I want to update my columns for rows specified by WHERE command, but I want to update my field in a way that it extracts number part of the string from each specified field, multiplies that with a number (that I will specify) and give number output in all those specific fields extracted by WHERE command in that column.
For example, assume I want to update all my fields in a column which are like (5.6 AUD/1000, 4.5 AUD/1000, 9.7 AUD/1000), so I want to first identify fields ending with /1000 and update only those fields in the column by multiplying the number part of the string (which is 5.6, 4.5, 9.7) with any number (let's say 10). I want that other fields on the column remains unchanged.
SELECT * from sorted WHERE Column8 REGEXP '/1000$';
gives me all the specific fields that I wish to update. But I want to update them in the way I specified above, which is that I want to extract number part from the string and multiply that with a number and update those fields only.
I am able to extract all the fields with the condition I mentioned, I'm facing difficulty in update these fields in the column.
SELECT * from sorted WHERE Column8 REGEXP '/1000$';
SELECT CAST(Column8 AS UNSIGNED)*10 FROM sorted
wHERE
column8 REGEXP '/1000$';
The above code gives me required updated fields, but I want them reflected in my column.
I expect my output to be a column where only those fields ending with '/1000' should get updated in a way that the number part of the string is multiplied with 10.
I have casted the varchar field named string to decimal type and multiplied with static value 10 . I have checked in sql server.
DECLARE #temp TABLE
(
string NVARCHAR(50)
)
INSERT INTO #temp (string)
VALUES
('5.6 AUD/1000'),
('4.5 AUD/1000'),
('9.7 AUD/1000')
select cast(left(string, patindex('%[^0-9./]%', string) - 1) As decimal(18,2))* 10
from #temp
I have a csv data as below where data comes every 10mins in the following format. I need to insert this data into hive by mapping column names with different column names. (columns don't come in constant order they change their order, we have total 10 columns sometimes we miss many columns like one example below below)
sample csv file :-
1 2 6 4
u f b h
a f r m
q r b c
now while inserting into hive i need to replace column names
for example
1 -> NBR
2 -> GMB
3 -> GSB
4 -> KTC
5 -> VRV
6 -> AMB
now I need to insert into hive table as below
NBR GMB GSB KTC VRV AMB
u f NULL h NULL b
a f NULL m NULL r
can anyone help me with this how to insert this values into hive
Assuming you can get column headers in you source CSV, you will need to map them from source number to their column names.
sed -i 's/1/NBR/g; s/2/GMB/g; s/3/GSB/g; s/4/KTC/g; s/5/VRV/g; s/6/AMB/g;...;...;...;...' input.csv
Since you only get an unknown subset of the total columns in your hive table, you will need to translate your CSV from
NBR,GMB,AMB,KTC
u,f,b,h
a,f,r,m
q,r,b,c
to
NBR,GMB,GSB,KTC,VRV,AMB,...,...,...,...
u,f,null,b,null,h,null,null,null,null
a,f,null,r,null,m,null,null,null,null
q,r,null,b,null,c,null,null,null,null
in order to properly insert them into your table.
From the Apache Wiki:
Values must be provided for every column in the table. The standard SQL syntax that allows the user to insert values into only some columns is not yet supported. To mimic the standard SQL, nulls can be provided for columns the user does not wish to assign a value to.
Standard Syntax:
INSERT INTO TABLE tablename [PARTITION (partcol1[=val1], partcol2[=val2] ...)] VALUES values_row [, values_row ...]
Where values_row is:
( value [, value ...] )
where a value is either null or any valid SQL literal
Using LOAD DATA INPATH, even with the tblproperties("skip.header.line.count"="1") set, still requires a valid SQL literal for all columns in the table. This is why youre missing columns.
If you can not get the producer of the CSV to create a file with 1,2,...9,10 columns in order with your table columns and either consecutive commas or a null character in the data, write some kind of script to add missing column names, in the order you need them in, and the required null values in the data.
If you will have header in csv like 1,2,3,4 (as you wrote in the comment), you could use the next syntax:
insert into table (columns where you want to insert) select 1,2,3,4 (columns) from csv_table;
So, if you could know the order of csv columns, you could write easily the insert, naming only the column that you need to populate, no matter the order in the target table.
Before you could run the above insert, you should create a table that reads from csv!
I have a table full of traffic accident data with column headers such as 'Vehicle_Manoeuvre' which contains integers for example 13 represents the vehicle manoeuvre which caused the accident was 'overtaking moving vehicle'.
I know the mappings from integers to text as I have a (quite large) excel file with this data.
An example of what I want to know is percentage of the accidents involved this type of manoeuvre but I don't want to have to open the excel file and find the mappings of integers to text every time I write a query.
I could manually change the integers of all the columns (write query with all the possible mappings of each column, add them as new column, then delete the orginial columns) but this sould take a long time.
Is it possible to create some type of variable (like an array with first column as integers and second column with the mapped text) that SQL could use to understand how text relates to the integers allowing me to write a query below:
SELECT COUNT(Vehicle_Manoeuvre) FROM traffictable WHERE Vehicle_Manoeuvre='overtaking moving vehicle';
rather than:
SELECT COUNT(Vehicle_Manoeuvre) FROM traffictable WHERE Vehicle_Manoeuvre=13;
even though the data in the table is still in integer form?
You would do this with a Maneeuvres reference table:
create table Manoeuvres (
ManoeuvreId int primary key,
Name varchar(255) unique
);
insert into Manoeuvres(ManoeuvreId, Name)
values (13, 'Overtaking');
You might even have such a table already, if you know that 13 has a special meaning.
Then use a join:
SELECT COUNT(*)
FROM traffictable tt JOIN
Manoeuvres m
ON tt.Vehicle_Manoeuvre = m.ManoeuvreId
WHERE m.name = 'Overtaking';
I am working on a project that was started with someone else. In the db instead for using a separate table the developer had opted for saving the 1 to many relationships on a single table with comma separated tables. The table structure is like this
CREATE TABLE pages(
pageid INT(6) AUTO_INCREMENT PRIMARY KEY,
newsid INT(6),
pages VARCHAR(30)
);
How can I search for a value 1 from the column pages. I have identified a few conditions that may appear, but was not been able to create a solution for it.
If I am searching for 1 the following patterns should be handles
1, match
11 shouldn't match
11, shouldn't match
,1, match
,1 match
1 match
21 shouldn't match
21, shouldn't match
I have been thinking about this for sometime, but no solution came up. I don't think normal %LIKE% can be used here
Sample sql on sqlfiddle
Also I need to search multiple values too like 1, 7 and 3
Use
FIND_IN_SET().
Example:
SELECT * FROM pages WHERE FIND_IN_SET('1', pages)
From the documentation:
FIND_IN_SET(str,strlist)
Returns a value in the range of 1 to N if the string str is in the string list strlist consisting of N substrings. A string list is a string composed of substrings separated by “,” characters. If the first argument is a constant string and the second is a column of type SET, the FIND_IN_SET() function is optimized to use bit arithmetic. Returns 0 if str is not in strlist or if strlist is the empty string. Returns NULL if either argument is NULL. This function does not work properly if the first argument contains a comma (“,”) character.
(highlighting added)
I have a VARCHAR field in a MySQL table like so -
CREATE TABLE desc(
`pk` varchar(10) NOT NULL UNIQUE,
...
);
The value in pk field is of the type - (xx0000001, xx0000002, ...). But when I insert these into my table the values in pk field get truncated to (xx1, xx2, ...).
How to prevent this?
UPDATE: Adding the INSERTstatement
INSERT INTO desc (pk) VALUES ("xx0000001");
It could be that the viewer you are using to LOOK at the values is displaying the info incorrectly because it is trying to interpret that string as a number, or that mysql may be interpreting your numbers as hexadecimal or something strange.
What happens if you do
INSERT INTO desc (pk) VALUES ("xx0000099");
Does it come back as xx99? or some other value?
Looks like you are referencing different tables in your two statements, text and desc?
Possibly somewhere along your program logic the value is interpreted as a hexadecimal or octal number?