I am trying to run the unload command on redshift to dump data from a table into a CSV file. This table has character and numeric fields. The character fields may contain a comma (,) , so I need quotes around them. However, I dont need quotes around my numeric columns.
The following command is the closest I have come, but cant seem to get rid of the quotes aroud my numeric data. How can I achieve the desired result?
unload ('select * from mytable') to
's3://mybucket/path/file.csv'
DELIMITER ',' ADDQUOTES
This results in data like:
"Henry, Jr","23","4.5"
"Henry, Sr","56","4.2"
What I would like is :
"Henry, Jr",23,4.5
"Henry, Sr",56,4.2
From reading the official documentation, it seems like that's not possible.
I can suggest two potential workarounds:
1) wrap your string columns with quotes in the query, i.e. instead of
select * from mytable
have
select int_col_1, int_col_2, '"'||str_col_1||'"','"'||str_col_2||'"' from mytable
2) export tab delimited files so the commas in text columns stop being a problem
Related
How do I query in MySql without putting all inserts in quotations? (I have a big list and it would take to much time to quote and unquote every word)
Example:
SELECT *
FROM names
WHERE names.first IN ("joe", "tom", "vincent")
Since you said the list is comma separated, simply use the 'find and replace' feature to find all commas and replace them with ","
The result should be joe","tom","vincent"," which you can simply copy into mysql.
All you then have to do is edit the start and end of the string
I'm trying to import a csv into Hive. I have a column which is a dollar value and is reported within the CSV as '$123,244.00.' I would like to convert this value into a float in Hive.
So I've loaded the csv into a temporary table, treating that column as a string. Next I want to load it into the final table, and in the process convert that string into a float or decimal.
Any suggestions on the best way to go about doing this?
This should work:
select float(regexp_replace(substr('$123,244.00', 2, length('$123,244.00')), ',', '')) from table;
You need to remove any commas as well as the dollar sign. You may find this link helpful as well: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types#LanguageManualTypes-NumericTypes
I converted a large amount of data from mssql to mysql and a few of them came in with the 'other' double quotes ... the sharper ones.
I would like to do a query in phpmyadmin for all the entries that have that symbol because its breaking my query (coming back as null) but cannot figure out how to write it ...
SELECT * FROM table where id LIKE '% %' <-- i dont actually know what the ascii symbol is for that one and this html ascii convention doesnt work anyway.
If you find one of them, can't you just copy it and paste it into your LIKE clause? That will prevent you from having to figure out the ASCII code for it.
Today I did something stupid: I had a list of card numbers in an excel file that I had to import to a DB table somehow. So i exported the numbers to CSV file, but without any quotes (don't ask me why). The file looked like:
123456
234567
345678
...
Then I created a table with a single VARCHAR(22) column and did a
LOAD DATA LOCAL INFILE 'numbers.csv' INTO TABLE cards
This worked fine, apart from many warnings, which I ignored (the other stupid thing I did).
After that I tried to query with this SQL:
SELECT * FROM cards WHERE number='123456'
which gave me an empty result. Whereas this works:
SELECT * FROM cards WHERE number=123456
Notice the missing quotes! So it seems, that I managed to populate my VARCHAR table with INTEGER data. I have no idea how that is possible at all.
I already tried to fix this with an UPDATE like this
UPDATE cards SET number = CAST(number AS CHAR(22))
But that didn't work.
So is there a way to fix this and how could this even happen?
This is the result of some implicit conversion in order to do a numerical comparison:
SELECT * FROM cards WHERE number='123'
This will only match against text fields that are literally "123" and will miss on " 123" and "123\r" if you have those. For some reason, "123 " and "123" are considered "equivalent" presumably do to trailing space removal on both sides.
When doing your import, don't forget LINES TERMINATED BY '\r\n'. If you're ever confused about what's in a field, including hidden characters, try:
SELECT HEX(number) FROM cards
This will show the hex-dumped output of each string. Things like 20 indicate space, just as %20 in a URL is a space.
You can also fix this by:
UPDATE cards SET number=REPLACE(number, '\r', '')
Ok here is my problem. I have a csv file that is created out of my control that has a data for different groupings on the same file. The first seven lines are table headers for each group which are different for each group. So first I import this file into Access into a single table. I have since created queries to access the individual groups for data analysis. The problem is that I need to use an expression on one of the fields but since it has to be text in order to import from the spreadsheet because each column contains numbers and characters because of the headers in the top and because sometimes the data is not in the correct column and needs to be massaged. So what I want to do is insert each group into their own table but I want to convert some of the columns to numbers so that my expression will work. I will post the expression that I am having problems with. Thanks.
Sum(IIf([2000 Query].[Field19]=1,IIf([5000 Query].[Field21]>0,-[5000 Query].[Field21],[5000 Query].[Field21]),[5000 Query].[Field21])) AS [ADJ Invoice Total]
CDec:
IIf(CDec([2000 Query].[Field19])=1 ...
It works like so:
?cdec(" 20,121.34 ")
20121.34
So commas and leading and trailing spaces should be okay.
CDec is available in VBA but not in MS Access queries. In queries, Val will work:
IIf(Val([2000 Query].[Field19])=1 ...
Or CDbl, which will accept comma thousand separators and leading and trailing spaces.