Unknown column error when uploading .csv file in MySQL - mysql

I am uploading a huge csv file with the following code:
LOAD DATA LOCAL INFILE 'file.csv'
INTO TABLE signal_vv.Action
CHARACTER SET latin1
FIELDS TERMINATED BY ','
-- ESCAPED BY '\b'
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS
(#visitdatetime,usersessionid,probability,#spottime,network,reqion,creative,origin,Region,
t_usersessions_cuserref,t_usersessions_useraddressid,country,isp,UserId,SessionRefID,
source_,t_usersessions_userrefid,postcode,appsessionid,UserIP,ServerAddress,Medium_,device,AdSpotID)
SET
visitdatetime= STR_TO_DATE(#visitdatetime,'%Y%m%d-%H:%i:%s'),
spottime=STR_TO_DATE(#spottime,'%Y%m%d-%H:%i:%s')
;
when I run it I got this error :
Error Code: 1054. Unknown column 'probability' in 'field list' 0.032 sec
the first 5 rows looks are:
visitdatetime,usersessionid,probability,spottime,network,reqion,creative,origin,Region,t_usersessions_cuserref,t_usersessions_useraddressid,country,isp,UserId,SessionRefID,source_,t_usersessions_userrefid,postcode,appsessionid,UserIP,ServerAddress,Medium_,device,AdSpotID
01/10/2016 06:14,13403176,0.009460106,01/10/2016 06:14,Movies 4 Men 1,national,VCCOOSI990030,None,GB/london/london,3.11137E+13,None,GB,TELEFONICAO2UK,None,2744429,None,None,None,None,82.132.238.96,None,web,None,41510
01/10/2016 06:14,13406873,0.009460106,01/10/2016 06:14,Movies 4 Men 1,national,VCCOOSI990030,None,GB/london/london,31148fc9500c58,None,GB,BT,None,2901890,None,None,SE10,None,109.147.90.149,None,web,None,41510
01/10/2016 06:14,13618866,0.009460106,01/10/2016 06:14,Movies 4 Men 1,national,VCCOOSI990030,None,GB/london/london,3191b1407c367e,None,GB,TELEFONICAO2UK,None,3063053,None,None,None,None,82.132.241.240,None,web,None,41510
01/10/2016 06:14,13407385,0.009460106,01/10/2016 06:14,Movies 4 Men 1,national,VCCOOSI990030,None,GB/london/london,3136d33c60e4c6,None,GB,TELEFONICAO2UK,None,2622421,None,None,None,None,82.132.222.151,None,web,None,41510
01/10/2016 06:14,13361612,0.009460106,01/10/2016 06:14,Movies 4 Men 1,national,VCCOOSI990030,None,GB/london/london,307808800c066f,None,GB,TELEFONICAO2UK,None,2805769,None,None,None,None,82.132.222.11,None,web,None,41510
The table has the field probability (double).
Could anybody help me with that?
thanks


Check the column name of your table.
Your LOAD DATA could be considering a .CSV that has a column with name "probability" and your table in MySQL could have a column with name "probabillity".

Related

How to create / modify a MySQL table to only get distinct values

I initially had a .csv file, and I imported the data into SQL. It comprises of footballers' data, so each footballer has a football club. Hence, when I create a reference table for the club, it becomes like this, since it reads the football club for each player entry.
id club big_club
1 Arsenal 1
2 Arsenal 1
3 Arsenal 1
......
15 Brighton 0
16 Brighton 0
17 Brighton 0
However, I want
id club big_club
1 Arsenal 1
2 Brighton 0
3 Chelsea 1
4 Everton 0
......
and so on. Currently, I'm thinking of 2 options.
1) Load data and filter directly (most preferred)
2) Load data in first, then update table to find the distinct values
I would like assistance in both. Option 2 sounds rather simple but unfortunately, I only know how to do it from a SELECT DISTINCT standpoint and not an UPDATE standpoint.
For loading data into the table, this is what I have.
LOAD DATA INFILE 'C:/ProgramData/MySQL/MySQL Server 8.0/Uploads/epldata_final.csv'
INTO TABLE big_clubs
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(#name, #club, #age, #position, #position_cat, #market_value, #page_views, #fpl_value,
#fpl_sel, #fpl_points, #region,
#nationality, #new_foreign, #age_cat, #club_id, #big_club, #new_signing)
SET id=#id, club_id = #club_id, club = #club;
I tried SET club_id = DISTINCT #club_id but that doesn't work.
Would appreciate help / guidance for both methods.

Converting to varbinary adds spaces

I'm trying in insert some data from Sql Server into MySQL using OPENQUERY
SELECT 1,1,'0123456789','Test Contact' INTO #tempcontacts
INSERT INTO OPENQUERY([PBXA_MYSQL],'SELECT relation_id,unit,name,number FROM asterisk.contacts')
SELECT id,unit_id,CONVERT(varbinary(100),name),CONVERT(varbinary(50),RIGHT(number,8)) FROM #tempcontacts
DROP TABLE #tempcontacts
The fields on MySQL are varbinary and i'm unable to change that. The data is inserted but every character has a space added. It looks like this:
name number
T e s t C o n t a c t 0 1 2 3 4 5 6 7 8 9
Is there a better way to convert to varbinary?
Thanks

Shows different value after exporting to excel

I have table as tab1 in mysql
Name Value
john 22
parker 334
tony 44
;
After exporting this table to .csv file the name column shows as
Name Value
"john" 22
"parker" 334
"tony" 44
Now how to eliminate those Quotation marks(") from the name column .
Thanks in advance

If you use Phpmyadmin....While exporting table change double quotes from the textbox Format-specific options:..available under "export options"
If you use Query try..MySQL - export to csv some columns with quotes and some without; OPTIONALLY ENCLOSED BY not working correctly

loading .csv file with multiple commas in the field using MySQL

I have a table named 'table1' with the columns:
CSMembers,BCID,Total,Email.
The excel sheet contains the data for this table in this format:
CSMembers BCID Total Email
abc 2,5,7,9,12,17,22,32 10,000 abc#gmail.com
xyz 1,3,5,7,9,12,17,20,22,33 12,500 xyz#gmail.com
pqr 2,5,7,9,12,17,22,32 11,000 pqr#gmail.com
ttt 2,5,7,9,12,17,22 9,800 ttt#gmail.com
the .csv file of this is :
CSMembers,BCID,Total,Email
abc,"2,5,7,9,12,17,22,32","10,000",abc#gmail.com
xyz,"1,3,5,7,9,12,17,20,22,33","12,500",xyz#gmail.com
pqr,"2,5,7,9,12,17,22,32","11,000",pqr#gmail.com
ttt,"2,5,7,9,12,17,22","9,800",ttt#gmail.com
I have used the following code:
load data local infile 'H:/abc.csv' into table table1
fields terminated by ','
optionally enclosed by '"'
lines terminated by '\n' ignore 1 lines
(CSMembers,BCID,Total,Email);
I am getting the following output:
CSMember BCID Total Email
abc 2 10 abc#gmail.com
xyz 1 12 xyz#gmail.com
pqr 2 11 pqr#gmail.com
ttt 2 9 ttt#gmail.com
But i need this output:
CSMembers BCID Total Email
abc 2,5,7,9,12,17,22,32 10,000 abc#gmail.com
xyz 1,3,5,7,9,12,17,20,22,33 12,500 xyz#gmail.com
pqr 2,5,7,9,12,17,22,32 11,000 pqr#gmail.com
ttt 2,5,7,9,12,17,22 9,800 ttt#gmail.com
Can anyone please tell me what is wrong?
if I should change the code or the csv file content or both?
plese help.

Your schema for the table is possibly wrong bcid and total is probably defined as an int of some kind instead of a string. Numeric fields won't know whether a comma is a separator to another field or simply dividing the number up so it is easily readable. Also input to numeric fields typically accepts the value up to the first non-numeric character i.e. the comma

Conditionally set value of column in MySQL select query

I have a simple query in MySQL
select AGE, NAME from Members;
which returns,
20 ABC
11 PQR
21 XYZ
16 REW
I need to conditionally select NAME from Members, i.e if AGE is less then 18 I need to print MINOR. So my output will be
20 ABC
11 MINOR
21 XYZ
16 MINOR
I know I should do this programmatically, but I am dumping the output directly to a file in CSV format using
INTO OUTFILE '$random_file_name' FIELDS TERMINATED BY ',' ENCLOSED BY '\"' LINES TERMINATED BY '\n'
What are my options in MySQL query itself to achieve this?

select AGE,
case when AGE < 18
then 'MINOR'
else NAME
end as NAME
from Members