This is sql file
# mo.sql
DROP TABLE IF EXISTS mo;
## _CREATE_TABLE_
CREATE TABLE mo
(
name CHAR(30),
age INT,
salary INT
);
## _CREATE_TABLE_
LOAD DATA LOCAL INFILE 'moja-2001.txt' INTO TABLE mo;
I run it from Linux terminal
mysql -p cookbook < mo.sql
No warning,no errors.
SELECT * FROM mo;
+--------------------------+------+--------+
| name | age | salary |
+--------------------------+------+--------+
| jova jovic | 24 | NULL |
| ceda prashak | 25 | NULL |
| toma grobar 28 20001 | NULL | NULL |
+--------------------------+------+--------+
I have created txt file with geany text editor
jova jovic 24 999
ceda prashak 25 1000
toma grobar 28 20001
Why is salary column wrong?Why is third row wrong also?
The last row does not use the correct separators for values. I suspect the other rows separate the values using tabs and the last one uses spaces. The same for column salary.
Make sure you use the same separator for values. It's better to use comma (it is visible and less error prone) and use the FIELDS TERMINATED BY clause of the LOAD DATA statement to inform MySQL about it.
Change the file to look like this:
jova jovic,24,999
ceda prashak,25,1000
toma grobar,28,20001
and import it like this:
LOAD DATA LOCAL INFILE 'moja-2001.txt' INTO TABLE mo FIELDS TERMINATED BY ',';
Read more about the LOAD DATA statement.
Related
I am trying to import a csv file that contains 5 columns and 100 rows. When I open the terminal and type SELECT * FROM cmc2 it only shows 1 row.
mysql> SELECT * FROM cmc2;
+--------------------------------+-------+------------+--------------------+--------+
| alt_name | price | market_cap | circulating_supply | volume |
+--------------------------------+-------+------------+--------------------+--------+
| Alt coin name;Price;Market cap | 554 | 714 | 630 | NULL |
+--------------------------------+-------+------------+--------------------+--------+
1 row in set (0.01 sec)
this is my table that i created:
mysql> create table cmc2 (alt_name varchar(30), price int(20), market_cap int(20), circulating_supply int(20), volume int(20));
As per the documentation "CSV files must have one line for each row of data and have comma-separated fields." So you should replace the semicolons in your file with commas.
This should be an easy task if you have a tool like Word, Excel, etc... Where you can do a search (Control + F) and select replace with... then import the CSV file with the required separator
I need to load file content into a table. The file contains text separated by commas. It is a very large file. I can not change the file it is already given to me like this.
12.com,128.15.8.6,TEXT1,no1,['128.15.8.6']
23com,122.14.10.7,TEXT2,no2,['122.14.10.7']
45.com,91.33.10.4,TEXT3,no3,['91.33.10.4']
67.com,88.22.88.8,TEXT4,no4,['88.22.88.8', '5.112.1.10']
I need to load the file into a table of four columns. So for example, the last row above should be in the table as follows:
table.col1: 67.com
table.col2: 88.22.88.8
table.col3: TEXT3
table.col4: no3
table.col5: ['88.22.88.8', '5.112.1.10']
Using MySQL workbench, I created a table with five columns all are of type varchar. Then I run the following SQL command:
LOAD DATA INFILE '/var/lib/mysql-files/myfile.txt'
INTO TABLE `mytable`.`myscheme`
fields terminated BY ','
The last column string (which contains commas that I do not want to separate) causes an issue.
Error:
Error Code: 1262. Row 4 was truncated; it contained more data than there were input columns
How can I overcome this problem please.
Not that difficult simply using load data infile - note the use of a variable.
drop table if exists t;
create table t(col1 varchar(20),col2 varchar(20), col3 varchar(20), col4 varchar(20),col5 varchar(100));
truncate table t;
load data infile 'test.csv' into table t LINES TERMINATED BY '\r\n' (#var1)
set col1 = substring_index(#var1,',',1),
col2 = substring_index(substring_index(#var1,',',2),',',-1),
col3 = substring_index(substring_index(#var1,',',3),',',-1),
col4 = substring_index(substring_index(#var1,',',4),',',-1),
col5 = concat('[',(substring_index(#var1,'[',-1)))
;
select * from t;
+--------+-------------+-------+------+------------------------------+
| col1 | col2 | col3 | col4 | col5 |
+--------+-------------+-------+------+------------------------------+
| 12.com | 128.15.8.6 | TEXT1 | no1 | ['128.15.8.6'] |
| 23com | 122.14.10.7 | TEXT2 | no2 | ['122.14.10.7'] |
| 45.com | 91.33.10.4 | TEXT3 | no3 | ['91.33.10.4'] |
| 67.com | 88.22.88.8 | TEXT4 | no4 | ['88.22.88.8', '5.112.1.10'] |
+--------+-------------+-------+------+------------------------------+
4 rows in set (0.00 sec)
In this case for avoid the problem related with the improper presence of comma you could import the rows .. in single column table .. (of type TEXT on Medimun TEXT ..as you need)
ther using locate (one for 1st comma , one for 2nd, one for 3th .. ) and substring you could extract form each rows the four columns you need
and last with a insert select you could populate the destination table .. separating the columns as you need ..
This is too long for a comment.
You have a horrible data format in your CSV file. I think you should regenerate the file.
MySQL has facilities to help you handle this data, particularly the OPTIONALLY ENCLOSED BY option in LOAD DATA INFILE. The only caveat is that this allows one escape character rather than two.
My first suggestion would be to replace the field separates with another character -- tab or | come to mind. Any character that is not used for values within a field.
The second is to use a double quote for OPTIONALLY ENCLOSED BY. Then replace '[' with '"[' and ] with ']"' in the data file. Even if you cannot regenerate the file, you can pre-process it using something like grep or pearl or python to make this simple substitution.
Then you can use the import facilities for MySQL to load the file.
I have been banging my head against a wall trying to import datetime values from a .csv file.
Here's the import statement.
LOAD DATA LOCAL INFILE 'myData.csv'
INTO TABLE equity_last_import
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
(equity,last,#last_date)
SET last_date = STR_TO_DATE( #last_date, '%Y-%m-%d %H:%i:%s')
Here's a sample of the data:
4108,48.74,"2013-09-16 16:15:04"
4249,8.1,"2013-09-16 16:15:04"
4197,3.81,"2013-09-16 17:20:00"
4139,26.81,"2013-09-16 16:15:04"
4218,24.83,"2013-09-16 17:20:00"
4260,79.72,"2013-09-16 16:15:04"
4270,450.12,"2013-09-16 17:20:00"
4242,30.38,"2013-09-16 16:15:04"
4193,1.42,"2013-09-16 16:15:04"
4134,3.77,"2013-09-16 16:15:04"
I am able to import date values using STR_TO_DATE() but I not able to get datetime values to import. I have tried several different date formats other than '%Y-%m-%d %H:%i:%s' and I always get a null datetime [0000-00-00 00:00:00]. I have also tried not using STR_TO_DATE(), since the string is in the default MySQL datetime format.
Any help will be appreciated.
I ran into the same problem. I fixed it by changing the format for the date column in my CSV file to match the MySQL datetime format.
Open CSV in Excel.
Highlight the column.
Right-click on the column.
Click on Format Cells.
Pick Custom.
Use yyyy/mm/dd hh:mm:ss in the Type field.
Click ok
My CSV successfully imported after I changed the datetime format as above.
The date in your data file is already in a format MySQL should natively understand. It's just enclosed in double quotes. You need to tell LOAD DATA INFILE how to deal with the quotes. Try something like this:
LOAD DATA LOCAL INFILE 'myData.csv'
INTO TABLE equity_last_import
FIELDS OPTIONALLY ENCLOSED BY '"' TERMINATED BY ','
LINES TERMINATED BY '\n'
(equity,last,last_date)
Update:
Since you've said it doesn't work, I created a test table and verified that it does work. Here's the proof:
I've highlighted your csv data from the question and pasted into a new file called myData.csv in my system's /tmp folder. Then I connected to the mysql console, switched to the test database and ran the following:
mysql> create table equity_last_import (equity int, last decimal(10,2), last_date datetime) engine=innodb;
Query OK, 0 rows affected (0.02 sec)
mysql> LOAD DATA LOCAL INFILE '/tmp/myData.csv'
-> INTO TABLE equity_last_import
-> FIELDS OPTIONALLY ENCLOSED BY '"' TERMINATED BY ','
-> LINES TERMINATED BY '\n'
-> (equity,last,last_date);
Query OK, 10 rows affected (0.00 sec)
Records: 10 Deleted: 0 Skipped: 0 Warnings: 0
mysql> select * from equity_last_import;
+--------+--------+---------------------+
| equity | last | last_date |
+--------+--------+---------------------+
| 4108 | 48.74 | 2013-09-16 16:15:04 |
| 4249 | 8.10 | 2013-09-16 16:15:04 |
| 4197 | 3.81 | 2013-09-16 17:20:00 |
| 4139 | 26.81 | 2013-09-16 16:15:04 |
| 4218 | 24.83 | 2013-09-16 17:20:00 |
| 4260 | 79.72 | 2013-09-16 16:15:04 |
| 4270 | 450.12 | 2013-09-16 17:20:00 |
| 4242 | 30.38 | 2013-09-16 16:15:04 |
| 4193 | 1.42 | 2013-09-16 16:15:04 |
| 4134 | 3.77 | 2013-09-16 16:15:04 |
+--------+--------+---------------------+
10 rows in set (0.00 sec)
See? It works perfectly.
Another Update:
You've specified that you're getting the following error now:
Out of range value for column 'last_date' at row 1
Does your CSV file have a header? If so, you may want to add IGNORE 1 LINES to your LOAD DATA INFILE command to tell MySQL to skip over the header.
Pulled my hair out over this also because I'm not importing in the above suggested way.
Workaround: Created a temporary field temp_date of type "VARCHAR" on your import table and have no problems loading the data. I then was able to perform an update on my date column which is of type date.
update table
set date = temp_date
I was having the same trouble and here's what I discovered, I think it might help you
The problem regards a GMT conflict:
The database I was extracting the .csv file was on GMT 00:00, so there wasn't daylight saving time.
My local server (which was the one I was trying to insert the .csv file) was running on my computer (system's) GMT, by default. In my case it was GMT -3, which is affected by daylight saving time.
SQL has a special way to deal with daylight saving time, it ignores the exact date and time when the daylight saving time starts to happen. In my case, it was october 20 and all the records between 00 and 1 AM of that day simply weren't recognized by the server, even if the format was correct, because they simply 'didn't exist' (see more here). This was affecting all timestamp records, not only the specifics of daylight saving time.
My solution was to set the local server to GMT +00, before creating the new table and import the .csv, using
SET time_zone='+00:00';
And then when I imported the .csv file all the records were read properly. I think if you set the time zone equal to the one that generated the .csv file should work!
Open CSV in Excel.
Highlight the column.
Right-click on the column.
Click on Format Cells.
Pick Custom.
Use same format as SQL date format yyyy-mm-dd
Click ok and save
it's working fine for me
What's the right syntax to insert a value inside a column of type bit(1) in `MySQL'?
My column definition is:
payed bit(1) NOT NULL
I'm loading the data from a csv where the data is saved as 0 or 1.
I've tried to do the insert using:
b'value' or 0bvalue (example b'1' or 0b1)
As indicated from the manual.
But I keep getting this error:
Warning | 1264 | Out of range value for column 'payed' at row 1
What's the right way to insert a bit value?
I'm not doing the insert manually but I'm loading the data from a csv (using load data infile) in which the data for the column is 0 or 1.
This is my load query, I've renamed the fields for privacy questions, there's no error in that definition:
load data local infile 'input_data.csv' into table table
fields terminated by ',' lines terminated by '\n'
(id, year, field1, #date2, #date1, field2, field3, field4, field5, field6, payed, field8, field9, field10, field11, project_id)
set
date1 = str_to_date(#date1, '%a %b %d %x:%x:%x UTC %Y'),
date2 = str_to_date(#date2, '%a %b %d %x:%x:%x UTC %Y');
show warnings;
This is an example row of my CSV:
200014,2013,0.0,Wed Feb 09 00:00:00 UTC 2014,Thu Feb 28 00:00:00 UTC 2013,2500.0,21,Business,0,,0,40.0,0,PROSPECT,1,200013
Update:
I didn't find a solution with the bit, so I've changed the column data type from bit to tinyint to make it work.
I've finally found the solution and I'm posting it here for future reference. I've found help in the mysql load data manual page.
So for test purpose my table structure is:
+--------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+--------+-------------+------+-----+---------+-------+
| id | int(11) | NO | PRI | NULL | |
| nome | varchar(45) | YES | | NULL | |
| valore | bit(1) | YES | | NULL | |
+--------+-------------+------+-----+---------+-------+
My csv test file is:
1,primo_valore,1
2,secondo_valore,0
3,terzo_valore,1
The query to load the csv into the table is:
load data infile 'test.csv' into table test
fields terminated by ',' lines terminated by '\n'
(id, nome, #valore) set
valore=cast(#valore as signed);
show warnings;
As you can see do load the csv you need to do a cast cast(#valore as signed) and in your csv you can use the integer notation 1 or 0 to indicate the bit value. This is because BIT values cannot be loaded using binary notation (for example, b'011010').
Replace the "0" values in the csv by no value at all. That worked for me.
You can use BIN() function like this :
INSERT INTO `table` VALUES (`column` = BIN(1)), (`column` = BIN(0));
Let me guess, but I think you should ignore 1st line of your CSV file in LOAD query.
See "IGNORE number LINES"
Pseudo table:
| primary_key | first_name | last_name | date_of_birth |
| 1 | John Smith | | 07/04/1982 |
At the moment first_name contains a users full name for many rows. The desired outcome is to split the data, so first_name contains "John" and last_name contains "Smith".
I have a CSV file which contains the desired format of data:
| primary_key | first_name | last_name |
| 1 | John | Smith |
Is there a way of using the LOAD DATA INFILE command to process the CSV file to UPDATE all rows in this table using the primary_key - and not replace any other data in the row during the process (i.e. date_of_birth)?
In this situation I usually LOAD DATA INFILE to a temp table with identical structure. Then I do INSERT with ON DUPLICATE KEY UPDATE from the temp table to the real table. This allows for data type checking without wrecking your real table; it's relatively quick and it doesn't require fiddling with your .csv file.
No. While LOAD DATA INFILE has a REPLACE option, it will actually replace the row in question - that is, delete the existing one and insert a new one.
If you configure your LOAD DATA INFILE to only insert certain columns all others will be set to their default values, not to values they currently contain.
Can you modify your CSV file to contain a bunch of UPDATE statements instead? Should be reasonably straightforward via some regex replaces.