I have a form where user updates the table in database with serial numbers, the problem is that in my .csv file serial number has value 0 and after inserting it, it has 000000, same for the 1, after inserting it is 000001. I need it in exact way like it is in .csv file. My code for the LOAD is:
LOAD DATA LOCAL INFILE path_to_file.csv
INTO TABLE im_seriennummer CHARACTER SET latin1
FIELDS TERMINATED BY ";"
IGNORE 1 LINES
(sn,description_sn)
In .csv file it is like this:
0
1
And in database
000000
000001
In the database sn is varchar(16).
Is this problem familiar to anyone? Please don't tell me to change the type of field, I need to have it in varchar since some serial numbers are like this MT 002
The solution,i think, is to use a temp table from import the csv.
CREATE TEMPORARY TABLE tmptab LIKE im_seriennummer;
LOAD DATA LOCAL INFILE path_to_file.csv
INTO TABLE tmptab CHARACTER SET latin1
FIELDS TERMINATED BY ";"
IGNORE 1 LINES
(sn,description_sn)
UPDATE tmptab SET SERIAL = RIGHT(CONCAT('000000', SERIAL), 6)
INSERT INTO im_seriennummer
SELECT * FROM tmptab
DROP TEMPORARY TABLE tmptab;
Related
I have a csv file (12 gb) which exported from oracle database,
formatted like
6436,,N,,,,,,,,,,,,04/01/1999,04/01/1999,352,1270,1270,406,406,1999,1,31/01/1999,0,88,0,A,11/12/2005,N,0,11/12/2005,,,,1270,1,0,,2974,,,,,,,,,,,,,,,,,,,,,,,,
As you see it has a lot of null values (mostly integer),
And when i import it to mysql database, It fills null values with zero
Like,
6436,0,0,,0,0,0,0,0,0,0,0,0,0,4,04/01/1999,04/01/1999,1270,1270,406,406,1999,1,31,31/01/1999,88,0,A,11/12/2005,N,0,11/12/2005,0,0,0,1270,1,0,,2974,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,,,0,0,00/00/0000,0,,0,0
What is the real issue here?
Thanks.
I figure it out with helpful comment of #Marc B
I wrote something like
LOAD DATA LOCAL INFILE '/path/data.csv' INTO TABLE table_name
FIELDS TERMINATED BY ','
IGNORE 1 LINES
(columns with #)
SET
column = IF(length(#column)= 0,null,#column),
date = str_to_date(#date, '%d/%m/%Y');
i use below script for insert data to sql from textpad.
#!/bin/bash
mysql --utest -ptest test << EOF
LOAD DATA INFILE 'test.txt'
INTO TABLE content_delivery_process
FIELDS TERMINATED BY ',';
EOF
in my test file i have a format like,
cast , date , name , buy
i can insert but i need format like below,
S.NO | date | name | buy | cast
You can specify the columns you want to import:
From the MySQL Manual:
MySQL LOAD DATA INFILE
The following example loads all columns of the persondata table:
LOAD DATA INFILE 'persondata.txt' INTO TABLE persondata;
By default, when no column list is provided at the end of the LOAD
DATA INFILE statement, input lines are expected to contain a field for
each table column.
If you want to load only some of a table's columns, specify a column
list:
LOAD DATA INFILE 'persondata.txt' INTO TABLE persondata (col1,col2,...);
You must also specify a column list if the order of the fields in the
input file differs from the order of the columns in the table.
Otherwise, MySQL cannot tell how to match input fields with table
columns.
You would include "FIELDS TERMINATED BY '|';" at the end to import data delimited with a '|' symbol.
Hope this helps.
create table [YOUR TABLE] ( `S.NO` INT AUTO_INCREMENT, date DATETIME, name VARCHAR(50), buy VARCHAR(50), cast VARCHAR(50));
Load data local infile 'test.txt' ignore into table [YOUR TABLE] fields terminated by ',' lines terminated by '\n'(cast , date , name , buy);
I have 35 CSV files which I want to import to MYSQL table(say 'test'). I want to create one column in 'test' table( say 'file_name'). This column will contain name of the CSV from which data has been imported. The file names are unique IDs, that is why I want to get file name as input in the table.
Suppose I have CSV files like X1.csv, X2.CSV, X3.csv .... X35.csv. I want a column in 'test' table as 'file_name' such that 'test' table looks something like:
col1 -> a, b, c, d
col2 -> x, y, w, z
...
...
... ....
file_name -> X1, X1, X2, X3
Note: I tried to search this question on forum but I could not find any suitable solution. Also I am new to MYSQL, please help even it is a trivial thing.
I'm not sure this is exactly what you are looking for, but at first sight, you should investigate the LOAD DATA INFILE statement:
LOAD DATA INFILE 'X1.csv' INTO TABLE tbl_name -- Load the content of the CSV file
FIELDS TERMINATED BY ',' ENCLOSED BY '"' -- assuming fields separate by ",", enclosed by "'"
LINES TERMINATED BY '\r\n' -- assuming end-of-line being '\r\n'
IGNORE 1 LINES -- assuming first line is a header and should be ignored
SET file_name = 'X1'; -- force the column `file_name` to be the name of the file
Please note that with such statement, each field will go in its own column of the table. And each line of the CSV data file will be loaded a one row in the table. This will imply that there will be several rows in the result table with the same file name. In fact one row per data line.
Initially I have uploaded Using load Data Infile row is having like 100000 Im Using Ubuntu
Example:data
ToneCode....Artist...MovieName...Language
1....................Mj..........Null........... English
3....................AB..........Null........... English
4....................CD.........Null........... English
5....................EF..........Null........... English
But Now I have To update Column MovieName Starting From ToneCode 1 till 100000 row I’m having data in .csv file to update .
Please suggest how to upload the .Csv file for existing table with data
I think the fastest way to do this, using purely MySQL and no extra scripting, would be as follows:
CREATE a temporary table, two columns ToneCode and MovieName same as in your target table
load the data from your new CSV file into that using LOAD DATA INFILE
UPDATE your target table using the INNER JOIN-like syntax that http://dev.mysql.com/doc/refman/5.1/en/update.html describes:
UPDATE items,month SET items.price=month.price WHERE items.id=month.id;
this would “join” the two tables items and month (by using just the “comma-syntax” for an INNER JOIN) using the id column as the join criterion, and update the items.price column with the value of the month.price column.
I Have found a solution as u Guys mentioned above
Soln: example
create table A(Id int primary Key, Name Varchar(20),Artist Varchar(20),MovieName Varchar(20));
Add all my 100000 row using
Load data infile '/Path/file.csv' into table tablename(A) fields terminated by ',' enclosed by'"'
lines terminated by '\n'
(Id,Name,Artist) here movie value is null
create temporary table TA(Id int primary Key,MovieName Varchar(20));
Uploaded data to temporary table TA
Load data infile '/Path/file.csv' into table tablename(A) fields terminated by ',' enclosed by'"'
lines terminated by '\n'(IDx,MovieName)
Now using join as u said
Update Tablename(TA),TableName(A) set A.MovieName=TA.MovieName Where A.Id=TA.Id
Need experts help to get whole original line using LOAD DATA LOCAL INFILE and put into my db column
Sample
table
DROP TABLE IF EXISTS `syslog`;
CREATE TABLE `syslog` (
`the_time` VARCHAR(80) NOT NULL,
`the_key` VARCHAR(30) NOT NULL,
`the_log` VARCHAR(1024) NOT NULL
)
ENGINE = MyISAM;
File : D:/rnd/syslog.csv
"device","date_time","src_ip","dst_ip","log_type","message"
"Fortigate","2012-05-02 12:02:03","192.168.1.1","192.168.1.11","vpn","Sample message1"
"Fortigate","2012-05-02 12:02:04","192.168.1.2","192.168.1.12","vpn","Sample message2"
"Fortigate","2012-05-02 12:02:05","192.168.1.3","192.168.1.13","traffic","Sample message3"
"Fortigate","2012-05-02 12:02:06","192.168.1.4","192.168.1.14","traffic","Sample message4"
"Fortigate","2012-05-02 12:02:07","192.168.1.5","192.168.1.15","vpn","Sample message5"
"Fortigate","2012-05-02 12:02:08","192.168.1.6","192.168.1.16","vpn","Sample message6"
Mysql Statement
SET #delimeter = ",";
LOAD DATA LOCAL INFILE
"D:/rnd/syslog.csv"
INTO TABLE syslog
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES
( #device,
#date_time,
#src_ip,
#dst_ip,
#log_type,
#message)
SET the_time = #date_time,
the_key=CONCAT(#src_ip, "~" , #dst_ip),
the_log=CONCAT(#device,#delimeter,#date_time,#delimeter,#src_ip,#delimeter,#dst_ip,#delimeter,#log_type,#delimeter,#message);
Currently is is just working with manual setting like
the_log=CONCAT(#device,#delimeter,#date_time,#delimeter,#src_ip,#delimeter,#dst_ip,#delimeter,#log_type,#delimeter,#message)
Is there any other way to get the whole line since the actual column is 60 and it is not a good idea to do it manually inside the code + not easy to maintain later.
Objective : To use the csv data and manipulate it into my own table (Means the column does not same as the csv)
The other way is to create table structure as CSV-file, store data as is, and use CONCAT only when you SELECT data from log table.