As part of a Linux-script, i tried to insert data from a file into a table.
If I try to use the following SQL statement I get this answer from the Server:
./load_test.sh: line 28: -h$SERVER: command not found.
What does it mean?
$MYSQL -h$SERVER -P3306 -uroot -p $DB -e "load data local infile '/var/lib/mysql/scripts/load_test/test.del' INTO TABLE TAB_SNAP_GET_DB_V97 FIELDS TERMINATED BY ',' ENCLOSED BY '\"' LINES TERMINATED BY '\n'"
Related
can someone please assist, I am new to mysql, and i have noticed that "LOAD DATA LOCAL INFILE" does not work in mysql event scheduler to update my databases from a normal .csv
So I'm trying to setup a "cron job" in linux to run a shell script to do the LOAD DATA INFILE to my databases, but im getting errors on the following shell script, please help correct it, see my script layout below...
#!/bin/bash
mysql -u root -p xxxxxxx testdb --local_infile=1 -e"LOAD DATA LOCAL INFILE '/mnt/mysqldb/mysqldb-new/mysql/CK-BATCH-FTP/Acelity/activity.csv'
INTO TABLE acelity_activity
FIELDS TERMINATED BY ';'
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS;
Thank you for helping me
You must enclose the Double quotes in ENCLOSED BY and i think that you have a finalizing Double quote in you original code
#!/bin/bash
mysql -u root -p xxxxxxx testdb --local_infile=1 -e"LOAD DATA LOCAL INFILE '/mnt/mysqldb/mysqldb-new/mysql/CK-BATCH-FTP/Acelity/activity.csv'
INTO TABLE acelity_activity
FIELDS TERMINATED BY ';'
ENCLOSED BY '\"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS;"
And you should test the command, in wokbench or phpmyadmin
Thanks for the help #nbk
So i modified the script above a little, and the following worked for me: (I removed the db name in the mysql syntax and add it in the query)
mysql -u root -pxxxxxxx -e"LOAD DATA LOCAL INFILE '/mnt/mysqldb/mysqldb-new/mysql/CK-BATCH-FTP/File/XXXX.csv'
INTO TABLE <DB NAME>.<TABLE NAME>
FIELDS TERMINATED BY ';'
ENCLOSED BY '\"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS;"
If you are using Mysql8 then this will help you.
mysql -udbuser -pXXXXX -h host-pc test1 --local_infile=1 -e "LOAD DATA LOCAL INFILE '/home/abc/data.csv' INTO TABLE tempTable FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"' LINES TERMINATED BY '\n' (col1, col2);";
Make sure there is no space in username and password i.e. it should be like this as mentioned in the above command.
Also ENCLOSED BY needs to write in the mentioned format only.
I am trying to use mysql load via the shell prompt. I want to load a CSV file directly into the database.
mysql -u root -p -h mysql -e "LOAD DATA INFILE 'Subscriber.csv' INTO TABLE temp_data FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n'IGNORE 1 ROWS" psi
I am getting the error ERROR 1083 (42000) at line 1: Field separator argument is not what is expected
Below is my CSV file
misdn,city,age,gender
771XXXXXX,MUTOKO,24,MALE
771XXXXXX,MUTOKO,32,MALE
771XXXXXX,MUTOKO,37,Male
771XXXXXX,MUTOKO,36,MALE
771XXXXXX,MUTOKO,25,Male
771XXXXXX,HWEDZA,26,MALE
771XXXXXX,HWEDZA,33,MALE
771XXXXXX,MUTOKO,26,MALE
771XXXXXX,HWEDZA,34,MALE
771XXXXXX,HWEDZA,34,MALE
771XXXXXX,MUTOKO,21,MALE
771XXXXXX,MUTOKO,22,MALE
771XXXXXX,MUTOKO,30,MALE
771XXXXXX,MUTOKO,28,Male
771XXXXXX,MUTOKO,33,Male
771XXXXXX,MUTOKO,23,Male
771XXXXXX,ZVISHAVANE,31,Male
771XXXXXX,ZVISHAVANE,39,MALE
Please help, what I am doing wrong?
You used ENCLOSED BY '"', but your fields do not contain double quotes at all. I think you should have used OPTIONALLY ENCLOSED BY '"'. Try this version:
mysql -u root -p -h mysql -e "LOAD DATA INFILE 'Subscriber.csv'
INTO TABLE temp_data
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\n'IGNORE 1 ROWS" psi
Or, if you are certain that no fields will ever have double quotes, you could remove the ENCLOSED BY clause entirely.
I have 3000 csv files that are " delimited and , separated. Each is named after the server the data was collected from. I need to import them into a database to allow me to work with the data.
For an individual file I've tried :
mysql -uuser -ppassword --local-infile mydatabase -e "LOAD DATA LOCAL INFILE '/root/downloads/test/reports/mytestcsvfile.csv' INTO TABLE results FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' IGNORE 1 LINES FIELDS FIELDS ESCAPED BY '\' "
and this works fine except that every data element is enclosed in quotes. If I add
ENCLOSED BY '"'
then the " in that parameter stops the import working and just gives me a > prompt.
If I escape the " with
ENCLOSED BY '\"'
then I get an error :
ERROR 1064 (42000) at line 1: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'FIELDS ENCLOSED BY '"' FIELDS ESCAPED BY '\'' at line 1
I'll need to enclose the whole lot in something like :
#!/bin/bash
FILES= /root/downloads/smtptest/reports/
for f in $FILES
do
echo "processing $f..."
mysql -uuser -ppassword --local-infile mydatabase -e "LOAD DATA LOCAL INFILE '/root/downloads/test/reports/$f' INTO TABLE results FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' IGNORE 1 LINES FIELDS FIELDS ESCAPED BY '\' "
done
How can I run the import and strip the ". Mysqlimport looks to be ruled out as it imports into a table based on filename, not something I want.
And...
To allow me to run it for each file at the moment I'm having to run this first:
for f in file*.csv; do echo ":" $f ":"; sed -i "s/^/\"$f\",/" "$f"; done
This adds each server name (the filename) to the start of each line in each file so that that becomes part of the record in the database. Is there a more elegant way of doing this?
I found the problem: I was using "fields" multiple times, to define the enclosed, terminated and escaped by parameters. When I removed that it all started working
This worked :
!/bin/bash
for filename in /root/downloads/smtptest/reportsmod/*.csv; do
echo "processing $filename"
mysql -uuser -ppassword --local-infile smtpsslcheck -e "LOAD DATA LOCAL INFILE '$filename' INTO TABLE results FIELDS TERMINATED BY ',' ENCLOSED BY '\"' LINES TERMINATED BY '\n' IGNORE 1 LINES"
done
So i'm trying to create a script that I can run that will do a batch import of csv files into a table.
I'm having trouble getting the script to work.
Here is the script i'm running:
#!/bin/bash
for f in *.csv
do
"/opt/lampp/bin/mysql -e use test -e LOAD DATA LOCAL INFILE ("$f") INTO TABLE temp_table FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' LINES TERMINATED BY '\n' IGNORE 1 LINES (DATE, TIME, SITE_NAME, SITE_IP, TOTAL_TALKTIME, EDGE_UL_BYTES, EDGE_DL_BYTES);"
done
When I run the script I receive the following error message:
./script.sh: line 5: unexpected EOF while looking for matching `''
./script.sh: line 7: syntax error: unexpected end of file
The load data local infile command works fine directly in mysql.
When you want to use literal double quotes in double quoted strings, escape them with \". Since mysql doesn't care about line feeds, you can also break the line to make it more readable:
#!/bin/bash
for f in *.csv
do
/opt/lampp/bin/mysql -e "use test" -e "
LOAD DATA LOCAL INFILE '$f'
INTO TABLE temp_table
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '\"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(DATE, TIME, SITE_NAME, SITE_IP, TOTAL_TALKTIME,
EDGE_UL_BYTES, EDGE_DL_BYTES);"
done
mysql -u<username> -p<password> -h<hostname> <db_name> --local_infile=1 -e "use <db_name>" -e"LOAD DATA LOCAL INFILE '<path/file_name>'
IGNORE INTO TABLE <table_name>
FIELDS TERMINATED BY '\t'
OPTIONALLY ENCLOSED BY '\"'"
I'm trying to simplify a load data local infile, by putting it into a .sh file, and bash to run it.
Here is my count_portal.sh file
mysql -h host -u root -p password
load data local infile
"/workplace/user/dump/count_portal.txt"
replace into table test.icqa_count_portal
fields terminated by '\t' lines terminated by '\n' ignore 1 lines;
And here is my bash script
bash /home/user/Desktop/count_portal.sh I get an output that doesn't do what it is designed to do. When I simply make the count_portal.sh contain mysql -h host it longs in when running the script.
I figured it out. Here is my file.
#!/bin/bash
/usr/bin/mysql --host=host --user=root --password=password --database=test<<EOFMYSQL
load data local infile '/workplace/user/ETLdump/count_portal.txt' replace INTO TABLE count_portal fields terminated by '\t' LINES
TERMINATED BY '\n' ignore 1 lines;
EOFMYSQL
Works flawlessly!
Try removing the space between the user and password switches. So it would be something like:
mysql -h host -uroot -ppassword etc....
The spaces seem to cause problems for me when doing similar things inside bash scripts.
missed -e ?
mysql -h host -u root -ppassword dbname -e "load data local infile '/workplace/user/dump/count_portal.txt' replace into table test.icqa_count_portal fields terminated by '\t' lines terminated by '\n' ignore 1 lines;"
I agree with #PasteBT, his answer should work well. I think there are escape problem or shell variable is empty.
Have you tried this?:
echo "LOAD DATA LOCAL INFILE '/workplace/user/dump/count_portal.txt' REPLACE INTO TABLE test.icqa_count_portal FIELDS TERMINATED BY '\t' LINES TERMINATED BY '\n' IGNORE 1 LINES;" | mysql -h host -u root -ppassword
and you original shell script is wrong. could you post again what did you do?
mysql -u<username> -p<password> -h<hostname> <db_name> --local_infile=1 -e "use <db_name>" -e"LOAD DATA LOCAL INFILE '<path/file_name>'
IGNORE INTO TABLE <table_name>
FIELDS TERMINATED BY '\t'
OPTIONALLY ENCLOSED BY '\"'"