I want to use a variable as the file name in Load data Infile. I run a below code:
Set #d1 = 'C:/Users/name/Desktop/MySQL/1/';
Set #d2 = concat( #d1, '20130114.txt');
load data local infile #d2 into table Avaya_test (Agent_Name, Login_ID,ACD_Time);
Unfortunately after running, there is a error with the commment like below:
"Error Code: 1064. You have an error in your SQL syntax ......"
Variable "#D2" is underlined in this code so it means that this error is caused by this variable.
Can you help me how to define correctly a variable of file name in LOAD DATA #variable infile ?
Thank you.
A citation from MySQL documentation:
The file name must be given as a literal string. On Windows, specify backslashes in path names as forward slashes or doubled backslashes. The character_set_filesystem system variable controls the interpretation of the file name.
That means that it can not be a parameter of a prepared statement, stored procedure, or anything "server-side". The string/path evaluation must be done client side.
In general, it is not possible directly but you can use a temporary file.
The procedure is similar to using a combination PREPARE-EXECUTE. Notice that you cannot use PREPARE with LOAD DATA INFILE, for this reason you require a temporary file.
Here's an example of how to read a file with today's date:
SET #sys_date = CURRENT_DATE();
SET #trg_file = CONCAT("LOAD DATA INFILE '/home/data/file-",#sys_date, "' INTO TABLE new_data FIELDS TERMINATED BY ' ';");
SELECT #trg_file INTO OUTFILE '/tmp/tmp_script.sql';
SOURCE /tmp/tmp_script.sql;
WARNING: You cannot overwrite files with mysql, for this reason the temporary file must not exist. It is a serious problem if you want to automatize the previous example.
Unfortunately, this does not seem to be possible in mysql.
In addition to Binary Alchemist's answer, it's also not possible with prepaired statements either, as it is not on this list: SQL Syntax Allowed in Prepared Statements
You could use soemthing external to generate your load data infile statement, and then run the sql. For instance you could create it in Excel.
This sysntax can't work since the variables are interpreted by the server, and the file is read by the mysql client.
Therefore for the client #d2 is an illegal file name.
My solution works on Linux/Mac/[Windows with bash] (e.g. cygwin)
Create a template with the load SQL, e.g. load.tpl -- note the %FILENAME% placeholder
LOAD DATA LOCAL INFILE '/path/to/data/%FILENAME%'
INTO TABLE Avaya_test ( Agent_Name, Login_ID, ACD_Time );
Then run this command:
for n in `ls *.txt`; do echo $n; sed -e 's/%FILENAME%/'$n'/g' load.tpl | mysql -u databaseUser databaseName; done
Related
The following triggers a LOAD DATA INFILE statement in mysql8:
util.importTable("sample.csv", {schema: "myschema", table: "mytable", dialect: "csv-unix", fieldsTerminatedBy: ";", showProgress: true})
Question: how can I add the IGNORE INTO flag?
util.importTable is using LOAD DATA LOCAL INFILE and ignores duplicate-keys by default[1]. If you want replace duplicate-keys set replaceDuplicates option to true.
[1] MySQL Reference Manual for LOAD DATA Syntax
Duplicate-Key Handling
(…) With LOCAL, the default behavior is the same as if IGNORE is specified; this is because the server has no way to stop transmission of the file in the middle of the operation.
I've created a huge .csv with only one column, each column is a valid sql string update like:
UPDATE TBL SET LAST = 0 WHERE ID = 1534781;
Is there a way to execute each row as a single sql query? Also, I'm using datagrip, if anyone knows of a sort of tool I would be happy.
To execute a file against your database in DataGrip just use the context menu when observing your file in Files tool window
A CSV file that contains one column is just called a "file." :-)
The most common way of executing a series of SQL statements in a file is with the command-line mysql client:
mysql -e "source myfile.csv"
How about script:
begin
update ...
update ...
update ...
...
end;
I'm working with JMeter to load test queries on a mySQL database (memsql server, mySQL syntax). I'm using a gui version of JMeter to create a test plan xml file and then go to another server and run that xml in non-gui mode.
I have two queries running, one that selects and one that inserts. Both queries use parameters taken from a csv file I made with a script.
My SELECT statement works just fine with parameters taken from a csv file but I run into syntax errors with my INSERT statement:
INSERT INTO customer_transactions_current (`column_name1`, ... , `column_name12`)
VALUES ((${r1},${r2},${r3},${r4},${r5},${r6},${r7},${r8},${r9},${r10},${r11},${r12}));
In the section of the query in the gui mode under 'CSV Data Set Config' I choose to delimit the data by ',' and the variable names are r1,..,r12.
Under the query itself I entered the parameter types and again the same names, just as I did for the working SELECT query.
When I run the query I run into a syntax error on the first column (which is of type datetime):
java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '19:00:00,75400492,936988,56,1115,5,2156,8,2,3,909,3))' at line 2
The dates I'm entering are of the form: '2018-11-2 20:00:00' and in the csv file they are present without apostrophes.
It seems that the syntax error has something to do with the date and in the position where it holds a space. I tried entering the STR_TO_DATE function for that column but kept getting syntax errors.
BUT when I try to take some values from the file and manually run the query it works fine! So my thoughts are that it has something to do JMeter's conversion of spaces before sending out the query.
Is the problem with my configurations of JMeter? Since the query is OK manually.
Add apostrophes to insert and remove unnecessary parenthesis
INSERT INTO customer_transactions_current ('column_name1', ... , 'column_name12')
VALUES ('${r1}','${r2}','${r3}','${r4}','${r5}','${r6}','${r7}','${r8}','${r9}','${r10}','${r11}','${r12}');
If you have date issue see using STR_TO_DATE
I want to pass SOME VARIABLES to mysql file from bash shell script.
Here is my shell script.
#!/bin/bash
echo $0 Started at $(date)
mysql -uroot -p123xyzblabla MyMYSQLDBName<mysqlfile.sql PARAM_TABLE_NAME
Please note that it is MYSQL and not SQLPLUS
My MYSQL.sql , I want to read and use passed parameter/argument (PARAM_TABLE_NAME)
select count(*) from PARAM_TABLE_NAME
Question 1: What is the correct syntax to pass variable(PARAM_TABLE_NAME) to sql file (mysqlfile.sql)?
Question 2: How can I print variable(PARAM_TABLE_NAME) in sql file (mysqlfile.sql)?
Basically, I want to make generic SQL script which can load or select data from tables based on received inputs.
Thanks
There is no such thing as passing a parameter to a SQL file. A SQL file is no more than a text file that contains a list of SQL statements. These statements are interpreted by the mysql client program exactly as if you typed them on your keyboard.
The mysql client does not provide the feature you are looking for.
But I can think of a few tricks to achieve a similar effet:
create/populate a configuration table prior to reading your SQL file. Then write your SQL file so that it takes this table contents into account:
bash> mysql -e "INSERT INTO config_table VALUES(1, 2, 3)"
bash> mysql < script.sql
prepend your SQL file with some variables declarations. Then use these variables in the rest of your script:
bash> (echo "SET #var=123;" ; cat script.sql) |mysql
[example script.sql]
SELECT * FROM mytable WHERE id = #var;
write your SQL file with some placeholders that your replace on the fly, e.g with sed:
bash> sed "s/__VAR_A__/mytable/g" script.sql |mysql
[example script.sql]
SELECT * FROM __VAR_A__ WHERE id = 123;
All the above is quite dirty. A much cleaner solution would involve stored procedures or functions. Then you would just pass your parameters as procedure parameters:
bash> PARAM1='foo'; PARAM2='bar'
bash> mysql -e "CALL MyProc($PARAM1);"
bash> mysql -e "SELECT MyFunc($PARAM2);"
note: it is not possible to parametrize a table name in SQL, so you will need to resort to dynamic SQL like this in all cases (except for the sed-based hack, which I do not recommend)
This is an old thread but I think may still be useful to some people. Something like this should work:
mysql -uroot -p123xyzblabla MyMYSQLDBName -e "set #testVar='customer_name'; source mysqlfile.sql;"
Now #testVar (customer_name) is available for you to use in mysqlfile.sql file.
HTH
The way to pass parameters has already been answered in this or other threads. However, specific to the sample in you question, I'd like to add that you can't use the variables declaration method as a placeholder for a table name, as the documentation says:
User variables are intended to provide data values. They cannot be used directly in an SQL statement as an identifier or as part of an identifier, such as in contexts where a table or database name is expected
If you want to use a table name parameter, you can still use the sed or the stored procedures or functions as answered by #RandomSeed
In addition to that, another way is using PREPARE and EXECUTE in your script. The following example allows you to create a database/schema (in case you wanted to use stored procedures you need to have them already stored in a database), like this:
[myscript.sql]
set #s=CONCAT("CREATE DATABASE ", #dbname);
PREPARE stmt FROM #s;
EXECUTE stmt;
Then use any of the proposed syntax in the other questions to set the #dbname variable:
mysql -uroot -p123xyzblabla MyMYSQLDBName -e "set #dbname='mydatabase'; source myscript.sql;"
SQL has the option to dump data into a file, using the INTO OUTFILE option, for exmaple
SELECT * from FIshReport INTO OUTFILE './FishyFile'
The problem is, this command is only allowed if the file didn't exist before it. It creates the file and then enters the data.
So, is there any way to append data to a file this way?
As the MySQL page on SELECT syntax suggests:
http://dev.mysql.com/doc/refman/5.0/en/select.html
the alternative to this is to issue the SELECT from the MySQL client:
However, if the MySQL client software is installed on the remote machine,
you can instead use a client command such as mysql -e "SELECT ..." > file_name
to generate the file on the client host.
which, in your case, would be modified to be:
mysql -e "SELECT * from FishReport" >> file_name
so that you simply append to the file.
From your Tcl script, you could simply issue this as an exec command:
http://www.tcl.tk/man/tcl/tutorial/Tcl26.html
I think MySQL does not allow appending data to an existing file or overwriting an existing file for security reasons.
A work around could be to save resuts in seperate files and then append the using file IO.
You could always append the output from your SQL script to a file using >>
For example (for Sybase):
isql < script.sql >> outputfile.out
I can't tell you what the equivalent is for MySQL but the principle should be the same.
Of course output will all go to one file so if your SQL script is outputting various SQL selects to different output files then you'd need to split the script up.
You could just add it to a variable. Then use a SELECT with UNION.
declare t varchar(100);
set #myvar = concat('
select * INTO OUTFILE \'',file,'\'
from (
select \'',t,'\'
union all
SELECT col from tbl where x
) a'
);
PREPARE stmt1 FROM #myvar;
EXECUTE stmt1;
Deallocate prepare stmt1;