I am attempting a LOAD DATA INFILE and getting the above error.
LOAD DATA INFILE '$file'
REPLACE INTO TABLE $custom_parts
FIELDS TERMINATED BY ',' ESCAPED BY '\\\\'
LINES TERMINATED BY '\\r\\n'
IGNORE 1 LINES
(`partsno`, `mfg`, `cond`, `price`, `is_deleted`, #date_added)
SET `date_added` = STR_TO_DATE(#date_added, '%c/%e/%Y'),
`prtky` = REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(partsno, ' ', '' )
, '\\\\', '' ) , '/', '' ) , '_', '' ) , '.', '' ) , '-', '' )
The columns of the file are so
Part Number,MFR,Condition,price,is_deleted,date_added
Whenever I run the PHP to load this, I get the error. I am entrigued as why this is occuring. I believe that it is an issue with the user variable not being assigned and I am just looking for verification.
You cannot use variables for dynamic table names.
You can only do this using prepared statements,
However
You cannot use load data infile inside a stored procedure and I'm not sure you can use it in a prepared statement either.
If you use MySQL from a higher level program (php, pascal, whatever) you can
resolve the variable before constructing the query;
Check the columnname against a whitelist to prevent SQL-injection
Feed MySQL the expanded statement.
See this question for sample code: How to prevent SQL injection with dynamic tablenames?
Also, if you are using dynamic tablenames, escape them using ` backticks. This prevents MySQL from bombing if the tablenames contains funny chars or is a reserved word.
I answered my own question. ---- User variables may be used in most contexts where expressions are permitted. This does not currently include contexts that explicitly require a literal value, such as in the LIMIT clause of a SELECT statement, or the IGNORE N LINES clause of a LOAD DATA statement.
Related
set #tmpGuid = REPLACE( uuid(),'-',''); set #fieldName = concat('deneme' , '_' , #tmpGuid); ALTER TABLE table_name RENAME COLUMN deneme to #fieldName;
https://i.stack.imgur.com/ciQ44.png
how can use my variable on mysql commands
Variables are meant to store data, not code. In SQL, table and column names count as the latter. From User-Defined Variables:
User variables are intended to provide data values. They cannot be used directly in an SQL statement as an identifier or as part of an identifier, such as in contexts where a table or database name is expected, or as a reserved word such as SELECT.
Many DBMS don't allow DDL statements to be parameterised. I.e. alter, create statements and such.
However this doesn't mean it's impossible. What you'd need to do is convert your statement to dynamic SQL, which you can manipulate in every way possible (including inserting parameters into the string).
Then simply executing it.
In your example:
SET #tmpGuid = REPLACE( uuid(),'-','');
SET #fieldName = concat('deneme' , '_' , #tmpGuid);
SET #sql = CONCAT('ALTER TABLE table_name RENAME COLUMN deneme to ', #fieldName);
PREPARE dynamic_SQL FROM #sql;
EXECUTE dynamic_SQL;
DEALLOCATE PREPARE dynamic_SQL;
Ofcourse using dynamic SQL has it's own troubles, and opens up potential risks. But it's the only way to do something like this.
See it in action: Here
I have a dataset as follows,
Date,Time,ID,Name,Count
01-MAY-2009,00:00,4,Town Hall (West),209
01-MAY-2009,02:00,17,Collins Place (South),28
01-MAY-2009,23:00,18,Collins Place (North),36
For this I have created table with following schema,
CREATE TABLE table_name(date DATE, time TIME, id int, name VARCHAR(50), count int);
And for loading the table from ".csv" file as,
LOAD DATA INFILE '/home/cloudera/dataset.csv'
INTO TABLE table_name
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
(#var1, #var2, id, name, count)
SET date = STR_TO_DATE(#var1, '%d-%b-%Y')
SET time = TIME(#var2, "%H:%i");
But I get an error as,
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'SET time = TIME(#var2, "%H:%i")' at line 1
I can't understand what the error is. I have tried going through MySQL website and documentation, but can't make out what the correct format is. Can anyone please me. Thanks in advance.
I don't think that you even need to be using the TIME function here. Your current hour:minute string time literals should be good enough, q.v. the documentation:
Be careful about assigning abbreviated values to a TIME column. MySQL interprets abbreviated TIME values with colons as time of the day. That is, '11:12' means '11:12:00', not '00:11:12'
As valid literals, your times would be interpreted as having a zero second component. So, if you just insert those time strings as is, I think it should work. Try the following code:
LOAD DATA INFILE '/home/cloudera/dataset.csv'
INTO TABLE table_name
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
(#var1, Time, id, name, count)
SET date = STR_TO_DATE(#var1, '%d-%b-%Y');
Tim is right in pointing out that you do not need to convert your time data in the load data infile statement.
Just to answer why you get a syntax error: load data infile can only have a single set clause, in which assignments to multiple columns are separated by comma. Your code has multiple set clauses, hence it fails.
Also, the time() function does not have a 2nd parameter for a pattern. The function you need to use is called str_to_date().
So, it should look like as follows:
...
SET date = STR_TO_DATE(#var1, '%d-%b-%Y'), time = TIME(str_to_date(#var2, "%H:%i"));
I have hundreds of databases with some structurally identical (but over time changing) tables. Data of a certain table (from all DBS) should be copied into one central table ('ex_objects') in a central database ('db_central'; there are no pk conflicts). I've used a trigger in each DB for this purpose. But since the table structure is changing almost on a daily basis, it's a pain to update the fields in the ON DUPLICATE KEY part of the trigger's query. And someone could forget to modify the trigger after modifying the table structure. So I came across a possible solution to build that particular part of the query dynamically. This actually works on a script (PHP) basis, but I don't get the trigger done. I don't see what I am missing here.
BEGIN
DECLARE VAL_FIELDS TEXT;
SET VAL_FIELDS = (SELECT GROUP_CONCAT( CONCAT(COLUMN_NAME,"=values(", COLUMN_NAME,")") SEPARATOR ", ") FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = 'db_central' AND TABLE_NAME = 'ex_objects');
-- SELECT GROUP_CONCAT( CONCAT(COLUMN_NAME,"=values(", COLUMN_NAME,")") SEPARATOR ", ") INTO VAL_FIELDS FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = 'db_central' AND TABLE_NAME = 'ex_objects';
-- SELECT #VAL_FIELDS := GROUP_CONCAT( CONCAT(COLUMN_NAME,"=values(", COLUMN_NAME,")") SEPARATOR ", ") INTO VAL_FIELDS FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = 'db_central' AND TABLE_NAME = 'ex_objects';
IF NEW.online = 1 THEN
INSERT INTO db_central.ex_objects
SELECT * FROM ex_objects WHERE id = NEW.id AND client_id = NEW.client_id AND NEW.online = 1
ON DUPLICATE KEY UPDATE VAL_FIELDS;
END IF;
END
I get the error that there's something wrong at ; END IF; END. Well, that means for me, that either the VAL_FIELDS variable after KEY UPDATE isn't recognized at all or the parser expects at least one equation (something like VAL_FIELDS = whatever). But in this case, it wouldn't solve my underlying problem at all.
The SELECT GROUP_CONCAT( CONCAT(COLUMN_NAME,"=v ... FROM INFORMATION_SCHEMA.C ... Query works well and results in something similar to id=values(id), xfield=values(xfield), yfield=values(yfield) (but with a few hundred fields, since the table is actually pretty huge).
The full error: SQL Error (1064): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ';
END IF;
END' at line 10
You must build the SQL using CONCAT, etc, then prepare and execute it. There is no 'interpolation'.
Since it is a TRIGGER, that won't work.
However, since a TRIGGER applies to a particular table, you may as well simply spell out the query, not construct it. You have most of what it takes to manually get the query generated for you (SELECT ... I_S ...). Add some more columns for OLD.col to fill out the thing; viola, you have the query that you need.
I am trying to load data from a CSV into a database in MySQL workbench. The table I am loading into has an auto increment ID column. I am trying to get the query to recognize that I want it to keep the first column as Null, and I put NULL as the value in the CSV, but I cannot get the SET ... NULL command to recognize the name of the ID column. Here is the SQL I am using:
load data infile 'filenam.csv'
INTO TABLE table_name
fields Terminated By ','
LINES TERMINATED BY ',,'
SET column_name = null
I suspect I am making a simple syntax error that is causing the problem. But I cannot figure out what it is.
If you put NULL as the value in the CSV file then you shouldn't need the "SET column_name = null" in the statement. AFAIK, the SET value should be used to supply values not derived from the input file or to perform calculations to the value before insertion. This statement should work fine since you said you specified NULL in the CSV. However, make sure you specified NULL "properly" according to the documentation. I always use \N in my import files.
LOAD DATA INFILE 'filename.csv'
INTO TABLE table_name
FIELDS TERMINATED BY ','
LINES TERMINATED BY ',,'
Secondly, you can discard the NULL specified in the CSV file by assigning it to a user variable and then specifying the column value with SET. However, you need to specify a column list
LOAD DATA INFILE 'filename.csv'
INTO TABLE table_name (#dummy, column_2, column_3)
FIELDS TERMINATED BY ','
LINES TERMINATED BY ',,'
SET column_name = NULL
I have one other thought based on the MySQL docs and dependent upon how your server is configured. Comment if this does not work and I will provide more options.
http://dev.mysql.com/doc/refman/5.1/en/load-data.html
I'm importing a CSV file with dotted german dates into a MySQL database. I want the dates in the CSV to automatically be formatted correctly to the correct data type fields used by MySQL.
I'm using Sequel Pro for the import. I gather I'm supposed to use the STR_TO_DATE function, but I just can't wrap my head around how to use Add Value or Expression in the program.
German date
Here are the dates in the CSV file:
DD.MM.YYYY e.g.: 28.01.1978
MySQL date
Here is what I want to end up with in the database:
YYYY-MM-DD
e.g.: 1978-01-28
Here's what I've tried
I put in STR_TO_DATE('$5', '%d.%m.%Y'); into Add Value or Expression, but this only gives the following error message:
[ERROR in row 1] You have an error in your SQL syntax; check the manual that corresponds to your
MySQL server version for the right syntax to use near '06.04.1997'', '%d.%m.
%Y');,'2KMX','43354997')' at line 2
Any ideas?
You need import the date field in a varchar fields (temp_varchar_field) first, after that, you can use something like:
update table_name set final_date=STR_TO_DATE(temp_varchar_field,'%d.%m.%Y');
You should do something like:
Create a temporary field: alter table table_name add column temp_varchar_field varchar(10);
Import, using Sequel Pro, the CVS file but using the temp_varchar_field for the date.
update table_name set final_date=STR_TO_DATE(temp_varchar_field,'%d.%m.%Y');
Delete the temp field if everything was imported properly. Using: alter table_name drop column temp_varchar_field;
I just got it to work with this piece of SQL-code:
load data local infile 'myfile.csv' into table `mytable`
fields terminated by ','
enclosed by '"'
lines terminated by '\n'
(surname, name, #germandate, telephone, etc)
set birthyear = STR_TO_DATE(#germandate , "%d.%m.%Y")
;
The clue here being the #germandate variable which is turned into the default MySQL date by setting the respective column with STR_TO_DATE(). No hacks needed! :)
It is easier if your CSV import would contain a date as a MySQL string, but it can be done otherwise too:
Step 1:
Define a varchar(10) for your german dates and import the data.
Step 2:
Add another field to your table:
ALTER TABLE `yourtable`
ADD COLUMN `your_mysql_date` DATE NULL;
Step 3:
Move the data:
UPDATE yourtable
SET your_mysql_date = CONCAT(
RIGHT(your_german_date,4),
'-',
MID(your_german_date,4,2),
'-',
LEFT(your_german_date,2)
);
...done!
There might be an easier way to solve this, but this way you have alot of control over the data and the formatting.