mysql infile can't access #user variable in sub query - mysql

Question

I am trying to read a csv file and insert rows into a table. I am able to insert them without any problem when I am assigning the value. But the same sql stops working when I try to use a #user variable.
All help is appreciated.
This works:
LOAD DATA INFILE 'tmp/test.csv'
INTO TABLE T1
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES
( #var1 )
set create_timestamp = now(),
col1 = ( select max(id) from T2 where
col2 = 1234 );
This doesn't work:
LOAD DATA INFILE 'tmp/test.csv'
INTO TABLE T1
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES
( #var1 )
set create_timestamp = now(),
col1 = ( select max(id) from T2 where
col2 = #var1 );

Answer 1

I suspect that the subquery ( select max(id) from is being executed before a value is assigned to #var1.
I'm actually surprised that the first example works at all; I've never tried running a subquery in a LOAD DATA.
When you say "This doesn't work" do you mean that the LOAD DATA throws an error? Or do you mean that the LOAD DATA completes successfully, but with unexpected results?
In the latter case, I'd recommend a simple test, doing a separate SET #var1 = 1234; statement before the LOAD DATA, and see what happens for the first row.
If it's throwing an error, it may be that a subquery isn't supported in that context. (We'd need to consult the MySQL Reference Manual, to see if that is supported.)
That's my guesses. 1) unexpected order of operations (#var1 is being evaluated before the value from the row is assigned to #var1), or 2) a subquery isn't valid in that context.
EDIT
According the the MySQL Reference Manual 5.7
http://dev.mysql.com/doc/refman/5.7/en/load-data.html
It looks like a subquery is supported in the context of a LOAD DATA statement.

Answer 2

I'm having the same problems. Part of my problem was that, using your table definition as an example, T1.col1 is set to NOT NULL but the expression "select max(id) from T2 where col2 = #var1" resulted in nothing being returned and so MySQL attempted to assign a null value to col1. That didn't solve everything, but it was a start.
Here's what I'm working with:
LOAD DATA LOCAL INFILE 'streets.csv'
INTO TABLE streets
CHARACTER SET latin1
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(pre_dir, name, suffix, post_dir, #community, #state)
SET community_id = (SELECT community_id FROM communities_v WHERE community_name = #community AND state_abbr = #state);
EDIT: I had originally put a bunch of other stuff here that I thought was related to the problem, but as it turns out, didn't. It really was a data problem. The subquery I'm using had certain combinations of #community and #state that had no value for community_id, so I would double-check your own subquery to see if it returns a value in all cases.