I m loading data in a mysql table with the instruction :
LOAD DATA LOCAL INFILE "/home/user123/Documents/PartageVB/export tables/pays.csv" INTO TABLE T_PAYS FIELDS TERMINATED BY ";" LINES TERMINATED BY "\n"(id, name, cit,actif);
In the file pays.csv, the column cit is boolean, and gets the value 1 or 0.
My problem is that once LOAD DATA is done, cit always gets the value 1 in the mysql table.
Does anyone know where I m wrong?
In the table, cit type is Bit(1).
Thanks
Please use "TINYINT(1)" instead of "Bit(1)".It might work.
Refer this link
Related
First time using sqlldr, simply trying to load a file with single column, and cannot find a proper way to do it :(
What do I put as a delimiter?
Here's my .ctl file :
load data
infile 'myfile.dat'
into table mytable
fields terminated by ''
(mycolumn)
I keep getting errors in the .log like :
Record 4: Rejected - Error on table ..., column ....
ORA-12899: value too large for column "... (actual: 80, maximum: 24)
Even though the values in the file are max 8 chars each or smth :
string1
string2
string3
Any help will be much appreciated.
Many thanks,
G
You don't need the fields terminated by line in this case but you should have a TRUNCATE or APPEND depending on if you want to keep existing data or not.
load data
infile 'myfile.dat'
truncate
into table mytable
(mycolumn)
Why not just change the size of your column to 80
Alter table modify varchar2(80)
I'm trying to use mysql's LOAD DATA LOCAL INFILE syntax to load a .csv file into an existing table. Here is one record from my .csv file (with headers):
PROD, PLANT,PORD, REVN,A_CPN, A_CREV,BRDI, DTE, LTME
100100128144,12T1,2070000,04,3DB18194ACAA,05_01,ALA13320004,20130807,171442
The issue is that I want 3 extra things done during import:
A RECORDID INT NOT NULL AUTO_INTEGER PRIMARY_KEY field should be incremented as each row gets inserted (this table column and structure already exists within the mysql table)
DTE and LTME should be concatenated and converted to a mysql DATETIME format and inserted into an existing mysql column named TRANS_OCR
A CREATED TIMESTAMP field should be set to the current unix timestamp on row insertion (this table column and structure already exists as well within the mysql table)
I'm trying to import this data into the mysql table with the following command:
LOAD DATA LOCAL INFILE 'myfile.csv' INTO TABLE seriallog
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(FLEX_PN, FLEX_PLANT, FLEX_ORDID, FLEX_REV, CUST_PN, CUST_REV, SERIALID)
SET CREATED = CURRENT_TIMESTAMP;
I think I have the CREATED column set properly but the others are causing a mysql warning to be issued:
Warning: Out of range value for column 'FLEX_PN' at row 1
Warning: Row 1 was truncated; it contained more data than there were input columns
Can someone help me with the syntax, the LOAD DATA LOCAL INFILE module is confusing to me...
Figured out the proper syntax to make this work:
sql = """LOAD DATA LOCAL INFILE %s INTO TABLE seriallog_dev
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"'
LINES TERMINATED BY '\\n'
IGNORE 1 LINES
(FLEX_PN, FLEX_PLANT, FLEX_ORDID, FLEX_REV, CUST_PN, CUST_REV, SERIALID, #DTE, #LTME)
SET RECORDID = NULL,
TRANS_OCR = STR_TO_DATE(CONCAT(#DTE,'',#LTME), "%%Y%%m%%d%%H%%i%%s"),
CREATED = CURRENT_TIMESTAMP;"""
params = (file,)
self.db.query( sql, params )
Mind you--this is done with python's mysqldb module.
CAVEAT
The only issue with this solution is that for some reason my bulk insert only inserts the first 217 rows of data from my file. My total file size is 19KB so I can't imagine that it is too large for the mysql buffers... so what gives?
more info
Also, I just tried this syntax directly within the msyql-server CLI and it works for all 255 records. So, obviously it is some problem with python, the python mysqldb module, or the mysql connection that the mysqldb module makes...
DONE
I JUST figured out the problem, it had nothing to do with the load data local infile command but rather the method I was using to convert my original .dbf file into the .csv before attempting to import the .csv. For some reason the mysql import method was running on the .csv before .dbf to .csv conversion method finished -- resulting in a partial data set being found in the .csv file and imported... sorry to waste everyone's time!
I have a mysql database with a single table, that includes an autoincrement ID, a string and two numbers. I want to populate this database with many strings, coming from a text file, with all numbers initially reset to 0.
Is there a way to do it quickly? I thought of creating a script that generates many INSERT statements, but that seems somewhat primitive and slow. Especially since mysql is on remote site.
Yes - use LOAD DATA INFILE docs are here Example :
LOAD DATA INFILE 'csvfile'
INTO TABLE table
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
IGNORE 0 LINES
(cola, colb, colc)
SET cold = 0,
cole = 0;
Notice the set line - here is where you set a default value.
Depending on your field separator change the line FIELDS TERMINATED BY ','
The other answers only respond to half of your question. For the other half (zeroing numeric columns):
Either:
Set the default value of your number columns to 0,
In your text file, simply delete the numeric values,
This will cause the field to be read by LOAD INFILE as null, and the defauly value will be assigned, which you have set to 0.
Or:
Once you have your data in the table, issue a MySQL command to zero the fields, like
UPDATE table SET first_numeric_column_name = 0, second_numeric_column_name = 0 WHERE 1;
And to sum everything up, use LOAD DATA INFILE.
If you have access to server's file system, you can utilize LOAD DATA
If you don't want to fight with syntax, easiest way (if on windows) is to use HeidiSQL
It has friendly wizard for this purpose.
Maybe I can help you with right syntax, if you post sample line from text file.
I recommend you to use SB Data Generator by Softbuilder (which I work for), download and install the free trial.
First, create a connection to your MySQL database then go to “Tools -> Virtual Data” and import your test data (the file must be in CSV format).
After importing the test data, you will be able to preview them and query them in the same window without inserting them into the database.
Now, if you want to insert test data into your database, go to “Tools -> Data Generation” and select "generate data from virtual data".
SB data generator from Softbuilder
I have created a database and a table. I have also created all the fields I will be needing. I have created 46 fields including one that is my ID for the row. The CSV doesn't contain the ID field, nor does it contain the headers for the columns. I am new to all of this but have been trying to figure this out. I'm not on here being lazy asking for the answer, but looking for directions.
I'm trying to figure out how to import the CSV but have it start importing data starting at the 2nd field, since I'm hoping the auto_increment will fill in the ID field, which is the first field I created.
I tried these instructions with no luck. Can anyone offer some insight?
The column names of your CSV file must match those of your table
Browse to your required .csv file
Select CSV using LOAD DATA options
Check box 'ON' for Replace table data with file
In Fields terminated by box, type ,
In Fields enclosed by box, "
In Fields escaped by box, \
In Lines terminated by box, auto
In Column names box, type column name separated by , like column1,column2,column3
Check box ON for Use LOCAL keyword.
Edit:
The CSV file is 32.4kb
The first row of my CSV is:
Test Advertiser,23906032166,119938,287898,,585639051,287898 - Engager - 300x250,88793551,Running,295046551,301624551,2/1/2010,8/2/2010,Active,,Guaranteed,Publisher test,Maintainer test,example-site.com,,All,All,,Interest: Dental; custom geo zones: City,300x250,-,CPM,$37.49 ,"4,415","3,246",3,0,$165.52 ,$121.69 ,"2,895",805,0,0,$30.18 ,$37.49 ,0,$0.00 ,IMPRESSIONBASED,NA,USD
You can have MySQL set values for certain columns during import. If your id field is set to auto increment, you can set it to null during import and MySQL will then assign incrementing values to it. Try putting something like this in the SQL tab in phpMyAdmin:
LOAD DATA INFILE 'path/to/file.csv' INTO TABLE your_table FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' SET id=null;
Please look at this page and see if it has what you are looking for. Should be all you need since you are dealing with just one table. MYSQL LOAD DATA INFILE
So for example you might do something like this:
LOAD DATA INFILE 'filepath' INTO TABLE 'tablename' FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' (column2, column3, column4);
That should give you an idea. There are of course more options that can be added as seen in the above link.
be sure to use LOAD DATA LOCAL INFILE if the import file is local. :)
I am using the following statement to load data from a file into a table:
LOAD DATA LOCAL INFILE '/home/100000462733296__Stats"
INTO TABLE datapoints
FIELDS TERMINATED BY '|'
LINES TERMINATED BY '\n'
(uid1, uid2, status);
Now, if I want to enter a custom value into uid1, say 328383 without actually asking it to read it from a file, how would I do that? There are about 10 files and uid1 is the identifier for each of these files. I am looking for something like this:
LOAD DATA LOCAL INFILE '/home/100000462733296__Stats"
INTO TABLE datapoints
FIELDS TERMINATED BY '|'
LINES TERMINATED BY '\n'
(uid1="328383", uid2, status);
Any suggestions?
The SET clause can be used to supply values not derived from the input file:
LOAD DATA LOCAL INFILE '/home/100000462733296__Stats"
INTO TABLE datapoints
FIELDS TERMINATED BY '|'
LINES TERMINATED BY '\n'
(uid1, uid2, status)
SET uid1 = '328383';
It's not clear what the data type of uid1 is, but being that you enclosed the value in double quotes I assumed it's a string related data type - remove the single quotes if the data type is numeric.
There's more to read on what the SET functionality supports in the LOAD FILE documentation - it's a little more than 1/2 way down the page.
You could use a python interactive shell instead of MySQL shell to interactvely provide values for MySQL tables.
Install the python inerpreter from python.org (only needed if you are under windows, otherwise you have it already), and the mysql connector from http://sourceforge.net/projects/mysql-python/files/ (ah, I see you are on Lunux/Unix --just install teh mysqldb package then)
After that, you type these three lines in the python shell:
import MySQLdb
connection = MySQLdb.connect(" <hostname>", "< user>", "<pwd>", [ "<port>"] )
cursor = connection.cursor
Adter that you can use the cursor.execute method to issue SQL statements, but retaining th full flexibility of python to change your data.
For example, for this specific query:
myfile = open("/home/100000462733296__Stats")
for line in file:
uid1, uid2, status = line.split("|")
status = status.strip()
cursor.execute("""INSERT INTO datapoints SET uid1="328383", uid2=%s, status=%s""" %(uid2, status) )
voilá !
(maybe with a try: clause around the the "line.split " line to avoid an exception on the last line)
If you don't already, you may learn Python in under one hour with the tutorial at python.org
-- it is really worth it, even if the only things you do at computers is to import data into databases.
2 quick thought (one might be applicable :)):
change the value of uid1 in the file to 328383 in every line.
temporarily change the uid1 column in the table to be non-mandatory, load the contents of the file, then run a query that sets the value to 328383 in every row. Finally, reset the column to mandatory.