I'm working on importing a very large CSV file into SQLite. My understanding is that LOAD DATA INFILE is my best bet. I've created a table for it to reside in, and am attempting to execute the following query
LOAD DATA LOCAL INFILE 'F:/Downloads/NielsonReport.csv'
INTO TABLE neilson;
IGNORE 1 LINES
but, I get the following error:
Error while executing SQL query on database 'test': near "LOAD": syntax error
I seem to be getting an error along these lines regardless of what I'm trying to execute.
I feel like I'm missing something very basic, and would appreciate any help resolving this problem (I've been referencing this page for information so far)
When you are using SQLite, it would be a good idea to reference the SQLite documentation instead.
Anyway, SQLite itself does not have a CSV import function. But the sqlite3 command-line shell allows to import CSV files with the .import command.
Use import command like this.
.import '/Users/haseeb/Desktop/names_data.txt' Names
Related
I am learning to use MySQL Workbench and am having trouble importing data. I have created a database, a schema, and a table. The table has three columns. I have been using the following query to try to populate the table using a .txt file. The file uses new lines to separate rows and two spaces to separate columns.
LOAD DATA LOCAL INFILE 'C:\Users\Nick\Desktop\toImport.txt'
INTO TABLE schema.table
FIELDS TERMINATED BY ' '
LINES STARTING BY '\n';
Unfortunately, I keep getting the following error:
Error Code: 2. File 'C:UsersNickDesktoptoImport.txt' not found (Errcode: 2 - No such file or directory)
Any suggestions on what I am doing wrong? This is my first go at learning SQL so apologies if this is quite basic. Thanks!
As suggested in the comments, using "/" allowed the script to execute. However, using "\" did not work.
Once I removed a column from the table, all data imported perfectly.
Thanks for the help!
I have a csv file I want to load into a database, but can't seem to have a row as a value for the insert function
for row in readCSV:
cur.execute("INSERT INTO IPaddresses(Start) VALUES(row[0])")
I use the module pymysql
Not sure why you're not using "load data [local] infile". It'd be much easier. Check out the command here: http://dev.mysql.com/doc/refman/5.0/en/load-data.html and someone with a similar problem here Python/MySQL - LOAD DATA LOCAL INFILE
What exactly happens when you run the code? Errors? Unexpected values in your table?
I have a CSV that is a partial projection of an origin table. I have the same table structure in my DB.
I would like to import only those columns into my DB, given that no additional NOT NULL constraints are in place (I explicitly disabled some of them). I don't know how to import them.
I have tried the following: from MySQL Workbench, right click on table and then Edit table data, then on the screen I tried the "Import records from an external file" button, loaded the CSV file but I got the following error:
[Window Title]
MySQL Workbench
[Main Instruction]
Error importing recordset
[Content]
error calling Python module function SQLIDEUtils.importRecordsetDataFromFile
[OK]
The column names are the same as in the DB but these are partial (not all columns as DB). The table is currently empty.
What can I do to import the data into MySQL?
It turns out that is the error that tool gives for ANY problems importing CSV data. They have opened a bug for more descriptive error responses.
For me it turned out that it can not work with non-Windows line breaks. So if your file came from Unix/Linux or Mac it will not work. You can just open it in Excel though and re-save it as an MS-DOS CSV and then it works. Other things that can make it throw up are any use of ";".
Also the tool has NO column mapping options so you have to have your import file matched to the table setup perfectly. If you have columns mis-matched or data types mis-matched it will also throw-up.
I got the same question and error when I was importing a csv file into MAC Mysqlworkbench.
I restored my file as windows comma separated (.csv). This works for me.
Check do you have permissions to import (on insert), and if you import clearing old data check do you have permissions on delete.
If it won't help, create a new table with least restrictions which has absolutely identical columns as in CSV file, try to import to it. If ok - just update/insert into your target table
If it won't help, try to import on another database.
If it won't help, check CSV file - is all contents is ok? (may be it should try first)
I've tried several ways to convert the SQL file found here into an SQLite file. Ways include this script (which produces the table properly in SQLite but empty), this script, and I've tried outputting the data from MySQL as CSV which I then tried to import into SQLite, getting apparently the wrong amount of columns...
Is there no easy way to convert from MySQL to SQLite?
While trying to start uplaod the 3.9 GB sql file via BigDump there is error
UNEXPECTED: Can't set file pointer
behind the end of file
Dump of database was exported from PHPMyAdmin. File is not corrupted. What is the problem? What are other ways to import such a big database?
Bigdump uses a INSERT INTO table VALUES (....) kind of method.
This is a very slow way of inserting!
Use
LOAD DATA INFILE 'c:/filename.csv' INTO TABLE table1
Instead. Note the use of forward slashes even on Windows.
See: http://dev.mysql.com/doc/refman/5.1/en/load-data.html
This is the fastest way possible to insert data into a MySQL table.
It will only work if the input file is on the same server as the MySQL server though.
I get similar error: I can't seek into .sql
The reason for this error is, that BigDump tries to set pointer at the end of .sql-File and then find out its size (using fseek() and fteil() functions). As fseek() is failing when you work with files over 2GB, you get this error. Solution is to split your SQL-File into chunks of 1,5GB - 2GB size...