I have read many threads but can't find the right specific answer. I get this error message when I try to import additional data into an existing table. The field names are all aligned correctly, but not every row has data in every field. For example, although I have a field named middle_name, not every row has a middle name in it. During the import process, is this blank field not counted as a field and thus throwing off the field count?
I have managed to get most of the data to import by making sure I had a blank column to allow for the auto-increment of ID, as well as leaving the header row in the file but choosing 1 row to skip on the import.
Now the problem is the last row won't import - get error message Invalid format of CSV input on line 19. When I copy the file to Text Wrangler, the last row ends with ,,,,,. This accounts for the last 5 columns which are blank. I need to know what the trick is to get the last row to import.
Here are the settings I have been using:
I’ve had similar problems (with a tab-separated file) after upgrading from an ancient version of phpMyAdmin. The following points might be helpful:
phpMyAdmin must have he correct number of columns. In older versions of phpMyAdmin you could get away with not supplying empty values for columns at the end of the row, but this is no longer the case.
If you export an Excel file to text and columns at the start or end of rows are completely empty, Excel will not export blanks for those rows. You need to put something in, or leave blank then edit the resulting file in a text editor with regular expressions, e.g. to add a blank first row, search for ^ and replace with , (CSV file) or \t (tab file); to add two columns to the end search for $ and replace with ,, or \t\t etc.
Add a blank line to the bottom of the file to avoid the error message referring to the last line of data. This seems to be a bug that has been fixed in newer versions.
Whilst in the text editor, also check the file encoding as Excel sometimes saves as things like UTF-16 with BOM which phpMyAdmin doesn’t like.
I saw the same error while trying to import a csv file with 20,000 rows into a custom table in Drupal 7 using phpmyadmin. My csv file didn't have headers and had one column with many blanks in it.
What worked for me: I copied the data from Excel into Notepad (a Windows plain text editor) and then back into a new Excel spreadsheet and re-saved it as a csv. I didn't have to add headers and left the blank rows blank. All went fine after I did that.
You'll never have this problem if you keep your 1st row the header row, even if your table already has a header. You can delete the extra header row later.
Why this helps is that, then mysql knows how many cells can possibly contain data, and you wont have to fill in dummy data or edit the csv or any of those things.
I’ve had a similar problem, Then I have tried in Mysql Workbench.
table data import through CSV file easily and I have done my work perfectly in MySQL Workbench.
As long as the CSV file you're importing has the proper number of empty columns, it should be no problem for phpMyAdmin (and that looks like it's probably okay based on the group of commas you pasted from your last line).
How was the CSV file generated? Can you open it in a spreadsheet program to verify the line count of row 19?
Is the file exceptionally large? (At 19 rows, I can't imagine it is, but if so you could be hitting PHP resource limits causing early termination of the phpMyAdmin import).
Make sure you are trying to import the table into the database and not the database > table.
I had this same issue, tried all listed and finally realized I needed to go up a level to the database
I have a CSV that is a partial projection of an origin table. I have the same table structure in my DB.
I would like to import only those columns into my DB, given that no additional NOT NULL constraints are in place (I explicitly disabled some of them). I don't know how to import them.
I have tried the following: from MySQL Workbench, right click on table and then Edit table data, then on the screen I tried the "Import records from an external file" button, loaded the CSV file but I got the following error:
[Window Title]
MySQL Workbench
[Main Instruction]
Error importing recordset
[Content]
error calling Python module function SQLIDEUtils.importRecordsetDataFromFile
[OK]
The column names are the same as in the DB but these are partial (not all columns as DB). The table is currently empty.
What can I do to import the data into MySQL?
It turns out that is the error that tool gives for ANY problems importing CSV data. They have opened a bug for more descriptive error responses.
For me it turned out that it can not work with non-Windows line breaks. So if your file came from Unix/Linux or Mac it will not work. You can just open it in Excel though and re-save it as an MS-DOS CSV and then it works. Other things that can make it throw up are any use of ";".
Also the tool has NO column mapping options so you have to have your import file matched to the table setup perfectly. If you have columns mis-matched or data types mis-matched it will also throw-up.
I got the same question and error when I was importing a csv file into MAC Mysqlworkbench.
I restored my file as windows comma separated (.csv). This works for me.
Check do you have permissions to import (on insert), and if you import clearing old data check do you have permissions on delete.
If it won't help, create a new table with least restrictions which has absolutely identical columns as in CSV file, try to import to it. If ok - just update/insert into your target table
If it won't help, try to import on another database.
If it won't help, check CSV file - is all contents is ok? (may be it should try first)
I have a MySql database on my server with a table named table_1. However I imported a csv file which occasionally included "café". However the "é" was not inserted into the database table, so I have been left with the text "caf". So what I would like to know is how can I replace the word "caf" in my database table with "cafe"?
Looks like an encoding issue to me - make sure you're using UTF-8 throughout your DB, and reimport your CSV.
If you've used the LOAD DATA command in MySQL, you can pass it a CHARACTER SET, which, when set to 'utf8' should allow you to import that file correctly.
This is a common problem of encoding. I sugest that you change your mysql database to utf-8 via GUI or with this information
We're often faced with the need to send a data file to one of our clients with data from the database he/she needs to translate. Most of the time this export is CSV or XLS.
Most of the time we create a csv dump with phpmyadmin and get an xls file in return with the translated data. The problem is that most of the time the data is UTF8 and when the file is returned as xls each and every time we load the data into mysql again we end up with utf8 problems, characters not being displayed properly, etc ...
We've already doublechecked everything in mysql from my.conf to column charactersets and everything is set correctly to UTF8.
My question is not how to fix the encoding issue since that's been solved but how we would best proceed in the future handling this situation? What export format should we hand over? How should we import (just mysql load data infile or our own processing scripts). What is the general consensus on how to handle this situation?
We would like to continue using excel if possible since that's the format almost everybody expects including our clients' translation agencies. Our clients' ease of use is the most important factor here, without overloading us with major issues each time. The best of both worlds :)
The application I am currently working on includes the functionality of data import as well. The data is mostly encoded in utf-8.
My approach is to preprocess the imported CSV (or tab delimited)(in any encoding) file to a correct utf-8 encoded temporary CSV file in client script (Python) and load the contents of the file using LOAD DATA INFILE statement.
The encoding of the file is controled by character_set_database system variable (the variable should be set on the server level) and starting from MySQL 5.1.17 can be overridden by the CHARACTER SET clause of the LOAD DATA INFILE.
The only thing one should know is that MySQL stores up to 3 bytes for each character instead of 4 (that might be a problem for orient languages).
To export lots of data efficienly you can use SELECT ... INTO OUTFILE statement.
Can any one explain how to import a Microsoft Excel file in to a MySQL database?
For example, my Excel table looks like this:
Country | Amount | Qty
----------------------------------
America | 93 | 0.60
Greece | 9377 | 0.80
Australia | 9375 | 0.80
There's a simple online tool that can do this called sqlizer.io.
You upload an XLSX file to it, enter a sheet name and cell range, and it will generate a CREATE TABLE statement and a bunch of INSERT statements to import all your data into a MySQL database.
(Disclaimer: I help run SQLizer)
Below is another method to import spreadsheet data into a MySQL database that doesn't rely on any extra software. Let's assume you want to import your Excel table into the sales table of a MySQL database named mydatabase.
Select the relevant cells:
Paste into Mr. Data Converter and select the output as MySQL:
Change the table name and column definitions to fit your requirements in the generated output:
CREATE TABLE sales (
id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
Country VARCHAR(255),
Amount INT,
Qty FLOAT
);
INSERT INTO sales
(Country,Amount,Qty)
VALUES
('America',93,0.60),
('Greece',9377,0.80),
('Australia',9375,0.80);
If you're using MySQL Workbench or already logged into mysql from the command line, then you can execute the generated SQL statements from step 3 directly. Otherwise, paste the code into a text file (e.g., import.sql) and execute this command from a Unix shell:
mysql mydatabase < import.sql
Other ways to import from a SQL file can be found in this Stack Overflow answer.
Export it into some text format. The easiest will probably be a tab-delimited version, but CSV can work as well.
Use the load data capability. See http://dev.mysql.com/doc/refman/5.1/en/load-data.html
Look half way down the page, as it will gives a good example for tab separated data:
FIELDS TERMINATED BY '\t' ENCLOSED BY '' ESCAPED BY '\'
Check your data. Sometimes quoting or escaping has problems, and you need to adjust your source, import command-- or it may just be easier to post-process via SQL.
There are actually several ways to import an excel file in to a MySQL database with varying degrees of complexity and success.
Excel2MySQL. Hands down, the easiest and fastest way to import Excel data into MySQL. It supports all verions of Excel and doesn't require Office install.
LOAD DATA INFILE: This popular option is perhaps the most technical and requires some understanding of MySQL command execution. You must manually create your table before loading and use appropriately sized VARCHAR field types. Therefore, your field data types are not optimized. LOAD DATA INFILE has trouble importing large files that exceed 'max_allowed_packet' size. Special attention is required to avoid problems importing special characters and foreign unicode characters. Here is a recent example I used to import a csv file named test.csv.
phpMyAdmin: Select your database first, then select the Import tab. phpMyAdmin will automatically create your table and size your VARCHAR fields, but it won't optimize the field types. phpMyAdmin has trouble importing large files that exceed 'max_allowed_packet' size.
MySQL for Excel: This is a free Excel Add-in from Oracle. This option is a bit tedious because it uses a wizard and the import is slow and buggy with large files, but this may be a good option for small files with VARCHAR data. Fields are not optimized.
Not sure if you have all this setup, but for me I am using PHP and MYSQL. So I use a PHP class PHPExcel. This takes a file in nearly any format, xls, xlsx, cvs,... and then lets you read and / or insert.
So what I wind up doing is loading the excel in to a phpexcel object and then loop through all the rows. Based on what I want, I write a simple SQL insert command to insert the data in the excel file into my table.
On the front end it is a little work, but its just a matter of tweaking some of the existing code examples. But when you have it dialed in making changes to the import is simple and fast.
the best and easiest way is to use "MySQL for Excel" app that is a free app from oracle. this app added a plugin to excel to export and import data to mysql. you can download that from here
When using text files to import data, I had problems with quotes and how Excel was formatting numbers. For example, my Excel configuration used the comma as decimal separator instead of the dot.
Now I use Microsoft Access 2010 to open my MySql table as linked table. There I can simply copy and paste cells from Excel to Access.
To do this, first install the MySql ODBC driver and create an ODBC connection.
Then in access, in the "External Data" tab, open "ODBC Database" dialog and link to any table using the ODBC connection.
Using MySql Workbench, you can also copy and paste your Excel data into the result grid of MySql Workbench. I gave detailed instructions in this answer.
Fastest and simpliest way is to save XLS as ODS (open document spreasheet) and import it from PhpMyAdmin
For a step by step example for importing Excel 2007 into MySQL with correct encoding (UTF-8) search for this comment:
"Posted by Mike Laird on October 13 2010 12:50am"
in the next URL:
http://dev.mysql.com/doc/refman/5.1/en/load-data.html
You could use DocChow, a very intuitive GIU for importing Excel into MySQL, and it's free on most common platforms (including Linux).
More especially if you are concerned about date, datetime datatypes, DocChow easily handles datatypes. If you are working with multiple Excel spreadsheets that you want to import into one MySQL table DocChow does the dirty work.
Step 1 Create Your CSV file
Step 2 log in to your mysql server
mysql -uroot -pyourpassword
Step 3
load your csv file
load data local infile '//home/my-sys/my-excel.csv' into table my_tables fields terminated by ',' enclosed by '"' (Country, Amount,Qty);
Another useful tool, and as a MySQL front-end replacement, is Toad for MySQL. Sadly, no longer supported by Quest, but a brilliant IDE for MySQL, with IMPORT and EXPORT wizards, catering for most file types.
If you are using Toad for MySQL steps to import a file is as follows:
create a table in MySQL with the same columns that of the file to be imported.
now the table is created, goto > Tools > Import > Import Wizard
now in the import wizard dialogue box, click Next.
click Add File, browse and select the file to be imported.
choose the correct dilimination.("," seperated for .csv file)
click Next, check if the mapping is done properly.
click Next, select the "A single existing table" radio button also select the table that to be mapped from the dropdown menu of Tables.
Click next and finish the process.
If you don't like plugins, VBA and external tools, I have an excel file that using formulas only allows you to create INSERT/UPDATES. You only have to put the data on the cells:
As an extra, there's another tab in the file to CREATE TABLES:
The file can be found on the following link:
EXCEL FILE
I've had good results with the Tools / Import CSV File feature in HeidiSQL, with CSV files directly exported from Excel 2019 with "Save As..."
It uses LOAD DATA INFILE internally but with a GUI interface and also analyzes the CSV file before passing it to LOAD DATA INFILE so it can, for example, create the table using the first row as column names and guessing the column data type (<New table> option as shown in the picture)