I've filled my table of instruments using LOAD INTO FILE. It fills the rows successfully but then doesn't enclose the final column (status) with a vertical line. I didn't think this was an issue until I ran a query to check the number of column entries = "commissioning".
SELECT COUNT(*)
FROM instrument
WHERE status = 'commissioning';
All 60 rows contain "commissioning" so it should return 60, but instead it returns 0?
I retried the query with a wildcard search and returned the right result here (You can also see the table is not enclosed)
Perhaps something is going on when I imported from csv file, because a LENGTH(status) query returns 14 when "commissioning" is only 13 characters. Has anyone encountered this before or know what character could be causing this?
Heres the import from the csv file code for further clarity - but it worked fine with my other tables
The problem you are having is produced because Windows uses '\r\n' instead of '\n'. As you are telling the import statement to finish lines with '\n' you have an extra '\r' character in every line. You need to change your import statement as:
LOAD DATA INFILE 'instruments.csv'
INTO TABLE instruments
FILEDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 1 ROWS;
Related
For days I am trying to export from my SQL-server table and to import into MySQL-table.
I can't solve the problem with HTML-Mails in one field of the table, which contains everything the HTML-code can have, such as \r\n linebreaks, quotation marks, maybe even | pipe-sign.
I tried exporting a concatenated string from SQL such as 'Insert Into MYSQL_table (field1, field2, ...)
I tried CSV-Files with terminal.command
LOAD DATA LOCAL INFILE 'G:/Test2.csv'
INTO TABLE insectum.tblolnachrichten
CHARACTER SET utf8mb4
FIELDS TERMINATED BY '|##|'
ENCLOSED BY ''
ESCAPED BY '\n'
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES;
tried workbench, php with CSV-files, I think everything.
But everywhere I fail due to another occurence of any sign in the HTML-Code in this field.
There are about 5000 lines to be transfered intyo Mysql-table, more than 100 MB in CSV-File.
I even tried field separator like |##| .
The content of this one field is wrapped with like this:
|##|myHTML-field|##|
Did not work as well.
Any idea what I could do to tell Mysql at import to keep content of a field for import and do not make a break anwhere?
Well, as no one had an answer for me, I did it the boring but obviously easierst way:
I linked SQL and MySQL into empty MS Access database and copied from one to another by taking about 300 rows every copy.
It worked and as I just have to do ONE time, it is OK.
I am using the following command from the MySQL command line to try and import a csv file into one of my tables:
LOAD DATA INFILE 'file path' INTO TABLE table
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(BASEID, BIGID, StartDate, EndDate)
Some of my rows have no value for EndDate which I want to be represented as NULL values in the database.
Unfortunately when I execute the above command I get the following error:
for column 'EndDate' at row 141lue:
If I remove the rows with blank cells the command works, so it is clearly the blank values for EndDate which are causing the problem.
I have also tried changing my csv file so that the blank cells say NULL or \N. I have also tried the following command instead but I still get the same error message:
LOAD DATA INFILE 'file path' INTO TABLE Table
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(BASEID, BIGID, StartDate, #EndDate)
SET EndDate = nullif(#EndDate, ' ')
How can I load csv files which have some blank values? The suggest solutions I have seen on other posts don't seem to work as outlined above.
Is the issue that the value for the end date is missing, or that the column itself is missing? These are not the same thing. For the former case, I think LOAD DATA should be able to handle this, assuming that the target column for the end date can tolerate missing/null values.
What I suspect here is that some of your input lines look like this:
1,1,'2020-10-03'
That is, there is no fourth column present at all. If this be the case, then the most prudent thing to do here might be to run a simple regex over your input CSV flat file to fix these missing fourth column edge cases. You may try:
Find: ^([^,]+,[^,]+,'[^,]+')$
Replace: $1,
This would turn the sample line above into:
1,1,'2020-10-03',
Now, the date value is still missing, but at least LOAD DATA should detect that the line has four columns, instead of just three.
I have imported a CSV file where a specific column has a decimal number.
In the original excel file (before saving it to a CSV), the first number of the column shows up as 218,790. When I choose the cell, the number shows up as 218790.243077911.
In the CSV file the number shows up as 218790 and when I choose the cell it is 218,790.
When I import the file on mySQL and show the table I created, the number shows up as 218.000000000.
Here is the code I used:
create table Apolo_Test(
Leads decimal (15,9)
);
LOAD DATA LOCAL INFILE 'C:/Users/SCRIPTS/file.csv'
INTO TABLE Apolo_Test
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 7 ROWS
;
I tried updating the format with this :
update Apolo_Test set Leads = format(Leads, 10, 'de_DE');
but it did not work. I have never had a case where files had a comma before. I guess it is the UK version of numerical fields.
How is it possible to make it work on mySQL without using any MACROS in excel?
UPD:
It works but I get some warnings although I double checked the csv file and the fields :
create table Apolo_Test(
Ad_Group varchar(50),
Impacts int,
Leads decimal (10,3)
);
LOAD DATA LOCAL INFILE 'C:/Users/me/Desktop/SCRIPTS/11/Adalyser.csv'
INTO TABLE Apolo_Test
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 7 ROWS
(Ad_Group, Impacts, #Leads)
SET Leads = replace(#Leads, ',', '');
;
alter table Apolo_Test ADD IPL decimal (10,6) after Leads;
update Apolo_Test set IPL=Impacts/Leads;
select * from Apolo_Test;
You have to use this syntax:
LOAD DATA LOCAL INFILE 'C:/path/to/mytable.txt' IGNORE
INTO TABLE mytable
FIELDS TERMINATED BY '\t' LINES TERMINATED BY '\r\n'
(int_col, #float_col)
SET float_col = replace(#float_col, ',', '.');
For more information read here
The thousands-separator should not matter when moving data around -- Excel internal values and CSV files and MySQL internal values do not include it. Only "formatted" output includes it. And you should not use formatted output for moving numbers around.
Be careful with locale, such as de_DE.
The German "218.790" is the same as English "218,790".
"218790.243077911" is likely to be what Excel had internally for the number.
"218,790" is likely to be the English representation on the screen; note the English thousands separator.
In the CSV file the number shows up as 218790 and when I choose the cell it is 218,790.
What do you mean? Perhaps that there no comma or dot in the file, itself? But what you mean by "choose the cell"?
I can't see how to get "218.000000000" without truncation going on somewhere.
Im trying to import a text file containing:
http://pastebin.com/qhzrq3M7
Into my database using the command
Load data local infile 'C:/Users/Gary/Desktop/XML/jobs.txt'
INTO Table jobs
fields terminated by '\t';
But I keep getting the error Row 1-13 doesn't contain data for all columns
Make sure the last field of each row ends with \t. Alternatively, use LINES TERMINATED BY
LOAD DATA LOCAL INFILE 'C:/Users/Gary/Desktop/XML/jobs.txt' INTO TABLE jobs COLUMNS TERMINATED BY '\t' OPTIONALLY ENCLOSED BY '"' LINES TERMINATED BY '\r';
\r is a carriage return character, similar to the newline character (i.e. \n)
I faced same issue. How i fixed the issue:
Try to open the CSV file using Notepad++ (text editor)
I've seen a blank line at the end of my file, I've deleted it.
-- Hurrah, it resolved my issue.
Below URL also can help you out to resolve the issue.
http://www.thoughtspot.com/blog/5-magic-fixes-most-common-csv-file-problems
If you're on Windows, make sure to use the LINES TERMINATED BY \r\n as explained by the mariadb docs
sounds like load data local infile expects to see a value for each column.
You can edit the file by hand (to delete those rows -- could be blank lines), or you can create a temp table, insert the rows into a single column, and write a mysql command to split the rows on tab and insert the values into the target table
Make sure there are no "\"s at the end of any field. In the csv viewed as text this would look like "\," which is obviously a no-no, since that comma will be ignored so you won't have enough columns.
(This primarily applies when you don't have field encasings like quotes around each field.)
my CSV content looks like this
1234,123;123;123
5675,123;567;234;565
No Space is provided at the end of each row in CSV i.e. 1234,123;123;123(No space here)
Imported this using the following command
mysql> load data local infile 'E:\sample.csv' into table Test.Table1 fields
terminated by ',' lines terminated by '\n' (Column1,Colunm2);
It gets executed successfully and i can find all the records in the DB. But the second column ends with a pilcrow.
When i try to edit, the value looks like
123;123;123
<extra line here>
If i remove the extra line, the pilcrow disappears.
Type of the column1, column2 is varchar.
Any clues for the issue?
I believe your problem is because of EOL termination. The file probably is using \r\n instead of only \n, hence the "<extra line here>".