I am using phpmyadmin(Version : 4.6.4) as a platform to import CSV encoded with UTF-8 to database. I am able to import the data, but with no idea why is the first two character of at first-column first-row went missing and this happens every time i import a CSV.
raw: A1610011111-001,N,N,N,N,N,N,N,N,N,N,N,N,N,--
This is what the data supposed to be -> (A1)
CSV data
This is the imported data (A1 went missing)->imported data
If the data is more than one row, the result will be same(only 1st two character went missing)
I am not sure what is the problem and what is the solution. Please give me a hand on this.
Well for anyone still searching for an answer, this is what worked for me after numerous tries.
In Excel (365), you can chose between
save as csv UTF-8 (sperated by comma's)
save as csv (seperated by delimiter)
As contradictory as it may seem, when I use the first option, I lose my first 2 characters, whereas choosing the 2nd. option I lose nothing.
So saving without the UTF-8 seems to do the trick.
Related
Someone sent me a xlsx excel file and I opened it with excel and saved as csv with utf8 encoding.
I use mysql workbench import wizard to import an excel-made utf8 csv file to a database table. But the imported result missed some data (less than it should have).
And I think it has something to do with the quotechar.
By default the quotechar is double quote but I have some data like this (mixing single quote and double quote):
8'10" foo bar
4" x 6" foo foo bar
I've try to omit the value but it can't (see the error from the pic).
So here I want to figure out:
What does quotechar mean here? How does it work? Why does it matter? Can't it just import everything from the csv file?
How can I import the data correctly while my data mixes single quote and double quote (later I need to retrieve them and use as search keywords, so it'd be better to keep the original form)?
my data looks like this in excel:
You are going to export your data from Excel as a CSV, I assume, so how this looks in Excel is irrelevant.
When you export the data from excel as a CSV format it's going to encapsulate your data in double quotes. Any double quotes in the data itself is going to be escaped by a second double quote automatically by excel.
As an example, if your data is:
8"
When you export it will be:
"8"""
You have to tell Mysql that you are enclosing strings in character ". That is the quotechar it's talking about. It's the second field on that form you are filling out.
I'm not sure how picky MySQl is going to be here since I haven't imported CSV to Mysql in forever and ever and ever. The trick with the Excel CSV output is that if you have data like:
8"
8'
It will output it as CSV:
"8"""
8'
The second record/field doesn't gain the double quote encapsulation since it doesn't contain a character that requires encapsulation (A double quote, a carriage return, or a line feed).
Mysql might choke on that second record (Hopefully it's import process is robust enough to handle encapsulated and non-encaps'd fields though)
I'm trying to import some data from a CSV into my database with phpmyadmin.
Here's a row from the CSV:
20101,1,grams,Good,AU,0.9999,Caesar,2017-06-14,12:33:44,RP
The first number I have set as a unique bigint(16). Somehow though, this gets imported as "101" instead of "20101", which causes a duplicate error because i already have a "101".
Why wouldn't the number fully read as "20101"?
I think I figured this out. When saving the CSV from Excel, I was using the "CSV (UTF-8)" option. Don't know why that would make a difference, but when I switched to saving it as a plain CSV comma delineated, the file imported with no problems.
Importing csv in Rapidminer is not loading data properly in the attributes/ columns and returns errors.
I have set the parameter values correctly in the 'Data Import Wizard'.
Column Separation is set to comma and when I check the "Use Quotes" parameter I see that there are too many "?" appear in the columns even though there is data in the actual csv file.
And when I do not check the “Use Quotes” option then I notice that the content of the columns are distributed across different columns, i.e., data does not appear in the correct column. It also gives error for the date column.
How to resolve this? Any suggestions please? I saw a lot of Rapidminer videos and read about it but did not help.
I am trying to import twitter conversations data which I exported from a 3rd party SaaS tool which extracts Twitter data for us.
Could someone help me soon please? Thanks, Geeta
It's virtually impossible to debug this without seeing the data.
The use quotes option requires that each field is surrounded by double quotes. Do not use this if your data does not contain these because the input process will import everything into the first field.
When you use comma as the delimiter, the observed behaviour is likely to be because there are additional commas contained in the data. This seems likely if the data is based on Twitter. This confuses the import because it is just looking for commas.
Generally, if you can get the input data changed, try to get it produced using a delimiter that cannot appear in the raw text data. Good examples would be | or tab. If you can get quotes around the fields, this will help because it allows delimiter characters to appear in the field.
Dates formats can be handled using the data format parameter but my advice is to import the date field as a polynominal and then convert it later to date using the Nominal to Date operator. This gives more control especially when the input data is not clean.
I'm trying to export a CSV from my client's FluidSurvey's account and import it into a database I've created. I've never actually worked with a CSV before, so excuse my ignorance.
I've looked into this error and none of the solutions seem to be working for me, I'm at a loss, I've been trying to import this file for hours now.
Settings are as follows:
There is already a table with columns for this data to be inserted into.
What am I missing here?
You've showed exported csv file in Excel or Calc. It is impossible to understand how you columns are enclosed. Probably there is some sign other than ' or " Please show exported csv in notepad. This will clear the structure of csv.
I found that Fluidsurveys CSV files had the header two bytes incorrect.
They are only 7F7E. Changing them to the expected Unicode FFFE works as expected - they can be read into Excel with no garbage characters at the start.
I have 12 excel files, each one with lots of data organized in 2 fields (columns): id and text.
Each excel file uses a diferent language for the text field: spanish, italian, french, english, german, arabic, japanese, rusian, korean, chinese, japanese and portuguese.
The id field is a combination of letters and numbers.
I need to import every excel into a different MySQL table, so one table per language.
I'm trying to do it the following way:
- Save the excel as a CSV file
- Import that CSV in phpMyAdmin
The problem is that I'm getting all sorts of problems and I can't get to import them properly, probably because of codification issues.
For example, with the Arabic one, I set everything to UTF-8 (the database table field and the CSV file), but when I do the import, I get weird characters instead of the normal arabic ones (if I manually copy them, they show fine).
Other problems I'm getting are that some texts have commas, and since the CSV file uses also commas to separate fields, in texts that are imported are truncated whenever there's a comma.
Other problems are that, when saving as CSV, the characters get messed up (like the chinese one), and I can't find an option to tell excel what encoding I want to use in the CSV file.
Is there any "protocol" or "rule" that I can follow to make sure that I do it the right way? Something that works for each different language? I'm trying to pay attention to the character encoding, but even with that I still get weird stuff.
Maybe I should try a different method instead of CSV files?
Any advice would be much appreciated.
OK, how do I solved all my issues? FORGET ABOUT EXCEL!!!
I uploaded the excels to Googledocs spreadsheets, downloaded them as CSV, and all the characters were perfect.
Then I just imported into their corresponding fields of the tables, using a "utf_general_ci" collation, and now everything is uploaded perfectly in the database.
One standard thing to do in a CSV is to enclose fields containing commas with double quotes. So
ABC, johnny cant't come out, can he?, newfield
becomes
ABC, "johnny cant't come out, can he?", newfield
I believe Excel does this if you choose to save as file type CSV. A problem you'll have is that CSV is ANSI-only. I think you need to use the "Unicode Text" save-as option and live with the tab delimiters or convert them to commas. The Unicode text option also quotes comma-containing values. (checked using Excel 2007)
EDIT: Add specific directions
In Excel 2007 (the specifics may be different for other versions of Excel)
Choose "Save As"
In the "Save as type:" field, select "Unicode Text"
You'll get a Unicode file. UCS-2 Little Endian, specifically.