I'm trying to import a CVS made from an excel file, but is not working, i'm receiving the following error -> Invalid column count in CSV input on line 1.
Follow the table structure
Follow the cvs file content
Do I need to have an id field in the csv file? Since its auto incremented, do i need to add it?
Follow is the import config of phpmyadmin
Another doubt, for each entry of mysql table, i would like to add an timestamp, is the configuration right? For every entry of the table, a timestamp will be added?
Regards
Related
When using a copy activity in Azure Data Factory to copy a typical CSV file with a header row into Parquet sink, the SINK fails with the following error due to the column names in the CSV having spaces in the header.
The column name is invalid. Column name cannot contain these
character:[,;{}()\n\t=]
The CSV is pipe delimited and displays just fine using the preview feature of the dataset with the first row marked as the header. I see no options to handle this use-case on the parquet side (sink) of the copy activity. I realize this can probably be addressed using a data flow to transform column names to remove spaces, but does that mean the native copy activity is incapable of handling this condition where a space in included in a header row?
EDIT: I should have added that dataset uses default mappings so that we can use the same dataset for any CSV to PARQUET copy. The answer provided will work for explicit mappings, but we don't see any resolution for folks who use default/dynamic mappings since we do not have access to the column names to remove spaces.
As we can note from the official Doc here
Error code: ParquetInvalidColumnName
Message: The column name is invalid. Column name cannot contain these character:[,;{}()\n\t=]
Cause: The column name contains invalid characters.
Resolution: Add or modify the column mapping to make the sink column name valid.
If you would like continue to use copy activity, there are few workarounds
1. make sure you have selected Column delimiter as Pipe(|)
2. If feasible, in mapping settings > import schema and rename the column name without spaces in destination column.
This is still an ongoing issue or request, follow here for more.
Hello and thank you for take your time reading my issue.
I have the next issue with Cassandra cqlsh:
When I use the COPY command to load a .csv into my table, the command prompt never finishes the executing and loads nothing into the table if I stop it with ctrl+c.
Im using .csv's files from: https://www.kaggle.com/daveianhickey/2000-16-traffic-flow-england-scotland-wales
specifically from ukTrafficAADF.csv.
I put the code below:
CREATE TABLE first_query ( AADFYear int, RoadCategory text,
LightGoodsVehicles text, PRIMARY KEY(AADFYear, RoadCategory);
Im trying it:
COPY first_query (AADFYear, RoadCategory, LightGoodsVehicles) FROM '..\ukTrafficAADF.csv' WITH DELIMITER=',' AND HEADER=TRUE;
This give me the error below repeatedly:
Failed to import 5000 rows: ParseError - Invalid row length 29 should be 3, given up without retries
And never finishes.
Add that the .csv file have more columns that I need, and trying the previous COPY command with the SKIPCOLS reserved word including the unused columns does the same.
Thanks in advance.
In cqlsh COPY command, All column in the csv must be present in the table schema.
In your case your csv ukTrafficAADF has 29 column but in the table first_query has only 3 column that's why it's throwing parse error.
So in some way you have to remove all the unused column from the csv then you can load it into cassandra table with cqlsh copy command
Ok so I am trying to import data from a CSV file into mySQL database table. The table is called serial_code, however when I try to upload the CSV file I have an error message. I have tried taking the column names out, also adding NULL to the last column EngineSerialCode and I have also viewed the CSV in a text editor and it shows the columns correctly with , comma.
Invalid column count in CSV input on line 1.
Version of phpMyAdmin and mySQL
Database Fields
CSV Fields I want to import
Ok so I found the issue and it was quite frustrating. I had a serial code in my data that had a "," comma instead of a "-". Also I noticed that my CSV file EngineSerialCode was placed in column E and not D as I only have 4 columns in my database. From the image column C is overlapping column D hence my mistake.
I've read through some other posts and nothing quite answers my question specifically.
I have an existing database in phpMyAdmin - a set of pin codes we use to collect contest entries.
The DB has about 10,000 pin codes in it.
I need to add 250 "New" codes to it. I have an excel file that is stripped down to a single column .csv, no header - just codes.
What I need to do is import this into the table named "pin2" and add these to the row called "pin"
The other rows are where entrants would add names and phone numbers, so are all "null"
I've uploaded a screen grab of the structure.
DB Structure http://www.redpointdesign.ca/sql.png
any help would be appreciated!
You need to use a LOAD DATA query similar to this:
LOAD DATA INFILE 'pincodes.csv'
INTO TABLE pin2 (pin)
If the pin codes in the csv file are enclosed in quotes you may also need to include an ENCLOSED BY clause.
LOAD DATA INFILE 'pincodes.csv'
INTO TABLE pin2
FIELDS ENCLOSED BY '"'
(pin)
If you wants to do using csv
Then you need to need to follow these steps
Manually define autoincremented value in first comlumn.
In other column you have to externally define it as a NULL,
otherwise you will get Invalid column count in CSV input on line 1.
because column with no value is not consider by phpmyadmin
Them click on import in phpmyadmin and you are done ..
I am trying to export an ODS file to CSV, but when I import into phpmyadmin - I get "Invalid field count in CSV input on line 1."
File (it has more than two lines but the scheme is the same):
"Administração da Guarda Nacional Republicana"
"Administração de Publicidade e Marketing"
table:
CREATE TABLE IF NOT EXISTS `profession` (
`id_profession` int(11) NOT NULL,
`profession` varchar(45) DEFAULT NULL,
`formation_area_id_formation_area` int(11) NOT NULL,
PRIMARY KEY (`id_profession`),
UNIQUE KEY `profession_UNIQUE` (`profession`),
KEY `fk_profession_formation_area1` (`formation_area_id_formation_area`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
I never do something similar, probably i need to specify the columns. the csv only have one column and the table have three. In this case the file input belongs to profession column
If you use phpMyAdmin, then you are allowed to specify column names. When logged into the desired database:
Select the table you want to import to.
Click the Import tab.
Under Format of imported file, select CSV.
In Column names, write out a comma separated list of the columns you want the data imported to.
You can also use mysqlimport if you prefer the shell.
For Example:
shell>mysqlimport --columns=column1,column2 dbname imptest.txt
If you want this to import into that table you have 2 choices:
1) Add a comma before and after the data in every row of your file, or
2) Delete the first and third columns in your table, import the data and then add the 2 columns you deleted back.
In Excel I saved the file as "Microsoft Office Excel Comma Separated Values File (.csv)"
In Phpmyadmin:
Select database you want to import table into.
Click import tab.
Select your file. Set FORMAT to CSV
Leave Format-Specific Options alone except for ticking the "The first line of the file contains the table column names" box if you need to
Click GO
You then need to rename the table ( which will be called somthing like "TABLE 5" if its the 5th table in the DB). So select the table and using the Operations tab -> "Rename table to:"
Make sure your uploading csv file only not excel file and then follow the below steps
1 Import
2 Browse for your csv file.
3 Select CSV using LOAD DATA (rather than just CSV)
4 Change “Fields terminated by” from “;” to “,”
5 Make sure “Use LOCAL keyword” is selected.
Click “Go”
Go to Import tab
Browse for the csv file.
Select "CSV" on Format on imported file tab
Checklist the "Ignore duplicate rows"
Change “Fields terminated by” from “;” to “,”
note : you better check the data that already imported. the first row of the data usually contains the header of each column on the table which you imported to csv file. delete the first row after you imported the csv file
you need to check the "The first line of the file contains the table column names (if this is unchecked, the first line will become part of the data)" and then click on "GO".