I have a problem,
I want to export from excel to MySql database table, but the destination table field is dynamic.
Let's say:
table A for storing field_information (e.g. field_name, field_type)
table B for storing field_answers (e.g. field_info_id, value)
example of excel spreadsheet file (which I convert into cvs format):
name,school,news;
"test","test_school","test_news";
I know I can export from excel to MySql (with static field) using following syntax:
LOAD DATA LOCAL INFILE '\\path' INTO TABLE test FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\r\n'
but, what if dynamic field?
How could I programmatically know which row of the spreadsheet should go to which database table?
Anyone can help?Thanks.
How will you programmatically know which row of the spreadsheet should go to which database table? How about some example rows of the spreadsheet?
Assuming I'm understanding your question correctly,
I'd sort the spreadsheet and then do two imports, to the two db tables.
Related
My goal is to create a MySQL table containing data from my CSV file.
is there any way to do this
i don't know whether it is possible or not thorugh csv. the thing is like i want to generate mysql table using csv headers as filed names i know how to load csv into table which is already created manually but i want to generate table through csv using batch script and below is my batch script to load csv into table through manually
load data local infile "C:\\EQA\\project\\input1.csv"
into table request_table
character set latin1
fields terminated by','
ENCLOSED BY '"'
lines terminated by'\r\n'
IGNORE 1 ROWS
here above code is for to load csv into table in which request_table is existing in db.
but i want to load csv into table dynamically?
is that possible?if so can some one help me out to accomplish this?
I have data in excel sheet and have to export the data to MySQL. I am using the MySQL for Excel tool in Data tab to export the data to a new table.
I have a table with id (primary key) and name (unique, with index), around 1000 rows. Exporting it takes 3 - 5 minutes.
Why is it so slow? Any suggestions or tips on how I can speed up the export process?
I have a sheet with 150,000 rows. So need some help..
Found one way to do it:
Create the table from excel using MySQL plugin for Excel. It can be done manually, make sure to have the same columns in the excel and the table (order and data type).
Before saving the sheet as a CSV, replace all comma (,) characters in the sheet with blank string; because the comma acts as separator for columns. Save the file as a .csv file.
Open mysql command line tool and run the below sql to import data to the table. Replace the csv file path and table name.
LOAD DATA INFILE "C:/data.csv" INTO TABLE mytable COLUMNS TERMINATED BY "," LINES TERMINATED BY "\r\n";
This imported 155,000 columns in a matter of seconds.
We want our users to be able to upload CSV files into our software and have the files be put into a temporary table so they can be analyzed and populated with additional data (upload process id, user name, etc).
Currently our application allows users to upload files into the system, but the files end up as TEXT values in a MySQL table (technically BLOB, but for our purposes, I will call it TEXT, as the only type of files I am concerned with are CSVs).
After a user uploads the CSV and it becomes a TEXT value, I want to take the TEXT value and interpret it as a CSV import, with multiple columns, to populate another table within MySQL without using a file output.
A simple insert-select into won't work as the TEXT is parsed as one big chunk (as it should be) instead of multiple columns.
insert into db2.my_table (select VALUE from db1.FILE_ATTACHMENT where id = 123456)
Most examples I have found export data from the DB as a file, then import it back in, i.e. something like:
SELECT VALUE INTO OUTFILE '/tmp/test.csv'
followed by something like:
LOAD DATA INFILE '/tmp/test.csv' INTO TABLE db2.my_table;
But I would like to do the entire process within MySQL if possible, without using the above "SELECT INTO OUTFILE/LOAD DATA INFILE" method.
Is there a way to have MySQL treat the TEXT value as multiple columns instead of one big block? Or am I stuck exporting to a file and then re-importing?
Thanks!
There is flaw in your data load approach.
Instead of keep each row in a single column, keep each value in respective column.
For example, suppose csv file contains n column. Create a table with n columns.
LOAD DATA INFILE '/tmp/test.csv'
INTO TABLE table_name
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS; // for ignoring header if present in csv file.
The Excel sheet is huge; contains around 40,000 rows. Both the table and the excel sheet have an equal number of columns.
I am using Ubuntu Linux, Libre Office and MySQL workbench. All I could find when I searched on the web was to import using MySQL Server Studio(I don't think it is available for Ubuntu), and queries containing "Microsoft.Jet-OPENROWSET" query. I am new to performing tasks other than the basic operations in excel/SQL.
And also, what format should the excel be in? Should it contain the column names explicitly?
Is there a simple way to perform this operation?
Thank You in Advance.
You can use the following:
LOAD DATA LOCAL INFILE '/path/to/your/file' INTO TABLE tablename
FIELDS TERMINATED BY ',' lines terminated by '\n';
i am planning to get some Data Entry done for the records of Restaurant info which i later want to upload to MySQL table.
Is there some kinda rules i need to follow because i read some place that we have to mention some kinda delimiter like , ; etc. not sure how to do that in Excel.
Secondly can i use Google Forms which stores question into Google sheets for doing the same as filling the form will be a lot faster and typing it out in excel :D
You can save the file (in Excel and also in Google spreadsheet) as a CSV. You can load the CSV into mysql using the LOAD DATA INFILE syntax. For example:
LOAD DATA INFILE 'data.txt' INTO TABLE db2.my_table
FIELDS TERMINATED BY ',' ENCLOSED BY '' ESCAPED BY '\\'
LINES TERMINATED BY '\n' STARTING BY ''
http://dev.mysql.com/doc/refman/5.1/en/load-data.html
Excel is the enemy of database programmers. If you simply ask someone to fill in an Excel spreadsheet expect 1) data in the wrong column for some rows, 2) text in fields that must be numbers (eg "bet 10 and 20", 3) Incomplete date fields (eg "10/2010"), text that is too long to fit your column definition and other problems.
However, if you have clean data you can Save As CSV in Excel and that will produce a file that any tool that can read CSV will be able to parse and load.