How to bulk insert text files into MySQL table? - mysql

I just wanted to ask your expertise in bulk insert into mysql table, as of now this script is working fine:
LOAD DATA INFILE 'C:\\test\\cdr_june1.txt' INTO TABLE bod FIELDS TERMINATED BY ',';
But this past few days my text file have different file name, I was thinking to load all the file with .txt extension. How can I do this?

Related

how to create a mysql table by csv file header?

My goal is to create a MySQL table containing data from my CSV file.
is there any way to do this
i don't know whether it is possible or not thorugh csv. the thing is like i want to generate mysql table using csv headers as filed names i know how to load csv into table which is already created manually but i want to generate table through csv using batch script and below is my batch script to load csv into table through manually
load data local infile "C:\\EQA\\project\\input1.csv"
into table request_table
character set latin1
fields terminated by','
ENCLOSED BY '"'
lines terminated by'\r\n'
IGNORE 1 ROWS
here above code is for to load csv into table in which request_table is existing in db.
but i want to load csv into table dynamically?
is that possible?if so can some one help me out to accomplish this?

Scheduling Mysql procedure/macro to load CSV data

As I'm beginner to mysql ,I'm asking this question. Please help me.
I had .csv file and i'm loading this file data into mysql table. using the following command
"load data infile 'D:/xampp/htdocs/test/test.csv' into table offers fields terminated by ',' enclosed by '"' lines terminated by '\n' ignore 1 rows; "
It is inserting data into data into table successfully.
Now my question as follows
test.csv file(it has a huge volume of data)is going to update for every 24 hours. So that I want a stored procedure/macro( whatever it may be) to load the updated data into offers table it is going to call for every 24 hours, So that table data is in sync with .csv file.
Steps to remember
I want to truncate the offers table data before insert into table
and load the data using above command
Create a success log status in another log table(optional)
I heared that "load data" not going to work in stored procedure (I don't exactly).please give me any answer/suggesstions.

Excel file to mysql

I am creating a web application to collect some specific data from users. What I want is that the user uploads an excel file containing the data on the web page I created and that excel file stores its data on MySQL database.
Is it possible?How?
It's possible.
I would convert the Excel file to a csv file, or make the user upload a csv file instead. Excel already has this feature build in.
Then in MySQL you can turn the csv file into a tmp table with ease:
LOAD DATA LOW_PRIORITY LOCAL INFILE 'C:\\Users\\Desktop\\nameoffile.csv' REPLACE INTO TABLE `tmp_table` CHARACTER SET latin1 FIELDS TERMINATED BY ';' LINES TERMINATED BY '\r\n';
After that you transfer your data from the tmp table into the tables you'd like and finally you delete the temporary table.

Replace contents of MySQL table with contents of csv file on a remote server

I'm a newbie here trying to import some data into my wordpress database (MySQL) and I wonder if any of you SQL experts out there can help?
Database type: MySQL
Table name: wp_loans
I would like to completely replace the data in table wp_loans with the contents of file xyz.csv located on a remote server, for example https://www.mystagingserver.com/xyz.csv
All existing data in the table should be replaced with the contents of the CSV file.
The 1st row of the CSV file is the table headings so can be ignored.
I'd also like to automate the script to run daily at say 01:00 in the morning if possible.
UPDATE
Here is the SQL I'm using to try and replace the table contents:
LOAD DATA INFILE 'https://www.mystagingserver.com/xyz.csv'
REPLACE
INTO TABLE wp_loans
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
IGNORE 1 LINES
I would recommend a cron job to automate the process, and probably use BCP (bulk copy) to insert the data into a table... But seeing as you are using MySQL, instead of BCP, try load data in file - https://mariadb.com/kb/en/load-data-infile/

loading very large text file into my sql table

I have a very large text file of 6.2 GB. I have loaded this to a table in mysql using
load data local infile 'sample.txt'
into table table1 fields terminated by ','
(It took about 20 mins)But the dump was not successful and the data is not readable and not meaningful. I had a similar file but of smaller size in csv format and could successfully load that into sql table. I am unable to open the large text file, How should I convert the text to csv without opening it? or Can anyone share an idea to successfully load the text file to mysql table