I have written the below query to insert data from xml file to a mysql table:
load XML local infile 'D:\\a.xml' into table asdb.abc rows identified by '<ApplicantName>';
It inserts the values in the table. I have a column named AckNo in the abc table. I need to insert the xml values where ackNo is equal to a particular value. e.g. i tried writing the below query but its not working:
load XML local infile 'D:\\a.xml' into table asdb.abc rows identified by '<ApplicantName>' where ackNo='1';
Would really appreciate someones help.
I don't think that you can use a WHERE clause with LOAD INFILE.
Check the documentation.
I think you have to parse your XML file and keep only values you want, then generates another XML file, before using LOAD DATA.
Or easier and quicker :
Load everything
DELETE FROM your_table WHERE your_id != "the_value_you_want"
Related
I'm trying to extract information from 50 million HTML files into a MySQL database. My question is at what point during the process should I store the information into the MySQL database. For example, I'm considering these options:
Open each file and extract the information I need. Perform an INSERT after each file gets parsed.
Open each file and extract the information I need. Store the information into a CSV file as an intermediary. After all the files have been parsed into the CSV, perform a bulk upload using LOAD DATA INFILE
I know that LOAD DATA INFILE is much faster than individual INSERT statements if I already have the information in a CSV. However, if I don't have the information already in a CSV, I don't know if it's faster to create the CSV first.
At the crux of the question: Is writing to a local CSV faster or about the same as a single INSERT statement?
I'm using PHP in case it matters. Thanks in advance!
They key is not to do one insert per entry, but batch the entries in memory then perform a batch insert.
See: https://dev.mysql.com/doc/refman/5.7/en/insert.html
INSERT statements that use VALUES syntax can insert multiple rows. To do this, include multiple lists of column values, each enclosed within parentheses and separated by commas. Example:
INSERT INTO tbl_name (a,b,c) VALUES(1,2,3),(4,5,6),(7,8,9);
ORMs like SQLAlchemy or Hibernate are smart enough (depending on configuration) to automatically batch your inserts.
Suppose we have table with a DECIMAL column with values, for example: 128.98, 283.98, 21.20.
I want to import some CSV Files to this table. However, in the columns of these files, I have values like 235,69, 23,23, with comma instead of points.
I know I can REPLACE that column, but is there some way of doing that before LOAD INFILE?
I do not believe you can simultaneously replace that column and load the data. Looks like you will have to do multiple steps to get the results you want.
Load the data first into a raw table using the LOAD INFILE command. This table can be identical to the main table. You can use the Create Table like command to create the table.
Process the data (i.e. change the comma to a . where applicable) in the raw table.
select the data from the raw table and insert into main table either with row by row processing or bulk insert.
This can all be done in a stored procedure (SP) or by a 3rd party script written in python, php, etc...
If you want to know more about SP's in Mysql, Here is a useful link.
I have a machine that output the data into a text file.
This text file contains the raw data.
Now I would like to insert the raw data from the text file by using the MYSQL LOAD DATA.
My condition is to append the new data to the existing table without duplicates.
I made raw_data column as primary key.
Everytime i run the LOAD DATA command it terminates when duplicates encountered and will not continue to load the rest of none duplicate raw data.
Example:
LOAD DATA INFILE '/mnt/A3/rawdata.txt' INTO TABLE test(raw_data);
Error Msg:for key 'PRIMARY' Duplicate entry 'aabbcc'
My question is how can I load and append the raw data to existing table without duplicates?
Look at the documentation, keywords REPLACE or IGNORE
I know this is a really basic question but I am struggling on my first import of data from an xml file. I have created the table "Regions" which has just two columns - ID and Name. The xml file contains the same column names.
In order to bulk import the data, I am using the following SQL command:
LOAD XML LOCAL INFILE 'C:\Users\Dell\Desktop\regions.xml'
INTO TABLE Regions (ID, Name)
but I am getting the error #1148 - The used command is not allowed with this MySQL version
Now having researched the internet, to allow this command requires a change in one of the command files but my service provider doesn't allow me access to it. Is there an alternative way to write the SQL code and do exactly the same thing as the code above which is basically just import the data from an xml file?
Many thanks
Since LOAD DATA INFILE isn't enabled for you, it appears you have only one more option and that's to create a set of INSERT statements for each row. If you converted your XML file to CSV using Excel, that's an easy step. Assuming you have a rows of data like this
A | B
-----|-------------------------
1 | Region 1
2 | Region 2
I would create a formula like this in column C
=CONCATENATE("INSERT INTO Regions(ID,Name) VALUES(",A1,",'",B1,"');")
This will result in INSERT INTO Regions(ID,Name) VALUES(1,'Region 1'); for your first row. File this down to the last row of your spreadsheet. Select all the insert statements and copy them into a Query text box inside PHPMyAdmin and you should be able to insert your values.
I've used this method many times when I needed to import data into a database.
I am very confused about LOAD DATA INFILE
after searching SO and google I have found no help on what I am attempting to do.
I want to create a new table, and load the contents of a csv file. The csv files first row is the column names I want.
Or if that cannot be done, how can I load the file without knowing how many columns exist?
LOAD DATA INFILE 'file.txt'
INTO TABLE t1
(... unknown number of columns ...);
You're going to need a tool other than mysql to load csvs without a predetermined schema.