I have a table with 18 columns and I want to update all rows of the column page_count using LOAD DATA INFILE. The file with the data is extremely simple, just \n separated values. I'm not trying to pull off anything impressive here; I just want to update the values in that one single column - about 3000 records. The code I'm using is
LOAD DATA INFILE '/tmp/sbfh_counter_restore' REPLACE INTO TABLE obits FIELDS TERMINATED BY ',' LINES TERMINATED BY '\r\n' (page_count);
And all this does is add one new row with the entire contents of the file dumped into the column page_count. What am I doing wrong? Shoud I use phpmyadmin? I'd be happy to use that as it better fits my skill set ;)
I created the file using SELECT page_count FROM obits INTO outfile '/tmp/sbfh_counter_restore'
based what I can understand from MySQL document, it does not support loading data into ONE COLUMN, but it will require ALL COLUMNS to be present in the file in correct order.
At least, you should use SELECT * FROM obits INTO outfile, then load it back as it will ensure the column order is consistent.
As all your file content was loaded into a new row, I think you should check the primary key (or unique key) of your table. The rows will be matched by the key and update or insert based on the matching result. It is likely that page_count is not your primary or unique key.
Hope that helps.
Related
I have a CSV file that I am loading into my database. I want the previous data in the table to be overwritten and not appended every time I load my CSV file. Is it possible to do this within a single query?
Is the only solution to TRUNCATE the table and then utilize the LOAD DATA INFILE queries?
Assuming you have a primary key, you can use REPLACE. As the documentation states:
The REPLACE and IGNORE modifiers control handling of input rows that
duplicate existing rows on unique key values:
If you specify REPLACE, input rows replace existing rows. In other words, rows that have the same value for a primary key or unique index
as an existing row. See Section 13.2.9, “REPLACE Statement”.
However, if you want to replace the existing table, then truncate the table first and then load.
I am googling around and reading posts on this site but not clear what I should I do go with insert unique, records.
I basically have a giant file that that has a single column of data in it that needs to be imported into a db table where several thousand of the records from my text file already exist.
There are no id's in the text file I need to import and what I am reading insert ignore looks to be solving for duplicate ID's. I want new ids created for any of the new records added but obviously I can't have duplicates.
This would ideally be done with load data infile...but really not sure:
Insert IGNORE?
ON Duplicate Key?
The easiest way to achieve what you want is to read in the entire file using LOAD DATA, and then insert all non duplicates into the table which already exists.
LOAD DATA LOCAL INFILE 'large_file.txt' INTO TABLE new_table
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
(name);
This assumes that the first line of your giant file does not contain a column header. If it does, then you can add IGNORE 1 LINES to the statement above.
You can now INSERT the names in new_table into your already-existing table using INSERT INTO IGNORE:
INSERT IGNORE INTO your_table_archive (name)
SELECT name
FROM new_table
Any duplicate records which MySQL encounters will not be inserted, and instead the original ones will be retained. And you can drop new_table once you have what you need from it.
I have an existing table on MySQL with 7 fields - 4 are integer(one acting as primary key and auto incremete) and 3 text. This table already has data.
I am collecting new data in Excel and wish to bulk-add the data to the Mysql table. Some of the cells in Excel will be blank and need to show as null in MySQL and the primary key won't be part of excel, I let MYSQL auto add it. Previously I was manually adding each record through PHPmyadmin but it is far too time consuming (I have 1000's of records to add).
How do I go about this in terms of setting the Excel sheet in the right way and making sure I add to the exsisting data instead of replacing it? I have heard CSV files are the way? I use PHPmyadmin if that helps.
Thanks
If you want to append data to a MySQL table, I would use .csv files to do it.
In MySQL:
load data local infile 'yourFile'
into table yourTable (field1, field2, field3)
fields terminated by ',' optionally enclosed by '"'
lines terminated by '\n' -- On windows you may need to use `\r\n'
ignore 1 lines; -- Ignore the header line
Check the reference manual: http://dev.mysql.com/doc/refman/5.0/en/load-data.html
How to add a custom value using LOAD DATA LOCAL INFILE?
The column time_added is the 7th column and the file has only 2 values for the first and the second column. For the 7th column, time_added I want to use the unix timestamp when loading from file.
This code isn't working:
$result = mysql_query("LOAD DATA LOCAL INFILE '{$myFile}' INTO TABLE {$table} FIELDS TERMINATED BY ':' LINES TERMINATED BY '\n' SET `time_added`=unix_timestamp()");
Why wouldn't this work?
LOAD DATA INFILE 'file.txt'
INTO TABLE t1
(column1, column2)
SET column7 = unix_timestamp();
The answer given by #iouri indicates the key element to address your question, namely the explicit listing of the columns populated by the .csv file, (column1, column2). This line informs the LOAD function to only consider these columns when loading data from the .csv file and avoids an error similar to Row 1 doesn't contain data for all columns.
You will still need to list all columns, including custom columns, in the table definition. Also, the column names listed in the parentheses should match the names of the columns defined in the table definition. For example, if the table definition specifies two columns named user and id then you would need to have the line (user, id) above the SET column7 = unix_timestamp() line.
You may also want to double check that you want LOAD DATA LOCAL INFILE instead of LOAD DATA INFILE (no LOCAL). As specified in the documentation for load-data, the LOCAL keyword affects the expected location of the file and both the server and client must be configured properly to allow for using the LOCAL option.
Here's my query for loading mysql table using csv file.
LOAD DATA LOCAL INFILE table.csv REPLACE INTO TABLE table1 FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\'' LINES TERMINATED BY 'XXX' IGNORE 1 LINES
SET date_modified = CURRENT_TIMESTAMP;
Suppose my CSV contains 500 records with 15 columns. I changed three rows and terminated them with 'XXX'. I now want to update the mysql table with this file. My primary key is auto-incremented value. When I run this query, all the 500 rows are getting updated with old data and the rows I changed are getting added as new ones. I dont want the new ones. I want my table to be replaced with csv as-is. I tried changing my primary key to non-AI, it still didnt work. Any pointers please?? Thanks.
I am making some assumptions here.
1) You dont have the autonumber value in your file.
Since your primary key is not in your file MySQL will not be able to match rows. A autonumber primary key is a artificial key thus it is not part of the data. MySQL adds this artificial primary key when the row is inserted.
Lets assume your file contained some unique identifier lets call it Identification_Number. This number is both in the file and your table uses it as a primary key in this case MySQL will be able to identify the rows from the file and match them to the rows in the table.
While a lot of people will only use autonumbers in a database I always check if there is not a natural key in the data. If I identify one I do some performance testing with this natural key in a table. Then based on the performance metrics of both I then decide on a key.
Hopefully I did not get your question wrong but I suspect this might be the case.