There are approximately 26K products (posts) and each product has meta values like this:
The post_id column is the product id in db and the _sku (meta_key) is the unique id for each product.
I've received a new CSV file that updates all of the values (meta_value) for _sale_price (meta_key) of each product. The CSV file looks like:
SKU, Sale Price
How do I import this CSV to update only the _sale_price row based on the post_id (product id) & _sku value?
Output Example:
I know how to do this in PHP by looping through the CSV and selecting & executing an update for each single product but this seems inefficient.
Preferably with phpMyAdmin and by using LOAD DATA INFILE.
You can use temporary table to hold the update data and then run single update statement.
CREATE TEMPORARY TABLE temp_update_table (meta_key, meta_value)
LOAD DATA INFILE 'your_csv_pathname'
INTO TABLE temp_update_table FIELDS TERMINATED BY ';' (meta_key, meta_value);
UPDATE "table"
INNER JOIN temp_update_table on temp_update_table.meta_key = "table".meta_key
SET "table".meta_value = temp_update_table.meta_value;
DROP TEMPORARY TABLE temp_update_table;
If product_id is the unique column of that table, you can do that using CSV:
Have a CSV file of those you want to import with their unique ID. CSV file must be in same order of the table column, put all your columns and no column name
Then in phpMyAdmin, go to the table of database, click import
Select CSV in the drop-down of Format field
Make sure "Update data when duplicate keys found on import (add ON DUPLICATE KEY UPDATE)" is checked.
You can import the new data into another table (table2). Then update your primary table (table1) using a update with a sub-select:
UPDATE table1 t1 set
sale_price = (select meta_value from table2 t2 where t2.post_id = t1.product_id)
WHERE
(select count(*) from table2 t2 where t1.product_id = t2.post_id) > 0
This is obviously a simplification and you will most likely need to constrain your query a little further.
Make sure to backup your full database before attempting. I recommend you work on a non-production database until the process works flawlessly.
It seems to me that rAndom69's answer does not work on postgresql 12 but the join with the WHERE work:
UPDATE tableA
SET fieldToPopulateInTableA = temp_update_table.fieldPopulated
FROM temp_update_table
WHERE tableA.correspondingField = temp_update_table.correspondingField
Related
I`m trying to upload a file in my opencart database, I want to bulk upload the product UPC numbers and I'm using this query:
CREATE TEMPORARY TABLE `my_temp_table`
(
product_id INT(11),
name VARCHAR(255),
upc VARCHAR(12)
);
LOAD DATA LOCAL INFILE '/home/mupckuco/dev/oc_product.csv'
INTO TABLE `my_temp_table`
FIELDS TERMINATED BY ','
(`product_id`, `name`, `upc`);
UPDATE `oc_product`
INNER JOIN `my_temp_table` on `my_temp_table`.`product_id` = `oc_product`.`product_id`
SET `oc_product`.`upc` = `my_temp_table`.`upc`;
DROP TEMPORARY TABLE `my_temp_table`;
Firstly I have exported the table with product id, product name and the column with the product UPC with the following query:
SELECT `oc_product`.`product_id` , `oc_product_description`.`name` , `oc_product`.`upc`
FROM `oc_product`
INNER JOIN `oc_product_description` ON `oc_product`.`product_id` = `oc_product_description`.`product_id`
BUT unfortunately, this line
UPDATE `oc_product` INNER JOIN `my_temp_table` on `my_temp_table`.`product_id` = `oc_product`.`product_id` SET `oc_product`.`upc` = `my_temp_table`.`upc`
is inserting NULLs in the table.
I have tried firstly to create a normal table (not temporary) so there i saw that everything in the UPC column is inserted as NULL.
I'm doing export as CSV for Microsoft Excel and I'm saving the file with the same extension. Where can be the problem?
This are some examples from the csv file and the my_temp_table results after updating:
Wow, I have never seem someone try to import a CSV directly in to the database and then run an update with a JOIN, perhaps your background is as a DBA rather than a PHP developer?
Anyway, I suggest you import the CSV in to an array in PHP:
$csv = array_map('str_getcsv', file('/home/mupckuco/dev/oc_product.csv'));
And then loop over it to generate the UPDATE statement.
How can I load data from a file without replacing the existing columns but just adding missing values? For example if I already had a table and one of the rows would be
Id: 25, username: john, password: #hash
And then I add new columns bday, height, surname to my database and populate a csv file with them.
Is it possible to load those into a file without changing the id's of the users?
The simplest way is based on import the data in a new temporary table and then perform the update on the orginal table for the column you need join the rows between the 2 tables
eg .
table1 (id, key1, col1, col2_added)
table_temp(id, key1, col1, col2)
once you imported the files in table_temp you can
update table1
join table_temp = table1.key = table_temp.key
set col2_add = col2;
Alright, I have multiple MySQL statements that lead into an issue I'm having updating a particular table. First let me show you my code, then I'll explain what I'm trying to do:
/*STEP 1 - create a temporary table to temporarily store the loaded csv*/
CREATE TEMPORARY TABLE IF NOT EXISTS `temptable1` LIKE `first60dayactivity`;
/*STEP 2. load the csv into the previously created temporary table*/
LOAD DATA LOCAL INFILE '/Users/me/Downloads/some.csv'
IGNORE INTO TABLE `{temptable}`
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"'
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES
SET CUSTID = 1030,
CREATED = NOW(),
isactive = 1;
/*STEP 3. update first60dayactivity table changing isactive for records that are not in the temptable*/
UPDATE `first60dayactivity` fa
INNER JOIN `temptable1` temp
ON temp.`mid` = fa.`mid`
AND temp.`primarypartnername` = fa.`primarypartnername`
AND temp.`market` = fa.`market`
AND temp.`agedays` = fa.`agedays`
AND temp.`opendate` = fa.`opendate`
AND temp.`CUSTID` = fa.`CUSTID`
SET fa.isactive = IF( temp.`mid` IS NULL, 0, 1 );
/*STEP 4. insert the temp table records into the real table*/
.....blah blah blah.....
Ok, first create a temporary table so that we have a table to hold the imported .csv data. Next, import the .csv data into the temporary table (all this works perfectly so far).
Here is where I run into an issue. I'm wanting to update the isactive column of each record of the first60dayactivity table to 0 if the record is NOT found in temptable1 (after my import). Ultimately, I'm gathering a .csv, the .csv has the new live data that should be considered "active" and I need to set the old data to inactive. So, the update does an INNER JOIN to match on several column to see if the record is found in the temptable1, if it isn't then set the activity to 0, if it is found in temptable1 then ensure the activity status is 1.
The problem here is that all records in first60dayactivity are retaining the 1 property to indicate it is active. Nothing is getting updated to 0 even though I have proof new records exist within temptable1... Can someone tell me what I'm doing wrong in my query?
Thanks in advance!
temp.mid can never be NULL because you use this column in your join condition and you use an INNER JOIN.
Your join (without the insert) should return the matching rows. Using a LEFT JOIN for the update should do what I suppose you want to do.
I have a MySQL table like this,
id (primary key) | name | scores
and I am reading a large file to insert records into the MySQL table.
New records will be added into this file but the old records are not deleted, so when I read the file, a lot of records are already in the database.
Except to use SELECT COUNT to see if a record is already in the database, is there a best way to check it (to save processing time & database load)?
Or maybe I should just INSERT it directly? (The database will not allow records with duplicate id anyway.)
I usually use update + insert method.
first i will run the update statement. the update query will act like a select query + directly update the data.
update t1 set t1.Name = 'Name', t1.Scores = 99
where t1.Name = 'Name' and t1.Scores = 99
then check if there is a row affected by the above query. if not run the insert statement
if ##RowCount = 0
insert into t1 (Name, Scores) values ('Name',99)
Serch examples for
INSERT IGNORE INTO table
Simple example for this is
INSERT IGNORE INTO `transcripts`
SET `ensembl_transcript_id` = ‘ENSORGT00000000001′,
`transcript_chrom_start` = 12345
`transcript_chrom_end` = 12678;
I have two tables: t1 and t2
- t2 has only 1 column named stuff (60.000 entries).
- t1 has 15 columns, including stuff (empty). t1 has about 650.000 entries.
How can I import the data from t2.stuff in t1.stuff when I have nothing to match it against? (I just want to populate empty fields of t1.stuff with data from t2.stuff and don't care about matching ids or anything.)
The best case (i think) would be, that if I run this query about 11 times, all fields of t1.stuff would be populated, because no empty field in t1.stuff is left over.
Here is an example what the tables look like:
t1:
|__a___|_b_|_c_|stuff|...|
|___308|foo|bar|_____|baz|
|___312|foo|bar|_____|baz|
...
|655578|foo|bar|_____|baz|
t2:
|___stuff___|
|some_info_1|
|some_info_2|
...
|some_info_n|
Maybe there are multiple steps required...
UPDATE
Here is the SOLUTION I went with in case someone has a similar problem - All credits go to user nurdglaw for pointing me in the right direction. So here we go:
Add a new column to your table in question populated with autoincrementing numbers (I set alter table t1 auto_increment = 1 and temporary disabled autoincrementing on my primary key, to avoid an error with this code) ALTER TABLE t1 ADD COLUMN new_column INTEGER UNIQUE AUTO_INCREMENT;
Did the same thing for t2. If you don't already have a second table, you can do something like this:
CREATE TABLE t2 (id INTEGER PRIMARY KEY AUTO_INCREMENT,t2_data_column VARCHAR(255)); <-- adjust number to your needs
and import your data with:
LOAD DATA LOCAL INFILE 'path_on_your_server/data_file.csv'
INTO TABLE t2
LINES TERMINATED BY '\r\n' <-- adjust to your linebreak needs
(t2_data_column)
Now that you have something to match against, you can INNER JOIN t1 with t2 by doing the following: Add the data from t2 to t1
UPDATE t1 AS s
JOIN t2 AS t ON t.id=s.new_column
SET s.stuff=t.t2_data_column; <-- stuff was the column in t1 I wanted to import the data to.
Tidy up the mess
DROP TABLE t2;
ALTER TABLE t1 DROP COLUMN new_column;
Enable autoincrement on your primary key again and set it to the number you need for new rows, if you used one before.
That is it, you're done!
One further note: I decided to adjust my data offline and import the 650.000 entries needed with this method in one go, rather than doing it with only the 60.000 I put in the initial question. But you'll get the idea of doing it with any number of data and match it with whatever you need.
INSERT statements create new rows in your table.
You need an UPDATE on the already existing rows
An easy way to do that is using an extern scripting language
; here is a rebol example
; assumming you use the mysql library from softinnov
; and a_ is the name of the unique key to a row in t1
db: open mysql://user:pass#mysql
insert db {select * from t1}
t1rows: copy db
insert db {select * from t2}
t2rows: copy db
foreach row t1rows [
insert db [ {update t1 set t1.stuff = ? where t1.a_ = ?} t2rows/1/1 row/1]
either tail? next t2rows [
t2rows: head t2rows
] [
t2rows: next t2rows
]
]
sorry, I still have difficulties with the formatting and the variables in your example
Try this
INSERT INTO t1 (stuff)
SELECT DISTINCT stuff FROM t2
I hope it helps