I`m trying to upload a file in my opencart database, I want to bulk upload the product UPC numbers and I'm using this query:
CREATE TEMPORARY TABLE `my_temp_table`
(
product_id INT(11),
name VARCHAR(255),
upc VARCHAR(12)
);
LOAD DATA LOCAL INFILE '/home/mupckuco/dev/oc_product.csv'
INTO TABLE `my_temp_table`
FIELDS TERMINATED BY ','
(`product_id`, `name`, `upc`);
UPDATE `oc_product`
INNER JOIN `my_temp_table` on `my_temp_table`.`product_id` = `oc_product`.`product_id`
SET `oc_product`.`upc` = `my_temp_table`.`upc`;
DROP TEMPORARY TABLE `my_temp_table`;
Firstly I have exported the table with product id, product name and the column with the product UPC with the following query:
SELECT `oc_product`.`product_id` , `oc_product_description`.`name` , `oc_product`.`upc`
FROM `oc_product`
INNER JOIN `oc_product_description` ON `oc_product`.`product_id` = `oc_product_description`.`product_id`
BUT unfortunately, this line
UPDATE `oc_product` INNER JOIN `my_temp_table` on `my_temp_table`.`product_id` = `oc_product`.`product_id` SET `oc_product`.`upc` = `my_temp_table`.`upc`
is inserting NULLs in the table.
I have tried firstly to create a normal table (not temporary) so there i saw that everything in the UPC column is inserted as NULL.
I'm doing export as CSV for Microsoft Excel and I'm saving the file with the same extension. Where can be the problem?
This are some examples from the csv file and the my_temp_table results after updating:
Wow, I have never seem someone try to import a CSV directly in to the database and then run an update with a JOIN, perhaps your background is as a DBA rather than a PHP developer?
Anyway, I suggest you import the CSV in to an array in PHP:
$csv = array_map('str_getcsv', file('/home/mupckuco/dev/oc_product.csv'));
And then loop over it to generate the UPDATE statement.
Related
I have a csv with two columns, col1 is a barcode, col2 is stock quantity.
I have the 3 tables.
Table1:product_option_value
Fields: product_option_value_id, product_option_id, product_id, option_id, option_value_id, quantity, subtract, price, price_prefix, points, points_prefix, weight, weight_prefix
Table2: product_option_newvalue
Fields: product_id, product_option_value_id, sku, upc
I am trying to update the QUANTITY field of the table product_option_value using the sku and quantity in my CSV file, the part I’m having trouble with is I have to use product_option_value_id in the product_option_newvalue table to update QUANTITY field in product_option_value, how would I reference between the two?
Here is what I have. It does not work.
CREATE TABLE oc_product_import LIKE oc_product_option_value;
LOAD DATA INFILE '/var/lib/mysql-files/out.csv'
INTO TABLE oc_product_import
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
(sku, quantity);
UPDATE oc_product_option_value AS R
INNER JOIN oc_product_import AS P
ON R.product_option_value_id = P.product_option_value_id
SET R.quantity = P.sku;
DROP TABLE product_import;"
Edit: my issue seems to be that the barcode is only stored in product_option_newvalue, and can only be linked to quantity by referencing product_option_value_id, in both tables, To update the quantity in the product_option_value table.
EDIT2: This is similar code that is working for me. But it does not have to reference PRODUCT_OPTION_VALUE_ID issue I’m working with in the two table as the barcode is included in the product table and not an additional table by reference
DROP TABLE IF EXISTS oc_product_import;
CREATE TABLE oc_product_import LIKE oc_product
LOAD DATA INFILE '/var/lib/mysql-files/out.csv'
INTO TABLE oc_product_import
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY ';'
(sku, quantity);
UPDATE oc_product AS R
INNER JOIN oc_product_import AS P
ON R.sku = P.sku
SET R.quantity = P.quantity;
DROP TABLE oc_product_import;"
I would recommend using MySQL Workbench. It isn't the best, but it definitely makes situations like these easier to manage.
After you get it setup you can right click on a table and select table data import wizard and manually add a csv file that way.
If you get any errors with this let me know.
Also, if you are on mac you can simply brew cask install mysqlworkbench in terminal.
I have a table in a database, and I'd like to update a column which I have offline on a local file. The file itself has two columns
an ID which corresponds to an ID column in the table, and
the actual value.
I've been able to create new rows using
LOAD DATA INFILE 'file.txt' INTO TABLE table
FIELDS TERMINATED BY ','
But I'm not sure how I can specifically insert values in such a way that the ID column in the file is joined to the ID column in the table. Can someone help with the SQL syntax?
I suggest you load your data into a temporary table, then use an INSERT ... SELECT ... ON DUPLICATE KEY UPDATE; for example:
CREATE TEMPORARY TABLE temptable (
id INT UNSIGNED NOT NULL,
val INT,
PRIMARY KEY (id)
) ENGINE = MEMORY;
LOAD DATA LOCAL INFILE '/path/to/file.txt' INTO temptable FIELDS TERMINATED BY ',';
INSERT INTO my_table
SELECT id, val FROM temptable
ON DUPLICATE KEY UPDATE val = VALUES(val);
DROP TEMPORARY TABLE temptable;
Another way could be ...
Since you already know the table name as well have the ID and actual value ... what you can do is ... directly write the update statements in a file, like
update mytable set value_col = value where ID_col = ID;
Second Update Statement
Third Update statement
.......
Save the file as *.sql like, updatescript.sql and then execute that script directly like
mysql -h <hostname> -u root -p <your_db_name> < "E:/scripts/sql/updatescript.sql"
It depends of the no of rows ,
If it is in hundreds make a script of update column and run it , but if it is in large volume import that file in to a new table and update your table with a join , and then drop the table
Alright, I have multiple MySQL statements that lead into an issue I'm having updating a particular table. First let me show you my code, then I'll explain what I'm trying to do:
/*STEP 1 - create a temporary table to temporarily store the loaded csv*/
CREATE TEMPORARY TABLE IF NOT EXISTS `temptable1` LIKE `first60dayactivity`;
/*STEP 2. load the csv into the previously created temporary table*/
LOAD DATA LOCAL INFILE '/Users/me/Downloads/some.csv'
IGNORE INTO TABLE `{temptable}`
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"'
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES
SET CUSTID = 1030,
CREATED = NOW(),
isactive = 1;
/*STEP 3. update first60dayactivity table changing isactive for records that are not in the temptable*/
UPDATE `first60dayactivity` fa
INNER JOIN `temptable1` temp
ON temp.`mid` = fa.`mid`
AND temp.`primarypartnername` = fa.`primarypartnername`
AND temp.`market` = fa.`market`
AND temp.`agedays` = fa.`agedays`
AND temp.`opendate` = fa.`opendate`
AND temp.`CUSTID` = fa.`CUSTID`
SET fa.isactive = IF( temp.`mid` IS NULL, 0, 1 );
/*STEP 4. insert the temp table records into the real table*/
.....blah blah blah.....
Ok, first create a temporary table so that we have a table to hold the imported .csv data. Next, import the .csv data into the temporary table (all this works perfectly so far).
Here is where I run into an issue. I'm wanting to update the isactive column of each record of the first60dayactivity table to 0 if the record is NOT found in temptable1 (after my import). Ultimately, I'm gathering a .csv, the .csv has the new live data that should be considered "active" and I need to set the old data to inactive. So, the update does an INNER JOIN to match on several column to see if the record is found in the temptable1, if it isn't then set the activity to 0, if it is found in temptable1 then ensure the activity status is 1.
The problem here is that all records in first60dayactivity are retaining the 1 property to indicate it is active. Nothing is getting updated to 0 even though I have proof new records exist within temptable1... Can someone tell me what I'm doing wrong in my query?
Thanks in advance!
temp.mid can never be NULL because you use this column in your join condition and you use an INNER JOIN.
Your join (without the insert) should return the matching rows. Using a LEFT JOIN for the update should do what I suppose you want to do.
There are approximately 26K products (posts) and each product has meta values like this:
The post_id column is the product id in db and the _sku (meta_key) is the unique id for each product.
I've received a new CSV file that updates all of the values (meta_value) for _sale_price (meta_key) of each product. The CSV file looks like:
SKU, Sale Price
How do I import this CSV to update only the _sale_price row based on the post_id (product id) & _sku value?
Output Example:
I know how to do this in PHP by looping through the CSV and selecting & executing an update for each single product but this seems inefficient.
Preferably with phpMyAdmin and by using LOAD DATA INFILE.
You can use temporary table to hold the update data and then run single update statement.
CREATE TEMPORARY TABLE temp_update_table (meta_key, meta_value)
LOAD DATA INFILE 'your_csv_pathname'
INTO TABLE temp_update_table FIELDS TERMINATED BY ';' (meta_key, meta_value);
UPDATE "table"
INNER JOIN temp_update_table on temp_update_table.meta_key = "table".meta_key
SET "table".meta_value = temp_update_table.meta_value;
DROP TEMPORARY TABLE temp_update_table;
If product_id is the unique column of that table, you can do that using CSV:
Have a CSV file of those you want to import with their unique ID. CSV file must be in same order of the table column, put all your columns and no column name
Then in phpMyAdmin, go to the table of database, click import
Select CSV in the drop-down of Format field
Make sure "Update data when duplicate keys found on import (add ON DUPLICATE KEY UPDATE)" is checked.
You can import the new data into another table (table2). Then update your primary table (table1) using a update with a sub-select:
UPDATE table1 t1 set
sale_price = (select meta_value from table2 t2 where t2.post_id = t1.product_id)
WHERE
(select count(*) from table2 t2 where t1.product_id = t2.post_id) > 0
This is obviously a simplification and you will most likely need to constrain your query a little further.
Make sure to backup your full database before attempting. I recommend you work on a non-production database until the process works flawlessly.
It seems to me that rAndom69's answer does not work on postgresql 12 but the join with the WHERE work:
UPDATE tableA
SET fieldToPopulateInTableA = temp_update_table.fieldPopulated
FROM temp_update_table
WHERE tableA.correspondingField = temp_update_table.correspondingField
I have a table that looks like this:
products
--------
id, product, sku, department, quantity
There are approximately 800,000 entries in this table. I have received a new CSV file that updates all of the quantities of each product, for example:
productA, 12
productB, 71
productC, 92
So there are approximately 750,000 updates (50,000 products had no change in quantity).
My question is, how do I import this CSV to update only the quantity based off of the product (unique) but leave the sku, department, and other fields alone? I know how to do this in PHP by looping through the CSV and executing an update for each single line but this seems inefficient.
You can use LOAD DATA INFILE to bulk load the 800,000 rows of data into a temporary table, then use multiple-table UPDATE syntax to join your existing table to the temporary table and update the quantity values.
For example:
CREATE TEMPORARY TABLE your_temp_table LIKE your_table;
LOAD DATA INFILE '/tmp/your_file.csv'
INTO TABLE your_temp_table
FIELDS TERMINATED BY ','
(id, product, sku, department, quantity);
UPDATE your_table
INNER JOIN your_temp_table on your_temp_table.id = your_table.id
SET your_table.quantity = your_temp_table.quantity;
DROP TEMPORARY TABLE your_temp_table;
I would load the update data into a seperate table UPDATE_TABLE and perform an update within MySQL using:
UPDATE PRODUCTS P SET P.QUANTITY=(
SELECT UPDATE_QUANTITY
FROM UPDATE_TABLE
WHERE UPDATE_PRODUCT=P.PRODUCT
)
I dont have a MySQL at hand right now, so I can check the syntax perfectly, it might be you need to add a LIMIT 0,1 to the inner SELECT.
Answer from #ike-walker is indeed correct but also remember to double check how your CSV data if formatted. Many times for example CSV files can have string fields enclosed in double quotes ", and lines ending with \r\n if working on Windows.
By default is assumed that no enclosing character is used and line ending is \n.
More info and examples here https://mariadb.com/kb/en/importing-data-into-mariadb/
This can be fixed by using additional options for FIELDS and LINES
CREATE TEMPORARY TABLE your_temp_table LIKE your_table;
LOAD DATA INFILE '/tmp/your_file.csv'
INTO TABLE your_temp_table
FIELDS
TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"' -- new option
LINES TERMINATED BY '\r\n' -- new option
(id, product, sku, department, quantity);
UPDATE your_table
INNER JOIN your_temp_table on your_temp_table.id = your_table.id
SET your_table.quantity = your_temp_table.quantity;
DROP TEMPORARY TABLE your_temp_table;