UPDATE statement using GROUP_CONCAT() - mysql

I'm attempting to create a single query that UPDATES another table but the SUBQUERY/DERIVED-QUERY that I would use requires me to have them GROUP BY and GROUP_CONCAT().
I was able to get my desired output but to do so I had to create a temporary table to store the "grouped/ concated" data and then push that "re-organized" data to the destination table. TO do so, I have to literally run 2 separate queries one that populates the temp table with the "organized" data in it's fields and then run another UPDATE that pushes the "organized" data from the temp table to the final destination table.
I've created a REPREX that exemplifies what I'm trying to achieve below:
/*
Create a simplified sample table:
*/
CREATE TABLE `test_tbl` (
`equipment_num` varchar(20),
`item_id` varchar(40),
`quantity` decimal(10,2),
`po_num` varchar(20)
)
--
-- Dumping data for table `test_tbl`
--
INSERT INTO `test_tbl` (`equipment_num`, `item_id`, `quantity`, `po_num`) VALUES
(TRHU8399302, '70-8491', '5.00', 'PO10813-Air'),
(TRHU8399302, '40-21-72194', '22.00', '53841'),
(TRHU8399302, '741-PremBundle-CK', '130.00', 'NECTAR-PMBUNDLE-2022'),
(TRHU8399302, '741-GWPBundle-KG', '650.00', 'NECTAR2021MH185-Fort'),
(TRHU6669420, '01-DGCOOL250FJ', '76000.00', '4467'),
(TRHU6669420, '20-2649', '450.00', 'PO9994'),
(TRHU6669420, 'PFL-PC-GRY-KG', '80.00', '1020'),
(TRHU6669420, '844067025947', '120.00', 'Cmax 2 15 22'),
(TRHU5614145, 'Classic Lounge Chair Walnut leg- A XH301', '372.00', 'P295'),
(TRHU5614145, '40-21-72194', '22.00', '53837'),
(TRHU5614145, 'MAR-PLW-55K-BX', '2313.00', 'SF220914R-CA'),
(TRHU5614145, 'OPCP-BH1-L', '150.00', 'PO-00000429B'),
(TRHU5367889, 'NL1000WHT', '3240.00', 'PO1002050'),
(TRHU4692842, '1300828', '500.00', '4500342008'),
(TRHU4560701, 'TSFP-HB2-T', '630.00', 'PO-00000485A'),
(TRHU4319443, 'BGS21ASFD', '20.00', 'PO10456-1'),
(TRHU4317564, 'CSMN-AM1-X', '1000.00', 'PO-00000446'),
(TRHU4249449, '4312970', '3240.00', '4550735164'),
(TRHU4238260, '741-GWPBundle-TW', '170.00', 'NECTAR2022MH241'),
(TRHU3335270, '1301291', '60000.00', '4500330599'),
(TRHU3070607, '36082233', '150.00', '11199460'),
(TLLU8519560, 'BGM03AWFX', '360.00', 'PO10181A'),
(TLLU8519560, '10-1067', '9120.00', 'PO10396'),
(TLLU8519560, 'LUNA-KP-SS', '8704.00', '4782'),
(TLLU5819760, 'GS-1319', '10000.00', '62719'),
(TLLU5819760, '2020124775', '340.00', '3483'),
(TLLU5389611, '1049243', '63200.00', '4500343723'),
(TLLU4920852, '40-21-72194', '22.00', '53839'),
(TRHU3335270, '4312904', '1050.00', '4550694829'),
(TLLU4540955, '062-06-4580', '86.00', '1002529'),
(TRHU3335270, 'BGM03AWFK', '1000.00', 'PO9912'),
(TLLU4196942, 'Classic Dining Chair,Walnut Legs, SF XH1', '3290.00', 'P279'),
(TLLU4196942, 'BGM61AWFF', '852.00', 'PO10365');
---
--- The data above is a subsample of what I have on the db, what I'm trying to do is to update another table based off this info but with some GROUP_CONCAT()
--- With the data from above, I need to GROUP_CONCAT(item_id),GROUP_CONCAT(quantity), GROUP_CONCAT(po_num) -- grouping by equipment_num field.
---
--- What I'm attempting to do is to do an UPDATE to another table with the GROUPED by equipment_num with and the Group_concats for the fields described above.
---
--- The only way I was able to do what I desired was with a intermediary TEMPORARY table.
---
--- Create the temp table:
--- Since what I need is a "list" of the quantities, I had to do a GROUP_CONCAT(CONCAT(quantity,''))
DROP TABLE __tmp__; CREATE TABLE __tmp__
SELECT equipment_num, GROUP_CONCAT( item_id ), GROUP_CONCAT(CONCAT( quantity , '' ) ), GROUP_CONCAT( po_num )
FROM `test_tbl`
GROUP BY equipment_num
--- Then FINALLY pull the information in the format I desire to the destination table:
UPDATE `dest_tbl` AS ms INNER JOIN `__tmp__` AS isn ON ( ms.equipment_num = isn.equipment_num ) SET ms.item_id = isn.item_id,
ms.piece_count = isn.quantity,
ms.pieces_detail = isn.po_num
I'm trying to create a single queries that generates a derived query that does the group_concat part and then pushes that derived query result to the final destination table.
Any suggestions would be greatly appreciated.
Thank you for your time.
TB.
EDIT: Thank you for the replies I've got, but I'm trying to AVOID using the temp table.
I'm trying to AVOID creating a temp table.... I'm wondering how to do it in one go...
I was thinking something along the lines of:
UPDATE dest
INNER JOIN(
SELECT src.equipment_num, GROUP_CONCAT(src.item_id) as item_id,
GROUP_CONCAT(CONCAT(src.quantity)) as quantity,
GROUP_CONCAT(src.po_num) as po_num
FROM `item_shipped_ns` as src
INNER JOIN milestone_test_20221019 as dest ON(src.equipment_num=dest.equipment_num)
WHERE src.importer_id='123456'
GROUP BY src.equipment_num
) as tmp ON(src.equipment_num=tmp.equipment_num)
SET
dest.item_num=tmp.item_id,
dest.piece_count=tmp.quantity,
dest.pieces_detail=tmp.po_num;
Unfortunately, the above doesn't work, I get the following error msg.
#1146 - Table 'fgcloud.dest' doesn't exist
Edit 2: I had a missing brackets in the above which caused a different error, I've fixed it but having issues with the table aliases. The table in question that should be updated is the "milestone_test_20221019" - it is declared as "dest", yet I it says it cannot find it, suggestions? The source table which I need to get the info and aggregate before updating "milestone_test_20221019" is the "item_shipped_ns" and I believe that "tmp" table is the derived/sub-query table alias...


You need to give an alias to the GROUP_CONCAT() so you'll get a column named item_id. It won't use the argument to GROUP_CONCAT() as the name of the resulting column automatically.
CREATE TABLE __tmp__
SELECT equipment_num,
GROUP_CONCAT( item_id ) AS item_id,
GROUP_CONCAT( quantity ) AS quantity,
GROUP_CONCAT( po_num ) AS po_num
FROM `test_tbl`
GROUP BY equipment_num
To do this in a single query without creating the __tmp__ table, just put the query used to create __tmp__ in a subquery in the UPDATE.
UPDATE milestone_test_20221019 AS dest
JOIN (
SELECT equipment_num,
GROUP_CONCAT( item_id ) AS item_id,
GROUP_CONCAT( quantity ) AS quantity,
GROUP_CONCAT( po_num ) AS po_num
FROM item_shipped_ns
GROUP BY equipment_num
) AS src ON dest.equipment_num = src.equipment_num
SET dest.item_id = src.item_id,
dest.quantity = src.quantity,
dest.po_num = src.po_num


Thanks for the assistance, after a few more test and tweaks I was able to achieve what I desired.
Below is an example of how to use an UPDATE with GROUP_CONCAT() as well an implicit-explicit casting for the quantity field.
UPDATE milestone_test_20221019 as dest
INNER JOIN(
SELECT src.equipment_num, GROUP_CONCAT(src.item_id) as item_id,
GROUP_CONCAT(CONCAT(src.quantity,'')) as quantity,
GROUP_CONCAT(src.po_num) as po_num
FROM item_shipped_ns as src
INNER JOIN milestone_test_20221019 as t1 ON(src.equipment_num=t1.equipment_num)
WHERE src.importer_id='4081836'
GROUP BY src.equipment_num
) AS tmp ON(tmp.equipment_num=dest.equipment_num)
SET
dest.item_num=tmp.item_id,
dest.piece_count=tmp.quantity,
dest.pieces_detail=tmp.po_num;
Thank you for the people that commented and assisted me with their inputs.
Best regards,
TB.

Related

What sql query to use for only deleting duplicate results for wp_comments table?

I need to finish the select query below. The query shows me the count of comments with the same comment_id.I just ultimately want to delete the duplicates and leave the non duplicates alone.This is a wordpress database
screenshot of my current query results
SELECT `comment_ID`, `comment_ID`, count(*) FROM `wp_comments` GROUP BY `comment_ID` HAVING COUNT(*) > 1 ORDER BY `count(*)` ASC
example of 2 entries I need to delete one

First back up your bad table in case you goof something up.
CREATE TABLE wp_commments_bad_backup SELECT * FROM wp_comments;
Do you actually have duplicate records here (duplicate in all columns) ? If so, try this
CREATE TABLE wp_comments_deduped SELECT DISTINCT * FROM wp_comments;
RENAME TABLE wp_comments TO wp_comments_not_deduped;
RENAME TABLE wp_comments_deduped TO wp_comments;
If they don't have exactly the same contents and you don't care which contents you keep from each pair of duplicate rows, try something like this:
CREATE TABLE wp_comments_deduped
SELECT comment_ID,
MAX(comment_post_ID) comment_post_ID,
MAX(comment_author) comment_author,
MAX(comment_author_email) comment_author_email,
MAX(comment_author_url) comment_author_url,
MAX(comment_author_IP) comment_author_IP,
MAX(comment_date) comment_date,
MAX(comment_date_gmt) comment_date_gmt,
MAX(comment_content) comment_content,
MAX(comment_karma) comment_karma,
MAX(comment_approved) comment_approved,
MAX(comment_agent) comment_agent,
MAX(comment_type) comment_type,
MAX(comment_parent) comment_parent,
MAX(user_id) user_id
FROM wp_comments
GROUP BY comment_ID;
RENAME TABLE wp_comments TO wp_comments_not_deduped;
RENAME TABLE wp_comments_deduped TO wp_comments;
Then you'll need to doublecheck whether your deduplicating worked:
SELECT comment_ID, COUNT(*) num FROM wp_comments GROUP BY comment_ID;
Then, once you're happy with it, put back WordPress's indexes.
Pro tip: Use a plugin like Duplicator when you migrate from one WordPress setup to another; its authors have sorted out all this data migration for you.

I would recommand add a unique key to the table make it auto incremental call it tempId , so you would be able to to distinguish between one duplicate set, use below query to remove duplicate copies and at the end remove that '`tempid' column:
DELETE FROM `wp_comments`
WHERE EXISTS (
SELECT `comment_ID` , MIN(`tempid`) AS `tempid`
FROM `wp_comments` as `dups`
GROUP BY `comment_ID`
HAVING
COUNT(*) > 1
AND `dups`.`comment_ID` = `wp_comments`.`comment_ID`
AND `dups`.`tempid` = `wp_comments`.`tempid`
)

I'm not clear on why there appear to be two different fields both named 'column_ID' from the same table, but I believe this will delete only the first of the two identical records. Before running a DELETE statement, however, be sure to make a backup of the original table.
DELETE
TOP 1 *
FROM
'wp_comments'
WHERE
comment_ID IN
(
SELECT
comment_ID,
r,
(comment_ID + '_' + r) AS unique
FROM
(
SELECT
`comment_ID`,
`comment_ID`,
RANK() OVER (PARTITION BY 'comment_id' ORDER BY 'comment_id') AS r
FROM
'wp_comments'
)
WHERE
r>1
)

MySQL Query that traverses pointer fields

Due to some legacy code, I have 2 MySQL tables with the below structure (simplified):
Invoice (ID, InvoiceNo, First_Item)
InvoiceItem (ID, Details, Next_Item)
Obviously, there are many InvoiceItems for each Invoice.
The legacy app expects you to load the Invoice row first, then load the first item from the InvoiceItem table using the Invoice's First_Item value. To get each successive InvoiceItem row, you would then follow its Next_Item value until you hit a null value.
Is there a way to write MySQL SQL that would bring back all InvoiceItem(s) for a given Invoice? i.e follow the Invoice's First_Item and then traverse all the Invoice_Items's Next_Item pointers.
Thanks
Bill.

You want do a recursive query , but mysql < 8 does not support it .
This is a solution that work on you small dataset ( from sqlfiddle )
select id,
details ,
next_item
from (select * from invoiceitem
order by id ) inv_itm,
(select #iis := 0 ) init
where find_in_set(id, #iis)
and not find_in_set(9999999999, #iis)
and length(#iis := concat(#iis, ',', ifnull(next_item,9999999999))) ;
This solution will work only if for each invoices id of items are in ascending order .
This solution is inspired by How to create a MySQL hierarchical recursive query
You need to plan a upgrade to 5.7 or 8.0 , because bellow you will have no security update soon .
see https://en.wikipedia.org/wiki/MySQL#Release_history

Table is specified twice, both as a target for 'UPDATE' and as a separate source for data in mysql

I have below query in mysql where I want to check if branch id and year of finance type from branch_master are equal with branch id and year of manager then update status in manager table against branch id in manager
UPDATE manager as m1
SET m1.status = 'Y'
WHERE m1.branch_id IN (
SELECT m2.branch_id FROM manager as m2
WHERE (m2.branch_id,m2.year) IN (
(
SELECT DISTINCT branch_id,year
FROM `branch_master`
WHERE type = 'finance'
)
)
)
but getting error
Table 'm1' is specified twice, both as a target for 'UPDATE' and as a
separate source for data

This is a typical MySQL thing and can usually be circumvented by selecting from the table derived, i.e. instead of
FROM manager AS m2
use
FROM (select * from manager) AS m2
The complete statement:
UPDATE manager
SET status = 'Y'
WHERE branch_id IN
(
select branch_id
FROM (select * from manager) AS m2
WHERE (branch_id, year) IN
(
SELECT branch_id, year
FROM branch_master
WHERE type = 'finance'
)
);

The correct answer is in this SO post.
The problem with here accepted answer is - as was already mentioned multiple times - creating a full copy of the whole table. This is way far from optimal and the most space complex one. The idea is to materialize the subset of data used for update only, so in your case it would be like this:
UPDATE manager as m1
SET m1.status = 'Y'
WHERE m1.branch_id IN (
SELECT * FROM(
SELECT m2.branch_id FROM manager as m2
WHERE (m2.branch_id,m2.year) IN (
SELECT DISTINCT branch_id,year
FROM `branch_master`
WHERE type = 'finance')
) t
)
Basically you just encapsulate your previous source for data query inside of
SELECT * FROM (...) t

Try to use the EXISTS operator:
UPDATE manager as m1
SET m1.status = 'Y'
WHERE EXISTS (SELECT 1
FROM (SELECT m2.branch_id
FROM branch_master AS bm
JOIN manager AS m2
WHERE bm.type = 'finance' AND
bm.branch_id = m2.branch_id AND
bm.year = m2.year) AS t
WHERE t.branch_id = m1.branch_id);
Note: The query uses an additional nesting level, as proposed by #Thorsten, as a means to circumvent the Table is specified twice error.
Demo here

Try :::
UPDATE manager as m1
SET m1.status = 'Y'
WHERE m1.branch_id IN (
(SELECT DISTINCT branch_id
FROM branch_master
WHERE type = 'finance'))
AND m1.year IN ((SELECT DISTINCT year
FROM branch_master
WHERE type = 'finance'))

The problem I had with the accepted answer is that create a copy of the whole table, and for me wasn't an option, I tried to execute it but after several hours I had to cancel it.
A very fast way if you have a huge amount of data is create a temporary table:
Create TMP table
CREATE TEMPORARY TABLE tmp_manager
(branch_id bigint auto_increment primary key,
year datetime null);
Populate TMP table
insert into tmp_manager (branch_id, year)
select branch_id, year
from manager;
Update with join
UPDATE manager as m, tmp_manager as tmp_m
inner JOIN manager as man on tmp_m.branch_id = man.branch_id
SET status = 'Y'
WHERE m.branch_id = tmp_m.branch_id and m.year = tmp_m.year and m.type = 'finance';

This is by far the fastest way:
UPDATE manager m
INNER JOIN branch_master b on m.branch_id=b.branch_id AND m.year=b.year
SET m.status='Y'
WHERE b.type='finance'
Note that if it is a 1:n relationship the SET command will be run more than once. In this case that is no problem. But if you have something like "SET price=price+5" you cannot use this construction.

Maybe not a solution, but some thoughts about why it doesn't work in the first place:
Reading data from a table and also writing data into that same table is somewhat an ill-defined task. In what order should the data be read and written? Should newly written data be considered when reading it back from the same table? MySQL refusing to execute this isn't just because of a limitation, it's because it's not a well-defined task.
The solutions involving SELECT ... FROM (SELECT * FROM table) AS tmp just dump the entire content of a table into a temporary table, which can then be used in any further outer queries, like for example an update query. This forces the order of operations to be: Select everything first into a temporary table and then use that data (instead of the data from the original table) to do the updates.
However if the table involved is large, then this temporary copying is going to be incredibly slow. No indexes will ever speed up SELECT * FROM table.
I might have a slow day today... but isn't the original query identical to this one, which souldn't have any problems?
UPDATE manager as m1
SET m1.status = 'Y'
WHERE (m1.branch_id, m1.year) IN (
SELECT DISTINCT branch_id,year
FROM `branch_master`
WHERE type = 'finance'
)

fixing duplicate fields in mysql table with update statement

I have inherited a table with a field "sku" with should be unique, but thanks to a failing sku-generating method is now littered with dozens of duplicates all around.
I need to quickly fix these duplicates (other parts of the application are failing when encountering these duplicate records) by running an update and appending the record ID to the SKU (which is a valid solution for the time being for this application).
I'm trying to run:
UPDATE
main_product_table
SET sku = CONCAT(sku, '-', CAST(product_id as CHAR) )
WHERE sku IN (
SELECT sku FROM main_product_table
GROUP BY sku
HAVING COUNT(*) > 1
);
But I receive:
You can't specify target table 'main_product_table' for update in FROM clause
Is there a way to accomplish the same? Is mysql complaining about me having main_product_table both in the update and in the subquery to get the duplicates?
Thanks!

Try this:
UPDATE
main_product_table
SET sku = CONCAT(sku, '-', CAST(product_id as CHAR) )
WHERE sku IN (
select * from ( SELECT sku FROM main_product_table
GROUP BY sku
HAVING COUNT(*) > 1) as p
);
Added table alias in inner query.

MySQL INSERT INTO table 1 SELECT table 2 with different column name

I have a table (pdt_1) in database (db_1) and another table (pdt_2) in another database (db_2).
I met pdt_1 and pdt_2 to find pdt_1 products not present and published in pdt_2.
functional code :
SELECT * FROM db_1.pdt_1 AS lm
WHERE lm.product_sku
NOT IN (SELECT DISTINCT product_cip7 FROM db_2.pdt_2)
AND lm.product_publish=‘Y'
finally, I need to insert the result of this query in pdt_2.
However, the structure of pdt_1 and pdt_2 are different.
Example:
- columns's names
- columns's numbers
I also need an auto_increment id for pdt_1 products inserted into pdt_2.
I need help.
NB : sorry for my poor english :(

If you want a new table with just the id and product_sku, try:
INSERT INTO new_table # with id and product_sku from first table
SELECT pdt_1.id,
pdt_1.product_sku
FROM db_1.pdt_1
LEFT JOIN db_2.pdt_2
ON pdt_1.product_sku = pdt_2.product_cip7
WHERE pdt_2.product_cip7 IS NULL
AND pdt_1.product_publish = 'Y'