TL;DR (i.e. asking the question first):
Is there any way to write an INSERT INTO...SELECT FROM...GROUP BY...ON DUPLICATE KEY UPDATE statement using row alias(es) in the ON DUPLICATE KEY UPDATE clause instead of the col1 = VALUES(col1) syntax that has been deprecated and will be removed from future MySQL releases?
My searches of SO relating to this issue tend to all suggest using the deprecated VALUES() function, which is why I believe that my question is not a duplicate.
BACKGROUND (i.e. more info on how to reproduce the issue)
I have a table that comprises grouped records from another table. For simplicity in describing this issue, I've created two sample tables purely to illustrate:
items:
item_groups (below) was populated using the following SQL:
insert into item_groups (item_type,quantity) (select item_type, count(*) from items group by item_type order by item_type)
It also has a unique index on item_type:
Now, let's say that I add two more items to the items table, one with an item_type of 4 and one with a new item_type of 5. The quantity of item_type 4 in item_groups should be updated to 3 and a new row inserted for the item_type of 5 with quantity of 1.
Using the same INSERT statement I used above to initially populate the item_groups table, I now get an error, which is expected because of a duplicate key (4 of the 5 item_types currently in the items table are duplicates of the item_types that currently exist in the item_groups table):
Zero updates or inserts were completed due to this error. To remedy this, we would have historically used the ON DUPLICATE KEY UPDATE (occasionally abbreviated to ODKU below) clause like so including the VALUES() function:
insert into item_groups (item_type,quantity) (select item_type, count(*) from items group by item_type order by item_type) ON DUPLICATE KEY UPDATE quantity = VALUES(quantity);
The above INSERT...ON DUPLICATE KEY UPDATE statement with VALUES() DOES work (currently)...
However, I am also greeted with the following warning:
'VALUES function' is deprecated and will be removed in a future
release. Please use an alias (INSERT INTO ... VALUES (...) AS alias)
and replace VALUES(col) in the ON DUPLICATE KEY UPDATE clause with
alias.col instead
Now, I know how to write a simple INSERT...ODKU statement to be future-proof against the warning above (generically):
INSERT INTO `my_table` (col1,col2,col3) VALUES (1,2,3) AS new ON DUPLICATE KEY UPDATE col1 = new.col1, col2 = new.col2, col3 = new.col3
But let's insert more items into my items table and then use the above syntax for my more complicated INSERT...SELECT...ODKU statement into item_groups:
insert into item_groups (item_type,quantity) (select item_type, count(*) from items group by item_type order by item_type) AS new ON DUPLICATE KEY UPDATE quantity = new.quantity;
I get this error:
ERROR 1064 (42000): You have an error in your SQL syntax; check the
manual that corresponds to your MySQL server version for the right
syntax to use near 'AS new ON DUPLICATE KEY UPDATE quantity =
new.quantity' at line 1
Adding "VALUES" prior to my SELECT subquery, like so...
insert into item_groups (item_type,quantity) VALUES (select item_type, count(*) from items group by item_type order by item_type) AS new ON DUPLICATE KEY UPDATE quantity = new.quantity;
I now get a new syntax error:
ERROR 1064 (42000): You have an error in your SQL syntax; check the
manual that corresponds to your MySQL server version for the right
syntax to use near 'select item_type, count(*) from items group by
item_type order by item_type) AS ' at line 1
Finally, at my wit's end, I try adding another set of parentheses around the SELECT sub-query...
insert into item_groups (item_type,quantity) VALUES ((select item_type, count(*) from items group by item_type order by item_type)) AS new ON DUPLICATE KEY UPDATE quantity = new.quantity;
...and I still get an error:
ERROR 1136 (21S01): Column count doesn't match value count at row 1
This appears to be "progress" as I'm no longer getting syntax errors; however, I don't understand why the column count doesn't match the value count. My SELECT subquery pulls in 2 values for each row and the INSERT attempts to insert those into 2 columns for each row. So it would seem to me that 2 values -> 2 columns should not be an issue; yet it is.
CONCLUSION
I'm frankly not even sure what else to try, and I'm about ready to give up doing it this way and just write a simple SELECT, store those retrieved values in variables, and then use a simple INSERT to insert those values (wrapping everything in a transaction). However, if there is a way to do what I'm trying to do in one statement, I would appreciate anyone who can help me to do this.
From MySQL docs
Beginning with MySQL 8.0.20, an INSERT ... SELECT ... ON DUPLICATE KEY
UPDATE statement that uses VALUES() in the UPDATE clause, like this
one, throws a warning:
INSERT INTO t1 SELECT c, c+d FROM t2 ON DUPLICATE KEY UPDATE b =
VALUES(b); You can eliminate such warnings by using a subquery
instead, like this:
INSERT INTO t1 SELECT * FROM (SELECT c, c+d AS e FROM t2) AS dt ON
DUPLICATE KEY UPDATE b = e;
In simple words you could use a subquery as follows:
insert into item_groups (item_type,
quantity)
select * from ( select item_type , count(*) as new_quantity from items group by item_type ) as tbl
ON DUPLICATE KEY UPDATE quantity = new_quantity;
https://dbfiddle.uk/HoMLKMfd
You need a version mysql that is newer
8.0.30 and 8.0,31 this works
The use of VALUES() to refer to the new row and columns is deprecated beginning with MySQL 8.0.20, and is subject to removal in a future version of MySQL. Instead, use row and column aliases, as described in the next few paragraphs of this section.
so it shpuld work with 8.0.20 too
Besides security risks, with very update come new functions and old bugs are fixed.
Deploying for two Versions is bad, as you need more and more code to support more database version or to simulate functions you need, that you open your code to more and more bugs and insecurities.
So make a cut and use the latest Version
CREATE TABLE `my_table` (col1 int unique,col2 int ,col3 int)
INSERT INTO `my_table` (col1,col2,col3) VALUES (1,2,3) AS new
ON DUPLICATE KEY UPDATE col1 = new.col1, col2 = new.col2, col3 = new.col3
SELECT * FROM `my_table`
col1
col2
col3
1
2
3
fiddle
Related
Clearly the following is incorrect.
INSERT INTO `aTable` (`A`,`B`) VALUES((SELECT MAX(`A`) FROM `aTable`)*2),'name');
I get the value:
SQL query:
INSERT INTO `aTable` (`A`, `B` )
VALUES
(
(
SELECT MAX(`A`)
FROM `aTable`
) *2
, 'name'
)
MySQL said:
1093 - You can't specify target table 'aTable' for update in FROM clause
So, I'm trying to make a bitmap table, each row corresponds to one Bit, and has a 'map' value.
To insert in the table, I don't want to do two queries, I want to do one.
How should I do this?
No one commented on this, but since I am trying to make a bitmap, it should be * 2 not ^ 2, my mistake, please note that is why the comments often say ^ 2, it was an error in the version that the commenters read.
try:
insert into aTable select max(a)^2, 'name' from aTable;
or
insert into aTable select max(a)^2, 'name' from aTable group by B;
If you need a join, you can do this:
insert into aTable select max(a)^2, 'name' from aTable, bTable;
My "Server version" is "5.0.51b-community-nt MySQL Community Edition (GPL)"
Actually, you can alias the table on the insert. I've seen this question all over the place, but no one seems to have tried that. Use a subquery to get the max from the table, but alias the table in the subquery.
INSERT INTO tableA SET fieldA = (SELECT max(x.fieldA) FROM tableA x)+1;
A more complex example, where you have a corresponding secondary key and might be inserting the FIRST record for the corresponding secondary key:
INSERT INTO tableA SET secondaryKey = 123, fieldA = COALESCE((SELECT max(x.fieldA) FROM tableA x WHERE x.secondaryKey = 123)+1,1);
By aliasing the table, it doesn't throw the error and seems to work. I just did this while coding something, although I can't see if there area any silly syntax errors above, I would try that type of syntax.
I take it that INSERT ... SELECT isn't working? I see this in the documentation for it:
The target table of the INSERT
statement may appear in the FROM
clause of the SELECT part of the
query. (This was not possible in some
older versions of MySQL.) In this
case, MySQL creates a temporary table
to hold the rows from the SELECT and
then inserts those rows into the
target table.
Out of curiosity, which version of MySQL are you using?
I think you need to drop the "VALUES", and have a valid select statement.
see this link
I'm not particularly a mySQL guy, I use MSSQL mostly. But If you format the select statement correctly, It should work.
as soon as the Select is correct you can do this.
Code:
UPDATE COMPANY SET id='21'
WHERE id='20';
Error:
SQLException: Duplicate entry '21' for key 'PRIMARY'
I want to UPDATE the primary key field in this case it's called 'id' to another value but if the value exists already it throws the error above. How would I do a conditional UPDATE based on if the 'id' doesn't exist in the COMPANY table already, to avoid throwing that error using MariaDB syntax.
NOTE:
I am NOT talking about doing a conditional INSERT that uses "ON DUPLICATE KEY UPDATE" as shown below.
INSERT INTO COMPANY(id,first,last,age)
VALUES('1','Tim','Jones','70')
ON DUPLICATE KEY UPDATE id='1';
You can use UPDATE IGNORE:
UPDATE IGNORE COMPANY
SET id='21'
WHERE id = '20'
See a simplified demo.
You can count the number of values already in the table:
UPDATE COMPANY C CROSS JOIN
(SELECT COUNT(*) as cnt
FROM COMPANY
WHERE id = 21
) CC
SET c.id = 21
WHERE id = 20 AND cnt = 0;
Note: In most databases, you would use NOT EXISTS in the WHERE clause, but MySQL/MariaDB doesn't support references to the table being updated.
UPDATE AggregatedData SET datenum="734152.979166667",
Timestamp="2010-01-14 23:30:00.000" WHERE datenum="734152.979166667";
It works if the datenum exists, but I want to insert this data as a new row if the datenum does not exist.
UPDATE
the datenum is unique but that's not the primary key
Jai is correct that you should use INSERT ... ON DUPLICATE KEY UPDATE.
Note that you do not need to include datenum in the update clause since it's the unique key, so it should not change. You do need to include all of the other columns from your table. You can use the VALUES() function to make sure the proper values are used when updating the other columns.
Here is your update re-written using the proper INSERT ... ON DUPLICATE KEY UPDATE syntax for MySQL:
INSERT INTO AggregatedData (datenum,Timestamp)
VALUES ("734152.979166667","2010-01-14 23:30:00.000")
ON DUPLICATE KEY UPDATE
Timestamp=VALUES(Timestamp)
Try using this:
If you specify ON DUPLICATE KEY UPDATE, and a row is inserted that would cause a duplicate value in a UNIQUE index orPRIMARY KEY, MySQL performs an [UPDATE`](http://dev.mysql.com/doc/refman/5.7/en/update.html) of the old row...
The ON DUPLICATE KEY UPDATE clause can contain multiple column assignments, separated by commas.
With ON DUPLICATE KEY UPDATE, the affected-rows value per row is 1 if the row is inserted as a new row, 2 if an existing row is updated, and 0 if an existing row is set to its current values. If you specify the CLIENT_FOUND_ROWS flag to mysql_real_connect() when connecting to mysqld, the affected-rows value is 1 (not 0) if an existing row is set to its current values...
This is not too bad, but we could actually combine everything into one query. I found different solutions on the internet. The simplest, but MySQL only solution is this:
INSERT INTO wp_postmeta (post_id, meta_key)
SELECT
?id,
‘page_title’
FROM
DUAL
WHERE
NOT EXISTS (
SELECT
meta_id
FROM
wp_postmeta
WHERE
post_id = ?id
AND meta_key = ‘page_title’
);
UPDATE
wp_postmeta
SET
meta_value = ?page_title
WHERE
post_id = ?id
AND meta_key = ‘page_title’;
Link to documentation.
I had a situation where I needed to update or insert on a table according to two fields (both foreign keys) on which I couldn't set a UNIQUE constraint (so INSERT ... ON DUPLICATE KEY UPDATE won't work). Here's what I ended up using:
replace into last_recogs (id, hasher_id, hash_id, last_recog)
select l.* from
(select id, hasher_id, hash_id, [new_value] from last_recogs
where hasher_id in (select id from hashers where name=[hasher_name])
and hash_id in (select id from hashes where name=[hash_name])
union
select 0, m.id, h.id, [new_value]
from hashers m cross join hashes h
where m.name=[hasher_name]
and h.name=[hash_name]) l
limit 1;
This example is cribbed from one of my databases, with the input parameters (two names and a number) replaced with [hasher_name], [hash_name], and [new_value]. The nested SELECT...LIMIT 1 pulls the first of either the existing record or a new record (last_recogs.id is an autoincrement primary key) and uses that as the value input into the REPLACE INTO.
I'm using the mysql node module.
My query is:
insert into myTableA (fk_1, fk_2)
values (
(select id from myTableB where name = ?),
(select id from myTableC where name = ?)
)
My method call looks like:
db.query(q, [values], (err) => { ... })
Each item in values is an array whose items are respectively the names in myTableB and myTableC. As shown above, I made sure to wrap values in another array.
The error I get is:
ER_PARSE_ERROR: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ...`
Help please...
EDIT 1:
The values I am trying to insert look like the following:
var values = [
[name00, name01],
[name10, name11],
[name20, name21],
...
]
EDIT 2:
Here's to clarify my confusion... Basically, I am looking for a way that would allow me to use the INSERT INTO query to combine the following 2 queries:
INSERT INTO MyTable (col_1, col_2, col_3) VALUES ?
and:
INSERT INTO MyTableA (col_1, col_2, col_3) VALUES (
(select id from MyTableB where name=?),
(select id from MyTableC where name=?),
?
)
The query I am looking for would let me pass a values variable in the following form:
var values = [
[row_0_col_1, row_0_col_2, row_0_col_3],
[row_1_col_1, row_1_col_2, row_1_col_3],
[row_2_col_1, row_2_col_2, row_2_col_3],
...
]
myTableA appears to be a many-to-many table, and you're trying to populate the rows to be foreign keys references to myTableB and myTableC.
With the INSERT ... VALUES syntax, you can specify rows of data values, but each row must be specified. Like this:
INSERT INTO myTableA (fk_1, fk_2) VALUES
('name00', 'name01');
('name10', 'name11');
('name20', 'name21');
Each row is independent and self-contained. If you use subqueries, each subquery must return a single value (this is called a scalar subquery). You can't use subqueries that return multiple values.
You can also use INSERT ... SELECT syntax, so instead of listing rows one by one, the INSERT just uses the rows returned by a SELECT statement.
INSERT INTO myTableA (fk_1, fk_2)
SELECT b.id, c.id
FROM myTableB JOIN myTableC ON (...);
But you would have to provide some join expression to describe how to match values from the two respective tables in the join.
This would require inventing the many-to-many relationship out of thin air.
As for the syntax error returned by node, the error you included in your post above is truncated where you typed .... This isn't a complete error message. The query you posted is obviously edited (I suppose your table names are not myTableA, myTableB, etc.). You're asking for help but you have provided neither the real query you executed, nor the real error message. Given that, there's no way for anyone on Stack Overflow to guess at the cause of the error, or any fix for it.
Try using SQL Join feature, to insert the selected values from multiple tables.
An example for your reference:
INSERT INTO Table (aID, bID)
SELECT a.ID, B.ID
FROM A, B
WHERE A.Name='Me'
AND B.Class='Math';
I am trying to insert records into MySQL database from a MS SQL Server using the "OPENQUERY" but what I am trying to do is ignore the duplicate keys messages. so when the query run into a duplicate then ignore it and keep going.
What ideas can I do to ignore the duplicates?
Here is what I am doing:
pulling records from MySQL using "OpenQuery" to define MySQL "A.record_id"
Joining those records to records in MS SQL Server "with a specific criteria and not direct id" from here I find a new related "B.new_id" record identifier in SQL Server.
I want to insert the found results into a new table in MySQL like so A.record_id, B.new_id Here in the new table I have A.record_id set as a primary key for that table.
The problem is that when joining table A to Table B some times I find 2+ records into table B matching the criteria that I am looking for which causes the value A.record_id to 2+ times in my data set before inserting that into table A which causes the problem. Note I can use aggregate function to eliminate the records.
I don't think there is a specific option. But it is easy enough to do:
insert into oldtable(. . .)
select . . .
from newtable
where not exists (select 1 from oldtable where oldtable.id = newtable.id)
If there is more than one set of unique keys, you can add additional not exists statements.
EDIT:
For the revised problem:
insert into oldtable(. . .)
select . . .
from (select nt.*, row_number() over (partition by id order by (select null)) as seqnum
from newtable nt
) nt
where seqnum = 1 and
not exists (select 1 from oldtable where oldtable.id = nt.id);
The row_number() function assigns a sequential number to each row within a group of rows. The group is defined by the partition by statement. The numbers start at 1 and increment from there. The order by clause says that you don't care about the order. Exactly one row with each id will have a value of 1. Duplicate rows will have a value larger than one. The seqnum = 1 chooses exactly one row per id.
If you are on SQL Server 2008+, you can use MERGE to do an INSERT if row does not exist, or an UPDATE.
Example:
MERGE
INTO dataValue dv
USING tmp_holding_DataValue t
ON t.dateStamp = dv.dateStamp
AND t.itemId = dv.itemId
WHEN NOT MATCHED THEN
INSERT (dateStamp, itemId, value)
VALUES (dateStamp, itemId, value)