This question is about linq2db.MySql 3.1.5 and MySQL 8.0.21
I am getting "You have an error in your SQL syntax..." when trying to run an update (marking duplicate emails) using CTE on MySql as per below
var cte = (
from x in db.GetTable<PEmailCleanupJobItem>()
group x by x.EmailAddress into eGroup
where eGroup.Count() > 1
select new { EmailAddress = eGroup.Key }
).AsCte();
var qry = (
from e in db.GetTable<PEmailCleanupJobItem>()
from d in cte
.InnerJoin(d => e.EmailAddress == d.EmailAddress)
select e
);
await qry
.Set(e => e.IsDuplicate, true)
.UpdateAsync();
This generates and sends the following to MySQL
-- MySql.Official MySql (asynchronously)
DECLARE #IsDuplicate UByte -- Boolean
SET #IsDuplicate = 1
UPDATE `CTE_1` (`EmailAddress`)
AS
(
SELECT
`t1`.`email_address`
FROM
`emails` `t1`
GROUP BY
`t1`.`email_address`
HAVING
Count(*) > 1
)
FROM
`emails` `d`
INNER JOIN `CTE_1` `e` ON (`d`.`email_address` IS NULL AND `e`.`EmailAddress` IS NULL OR `d`.`email_address` = `e`.`EmailAddress`)
SET
`d`.`is_duplicate` = #IsDuplicate
Note how UPDATE `CTE_1` (`EmailAddress`) is generated instead of WITH `CTE_1` (`EmailAddress`)
I would manually write this query (which works) as
WITH `CTE_1` (`EmailAddress`)
AS
(
SELECT
`t1`.`email_address`
FROM
`emails` `t1`
GROUP BY
`t1`.`email_address`
HAVING
Count(*) > 1
)
UPDATE
`emails` `d`
INNER JOIN `CTE_1` `e` ON (`d`.`email_address` IS NULL AND `e`.`EmailAddress` IS NULL OR `d`.`email_address` = `e`.`EmailAddress`)
SET
`d`.`is_duplicate` = 1
Is this a bug with linq2db or am I missing something?
Related
I am trying to update a single campaign.id with minimum used_time (datetime) based on user.id but the following code need about 5 seconds to execute. Backlinks table contains 1 million rows.
UPDATE `backlinks`
SET
`backlinks`.`crawler_id` = 'test',
`backlinks`.`used_time`=NOW()
WHERE
`backlinks`.`campaign_id`=(
SELECT `id` FROM `campaigns`
WHERE `campaigns`.`completed`=false
AND `campaigns`.`status`=true
GROUP BY `campaigns`.`user_id`
ORDER BY `campaigns`.`used_time` ASC
limit 1
)
AND `backlinks`.`googlebot_id` IS NULL
AND `backlinks`.`used_time` IS NULL
LIMIT 1;
You can try to UPDATE with JOIN by a subquery.
UPDATE `backlinks` b
JOIN (
SELECT c.id
FROM campaigns c
WHERE exists (
SELECT 1
FROM campaigns cc
WHERE c.user_id = cc.user_id
GROUP BY cc.user_id
HAVING min(cc.used_time) = c.used_time
)
) t1 on b.`campaign_id` = t1.id
SET
b.`crawler_id` = 'test',
b.`used_time`=NOW()
WHERE
b.`googlebot_id` IS NULL
AND
b.`used_time` IS NULL
I have MySQL query currently selecting and joining 13 tables and finally grouping ~60k rows. The query without grouping takes ~0ms but with grouping the query time increases to ~1.7sec. The field, which is used for grouping is primary field and is indexed. Where could be the issue?
I know group by without aggregate is considered invalid query and bad practise but I need distinct base table rows and can not use DISTINCT syntax.
The query itself looks like this:
SELECT `table_a`.*
FROM `table_a`
LEFT JOIN `table_b`
ON `table_b`.`invoice` = `table_a`.`id`
LEFT JOIN `table_c` AS `r1`
ON `r1`.`invoice_1` = `table_a`.`id`
LEFT JOIN `table_c` AS `r2`
ON `r2`.`invoice_2` = `table_a`.`id`
LEFT JOIN `table_a` AS `i1`
ON `i1`.`id` = `r1`.`invoice_2`
LEFT JOIN `table_a` AS `i2`
ON `i2`.`id` = `r2`.`invoice_1`
JOIN `table_d` AS `_u0`
ON `_u0`.`id` = 1
LEFT JOIN `table_e` AS `_ug0`
ON `_ug0`.`user` = `_u0`.`id`
JOIN `table_f` AS `_p0`
ON ( `_p0`.`enabled` = 1
AND ( ( `_p0`.`role` < 2
AND `_p0`.`who` IS NULL )
OR ( `_p0`.`role` = 2
AND ( `_p0`.`who` = '0'
OR `_p0`.`who` = `_u0`.`id` ) )
OR ( `_p0`.`role` = 3
AND ( `_p0`.`who` = '0'
OR `_p0`.`who` = `_ug0`.`group` ) ) ) )
AND ( `_p0`.`action` = '*'
OR `_p0`.`action` = 'read' )
AND ( `_p0`.`related_table` = '*'
OR `_p0`.`related_table` = 'table_name' )
JOIN `table_a` AS `_e0`
ON ( ( `_p0`.`related_id` = 0
OR `_p0`.`related_id` = `_e0`.`id`
OR `_p0`.`related_user` = `_e0`.`user`
OR `_p0`.`related_group` = `_e0`.`group` )
OR ( `_p0`.`role` = 0
AND `_e0`.`user` = `_u0`.`id` )
OR ( `_p0`.`role` = 1
AND `_e0`.`group` = `_ug0`.`group` ) )
AND `_e0`.`id` = `table_a`.`id`
JOIN `table_d` AS `_u1`
ON `_u1`.`id` = 1
LEFT JOIN `table_e` AS `_ug1`
ON `_ug1`.`user` = `_u1`.`id`
JOIN `table_f` AS `_p1`
ON ( `_p1`.`enabled` = 1
AND ( ( `_p1`.`role` < 2
AND `_p1`.`who` IS NULL )
OR ( `_p1`.`role` = 2
AND ( `_p1`.`who` = '0'
OR `_p1`.`who` = `_u1`.`id` ) )
OR ( `_p1`.`role` = 3
AND ( `_p1`.`who` = '0'
OR `_p1`.`who` = `_ug1`.`group` ) ) ) )
AND ( `_p1`.`action` = '*'
OR `_p1`.`action` = 'read' )
AND ( `_p1`.`related_table` = '*'
OR `_p1`.`related_table` = 'table_name' )
JOIN `table_g` AS `_e1`
ON ( ( `_p1`.`related_id` = 0
OR `_p1`.`related_id` = `_e1`.`id`
OR `_p1`.`related_user` = `_e1`.`user`
OR `_p1`.`related_group` = `_e1`.`group` )
OR ( `_p1`.`role` = 0
AND `_e1`.`user` = `_u1`.`id` )
OR ( `_p1`.`role` = 1
AND `_e1`.`group` = `_ug1`.`group` ) )
AND `_e1`.`id` = `table_a`.`company`
WHERE `table_a`.`date_deleted` IS NULL
AND `table_a`.`company` = 4
AND `table_a`.`type` = 1
AND `table_a`.`date_composed` >= '2016-05-04 14:43:55'
GROUP BY `table_a`.`id`
The ORs kill performance.
This composite index may help: INDEX(company, type, date_deleted, date_composed).
LEFT JOIN table_b ON table_b.invoice = table_a.id seems to do absolutely nothing other than slow down the processing. No fields of table_b are used or SELECTed. Since it is a LEFT join, it does not limit the output. Etc. Get rid if it, or justify it.
Ditto for other joins.
What happens with JOIN and GROUP BY: First, all the joins are performed; this explodes the number of rows in the intermediate 'table'. Then the GROUP BY implodes the set of rows.
One technique for avoiding this explode-implode sluggishness is to do
SELECT ...,
( SELECT ... ) AS ...,
...
instead of a JOIN or LEFT JOIN. However, that works only if there is zero or one row in the subquery. Usually this is beneficial when an aggregate (such as SUM) can be moved into the subquery.
For further discussion, please include SHOW CREATE TABLE.
I have this update query which works as well:
UPDATE tbname t CROSS JOIN ( SELECT related FROM tbname WHERE id = 5 ) x
SET AcceptedAnswer = ( id = 5 )
WHERE t.related = x.related
I also have two select statements which validates somethings. Actually I want to check these to conditions before updating:
Condition1:
(SELECT 1 FROM tbname
WHERE id = x.related AND
author_id = 29
)
Condition2:
(SELECT 1 FROM tbname
WHERE id = x.related AND
(
( amount IS NOT NULL AND
NOT EXISTS ( SELECT 1 FROM tbname
WHERE related = x.related AND
AcceptedAnswer = 1 )
) OR amount IS NULL
)
)
How can I combine those two conditions with that updating query?
Here is what I've tried so far but it doesn't work and throws this error:
UPDATE tbname CROSS JOIN ( SELECT related FROM tbname WHERE id = 5 ) x
SET AcceptedAnswer = ( id = 5 )
WHERE q.related = x.related
AND
(SELECT 1 FROM tbname
WHERE id = x.related AND
author_id = 29
) AND
(SELECT 1 FROM tbname
WHERE id = x.related AND
(
( amount IS NOT NULL AND
NOT EXISTS ( SELECT 1 FROM tbname
WHERE related = x.related AND
AcceptedAnswer = 1 )
) OR amount IS NULL
)
)
#1093 - You can't specify target table 'tbname' for update in FROM clause
Seems your update is equivalent to this
update tbname as a
inner join tbname as b on a.related = b.related and b.id = 5
set AcceptedAnswer = (id = 5)
your query seem set to true (1) the AccepetdAnswer of the row with id = 5 for the row that have acceppeted equalt to the accepted value of th row with id = 5 (false / 0) in the other case ..
for test use
select * from tbname as a
inner join tbname as b on a.related = b.related and b.id = 5
and (b.related = a.id and a.author_id = 29)
and (b.related = a.id and
(a.amont is not null and (a.related = b.related and a.AcceptedAnswer = 1)))
I'm not pretty sure what is the purpose of the SET clause (id =5)
anyway this way avoids the use of the cross join provided that you
don't use the table "x" to get something beyond the "related" items.
UPDATE tbname
SET
AcceptedAnswer = ( id = 5 )
WHERE
#THIS IS EQUIVALENT TO THE JOIN CLAUSE
id IN ( SELECT related FROM tbname WHERE id = 5 )
#THIS IS THE CONDITION 1 POINTING tnname
AND author_id = 29
#THIS IS THE CONDITION 2 POINTING tbname
AND (
( amount IS NOT NULL
AND NOT AcceptedAnswer = 1
) OR amount IS NOT NULL
)
;
EDIT: Working now, solution in the bottom of this post.
I have tried to create this query for hours now, without success:
UPDATE tasks
SET `Order`=
(
CASE WHEN
(
SELECT EXISTS
(
SELECT 1
FROM user_job_tasks ujt
WHERE ujt.JobID = :jobID AND ujt.TaskID = LAST_INSERT_ID()
)
)
THEN
(
SELECT `order` FROM
(
SELECT MAX(t.`Order`)+1 AS `Order`
FROM user_job_tasks ujt
LEFT OUTER JOIN tasks t ON ujt.TaskID = t.ID
WHERE ujt.JobID = :jobID
) AS temp
)
ELSE
(
1
)
END
)
WHERE ID = LAST_INSERT_ID()
Now, the point of this is to update tasks table's column Order where ID equals something (LAST_INSERT_ID() here).
If any records exist in user_job_tasks where jobID and taskID match the values, I want to set Order as the highest value + 1 of Order found in user_job_tasks where jobID matches the value.
If not, Order will be set 1.
I am only getting an error General error: 1093 You can't specify target table 'tasks' for update in FROM clause. I fail to find anything wrong in the query syntax.
Sorry for confusing explanation, perhaps I should just go to sleep.
Working solution:
UPDATE tasks
SET `Order`=
(
COALESCE
(
(
SELECT `Order` FROM
(
SELECT MAX(t.`Order`)+1 AS `Order`
FROM user_job_tasks ujt
LEFT OUTER JOIN tasks t ON ujt.TaskID = t.ID
WHERE ujt.JobID = :jobID
) AS temp
),
1
)
)
WHERE ID = LAST_INSERT_ID()
I am not 100% sure this does what you want, but I think it does:
UPDATE tasks
SET `Order`= coalesce((SELECT MAX(t.`Order`) + 1 AS `Order`
FROM user_job_tasks ujt JOIN
tasks t
ON ujt.TaskID = t.ID
WHERE ujt.JobID = :jobID
), 1)
WHERE ID = LAST_INSERT_ID();
This just looks for the maximum id (+ 1) for your records and then uses 1 if this doesn't exist.
EDIT:
MySQL doesn't like using the table being updated in a from clause. So the above is correctly stated using a second subquery:
UPDATE tasks
SET `Order`= coalesce((SELECT `Order`
FROM (SELECT MAX(t.`Order`) + 1 AS `Order`
FROM user_job_tasks ujt JOIN
tasks t
ON ujt.TaskID = t.ID
WHERE ujt.JobID = :jobID
)
), 1)
WHERE ID = LAST_INSERT_ID();
This limitation would also apply to your query.
How to check in MySQL table if my ID exist in the results and return TRUE, but this is a little complicated as I would like to check GROUP of records and not the last one from that GROUP, code below for the table and query which doesn't work, I would like to check if my $session_user_id exist in the GROUP OF auction_bid_item_id but not check last row as the last row is a WINNING ROW.
CREATE TABLE `auction_bids` (
`auction_bid_id` int(11) NOT NULL AUTO_INCREMENT,
`auction_bid_user_id` int(11) NOT NULL,
`auction_bid_seller_id` int(11) NOT NULL,
`auction_bid_item_id` int(11) NOT NULL,
PRIMARY KEY (`auction_bid_id`)
);
MySQL Query:
SELECT * FROM auction_bids
WHERE auction_bid_user_id = '$session_user_id'
GROUP BY auction_bid_item_id;
Join against a subselect which determines if the user is in that group.
Ie, the following will get you the bids and a field saying if the user is found.
SELECT auction_bids.*, IF(Sub1.auction_bid_item_id IS NULL, 'User not found', 'User found')
FROM auction_bids
LEFT OUTER JOIN
(
SELECT DISTINCT auction_bid_item_id
FROM auction_bids
WHERE auction_bid_user_id = '$session_user_id'
) Sub1
ON auction_bids.auction_bid_item_id = Sub1.auction_bid_item_id
GROUP BY auction_bids.auction_bid_item_id
EDIT - try this to try and ignore the latest bid
SELECT auction_bids.*, IF(Sub1.auction_bid_item_id IS NULL, 'User not found', 'User found')
FROM auction_bids
LEFT OUTER JOIN
(
SELECT DISTINCT auction_bids.auction_bid_item_id
FROM auction_bids
LEFT OUTER JOIN
(
SELECT auction_bid_item_id, MAX(auction_bid_id) AS LastBid
FROM auction_bids
GROUP BY auction_bid_item_id
) Sub2
ON auction_bids.auction_bid_item_id = Sub2.auction_bid_item_id
AND auction_bids.auction_bid_id = Sub2.LastBid
WHERE Sub2.auction_bid_item_id IS NULL
AND auction_bid_user_id = '$session_user_id'
) Sub1
ON auction_bids.auction_bid_item_id = Sub1.auction_bid_item_id
GROUP BY auction_bids.auction_bid_item_id
Something like this? (with php)
$id_exists = false;
$id = mysql_real_escape_string($_GET['id']);
$select = "SELECT auction_bid_id FROM auction_bids WHERE auction_bid_id ='$id'";
$result = mysql_query($select);
if(mysql_num_rows($result) > 0){
$id_exists = true;
}else{
$id_exists = false;
}