I am trying to update a single campaign.id with minimum used_time (datetime) based on user.id but the following code need about 5 seconds to execute. Backlinks table contains 1 million rows.
UPDATE `backlinks`
SET
`backlinks`.`crawler_id` = 'test',
`backlinks`.`used_time`=NOW()
WHERE
`backlinks`.`campaign_id`=(
SELECT `id` FROM `campaigns`
WHERE `campaigns`.`completed`=false
AND `campaigns`.`status`=true
GROUP BY `campaigns`.`user_id`
ORDER BY `campaigns`.`used_time` ASC
limit 1
)
AND `backlinks`.`googlebot_id` IS NULL
AND `backlinks`.`used_time` IS NULL
LIMIT 1;
You can try to UPDATE with JOIN by a subquery.
UPDATE `backlinks` b
JOIN (
SELECT c.id
FROM campaigns c
WHERE exists (
SELECT 1
FROM campaigns cc
WHERE c.user_id = cc.user_id
GROUP BY cc.user_id
HAVING min(cc.used_time) = c.used_time
)
) t1 on b.`campaign_id` = t1.id
SET
b.`crawler_id` = 'test',
b.`used_time`=NOW()
WHERE
b.`googlebot_id` IS NULL
AND
b.`used_time` IS NULL
Related
I have an issue with the following mySQL query where it fails when Max date is introduced as shown below.
I get the following error
Error Code: 1054. Unknown column 'order_items.ORDER_ITEM_ID' in 'where
clause'
SET #UserID = 160;
SET #OrderDateTime = '2018-11-13 09:23:45';
SELECT
order_items.ORDER_ID,
listing_region.LIST_REGION_REGION_ID,
listings.LISTING_ID,
order_items.ORDER_REQUIRED_DATE_TIME,
listings.LISTING_NICK_NAME,
order_items.ORDER_QUANTITY,
order_price.ORDER_PRICE_ID,
order_items.ORDER_PORTION_SIZE,
t.LATEST_DATE,
t.ORDER_STATUS
FROM order_status_change, order_items
INNER JOIN listings ON listings.LISTING_ID = order_items.ORDER_LISTING_ID
INNER JOIN listing_region ON listing_region.LIST_REGION_LISTING_ID = listings.LISTING_ID
INNER JOIN order_price ON order_price.ORDERP_ITEM_ID = order_items.ORDER_ITEM_ID
INNER JOIN
(
SELECT MAX(order_status_change.ORDER_STATUS_CHANGE_DATETIME) AS LATEST_DATE, order_status_change.ORDER_ITEM_ID, order_status_change.ORDER_STATUS
FROM order_status_change
WHERE order_status_change.ORDER_ITEM_ID = order_items.ORDER_ITEM_ID
) AS t ON order_status_change.ORDER_ITEM_ID = t.ORDER_ITEM_ID AND order_status_change.ORDER_STATUS_CHANGE_DATETIME = t.LATEST_DATE
WHERE ((order_items.ORDER_USER_ID = #UserID) AND DATE(order_items.ORDER_REQUIRED_DATE_TIME) = DATE(#OrderDateTime))
Any help ?
I have assumed you can join order_status_change on order_items.ID = order_status_change.ORDER_ITEM_ID
If that is valid then I think this will achieve what you are after:
SET #UserID = 160;
SET #OrderDateTime = '2018-11-13 09:23:45';
SELECT
order_items.ORDER_ID
, listing_region.LIST_REGION_REGION_ID
, listings.LISTING_ID
, order_items.ORDER_REQUIRED_DATE_TIME
, listings.LISTING_NICK_NAME
, order_items.ORDER_QUANTITY
, order_price.ORDER_PRICE_ID
, order_items.ORDER_PORTION_SIZE
, t.LATEST_DATE
, order_status_change.ORDER_STATUS
FROM order_items
INNER JOIN listings ON listings.LISTING_ID = order_items.ORDER_LISTING_ID
INNER JOIN listing_region ON listing_region.LIST_REGION_LISTING_ID = listings.LISTING_ID
INNER JOIN order_price ON order_price.ORDERP_ITEM_ID = order_items.ORDER_ITEM_ID
INNER JOIN order_status_change ON order_items.ID = order_status_change.ORDER_ITEM_ID
INNER JOIN (
SELECT
MAX( mc.ORDER_STATUS_CHANGE_DATETIME ) AS LATEST_DATE
, mc.ORDER_ITEM_ID
FROM order_status_change AS mc
GROUP BY
mc.ORDER_ITEM_ID
) AS t
ON order_status_change.ORDER_ITEM_ID = t.ORDER_ITEM_ID
AND order_status_change.ORDER_STATUS_CHANGE_DATETIME = t.LATEST_DATE
WHERE order_items.ORDER_USER_ID = #UserID
AND DATE( order_items.ORDER_REQUIRED_DATE_TIME ) = DATE( #OrderDateTime )
You need to avoid this in future:
FROM order_status_change , order_items
That comma between the 2 table names IS a join, but it is from an older syntax and it is LOWER in precedence than the other joins of your query. Also, by default this comma based join acts as an equivalent to a cross join which MULTIPLIES the number of rows. In brief, please do NOT USE commas between table names.
The other issue is that you were missing a group by clause and I believe you just want to get the "latest" date from this aggregation, once that is determined link back to that table to get the status relevant to that date. (i.e. you can't group by status in the subquery, otherwise you get the latest dateS (one for each status).
Here's a simplified version to illustrate the problem.
DROP TABLE IF exists t,t1;
create table t (id int);
create table t1(id int,dt date);
insert into t values (1),(2);
insert into t1 values (1,'2018-01-01'),(1,'2018-02-01'),(2,'2018-01-01');
select t.*,t2.maxdt
from t
join (select max(dt) maxdt,t1.id from t1 where t1.id = t.id) t2
on t2.id = t.id;
ERROR 1054 (42S22): Unknown column 't.id' in 'where clause'
You could group by in the sub query and then the on clause will come into play
select t.*,t2.maxdt
from t
join (select max(dt) maxdt,t1.id from t1 group by t1.id) t2
on t2.id = t.id;
+------+------------+
| id | maxdt |
+------+------------+
| 1 | 2018-02-01 |
| 2 | 2018-01-01 |
+------+------------+
2 rows in set (0.00 sec)
If you want an answer closer to your problem please add sample data and expected output to the question as text of to sqlfiddle.
In following query I indexed every field like:
menu.id
pricelist.menu_id
vendors.id
pricelist.vendor
orders.pricelist_id
pricelist.id
users.id
orders.user_id
orders.free
pricelist.menu_id.
When I run the below query it takes much time, we have 13 million records in orders table and other table has few thousands.
SELECT
`orders`.`itusername`,
`orders`.`iturl`,
`orders`.`error_message`,
`orders`.`return` AS return1,
`orders`.`coupon`,
DATEDIFF( users.reseller_expiry, now( ) ) AS edays,
`users`.`email`,
`menu`.`menuname`,
`orders`.`error_status`,
`orders`.`auto_status`,
`vendors`.`name`,
`vendors`.`id` AS venderid,
`pricelist`.`servicename`,
`orders`.`email_order` AS paypal_order_email,
`orders`.`user_id`,
`orders`.`services_order`,
`orders`.`created_dt`,
`orders`.`id`,
`orders`.`transaction_comment`,
`orders`.`url`,
`orders`.`requireviews`,
`orders`.`youtubeviews`,
`orders`.`total_views_completed`,
`orders`.`aff`,
`orders`.`is_package`,
`orders`.`price`,
`orders`.`order_from_site`,
`orders`.`cost_per_unit_order`,
`orders`.`service_name_order`,
`orders`.`status`,
`orders`.`start_api_date`,
`orders`.`end_api_date`,
`orders`.`allow_setting`,
`orders`.`return2`,
`users`.`balance` AS user_balance
FROM
( `pricelist` )
JOIN `menu` ON `menu`.`id` = `pricelist`.`menu_id`
JOIN `vendors` ON `vendors`.`id` = `pricelist`.`vendor`
JOIN `orders` ON `orders`.`pricelist_id` = `pricelist`.`id`
JOIN `users` ON `users`.`id` = `orders`.`user_id`
WHERE
`orders`.`free` != 1
AND pricelist.menu_id = 3
ORDER BY
`orders`.`id` DESC
LIMIT 10.
.................................
The trick is to do the LIMIT before running around to 5 tables.
SELECT ...
FROM
( SELECT orders.pricelist_id AS id
FROM pricelist
JOIN `orders` ON `orders`.`pricelist_id` = `pricelist`.`id`
WHERE `orders`.`free` != 1
AND pricelist.menu_id = 3
ORDER BY `orders`.`id` DESC
LIMIT 10
) AS x
JOIN `pricelist` ON pricelist.id = x.id
JOIN `menu` ON `menu`.`id` = `pricelist`.`menu_id`
JOIN `vendors` ON `vendors`.`id` = `pricelist`.`vendor`
JOIN `orders` ON `orders`.`pricelist_id` = x.`id`
JOIN `users` ON `users`.`id` = `orders`.`user_id`
ORDER BY `orders`.`id` DESC
There is a unknown -- are the tables 1:1 or 1:many or many:1 or many:many? With the 'wrong' answers, you will get more than 10 rows, and the design is flawed.
I have a MySQL query and it takes about 25 sec. There are not many rows (just about 200) but I don't understand why it takes long time.
Query:
SELECT *
, c.id c_id
FROM campaign c
JOIN campaign_category cc
ON c.campaign_type = cc.id
WHERE c.is_deleted = 0
AND c.status = 1
AND c.id NOT IN (SELECT campaign_id FROM user_reviews WHERE user_id = 4)
AND c.amt_req > (SELECT COUNT(id)
FROM reserved_reviews
WHERE camping_id = c.id
AND user_id != 4)
+ (SELECT COUNT(id)
FROM user_reviews
WHERE campaign_id = c.id)
Edit:
I tried with JOIN like this but i got no result:
SELECT
*, `c`.`id` as `c_id`,COUNT(`ur`.`id`) as `total_reviewed`, COUNT(`rr`.`id`) as `total_reserved`
FROM
`campaign` `c`
JOIN `campaign_category` `cc` ON `c`.`campaign_type`=`cc`.`id`
JOIN `user_reviews` `ur` ON `ur`.`campaign_id`=`c`.`id`
JOIN `reserved_reviews` `rr` ON `rr`.`camping_id`=`c`.`id`
WHERE
`c`.`is_deleted` =0
AND
`c`.`status` = 1
AND
`ur`.`user_id` != 4
GROUP BY `c`.`id`
HAVING `c`.`amt_req` > COUNT(`ur`.`id`) + COUNT(`rr`.`id`)
Edit: Table structures: First Image - user_reviews Table, Second image campagin Table, Third image: reserved_reviews Table.
http://imgur.com/GI4817B,SdnSxuz,truxHM6#0
You can improve this query with indexes;
SELECT *, c.id c_id
FROM campaign c JOIN
campaign_category cc
ON c.campaign_type = cc.id
WHERE c.is_deleted = 0 AND
c.status = 1 AND
c.id NOT IN (SELECT campaign_id FROM user_reviews WHERE user_id = 4)
c.amt_req > (SELECT COUNT(*)
FROM reserved_reviews
WHERE campaign_id = c.id AND user_id <> 4)
) +
(SELECT COUNT(id)
FROM user_reviews
WHERE campaign_id = c.id
) ;
For the outer query and joins: campaign(status, is_deleted, id, amt_req) and campaign_category(id) (you should have the latter if it is defined as a primary key.
Then: user_reviews(user_id, campaign_id), reserved_reviews(campaign_id, user_id), and user_reviews(campaign_id).
I'm concerned about the performance of the query below once the tables are fully populated. So far it's under development and performs well with dummy data.
The table "adress_zoo" will contain about 500 million records once fully populated. "adress_zoo" table looks like this:
CREATE TABLE `adress_zoo`
( `adress_id` int(11) NOT NULL, `zoo_id` int(11) NOT NULL,
UNIQUE KEY `pk` (`adress_id`,`zoo_id`),
KEY `adress_id` (`adress_id`) )
ENGINE=InnoDB DEFAULT CHARSET=latin1;
The other tables will contain maximum 500 records each.
The full query looks like this:
SELECT a.* FROM jos_zoo_item AS a
JOIN jos_zoo_search_index AS zsi2 ON zsi2.item_id = a.id
WHERE a.id IN (
SELECT r.id FROM (
SELECT zi.id AS id, Max(zi.priority) as prio
FROM jos_zoo_item AS zi
JOIN jos_zoo_search_index AS zsi ON zsi.item_id = zi.id
LEFT JOIN jos_zoo_tag AS zt ON zt.item_id = zi.id
JOIN jos_zoo_category_item AS zci ON zci.item_id = zi.id
**JOIN adress_zoo AS az ON az.zoo_id = zi.id**
WHERE 1=1
AND ( (zci.category_id != 0 AND ( zt.name != 'prolong' OR zt.name is NULL))
OR (zci.category_id = 0 AND zt.name = 'prolong') )
AND zi.type = 'telefoni'
AND zsi.element_id = '44d3b1fd-40f6-4fd7-9444-7e11643e2cef'
AND zsi.value = 'Small'
AND zci.category_id > 15
**AND az.adress_id = 5**
GROUP BY zci.category_id ) AS r
)
AND a.application_id = 6
AND a.access IN (1,1)
AND a.state = 1
AND (a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <= '2012-06-07 07:51:26')
AND (a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >= '2012-06-07 07:51:26')
AND zsi2.element_id = '1c3cd26e-666d-4f8f-a465-b74fffb4cb14'
GROUP BY a.id
ORDER BY zsi2.value ASC
The query will usually return about 25 records.
Based on your experience, will this query perform acceptable (respond within say 3 seconds)?
What can I do to optimise this?
As adviced by #Jack I ran the query with EXPLAIN and got this:
This part is an important limiter:
az.adress_id = 5
MySQL will limit the table to only those records where adress_id matches before joining it with the rest of the statement, so it will depend on how big you think that result set might be.
Btw, you have a UNIQUE(adress_id, zoo_id) and a separate INDEX. Is there a particular reason? Because the first part of a spanning key can be used by MySQL to select with as well.
What's also important is to use EXPLAIN to understand how MySQL will "attack" your query and return the results. See also: http://dev.mysql.com/doc/refman/5.5/en/execution-plan-information.html
To avoid subquery you can try to rewrite your query as:
SELECT a.* FROM jos_zoo_item AS a
JOIN jos_zoo_search_index AS zsi2 ON zsi2.item_id = a.id
INNER JOIN
(
SELECT ** distinct ** r.id FROM (
SELECT zi.id AS id, Max(zi.priority) as prio
FROM jos_zoo_item AS zi
JOIN jos_zoo_search_index AS zsi ON zsi.item_id = zi.id
LEFT JOIN jos_zoo_tag AS zt ON zt.item_id = zi.id
JOIN jos_zoo_category_item AS zci ON zci.item_id = zi.id
**JOIN adress_zoo AS az ON az.zoo_id = zi.id**
WHERE 1=1
AND ( (zci.category_id != 0 AND ( zt.name != 'prolong' OR zt.name is NULL))
OR (zci.category_id = 0 AND zt.name = 'prolong') )
AND zi.type = 'telefoni'
AND zsi.element_id = '44d3b1fd-40f6-4fd7-9444-7e11643e2cef'
AND zsi.value = 'Small'
AND zci.category_id > 15
**AND az.adress_id = 5**
GROUP BY zci.category_id ) AS r
) T
on a.id = T.id
where
AND a.application_id = 6
AND a.access IN (1,1)
AND a.state = 1
AND (a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <= '2012-06-07 07:51:26')
AND (a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >= '2012-06-07 07:51:26')
AND zsi2.element_id = '1c3cd26e-666d-4f8f-a465-b74fffb4cb14'
GROUP BY a.id
ORDER BY zsi2.value ASC
This approach don't perform subquery for each candidate row. Performance may be increased only if T is calculated in few milliseconds.
HI im running socialengine3 and need optimization on the a custom Mutual friends query.
Its currently taking 15 seconds to execute
Friends table
friend_id
friend_user_id1
friend_user_id2
friend_status
friend_type
users
user_id
Edited
I have converted in into exists and still its now executing in 20 seconds.
below is the updated query.
SELECT friendlist.friend_user_id2, se_users.username, se_users.id, se_users.image, se_users.name, se_users.surname, count( * ) AS mutral_friends
FROM `se_friends` friendlist
INNER JOIN `users` se_users ON friendlist.friend_user_id2 = `se_users`.id
WHERE EXISTS (
SELECT se.friend_user_id2
FROM se_friends se
WHERE se.friend_user_id1 = '105012'
AND se.friend_status = '1'
AND se.friend_user_id2 = friendlist.friend_user_id1
) AND NOT EXISTS (
SELECT se1.friend_user_id2
FROM `se_friends` se1
WHERE se1.friend_user_id1 = '105012'
AND friendlist.friend_user_id2 = se1.friend_user_id2
)
AND NOT (
friendlist.friend_user_id2 = '105012'
)
AND friendlist.friend_status = '1'
GROUP BY friendlist.friend_user_id2, se_users.username, se_users.id, se_users.image, se_users.name, se_users.surname
ORDER BY mutral_friends DESC
LIMIT 0 , 20
Orignal query
SELECT DISTINCT `se_friends`.friend_user_id2, se_users.username, se_users.id, se_users.image, se_users.name, se_users.surname, count(*) as mutral_friends
FROM `se_friends`
INNER JOIN `users` se_users` ON `se_friends`.friend_user_id2=`se_users`.id
WHERE
(se_friends.friend_user_id1 <> '30355' or se_friends.friend_user_id2 <> '30355') AND
se_friends.friend_user_id1 IN
(SELECT se_friends.friend_user_id2
FROM `se_friends`
WHERE se_friends.friend_user_id1='".$user_id."' AND se_friends.friend_status='1')
AND `se_friends`.friend_user_id2 NOT IN
(SELECT se_friends.friend_user_id2
FROM `se_friends`
WHERE se_friends.friend_user_id1='".$user_id."'
)
AND NOT(se_friends.friend_user_id2='".$user_id."') AND se_friends.friend_status='1'
GROUP BY `se_friends`.friend_user_id2, se_users.username, se_users.id, se_users.image, se_users.name, se_users.surname
ORDER BY mutral_friends DESC
LIMIT 0, 20
IN is very expensive operation.
try to replace it with EXISTS. eg
select * from table where user_id in (select user_id from users where active='A')
and
select * from table t where exists (select user_id from users u where t.user_id = u.user_id and u.active='A')
if it won't be helpful, it's better to look at execution plan