I have following Mysql query
SELECT c.`id`
,c.`category_name`
,c.`category_type`
,c.bookmark_count
,f.category_id cat_id
,f.unfollow_at
,(
CASE WHEN c.id = f.follower_category_id
THEN (
SELECT count(`user_bookmarks`.`id`)
FROM `user_bookmarks`
WHERE (`user_bookmarks`.`category_id` = cat_id)
AND ((`f`.`unfollow_at` > `user_bookmarks`.`created_at`) || (`f`.`unfollow_at` = '0000-00-00 00:00:00'))
)
ELSE 0 END
) counter
,c.id
,f.follower_category_id follow_id
,c.user_id
FROM categories c
LEFT JOIN following_follower_categories f ON f.follower_category_id = c.id
WHERE c.user_id = 26
ORDER BY `category_name` ASC
and here is output what i am getting after execuation
now i just want to count . here i have field id having value 172 against it i have counter 30,3, 2 and Bookmark_count is 4( i need to include only once)
and i am accepting output for id 172 is 30+3+2+4(bookmark_count only once).
I am not sure how to do this.
Can anybody help me out
Thanks a lot
The following may be the most inefficient query for that purpose, but I added a cover to your query in order to hint at grouping the results.
(I removed the second c.id, and my example may have errors since I couldn't try it.)
SELECT `id`,
`category_name`,
`category_type`,
max(`bookmark_count`),
`cat_id`,
`unfollow_at`,
sum(`counter`)+max(`bookmark_count`) counter,
follow_id`, `user_id`
FROM
(SELECT c.`id`
,c.`category_name`
,c.`category_type`
,c.bookmark_count
,f.category_id cat_id
,f.unfollow_at
,(
CASE WHEN c.id = f.follower_category_id
THEN (
SELECT count(`user_bookmarks`.`id`)
FROM `user_bookmarks`
WHERE (`user_bookmarks`.`category_id` = cat_id)
AND ((`f`.`unfollow_at` > `user_bookmarks`.`created_at`) || (`f`.`unfollow_at` = '0000-00-00 00:00:00'))
)
ELSE 0 END
) counter
,f.follower_category_id follow_id
,c.user_id
FROM categories c
LEFT JOIN following_follower_categories f ON f.follower_category_id = c.id
WHERE c.user_id = 26)
GROUP BY `id`, `category_name`, `category_type`, `cat_id`, `unfollow_at`, `follow_id`, `user_id`
ORDER BY `category_name` ASC
Related
I have a relatively long query (posted below for reference).
I have tried to debug why the query is so slow (2 seconds), and I have finally found the cause.
In the end of the query I do:
ORDER BY
-- order by date
DATE(p.date) DESC,
-- order by followed people
CASE WHEN n.id IS NULL THEN '0' ELSE '1' END DESC -- this case takes the query from 20ms to 2 seconds
If I remove the CASE order by, it'll execute in around 20 ms.
Why is this?
When I run the query with EXPLAIN, I notice the CASE will add "Using temporary" into the Extra field.
See EXPLAIN queries below:
Explain query with CASE WHEN n.id IS NULL THEN '0' ELSE '1' END DESC in order by
Explain query without CASE WHEN n.id IS NULL THEN '0' ELSE '1' END DESC in order by
The full query (if it may help):
SELECT
-- feed type
'1' AS feed_type,
-- fetch post data
p.id,
p.receiver,
p.date,
p.message,
p.system_msg,
p.type AS post_type,
-- fetch author data
u.user_id,
u.firstname,
u.lastname,
u.type,
u.permalink,
av.file AS avatar_file,
-- fetch receiever data
u2.user_id AS receiver_user_id,
u2.firstname AS receiver_firstname,
u2.lastname AS receiver_lastname,
u2.permalink AS receiver_permalink,
u2.type AS receiver_type,
-- fetch post comment count
(
SELECT
COUNT(*)
FROM
edu_posts pc
WHERE
pc.comment = p.id
AND pc.deleted IS NULL
) as commentCount,
-- fetch post like count
(
SELECT
COUNT(*)
FROM
edu_likes l
WHERE
l.like_entity = p.id
) as likeCount,
-- user follow state
CASE WHEN n.id IS NOT NULL THEN '1' ELSE '0' END as is_following,
-- check if user likes post
CASE WHEN l.like_id IS NOT NULL THEN '1' ELSE '0' END as user_likes
FROM
edu_posts p
INNER JOIN -- author information
edu_users u ON u.user_id = p.author
LEFT JOIN -- author avatar
edu_avatars av ON av.fk = p.author
AND av.temp = 0
AND av.fk_type = 1
LEFT JOIN -- receiver information (if any)
edu_users u2 ON u2.user_id = p.receiver
LEFT JOIN -- check if author/receiver is followed by current user
edu_notification_list n ON n.user = 1
AND n.following = 1
AND (
n.fk = p.author
OR n.fk = p.receiver
)
AND (
(
n.type = 5
AND p.type = 3
)
OR (
n.type = 2
AND p.type = 1
)
)
LEFT JOIN -- check if user likes the post
edu_likes l ON l.like_entity = p.id
AND l.like_author = 1
WHERE
p.deleted IS NULL
AND p.comment IS NULL
AND (
p.id = p.comment
OR 1 = 1
)
AND (
n.id IS NOT NULL
OR p.system_msg = 0
)
ORDER BY
-- order by date
DATE(p.date) DESC,
-- order by followed people
CASE WHEN n.id IS NULL THEN '0' ELSE '1' END DESC
LIMIT
20 OFFSET 0
NOTE: Please let me know, if you would like to see any of the other tables.
You perform an operation that can't make use of indexes. Instead try
ORDER BY DATE(p.date) DESC,
n.id IS NULL ASC
I have a strange problem with mysql sum() result.
SELECT `users_limits`.times, SUM( l.times ) AS `result`
FROM `users_limits`
INNER JOIN `users_limits` AS `l`
INNER JOIN `vacation_types` AS `v` ON l.id_vacation_type = v.id_vacation_type
WHERE l.year =2014 AND v.type =0
This query give me result:
times = 10;
result = 30;
But should give me result 10 too, because I have only one record in my db, with these conditions.
you need GROUP BY . try add this
GROUP BY l.times
like that
WHERE l.year =2014 AND v.type =0
GROUP BY l.times
EDIT-:
SELECT `users_limits`.times, SUM( l.times ) AS `result`
FROM `users_limits` l
INNER JOIN `vacation_types` AS `v` ON l.id_vacation_type = v.id_vacation_type
WHERE l.year =2014 AND v.type =0
GROUP BY l.times
I have a sql query that is running really slow and I am perplexed as to why. The query is:
SELECT DISTINCT(c.ID),c.* FROM `content` c
LEFT JOIN `content_meta` cm1 ON c.id = cm1.content_id
WHERE 1=1
AND c.site_id IN (14)
AND c.type IN ('a','t')
AND c.status = 'visible'
AND (c.lock = 0 OR c.site_id = 14)
AND c.level = 0
OR
(
( c.site_id = 14
AND cm1.meta_key = 'g_id'
AND cm1.meta_value IN ('12','13','7')
)
OR
( c.status = 'visible'
AND (
(c.type = 'topic' AND c.parent_id IN (628,633,624))
)
)
)
ORDER BY c.date_updated DESC LIMIT 20
The content table has about 1250 rows and the content meta table has about 3000 rows. This isn't a lot of data and I'm not quite sure what may be causing it to run so slow. Any thoughts/opinions would be greatly appreciated.
Thanks!
Is you where clause correct? You are making a series of AND statements and later you are executing a OR.
Wouldn't the correct be something like:
AND (c.lock = 0 OR c.site_id = 14)
AND (
( ... )
OR
( ... )
)
If it is indeed correct, you could think on changing the structure or treating the result in an script or procedure.
It might have to do with your "OR" clause at the end... your up-front are being utilized by the indexes where possible, but then you throw this huge OR condition at the end that can be either one or the other. Not knowing more of the underlying content, I would adjust to have a UNION on the inside so each entity can utilize its own indexes, get you qualified CIDs, THEN join to final results.
select
c2.*
from
( select distinct
c.ID
from
`content` c
where
c.site_id in (14)
and c.type in ('a', 't' )
and c.status = 'visible'
and c.lock in ( 0, 14 )
and c.level = 0
UNION
select
c.ID
from
`content` c
where
c.status = 'visible'
and c.type = 'topic'
and c.parent_id in ( 628, 633, 624 )
UNION
select
c.ID
from
`content` c
join `content_meta` cm1
on c.id = cm1.content_id
AND cm1.meta_key = 'g_id'
AND cm1.meta_value in ( '12', '13', '7' )
where
c.site_id = 14 ) PreQuery
JOIN `content` c2
on PreQuery.cID = c2.cID
order by
c2.date_updated desc
limit
20
I would ensure content table has an index on ( site_id, type, status ) another on (parent_id, type, status)
and the meta table, an index on ( content_id, meta_key, meta_value )
I'm concerned about the performance of the query below once the tables are fully populated. So far it's under development and performs well with dummy data.
The table "adress_zoo" will contain about 500 million records once fully populated. "adress_zoo" table looks like this:
CREATE TABLE `adress_zoo`
( `adress_id` int(11) NOT NULL, `zoo_id` int(11) NOT NULL,
UNIQUE KEY `pk` (`adress_id`,`zoo_id`),
KEY `adress_id` (`adress_id`) )
ENGINE=InnoDB DEFAULT CHARSET=latin1;
The other tables will contain maximum 500 records each.
The full query looks like this:
SELECT a.* FROM jos_zoo_item AS a
JOIN jos_zoo_search_index AS zsi2 ON zsi2.item_id = a.id
WHERE a.id IN (
SELECT r.id FROM (
SELECT zi.id AS id, Max(zi.priority) as prio
FROM jos_zoo_item AS zi
JOIN jos_zoo_search_index AS zsi ON zsi.item_id = zi.id
LEFT JOIN jos_zoo_tag AS zt ON zt.item_id = zi.id
JOIN jos_zoo_category_item AS zci ON zci.item_id = zi.id
**JOIN adress_zoo AS az ON az.zoo_id = zi.id**
WHERE 1=1
AND ( (zci.category_id != 0 AND ( zt.name != 'prolong' OR zt.name is NULL))
OR (zci.category_id = 0 AND zt.name = 'prolong') )
AND zi.type = 'telefoni'
AND zsi.element_id = '44d3b1fd-40f6-4fd7-9444-7e11643e2cef'
AND zsi.value = 'Small'
AND zci.category_id > 15
**AND az.adress_id = 5**
GROUP BY zci.category_id ) AS r
)
AND a.application_id = 6
AND a.access IN (1,1)
AND a.state = 1
AND (a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <= '2012-06-07 07:51:26')
AND (a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >= '2012-06-07 07:51:26')
AND zsi2.element_id = '1c3cd26e-666d-4f8f-a465-b74fffb4cb14'
GROUP BY a.id
ORDER BY zsi2.value ASC
The query will usually return about 25 records.
Based on your experience, will this query perform acceptable (respond within say 3 seconds)?
What can I do to optimise this?
As adviced by #Jack I ran the query with EXPLAIN and got this:
This part is an important limiter:
az.adress_id = 5
MySQL will limit the table to only those records where adress_id matches before joining it with the rest of the statement, so it will depend on how big you think that result set might be.
Btw, you have a UNIQUE(adress_id, zoo_id) and a separate INDEX. Is there a particular reason? Because the first part of a spanning key can be used by MySQL to select with as well.
What's also important is to use EXPLAIN to understand how MySQL will "attack" your query and return the results. See also: http://dev.mysql.com/doc/refman/5.5/en/execution-plan-information.html
To avoid subquery you can try to rewrite your query as:
SELECT a.* FROM jos_zoo_item AS a
JOIN jos_zoo_search_index AS zsi2 ON zsi2.item_id = a.id
INNER JOIN
(
SELECT ** distinct ** r.id FROM (
SELECT zi.id AS id, Max(zi.priority) as prio
FROM jos_zoo_item AS zi
JOIN jos_zoo_search_index AS zsi ON zsi.item_id = zi.id
LEFT JOIN jos_zoo_tag AS zt ON zt.item_id = zi.id
JOIN jos_zoo_category_item AS zci ON zci.item_id = zi.id
**JOIN adress_zoo AS az ON az.zoo_id = zi.id**
WHERE 1=1
AND ( (zci.category_id != 0 AND ( zt.name != 'prolong' OR zt.name is NULL))
OR (zci.category_id = 0 AND zt.name = 'prolong') )
AND zi.type = 'telefoni'
AND zsi.element_id = '44d3b1fd-40f6-4fd7-9444-7e11643e2cef'
AND zsi.value = 'Small'
AND zci.category_id > 15
**AND az.adress_id = 5**
GROUP BY zci.category_id ) AS r
) T
on a.id = T.id
where
AND a.application_id = 6
AND a.access IN (1,1)
AND a.state = 1
AND (a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <= '2012-06-07 07:51:26')
AND (a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >= '2012-06-07 07:51:26')
AND zsi2.element_id = '1c3cd26e-666d-4f8f-a465-b74fffb4cb14'
GROUP BY a.id
ORDER BY zsi2.value ASC
This approach don't perform subquery for each candidate row. Performance may be increased only if T is calculated in few milliseconds.
Can you please help me optimize this query. I use this query to get a list of friends along with their details and their status.
It takes about 0.08 secs to process this on a Athlon X2 6000
I cant use materizlized view as well because this is frequently changing.
SELECT p.userid, p.firstname, p.lastname, p.gender, p.dob, x.relationship,
IF(p.picture !=1,
IF(p.gender != 'm','/sc/f-t.jpg','/sc/m-t.jpg'),
concat('/sc/pthumb/', p.userid, '.jpg' )) AS picture
FROM `social` AS p
LEFT JOIN `friendlist` AS f1 ON (f1.`userid` = p.`userid` AND f1.`friendid` = 1 AND `f1`.`status` = 1)
LEFT JOIN `friendlist` AS f2 ON (f2.`friendid` = p.`userid` AND f2.`userid` = 1 AND `f2`.`status` = 1)
LEFT JOIN `x_relationship` AS x ON (x.`id` = p.`relationship`)
LEFT JOIN `auth` as a ON (a.`userid` = p.`userid`)
WHERE 1 AND (f1.`userid` IS NOT NULL OR f2.`userid` IS NOT NULL AND ((a.`banned` != 1 AND a.`deleted` != 1)))
ORDER BY RAND() LIMIT 0,10
This is your original query, formatted. Below it are a few thoughts.
SELECT
p.userid,
p.firstname,
p.lastname,
p.gender,
p.dob,
x.relationship,
IF(p.picture !=1, IF(p.gender != 'm', '/sc/f-t.jpg', '/sc/m-t.jpg'), concat('/sc/pthumb/', p.userid, '.jpg' )) AS picture
FROM
`social` AS p
LEFT JOIN `friendlist` AS f1 ON (f1.`friendid` = 1 AND f1.`userid` = p.`userid` AND `f1`.`status` = 1)
LEFT JOIN `friendlist` AS f2 ON (f2.`friendid` = p.`userid` AND f2.`userid` = 1 AND `f2`.`status` = 1)
LEFT JOIN `x_relationship` AS x ON (x.`id` = p.`relationship`)
LEFT JOIN `auth` AS a ON (a.`userid` = p.`userid`)
WHERE
1
AND (
f1.`userid` IS NOT NULL
OR f2.`userid` IS NOT NULL
AND (a.`banned` != 1 AND a.`deleted` != 1)
)
ORDER BY
RAND()
LIMIT
0,10
Think if some of your left joins could be inner joins. If so, change them.
Your WHERE clause is messed up. You are mixing AND and OR without properly prioritizing with parentheses. Try:
WHERE
a.`banned` != 1
AND a.`deleted` != 1
AND (
f1.`userid` IS NOT NULL
OR f2.`userid` IS NOT NULL
)
You could also try:
INNER JOIN `auth` AS a ON (
a.`userid` = p.`userid`
AND a.`banned` != 1
AND a.`deleted` != 1
)
WHERE
f1.`userid` IS NOT NULL
OR f2.`userid` IS NOT NULL
You seem to be joining against friendlist solely to check for record existence. You might want to give this a try as well:
FROM
`social` AS p
INNER JOIN `auth` AS a ON (
a.`userid` = p.`userid`
AND a.`banned` != 1
AND a.`deleted` != 1
)
LEFT JOIN `x_relationship` AS x ON (
x.`id` = p.`relationship`
)
WHERE
EXISTS (
SELECT 1 FROM `friendlist` WHERE `friendid` = 1 AND `userid` = p.`userid` AND `status` = 1
)
OR EXISTS (
SELECT 1 FROM `friendlist` WHERE `friendid` = p.`userid` AND `userid` = 1 AND `status` = 1
)
ORDER BY RAND() might not be the fastest thing on earth, but if this is what you need... Try ordering by a indexed column to see how much impact ORDER BY RAND() has.
Indexing:
You should create a composite index on friendlist(friendid, userid, status).
Make sure there is an index on relationship(id)
Make sure there is an index on auth(userid)