I have used below query for product listing. Query is working fine but it takes approximately 0.4534 seconds. How can I optimize the same query.
SELECT SQL_CALC_FOUND_ROWS DISTINCT tp.prod_id, tp.prod_name, tp.prod_shop, tp.prod_retail_price, tp.prod_sale_price, tp.prod_initial_price, tp.prod_stock, ts.shop_id, ts.shop_name, ts.shop_logo, ts.shop_description, ts.shop_title, tu.user_profile_image, ( SELECT pdiscount_price FROM tbl_product_discounts tpd WHERE tpd.pdiscount_product_id = tp.prod_id AND tpd.pdiscount_qty = '1' AND( ( tpd.pdiscount_start_date = '0000-00-00' OR tpd.pdiscount_start_date < NOW()) AND( tpd.pdiscount_end_date = '0000-00-00' OR tpd.pdiscount_end_date > NOW()) ) ORDER BY tpd.pdiscount_priority ASC, tpd.pdiscount_price ASC LIMIT 1 ) AS discount FROM tbl_products tp LEFT JOIN tbl_shops ts ON tp.prod_shop = ts.shop_id AND ts.shop_is_deleted = 0 INNER JOIN tbl_users tu ON ts.shop_user_id = tu.user_id WHERE tp.prod_is_deleted = '0' LIMIT 0, 20
Without checking you table or Requirement :
Try to use group by instead of DISTINCT
Do not use sub query if possible .
Try To use indexing in you table .
This will help you to optimize you query.
Related
I have a query where I do a join on a substring, the problem is that this is really slow to complete. Is there a more effecient way to write this?
SELECT *, SUM(s.pris*s.antall) AS total, SUM(s.antall) AS antall
FROM ecs_statistikk AS s
JOIN butikk_ordre AS bo ON ordreId=bo.ecs_ordre_id AND butikkNr=bo.site_id
JOIN ecs_supplier AS l ON SUBSTRING( s.artikkelId, 1,2 )=l.lev_id
WHERE s.salgsDato>='2016-6-01' AND s.salgsDato<='2016-09-30'
GROUP BY l.lev_id ORDER BY total DESC
First, I would check indexes. For this query:
SELECT *, SUM(s.pris*s.antall) AS total, SUM(s.antall) AS antall
FROM ecs_statistikk s JOIN
butikk_ordre bo
ON s.ordreId = bo.ecs_ordre_id AND
s.butikkNr = bo.site_id JOIN
ecs_supplier l
ON SUBSTRING(s.artikkelId, 1, 2 ) = l.lev_id
WHERE s.salgsDato >= '2016-06-01' AND s.salgsDato <= '2016-09-30'
GROUP BY l.lev_id
ORDER BY total DESC ;
You want indexes on ecs_statistikk(salgsDato, ordreId, butikkNr, artikkelId), butikk_ordre(ecs_ordre_id, site_id), and ecs_supplier(lev_id)`.
Next, I would question whether you need the last JOIN at all. Does this do what you want?
SELECT LEFT(s.artikkelId, 2) as lev_id, *,
SUM(s.pris*s.antall) AS total, SUM(s.antall) AS antall
FROM ecs_statistikk s JOIN
butikk_ordre bo
ON s.ordreId = bo.ecs_ordre_id AND
s.butikkNr = bo.site_id
WHERE s.salgsDato >= '2016-06-01' AND s.salgsDato <= '2016-09-30'
GROUP BY LEFT(s.artikkelId, 2)
ORDER BY total DESC ;
I have the query below, and it gets the correct data, but I need to limit the results of the join so that only the first row from the photos table is included. Here's what I have:
SELECT `listings`.*, `photos`.*
FROM `listings` JOIN `photos`
ON `photos`.`listing_id` = `listings`.`id`
WHERE `listings`.`date_expires` >= '2016-08-10' AND `listings`.`visible` = '1' AND `listings`.`spotlightListing` >= '2016-08-10'
How do I limit the above so I only get the first row from the photos table? Thanks!
SELECT `listings`.*, `photos`.*
FROM `listings`
JOIN `photos` ON `photos`.`listing_id` = `listings`.`id`
WHERE `listings`.`date_expires` >= '2016-08-10' AND `listings`.`visible` = '1'
AND `listings`.`spotlightListing` >= '2016-08-10'
limit 1
I am sure you are talking about first photo of the list record.
I want to speed up one of my slower queries.
The problem is that I can't access the outer colum value within a subquery.
What I have:
SELECT r.id AS room_id, r.room_name, coalesce(d.score,0) AS total_messages, d.latest
FROM cf_rooms_time_frames tf
INNER JOIN cf_rooms r on r.id = tf.room_id
INNER JOIN(
SELECT cf.room_id, count(*) as score, max(cf.id) as latest
FROM cf_rooms_messages cf
WHERE EXISTS(
SELECT NULL FROM cf_rooms_time_frames tf
WHERE tf.start <= cf.id AND ( tf.end IS NULL OR tf.end >= cf.id )
AND tf.room_id = cf.room_id AND tf.uid = 8
)
GROUP BY cf.room_id
ORDER BY latest
DESC ) d on d.room_id = r.id
WHERE tf.uid = 8
ORDER BY coalesce(latest, score) DESC LIMIT 0, 20
What I want:
SELECT r.id AS room_id, r.room_name, coalesce(d.score,0) AS total_messages, d.latest
FROM cf_rooms_time_frames tf
INNER JOIN cf_rooms r on r.id = tf.room_id
INNER JOIN(
SELECT cf.room_id, count(*) as score, max(cf.id) as latest
FROM cf_rooms_messages cf
/* line added here */
WHERE cf.room_id = tf.room_id
/* */
AND EXISTS(
SELECT NULL FROM cf_rooms_time_frames tf
WHERE tf.start <= cf.id AND ( tf.end IS NULL OR tf.end >= cf.id )
AND tf.room_id = cf.room_id AND tf.uid = 8
)
GROUP BY cf.room_id
ORDER BY latest
DESC ) d on d.room_id = r.id
WHERE tf.uid = 8
ORDER BY coalesce(latest, score) DESC LIMIT 0, 20
I think the markup explains what the query does.
It searches for "chatrooms" for a given user and orders them by the last message, gets the number of total message which ids are in a given range ( timeframes ), and the last message id.
I don't know why, but the first query returns all rows within the chatmessage table ( cf ) if I can trust EXPLAIN. It delivers the correct results but is kind of slow on a huge table.
I tested the second one with a "hardcoded" room_id and this one was very fast and doesn't "touched" the whole table.
In relation to the answer I accepted for this post, SQL Group By and Limit issue, I need to figure out how to create that query using SQLAlchemy. For reference, the query I need to run is:
SELECT t.id, t.creation_time, c.id, c.creation_time
FROM (SELECT id, creation_time
FROM thread
ORDER BY creation_time DESC
LIMIT 5
) t
LEFT OUTER JOIN comment c ON c.thread_id = t.id
WHERE 3 >= (SELECT COUNT(1)
FROM comment c2
WHERE c.thread_id = c2.thread_id
AND c.creation_time <= c2.creation_time
)
I have the first half of the query, but I am struggling with the syntax for the WHERE clause and how to combine it with the JOIN. Any one have any suggestions?
Thanks!
EDIT: First attempt seems to mess up around the .filter() call:
c = aliased(Comment)
c2 = aliased(Comment)
subq = db.session.query(Thread.id).filter_by(topic_id=122098).order_by(Thread.creation_time.desc()).limit(2).offset(2).subquery('t')
subq2 = db.session.query(func.count(1).label("count")).filter(c.id==c2.id).subquery('z')
q = db.session.query(subq.c.id, c.id).outerjoin(c, c.thread_id==subq.c.id).filter(3 >= subq2.c.count)
this generates the following SQL:
SELECT t.id AS t_id, comment_1.id AS comment_1_id
FROM (SELECT count(1) AS count
FROM comment AS comment_1, comment AS comment_2
WHERE comment_1.id = comment_2.id) AS z, (SELECT thread.id AS id
FROM thread
WHERE thread.topic_id = :topic_id ORDER BY thread.creation_time DESC
LIMIT 2 OFFSET 2) AS t LEFT OUTER JOIN comment AS comment_1 ON comment_1.thread_id = t.id
WHERE z.count <= 3
Notice the sub-query ordering is incorrect, and subq2 somehow is selecting from comment twice. Manually fixing that gives the right results, I am just unsure of how to get SQLAlchemy to get it right.
Try this:
c = db.aliased(Comment, name='c')
c2 = db.aliased(Comment, name='c2')
sq = (db.session
.query(Thread.id, Thread.creation_time)
.order_by(Thread.creation_time.desc())
.limit(5)
).subquery(name='t')
sq2 = (
db.session.query(db.func.count(1))
.select_from(c2)
.filter(c.thread_id == c2.thread_id)
.filter(c.creation_time <= c2.creation_time)
.correlate(c)
.as_scalar()
)
q = (db.session
.query(
sq.c.id, sq.c.creation_time,
c.id, c.creation_time,
)
.outerjoin(c, c.thread_id == sq.c.id)
.filter(3 >= sq2)
)
I'm concerned about the performance of the query below once the tables are fully populated. So far it's under development and performs well with dummy data.
The table "adress_zoo" will contain about 500 million records once fully populated. "adress_zoo" table looks like this:
CREATE TABLE `adress_zoo`
( `adress_id` int(11) NOT NULL, `zoo_id` int(11) NOT NULL,
UNIQUE KEY `pk` (`adress_id`,`zoo_id`),
KEY `adress_id` (`adress_id`) )
ENGINE=InnoDB DEFAULT CHARSET=latin1;
The other tables will contain maximum 500 records each.
The full query looks like this:
SELECT a.* FROM jos_zoo_item AS a
JOIN jos_zoo_search_index AS zsi2 ON zsi2.item_id = a.id
WHERE a.id IN (
SELECT r.id FROM (
SELECT zi.id AS id, Max(zi.priority) as prio
FROM jos_zoo_item AS zi
JOIN jos_zoo_search_index AS zsi ON zsi.item_id = zi.id
LEFT JOIN jos_zoo_tag AS zt ON zt.item_id = zi.id
JOIN jos_zoo_category_item AS zci ON zci.item_id = zi.id
**JOIN adress_zoo AS az ON az.zoo_id = zi.id**
WHERE 1=1
AND ( (zci.category_id != 0 AND ( zt.name != 'prolong' OR zt.name is NULL))
OR (zci.category_id = 0 AND zt.name = 'prolong') )
AND zi.type = 'telefoni'
AND zsi.element_id = '44d3b1fd-40f6-4fd7-9444-7e11643e2cef'
AND zsi.value = 'Small'
AND zci.category_id > 15
**AND az.adress_id = 5**
GROUP BY zci.category_id ) AS r
)
AND a.application_id = 6
AND a.access IN (1,1)
AND a.state = 1
AND (a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <= '2012-06-07 07:51:26')
AND (a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >= '2012-06-07 07:51:26')
AND zsi2.element_id = '1c3cd26e-666d-4f8f-a465-b74fffb4cb14'
GROUP BY a.id
ORDER BY zsi2.value ASC
The query will usually return about 25 records.
Based on your experience, will this query perform acceptable (respond within say 3 seconds)?
What can I do to optimise this?
As adviced by #Jack I ran the query with EXPLAIN and got this:
This part is an important limiter:
az.adress_id = 5
MySQL will limit the table to only those records where adress_id matches before joining it with the rest of the statement, so it will depend on how big you think that result set might be.
Btw, you have a UNIQUE(adress_id, zoo_id) and a separate INDEX. Is there a particular reason? Because the first part of a spanning key can be used by MySQL to select with as well.
What's also important is to use EXPLAIN to understand how MySQL will "attack" your query and return the results. See also: http://dev.mysql.com/doc/refman/5.5/en/execution-plan-information.html
To avoid subquery you can try to rewrite your query as:
SELECT a.* FROM jos_zoo_item AS a
JOIN jos_zoo_search_index AS zsi2 ON zsi2.item_id = a.id
INNER JOIN
(
SELECT ** distinct ** r.id FROM (
SELECT zi.id AS id, Max(zi.priority) as prio
FROM jos_zoo_item AS zi
JOIN jos_zoo_search_index AS zsi ON zsi.item_id = zi.id
LEFT JOIN jos_zoo_tag AS zt ON zt.item_id = zi.id
JOIN jos_zoo_category_item AS zci ON zci.item_id = zi.id
**JOIN adress_zoo AS az ON az.zoo_id = zi.id**
WHERE 1=1
AND ( (zci.category_id != 0 AND ( zt.name != 'prolong' OR zt.name is NULL))
OR (zci.category_id = 0 AND zt.name = 'prolong') )
AND zi.type = 'telefoni'
AND zsi.element_id = '44d3b1fd-40f6-4fd7-9444-7e11643e2cef'
AND zsi.value = 'Small'
AND zci.category_id > 15
**AND az.adress_id = 5**
GROUP BY zci.category_id ) AS r
) T
on a.id = T.id
where
AND a.application_id = 6
AND a.access IN (1,1)
AND a.state = 1
AND (a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <= '2012-06-07 07:51:26')
AND (a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >= '2012-06-07 07:51:26')
AND zsi2.element_id = '1c3cd26e-666d-4f8f-a465-b74fffb4cb14'
GROUP BY a.id
ORDER BY zsi2.value ASC
This approach don't perform subquery for each candidate row. Performance may be increased only if T is calculated in few milliseconds.