How to customize this query, It takes around 30 Second to take out results, total records in 'videos' table are about 0.5 million, 3 million members are present in 'Members' Table, is there any alternate Query or should i break this query in 2 select queries ?
user_id is Indexed
vid_id is Indexed
select a.ref_url , a.source , a.video_name , a.viewers , b.username ,
c.points from
members_videos a
inner join Members b on a.user_id = b.user_id
inner join rankings c on c.user_id = b.user_id
where a.cat_ids in (123,234,52,234,423,122) not in (110,99)
order by a.vid_id Desc limit 10
There are multiple factors:
Make sure members_videos, Members and rankings have index on user_id column.
Break the query; eliminate Order By clause. Get the data in temporary table.
Select the data from temporary table and put Order By Clause over there.
Alternate, try sub-query.
SELECT *
FROM
(
select a.ref_url , a.source , a.video_name , a.viewers , b.username ,
c.points from
members_videos a
inner join Members b on a.user_id = b.user_id
inner join rankings c on c.user_id = b.user_id
where a.cat_ids in (123,234,52,234,423,122) not in (110,99)
) as T
order by T.vid_id Desc limit 10
Related
How do I improve performance this query while also get all the information needed..
SELECT
tr.id, tr.request_status, tr.note, tr.created_date,
c.name AS customer_name, c.mobile_phONe,
u.full_name AS created_by_name, tt.name AS ticket_type_name
FROM
ticket_request tr
LEFT JOIN
ticket_type tt ON tt.id = tr.ticket_type_id
LEFT JOIN
users u ON u.id = tr.created_by
LEFT JOIN
customer c ON c.id = tr.customer_id
WHERE
tr.is_deleted != 1
AND tr.user_id IN (SELECT u.id FROM users u WHERE u.status = '1')
GROUP BY
tr.id
ORDER BY
tr.created_date DESC
LIMIT 0,20
Currently, this query runs in 7-10 seconds.
ticket_request table has about 100k rows
customers table has about 300k rows
users table and ticket_type don't have that much (about 1k rows)
The speedup technique below is to dispense with the LIMIT first, and only after that, do all the JOINs.
SELECT tr3.id, tr3.request_status, tr3.note, tr3.created_date,
c.name AS customer_name, c.mobile_phONe,
u2.full_name AS created_by_name,
tt.name AS ticket_type_name
FROM
(
SELECT tr1.id
FROM ticket_request tr1
JOIN users u1 ON u1.id = tr1.created_by
WHERE u1.status = '1'
AND tr1.is_deleted != 1
ORDER BY tr1.created_date DESC
LIMIT 0,20
) AS tr2
JOIN ticket_request AS tr3 ON tr3.id = tr2.id
JOIN user AS u2 ON u2.id = tr3.created_by
LEFT JOIN ticket_type tt ON tt.id = tr3.ticket_type_id
LEFT JOIN customer c ON c.id = tr3.customer_id
ORDER BY tr3.created_date
The JOINs, after the one in the "derived" table tr2, are touching only 20 rows; this is much of the speedup.
This may be equally good:
SELECT d.id, d.request_status, d.note, d.created_date,
c.name AS customer_name, c.mobile_phONe, d.created_by_name,
tt.name AS ticket_type_name
FROM
(
SELECT tr.id AS tr_id, tr.request_status, tr.note, tr.created_date,
tr.ticket_type_id, tr.customer_id
u.full_name AS created_by_name
FROM ticket_request tr
JOIN users u ON u.id = tr.created_by
WHERE u.status = '1'
AND tr.is_deleted != 1
ORDER BY tr.created_date DESC
LIMIT 0,20
) AS d
LEFT JOIN ticket_type tt ON tt.id = d.ticket_type_id
LEFT JOIN customer c ON c.id = d.customer_id
ORDER BY d.created_date
I'm assuming that you are using MySQL. If not, this answer can be slightly modified to fit another database, but the concept should remain the same. You can add indices to all the ID columns which are involved in the right hand side of your left joins with the ticket_request column:
ALTER TABLE ticket_type ADD INDEX (id);
ALTER TABLE users ADD INDEX (id);
ALTER TABLE customer ADD INDEX (id); -- important
To explain why an index would help, consider the first LEFT JOIN between your ticket_request table and the ticket_type table. Without an index, for each record in ticket_request the database would have to potentially scan the entire ticket_type table to find records which match the join condition. This is costly from a performance point of view. But with an index, the database can complete this operation much faster, since it "knows" where to look exactly (or almost exactly) for the matching records.
Though you mentioned that only the customer table is very large, you can still add indices to the other tables. In the future, they might get larger too. Most likely the join involving customer is the bottleneck in your query.
SELECT
tr.id, tr.request_status, tr.note, tr.created_date,
c.name AS customer_name, c.mobile_phONe,
u.full_name AS created_by_name, tt.name AS ticket_type_name
FROM
ticket_request tr
LEFT JOIN
ticket_type tt ON tt.id = tr.ticket_type_id and tr.is_deleted != 1
LEFT JOIN
users u ON u.id = tr.created_by
JOIN
users u1 ON u1.id = tr.user_id and u1.status = '1'
LEFT JOIN
customer c ON c.id = tr.customer_id
GROUP BY
tr.id
ORDER BY
tr.created_date DESC
LIMIT 0,20
try this it will work with improved performance and tweak as per your requirement
The biggest opportunity for optimization here is with LIMIT 0,20
GROUP BY tr.id make no sense and should be removed.
create index ticket_request_ix_is_deleted_created_date on ticket_request (is_deleted,created_date) and change tr.is_deleted != 1 to tr.is_deleted = 0.
Or
create index ticket_request_ix_created_date on ticket_request (created_date)
Other than indexing , On application level you can use Memcached ( in case you are using php) like stuffs. This will also give you great performance.
Hi I like to make a perfomant order by query.
This is my query:
SELECT a.status,a.title,a.id,
b.deployed,b.accepted,b.rejected,b.error
FROM a
LEFT JOIN b ON ( a.id = b.id )
WHERE b.agencyid = 1 AND a.userid = 3
ORDER BY a.date DESC
LIMIT 0,50
without the ORDER BY clause the query takes 0.006 s
with the ORDER BY clause it needs 1.1181 s !!!
The most time consumption comes from Copying To Tmp Table 1,1 s 99,78% 1 1,1 s
How I can decrease the Copy to tpm table time?
First, as noted by Strawberry, because you have the "b" alias referenced in the where clause, and not considering NULL, it in essence turns your left join to an INNER JOIN.
SELECT
a.status,
a.title,
a.id,
b.deployed,
b.accepted,
b.rejected,
b.error
FROM a JOIN b
ON a.id = b.id
WHERE
a.userid = 3
AND b.agencyid = 1
ORDER BY
a.date DESC
LIMIT
0,50
If you meant for it to be a LEFT-JOIN (all "a" records regardless of a match in the "b" table), then you would need to move your "b" qualifier to the JOIN clause.
SELECT
a.status,
a.title,
a.id,
b.deployed,
b.accepted,
b.rejected,
b.error
FROM a LEFT JOIN b
ON a.id = b.id
AND b.agencyid = 1
WHERE
a.userid = 3
ORDER BY
a.date DESC
LIMIT
0,50
That being said, you are looking for a specific user ordered by date. I would have your "a" table indexed on (userid, date) so the engine can be optimized on your WHERE qualifier AND the order by and should resolve your performance issue regardless of left and inner join...
To help the join condition to your "b" table, index it on (id, agencyid)
I have a SQL query which does most of what I need it to do but I'm running into a problem.
There are 3 tables in total. entries, entry_meta and votes.
I need to get an entire row from entries when competition_id = 420 in the entry_meta table and the ID either doesn't exist in votes or it does exist but the user_id column value isn't 1.
Here's the query I'm using:
SELECT entries.* FROM entries
INNER JOIN entry_meta ON (entries.ID = entry_meta.entry_id)
WHERE 1=1
AND ( ( entry_meta.meta_key = 'competition_id' AND CAST(entry_meta.meta_value AS CHAR) = '420') )
GROUP BY entries.ID
ORDER BY entries.submission_date DESC
LIMIT 0, 25;
The votes table has 4 columns. vote_id, entry_id, user_id, value.
One option I was thinking of was to SELECT entry_id FROM votes WHERE user_id = 1 and include it in an AND clause in my query. Is this acceptable/efficient?
E.g.
AND entries.ID NOT IN (SELECT entry_id FROM votes WHERE user_id = 1)
A left join with an appropriate where clause might be useful:
SELECT
entries.*
FROM
entries
INNER JOIN entry_meta ON (entries.ID = entry_meta.entry_id)
LEFT JOIN votes ON entries.ID = votes.entry_id
WHERE 1=1
AND (
entry_meta.meta_key = 'competition_id'
AND CAST(entry_meta.meta_value AS CHAR) = '420')
AND votes.entry_id IS NULL -- This will remove any entry with votes
)
GROUP BY entries.ID
ORDER BY entries.submission_date DESC
Here's an implementation of Andrew's suggestion to use exists / not exists.
select
e.*
from
entries e
join entry_meta em on e.ID = em.entry_id
where
em.meta_key = 'competition_id'
and cast(em.meta_value as char) = '420'
and (
not exists (
select 1
from votes v
where
v.entry_id = e.ID
)
or exists (
select 1
from votes v
where
v.entry_id = e.ID
and v.user_id != 1
)
)
group by e.ID
order by e.submission_date desc
limit 0, 25;
Note: it's generally not a good idea to put a function inside a where clause (due to performance reasons), but since you're also joining on IDs you should be OK.
Also, The left join suggestion by Barranka may cause the query to return more rows than your are expecting (assuming that there is a 1:many relationship between entries and votes).
I have three tables - tblpollquestions, tblpollanswers and tblpollresponses.
I want to select a random question that a user hasn't responded to yet, with the respective answers.
The SQL below returns exactly what I need, but I'm concerned that it takes three SELECTs to do it. There must surely be a more efficient way?
SELECT
poll.id,
poll.question,
a.answer
FROM tblpollquestions poll
INNER JOIN tblpollanswers a ON a.question_id = poll.id
INNER JOIN (
SELECT id FROM tblpollquestions WHERE id NOT IN(
SELECT question_id FROM tblpollresponses WHERE user_id = 1
) ORDER BY RAND() LIMIT 1
) as t ON t.id = poll.id
This could be made a bit better by switching NOT IN(SELECT...) into LEFT JOIN
SELECT
poll.id,
poll.question,
a.answer
FROM
tblpollquestions poll
INNER JOIN
tblpollanswers a
ON
a.question_id = poll.id
INNER JOIN (
SELECT
q.id
FROM
tblpollquestions AS q
LEFT JOIN
tblpollresponses AS r
ON
q.id = r.question_id
AND r.user_id = 1
WHERE
r.question_id IS NULL
ORDER BY RAND() LIMIT 1
) as t ON t.id = poll.id
ORDER BY RAND() can also be slow if there are many rows in tblpollquestions table. See this presentation from Bill Karwin (slide 142 and onwards) for some other ideas on selecting a random row.
http://www.slideshare.net/billkarwin/sql-antipatterns-strike-back
Is seems fine to me, although I would change it slightly:
SELECT
poll.id,
poll.question,
a.answer
FROM tblpollquestions poll
INNER JOIN tblpollanswers a ON a.question_id = poll.id
WHERE poll.id = (
SELECT id FROM tblpollquestions WHERE NOT EXISTS (
SELECT * FROM tblpollresponses WHERE user_id = 1 AND question_id = tblpollquestions.id )
ORDER BY RAND() LIMIT 1)
Written that way should do a better job of using indexes, and not checking the join conditions for every single tblpollanswers.
Make sure you have a UNIQUE index (or primary key) on tblpollresponses for (user_id, question_id) (in that order). If you need it for other queries, you can add an additional UNIQUE index with the columns in the reverse order.
Edit: Actually putting it in the where might not be so good http://jan.kneschke.de/projects/mysql/order-by-rand/ You will need to explain the query and compare.
Use left join like this:
SELECT ques.id, ques.question, ans.answer FROM tblpollquestions ques
INNER JOIN tblpollanswers ans ON(ans.question_id = ques.id)
left join tblpollresponses res on(res.question_id=ques.id and user_id = 1)
where res.question_id is null ORDER BY RAND() LIMIT 1;
I changed your table aliases to make better sense.
I'm trying to do a select statement and it works except that it's not limiting the number of results for each user (U.id) to 5.
SELECT F.id,F.created,U.username,U.fullname,U.id,I.id,I.cached_image
FROM favorites AS F
INNER JOIN users AS U
ON F.faver_profile_id = U.id
INNER JOIN items AS I
ON F.notice_id = I.id
WHERE faver_profile_id IN ('.$users.')
GROUP BY I.id HAVING COUNT(U.id) <= 5
ORDER BY F.faver_profile_id, F.created DESC
I'm grouping by I.id to eliminate duplicates. From my research it looks like you can only use HAVING COUNT if your also grouping by that column, but I cannot group by U.id or I'd lose results rows.
Instead of HAVING, can you slap a LIMIT 5 in there?
Edit: OP cannot LIMIT entire query,
and, AFAIK, MySQL does not support LIMIT in subqueries,
so you can create a temporary table with your five (5) user ids:
create table temp_table ( id INT );
insert into temp_table (id) SELECT U.id FROM users U LIMIT 5;
SELECT F.id,F.created,U.username,U.fullname,U.id,I.id,I.cached_image
FROM favorites AS F
INNER JOIN temp_table AS Ut
ON F.faver_profile_id = Ut.id
INNER JOIN items AS I
ON F.notice_id = I.id
WHERE faver_profile_id IN ('.$users.')
GROUP BY I.id
ORDER BY F.faver_profile_id, F.created DESC;
drop table temp_Table;
Let us know how that works.