I have a query that fetches data from Six tables but it takes too much time to fetch data.The browser loads and shows sometimes nothing as a result.When I run this query in the MySQL database, it takes a long time to execute.
SELECT SQL_CALC_FOUND_ROWS movies.*,
curriculums.name AS curriculum,
teachers.name AS teacher,
movie_sub_categories.name AS sub_cat_name,
movie_categories.name AS cat_name
FROM movies
LEFT JOIN curriculums on movies.curriculum_id = curriculums.id
LEFT JOIN teachers on movies.teacher_id = teachers.id
LEFT JOIN movies_movie_sub_categories on movies.id = movies_movie_sub_categories.movie_id
LEFT JOIN movie_sub_categories on movies_movie_sub_categories.movie_sub_category_id = movie_sub_categories.id
LEFT JOIN movie_categories on movie_sub_categories.movie_category_id = movie_categories.id
ORDER BY id LIMIT 0, 50
Here all of my table structure
That's not a very exciting query -- it simply delivers the first 50 rows of whichever table id belongs to. When JOINing, please qualify columns so we know what is going on.
Do you really need LEFT?
Assuming you need LEFT and id belongs to movies, then this should run a lot faster:
Meanwhile, find how many rows there are in movies only once, so you don't have to compute it every time.
SELECT movies.*, curriculums.name AS curriculum,
teachers.name AS teacher, movie_sub_categories.name AS sub_cat_name,
movie_categories.name AS cat_name
FROM ( SELECT id FROM movies ORDER BY id LIMIT 0, 50 ) AS m
JOIN movies USING(id)
LEFT JOIN curriculums AS c ON movies.curriculum_id = c.id
LEFT JOIN teachers AS t ON movies.teacher_id = t.id
LEFT JOIN movies_movie_sub_categories AS mmsc ON movies.id = mmsc.movie_id
LEFT JOIN movie_sub_categories AS msc ON mmsc.movie_sub_category_id = msc.id
LEFT JOIN movie_categories AS mc ON msc.movie_category_id = mc.id
ORDER BY m.id
Please use SHOW CREATE TABLE; we need to see if you have sufficient indexes, such as
mmsc: INDEX(movie_id)
the table movies_movie_sub_categories needs to have an index on movie_id and a separate index on movie_sub_category_id. Without those two indexes the query builder will be forced to scan every record twice (since the query has two separate join clauses that reference that table)
Related
I have 4 tables, album, artist, song_cover and song.
Am trying to join the 3 tables to the album table and include the total number of songs within each album.
The query I have so far returns results as expected but takes almost over a minute to execute.
SELECT frco_album.*,
COUNT(frco_song.song_id) AS TotalSongs,
artist_aka, artist_address,
cover_filename
FROM frco_album
LEFT JOIN frco_song ON frco_album.album_id =
frco_song.song_album_id
LEFT JOIN frco_artist ON frco_album.album_artist =
frco_artist.artist_id
LEFT JOIN frco_song_cover ON frco_album.album_cover_id =
frco_song_cover.cover_id
GROUP BY frco_album.album_id
ORDER BY album_month DESC LIMIT 0, 20;
When I get rid of the song table LEFT JOIN song ON album.album_id = song.song_album_id and COUNT(song.song_id) AS TotalSongs, the query executes fast, as expected.
What am I doing wrong here?
EDIT: I've edited the question to include the tables and changed the query to reflect the actual rows in the tables.
The left join will multiply rows and then you condense them back using group by. Assuming that there is one artist and cover per album I would try counting the songs inside the select clause:
SELECT album.*, artist_aka, artist_address, cover_filename, (
SELECT COUNT(*)
FROM songs
WHERE song.song_album_id = album.album_id
) AS TotalSongs
FROM album
LEFT JOIN artist ON album.album_artist_id = artist.artist_id
LEFT JOIN song_cover ON album.album_cover_id = song_cover.cover_id
ORDER BY album_plays DESC
LIMIT 0, 20
The query below is grabbing some information about a category of toys and showing the most recent sale price for three levels of condition (e.g., Brand New, Used, Refurbished). The price for each sale is almost always different. One other thing - the sales table row id's are not necessarily in chronological order, e.g., a toy with a sale id of 5 could have happened later than a toy with a sale id of 10).
This query works but is not performant. It runs in a manageable amount of time, usually about 1s. However, I need to add yet another left join to include some more data, which causes the query time to balloon up to about 9s, no bueno.
Here is the working but nonperformant query:
SELECT b.brand_name, t.toy_id, t.toy_name, t.toy_number, tt.toy_type_name, cp.catalog_product_id, s.date_sold, s.condition_id, s.sold_price FROM brands AS b
LEFT JOIN toys AS t ON t.brand_id = b.brand_id
JOIN toy_types AS tt ON t.toy_type_id = tt.toy_type_id
LEFT JOIN catalog_products AS cp ON cp.toy_id = t.toy_id
LEFT JOIN toy_category AS tc ON tc.toy_category_id = t.toy_category_id
LEFT JOIN (
SELECT date_sold, sold_price, catalog_product_id, condition_id
FROM sales
WHERE invalid = 0 AND condition_id <= 3
ORDER BY date_sold DESC
) AS s ON s.catalog_product_id = cp.catalog_product_id
WHERE tc.toy_category_id = 1
GROUP BY t.toy_id, s.condition_id
ORDER BY t.toy_id ASC, s.condition_id ASC
But like I said it's slow. The sales table has about 200k rows.
What I tried to do was create the subquery as a view, e.g.,
CREATE VIEW sales_view AS
SELECT date_sold, sold_price, catalog_product_id, condition_id
FROM sales
WHERE invalid = 0 AND condition_id <= 3
ORDER BY date_sold DESC
Then replace the subquery with the view, like
SELECT b.brand_name, t.toy_id, t.toy_name, t.toy_number, tt.toy_type_name, cp.catalog_product_id, s.date_sold, s.condition_id, s.sold_price FROM brands AS b
LEFT JOIN toys AS t ON t.brand_id = b.brand_id
JOIN toy_types AS tt ON t.toy_type_id = tt.toy_type_id
LEFT JOIN catalog_products AS cp ON cp.toy_id = t.toy_id
LEFT JOIN toy_category AS tc ON tc.toy_category_id = t.toy_category_id
LEFT JOIN sales_view AS s ON s.catalog_product_id = cp.catalog_product_id
WHERE tc.toy_category_id = 1
GROUP BY t.toy_id, s.condition_id
ORDER BY t.toy_id ASC, s.condition_id ASC
Unfortunately, this change causes the query to no longer grab the most recent sale, and the sales price it returns is no longer the most recent.
Why is it that the table view doesn't return the same result as the same select as a subquery?
After reading just about every top-n-per-group stackoverflow question and blog article I could find, getting a query that actually worked was fantastic. But now that I need to extend the query one more step I'm running into performance issues. If anybody wants to sidestep the above question and offer some ways to optimize the original query, I'm all ears!
Thanks for any and all help.
The solution to the subquery performance issue was to use the answer provided here: Groupwise maximum
I thought that this approach could only be used when querying a single table, but indeed it works even when you've joined many other tables. You just have to left join the same table twice using the s.date_sold < s2.date_sold join condition and make sure the where clause looks for the null value in the second table's id column.
I'm often on this site to find answers and I usualy find what I look for but this time it's really particular so I post.
I'm working on a code I didn't write and I'd like somebody to explain me in detail what does the following query mean :
SELECT count(*)
FROM ( SELECT Id_dormir
FROM (dormir INNER JOIN dormir_photo AS DP
ON Id_dormir = Id_dormir_photo) INNER JOIN communes as C ON
Commune_dormir = Id_commune
WHERE Type_dormir=3
GROUP BY Id_dormir, Commune_dormir ) AS T
I'l not a boss in mysql and all those inner join and as are not very clear (it's a query from a search engine.
If anyone can help, many thanks,
Laurent
The outer query count a number of rows, returned by inner query:
SELECT Id_dormir FROM dormir
INNER JOIN dormir_photo AS DP ON Id_dormir = Id_dormir_photo
INNER JOIN communes as C ON Commune_dormir = Id_commune
WHERE Type_dormir=3 GROUP BY Id_dormir, Commune_dormir
In this query author get information from 3 tables:
domir, domir_photo, but only rows, where Id_dormir_photo = Id_dormir and from communes rows, where Commune_dormir = Id_commune. Than he filter rows with Type_dormir=3 and groups this with Id_dormir, Commune_dormir to have only UNIQUE rows with Id_dormir, Commune_dormir.
Your query count rows, where unique pair of Id_dormir, Commune_dormir and isset records in dormir_photo and communes.
I recommend you to read about different types of JOIN operation, especially about INNER JOIN (standard JOIN) and LEFT JOIN, this types are most applicable.
While running this query:
SELECT
a.id,
pub.name AS publisher_name,
pc.name AS placement_name,
b.name AS banner_name,
a.lead_id,
a.partner_id,
a.type,
l.status,
s.correctness,
a.landing_page,
t.name AS tracker_name,
a.date_view,
a.date_action
FROM actions AS a
LEFT JOIN publishers AS pub ON a.publisher_id = pub.id
LEFT JOIN placements AS pc ON pc.publisher_id = pub.id
LEFT JOIN banners AS b ON b.campaign_id = a.campaign_id
LEFT JOIN leads l ON
l.lead_id = a.lead_id
AND l.created = (
SELECT MAX(created) from leads l2 where l2.lead_id = l.lead_id
)
LEFT JOIN statuses AS s ON l.status = s.status
LEFT JOIN trackers AS t ON t.id = a.tracker_id
LIMIT 10
I am able to sort by every column from actions table. However when I try to for example ORDER BY b.name (from banners table, joined on actions.banner_id) or ORDER BY l.lead_id (joined from leads on more complex condition as seen above) MySQL is running query for a loooong time (most tables have tens of thousands records). Is it possible, performance-wise, to sort by joined columns?
You should rewrite the query with a inner join on the table where the column you want to sort on is.
For example, if you sort on actions.banner_id
SELECT ...
FROM actions AS a
JOIN banners AS b ON b.campaign_id = a.campaign_id
LEFT JOIN *rest of the query*
You will get the same results unless there is not enough banners that can be joined to action to produce a total of 10 rows.
I'm guessing it's not the case otherwise you wouldn't be sorting on banner_id.
You could first filter (order by, where, etc.) your records in a subquery and then join the result with the rest of the tables.
I have movie database that has these tables: new_movies, ratings, critic_ratings, colors
I'm trying to execute this SELECT statement which will combine these 4 tables on the same movie using 'mid' (movie id):
SELECT DISTINCT
new_movies.*,
movies_db.*,
ratings.rating,
ratings.count,color,
critic_ratings.rating AS critic_ratings
FROM
new_movies
INNER JOIN
movies_db
ON
new_movies.mid = movies_db.mid
LEFT JOIN
ratings
ON
new_movies.mid = ratings.mid
LEFT JOIN
colors
ON
new_movies.mid = colors.mid
LEFT JOIN
critic_ratings
ON
new_movies.mid = critic_ratings.mid
ORDER BY
title ASC
But I get this error:
The SELECT would examine more than
MAX_JOIN_SIZE rows; check your WHERE
and use SET SQL_BIG_SELECTS=1 or SET
SQL_MAX_JOIN_SIZE=# if the SELECT is
okay
How do I properly do this query?
If you don't want to enable big selects, you could reform this using correlated sub-queries. (I don't know if you'll still hit the limit or not though.)
SELECT DISTINCT
new_movies.*,
movies_db.*,
(SELECT rating FROM ratings WHERE new_movies.mid = ratings.mid) AS rating,
(SELECT count FROM ratings WHERE new_movies.mid = ratings.mid) AS rating_count,
(SELECT color FROM colors WHERE new_movies.mid = colors.mid) AS colour,
(SELECT rating FROM critic_ratings WHERE new_movies.mid = critic_ratings.mid) AS critic_ratings
FROM
new_movies
INNER JOIN
movies_db
ON new_movies.mid = movies_db.mid
ORDER BY
title ASC
Also, worth a test to see if the LEFT JOINs are actually the cause, can you execute the following?
SELECT DISTINCT
new_movies.*,
movies_db.*
FROM
new_movies
INNER JOIN
movies_db
ON new_movies.mid = movies_db.mid
ORDER BY
title ASC
why do you have a movies and a new_movies table? surely a release date field would be sufficient for that - would cut out a join too...
to that end I would create a view of that data and query that instead.
But back to your query:
SELECT DISTINCT
new_movies.*,
movies_db.*,
ratings.rating,
ratings.count,
color,
critic_ratings.rating AS critic_ratings
FROM
new_movies
INNER JOIN
movies_db
ON
new_movies.mid = movies_db.mid
LEFT JOIN
ratings
ON
new_movies.mid = ratings.mid
LEFT JOIN
colors
ON
new_movies.mid = colors.mid
LEFT JOIN
critic_ratings
ON
new_movies.mid = critic_ratings.mid
ORDER BY
title ASC
I can't see anything obvious... perhaps you can post the results of an explain query?
There is no problem with your query per se. It's just that you're selecting all movies (no WHERE, no LIMIT) and since you're joining ratings for e.g., it will join all ratings to each movie. You are just reaching the max amount of data allowed for joins.
I'm not sure why you'd need to select all movies. Perhaps you can use a limit. Otherwise you can just try the solutions in the error message.