MySQL Query Slow with ORDER BY - mysql

So I'm having a problem when I add in ORDER BY date_last_access DESC the whole query slows down to 3secs and without it's about 0.2secs, why is it running so slow and how can I change the query to run faster?
There are also indexes on all the tables and fields used.
Users: 1+ million records
Likes: 5+ million records (over 1 billion in production)
Tables will be growing really fast once in production.
QUERY
SELECT
id,
sid,
first_name,
date_birth,
location,
date_created,
date_last_access,
(3956 * 2 * ASIN(
SQRT(
POWER(
SIN(
({LAT} - latitude) * pi() / 180 / 2
),
2
) + COS({LAT} * pi() / 180) * COS(latitude * pi() / 180) * POWER(
SIN(
({LON} - longitude) * pi() / 180 / 2
),
2
)
)
)) AS distance
FROM
users
WHERE
`id` != {UID} AND
`gender` = {GEND} AND
`date_birth` BETWEEN {DOB_MIN} AND {DOB_MAX} AND
`status` = 'active' AND
(SELECT COUNT(*) FROM likes WHERE likes.judged_user = users.id AND likes.user_id = {UID}) = 0
HAVING distance <= {DIST}
ORDER BY date_last_access DESC
LIMIT {ROWS}
EXPLAIN
1 PRIMARY users ref PRIMARY,Index_2,discovery,index_1 index_1 2 const 226184 Using index condition; Using where; Using filesort
2 DEPENDENT SUBQUERY likes eq_ref PRIMARY,index_1,index_2 PRIMARY 16 const,hello.users.id 1 Using index
INDEXES
LIKES - user_id, judged_user - NORMAL - BTREE
USERS - id, gender, date_birth, status, date_last_access - NORMAL - BTREE
When I order by id instead of date_last_access it seems to run much faster, could it be cause date_last_access is a datetime format?

First try run a EXPLAIN of your query. This will show you what fields and operations are slowing your query. Then try to make joins with indexed columns and filter you resultset with more specific values.

Simplyfying the subquery could be a better way to avoid extra processing time (COUNT):
(SELECT COUNT(*) FROM likes WHERE likes.judged_user = users.id AND likes.user_id = {UID}) = 0
could change to
(SELECT 1 FROM likes WHERE likes.judged_user = users.id AND likes.user_id = {UID} limit 1) IS NULL
Avoiding a subquery could be the best way to improve the performance of the query. You could check what options could be better for your case (an index for likes.user_id is required in this case)
FROM
users
LEFT JOIN (
SELECT distinct judged_user FROM likes WHERE likes.user_id = {UID}
) l ON l.judged_user=users.id
WHERE
`id` != {UID} AND
`gender` = {GEND} AND
`date_birth` BETWEEN {DOB_MIN} AND {DOB_MAX} AND
`status` = 'active' AND
l.judged_user is NULL

You should phrase the FROM clause as:
WHERE `id` <> {UID} AND
`gender` = {GEND} AND
`date_birth` BETWEEN {DOB_MIN} AND {DOB_MAX} AND
`status` = 'active' AND
NOT EXISTS (SELECT 1 FROM likes l WHERE l.judged_user = users.id AND l.user_id = {UID})
HAVING distance <= {DIST}
For this query, you can try two indexes:
LIKES(judged_user, user_id)
USERS(Gender, status, date_birth, id)

Related

MYSQL - Subqueries problem - Cant reuse the table

WITH t as (
SELECT *
FROM scd p
WHERE p.modified_date > FROM_UNIXTIME(1593060230)
AND ( p.main_id = 1
OR FIND_IN_SET(1, p.mult_ids) <> 0 )
ORDER BY modified_date DESC
LIMIT 2 OFFSET 0
),
del as (
SELECT
*
FROM t WHERE (status <> 1 AND status <> 2)
),
w_del as (
SELECT
*
FROM t WHERE (status = 1 OR status = 2)
)
SELECT w_del.*, del.* FROM w_del,del;
How do I achieve this with normal sub queries. I am using MySQL 5.7 and can't use CTEs. Im getting can't reuse table error if I use UNION/sub-queries. Is there a way to achieve this without temporary tables?
Please help.
You can just plug in the code for each alias . . . and keep doing that until you are at the base tables:
SELECT w_del.*, del.*
FROM (SELECT t.*
FROM (SELECT *
FROM scd p
WHERE p.modified_date > FROM_UNIXTIME(1593060230) AND
( p.main_id = 1 OR FIND_IN_SET(1, p.mult_ids) <> 0 )
ORDER BY modified_date DESC
LIMIT 2 OFFSET 0
) t
WHERE (status <> 1 AND status <> 2)
) w_del CROSS JOIN
(SELECT t.*
FROM (SELECT *
FROM scd p
WHERE p.modified_date > FROM_UNIXTIME(1593060230) AND
( p.main_id = 1 OR FIND_IN_SET(1, p.mult_ids) <> 0 )
ORDER BY modified_date DESC
LIMIT 2 OFFSET 0
) t
WHERE (status = 1 OR status = 2)
) del;
One critical point, though: The definition of t is using ORDER BY and LIMIT. If there are ties in the modified_date column, then the two subqueries could return different result sets. You have two choices to avoid a problem here:
Add additional keys to the ORDER BY to ensure that the sorting is stable (i.e. returns the same results each time because the combination of keys is unique).
Materialize the subquery using a temporary table.

Combining SQL Google Search with own SQL query

I am pretty new to SQL queries.
I have a google SQL Search example
SELECT cID,
(6371 * acos
(
cos(radians(51.455643))
* cos(radians(latCord))
* cos(radians(longCord) - radians(7.011555))
+ sin(radians(51.455643))
* sin(radians(latCord))
)
) AS distance
FROM breitengrade
HAVING distance < 50
ORDER BY distance
LIMIT 0, 20
and a own SQL query
SELECT breitengrade.cID
,breitengrade.latCord
,breitengrade.longCord
,Pages.cIsActive
FROM breitengrade
INNER JOIN Pages ON breitengrade.cID = Pages.cID
WHERE cIsActive = '1'
How can I combine these 2 queries into one so that I can get one single result set?
SELECT breitengrade.cID,
breitengrade.latCord,
breitengrade.longCord,
Pages.cIsActive
(6371 * acos
(
cos(radians(51.455643))
* cos(radians(latCord))
* cos(radians(longCord) - radians(7.011555))
+ sin(radians(51.455643))
* sin(radians(latCord))
)
) AS distance
FROM breitengrade
INNER JOIN Pages ON breitengrade.cID = Pages.cID
WHERE cIsActive = '1'
HAVING distance < 50
ORDER BY distance
LIMIT 0, 20

how to get category wise rendom records with different limit for every category

I have issue in one mysql query
Note I have checked all answer plz don't mark this question as repeat
i have one table with name of "questions" which contain que_id,cat_id, etc fields
I need to fetch 50 records from table which should be random 30 from cat_id=1 and random 20 from cat_id=2
I guess this should do it:
(SELECT * FROM questions
WHERE cat_id = 1
ORDER BY RAND()
LIMIT 30)
UNION ALL
(SELECT * FROM questions
WHERE cat_id = 2
ORDER BY RAND()
LIMIT 20)
(SELECT * FROM `questions` WHERE cat_id=2 and que_id >=
(SELECT FLOOR( MAX(que_id) * RAND()) FROM `questions` ) ORDER BY que_id LIMIT 20) UNION ALL
(SELECT * FROM `questions` WHERE cat_id=1 and que_id >=
(SELECT FLOOR( MAX(que_id) * RAND()) FROM `questions` ) ORDER BY que_id LIMIT 30)

mySQL trouble building query with multiple limits

I Have this query which works with the single limit imposed at the end.
select distinct
loc.mID,
loc.city,
loc.state,
loc.zip,
loc.country,
loc.latitude,
loc.longitude,
baseInfo.firstname,
baseInfo.lastname,
baseInfo.profileimg,
baseInfo.facebookID,
(((acos(sin(('37.816876'*pi()/180)) * sin((`latitude`*pi()/180))+cos(('37.816876'*pi()/180)) * cos((`latitude`*pi()/180)) * cos((('-121.285410' - `longitude`)*pi()/180))))*180/pi())*60*1.1515) AS `distance`,
teams.teamName,
teams.leagueType,
teams.teamType,
teams.subcat
FROM memb_geo_locations loc
left join memb_friends friends on (friends.mID = loc.mID or friends.friendID = loc.mID) and (friends.mID = '100018' or friends.friendID = '100018')
join memb_baseInfo baseInfo on baseInfo.mID = loc.mID
join memb_teams teams on teams.mID = loc.mID
where
loc.primaryAddress = '1'
and ((friends.mID is null or friends.friendID is null)
or (friends.isactive = 2))
and (
teams.teamName like '%Anaheim Ducks%'
or teams.teamName like '%San Jose Sharks%'
or teams.teamName like '%New England Patriots%'
or teams.teamName like '%New York Yankees%'
or teams.teamName like '%Orlando Magic%'
)
and loc.mID != 100018
having `distance` < 50
order by baseInfo.firstname
asc limit 30
However I want my results to be limited by the teamName to 3 results max per, And I have tried stuff to the extent of
select distinct
loc.mID,
loc.city,
loc.state,
loc.zip,
loc.country,
loc.latitude,
loc.longitude,
baseInfo.firstname,
baseInfo.lastname,
baseInfo.profileimg,
baseInfo.facebookID,
(((acos(sin(('37.816876'*pi()/180)) * sin((`latitude`*pi()/180))+cos(('37.816876'*pi()/180)) * cos((`latitude`*pi()/180)) * cos((('-121.285410' - `longitude`)*pi()/180))))*180/pi())*60*1.1515) AS `distance`,
teams.teamName,
teams.leagueType,
teams.teamType,
teams.subcat
FROM memb_geo_locations loc
left join memb_friends friends on (friends.mID = loc.mID or friends.friendID = loc.mID) and (friends.mID = '100018' or friends.friendID = '100018')
join memb_baseInfo baseInfo on baseInfo.mID = loc.mID
join memb_teams teams on teams.mID = loc.mID
where
loc.primaryAddress = '1'
and ((friends.mID is null or friends.friendID is null)
or (friends.isactive = 2))
and (
(select * from memb_teams where teamName like '%Buffalo Bills%' limit 2),
(select * from memb_teams where teamName like '%San Jose Sharks%' limit 2),
(select * from memb_teams where teamName like '%New England Patriots%' limit 2)
)
and loc.mID != 100018
having `distance` < 150
order by baseInfo.firstname
asc limit 30
With no success, usually just syntax errors.. or Operand Should 1 Column(s) so I am reaching out here hopefully someone can give me some idea how to refine my query a bit so I can limit the results to 3 per teamName.. rather than having staggered results where I could have 20 of one and 4 of another 2 of another and 1 and 1 (which is not desired). 3 or less per team is desired, Just don't know how. Ideas, that don't involve tackling a huge data set from the query and looping over it via server side code to output results I desire?
IN MSSQL I use ROW_NUMBER function, and it would be something like this:
SELECT * FROM dbo.MyTable WHERE recno
IN (SELECT recno FROM (SELECT Teamname, ROW_NUMBER() OVER (PARTITION BY Teamname ORDER BY recno DESC) AS intRow FROM dbo.MyTable) AS T
WHERE intRow IN (1,2,3))
recno=your unique record number
Basically your Subquery selects the top 3 records, adding a new "ROW NUMBER" column.
The Top Query selects all the records with Rownumber between 1 to 3.
I know there is no ROW_NUMBER() native function in MYSQL, so you could use this instead:
MySQL - Get row number on select

How to get mysql random integer range?

I am trying to generate a random integer for each row I select between 1 and 60 as timer.
SELECT downloads.date, products.*, (FLOOR(1 + RAND() * 60)) AS timer
I have searched and keep coming up to this FLOOR function as how to select a random integer in a range. This is giving me a 1 for every row.
What am I missing?
I am on mysql 5.0.75
Heres the rest of the query I belive it might be a nesting issue
SELECT *
FROM (
SELECT downloads.date, products.*, FLOOR(1 + (RAND() * 60)) AS randomtimer,
(
SELECT COUNT( * )
FROM distros
WHERE distros.product_id = products.product_id
) AS distro_count,
(SELECT COUNT(*) FROM downloads WHERE downloads.product_id = products.product_id) AS true_downloads
FROM downloads
INNER JOIN products ON downloads.product_id = downloads.product_id
) AS count_table
WHERE count_table.distro_count > 0
AND count_table.active = 1
ORDER BY count_table.randomtimer , count_table.date DESC LIMIT 10
This is working for me. Your mysql version maybe?
SELECT id, (FLOOR( 1 + RAND( ) *60 )) AS timer
FROM users
LIMIT 0 , 30
The output of the RAND function will always be a value between 0 and 1.
Try this:
SELECT downloads.date, products.*, (CAST(RAND() * 60 AS UNSIGNED) + 1) AS timer
Old question, but always actual problem.
Here a way to create a MySQL function random_integer() based on manual :
CREATE FUNCTION random_integer(value_minimum INT, value_maximum INT)
RETURNS INT
COMMENT 'Gets a random integer between value_minimum and value_maximum, bounds included'
RETURN FLOOR(value_minimum + RAND() * (value_maximum - value_minimum + 1));
SELECT ALL random_integer(1, 60) AS timer;
I'm running your query and it does give me a random number for each row.... maybe has something to do with the name of the random (timer)?
You can increase the number multiplied by the number of records in the table.
SELECT id,
(FLOOR( (SELECT MIN(id) FROM your_table ) + RAND( ) * 1000000 ) ) AS timer
FROM your_table
LIMIT 0 , 30