Related
I've got a MySQL database with about 80K products, 300 store locations, and pricing info. Some products have prices, some don't. I'm running search on the product names (text), tags associated with them (text), ranks (integers), omit_tags that searches & products that don't go together, and geolocation based on a users location and distance they specify from the stores location. My search times are averaging 1.3s to return 10 results. Would this be considered slow?
Here's what my query looks like.
SELECT DISTINCT t3.prod_id,
t3.awaiting_approval,
t3.brand,
t3.prod_name,
t3.size,
t3.units,
t3.category,
t3.image,
t3.url,
t3.quantity,
t3.rank,
t3.word_count,
t3.word_count
FROM ((SELECT item_info_mem.prod_id,
item_info_mem.awaiting_approval,
item_info_mem.prod_name,
item_info_mem.brand,
item_info_mem.size,
item_info_mem.units,
item_info_mem.category,
(SELECT rank
FROM search_tags_mem
WHERE prod_id = item_info_mem.prod_id
AND tag = "bread"
ORDER BY rank
LIMIT 1) AS
rank,
(SELECT image
FROM images_mem
WHERE prod_id = item_info_mem.prod_id
ORDER BY id
LIMIT 1) AS
image,
(SELECT url
FROM urls_mem
WHERE prod_id = item_info_mem.prod_id
AND disabled IS NULL
ORDER BY id DESC
LIMIT 1) AS
url,
(SELECT quantity
FROM shopping_list_mem
WHERE user_id = "1"
AND prod_id = item_info_mem.prod_id
ORDER BY id
LIMIT 1) AS
quantity,
( Substrcount(Lcase(item_info_mem.prod_name), Lcase("bread"))
+ Substrcount(Lcase(item_info_mem.brand), Lcase("bread")) ) AS
word_count,
( Substrcheck(Lcase(item_info_mem.prod_name),
Lcase(item_info_mem.brand), Lcase("bread")) ) AS
word_count_unique
FROM item_info_mem
WHERE NOT EXISTS (SELECT id
FROM search_tags_omit_mem
WHERE prod_id = item_info_mem.prod_id
AND tag = "bread"
ORDER BY id DESC
LIMIT 1)
AND ( item_info_mem.prod_name REGEXP "bread"
OR item_info_mem.brand REGEXP "bread" )
AND EXISTS(SELECT scans_mem.scan_id
FROM scans_mem,
stores_mem
WHERE scans_mem.price IS NOT NULL
AND scans_mem.expired IS NULL
AND item_info_mem.prod_id = scans_mem.prod_id
AND NOT
EXISTS (SELECT
user_stores_disabled_mem.user_id
FROM
user_stores_disabled_mem
,
stores_mem
WHERE scans_mem.store_id =
stores_mem.id
AND
user_stores_disabled_mem.chain =
stores_mem.chain
AND
user_stores_disabled_mem.user_id =
1)
AND ( Sqrt(Pow(111 * (stores_mem.gps_lat
- 40.748080
), 2)
+ Pow(111 * (stores_mem.gps_lng -
-73.990533) *
Cos(
40.748080 /
57.3)
,
2)) <= (SELECT user_mem.distance
FROM user_mem
WHERE id = 1))
AND stores_mem.id = scans_mem.store_id))
UNION ALL
(SELECT item_info_mem.prod_id,
item_info_mem.awaiting_approval,
item_info_mem.prod_name,
item_info_mem.brand,
item_info_mem.size,
item_info_mem.units,
item_info_mem.category,
search_tags_mem.rank,
(SELECT image
FROM images_mem
WHERE prod_id = item_info_mem.prod_id
ORDER BY id
LIMIT 1) AS image,
(SELECT url
FROM urls_mem
WHERE prod_id = item_info_mem.prod_id
AND disabled IS NULL
ORDER BY id DESC
LIMIT 1) AS
url,
(SELECT quantity
FROM shopping_list_mem
WHERE user_id = "1"
AND prod_id = item_info_mem.prod_id
ORDER BY id
LIMIT 1) AS
quantity,
( Substrcount(Lcase(item_info_mem.prod_name), Lcase("bread"))
+ Substrcount(Lcase(item_info_mem.brand), Lcase("bread")) ) AS
word_count,
( Substrcheck(Lcase(item_info_mem.prod_name),
Lcase(item_info_mem.brand), Lcase("bread")) ) AS
word_count_unique
FROM item_info_mem,
search_tags_mem,
scans_mem,
stores_mem
WHERE NOT EXISTS (SELECT id
FROM search_tags_omit_mem
WHERE prod_id = item_info_mem.prod_id
AND tag = "bread"
ORDER BY id DESC
LIMIT 1)
AND scans_mem.price IS NOT NULL
AND scans_mem.expired IS NULL
AND item_info_mem.prod_id = search_tags_mem.prod_id
AND search_tags_mem.tag = "bread"
AND item_info_mem.prod_id = scans_mem.prod_id
AND NOT EXISTS (SELECT user_stores_disabled_mem.user_id
FROM user_stores_disabled_mem,
stores_mem
WHERE scans_mem.store_id = stores_mem.id
AND user_stores_disabled_mem.chain =
stores_mem.chain
AND user_stores_disabled_mem.user_id = 1)
AND ( Sqrt(Pow(111 * (stores_mem.gps_lat - 40.748080), 2)
+ Pow(111 * (stores_mem.gps_lng - -73.990533) * Cos(
40.748080 /
57.3)
,
2)) <= (SELECT user_mem.distance
FROM user_mem
WHERE id = 1))
AND stores_mem.id = scans_mem.store_id)) t3
ORDER BY -rank DESC,
word_count_unique DESC,
word_count DESC,
Field(t3.category, "food", "grocery", "pantry, household & pets",
"confectionery and grocery") DESC,
Length(prod_name),
brand
LIMIT 0, 10
````
There's a lot of reasons why the script being fast or slow.
PC Specs capabilities, DB Server version, DB Structure including indexes, Query structure also with network connection between server and client.
If you are asking Would this be considered slow? with 80k records my answer is NO
This Guide will help you about Query Optimization.
I have a table called votes with 4 columns: id, name, choice, date.
****id****name****vote******date***
****1*****sam*******A******01-01-17
****2*****sam*******B******01-05-30
****3*****jon*******A******01-01-19
My ultimate goal is to count up all the votes, but I only want to count 1 vote per person, and specifically each person's most recent vote.
In the example above, the result should be 1 vote for A, and 1 vote for B.
Here is what I currently have:
select name,
sum(case when uniques.choice = A then 1 else 0 end) votesA,
sum(case when uniques.choice = B then 1 else 0 end) votesB
FROM (
SELECT id, name, choice, max(date)
FROM votes
GROUP BY name
) uniques;
However, this doesn't work because the subquery is indeed selecting the max date, but it's not including the correct choice that is associated with that max date.
Don't think "group by" to get the most recent vote. Think of join or some other option. Here is one way:
SELECT v.name,
SUM(v.choice = 'A') as votesA,
SUM(v.choice = 'B') as votesB
FROM votes v
WHERE v.date = (SELECT MAX(v2.date) FROM votes v2 WHERE v2.name = v.name)
GROUP BY v.name;
Here is a SQL Fiddle.
Your answer are close but need to JOIN self
Subquery get Max date by name then JOIN self.
select
sum(case when T.vote = 'A' then 1 else 0 end) votesA,
sum(case when T.vote = 'B' then 1 else 0 end) votesB
FROM (
SELECT name,Max(date) as date
FROM T
GROUP BY name
) AS T1 INNER JOIN T ON T1.date = T.date
SQLFiddle
Try this
SELECT
choice,
COUNT(1)
FROM
votes v
INNER JOIN
(
SELECT
id,
max(date)
FROM
votes
GROUP BY
name
) tmp ON
v.id = tmp.id
GROUP BY
choice;
Something like this (if you really need count only last vote of person)
SELECT
sum(case when vote='A' then cnt else 0 end) voteA,
sum(case when vote='B' then cnt else 0 end) voteB
FROM
(SELECT vote,count(distinct name) cnt
FROM (
SELECT name,vote,date,max(date) over (partition by name) maxd
FROM votes
)
WHERE date=maxd
GROUP BY vote
)
PS. MySQL v 8
select
name,
sum( case when choice = 'A' then 1 else 0 end) voteA,
sum( case when choice = 'B' then 1 else 0 end) voteB
from
(
select id, name, choice
from votes
where date = (select max(date) from votes t2
where t2.name = votes.name )
) t
group by name
Or output just one row for the total counts of VoteA and VoteB:
select
sum( case when choice = 'A' then 1 else 0 end) voteA,
sum( case when choice = 'B' then 1 else 0 end) voteB
from
(
select id, name, choice
from votes
where date = (select max(date) from votes t2
where t2.name = votes.name )
) t
Based on #d-shish solution, and since introduction (in MySQL 5.7) of ONLY_FULL_GROUP_BY, the GROUP BY statement must be placed in subquery like this :
SELECT v.`name`,
SUM(v.`choice` = 'A') as `votesA`,
SUM(v.`choice` = 'B') as `votesB`
FROM `votes` v
WHERE (
SELECT MAX(v2.`date`)
FROM `votes` v2
WHERE v2.`name` = v.`name`
GROUP BY v.`name` # << after
) = v.`date`
# GROUP BY v.`name` << before
Otherwise, it won't work anymore !
Have a table photos
photos.id
photos.user_id
photos.order
A) Is it possible via a single query to group all photos by user and then update the order 1,2,3..N ?
B) added twist, what if some of the photos already have an order value associated? Make sure that the new photos.order never gets repeated and fills in ant orders lower or higher than those existing (as best as possible)
My only thought is just to run a script on this and loop through it and re'order' everything?
photos.id int(10)
photos.created_at datetime
photos.order int(10)
photos.user_id int(10)
Right now data may look like this
user_id = 1
photo_id = 1
order = NULL
user_id = 2
photo_id = 2
order = NULL
user_id = 1
photo_id = 3
order = NULL
the desired result would be
user_id = 1
photo_id = 1
order = 1
user_id = 2
photo_id = 2
order = 1
user_id = 1
photo_id = 3
order = 2
A)
You can use a variable that increments with each row and resets with each user_ID to get the row count.
SELECT ID,
User_ID,
`Order`
FROM ( SELECT #r:= IF(#u = User_ID, #r + 1,1) AS `Order`,
ID,
User_ID,
#u:= User_ID
FROM Photos,
(SELECT #r:= 1) AS r,
(SELECT #u:= 0) AS u
ORDER BY User_ID, ID
) AS Photos
Example on SQL Fiddle
B)
My First solution was to just add Order to the sorting that adds the row number, therefore anything with an Order Gets sorted by its order first, but this only works if your ordering system has no gaps and starts at 1:
SELECT ID,
User_ID,
RowNumber AS `Order`
FROM ( SELECT #r:= IF(#u = User_ID, #r + 1,1) AS `RowNumber`,
ID,
User_ID,
#u:= User_ID
FROM Photos,
(SELECT #i:= 1) AS r,
(SELECT #u:= 0) AS u
ORDER BY User_ID, `Order`, ID
) AS Photos
ORDER BY `User_ID`, `Order`
Example using Order Field
ORDERING WITH GAPS
I have eventually found a way of maintaining the sort order even when there are gaps in the sequence.
SELECT ID, User_ID, `Order`
FROM Photos
WHERE `Order` IS NOT NULL
UNION ALL
SELECT Photos.ID,
Photos.user_ID,
Numbers.RowNum
FROM ( SELECT ID,
User_ID,
#r1:= IF(#u1 = User_ID,#r1 + 1,1) AS RowNum,
#u1:= User_ID
FROM Photos,
(SELECT #r1:= 0) AS r,
(SELECT #u1:= 0) AS u
WHERE `Order` IS NULL
ORDER BY User_ID, ID
) AS Photos
INNER JOIN
( SELECT User_ID,
RowNum,
#r2:= IF(#u2 = User_ID,#r2 + 1,1) AS RowNum2,
#u2:= User_ID
FROM ( SELECT DISTINCT p.User_ID, o.RowNum
FROM Photos AS p,
( SELECT #i:= #i + 1 AS RowNum
FROM INFORMATION_SCHEMA.COLLATION_CHARACTER_SET_APPLICABILITY,
( SELECT #i:= 0) AS i
) AS o
WHERE RowNum <= (SELECT COUNT(*) FROM Photos P1 WHERE p.User_ID = p1.User_ID)
AND NOT EXISTS
( SELECT 1
FROM Photos p2
WHERE p.User_ID = p2.User_ID
AND o.RowNum = p2.`Order`
)
AND p.`Order` IS NULL
ORDER BY User_ID, RowNum
) AS p,
(SELECT #r2:= 0) AS r,
(SELECT #u2:= 0) AS u
ORDER BY user_ID, RowNum
) AS numbers
ON Photos.User_ID = numbers.User_ID
AND photos.RowNum = numbers.RowNum2
ORDER BY User_ID, `Order`
However as you can see this is pretty complicated. This works by treating those with an order value separately to those without. The top query just ranks all photos with no order value in order of ID for each user. The bottom query uses a cross join to generates a sequential list from 1 to n for each user ID (up to the number of entries for each User_ID). So with a data set like this:
ID User_ID Order
1 1 NULL
2 2 NULL
3 1 NULL
4 1 1
5 1 3
6 2 2
7 2 3
It would generate
UserID RowNum
1 1
1 2
1 3
1 4
2 1
2 2
2 3
It then uses NOT EXISTS to elimiate all combinations already used by Photos with a non null order, and ranked in order of RowNum partitioned by User_ID giving
UserID RowNum Rownum2
1 2 1
1 4 2
2 1 1
The RowNum2 value can then be matched with the rownum value achieved in the from subquery, giving the correct order value. Long winded, but it works.
Example on SQL Fiddle
Worked for me. I needed to increment version grouping by 4 fields (host, folder, fileName, status) and sort by 1 (downloadedAtTicks).
This is is my SELECT
SET #status := NULL;
SET #version := NULL;
SELECT
id,
host,
folder,
fileName,
status,
downloadedAtTicks,
version,
IF(IF(status IS NULL, 0, status) = #status, #version := #version + 1, #version := 0) AS varVersion,
#status := IF(status IS NULL, 0, status) AS varStatus
FROM csvsource
ORDER BY host, folder, fileName, status, downloadedAtTicks;
And this is my UPDATE
SET #status := NULL;
SET #version := NULL;
UPDATE
csvsource csv,
(SELECT
id,
IF(IF(status IS NULL, 0, status) = #status, #version := #version + 1, #version := 0) AS varVersion,
#status := IF(status IS NULL, 0, status) AS varStatus
FROM csvsource
ORDER BY host, folder, fileName, status, downloadedAtTicks) AS sub
SET
csv.version = sub.varVersion
WHERE csv.id = sub.id;
I have the following tables in my game's database:
rankedUp (image_id, user_id, created_at)
globalRank (image_id, rank )
matchups (user_id, image_id1, image_id2)
All image_ids in globalRank table are assigned a rank which is a float from 0 to 1
Assuming I have the current logged in user's "user_id" value, I'm looking for a query that will return a pair of image ids (imageid1, imageid2) such that:
imageid1 has lower rank than imageid2 but is also the next highest rank less than imageid2
matchups table doesn't have (userid,imageid1,imageid2) or (userid,imageid2,imageid1)
rankedup table doesn't have (userid,imageid1) or if it does, the createdat column is older than X hours
What I have so far for requirement 1 is this:
SELECT lowerImages.image_id AS lower_image, higherImages.image_id AS higher_image
FROM global_rank AS lowerImages, global_rank AS higherImages
WHERE lowerImages.rank < higherImages.rank
AND lowerImages.image_id = (
SELECT image_id
FROM (
SELECT image_id
FROM global_rank
WHERE rank < higherImages.rank
ORDER BY rank DESC
LIMIT 1 , 1
) AS tmp
)
but it doesnt work because I can't reference higherImages.rank in the subquery.
Does anyone know how I could satisfy all of those requirements in one query?
Thanks for your help
EDIT:
I now have this query but I don't know about the efficiency and I need to test it for correctness:
SELECT lowerImages.image_id AS lower_image,
max(higherImages.image_id) AS higher_image
FROM global_rank AS lowerImages, global_rank AS higherImages
WHERE lowerImages.rank < higherImages.rank
AND 1 NOT IN (select 1 from ranked_up where
lowerImages.image_id = ranked_up.image_id
AND ranked_up.user_id = $user_id
AND ranked_up.created_at > DATE_SUB(NOW(), INTERVAL 1 DAY))
AND 1 NOT IN (
SELECT 1 from matchups where user_id = $userId
AND lower_image_id = lowerImages.image_id
AND higher_image_id = higherImages.image_id
UNION
SELECT 1 from matchups where user_id = $user_id
AND lower_image_id = higherImages.image_id
AND higher_image_id = lowerImages.image_id
)
GROUP BY 1
the "not in" statements I'm using are all indexed so they should run fast. The efficiency problem I have is the group by and selection of the global_rank tables
This question is a revision of Pretty Complex SQL Query, which should no longer be answered.
select
(
select image_id, rank from
rankedup inner join globalRank
on rankedup.image_id = globalRank .image_id
where user_id = XXX
limit 1, 1
) as highest,
(
select image_id, rank from
rankedup inner join globalRank
on rankedup.image_id = globalRank .image_id
where user_id = XXX
limit 2, 1
) as secondhighest
I normally use SQL Server, but this i think is the translation for mysql :)
This should do the trick:
SELECT lowerImages.*, higherImages.*
FROM globalrank AS lowerImages, globalrank AS higherImages
WHERE lowerImages.rank < higherImages.rank
AND lowerImages.image_id = (
SELECT image_id
FROM (
SELECT image_id
FROM globalrank
WHERE rank < higherImages.rank
ORDER BY rank DESC
LIMIT 1,1
) AS tmp
)
AND NOT EXISTS (
SELECT * FROM matchups
WHERE user_id = $user_id
AND ((image_id1 = lowerImages.image_id AND image_id2 = higherImages.image_id)
OR (image_id2 = lowerImages.image_id AND image_id1 = higherImages.image_id))
)
AND higherImages.image_id NOT IN (
SELECT image_id FROM rankedup
WHERE created_at < DATE_ADD(NOW(), INTERVAL 1 DAY)
AND USER_ID <> $user_id
)
ORDER BY higherImages.rank
I'm assuming the PKs of matchups and rankedup include all columns in those tables. This would allow the second 2 sub-queries to utilize the PK indexes. You would probably want an ordered index on globalrank.rank to speed up the first sub-query.
have a table like this
empid questionid options
1 1 A
2 1 A
3 1 B
4 1 C
now i need result like this
questionid responseA responseB responseC
1 50% 25% 25%
You could PIVOT;
SELECT questionid, (A / total) * 100 responseA, (B / total) * 100 responseB, (C / total) * 100 responseC FROM (
SELECT T1.questionid, T1.options, T2.total
FROM the_tbl T1
INNER JOIN (SELECT questionid, CAST(COUNT(*) AS MONEY) AS total FROM the_tbl GROUP BY questionID) T2 ON T2.questionid = T1.questionid
) T
PIVOT (
COUNT(options) FOR options IN ([A], [B], [C])
) AS pvt
ORDER BY questionid
T-SQL:
SELECT
questionid,
SUM(CASE options WHEN 'A' THEN 100.0 ELSE 0.0 END) / COUNT(options) AS responseA,
SUM(CASE options WHEN 'B' THEN 100.0 ELSE 0.0 END) / COUNT(options) AS responseB,
SUM(CASE options WHEN 'C' THEN 100.0 ELSE 0.0 END) / COUNT(options) AS responseC
FROM
answers
GROUP BY
questionid
Note: To avoid casting and multiplying 100, I used 100.0 and 0.0 values in CASE ... END expressions.
SELECT CAST(100/
( SELECT COUNT(*)
FROM your_Table as t2
WHERE t2.questionid = t1.questionid )
* COUNT(*) AS VARCHAR) + '%' AS 'Percent', OPTIONS, questionid
FROM your_Table as t1
--WHERE questionid = 2
GROUP BY OPTIONS, questionid
ORDER BY questionid;
This is one possible way you could do it
(works on SQL-Server but not sure if it does in MySql)