In this query I want a total of x number of records returned. within that query I have several sub-queries where I can't be sure if they'll return the max number of records. if one result is less than it's max limit I want to populate the remaining slots with the next query and so on. I can't do math inside a limit clause so I'm still trying to figure out how to do it. here is what I would do if math was available inside the limit clause.
select *
from
(
(select * from profile where size='local' order by rand() limit 7) as local
join
(select * from profile where size='regional' order by rand() limit (13-count(local.id)) as regional
join
(select * from profile where size='national' order by rand() limit (19-(count(local.id)+count(regional.id))) as national
join
(select * from profile where size='international' order by rand() limit (25-(count(local.id)+count(regional.id)+count(national.id)))) as international
)
I might have done this a needlessly complicated way, but it does seem to work:-
SELECT id, size
FROM
(
SELECT id, size,
#SeqLocal:=IF(size="local", IF(#SeqLocal <= 7, #SeqLocal + 1, #SeqLocal), #SeqLocal) AS SeqLocal,
#SeqRegional:=IF(size="regional", IF(#SeqLocal + #SeqRegional <= 14, #SeqRegional + 1, #SeqRegional), #SeqRegional) AS SeqRegional,
#SeqNational:=IF(size="national", IF(#SeqLocal + #SeqRegional + #SeqNational <= 21 , #SeqNational + 1, #SeqNational), #SeqNational) AS SeqNational,
#SeqInternational:=IF(size="international", IF(#SeqLocal + #SeqRegional + #SeqNational + #SeqInternational <= 28, #SeqInternational + 1, #SeqInternational), #SeqInternational) AS SeqInternational
FROM
(
select *
from profile
where size IN ("local", "regional", "national", "international")
order by FIELD(size, "local", "regional", "national", "international"), rand()
) Sub1
CROSS JOIN (SELECT #SeqLocal:=0, #SeqRegional:=0, #SeqNational:=0, #SeqInternational:=0) Sub2
) Sub3
WHERE (size = "local" AND SeqLocal != #SeqLocal)
OR (size = "regional" AND SeqRegional != #SeqRegional)
OR (size = "national" AND SeqNational != #SeqNational )
OR (size = "international" AND SeqInternational != #SeqInternational)
Sqlfiddle here:-
http://www.sqlfiddle.com/#!2/cc3b884/14
Related
I've got a MySQL database with about 80K products, 300 store locations, and pricing info. Some products have prices, some don't. I'm running search on the product names (text), tags associated with them (text), ranks (integers), omit_tags that searches & products that don't go together, and geolocation based on a users location and distance they specify from the stores location. My search times are averaging 1.3s to return 10 results. Would this be considered slow?
Here's what my query looks like.
SELECT DISTINCT t3.prod_id,
t3.awaiting_approval,
t3.brand,
t3.prod_name,
t3.size,
t3.units,
t3.category,
t3.image,
t3.url,
t3.quantity,
t3.rank,
t3.word_count,
t3.word_count
FROM ((SELECT item_info_mem.prod_id,
item_info_mem.awaiting_approval,
item_info_mem.prod_name,
item_info_mem.brand,
item_info_mem.size,
item_info_mem.units,
item_info_mem.category,
(SELECT rank
FROM search_tags_mem
WHERE prod_id = item_info_mem.prod_id
AND tag = "bread"
ORDER BY rank
LIMIT 1) AS
rank,
(SELECT image
FROM images_mem
WHERE prod_id = item_info_mem.prod_id
ORDER BY id
LIMIT 1) AS
image,
(SELECT url
FROM urls_mem
WHERE prod_id = item_info_mem.prod_id
AND disabled IS NULL
ORDER BY id DESC
LIMIT 1) AS
url,
(SELECT quantity
FROM shopping_list_mem
WHERE user_id = "1"
AND prod_id = item_info_mem.prod_id
ORDER BY id
LIMIT 1) AS
quantity,
( Substrcount(Lcase(item_info_mem.prod_name), Lcase("bread"))
+ Substrcount(Lcase(item_info_mem.brand), Lcase("bread")) ) AS
word_count,
( Substrcheck(Lcase(item_info_mem.prod_name),
Lcase(item_info_mem.brand), Lcase("bread")) ) AS
word_count_unique
FROM item_info_mem
WHERE NOT EXISTS (SELECT id
FROM search_tags_omit_mem
WHERE prod_id = item_info_mem.prod_id
AND tag = "bread"
ORDER BY id DESC
LIMIT 1)
AND ( item_info_mem.prod_name REGEXP "bread"
OR item_info_mem.brand REGEXP "bread" )
AND EXISTS(SELECT scans_mem.scan_id
FROM scans_mem,
stores_mem
WHERE scans_mem.price IS NOT NULL
AND scans_mem.expired IS NULL
AND item_info_mem.prod_id = scans_mem.prod_id
AND NOT
EXISTS (SELECT
user_stores_disabled_mem.user_id
FROM
user_stores_disabled_mem
,
stores_mem
WHERE scans_mem.store_id =
stores_mem.id
AND
user_stores_disabled_mem.chain =
stores_mem.chain
AND
user_stores_disabled_mem.user_id =
1)
AND ( Sqrt(Pow(111 * (stores_mem.gps_lat
- 40.748080
), 2)
+ Pow(111 * (stores_mem.gps_lng -
-73.990533) *
Cos(
40.748080 /
57.3)
,
2)) <= (SELECT user_mem.distance
FROM user_mem
WHERE id = 1))
AND stores_mem.id = scans_mem.store_id))
UNION ALL
(SELECT item_info_mem.prod_id,
item_info_mem.awaiting_approval,
item_info_mem.prod_name,
item_info_mem.brand,
item_info_mem.size,
item_info_mem.units,
item_info_mem.category,
search_tags_mem.rank,
(SELECT image
FROM images_mem
WHERE prod_id = item_info_mem.prod_id
ORDER BY id
LIMIT 1) AS image,
(SELECT url
FROM urls_mem
WHERE prod_id = item_info_mem.prod_id
AND disabled IS NULL
ORDER BY id DESC
LIMIT 1) AS
url,
(SELECT quantity
FROM shopping_list_mem
WHERE user_id = "1"
AND prod_id = item_info_mem.prod_id
ORDER BY id
LIMIT 1) AS
quantity,
( Substrcount(Lcase(item_info_mem.prod_name), Lcase("bread"))
+ Substrcount(Lcase(item_info_mem.brand), Lcase("bread")) ) AS
word_count,
( Substrcheck(Lcase(item_info_mem.prod_name),
Lcase(item_info_mem.brand), Lcase("bread")) ) AS
word_count_unique
FROM item_info_mem,
search_tags_mem,
scans_mem,
stores_mem
WHERE NOT EXISTS (SELECT id
FROM search_tags_omit_mem
WHERE prod_id = item_info_mem.prod_id
AND tag = "bread"
ORDER BY id DESC
LIMIT 1)
AND scans_mem.price IS NOT NULL
AND scans_mem.expired IS NULL
AND item_info_mem.prod_id = search_tags_mem.prod_id
AND search_tags_mem.tag = "bread"
AND item_info_mem.prod_id = scans_mem.prod_id
AND NOT EXISTS (SELECT user_stores_disabled_mem.user_id
FROM user_stores_disabled_mem,
stores_mem
WHERE scans_mem.store_id = stores_mem.id
AND user_stores_disabled_mem.chain =
stores_mem.chain
AND user_stores_disabled_mem.user_id = 1)
AND ( Sqrt(Pow(111 * (stores_mem.gps_lat - 40.748080), 2)
+ Pow(111 * (stores_mem.gps_lng - -73.990533) * Cos(
40.748080 /
57.3)
,
2)) <= (SELECT user_mem.distance
FROM user_mem
WHERE id = 1))
AND stores_mem.id = scans_mem.store_id)) t3
ORDER BY -rank DESC,
word_count_unique DESC,
word_count DESC,
Field(t3.category, "food", "grocery", "pantry, household & pets",
"confectionery and grocery") DESC,
Length(prod_name),
brand
LIMIT 0, 10
````
There's a lot of reasons why the script being fast or slow.
PC Specs capabilities, DB Server version, DB Structure including indexes, Query structure also with network connection between server and client.
If you are asking Would this be considered slow? with 80k records my answer is NO
This Guide will help you about Query Optimization.
completely stuck use different approaches but still no luck,
I am using MySQl and SparkSql
I have simple select
SELECT
rdw.merchant_id, hash_id, transaction_ts
FROM table_1 rdw
JOIN table_2 bl ON rdw.MERCHANT_ID = bl.MERCHANT_ID
WHERE brand = 'TBrand'
order by rand()
limit 36
and this is perfectly work, but here is the problem in limit 36 i need select 30 % from all records, I cannot pass number in it because spark failed on working with variables. I . trying to calculate % in limit but . its now alowed, any ideas how to load ?
Problem is, I dont know how many records will be returned by select, thats why i cannot set limit, it should be % from records
If you can deal with an approximate 30%, you can simply do:
where brand = 'TBrand' and rand() <= 0.3
If you want a better approximation to 30%, then you can do:
SELECT x.*
FROM (SELECT rdw.merchant_id, hash_id, transaction_ts,
(#rn := #rn + 1) as rn
FROM table_1 rdw JOIN
table_2 bl
ON rdw.MERCHANT_ID = bl.MERCHANT_ID CROSS JOIN
(SELECT #rn := 0) params
WHERE brand = 'TBrand'
ORDER BY rand()
) x
WHERE rn <= #rn * 0.30
SET #row_number = 0;
select merchant_id, hash_id, transaction_ts
from (
SELECT
(#row_number:=#row_number + 1) as num,
rdw.merchant_id, hash_id, transaction_ts
FROM table_1 rdw
JOIN table_2 bl ON rdw.MERCHANT_ID = bl.MERCHANT_ID
WHERE brand = 'TBrand'
order by rand()
)
where num <= FLOOR((#row_number * 3) /10 )
I am making multiple unions on the same tables
however i need to order the records of the second table by rand()
keeping in mind that I DO NOT want to have duplicate records since Iam using order by rand()
Example:
news table has the following data: (test1,test2,test3)
ads table has the following data: (ads1,ads2,ads3)
The result should be like this:
news are sorted by id
ads are sorted by rand() : which means ads2 may comes in the top of the list, and maybe ads1 comes in the top of the list and so on..
This is my sql statement:
(select news.title
from news
order by news.id desc limit 6) union
(select
advertisements.title
from advertisements
order rand() limit 1,1)
union
(select
news.title,
from news
order by news.id desc limit 6,6)
union
(select
advertisements.title
from advertisements
order by rand() limit 2,1)
Near as I can tell, you seem to want 6 news articles followed by an advertising one, and then repeated again. This is not what your query does, but I'm guessing that is the intention in using union.
I would suggest enumerating the values and then doing the sort outside:
select title
from ((select n.title, #rn := #rn + 1, 'n' as which, id
from news n cross join (select #rn := 0) params
order by n.id desc
limit 12
)
union all
(select a.title, (#rna := #rna + 1) as rn, 'a', NULL
from advertisements a cross join (select #rna := 0) params
order rand()
limit 2
)
) na
order by (case when which = 'n' and rn <= 6 then 1
when which = 'a' and rn = 1 then 2
when which = 'n' and rn <= 12 then 3
when which = 'a' and rn = 1 then 4
end),
id desc;
This type of question is asked every now and then. The queries provided works, but it affects performance.
I have tried the JOIN method:
SELECT *
FROM nbk_tabl
INNER JOIN (
SELECT ITEM_NO, MAX(REF_DATE) as LDATE
FROM nbk_tabl
GROUP BY ITEM_NO) nbk2
ON nbk_tabl.REF_DATE = nbk2.LDATE
AND nbk_tabl.ITEM_NO = nbk2.ITEM_NO
And the tuple one (way slower):
SELECT *
FROM nbk_tabl
WHERE REF_DATE IN (
SELECT MAX(REF_DATE)
FROM nbk_tabl
GROUP BY ITEM_NO
)
Is there any other performance friendly way of doing this?
EDIT: To be clear, I'm applying this to a table with thousands of rows.
Yes, there is a faster way.
select *
from nbk_table
order by ref_date desc
limit <n>
Where is the number of rows that you want to return.
Hold on. I see you are trying to do this for a particular item. You might try this:
select *
from nbk_table n
where ref_date = (select max(ref_date) from nbk_table n2 where n.item_no = n2.item_no)
It might optimize better than the "in" version.
Also in MySQL you can use user variables (Suppose nbk_tabl.Item_no<>0):
select *
from (
select nbk_tabl.*,
#i := if(#ITEM_NO = ITEM_NO, #i + 1, 1) as row_num,
#ITEM_NO := ITEM_NO as t_itemNo
from nbk_tabl,(select #i := 0, #ITEM_NO := 0) t
order by Item_no, REF_DATE DESC
) as x where x.row_num = 1;
I am trying to generate a random integer for each row I select between 1 and 60 as timer.
SELECT downloads.date, products.*, (FLOOR(1 + RAND() * 60)) AS timer
I have searched and keep coming up to this FLOOR function as how to select a random integer in a range. This is giving me a 1 for every row.
What am I missing?
I am on mysql 5.0.75
Heres the rest of the query I belive it might be a nesting issue
SELECT *
FROM (
SELECT downloads.date, products.*, FLOOR(1 + (RAND() * 60)) AS randomtimer,
(
SELECT COUNT( * )
FROM distros
WHERE distros.product_id = products.product_id
) AS distro_count,
(SELECT COUNT(*) FROM downloads WHERE downloads.product_id = products.product_id) AS true_downloads
FROM downloads
INNER JOIN products ON downloads.product_id = downloads.product_id
) AS count_table
WHERE count_table.distro_count > 0
AND count_table.active = 1
ORDER BY count_table.randomtimer , count_table.date DESC LIMIT 10
This is working for me. Your mysql version maybe?
SELECT id, (FLOOR( 1 + RAND( ) *60 )) AS timer
FROM users
LIMIT 0 , 30
The output of the RAND function will always be a value between 0 and 1.
Try this:
SELECT downloads.date, products.*, (CAST(RAND() * 60 AS UNSIGNED) + 1) AS timer
Old question, but always actual problem.
Here a way to create a MySQL function random_integer() based on manual :
CREATE FUNCTION random_integer(value_minimum INT, value_maximum INT)
RETURNS INT
COMMENT 'Gets a random integer between value_minimum and value_maximum, bounds included'
RETURN FLOOR(value_minimum + RAND() * (value_maximum - value_minimum + 1));
SELECT ALL random_integer(1, 60) AS timer;
I'm running your query and it does give me a random number for each row.... maybe has something to do with the name of the random (timer)?
You can increase the number multiplied by the number of records in the table.
SELECT id,
(FLOOR( (SELECT MIN(id) FROM your_table ) + RAND( ) * 1000000 ) ) AS timer
FROM your_table
LIMIT 0 , 30