MySQL variable in WHERE clause - mysql

I need a query that find the recommended TV shows for an user, based on the TV Shows he is following.
Do to this I have the following tables:
the table Progress that contains wich show the user is following and the percentage of seen episodes (to solve this problem we can assume I have only one user in the database)
the table Suggested that contains _id1,_id2 and value (value is the strength of the connections between the show with id=_id1 and the show with id=_id2: the more value is great, the more the shows have something in common).
Note that in this table applies the commutative property, so the strength of the connection between id1 and _id2 is the same of _id1 and _id2. Moreover there aren't two rows such as ROW1._id1=ROW2._id2 AND ROW1._id2 = ROW2._id1
the table ShowCache that contains the details about a TV Show, such as name etc..
The following query is what I'm trying to do, but the result is an empty set:
SET #a = 0; //In other tests this line seem to be necessary
SELECT `ShowCache`.*,
(SUM(value) * (Progress.progress)) as priority
FROM `Suggested`,`ShowCache`, Progress
WHERE
((_id2 = Progress.id AND _id1 NOT IN (SELECT id FROM Progress) AND #a:=_id1)//There is a best way to set a variable here?
OR
(_id1 = Progress.id AND _id2 NOT IN (SELECT id FROM Progress) AND #a:=_id2))
AND `ShowCache`._id = #a //I think that the query fails here
GROUP BY `ShowCache`._id
ORDER BY priority DESC
LIMIT 0,20
I know the problem is related to the usage of variables, but I can't solve it. Any help is really appreciated.
PS: the main problem is that (because of the commutative propriety), without variables I need two queries, wich takes about 3 secs to begin executed (the query is more complex than the above). I'm really trying to make a single query to do this task
PPS: I tied also with an XOR operation, that results in an infinite loop?!?!? Here's the WHERE clause I tried:
((_id2=Progress.id AND #a:=_id1) XOR (_id1=Progress.id AND #a:=_id2)) AND `ShowCache`._id = #a
EDIT:
I come up with this WHERE conditions without using any variable:
(_id2 = Progress.id OR _id1 = Progress.id)
AND `ShowCache`._id = IF(_id2 = Progress.id, _id1,_id2)
AND `ShowCache`._id NOT IN (SELECT id FROM Progress)
It works, but it is very slow.

Your attempt to use xor is clever. If you want to get the nonmatching value you want to use bitwise XOR which is ^
Progress.id ^_id1 ^ _id2
3 ^ 2 ^ 3 = 2
2 ^ 2 ^ 3 = 3
You can use this trick to setup a join and really simplify your query (eliminate the OR's and NOT IN's and do it in one query without variables.)
select users.name as username, showcache.name as show_name,
sum(progress * value) as priority from users
inner join progress on users.id = progress.user_id
inner join suggested on progress.show_id in (suggested.id_1, suggested.id_2)
inner join showcache on showcache.id =
(suggested.id_1 ^ suggested.id_2 ^ progress.show_id)
where showcache.id not in
(select show_id from progress where user_id = users.id)
group by showcache.id
order by priority desc;
I also setup a fiddle to demonstrate it:
http://sqlfiddle.com/#!2/2dcd8/24
To break it down. I created a users table with a single user (but the solution will work with multiple users.)
The select and join to progress is straightforward. The join to suggested uses IN as an alternative to writing it with OR
The join to showcache is where the bitwise XOR happens. One of the id's links up to the progress.show_id and we want to use the other one.
It does include a not in to exclude shows already watched from the results. I could have changed it to not exists? but it seems clearer this way.

You're setting #a's value twice within the where clause, meaning that the query is actually boiling down to:
...
WHERE ... AND `ShowCache`._id = _id2
MySQL evalutes variable assignments in a first-encountered order, so you should leave #a constant until the END of the clause, then assign a new value, e.g
mysql> set #a=5;
mysql> select #a, #a+1, #a*5, #a := #a + 1, #a;
+------+------+------+--------------+------+
| #a | #a+1 | #a*5 | #a := #a + 1 | #a |
+------+------+------+--------------+------+
| 0 | 1 | 0 | 1 | 1 |
| 1 | 2 | 5 | 2 | 2 |
| 2 | 3 | 10 | 3 | 3 |
+------+------+------+--------------+------+
Note that #a's value in the first 3 columns remains constant, UNTIL mysql reaches the #a := #a +1, after which #a has a new value
So perhaps your query should be
set #a = 0;
select #temp := #a, ..., #a := _id2
where
((_id2 = Progress.id AND _id1 NOT IN (SELECT id FROM Progress) AND #temp =_id1)
...
etc...

Related

How to fetch MySQL rows until the sum of a certain column exceeds the threshold?

I've been searching through StackOverflow and the closest one that I've found is to use custom variable inside subquery. But the suggested solution has two shortcomings.
Table
+----+-------+-------------+
| id | type | MyAmountCol |
+----+-------+-------------+
| 1 | 85482 | 10 |
+----+-------+-------------+
| 2 | 47228 | 20 |
+----+-------+-------------+
| 3 | 12026 | 40 |
+----+-------+-------------+
When every row has cannot meet the condition, (i.e. if every value is larger than the threshold value.) no row is returned.
Example Fiddle
Query
SET #runningTotal=0;
SELECT
O.Id,
O.Type,
O.MyAmountCol,
#runningTotal + O.MyAmountCol as 'RunningTotal',
#runningTotal := #runningTotal + O.MyAmountCol
FROM Table1 O
HAVING RunningTotal <=5;
Returned
0 Row(s)
When the condition is caught in the middle, (i.e. if the first two values are 10 and 20, and the threshold is 15) the sum of returned values are always less than or equal to the threshold.
Example Fiddle
Query
SET #runningTotal=0;
SELECT
O.Id,
O.Type,
O.MyAmountCol,
#runningTotal + O.MyAmountCol as 'RunningTotal',
#runningTotal := #runningTotal + O.MyAmountCol
FROM Table1 O
HAVING RunningTotal <=15;
Returned
1 85482 10 10 (1 Row)
The desired result is this. In the first example fiddle, I want the first row (id=1, type=85842) returned. In the second example fiddle, I want rows with id=1 , type=85842 and id=2, type=47228 returned.
Putting it differently, what I'm trying to do is slightly different from what I've found and it doesn't seem to achieve it with that approach. I want the fewest number of sequential rows that exceed the target value. Is there any way for this with only MySQL (query), or should I solve this in the application level?
JOIN the table to itself on lesser ids (because that's what you're ordering on), summing all the rows from the join and keeping those rows whose sum is less than (not less than or equal) the threshold:
SELECT
a.Id,
a.Type,
a.MyAmountCol
FROM Table1 a
LEFT JOIN Table1 b on b.id < a.id
GROUP BY 1,2,3
HAVING COALESCE(SUM(b.MyAmountCol), 0) < 15
The COALESCE() call is added to cater for the lowest id, which we want to keep (always), having no rows to join with.
Disclaimer: Code may not compile or work as it was thumbed in on my phone (but there's a reasonable chance it will work)

select one row multiple time when using IN()

I have this query :
select
name
from
provinces
WHERE
province_id IN(1,3,2,1)
ORDER BY FIELD(province_id, 1,3,2,1)
the Number of values in IN() are dynamic
How can I get all rows even duplicates ( in this example -> 1 ) with given ORDER BY ?
the result should be like this :
name1
name3
name2
name1
plus I shouldn't use UNION ALL :
select * from provinces WHERE province_id=1
UNION ALL
select * from provinces WHERE province_id=3
UNION ALL
select * from provinces WHERE province_id=2
UNION ALL
select * from provinces WHERE province_id=1
You need a helper table here. On SQL Server that can be something like:
SELECT name
FROM (Values (1),(3),(2),(1)) As list (id) --< List of values to join to as a table
INNER JOIN provinces ON province_id = list.id
Update: In MySQL Split Comma Separated String Into Temp Table can be used to split string parameter into a helper table.
To get the same row more than once you need to join in another table. I suggest to create, only once(!), a helper table. This table will just contain a series of natural numbers (1, 2, 3, 4, ... etc). Such a table can be useful for many other purposes.
Here is the script to create it:
create table seq (num int);
insert into seq values (1),(2),(3),(4),(5),(6),(7),(8);
insert into seq select num+8 from seq;
insert into seq select num+16 from seq;
insert into seq select num+32 from seq;
insert into seq select num+64 from seq;
/* continue doubling the number of records until you feel you have enough */
For the task at hand it is not necessary to add many records, as you only need to make sure you never have more repetitions in your in condition than in the above seq table. I guess 128 will be good enough, but feel free to double the number of records a few times more.
Once you have the above, you can write queries like this:
select province_id,
name,
#pos := instr(#in2 := insert(#in2, #pos+1, 1, '#'),
concat(',',province_id,',')) ord
from (select #in := '0,1,2,3,1,0', #in2 := #in, #pos := 10000) init
inner join provinces
on find_in_set(province_id, #in)
inner join seq
on num <= length(replace(#in, concat(',',province_id,','),
concat(',+',province_id,',')))-length(#in)
order by ord asc
Output for the sample data and sample in list:
| province_id | name | ord |
|-------------|--------|-----|
| 1 | name 1 | 2 |
| 2 | name 2 | 4 |
| 3 | name 3 | 6 |
| 1 | name 1 | 8 |
SQL Fiddle
How it works
You need to put the list of values in the assignment to the variable #in. For it to work, every valid id must be wrapped between commas, so that is why there is a dummy zero at the start and the end.
By joining in the seq table the result set can grow. The number of records joined in from seq for a particular provinces record is equal to the number of occurrences of the corresponding province_id in the list #in.
There is no out-of-the-box function to count the number of such occurrences, so the expression at the right of num <= may look a bit complex. But it just adds a character for every match in #in and checks how much the length grows by that action. That growth is the number of occurrences.
In the select clause the position of the province_id in the #in list is returned and used to order the result set, so it corresponds to the order in the #in list. In fact, the position is taken with reference to #in2, which is a copy of #in, but is allowed to change:
While this #pos is being calculated, the number at the previous found #pos in #in2 is destroyed with a # character, so the same province_id cannot be found again at the same position.
Its unclear exactly what you are wanting, but here's why its not working the way you want. The IN keyword is shorthand for creating a statement like ....Where province_id = 1 OR province_id = 2 OR province_id = 3 OR province_id = 1. Since province_id = 1 is evaluated as true at the beginning of that statement, it doesn't matter that it is included again later, it is already true. This has no bearing on whether the result returns a duplicate.

MySQL: Select first record, last record and 200 evenly spaced records in-between in table

I have a MySQL-table of values with 10.000+ records collected from a datalogging device.
The table has the following columns:
––––––––––––––––––––––––––––––
| ID | Time | par1 | par2 |
––––––––––––––––––––––––––––––
| 0 | .. | .. | .. |
––––––––––––––––––––––––––––––
| 1 | .. | .. | .. |
––––––––––––––––––––––––––––––
.. and so on.
The ID is auto-incrementing.
I would like to query the table for values to plot, so I'm not interested in selecting all 10.000+ records.
If it's possible, I would like select the first record, the last record and 200 evenly spaced records in-between with a single MySQL-query, so that the result can be passed to my plotting algorithm.
I have seen other similar solutions, where every n'th row starting at (1,2,3.. etc) is selected, like explained here:
How to select every nth row in mySQL starting at n
So far I have accomplished to select the first record + 200 evenly spaced records
set #row:=-1;
set #numrows := (SELECT COUNT(*) FROM mytable);
set #everyNthRow := FLOOR(#numrows / 200);
SELECT r.*
FROM (
SELECT *
FROM mytable
ORDER BY ID ASC
) r
CROSS
JOIN ( SELECT #i := 0 ) s
HAVING ( #i := #i + 1) MOD #everyNthRow = 1
But I can't figure out how to include the last record as well.
How can this be accomplished?
EDIT: Furthermore, I would like to check whether the table is actually containing more than 200 records, before applying the desired logic. If not, the select statement should just output every record (so that the first 200 entries of the datalogging-session will also appear).
Try expanding your HAVING clause to include the last row.
HAVING ( #i := #i + 1) MOD #everyNthRow = 1 OR #i = #numrows
You may need to experiment as it could be #i = #numrows - 1 that gives the right result.

Why is this only returning 1 row?

I am trying to query 2 tables with a join. I expect to get 2 rows but only get 1:
SELECT tmp.pk, tmp.domain, count(crawl.pk)
FROM (
SELECT * FROM domains
WHERE domain IN('www.google.com', 'www.yahoo.com')
AND pk < 10000
) tmp
JOIN crawl ON crawl.domain=tmp.pk
AND crawl.date_crawled <= 3
HAVING COUNT(crawl.pk) < 1000
Result:
+-------+--------------------+-----------------+
| pk | domain | count(crawl.pk) |
+-------+--------------------+-----------------+
| 14929 | www.yahoo.com | 88 |
+-------+--------------------+-----------------+
1 row in set (0.03 sec)
If I remove 'www.yahoo.com' from the IN statement then I get 'www.google.com' in the result (therefore, I know that both www.google.com and www.yahoo.com pass my criteria).
Why is it returning only 1 row, when it should be returning 2?
Don't know why you're using a sub-query. Try this one...
SELECT d.pk, d.domain, count(c.pk)
FROM domains d
INNER JOIN crawl c ON d.pk = c.domain
WHERE d.pk < 10000
AND d.domain in ('www.google.com', 'www.yahoo.com')
AND c.date_crawled <= 3
GROUP BY d.pk, d.domain
HAVING COUNT(c.pk) < 1000
If you're still having issues, I'd try removing the HAVING clause as well as the d.pk < 10000
This doesn't make sense because tmp.pk is suppose to be less than 10000 yet your example recordset shows tmp.pk being greater than 10000.
And, your join looks incorrect. Seems you are trying to join a string-based field with a numerical one. I am referring specifically to: crawl.domain=tmp.pk
I would suggest you try JOIN by tmp.domain.
Alternatively, I would recommend removing the subquery and restructuring your query to represent more what #Phil suggested in his answer.

How to use result of an subquery multiple times into an query

A MySQL query needs the results of a subquery in different places, like this:
SELECT COUNT(*),(SELECT hash FROM sets WHERE ID=1)
FROM sets
WHERE hash=(SELECT hash FROM sets WHERE ID=1)
and XD=2;
Is there a way to avoid the double execution of the subquery (SELECT hash FROM sets WHERE ID=1)?
The result of the subquery always returns an valid hash value.
It is important that the result of the main query also includes the HASH.
First I tried a JOIN like this:
SELECT COUNT(*), m.hash FROM sets s INNER JOIN sets AS m
WHERE s.hash=m.hash AND id=1 AND xd=2;
If XD=2 doesn't match a row, the result is:
+----------+------+
| count(*) | HASH |
+----------+------+
| 0 | NULL |
+----------+------+
Instead of something like (what I need):
+----------+------+
| count(*) | HASH |
+----------+------+
| 0 | 8115e|
+----------+------+
Any ideas? Please let me know! Thank you in advance for any help.
//Edit:
finally that query only has to count all the entries in an table which has the same hash value like the entry with ID=1 and where XD=2. If no rows matches that (this case happend if XD is set to an other number), so return 0 and simply hash value.
SELECT SUM(xd = 2), hash
FROM sets
WHERE id = 1
If id is a PRIMARY KEY (which I assume it is since your are using a single-record query against it), then you can just drop the SUM:
SELECT xd = 2 AS cnt, hash
FROM sets
WHERE id = 1
Update:
Sorry, got your task wrong.
Try this:
SELECT si.hash, COUNT(so.hash)
FROM sets si
LEFT JOIN
sets so
ON so.hash = si.hash
AND so.xd = 2
WHERE si.id = 1
I normally nest the statements like the following
SELECT Count(ResultA.Hash2) AS Hash2Count,
ResultA.Hash1
FROM (SELECT S.Hash AS Hash2,
(SELECT s2.hash
FROM sets AS s2
WHERE s2.ID = 1) AS Hash1
FROM sets AS S
WHERE S.XD = 2) AS ResultA
WHERE ResultA.Hash2 = ResultA.Hash1
GROUP BY ResultA.Hash1
(this one is hand typed and not tested but you should get the point)
Hash1 is your subquery, once its nested, you can reference it by its alias in the outer query. It makes the query a little larger but I don't see that as a biggy.
If I understand correctly what you are trying to get, query should look like this:
select count(case xd when 2 then 1 else null end case), hash from sets where id = 1 group by hash
I agree with the other answers, that the GROUP BY may be better, but to answer the question as posed, here's how to eliminate the repetition:
SELECT COUNT(*), h.hash
FROM sets, (SELECT hash FROM sets WHERE ID=1) h
WHERE sets.hash=h.hash
and sets.ID=1 and sets.XD=2;