I have a MySQL database with a structure like this...
Site has many Sensors
Sensors has many SensorReadings
I want to get all Sensors for a Site and the last 5 SensorReadings for all those Sensors. I suspect I'm going to have to do something with a stored procedure and temporary tables (if they even exist in MySQL.
Possibly Something like...
SELECT reading,
date
FROM (select sensor_id,
reading,
date,
#num := if(#sensor_id = sensor_id, #num + 1, 1) as row_number,
#sensor_id := sensor_id as dummy
from sensor_readings
order by sensor_id,
date desc) T
WHERE row_number<=5
Please give your actual table structure(s) in your question.
Full example using MySQL variables. For brevity, this displays the top 2 readings per sensor.
drop table if exists Sensors;
create table Sensors (Id int);
insert Sensors (id) values (1), (2), (3);
drop table if exists SensorReadings;
create table SensorReadings (SensorId int, RecordDate date);
insert SensorReadings (SensorId, RecordDate) values
(1, '2011-01-01'),
(1, '2011-01-02'),
(1, '2011-01-03'),
(2, '2011-01-01'),
(2, '2011-01-02'),
(2, '2011-01-03');
set #num = -1;
set #SensorId = -1;
select *
from Sensors s
join (
select *
, #num := if(#SensorId = SensorId, #num + 1, 1) as rn
, #SensorId := SensorId
from SensorReadings sr
order by
SensorId
, RecordDate desc
) as numbered
on numbered.SensorId = s.Id
where numbered.rn < 3;
Based on Andomar dump
select * from SensorReadings as t1
where (select count(*) from SensorReadings as t2
where t1.sensorid = t2.sensorid and t2.recordDate > t1.recordDate) <2
Related
I'm trying in MySql to count the number of users created each day and then get an accumulative figure on a row by row basis. I have followed other suggestions on here, but I cannot seem to get the accumulation to be correct.
The problem is that it keeps counting from the base number of 200 and not taking account of previous rows.
Where was I would expect it to return
My Sql is as follows;
SELECT day(created_at), count(*), (#something := #something+count(*)) as value
FROM myTable
CROSS JOIN (SELECT #something := 200) r
GROUP BY day(created_at);
To create the table and populate it you can use;
CREATE TABLE myTable (
id INT AUTO_INCREMENT,
created_at DATETIME,
PRIMARY KEY (id)
);
INSERT INTO myTable (created_at)
VALUES ('2018-04-01'),
('2018-04-01'),
('2018-04-01'),
('2018-04-01'),
('2018-04-02'),
('2018-04-02'),
('2018-04-02'),
('2018-04-03'),
('2018-04-03');
You can view this on SqlFiddle.
Use a subquery:
SELECT day, cnt, (#s := #s + cnt)
FROM (SELECT day(created_at) as day, count(*) as cnt
FROM myTable
GROUP BY day(created_at)
) d CROSS JOIN
(SELECT #s := 0) r;
GROUP BY and variables have not worked together for a long time. In more recent versions, ORDER BY also needs a subquery.
I am working on a scraping project to crawl items and their scores over different schedules.Schedule is a user defined period (date) when the script is intended to run.
Table structure is as follows:
--
-- Table structure for table `test_join`
--
CREATE TABLE IF NOT EXISTS `test_join` (
`schedule_id` int(11) NOT NULL,
`player_name` varchar(50) NOT NULL,
`type` enum('celebrity','sportsperson') NOT NULL,
`score` int(11) NOT NULL,
PRIMARY KEY (`schedule_id`,`player_name`,`type`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
--
-- Dumping data for table `test_join`
--
INSERT INTO `test_join` (`schedule_id`, `player_name`, `type`, `score`) VALUES
(1, 'sachin', 'sportsperson', 100),
(1, 'ganguly', 'sportsperson', 80),
(1, 'dravid', 'sportsperson', 60),
(1, 'sachin', 'celebrity', 100),
(2, 'sachin', 'sportsperson', 120),
(2, 'ganguly', 'sportsperson', 100),
(2, 'sachin', 'celebrity', 120);
The scraping is done over periods and for each schedule it is expected to have about 10k+ entries.The schedules could be made in daily basis,hence the data would grow to be be around 2 million in 5-6 months.
Over this data I need to perform queries to aggregate the player who come across each schedules in a selected range of schedules.
For example:
I need aggregate same players who come across multiple schedules. If schedule 1 and 2 are selected,items which come under both of the schedules only will be selected.
I am using the following query to aggregate results based on the type,
For schedule 1:
SELECT fullt.type,COUNT(*) as count,SUM(fullt.score) FROM
(SELECT tj.*
FROM `test_join` tj
RIGHT JOIN
(SELECT `player_name`,`type`,COUNT(`schedule_id`) as c FROM `test_join` WHERE `schedule_id` IN (1,2) GROUP BY `player_name`,`type` HAVING c=2) stj
on tj.player_name = stj.player_name
WHERE tj.`schedule_id`=1
GROUP BY tj.`type`,tj.`player_name`)AS fullt
GROUP BY fullt.type
Reason for c = 2;
WHERE `schedule_id` IN (1,2) GROUP BY `player_name`,`type` HAVING c=2
Here we are selecting two schedules,1 and 2.Hence the count 2 is taken to make the query to to fetch records which belongs to both the schedules and occurs twice.
It would generate a results as follows,
Schedule 1 :Expected Results
Schedule 2 :Expected Results
This is my expected result and the query returns the results as above.
(In the real case I have to work across pretty big MySQL tables)
On my understanding of standardized MySQL queries, using sub queries,WHERE IN, varchar comparison fields ,multiple GROUP BY's would affect in the query performance.
I need the aggregate results in real time and query speed and well as standards are a concern too.How this could be optimized for better performance in this context.
EDIT:
I had reduced sub queries now:
SELECT fullt.type,COUNT(*) as count,SUM(fullt.score) FROM (
SELECT t.*
FROM `test_join` t
INNER JOIN test_join t1 ON t.`player_name` = t1.player_name AND t1.schedule_id = 1
INNER JOIN test_join t2 ON t.player_name = t2.player_name AND t2.schedule_id = 2
WHERE t.schedule_id = 2
GROUP BY t.`player_name`,t.`type`) AS fullt
GROUP BY fullt.type
Is this a better way to do so.I had replaced WHERE IN with JOINS.
Any advise would be highly appreciated.I would be happy to provide any supporting information if needed.
try below SQL Query in MYSQL:
SELECT tj.`type`,COUNT(*) as count,SUM(tj.`score`) FROM
`test_join` tj
where tj.`schedule_id`=1
and `player_name` in
(
select tj1.`player_name` from `test_join` tj1
group by tj1.`player_name` having count(tj1.`player_name`) > 1
)
group by tj.`type`
Actuallly I tried same data in Sybase as i dont have MySQL installed in my machine.It worked as exepected !
CREATE TABLE #test_join
(
schedule_id int NOT NULL,
player_name varchar(50) NOT NULL,
type1 varchar(15) NOT NULL,
score int NOT NULL,
)
INSERT INTO #test_join (schedule_id, player_name, type1, score) VALUES
(1, 'sachin', 'sportsperson', 100)
INSERT INTO #test_join (schedule_id, player_name, type1, score) VALUES(1, 'ganguly', 'sportsperson', 80)
INSERT INTO #test_join (schedule_id, player_name, type1, score) VALUES(1, 'dravid', 'sportsperson', 60)
INSERT INTO #test_join (schedule_id, player_name, type1, score) VALUES(1, 'sachin', 'celebrity', 100)
INSERT INTO #test_join (schedule_id, player_name, type1, score) VALUES(2, 'sachin', 'sportsperson', 120)
INSERT INTO #test_join (schedule_id, player_name, type1, score) VALUES(2, 'ganguly', 'sportsperson', 100)
INSERT INTO #test_join (schedule_id, player_name, type1, score) VALUES(2, 'sachin', 'celebrity', 120)
select * from #test_join
Print 'Solution #1 : Inner join'
select type1,count(*),sum(score) from
#test_join
where schedule_id=1 and player_name in (select player_name from #test_join t1 group by player_name having count(player_name) > 1 )
group by type1
select player_name,type1,sum(score) Score into #test_join_temp
from #test_join
group by player_name,type1
having count(player_name) > 1
Print 'Solution #2 using Temp Table'
--select * from #test_join_temp
select type1,count(*),sum(score) from
#test_join
where schedule_id=1 and player_name in (select player_name from #test_join_temp )
group by type1
I hope This Helps :)
Any SQL experts out there who can explain how I select say the top 3 items. When the 3rd item is equal to the 4th and 5th I'd like those included too but only in that situation. e.g. with the following list
40,
30,
15,
15,
15,
10
it would return 40,30,15,15,15 not 40,30,15.
If you want to output only the score field, you can do it like this:
SET #ranks = 3;
PREPARE stmt_top3 FROM
'SELECT score FROM (
SELECT score, (#row := #row + 1), IF (#row = ?, #min_score := score, NULL)
FROM user_score
WHERE score IN (
SELECT top_score.score FROM (
SELECT DISTINCT score,
(#row := 0), (#min_score := 0)
FROM user_score
ORDER BY score DESC
LIMIT ?
) AS top_score
)
ORDER BY score DESC
) AS score_rows
WHERE score >= #min_score
ORDER BY score DESC';
EXECUTE stmt_top3 USING #ranks, #ranks;
DEMO # SQL Fiddle
I used a prepared statement, so the number of ranks to show is flexible. If you don't want that, just hardcode a 3 instead of the 2 ?.
Otherwise, if you also need the rank and the user_id, i.e., here's a more extensive approach:
Highscore-Like Ranking (Shared Ranks)
Selects all rows that share the first rank, no matter how many there are.
As long as the number of these rows is not greater than or equal to the maximum of ranks allowed (3 in your example) the next rank is the number of rows + 1.
And everything begins from the start and so on ...
SET #ranks = 3;
PREPARE stmt_top3 FROM
'SELECT user_score.user_id, score_rank.rank, score_rank.score
FROM user_score
INNER JOIN (
SELECT (#last_rank := #last_rank + #last_equal_score) AS rank,
score, (#last_equal_score := count(score)) AS equal_score
FROM user_score
WHERE score IN (
SELECT top_score.score FROM (
SELECT DISTINCT score,
(#last_rank := 1), (#last_equal_score := 0)
FROM user_score
ORDER BY score DESC
LIMIT ?
) AS top_score
)
GROUP BY score
ORDER BY score DESC
) AS score_rank
ON user_score.score = score_rank.score
WHERE score_rank.rank <= ?
ORDER BY score_rank.rank ASC';
EXECUTE stmt_top3 USING #ranks, #ranks;
DEMO # SQL Fiddle
Due to the fact that MySQL does not support LIMIT in subqueries for certain subquery operators such as IN, you have to wrap your subquery that contains a LIMIT clause in another simple subquery to avoid the following error:
ERROR 1235 (42000): This version of MySQL doesn't yet support 'LIMIT
& IN/ALL/ANY/SOME subquery'
Restrictions on Subqueries
I used the following setup for testing:
CREATE TABLE user (
id INT UNSIGNED AUTO_INCREMENT PRIMARY KEY
) ENGINE=InnoDB;
INSERT INTO user (id) VALUES
(NULL),
(NULL),
(NULL),
(NULL),
(NULL),
(NULL);
CREATE TABLE user_score (
user_id INT UNSIGNED NOT NULL UNIQUE,
score INT NOT NULL DEFAULT 0,
INDEX (score),
FOREIGN KEY (user_id) REFERENCES user (id)
) ENGINE=InnoDB;
INSERT INTO user_score (user_id, score) VALUES
(1, 40),
(2, 30),
(3, 15),
(4, 15),
(5, 15),
(6, 10);
SELECT *
FROM myTable
WHERE Field1 IN(
SELECT *
FROM (
SELECT DISTINCT Field1
FROM myTable
ORDER BY Field1 DESC
LIMIT 3)
) t
The only issue you might have from your example is if you specifically DON'T want to include duplicates of 40 & 30?
Let's say I was looking for the second most highest record.
Sample Table:
CREATE TABLE `my_table` (
`id` int(2) NOT NULL AUTO_INCREMENT,
`name` varchar(255) NOT NULL,
`value` int(10),
PRIMARY KEY (`id`)
);
INSERT INTO `my_table` (`id`, `name`, `value`) VALUES (NULL, 'foo', '200'), (NULL, 'bar', '100'), (NULL, 'baz', '0'), (NULL, 'quux', '300');
The second highest value is foo. How many ways can you get this result?
The obvious example is:
SELECT name FROM my_table ORDER BY value DESC LIMIT 1 OFFSET 1;
Can you think of other examples?
I was trying this one, but LIMIT & IN/ALL/ANY/SOME subquery is not supported.
SELECT name FROM my_table WHERE value IN (
SELECT MIN(value) FROM my_table ORDER BY value DESC LIMIT 1
) LIMIT 1;
Eduardo's solution in standard SQL
select *
from (
select id,
name,
value,
row_number() over (order by value) as rn
from my_table t
) t
where rn = 1 -- can pick any row using this
This works on any modern DBMS except MySQL. This solution is usually faster than solutions using sub-selects. It also can easily return the 2nd, 3rd, ... row (again this is achievable with Eduardo's solution as well).
It can also be adjusted to count by groups (adding a partition by) so the "greatest-n-per-group" problem can be solved with the same pattern.
Here is a SQLFiddle to play around with: http://sqlfiddle.com/#!12/286d0/1
This only works for exactly the second highest:
SELECT * FROM my_table two
WHERE EXISTS (
SELECT * FROM my_table one
WHERE one.value > two.value
AND NOT EXISTS (
SELECT * FROM my_table zero
WHERE zero.value > one.value
)
)
LIMIT 1
;
This one emulates a window function rank() for platforms that don't have them. It can also be adapted for ranks <> 2 by altering one constant:
SELECT one.*
-- , 1+COALESCE(agg.rnk,0) AS rnk
FROM my_table one
LEFT JOIN (
SELECT one.id , COUNT(*) AS rnk
FROM my_table one
JOIN my_table cnt ON cnt.value > one.value
GROUP BY one.id
) agg ON agg.id = one.id
WHERE agg.rnk=1 -- the aggregate starts counting at zero
;
Both solutions need functional self-joins (I don't know if mysql allows them, IIRC it only disallows them if the table is the target for updates or deletes)
The below one does not need window functions, but uses a recursive query to enumerate the rankings:
WITH RECURSIVE agg AS (
SELECT one.id
, one.value
, 1 AS rnk
FROM my_table one
WHERE NOT EXISTS (
SELECT * FROM my_table zero
WHERE zero.value > one.value
)
UNION ALL
SELECT two.id
, two.value
, agg.rnk+1 AS rnk
FROM my_table two
JOIN agg ON two.value < agg.value
WHERE NOT EXISTS (
SELECT * FROM my_table nx
WHERE nx.value > two.value
AND nx.value < agg.value
)
)
SELECT * FROM agg
WHERE rnk = 2
;
(the recursive query will not work in mysql, obviously)
You can use inline initialization like this:
select * from (
select id,
name,
value,
#curRank := #curRank + 1 AS rank
from my_table t, (SELECT #curRank := 0) r
order by value desc
) tb
where tb.rank = 2
SELECT name
FROM my_table
WHERE value < (SELECT max(value) FROM my_table)
ORDER BY value DESC
LIMIT 1
SELECT name
FROM my_table
WHERE value = (
SELECT min(r.value)
FROM (
SELECT name, value
FROM my_table
ORDER BY value DESC
LIMIT 2
) r
)
LIMIT 1
I'm trying to insert different numbers of rows into my table by adjusting #s:=1 and #s<1000 but it only inserts one at a time. What am I doing wrong?
INSERT INTO
rent
(
id
)
select #s:=#s+1 as seq
FROM (SELECT #s:=1) AS baseview, rent
WHERE #s<1000
ORDER by #s ASC;
Can you try this?:
INSERT INTO
rent
(
id
)
SELECT seq FROM (
select #s:=#s+1 as seq
FROM (SELECT #s:=1) AS baseview, rent
WHERE #s<1000
ORDER by #s ASC
) seqSource;