select from tables with different numbers of rows - mysql

I'm hoping there is a simple answer to this. Competitors race over a series of 3 races. Some competitors only show up for one race. How could I show a final result for ALL competitors?
race 1
+------+--------+
| name | result |
+------+--------+
| Ali | 30 |
| Bob | 28 |
| Cal | 26 |
+------+--------+
race 2
+------+--------+
| name | result |
+------+--------+
| Ali | 32 |
| Bob | 31 |
| Dan | 24 |
+------+--------+
race 3
+------+--------+
| name | result |
+------+--------+
| Eva | 23 |
| Dan | 25 |
+------+--------+
The final result should look like this:
+------+--------+--------+--------+
| name | result | result | result |
+------+--------+--------+--------+
| Ali | 30 | 32 | |
| Bob | 28 | 31 | |
| Cal | 26 | | |
| Dan | | 24 | 25 |
| Eva | | | 23 |
+------+--------+--------+--------+
The problem I have is with ordering by name from multiple tables.
Here is the example data:
CREATE TABLE race (name varchar(20), result int);
CREATE TABLE race1 LIKE race;
INSERT INTO race1 VALUES ('Ali', '30'), ('Bob', '28'), ('Cal', '26');
CREATE TABLE race2 like race;
insert INTO race2 VALUES ('Ali', '32'), ('Bob', '31'), ('Dan', '24');
CREATE TABLE race3 LIKE race;
INSERT INTO race3 VALUES ('Eva', '23'), ('Dan', '25');
Many thanks!

Here we go !!!
select race1.name as name, race1.result, race2.result, race3.result from race1
left join race2 on race2.name = race1.name
left join race3 on race3.name = race1.name
union
select race2.name as name, race1.result, race2.result, race3.result from race2
left join race1 on race1.name = race2.name
left join race3 on race3.name = race2.name
union
select race3.name as name, race1.result, race2.result, race3.result from race3
left join race1 on race1.name = race3.name
left join race2 on race2.name = race3.name;
It is working :)

select s.name,
max(case when s.R = 'Result1' then s.result else '' end) as result1,
max(case when s.R = 'Result2' then s.result else '' end) as result2,
max(case when s.R = 'Result3' then s.result else '' end) as result3
from
(
select 'Result1' as R,r1.* from race1 r1
union all
select 'Result2' as R,r2.* from race2 r2
union all
select 'Result3' as R,r3.* from race3 r3
) s
group by s.name
result
+------+---------+---------+---------+
| name | result1 | result2 | result3 |
+------+---------+---------+---------+
| Ali | 30 | 32 | |
| Bob | 28 | 31 | |
| Cal | 26 | | |
| Dan | | 24 | 25 |
| Eva | | | 23 |
+------+---------+---------+---------+
5 rows in set (0.00 sec)

I personally would create the schema in a different way.
One table for the users, one for the races and one that connects both:
-- Create syntax for TABLE 'races'
CREATE TABLE `races` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(255) DEFAULT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
-- Create syntax for TABLE 'users'
CREATE TABLE `users` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(255) DEFAULT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
-- Create syntax for TABLE 'race_results'
CREATE TABLE `race_results` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`race_id` int(11) NOT NULL,
`user_id` int(11) NOT NULL,
`result` int(11) NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Let's insert some data (should be equal to your data set).
-- Insert data
INSERT INTO users (name)values('Ali'),('Bob'),('Cal'),('Dan'), ('Eva');
INSERT INTO races (name)values('Race1'),('Race2'),('Race3');
INSERT INTO race_results (user_id, race_id, result)values(1,1,30),(2,1,30),(1,2,28),(2,2,31),(3,1,26),(4,2,24),(4,3,25),(5,3,23);
Then you could write the query like this:
-- Static version
SELECT us.name, sum(if(ra.name='Race1', result, null)) as Race1, sum(if(ra.name='Race2', result, null)) as Race2, sum(if(ra.name='Race3', result, null)) as Race3
FROM race_results as rr
LEFT JOIN users as us on us.id = rr.user_id
LEFT JOIN races as ra on ra.id = rr.race_id
GROUP BY us.id;
Which gives you the result you're looking for. (I changed the column names to make it more obvious which result belongs to which race.)
But I've to admit that this works fine for 3 races but what if you have 30 or more?
Here is a more dynamic version of the above query, which kind of creates itself ;)
-- Dynamic version
SET #sql = '';
SELECT
#sql := CONCAT(#sql,if(#sql='','',', '),temp.output)
FROM
(SELECT
CONCAT("sum(if(ra.name='", race.name, "', result, null)) as ", race.name) as output
FROM races as race
) as temp;
SET #sql = CONCAT("SELECT us.name,", #sql, " FROM race_results as rr LEFT JOIN users as us on us.id = rr.user_id LEFT JOIN races as ra on ra.id = rr.race_id GROUP BY 1;");
SELECT #sql;
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;

Related

Group by Query - Select row with maximum date

Hi I have this scores table, And in my report on front end, I have to display the keyword and url and score for latest scan.
CREATE TABLE `scores` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`keyword` varchar(200) DEFAULT NULL,
`url` varchar(200) DEFAULT NULL,
`score` int(11) DEFAULT NULL,
`check_date` date DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=latin1;
Here is my Example Data:
Sample Data
| id | keyword | url | score | check_date |
|----|----------|----------------------|-------|------------|
| 1 | facebook | https://facebook.com | 10 | 2020-10-21 |
| 2 | facebook | https://facebook.com | 30 | 2020-10-25 |
| 3 | fb | https://facebook.com | 55 | 2020-10-23 |
| 4 | fb | https://facebook.com | 20 | 2020-10-24 |
My Query
SELECT s1.*
FROM scores s1
JOIN scores s2
ON s1.id = s2.id
WHERE s1.check_date = s2.check_date
GROUP BY keyword,url
It returns correct check_date for a specific keyword, url but score is not according to that date. Please help.
Do not use aggregation for this. A simple method is a correlated subquery:
select s.*
from scores s
where s.check_date = (select max(s2.check_date)
from scores s2
where s2.keyword = s.keyword and s2.url = s.url
);
If you are intent on using an explicit join you can use a left join, look for a larger date, and return the rows that have no larger date:
select s.*
from scores s left join
scores slater
on slater.keyword = s.keyword and
slater.url = s.url and
slater.check_date > s.check_date
where slater.check_date is null;

SQL JOIN : Prefix fields with table name

I have the following tables
CREATE TABLE `constraints` (
`id` int(11),
`name` varchar(64),
`type` varchar(64)
);
CREATE TABLE `groups` (
`id` int(11),
`name` varchar(64)
);
CREATE TABLE `constraints_to_group` (
`groupid` int(11),
`constraintid` int(11)
);
With the following data :
INSERT INTO `groups` (`id`, `name`) VALUES
(1, 'group1'),
(2, 'group2');
INSERT INTO `constraints` (`id`, `name`, `type`) VALUES
(1, 'cons1', 'eq'),
(2, 'cons2', 'inf');
INSERT INTO `constraints_to_group` (`groupid`, `constraintid`) VALUES
(1, 1),
(1, 2),
(2, 2);
I want to get all constraints for all groups, so I do the following :
SELECT groups.*, t.* FROM groups
LEFT JOIN
(SELECT * FROM constraints
LEFT JOIN constraints_to_group
ON constraints.id=constraints_to_group.constraintid) as t
ON t.groupid=groups.id
And get the following result :
id| name | id | name type groupid constraintid
-----------------------------------------------------
1 | group1 | 1 | cons1 | eq | 1 | 1
1 | group1 | 2 | cons2 | inf | 1 | 2
2 | group2 | 2 | cons2 | inf | 2 | 2
What I'd like to get :
group_id | group_name | cons_id | cons_name | cons_type | groupid | constraintid
-------------------------------------------------------------------------------------
1 | group1 | 1 | cons1 | eq | 1 | 1
1 | group1 | 2 | cons2 | inf | 1 | 2
2 | group2 | 2 | cons2 | inf | 2 | 2
This is an example, in my real case my tables have much more columns so using the SELECT groups.name as group_name, ... would lead to queries very hard to maintains.
Try this way
SELECT groups.id as group_id, groups.name as group_name ,
t.id as cons_id, t.name as cons_name, t.type as cons_type,
a.groupid , a.constraintid
FROM constraints_to_group as a
JOIN groups on groups.id=a.groupid
JOIN constraints as t on t.id=a.constraintid
The only difference I see are the names of the columns? Use for that mather an AS-statement.
SELECT
groups.id AS group_id,
groups.name AS group_name,
t.id AS cons_id,
t.name AS cons_name,
t.groupid, t.constraintid
FROM groups
LEFT JOIN
(SELECT * FROM constraints
LEFT JOIN constraints_to_group
ON constraints.id=constraints_to_group.constraintid) as t
ON t.groupid=groups.id
Besides, a better join-construction is:
SELECT G.id AS group_id,
G.name AS group_name,
CG.id AS cons_id,
CG.name AS cons_name,
C.groupid, C.constraintid
FROM constraints_to_group CG
LEFT JOIN constraints C
ON CG.constraintid = C.id
LEFT JOIN groups G
ON CG.groupid = G.id;
Possible duplicate of this issue

Can a sql query perform this job?

Here is my schema:
CREATE TABLE `tbltransactions` (
`transactionid` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`transactiondate` datetime DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`customerid` bigint(20) unsigned NOT NULL,
`transactiondetail` varchar(255) NOT NULL,
`transactionamount` decimal(10,2) NOT NULL,
UNIQUE KEY `transactionid` (`transactionid`),
KEY `customerid` (`customerid`),
CONSTRAINT `tbltransactions_ibfk_1` FOREIGN KEY (`customerid`) REFERENCES `tblcustomers` (`customerid`)
) ENGINE=InnoDB AUTO_INCREMENT=240 DEFAULT CHARSET=latin1;
transactionamount contains positive values for purchases and negative values for payments.
I wish I could list all records onwards from tbltransactions where customer's balance was zero at the latest. Any help?
EDIT: Please consider this dataset:
purchase 10
payment -10 // balance is zero
purchase 5
payment -5 // balance again zero
purchase 7 // show this transaction and onwards
purchase 2
payment -5 // show this also since balance is not zero
EDIT: sample of actual data:
INSERT INTO `tbltransactions` VALUES (1,'2014-06-22 22:51:00',39,'Balance when computerized',8851.00),(2,'2014-06-22 22:55:05',35,'Balance when computerized',5395.00),(3,'2014-06-22 22:56:17',53,'Balance when computerized',60.00),(4,'2014-06-22 22:57:15',54,'Balance when computerized',2671.00),(5,'2014-06-22 22:57:41',55,'Balance when computerized',1586.00),(6,'2014-06-22 22:58:34',61,'Balance when computerized',50.00),(7,'2014-06-22 22:59:22',56,'Balance when computerized',344.00),(8,'2014-06-22 22:59:42',71,'Balance when computerized',650.00),(9,'2014-06-22 23:01:10',63,'Balance when computerized',1573.00),(10,'2014-06-22 23:01:51',32,'Balance when computerized',7515.00),(11,'2014-06-22 23:02:22',72,'Balance when computerized',466.00),(12,'2014-06-22 23:03:10',64,'Balance when computerized',4774.00),(13,'2014-06-22 23:03:32',42,'Balance when computerized',2992.00),(14,'2014-06-22 23:05:24',41,'Balance when computerized',2218.00),(15,'2014-06-22 23:05:39',40,'Balance when computerized',7149.00),(16,'2014-06-22 23:06:25',80,'Balance when computerized',2607.00),(17,'2014-06-22 23:09:18',67,'Balance when computerized',357.00),(18,'2014-06-22 23:20:39',10,'Balance when computerized',677.00),(19,'2014-06-22 23:13:17',57,'Balance when computerized',135.00),(20,'2014-06-22 23:13:47',58,'Balance when computerized',5872.00),(21,'2014-06-24 11:36:10',73,'Balance when computerized',355.00),(22,'2014-06-22 23:14:30',74,'Balance when computerized',173.00),(23,'2014-06-22 23:16:45',59,'Balance when computerized',995.00),(24,'2014-06-22 23:17:44',19,'Balance when computerized',1704.00),(25,'2014-06-22 23:19:00',23,'Balance when computerized',690.00),(26,'2014-06-22 23:21:17',34,'Balance when computerized',10331.00),(27,'2014-06-22 23:21:43',38,'Balance when computerized',495.00),(28,'2014-06-22 23:22:01',65,'Balance when computerized',6676.00),(29,'2014-06-22 23:23:31',8,'Balance when computerized',4148.00),(30,'2014-06-22 23:23:53',24,'Balance when computerized',3124.00),(31,'2014-06-22 23:27:02',68,'Balance when computerized',3364.00),(35,'2014-06-22 23:35:22',46,'Balance when computerized',19105.00),(36,'2014-06-22 23:36:26',36,'Balance when computerized',2471.00),(37,'2014-06-22 23:36:42',60,'Balance when computerized',910.00),(38,'2014-06-22 23:37:11',75,'Balance when computerized',5203.00),(39,'2014-06-22 23:37:29',77,'Balance when computerized',2342.00),(40,'2014-06-22 23:37:42',13,'Balance when computerized',4555.00),(41,'2014-06-22 23:38:24',62,'Balance when computerized',271.00),(42,'2014-06-22 23:42:43',26,'Balance when computerized',5040.00),(43,'2014-06-22 23:43:13',33,'Balance when computerized',6792.00),(44,'2014-06-22 23:43:57',9,'Balance when computerized',1101.00),(45,'2014-06-22 23:44:27',21,'Balance when computerized',1010.00),(46,'2014-06-22 23:45:16',69,'Balance when computerized',89.00),(47,'2014-06-22 23:45:52',81,'Balance when computerized',220.00),(48,'2014-06-22 23:46:37',82,'Balance when computerized',205.00),(49,'2014-06-22 23:47:26',83,'Balance when computerized',731.00),(50,'2014-06-22 23:48:00',84,'Balance when computerized',155.00),(51,'2014-06-22 23:48:54',5,'Balance when computerized',475.00),(52,'2014-06-22 23:50:13',85,'Balance when computerized',1375.00),(53,'2014-06-22 23:51:04',86,'Balance when computerized',28.00),(54,'2014-06-22 23:51:39',87,'Balance when computerized',26.00),(55,'2014-06-22 23:52:23',88,'Balance when computerized',30.00),(56,'2014-06-22 23:52:53',89,'Balance when computerized',45.00),(57,'2014-06-22 23:53:23',90,'Balance when computerized',140.00),(58,'2014-06-22 23:54:13',91,'Balance when computerized',40.00),(59,'2014-06-22 23:55:38',93,'Balance when computerized',3350.00),(60,'2014-06-22 23:57:13',3,'Balance when computerized',60.00),(61,'2014-06-22 23:59:05',94,'Balance when computerized',3372.00),(62,'2014-06-23 00:00:12',20,'Balance when computerized',562.00),(63,'2014-06-23 00:00:48',18,'Balance when computerized',3227.00),(64,'2014-06-23 00:01:26',7,'Balance when computerized',1023.00),(65,'2014-06-23 00:01:46',29,'Balance when computerized',20.00),(66,'2014-06-23 00:02:57',15,'Balance when computerized',160.00),(67,'2014-06-23 00:04:14',11,'Balance when computerized',345.00),(68,'2014-06-23 00:04:50',31,'Balance when computerized',45.00),(69,'2014-06-23 00:08:45',50,'Balance when computerized',50.00),(70,'2014-06-23 00:09:05',6,'Balance when computerized',2880.00),(71,'2014-06-23 00:11:29',96,'Balance when computerized',1300.00),(72,'2014-06-23 00:12:40',4,'Balance when computerized',601.00),(74,'2014-06-24 10:21:26',97,'Balance when computerized',1250.00),(76,'2014-06-24 10:35:31',32,'1.5 ltr etc.',510.00),(77,'2014-06-24 15:04:13',97,'parchi',535.00),(78,'2014-06-24 15:05:51',32,'parchi',400.00),(79,'2014-06-24 15:08:08',32,'parchi',1924.00),(80,'2014-06-24 15:14:38',35,'suger berd',840.00),(81,'2014-06-24 15:16:49',39,'bottel',85.00),(82,'2014-06-24 15:21:51',20,'salt tusho',250.00),(83,'2014-06-24 15:23:49',26,'eggs',45.00),(84,'2014-06-24 15:24:54',38,'waldah',200.00),(85,'2014-06-24 15:26:12',78,'Balance when computerized',1557.00),(86,'2014-06-24 15:27:12',78,'haldi',70.00),(87,'2014-06-24 15:28:37',68,'eggs butter',87.00),(88,'2014-06-24 15:30:19',98,'Balance when computerized',550.00),(89,'2014-06-24 15:32:13',44,'2 coke',50.00),(90,'2014-06-24 15:33:05',81,'self',-220.00),(91,'2014-06-24 15:33:52',46,'razor',30.00),(92,'2014-06-24 15:34:37',75,'dues',40.00),(93,'2014-06-24 15:35:35',9,'oil ghee',625.00),(94,'2014-06-24 15:36:57',99,'bread',93.00),(95,'2014-06-24 15:38:14',100,'bottle razor',55.00),(96,'2014-06-24 15:38:54',7,'dues',40.00),(97,'2014-06-24 15:39:41',75,'ltr',60.00),(98,'2014-06-24 15:40:08',69,'1.5 ltr',60.00),(99,'2014-06-24 15:40:27',42,'2 1.5 ltr',120.00),(100,'2014-06-24 15:42:02',26,'bread bottle',110.00),(101,'2014-06-24 15:45:39',78,'saman',140.00),(102,'2014-06-26 15:19:20',101,'Oil dues',105.00),(103,'2014-06-26 15:19:59',26,'bread etc',55.00),(104,'2014-06-26 15:20:15',97,'parchi',290.00),(105,'2014-06-26 15:20:33',35,'parchi',355.00),(106,'2014-06-26 15:20:46',81,'bread',100.00),(107,'2014-06-26 15:21:26',102,'razor',40.00),(108,'2014-06-26 15:22:51',38,'dues',30.00),(109,'2014-06-26 15:23:35',20,'register, bottle',275.00),(110,'2014-06-26 15:23:55',46,'bottle dues etc',540.00),(112,'2014-06-26 15:26:08',46,'wife',-5000.00),(113,'2014-06-26 15:26:52',39,'bottle',65.00),(114,'2014-06-26 15:27:05',66,'1.5 ltr',85.00),(115,'2014-06-26 15:27:22',34,'cheeni etc',780.00),(116,'2014-06-26 15:27:46',97,'parchi',260.00),(117,'2014-06-26 15:28:04',81,'surf',370.00),(118,'2014-06-26 15:28:38',103,'rooh afza',150.00),(119,'2014-06-26 15:28:57',35,'parchi oil etc',623.00),(120,'2014-06-26 15:29:19',52,'easy paisa',1060.00),(121,'2014-06-26 15:29:51',35,'cake 1.5 ltr',185.00),(122,'2014-06-26 15:30:06',97,'parchi',243.00),(123,'2014-06-26 15:32:04',18,'dues',13.00),(124,'2014-06-26 15:32:28',26,'bread',50.00),(125,'2014-06-26 15:33:47',78,'bread',150.00),(126,'2014-06-26 15:34:52',9,'cheeni',280.00),(127,'2014-06-26 15:36:17',20,'oil',205.00),(128,'2014-06-26 15:39:31',96,'more load',500.00),(129,'2014-06-26 15:40:38',75,'water etc',125.00),(130,'2014-06-26 15:40:57',35,'dues',30.00),(131,'2014-06-26 15:41:10',18,'half role',90.00),(132,'2014-06-26 15:41:32',88,'geometery dues',20.00),(133,'2014-06-26 15:41:56',4,'dues',10.00),(134,'2014-06-26 15:42:18',41,'dues',60.00),(135,'2014-06-26 15:42:36',20,'ciggeret half role',190.00),(136,'2014-06-26 15:43:02',87,'always',30.00),(137,'2014-06-26 15:43:42',104,'dues',73.00),(138,'2014-06-26 15:44:07',13,'dues',946.00),(139,'2014-06-26 15:44:20',18,'surf',130.00),(140,'2014-06-26 15:44:29',35,'parchi',240.00),(141,'2014-06-26 15:44:46',85,'dues',30.00),(142,'2014-06-26 15:45:05',75,'milk',140.00),(143,'2014-06-26 15:45:24',74,'cream',40.00),(144,'2014-06-26 15:45:39',88,'milk',40.00),(145,'2014-06-26 15:46:00',38,'perfume',90.00),(146,'2014-06-26 15:46:20',32,'chilka etc',70.00),(147,'2014-06-26 15:47:05',90,'payment',-140.00),(148,'2014-06-26 15:47:26',18,'ghee dues',30.00),(149,'2014-06-26 15:47:45',98,'color',15.00),(150,'2014-06-26 15:48:00',85,'taala',50.00),(151,'2014-06-26 15:48:25',103,'ball',15.00),(153,'2014-06-26 15:51:21',64,'catchup',130.00),(154,'2014-06-26 15:51:42',65,'dues',10.00),(155,'2014-06-26 15:52:10',20,'dues',10.00),(156,'2014-06-26 15:52:35',18,'mirch',115.00),(157,'2014-06-26 15:52:56',18,'dues',10.00),(158,'2014-06-26 15:53:13',46,'half role etc',150.00),(159,'2014-06-26 15:53:37',33,'ghee',330.00),(160,'2014-06-26 15:54:06',36,'dues',10.00),(161,'2014-06-26 15:54:37',18,'dues',10.00),(162,'2014-06-26 15:54:50',18,'dues',30.00),(163,'2014-06-26 15:55:20',99,'dues',10.00),(164,'2014-06-26 15:58:14',92,'maidah',30.00),(165,'2014-06-26 16:16:23',26,'dues',856.00),(166,'2014-06-26 16:18:28',20,'load plus others',562.00),(167,'2014-06-26 16:51:35',75,'chanay',50.00),(168,'2014-06-26 16:54:22',103,'dettol',17.00),(169,'2014-06-26 16:55:00',42,'load',100.00),(171,'2014-06-26 17:15:23',85,'dues',125.00),(172,'2014-06-26 17:17:40',46,'tape',25.00),(173,'2014-06-26 17:33:50',66,'chana',40.00),(174,'2014-06-26 17:35:11',75,'shampoo',5.00),(175,'2014-06-26 17:36:37',106,'wiper',50.00),(176,'2014-06-26 17:37:23',43,'bottle',15.00),(177,'2014-06-26 17:37:51',87,'dues',60.00),(178,'2014-06-26 17:38:05',100,'bottle brush',125.00),(179,'2014-06-26 17:38:29',36,'shampoo',180.00),(180,'2014-06-26 17:39:49',32,'dues',20.00),(181,'2014-06-26 17:40:01',55,'dues',7.00),(182,'2014-06-26 17:41:01',41,'dues',15.00),(183,'2014-06-26 18:55:39',66,'bar haf',50.00),(184,'2014-06-26 19:40:30',103,'payment',-150.00),(185,'2014-06-26 20:24:00',61,'chohay maar',30.00),(186,'2014-06-26 21:47:45',97,'payment',-2578.00),(187,'2014-06-26 23:51:17',35,'boteletc',70.00),(188,'2014-06-27 00:00:18',66,'half',17.00),(189,'2014-06-27 00:02:05',99,'self',-107.00),(190,'2014-06-27 00:03:00',68,'tazab',30.00),(192,'2014-06-27 00:07:15',75,'due',25.00),(193,'2014-06-27 00:12:15',108,'dal',35.00),(194,'2014-06-27 00:14:54',57,'due',20.00),(195,'2014-06-27 00:15:30',65,'sig',45.00),(196,'2014-06-27 00:16:21',69,'shapener',15.00),(197,'2014-06-27 00:17:36',39,'botel',150.00),(198,'2014-06-27 00:19:27',37,'ice juice',140.00),(199,'2014-06-27 00:20:31',8,'sweet',250.00),(200,'2014-06-27 00:22:27',106,'botel',20.00),(201,'2014-06-27 00:23:24',22,'due',15.00),(202,'2014-06-27 00:24:08',81,'due',15.00),(203,'2014-06-27 00:26:31',19,'juice',50.00),(204,'2014-06-27 00:29:03',91,'kochyetc',30.00),(205,'2014-06-27 10:16:40',20,'payment',-2054.00),(206,'2014-06-27 10:39:38',78,'bread eggs',135.00),(208,'2014-06-27 10:41:27',74,'payment',-120.00),(209,'2014-06-27 10:45:24',109,'Balance when computerized',12287.00),(210,'2014-06-27 11:04:40',57,'payment',-155.00),(211,'2014-06-27 11:04:55',68,'blue band',55.00),(212,'2014-06-27 11:14:37',32,'sarmad soday',959.00),(213,'2014-06-27 11:28:59',78,'biscuit',40.00),(214,'2014-06-27 11:54:03',71,'bun',30.00),(215,'2014-06-27 15:26:06',92,'cocomo',20.00),(216,'2014-06-27 15:26:20',100,'paste',110.00),(217,'2014-06-27 15:26:48',71,'20 out of 30',-20.00),(218,'2014-06-27 15:27:22',32,'chetos',30.00),(219,'2014-06-27 15:27:44',75,'ghee cheeni',233.00),(220,'2014-06-27 15:29:32',18,'dues',45.00),(221,'2014-06-27 15:30:25',3,'ball',50.00),(222,'2014-06-27 15:31:15',100,'2 bottles',50.00),(223,'2014-06-27 15:32:10',3,'payment',-110.00),(224,'2014-06-27 15:32:38',4,'chips',40.00),(225,'2014-06-27 15:34:11',75,'ghee cheeni daal chawal',433.00),(226,'2014-06-27 15:34:52',41,'spray',400.00),(227,'2014-06-27 21:24:38',40,'katch bred',351.00),(228,'2014-06-27 22:02:04',8,'botel',60.00),(229,'2014-06-27 23:58:40',78,'half',90.00),(230,'2014-06-27 23:59:56',68,'rice',190.00),(231,'2014-06-28 00:00:40',97,'parchi',400.00),(232,'2014-06-28 00:01:15',97,'milk',70.00),(233,'2014-06-28 00:01:53',16,'ice',250.00),(234,'2014-06-28 00:02:53',35,'sig cake',20.00),(235,'2014-06-28 00:03:41',46,'botel cake',95.00),(236,'2014-06-28 00:05:17',75,'parchi rice bottels',750.00),(237,'2014-06-28 00:06:47',78,'sigret wife etc',230.00),(238,'2014-06-28 00:07:23',37,'nimko',10.00),(239,'2014-06-28 00:07:59',41,'rice',160.00);
SELECT a.*
FROM tbltransactions AS a
JOIN (
SELECT customerid, MAX(transactiondate) last_zero_bal FROM (
SELECT customerid, transactiondate,
#balance := IF (customerid = #prev_cust,
#balance + transactionamount,
transactionamount) AS balance,
#prev_cust := customerid
FROM (SELECT *
FROM tbltransactions
ORDER BY customerid, transactiondate) AS t
CROSS JOIN (SELECT #balance := 0, #prev_cust := NULL) AS v
) AS running_balances
WHERE balance = 0
GROUP BY customerid
) AS b ON a.customerid = b.customerid AND a.transactiondate > b.last_zero_bal
The subquery with the alias running_balances calculates each customer's running balance. Then the subquery b finds the most recent date where each customer had a zero balance. Finally, this is joined with the original transaction table to show all the transactions after this.
DEMO
Here you go
select b.*
from (
select a.customerid, max(a.transactionid) last_zero_transactionid
from (
select a.customerid, a.transactionid,
if(#prev_customer_id=customerid, #running_total:=#running_total+#last_transaction, #running_total:=0) running_total,
#prev_customer_id:=customerid prev_customer_id,
#last_transaction:=transactionamount last_transaction
from tbltransactions a
join (select #prev_customer_id:=0, #running_total:=0, #last_transaction:=0) b
order by customerid, transactionid) a
where a.running_total = 0
group by a.customerid) a
join tbltransactions b on a.customerid = b.customerid and a.last_zero_transactionid <= b.transactionid;
SELECT t1.* FROM
tbltransactions t1
JOIN (
SELECT MAX(t1.transactionid) max_zero_id, t1.customerid
FROM tbltransactions t1
JOIN tbltransactions t2 ON t1.customerid = t2.customerid
AND t1.transactiondate >= t2.transactiondate
GROUP BY t1.transactiondate, t1.customerid
HAVING SUM(t2.transactionamount) = 0
) t2 ON t1.transactionid > t2.max_zero_id AND t2.customerid = t1.customerid
ORDER BY t1.customerid, t1.transactionid
Output
+---------------+---------------------+------------+-------------------+-------------------+
| transactionid | transactiondate | customerid | transactiondetail | transactionamount |
+---------------+---------------------+------------+-------------------+-------------------+
| 106 | 2014-06-26 15:20:46 | 81 | bread | 100.00 |
| 117 | 2014-06-26 15:28:04 | 81 | surf | 370.00 |
| 202 | 2014-06-27 00:24:08 | 81 | due | 15.00 |
| 231 | 2014-06-28 00:00:40 | 97 | parchi | 400.00 |
| 232 | 2014-06-28 00:01:15 | 97 | milk | 70.00 |
+---------------+---------------------+------------+-------------------+-------------------+
Query Plan
+----+-------------+------------+------+--------------------------+------------+---------+--------------------+------+---------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+------------+------+--------------------------+------------+---------+--------------------+------+---------------------------------+
| 1 | PRIMARY | <derived2> | ALL | NULL | NULL | NULL | NULL | 6 | Using temporary; Using filesort |
| 1 | PRIMARY | t1 | ref | transactionid,customerid | customerid | 8 | t2.customerid | 1 | Using where |
| 2 | DERIVED | t1 | ALL | customerid | NULL | NULL | NULL | 229 | Using temporary; Using filesort |
| 2 | DERIVED | t2 | ref | customerid | customerid | 8 | test.t1.customerid | 1 | Using where |
+----+-------------+------------+------+--------------------------+------------+---------+--------------------+------+---------------------------------+

How to rewrite a NOT IN subquery as join

Let's assume that the following tables in MySQL describe documents contained in folders.
mysql> select * from folder;
+----+----------------+
| ID | PATH |
+----+----------------+
| 1 | matches/1 |
| 2 | matches/2 |
| 3 | shared/3 |
| 4 | no/match/4 |
| 5 | unreferenced/5 |
+----+----------------+
mysql> select * from DOC;
+----+------+------------+
| ID | F_ID | DATE |
+----+------+------------+
| 1 | 1 | 2000-01-01 |
| 2 | 2 | 2000-01-02 |
| 3 | 2 | 2000-01-03 |
| 4 | 3 | 2000-01-04 |
| 5 | 3 | 2000-01-05 |
| 6 | 3 | 2000-01-06 |
| 7 | 4 | 2000-01-07 |
| 8 | 4 | 2000-01-08 |
| 9 | 4 | 2000-01-09 |
| 10 | 4 | 2000-01-10 |
+----+------+------------+
The columns ID are the primary keys and the column F_ID of table DOC is a not-null foreign key that references the primary key of table FOLDER. By using the 'DATE' of documents in the where clause, I would like to find which folders contain only the selected documents. For documents earlier than 2000-01-05, this could be written as:
SELECT DISTINCT d1.F_ID
FROM DOC d1
WHERE d1.DATE < '2000-01-05'
AND d1.F_ID NOT IN (
SELECT d2.F_ID
FROM DOC d2 WHERE NOT (d2.DATE < '2000-01-05')
);
and it correctly returns '1' and '2'. By reading
http://dev.mysql.com/doc/refman/5.5/en/rewriting-subqueries.html
the performance for big tables could be improved if the subquery is replaced with a join. I already found questions related to NOT IN and JOINS but not exactly what I was looking for. So, any ideas of how this could be written with joins ?
The general answer is:
select t.*
from t
where t.id not in (select id from s)
Can be rewritten as:
select t.*
from t left outer join
(select distinct id from s) s
on t.id = s.id
where s.id is null
I think you can apply this to your situation.
select distinct d1.F_ID
from DOC d1
left outer join (
select F_ID
from DOC
where date >= '2000-01-05'
) d2 on d1.F_ID = d2.F_ID
where d1.date < '2000-01-05'
and d2.F_ID is null
If I understand your question correctly, that you want to find the F_IDs representing folders which only contains documents from before '2000-01-05', then simply
SELECT F_ID
FROM DOC
GROUP BY F_ID
HAVING MAX(DATE) < '2000-01-05'
Sample Table and Insert Statements
CREATE TABLE `tleft` (
`id` int(2) NOT NULL,
`name` varchar(100) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
CREATE TABLE `tright` (
`id` int(2) NOT NULL,
`t_left_id` int(2) DEFAULT NULL,
`description` varchar(100) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
INSERT INTO `tleft` (`id`, `name`)
VALUES
(1, 'henry'),
(2, 'steve'),
(3, 'jeff'),
(4, 'richards'),
(5, 'elon');
INSERT INTO `tright` (`id`, `t_left_id`, `description`)
VALUES
(1, 1, 'sample'),
(2, 2, 'sample');
Left Join : SELECT l.id,l.name FROM tleft l LEFT JOIN tright r ON l.id = r.t_left_id ;
Returns Id : 1, 2, 3, 4, 5
Right Join : SELECT l.id,l.name FROM tleft l RIGHT JOIN tright r ON l.id = r.t_left_id ;
Returns Id : 1,2
Subquery Not in tright : select id from tleft where id not in ( select t_left_id from tright);
Returns Id : 3,4,5
Equivalent Join For above subquery :
SELECT l.id,l.name FROM tleft l LEFT JOIN tright r ON l.id = r.t_left_id WHERE r.t_left_id IS NULL;
AND clause will be applied during the JOIN and WHERE clause will be applied after the JOIN .
Example : SELECT l.id,l.name FROM tleft l LEFT JOIN tright r ON l.id = r.t_left_id AND r.description ='hello' WHERE r.t_left_id IS NULL ;
Hope this helps

MySQL Query Optimization with MAX()

I have 3 tables with the following schema:
CREATE TABLE `devices` (
`device_id` int(11) NOT NULL auto_increment,
`name` varchar(20) default NULL,
`appliance_id` int(11) default '0',
`sensor_type` int(11) default '0',
`display_name` VARCHAR(100),
PRIMARY KEY USING BTREE (`device_id`)
)
CREATE TABLE `channels` (
`channel_id` int(11) NOT NULL AUTO_INCREMENT,
`device_id` int(11) NOT NULL,
`channel` varchar(10) NOT NULL,
PRIMARY KEY (`channel_id`),
KEY `device_id_idx` (`device_id`)
)
CREATE TABLE `historical_data` (
`date_time` datetime NOT NULL,
`channel_id` int(11) NOT NULL,
`data` float DEFAULT NULL,
`unit` varchar(10) DEFAULT NULL,
KEY `devices_datetime_idx` (`date_time`) USING BTREE,
KEY `channel_id_idx` (`channel_id`)
)
The setup is that a device can have one or more channels and each channel has many (historical) data.
I use the following query to get the last historical data for one device and all it's related channels:
SELECT c.channel_id, c.channel, max(h.date_time), h.data
FROM devices d
INNER JOIN channels c ON c.device_id = d.device_id
INNER JOIN historical_data h ON h.channel_id = c.channel_id
WHERE d.name = 'livingroom' AND d.appliance_id = '0'
AND d.sensor_type = 1 AND ( c.channel = 'ch1')
GROUP BY c.channel
ORDER BY h.date_time, channel
The query plan looks as follows:
+----+-------------+-------+--------+-----------------------+----------------+---------+---------------------------+--------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+-----------------------+----------------+---------+---------------------------+--------+-------------+
| 1 | SIMPLE | c | ALL | PRIMARY,device_id_idx | NULL | NULL | NULL | 34 | Using where |
| 1 | SIMPLE | d | eq_ref | PRIMARY | PRIMARY | 4 | c.device_id | 1 | Using where |
| 1 | SIMPLE | h | ref | channel_id_idx | channel_id_idx | 4 | c.channel_id | 322019 | |
+----+-------------+-------+--------+-----------------------+----------------+---------+---------------------------+--------+-------------+
3 rows in set (0.00 sec)
The above query is currently taking approximately 15 secs and I wanted to know if there are any tips or way to improve the query?
Edit:
Example data from historical_data
+---------------------+------------+------+------+
| date_time | channel_id | data | unit |
+---------------------+------------+------+------+
| 2011-11-20 21:30:57 | 34 | 23.5 | C |
| 2011-11-20 21:30:57 | 9 | 68 | W |
| 2011-11-20 21:30:54 | 34 | 23.5 | C |
| 2011-11-20 21:30:54 | 5 | 316 | W |
| 2011-11-20 21:30:53 | 34 | 23.5 | C |
| 2011-11-20 21:30:53 | 2 | 34 | W |
| 2011-11-20 21:30:51 | 34 | 23.4 | C |
| 2011-11-20 21:30:51 | 9 | 68 | W |
| 2011-11-20 21:30:49 | 34 | 23.4 | C |
| 2011-11-20 21:30:49 | 4 | 193 | W |
+---------------------+------------+------+------+
10 rows in set (0.00 sec)
Edit 2:
Mutliple channel SELECT example:
SELECT c.channel_id, c.channel, max(h.date_time), h.data
FROM devices d
INNER JOIN channels c ON c.device_id = d.device_id
INNER JOIN historical_data h ON h.channel_id = c.channel_id
WHERE d.name = 'livingroom' AND d.appliance_id = '0'
AND d.sensor_type = 1 AND ( c.channel = 'ch1' OR c.channel = 'ch2' OR c.channel = 'ch2')
GROUP BY c.channel
ORDER BY h.date_time, channel
I've used OR in the c.channel where clause because it was easier to generated pro grammatically but it can be changed to use IN if necessary.
Edit 3:
Example result of what I'm trying to achieve:
+-----------+------------+---------+---------------------+-------+
| device_id | channel_id | channel | max(h.date_time) | data |
+-----------+------------+---------+---------------------+-------+
| 28 | 9 | ch1 | 2011-11-21 20:39:36 | 0 |
| 28 | 35 | ch2 | 2011-11-21 20:30:55 | 32767 |
+-----------+------------+---------+---------------------+-------+
I have added the device_id to the example but my select will only need to return channel_id, channel, last date_time i.e max and the data. The results should be the last record from the historical_data table for each channel for one device.
It seems that removing an re-creating the index on date_time by deleting and creating it again sped up my original SQL up to around 2secs
I haven't been able to test this, so I'd like to ask you to run it and let us know what happens.. if it gives you the desired result and if it runs faster than your current:
CREATE DEFINER=`root`#`localhost` PROCEDURE `GetLatestHistoricalData_EXAMPLE`
(
IN param_device_name VARCHAR(20)
, IN param_appliance_id INT
, IN param_sensor_type INT
, IN param_channel VARCHAR(10)
)
BEGIN
SELECT
h.date_time, h.data
FROM
historical_data h
INNER JOIN
(
SELECT c.channel_id
FROM devices d
INNER JOIN channels c ON c.device_id = d.device_id
WHERE
d.name = param_device_name
AND d.appliance_id = param_appliance_id
AND d.sensor_type = param_sensor_type
AND c.channel = param_channel
)
c ON h.channel_id = c.channel_id
ORDER BY h.date_time DESC
LIMIT 1;
END
Then to run a test:
CALL GetLatestHistoricalData_EXAMPLE ('livingroom', 0, 1, 'ch1');
I tried working it into a stored procedure so that even if you get the desired results using this for one device, you can try it with another device and see the results... Thanks!
[edit] : : In response to Danny's comment here's an updated test version:
CREATE DEFINER=`root`#`localhost` PROCEDURE `GetLatestHistoricalData_EXAMPLE_3Channel`
(
IN param_device_name VARCHAR(20)
, IN param_appliance_id INT
, IN param_sensor_type INT
, IN param_channel_1 VARCHAR(10)
, IN param_channel_2 VARCHAR(10)
, IN param_channel_3 VARCHAR(10)
)
BEGIN
SELECT
h.date_time, h.data
FROM
historical_data h
INNER JOIN
(
SELECT c.channel_id
FROM devices d
INNER JOIN channels c ON c.device_id = d.device_id
WHERE
d.name = param_device_name
AND d.appliance_id = param_appliance_id
AND d.sensor_type = param_sensor_type
AND (
c.channel IN (param_channel_1
,param_channel_2
,param_channel_3
)
c ON h.channel_id = c.channel_id
ORDER BY h.date_time DESC
LIMIT 1;
END
Then to run a test:
CALL GetLatestHistoricalData_EXAMPLE_3Channel ('livingroom', 0, 1, 'ch1', 'ch2' , 'ch3');
Again, this is just for testing, so you'll be able to see if it meets your needs..
I would first add an index on the devices table ( appliance_id, sensor_type, name ) to match your query. I don't know how many entries are in this table, but if large, and many elements per device, get right to it.
Second, on your channels table, index on ( device_id, channel )
Third, on your history data, index on ( channel_id, date_time )
then,
SELECT STRAIGHT_JOIN
PreQuery.MostRecent,
PreQuery.Channel_ID,
PreQuery.Channel,
H2.Data,
H2.Unit
from
( select
c.channel_id,
c.channel,
max( h.date_time ) as MostRecent
from
devices d
join channels c
on d.device_id = c.device_id
and c.channel in ( 'ch1', 'ch2', 'ch3' )
join historical_data h
on c.channel_id = c.Channel_id
where
d.appliance_id = 0
and d.sensor_type = 1
and d.name = 'livingroom'
group by
c.channel_id ) PreQuery
JOIN Historical_Data H2
on PreQuery.Channel_ID = H2.Channel_ID
AND PreQuery.MostRecent = H2.Date_Time
order by
PreQuery.MostRecent,
PreQuery.Channel