Aggregate rows by id comparing column values - mysql
I have the following table that groups users by their permissions
userIds permissions
4,5,7,8 100,1600,500,501,502,400,401,1500,1501
The numbers in the permissions column are the sections ids.
Some of these sections may have other data associated which I retrieved and stored in another table.
sectionId userId resourceId
100 4 NULL
1600 4 NULL
500 4 NULL
501 4 NULL
502 4 NULL
400 4 NULL
401 4 1
1500 4 NULL
1501 4 NULL
100 5 NULL
1600 5 NULL
500 5 NULL
501 5 NULL
502 5 NULL
400 5 NULL
401 5 1,2
1500 5 NULL
1501 5 NULL
100 7 NULL
1600 7 NULL
500 7 NULL
501 7 NULL
502 7 NULL
400 7 NULL
401 7 2
1500 7 NULL
1501 7 NULL
100 8 NULL
1600 8 NULL
500 8 NULL
501 8 NULL
502 8 NULL
400 8 NULL
401 8 1
1500 8 NULL
1501 8 NULL
My goal is to compare, for each user in the userIds column of the first table (splitted by comma), every row of the second table in order to check if each user has the same resourceId value for that specific sectionId.
If one or more users have the same resourceId value for each section I want to keep them group together, otherwise they need to be on different rows.
This is the output I'm expecting from the sample data provided:
userIds permissions
4,8 100,1600,500,501,502,400,401,1500,1501
5 100,1600,500,501,502,400,401,1500,1501
7 100,1600,500,501,502,400,401,1500,1501
UPDATE
I managed to get the desidered output in the following way:
-- Numbers table creation
DROP temporary TABLE IF EXISTS tally;
CREATE temporary TABLE tally
(
n INT NOT NULL auto_increment PRIMARY KEY
);
INSERT INTO tally
(n)
SELECT NULL
FROM (SELECT 0 AS N
UNION ALL
SELECT 1
UNION ALL
SELECT 2
UNION ALL
SELECT 3
UNION ALL
SELECT 4
UNION ALL
SELECT 5
UNION ALL
SELECT 6
UNION ALL
SELECT 7
UNION ALL
SELECT 8
UNION ALL
SELECT 9) a,
(SELECT 0 AS N
UNION ALL
SELECT 1
UNION ALL
SELECT 2
UNION ALL
SELECT 3
UNION ALL
SELECT 4
UNION ALL
SELECT 5
UNION ALL
SELECT 6
UNION ALL
SELECT 7
UNION ALL
SELECT 8
UNION ALL
SELECT 9) b;
-- Split users by comma from first table
DROP temporary TABLE IF EXISTS tmppermissions2;
CREATE temporary TABLE tmppermissions2
(
userid VARCHAR(255) NOT NULL,
permissions TEXT NOT NULL
);
INSERT INTO tmppermissions2
SELECT userid,
permissions
FROM (SELECT Substring_index(Substring_index(t.userids, ',', tally.n), ',', -1
)
userId,
t.permissions
permissions
FROM tally
INNER JOIN tmppermissions t
ON Char_length(t.userids) - Char_length(
REPLACE(t.userids, ',',
'')) >=
tally.n - 1
ORDER BY n) AS split;
-- Gets the users with the same permissions
DROP temporary TABLE IF EXISTS sharedprofiles;
CREATE temporary TABLE sharedprofiles
(
userids VARCHAR(255) NOT NULL,
permissions TEXT NOT NULL,
profileid INT(11)
);
INSERT INTO sharedprofiles
SELECT Group_concat(userid),
permissions,
NULL
FROM tmppermissions2
WHERE userid NOT IN (SELECT split.userid
FROM (SELECT Substring_index(Substring_index(r.userids,
',',
t.n), ',', -1)
userId
FROM tally t
INNER JOIN tmppermissions r
ON Char_length(r.userids)
- Char_length(
REPLACE(r.userids, ',',
'')) >=
t.n - 1
WHERE Position(',' IN r.userids) > 0
ORDER BY n) AS split
WHERE split.userid IN (SELECT *
FROM (SELECT Group_concat(userid
ORDER
BY userid ASC)
AS
users
FROM
tmpcurrentresources2
GROUP BY resourceid,
sectionid
ORDER BY users) b
WHERE Position(',' IN b.users) =
0))
GROUP BY permissions
ORDER BY Group_concat(userid);
-- Gets the users with specific permissions
DROP temporary TABLE IF EXISTS singleprofiles;
CREATE temporary TABLE singleprofiles
(
userid VARCHAR(255) NOT NULL,
permissions TEXT NOT NULL,
profileid INT(11)
);
INSERT INTO singleprofiles
SELECT userid,
permissions,
NULL
FROM tmppermissions2
WHERE userid IN (SELECT split.userid
FROM (SELECT Substring_index(Substring_index(r.userids, ',',
t.n),
',', -1)
userId
FROM tally t
INNER JOIN tmppermissions r
ON Char_length(r.userids) -
Char_length(
REPLACE(r.userids, ',',
'')) >=
t.n - 1
WHERE Position(',' IN r.userids) > 0
ORDER BY n) AS split
WHERE split.userid IN (SELECT *
FROM (SELECT Group_concat(userid
ORDER BY
userid ASC)
AS
users
FROM tmpcurrentresources2
GROUP BY resourceid,
sectionid
ORDER BY users) b
WHERE Position(',' IN b.users) = 0))
ORDER BY userid;
-- Merge the results
SELECT *
FROM sharedprofiles
UNION
SELECT *
FROM singleprofiles;
I'm wondering if there is a more concise way to accomplish the same result.
The solution (as I suspect you already know) is to normalise your schema.
So instead of...
userIds permissions
4,5 100,1600,500
...you might have
userIds permissions
4 100
4 1600
4 500
5 100
5 1600
5 500
Related
Use previous row result when current row is null
I have a table that has daily records of transactions and some rows are missing data which will make plotting a daily graph inconsistent. I want a query to use the last row result when the current one is null so that it can look something like this: The structure of my table looks like this: I have tried working on this query to select the previous row and update the current row if it is null but is not dynamic. SELECT BALANCE FROM tbl_batch_balances_null WHERE id = (select min(id) from tbl_batch_balances_null where id < '2' and balance is not null)
Schema (MySQL v5.7) DROP TABLE IF EXISTS alfie; CREATE TABLE alfie (id INT AUTO_INCREMENT PRIMARY KEY, store CHAR(1) NULL, product INT NULL ); INSERT INTO alfie VALUES ( 7,'a',3), ( 8,null,null), ( 9,null,null), (10,null,null), (11,'a',1), (12,'a',1), (13,'a',1), (14,null,null), (15,null,null), (16,'b',2), (17,null,null), (18,null,null), (19,null,null); Query #1 SELECT a.id, COALESCE(a.store, c.store) store, COALESCE(a.product,c.product) product FROM alfie a LEFT JOIN ( SELECT x.*, MAX(y.id) y_id FROM alfie x JOIN alfie y ON y.id < x.id AND y.store IS NOT NULL WHERE x.store IS NULL GROUP BY x.id ) b ON b.id = a.id LEFT JOIN alfie c ON c.id = b.y_id ORDER BY id; id store product 7 a 3 8 a 3 9 a 3 10 a 3 11 a 1 12 a 1 13 a 1 14 a 1 15 a 1 16 b 2 17 b 2 18 b 2 19 b 2 View on DB Fiddle
MySQL: Select newest two rows per Group
I have a table like this: CREATE TABLE `data` ( `id` int(11) NOT NULL, `deviceId` int(11) NOT NULL, `position_x` int(11) NOT NULL, `position_y` int(11) NOT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; ALTER TABLE `data` ADD PRIMARY KEY (`id`); COMMIT; id, deviceId, position_x, position_y 1 1 100 0 2 2 150 50 3 3 200 20 4 1 220 20 5 1 210 10 6 2 100 40 7 3 120 50 8 3 130 60 9 2 240 15 I need the "newest" two rows per DeviceID, where a bigger ID means newer. Right now, I'm selecting the newest row per Device via this query: SELECT id, deviceId, position_x, position_y FROM data WHERE deviceId > 0 AND id IN (SELECT MAX(id) FROM data GROUP BY deviceId) And in a loop, where I output the data, I select the second latest row for every deviceId in an individual query, which is kinda slow/dirty: SELECT position_x position_y FROM data WHERE deviceId = :deviceId AND id < :id ORDER BY id DESC LIMIT 1 Is there a way to combine both queries or at least, in one query, select the second row for every deviceId from query 1? Thanks
You can try using row_number() select * from ( SELECT id, deviceId, position_x, position_y,row_number() over(partition by deviceid order by id desc) as rn FROM data WHERE deviceId > 0 )A where rn=2
You can use a correlated subquery for this as well: SELECT d.* FROM data d WHERE d.deviceId > 0 AND d.id = (SELECT d2.id FROM data d2 WHERE d2.deviceId = d.deviceId ORDER BY d2.id DESC LIMIT 1, 1 ); With an index on data(deviceId, id desc), you might be impressed at the performance.
Mysql split column string into three rows
This query SELECT user_Id,CAST(leave_dates_object AS CHAR(10000) CHARACTER SET utf8) AS LeaveDates FROM `lms_leaves` returns Now I want three rows user_Id and Date and F_or_H from the formatted string return from leave date. I modified and tried code from this link but can't able to get result. Expected Output user_Id Date LeaveType 85 2016-09-06 F 85 2016-09-07 F 85 2016-09-06 H 63 2016-03-25 F 63 2016-03-02 F 63 2016-03-03 H Please Help me.
Suppose you have maximally 8 dates in one record: create table users (userId int not null, leaveDates varchar(1000) not null); INSERT users (userId,leaveDates) VALUES (85,'--- \r-2016-09-06:F\r-2016-09-07:F'),(85,'---\r-2016-09-06:H'),(63,'---\r-2016-03-25:F'),(63,'---\r-2016-03-02:F\r-2016-03-03:H'); SELECT s.userId AS userId,LEFT(s.leaveDate,CHAR_LENGTH(s.leaveDate)-2) AS ldate, RIGHT(s.leaveDate,1) AS lflag FROM ( SELECT u.userId, REPLACE(SUBSTRING(SUBSTRING_INDEX(u.leaveDates, '\r-', n.number), CHAR_LENGTH(SUBSTRING_INDEX(u.leaveDates, '\r-', n.number -1)) + 1), '\r-', '') as leaveDate FROM (SELECT u0.userId,REPLACE(u0.leaveDates,'---','') AS leaveDates FROM users u0) u INNER JOIN (SELECT 1 AS number UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8) n ON (char_length(u.leaveDates) - char_length(replace(u.leaveDates, '\r-', '')) >= n.number-1)) s WHERE s.leaveDate<>'' ORDER BY userId,ldate,lflag;
Use Order By, and Group by on a Union result from 2 different columns
This My Table ID Value SenderID RecieverID 1 Hello There 2 7 2 etc etc 7 5 3 etc 2 6 4 ee 7 2 5 asdas 2 7 6 asdas 2 5 7 asdas 7 5 What I want is the value from either senderID or receiverID from all the rows in which a specific value let say 2 occurs in any of these 2 columns I used this query SELECT `SenderID` FROM `messages` WHERE `RecieverID` = 2 UNION SELECT `ReceiverID` FROM `messages` WHERE `SenderID` = 2 Gives unique answer but in wrong order like this ReceiverID 7 6 5 I'm expecting the answer of this query to be ordered by ID DESC in which a specific sender or reciever id occures for example in my tables the msg between senderid 2 and reverid 7 is at id 5 and latest id btweend sendr2 and 6 is at id 3 and btweed sndr2 and 5 it is ID 7 sot the above answer should be sorted like this 5, 7, 6 instead of 7,6,5
This one will order the sender/receiver IDs by their most recent conversation: SELECT senderID -- , MAX(ID) as maxID -- uncomment to see maxID FROM ( SELECT ID, `SenderID` FROM `messages` WHERE `RecieverID` = 2 UNION ALL SELECT ID, `RecieverID` FROM `messages` WHERE `SenderID` = 2 ) as sub GROUP BY (SenderID) ORDER BY MAX(ID) DESC http://sqlfiddle.com/#!9/6d7bc0/1
you need to use inner query in a brackets in order to set what the order by refers to: SELECT id, senderID from (SELECT ID, `SenderID` FROM `messages` WHERE `RecieverID` = 2 UNION SELECT ID, `RecieverID` as senderId FROM `messages` WHERE `SenderID` = 2 ) as A GROUP BY (SenderID) order by ID ASC
SELECT ID, IF(`SenderID` = 2,`RecieverID`,`SenderID`) FROM `messages` WHERE `RecieverID` = 2 OR `SenderID` = 2 group by IF(`SenderID` = 2,`RecieverID`,`SenderID`) order by ID ASC
MySQL- Select at least n rows per group
Suppose you have id / value 1 2 1 3 1 6 2 3 3 1 3 3 3 6 And I want to retrieve at least n rows per id group, let's say n = 4. In addition, it would help if a counter is added as a column. So the results should be like: counter / id / value 1 1 2 2 1 3 3 1 6 4 null null 1 2 3 2 null null 3 null null 4 null null 1 3 1 2 3 3 3 3 6 4 null null regards
I'm assuming that the combination of id and value is unique. Here's how you can do it without using MySQL variables: SELECT a.n AS counter, b.id, b.value FROM ( SELECT aa.n, bb.id FROM ( SELECT 1 AS n UNION ALL SELECT 2 AS n UNION ALL SELECT 3 AS n UNION ALL SELECT 4 AS n ) aa CROSS JOIN ( SELECT DISTINCT id FROM tbl ) bb ) a LEFT JOIN ( SELECT aa.id, aa.value, COUNT(*) AS rank FROM tbl aa LEFT JOIN tbl bb ON aa.id = bb.id AND aa.value >= bb.value GROUP BY aa.id, aa.value ) b ON a.id = b.id AND a.n = b.rank ORDER BY a.id, a.n
The next blog post describes the solution to your query: SQL: selecting top N records per group. It requires an additional small table of numbers, which is utilized to "iterate" the top N values per group via String Walking technique. It uses GROUP_CONCAT as a way to overcome the fact MySQL does not support Window Functions. This also means it's not a pretty sight! An advantage of this technique is that it does not require subqueries, and can optimally utilize an index on the table. To complete the answer to your question, we must add an additional columns: you have requested a counter per item per group. Here's an example using the world sample database, choosing top 5 largest counties per continent: CREATE TABLE `tinyint_asc` ( `value` tinyint(3) unsigned NOT NULL default '0', PRIMARY KEY (value) ) ; INSERT INTO `tinyint_asc` VALUES (0),(1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13),(14),(15),(16),(17),(18),(19),(20),(21),(22),(23),(24),(25),(26),(27),(28),(29),(30),(31),(32),(33),(34),(35),(36),(37),(38),(39),(40),(41),(42),(43),(44),(45),(46),(47),(48),(49),(50),(51),(52),(53),(54),(55),(56),(57),(58),(59),(60),(61),(62),(63),(64),(65),(66),(67),(68),(69),(70),(71),(72),(73),(74),(75),(76),(77),(78),(79),(80),(81),(82),(83),(84),(85),(86),(87),(88),(89),(90),(91),(92),(93),(94),(95),(96),(97),(98),(99),(100),(101),(102),(103),(104),(105),(106),(107),(108),(109),(110),(111),(112),(113),(114),(115),(116),(117),(118),(119),(120),(121),(122),(123),(124),(125),(126),(127),(128),(129),(130),(131),(132),(133),(134),(135),(136),(137),(138),(139),(140),(141),(142),(143),(144),(145),(146),(147),(148),(149),(150),(151),(152),(153),(154),(155),(156),(157),(158),(159),(160),(161),(162),(163),(164),(165),(166),(167),(168),(169),(170),(171),(172),(173),(174),(175),(176),(177),(178),(179),(180),(181),(182),(183),(184),(185),(186),(187),(188),(189),(190),(191),(192),(193),(194),(195),(196),(197),(198),(199),(200),(201),(202),(203),(204),(205),(206),(207),(208),(209),(210),(211),(212),(213),(214),(215),(216),(217),(218),(219),(220),(221),(222),(223),(224),(225),(226),(227),(228),(229),(230),(231),(232),(233),(234),(235),(236),(237),(238),(239),(240),(241),(242),(243),(244),(245),(246),(247),(248),(249),(250),(251),(252),(253),(254),(255); SELECT Continent, SUBSTRING_INDEX( SUBSTRING_INDEX( GROUP_CONCAT(Name ORDER BY SurfaceArea DESC), ',', value), ',', -1) AS Name, CAST( SUBSTRING_INDEX( SUBSTRING_INDEX( GROUP_CONCAT(SurfaceArea ORDER BY SurfaceArea DESC), ',', value), ',', -1) AS DECIMAL(20,2) ) AS SurfaceArea, CAST( SUBSTRING_INDEX( SUBSTRING_INDEX( GROUP_CONCAT(Population ORDER BY SurfaceArea DESC), ',', value), ',', -1) AS UNSIGNED ) AS Population, tinyint_asc.value AS counter FROM Country, tinyint_asc WHERE tinyint_asc.value >= 1 AND tinyint_asc.value <= 5 GROUP BY Continent, value ;