MySQL - group_concat pulling in additional incorrect data - mysql

I'm having trouble with a JOIN and a GROUP_CONCAT. The query is concatenating additional data that should not be associated with the join.
Here's my table structure:
linkages
ID table_name tag_id
1 subcategories 6
2 categories 9
music
ID artwork
1 5
2 4
artwork
ID url_path
1 /some/file/path
2 /some/file/path
And here's my query:
SELECT music.*,
artwork.url_path AS artwork_url_path,
GROUP_CONCAT( linkages.tag_id ) AS tag_ids,
GROUP_CONCAT( linkages.table_name ) AS table_name
FROM music
LEFT JOIN artwork ON artwork.id = music.artwork
LEFT JOIN linkages ON music.id = linkages.track_id
WHERE music.id IN( '1356',
'1357',
'719',
'169',
'170',
'171',
'805' )
ORDER BY FIELD( music.id,
1356,
1357,
719,
169,
170,
171,
805 )
This is the result of the GROUP_CONCAT :
[tag_ids] => 3, 6, 9, 17, 19, 20, 26, 49, 63, 64, 53, 57, 63, 65, 67, 73, 79, 80, 85, 96, 98, 11, 53, 67, 3, 6, 15, 17, 26, 38, 50, 63, 74, 53, 56, 57, 62, 63, 65, 66, 67, 72, 85, 88, 98, 24, 69, 71, 3, 6, 15, 17, 26, 38, 50
The first portion of the result is correct:
[tag_ids] => 3, 6, 9, 17, 19, 20, 26, 49, 63, 64, 53, 57, 63, 65, 67, 73, 79, 80, 85, 96, 98, 11, 53, 67
Everything after the correct values seems random and most of the values don't exist in the result in the database, but it's still pulling it in. It seems to repeat a portion of the correct result (3, 6, 15, 17 - the 3, 6, 17 are correct, but 15 shouldn't be there, similar with a bunch of other numbers - 71, etc. I can't use DISTINCT because I need to match up the tag_ids and table_name results as a multidimensional array from the results.
Any thoughts as to why?
UPDATE:
I ended up solving it with the initial push from Gordon. It needed a GROUP_BY clause, otherwise it was putting every results tag id's in each result. The final query ended up becoming this:
SET SESSION group_concat_max_len = 1000000;
SELECT
music.*,
artwork.url_path as artwork_url_path,
GROUP_CONCAT(linkages.tag_id, ':', linkages.table_name) as tags
FROM music
LEFT JOIN artwork ON artwork.id = music.artwork
LEFT JOIN linkages ON music.id = linkages.track_id
WHERE music.id IN('1356', '1357', '719', '169', '170', '171', '805')
GROUP BY music.id
ORDER BY FIELD(music.id,1356,1357,719,169,170,171,805);

Your join is generating duplicate rows. I would suggest that you fix the root cause of the problem. But, a quick-and-dirty solution is to use group_concat(distinct):
GROUP_CONCAT(DISTINCT linkages.tag_id) as tag_ids,
GROUP_CONCAT(DISTINCT linkages.table_name) as table_name
You can put the columns in a single field using GROUP_CONCAT():
GROUP_CONCAT(DISTINCT linkages.tag_id, ':', linkages.table_name) as tags

Related

How parse json in postgres

{
"Store123": {
"2015-05-03": 55,
"2019-06-09": 39,
"2018-06-17": 37,
"2019-06-02": 35,
"2018-07-01": 36,
"2015-01-18": 79,
"2017-02-26": 43,
"2019-08-04": 38,
"2017-11-05": 46,
"2019-02-17": 38,
"2015-07-19": 59,
"2015-01-11": 85,
"2015-02-01": 67
}
}
I have the above JSON data. How do I parse them in POSTGRES so the table would look like below
StoreID Dates Score
123 2015-05-03 55
123 2019-06-09 39
You can unnest the outer level first to get the storeid, then the inner level:
with data(doc) as (
values ('
{
"Store123": {
"2015-05-03": 55,
"2019-06-09": 39,
"2018-06-17": 37,
"2019-06-02": 35,
"2018-07-01": 36,
"2015-01-18": 79,
"2017-02-26": 43,
"2019-08-04": 38,
"2017-11-05": 46,
"2019-02-17": 38,
"2015-07-19": 59,
"2015-01-11": 85,
"2015-02-01": 67
}
}'::jsonb)
)
select sl.storeid, st.dt::date as "date", st.score::int as score
from data d
cross join lateral jsonb_each(d.doc) as sl(storeid, sd)
cross join lateral jsonb_each(sl.sd) as st(dt, score)

MYSQL Join/Index optimization

I have a query that tries to find all shopping carts containing a set of given packages.
For each package I join the corresponding cartitem table once, because I am only interested in carts containing all given packages.
When I reach more than 15 packages(joins) the query performance rapidly drops.
I have two indeces on the corresponding foreign columns and am aware that mysql uses only one of them. When I add an index over the 2 columns(cartitem_package_id,cartitem_cart_id) it works, but is this the only way to solve this situation?
I would like to know why MYSQL suddently stucks in this situation and what may be the mysql internal problem, because I do not see any deeper problem with this definition and query? Does that may be an issue with the query optimizer and can I do something(e.g. adding brackets) to support or force a specific query execution? Or has anyone a different approach here, using another query?
The query looks something like this:
SELECT cart_id
FROM cart
INNER JOIN cartitem as c1 ON cart_id=c1.cartitem_cart_id AND c1.cartitem_package_id= 7
INNER JOIN cartitem as c2 ON cart_id=c2.cartitem_cart_id AND c2.cartitem_package_id= 8
INNER JOIN cartitem as c3 ON cart_id=c3.cartitem_cart_id AND c3.cartitem_package_id= 9
INNER JOIN cartitem as c4 ON cart_id=c4.cartitem_cart_id AND c4.cartitem_package_id= 10
INNER JOIN cartitem as c5 ON cart_id=c5.cartitem_cart_id AND c5.cartitem_package_id= 11
INNER JOIN cartitem as c6 ON cart_id=c6.cartitem_cart_id AND c6.cartitem_package_id= 12
INNER JOIN cartitem as c7 ON cart_id=c7.cartitem_cart_id AND c7.cartitem_package_id= 13
INNER JOIN cartitem as c8 ON cart_id=c8.cartitem_cart_id AND c8.cartitem_package_id= 14
INNER JOIN cartitem as c9 ON cart_id=c9.cartitem_cart_id AND c9.cartitem_package_id= 15
INNER JOIN cartitem as c10 ON cart_id=c10.cartitem_cart_id AND c10.cartitem_package_id= 16
INNER JOIN cartitem as c11 ON cart_id=c11.cartitem_cart_id AND c11.cartitem_package_id= 17
INNER JOIN cartitem as c12 ON cart_id=c12.cartitem_cart_id AND c12.cartitem_package_id= 18
INNER JOIN cartitem as c13 ON cart_id=c13.cartitem_cart_id AND c13.cartitem_package_id= 19
INNER JOIN cartitem as c14 ON cart_id=c14.cartitem_cart_id AND c14.cartitem_package_id= 20
INNER JOIN cartitem as c15 ON cart_id=c15.cartitem_cart_id AND c15.cartitem_package_id= 21
INNER JOIN cartitem as c16 ON cart_id=c16.cartitem_cart_id AND c16.cartitem_package_id= 22
INNER JOIN cartitem as c17 ON cart_id=c17.cartitem_cart_id AND c17.cartitem_package_id= 23
Output:
No result.
Consider the following sample structure:
CREATE TABLE IF NOT EXISTS `cart` (
`cart_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`cart_state` smallint(20) DEFAULT NULL,
PRIMARY KEY (`cart_id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 AUTO_INCREMENT=80 ;
INSERT INTO `cart` (`cart_id`, `cart_state`) VALUES
(1, 0),(2, 5),(3, 0),(4, 0),(5, 0),(6, 0),(7, 0),(8, 0),(9, 0),(10, 0),(11, 0),(12, 0),(13, 0),(14, 5),(15, 5),(16, 10),(17, 0),(18, 10),(19, 40),(20, 10),(21, 5),(22, 0),(23, 10),(24, 10),(25, 0),(26, 10),(27, 5),(28, 5),(29, 0),(30, 5),(31, 0),(32, 0),(33, 0),(34, 0),(35, 0),(36, 0),(37, 0),(38, 0),(39, 0),(40, 0),(41, 0),(42, 0),(43, 0),(44, 0),(45, 40),(46, 0),(47, 0),(48, 1),(49, 0),(50, 5),(51, 0),(52, 0),(53, 5),(54, 5),(55, 0),(56, 0),(57, 10),(58, 0),(59, 0),(60, 5),(61, 0),(62, 0),(63, 10),(64, 0),(65, 5),(66, 5),(67, 10),(68, 10),(69, 0),(70, 0),(71, 10),(72, 0),(73, 10),(74, 0),(75, 10),(76, 0),(77, 10),(78, 0),(79, 10);
CREATE TABLE IF NOT EXISTS `cartitem` (
`cartitem_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`cartitem_package_id` int(10) unsigned DEFAULT NULL,
`cartitem_cart_id` int(10) unsigned DEFAULT NULL,
`cartitem_price` decimal(7,2) NOT NULL DEFAULT '0.00',
PRIMARY KEY (`cartitem_id`),
KEY `cartitem_package_id` (`cartitem_package_id`),
KEY `cartitem_cart_id` (`cartitem_cart_id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 AUTO_INCREMENT=89 ;
INSERT INTO `cartitem` (`cartitem_id`, `cartitem_package_id`, `cartitem_cart_id`, `cartitem_price`) VALUES
(1, 4, 2, 200.00),(2, 7, 3, 30.00),(3, 14, 9, 255.00),(4, 14, 9, 255.00),(5, 22, 9, 120.00),(6, 22, 9, 120.00),(7, 13, 13, 300.00),(8, 13, 13, 300.00),(9, 7, 14, 450.00),(10, 13, 14, 250.00),(11, 17, 14, 150.00),(12, 7, 15, 450.00),(13, 13, 15, 250.00),(14, 18, 15, 127.50),(15, 7, 16, 450.00),(16, 17, 16, 150.00),(17, 7, 18, 450.00),(18, 7, 19, 450.00),(19, 17, 19, 150.00),(20, 21, 19, 25.00),(21, 13, 20, 300.00),(22, 7, 21, 550.00),(23, 19, 21, 105.00),(24, 22, 21, 120.00),(25, 17, 22, 150.00),(26, 7, 23, 550.00),(27, 11, 24, 245.00),(31, 7, 26, 450.00),(32, 21, 26, 25.00),(33, 21, 26, 25.00),(34, 22, 26, 120.00),(35, 23, 26, 120.00),(36, 10, 27, 382.50),(37, 22, 27, 120.00),(38, 13, 27, 250.00),(39, 10, 28, 297.50),(43, 7, 29, 550.00),(41, 20, 28, 82.50),(42, 22, 28, 120.00),(44, 7, 30, 550.00),(46, 22, 30, 120.00),(47, 23, 30, 120.00),(48, 21, 18, 25.00),(49, 21, 19, 25.00),(50, 17, 37, 150.00),(51, 17, 37, 150.00),(52, 21, 37, 25.00),(53, 21, 37, 25.00),(54, 4, 45, 1.20),(55, 6, 45, 0.00),(56, 7, 47, 450.00),(57, 4, 50, 200.00),(58, 13, 52, 250.00),(59, 13, 19, 300.00),(60, 9, 19, 0.00),(61, 17, 53, 150.00),(62, 7, 53, 450.00),(63, 22, 18, 120.00),(64, 7, 16, 450.00),(65, 7, 54, 450.00),(66, 7, 57, 450.00),(67, 17, 57, 150.00),(68, 7, 56, 450.00),(69, 17, 59, 150.00),(70, 7, 60, 450.00),(71, 17, 61, 150.00),(72, 17, 63, 150.00),(73, 21, 65, 25.00),(74, 7, 66, 450.00),(75, 7, 67, 450.00),(76, 11, 68, 385.00),(77, 7, 71, 450.00),(78, 11, 73, 385.00),(79, 13, 73, 300.00),(80, 4, 75, 200.00),(82, 7, 73, 30.00),(83, 18, 73, 127.50),(84, 23, 73, 120.00),(85, 7, 73, 30.00),(86, 10, 77, 382.50),(87, 7, 79, 550.00),(88, 17, 79, 150.00);
The given query was a possible edge case leading to no results in this example.
SELECT cart_id
FROM cart
INNER JOIN cartitem as c1 ON cart_id=c1.cartitem_cart_id AND c1.cartitem_package_id= 7
INNER JOIN cartitem as c3 ON cart_id=c3.cartitem_cart_id AND c3.cartitem_package_id= 9
INNER JOIN cartitem as c4 ON cart_id=c4.cartitem_cart_id AND c4.cartitem_package_id= 13
INNER JOIN cartitem as c5 ON cart_id=c5.cartitem_cart_id AND c5.cartitem_package_id= 17
INNER JOIN cartitem as c6 ON cart_id=c6.cartitem_cart_id AND c6.cartitem_package_id= 21
Output:
cart_id
-------------
19
19
The query should return all carts containing items that are connected to packages(7,9,13,17,21) in this case.
My approach to your problem would be:
SELECT
cart_id
FROM
cart
INNER JOIN
cartitem
ON
cart_id = cartitem_cart_id
WHERE
cartitem_package_id IN (7,9,13,17,21) -- items that got to be in the cart
GROUP BY
cart_id
HAVING
count(distinct cartitem_package_id) = 5 -- number of different packages
;
DEMO with your data
Explanation
The principle is to filter first with the list of the desired values, here your packages. Now count the different packages per cart (GROUP BY cart_id). If this count matches the number of values in your filter list, then every single package must be in this cart.
You can replace the value list of the IN clause with a subselect, if you get those values from a subselect.
You should see that this approach should be easy to adapt to similar needs.

How to use SUM IF() in MySQL without hard coding values in the query

I have a query that uses SUM IF() to do a cross-tab result set. In this query I have the value sin the SUM IF() hard coded. The problem is that new values are added to the database. Is there a way to write the query without hard coding the values in the SUM IF()? Here is the query:
select storeid, sum(if(marketsegmentid = 6, 1, 0)) as 6,
sum(if(marketsegmentid = 7, 1, 0))
as 7, sum(if(marketsegmentid = 12, 1, 0)) as 12, sum(if(marketsegmentid = 17, 1, 0)) as 17,
sum(if(marketsegmentid = 22, 1, 0)) as 22, sum(if(marketsegmentid = 27, 1, 0)) as 27,
sum(if(marketsegmentid = 32, 1, 0)) as 32, sum(if(marketsegmentid = 37, 1, 0)) as 37,
sum(if(marketsegmentid = 42, 1, 0)) as 42, sum(if(marketsegmentid = 47, 1, 0)) as 47,
sum(if(marketsegmentid = 52, 1, 0)) as 52, sum(if(marketsegmentid = 97, 1, 0)) as 97,
sum(if(marketsegmentid = 102, 1, 0)) as 102, sum(if(marketsegmentid = 107, 1, 0)) as 107,
sum(if(marketsegmentid = 112, 1, 0)) as 112, sum(if(marketsegmentid = 117, 1, 0)) as 117,
sum(if(marketsegmentid = 122, 1, 0)) as 122, sum(if(marketsegmentid = 127, 1, 0)) as 127,
sum(if(marketsegmentid = 132, 1, 0)) as 132, sum(if(marketsegmentid = 137, 1, 0)) as 137,
sum(if(marketsegmentid = 142, 1, 0)) as 142
from storemarketsegments
group by storeid;
The query is used in a report and the results are exported to CSV. The 1's are used as flags in the result set.
The table I am querying is set up like this:
CREATE TABLE storemarketsegments(id INT NOT NULL, marketsegmentid INT NOT NULL);
The marketsegments are kept in a separate table:
CREATE TABLE marketsegment(ID INT NOT NULL AUTO_INCREMENT, PRIMARY
KEY(id), name VARCHAR(45), description VARCHAR(45));
Any help would be greatly appreciated. I am not sure if there is a way to write the query without hard coding the values and I don't mind updating the query in the report whenever new marketsegments are added but thought I would check. Thank you in advance for your assistance.
You can return each count as a separate row, and then filter as needed in the application layer:
select storeid, marketsegmentid, count(*) as Count
from storemarketsegments
group by storeid, marketsegmentid;

To print the list of user_id from the string list

I have a problem, i have a query that just simply displays the user id in set,for retreieving the user id i am calling the function and it gives me the below list as string
SELECT u.user_id
FROM user u
WHERE u.user_id
IN (
'2, 3, 4, 5, 6, 7, 22, 33, 44, 55, 66, 77, 13, 23, 43, 53, 63, 73'
)
but when i execute this query it displays only the first user_id ie: 2 and all the user id are present in the database
So any help is deeply appreciated
Your code:
SELECT u.user_id FROM user u WHERE u.user_id IN ( '2, 3, 4, 5, 6, 7, 22, 33, 44, 55, 66, 77, 13, 23, 43, 53, 63, 73' )
http://forums.mysql.com/read.php?10,217174
Im almost surprised you had any match.. each of the numbers in your in list, need to be individual strings, eg '1','2','3' etc.
Remove single quotes like this and try code again-
SELECT u.user_id FROM user u WHERE u.user_id IN ( 2, 3, 4, 5, 6, 7, 22, 33, 44, 55, 66, 77, 13, 23, 43, 53, 63, 73);
Remove the single quotes:
IN ( 2, 3, 4, 5, 6, 7, 22, 33, 44, 55, 66, 77, 13, 23, 43, 53, 63, 73 )

Getting top distinct records in MySQL

This is probably something very simple, so forgive my blonde moment :)
I have a table 'album'
* albumId
* albumOwnerId (who created)
* albumCSD (create stamp date)
Now what I am trying to do is to select the top 10 most recently updated albums. But, I don't want 10 albums from the same person coming back - I only want one album per unique person. I.E 10 albums from 10 different people.
So, this is what I have below, but it is not working properly and I just can't figure out why. Any ideas?
Thanks
SELECT DISTINCT(albumOwnerId), albumId
FROM album
ORDER BY albumCSD DESC
LIMIT 0,10
Here is some example data, followed by what I am trying to get. Hope this makes it clearer.
DATA:
albumOwnerID, albumId, albumCSD
18, 194, '2010-10-23 11:02:30'
23, 193, '2010-10-22 11:39:59'
22, 192, '2010-10-12 21:48:16'
21, 181, '2010-10-12 20:34:11'
21, 178, '2010-10-12 20:20:16'
19, 168, '2010-10-12 18:31:55'
18, 167, '2010-10-11 21:06:55'
20, 166, '2010-10-11 21:01:47'
18, 165, '2010-10-11 21:00:32'
20, 164, '2010-10-11 20:50:06'
17, 145, '2010-10-10 18:54:24'
17, 144, '2010-10-10 18:49:28'
17, 143, '2010-10-10 18:48:08'
17, 142, '2010-10-10 18:46:54'
16, 130, '2010-10-10 16:17:57'
16, 129, '2010-10-10 16:17:26'
16, 128, '2010-10-10 16:07:21'
15, 119, '2010-10-10 15:24:28'
15, 118, '2010-10-10 15:24:11'
14, 100, '2010-10-09 18:22:49'
14, 99, '2010-10-09 18:18:46'
11, 98, '2010-10-09 15:50:13'
11, 97, '2010-10-09 15:44:09'
11, 96, '2010-10-09 15:42:28'
11, 95, '2010-10-09 15:37:25'
DESIRED DATA:
18, 194, '2010-10-23 11:02:30'
23, 193, '2010-10-22 11:39:59'
22, 192, '2010-10-12 21:48:16'
21, 181, '2010-10-12 20:34:11'
19, 168, '2010-10-12 18:31:55'
17, 145, '2010-10-10 18:54:24'
16, 130, '2010-10-10 16:17:57'
15, 119, '2010-10-10 15:24:28'
14, 100, '2010-10-09 18:22:49'
11, 98, '2010-10-09 15:50:13'
I get results, you want to have, with this query
SELECT albumOwnerID, albumId, albumCSD
FROM album
WHERE albumCSD in
(SELECT Max(album.albumCSD) AS MaxvonalbumCSD
FROM album
GROUP BY album.albumOwnerID);
However in MS Access
select albumOwnerID, albumID
from album
Group by albumOwnerID, albumID
Order by albumcsd desc
LIMIT 0,10
EDIT:
select albumOwnerID, albumID
from album
where albumOwnerID in (select distinct albumOwnerID from album order by albumCSD )
LIMIT 0,10