Concatenate non empty CONCAT_GROUP to parent column in MySQL - mysql

I need to get the list of categories from store_cat, with the child COUNT from store_item (amount of products) and GROUP_CONCAT from store_cat_attributes (list of attributes). The thing is, using CONCAT function I need to attach the GROUP_CONCAT value with name column in the parent table (store_cat), and that's where it gets tricky.
This works fine:
SELECT
store_cat.id_cat AS id,
store_cat.name AS name,
GROUP_CONCAT(store_cat_attribute.name SEPARATOR ", ") AS attributes,
COUNT(store_item.id_item) AS products,
store_cat.flg_public AS flg_public
FROM store_cat
LEFT JOIN store_item ON store_item.id_cat = store_cat.id_cat
LEFT JOIN store_cat_attribute ON store_cat_attribute.id_cat = store_cat.id_cat
WHERE store_cat.id_store = 1
GROUP BY store_cat.id_cat
ORDER BY name
But this is what I would actually need. The problem is that, when I execute this query, the store_cat.name value shows an empty value when there are no attributes:
SELECT
store_cat.id_cat AS id,
CONCAT(store_cat.name, " (", GROUP_CONCAT(store_cat_attribute.name SEPARATOR ", "), ")") AS name,
COUNT(store_item.id_item) AS products,
store_cat.flg_public AS flg_public
FROM store_cat
LEFT JOIN store_item ON store_item.id_cat = store_cat.id_cat
LEFT JOIN store_cat_attribute ON store_cat_attribute.id_cat = store_cat.id_cat
WHERE store_cat.id_store = 1
GROUP BY store_cat.id_cat ORDER BY name
Basically, the idea is that the store_cat.name column should contain the attributes list with CONCAT and GROUP_CONCAT, just like this:
Comidas
Correas (S, M, L, XL)
Juguetes
Medicinas
Here's the current SQLfiddle. By the way, something is off with the attributes order in the current GROUP_CONCAT. It is displaying (XL, S, M, L) instead of (S, M, L, XL).
Problems to solve:
Use GROUP_CONCAT to concatenate the attributes to the category name only when there are attributes.
Use the store_cat_attributes.position to set the order for the GROUP_CONCAT values.
Any ideas? Thanks!

The following expression should return the results that you expect :
CONCAT(
store_cat.name,
IFNULL(
CONCAT(
' (',
GROUP_CONCAT(
store_cat_attribute.name
ORDER BY store_cat_attribute.position
SEPARATOR ', '
),
')'
),
''
)
) AS name
Basically, this just tries to GROUP_CONCAT() the attributes, and if the result is NULL then it turns the attribute list to an empty string. Please note that GROUP_CONCAT support ORDER BY.
I also fixed the GROUP BY clause : in non-ancient versions of MySQL, all non-aggregared columns must appear in the where clause (you are missing store_cat.name).
Demo on DB Fiddle with your sample data :
SELECT
store_cat.id_cat AS id,
CONCAT(
store_cat.name,
IFNULL(
CONCAT(
' (',
GROUP_CONCAT(store_cat_attribute.name ORDER BY store_cat_attribute.position SEPARATOR ', '),
')'
),
''
)
) AS name,
COUNT(store_item.id_item) AS products,
store_cat.flg_public AS flg_public
FROM
store_cat
LEFT JOIN store_item ON store_item.id_cat = store_cat.id_cat
LEFT JOIN store_cat_attribute ON store_cat_attribute.id_cat = store_cat.id_cat
WHERE store_cat.id_store = 1
GROUP BY store_cat.id_cat, store_cat.name
ORDER BY name;
| id | flg_public | name | products |
| --- | ---------- | --------------------- | -------- |
| 3 | 1 | Comidas | 0 |
| 2 | 1 | Correas (S, M, L, XL) | 4 |
| 1 | 1 | Juguetes | 2 |
| 4 | | Medicinas | 0 |

Related

MySQL JSON_OBJECT instead of GROUP_CONCAT

I have the following sql query that works fine using GROUP_CONCAT:
SELECT orders.created_at,products.title, o_p.qty AS qty,
(SELECT GROUP_CONCAT(features.title,"\:", o_p_f.value, features.unit) FROM order_product_features AS o_p_f
LEFT JOIN product_features ON product_features.id = o_p_f.product_feature_id
LEFT JOIN features ON o_p_f.product_feature_id = product_features.id
AND features.id = product_features.feature_id
WHERE o_p_f.order_product_id = o_p.id
) AS prop
FROM orders
LEFT JOIN order_products AS o_p ON o_p.order_id = orders.id
LEFT JOIN products ON products.id = o_p.product_id
The above query returns result looks like the following screen shot:
Now, I want to replace GROUP_CONCAT with JSON_OBJECT or in other words, I want to have prop field to be JSON object. I have tried the following:
SELECT orders.created_at,products.title, o_p.qty AS qty,
JSON_OBJECT((SELECT GROUP_CONCAT("title",features.title,"value", o_p_f.value, 'unit',features.unit) FROM order_product_features AS o_p_f
LEFT JOIN product_features ON product_features.id = o_p_f.product_feature_id
LEFT JOIN features ON o_p_f.product_feature_id = product_features.id
AND features.id = product_features.feature_id
WHERE o_p_f.order_product_id = o_p.id
)) AS prop
FROM orders
LEFT JOIN order_products AS o_p ON o_p.order_id = orders.id
LEFT JOIN products ON products.id = o_p.product_id
However, the above query returns error:
1582 - Incorrect parameter count in the call to native function 'JSON_OBJECT'
On ≥ 5.7.22
JSON_OBJECTAGG(key, value) is more likely what you're after.
mysql> SELECT o_id, attribute, value FROM t3;
+------+-----------+-------+
| o_id | attribute | value |
+------+-----------+-------+
| 2 | color | red |
| 2 | fabric | silk |
| 3 | color | green |
| 3 | shape | square|
+------+-----------+-------+
4 rows in set (0.00 sec)
mysql> SELECT o_id, JSON_OBJECTAGG(attribute, value) FROM t3 GROUP BY o_id;
+------+----------------------------------------+
| o_id | JSON_OBJECTAGG(attribute, name) |
+------+----------------------------------------+
| 2 | {"color": "red", "fabric": "silk"} |
| 3 | {"color": "green", "shape": "square"} |
+------+----------------------------------------+
1 row in set (0.00 sec)
But as you don't provide your table structure I can't help you with the query.
It seem your handling of unit will grant you some extra work, as aggregate function use Key=>Value and will not take a third argument...
On ≥ 5.7
You'll have to make a bit of hand craft:
SELECT
o_id,
CONCAT(
'{',
GROUP_CONCAT(
TRIM(
LEADING '{' FROM TRIM(
TRAILING '}' FROM JSON_OBJECT(
`attribute`,
`value`
)
)
)
),
'}'
) json
FROM t3
GROUP BY o_id
≥ 5.5
Without any JSON function:
SELECT
o_id,
CONCAT(
'{',
GROUP_CONCAT(
CONCAT(
'"',
`attribute`,
'":"',
`value`,
'"'
)
),
'}'
) json
FROM t3
GROUP BY o_id

Concatenate results from SQL query base on another column

SELECT DISTINCT * FROM ( SELECT hostname, table2.user-id, table2.user-team from
INNER JOIN table2 on table1.id = table2.id)
So at the moment my SQL query outputs this:
hostname user-id user-team
a 1 alpha
a 2 beta
b 3 beta
c 4 alpha
c 1 null
c 3 alpha
but what I want is something like this:
hostname user-id user-team
a 1, 2 alpha, beta
b 3 beta
c 4, 1, 3 alpha
I'm trying to use a GROUP BY hostname statement at the end of my query, and a GROUP_CONCAT(DISTINCT user_id SEPARATOR ', '), GROUP_CONCAT(DISTINCT user_team SEPARATOR ', '),
But this then only returns the first hostname result and all the values possible for the user-id and team-id. I feel like I'm close, but I can't quite get it. Any help?
(At present it returns)
hostname user-id user-team
a 1,2,3,4 alpha, beta
with this as the SQL query
SELECT DISTINCT *
FROM ( SELECT hostname,
GROUP_CONCAT(DISTINCT table2.user-id SEPARATOR(', '),
GROUP_CONCAT(DISTINCT table2.team-id SEPARATOR(', ')
from table1
INNER JOIN table2 on table1.id = table2.id)
GROUP BY hostname
Although the queries arean't 100% accurate, that is the logic in them (they just contain far more columns in the real world problem I have.)
I think you miss the GROUP BY
SQL DEMO
SELECT hostname,
GROUP_CONCAT(DISTINCT user_id SEPARATOR ', ') as users,
GROUP_CONCAT(DISTINCT user_team SEPARATOR ', ') as teams
FROM Table1
GROUP BY hostname
OUTPUT
| hostname | users | teams |
|----------|---------|-------------|
| a | 1, 2 | alpha, beta |
| b | 3 | beta |
| c | 4, 1, 3 | alpha |
EDIT: After you edit your question, the problem was you put the GROUP BY outside of the subquery

Use a GROUP_CONCAT and JOIN in a subquery

I have a DB under MySQL with one main table with unique id, few tables that list different choices (with one id column, one text column to describe the item). So, for one record in the main table, I can have multiple choices associated to the choice table.
I'd like to create a View where all information could be visible, using GROUP_CONCAT to concatenate into one field the different choices from a given 'choice' table. However, my query repeats many times each list of choices when the record is related to other multiple choices from another 'choice' table. The query returns all the possible combinations between those choices...
Here my query (reduced to 2 'choice' tables -t_age, t_animal- for the example)
SELECT general.id_g,
GROUP_CONCAT(CAST(t_age AS CHAR) SEPARATOR ', ') AS age,
GROUP_CONCAT(CAST(t_animal AS CHAR) SEPARATOR ', ') AS animal
FROM general
LEFT JOIN interm_age
INNER JOIN t_age ON interm_age.id_age = t_age.id_age
ON general.id_g = interm_age.id_g
LEFT JOIN interm_animal
INNER JOIN t_animal ON interm_animal.id_animal = t_animal.id_animal
ON general.id_g = interm_animal.id_g
GROUP BY id_g;
I tried to include each CONCAT/JOIN within a subquery into a main SELECT as followed, but MySQL tells me "returns more than 1row", which is the case indeed. And?
SELECT id_g, (
SELECT GROUP_CONCAT(CAST(t_age AS CHAR) SEPARATOR ', ')
FROM general
LEFT JOIN interm_age
INNER JOIN t_age ON interm_age.id_age = t_age.id_age
ON general.id_g = interm_age.id_g
GROUP BY general.id_g )
FROM general;
[EDIT]
In more details, this is my DB (with FK)
general
-----------
id_g | date
902 | 2016/01/01
956 | 2016/02/01
959 | 2016/02/01
interm_age
-----------
id_age | id_g
1 | 902
3 | 902
1 | 956
4 | 956
interm_animal
-----------
id_animal | id_g
1 | 902
5 | 902
5 | 959
7 | 959
t_age
-----------
id_age | age
1 | <10y
3 | >10y
4 | >60y
t_animal
-----------
id_animal | animal
1 | bird
5 | mammal
7 | insect
And I would like something like :
id_g | date | age | animal
902 | 2016/01/01 | <10y, >10y | bird, mammal
and so on...
Thanks in advance!
Although it would be better if you provided some sample data, actual results, and expected results, I'll have a stab at the solution.
The issue probably is that you have multiple matching records in the joined tables, which leads to duplication of values within group_concat().
As MySQL documentation on group_concat() indicates, you can use the distinct keyword to remove duplicate values:
To eliminate duplicate values, use the DISTINCT clause.
SELECT general.id_g,
GROUP_CONCAT(DISTINCT CAST(t_age AS CHAR) SEPARATOR ', ') AS age,
GROUP_CONCAT(DISTINCT CAST(t_animal AS CHAR) SEPARATOR ', ') AS animal
FROM general
LEFT JOIN interm_age
INNER JOIN t_age ON interm_age.id_age = t_age.id_age
ON general.id_g = interm_age.id_g
LEFT JOIN interm_animal
INNER JOIN t_animal ON interm_animal.id_animal = t_animal.id_animal
ON general.id_g = interm_animal.id_g
GROUP BY id_g;
Try this
SELECT general.id_g,
GROUP_CONCAT(CAST(t_age AS CHAR) SEPARATOR ', ') AS age,
GROUP_CONCAT(CAST(t_animal AS CHAR) SEPARATOR ', ') AS animal
FROM general
LEFT JOIN interm_age
INNER JOIN t_age ON interm_age.id_age = t_age.id_age AND general.id_g = interm_age.id_g
LEFT JOIN interm_animal
INNER JOIN t_animal ON interm_animal.id_animal = t_animal.id_animal AND general.id_g = interm_animal.id_g
GROUP BY id_g;
Hope this helps.

MySql query on 3 tables not returning all rows

I have 3 tables: questions, options, comments_to_options(opt_comments).
I want to write a query that returns in each row the following values, concatenated:
A question, all options to it, all comments to each option.
My query is:
select
concat('{', '"qid":"', q.q_id, '", "qt":"', q.q_title,
'", "op":[', group_concat('{"oi":"', o.op_id, '", "ot":"', o.opt_value, '", ', oc_list, '}'
order by o.opt_upvotes desc), ']}')
as r
from questions q, options o,
(select o.op_id as ocid, concat('"oc":[', group_concat('{"oci":"', oc.opt_com_id, '", "occ":"', oc.opt_com_value, '"}'
order by oc.opt_com_added_at), ']')
as oc_list
from options o, opt_comments oc
where oc.opt_com_to=o.op_id
group by o.op_id)
as r2
where o.op_id=r2.ocid
and q.q_id=o.option_to
group by q.q_id
order by q.q_added_at desc
limit 3;
But the above query gives only those options that have at least one comment to them.
How should I modify?
You are using the old JOIN syntax with comma-separated lists of tables and subqueries. That syntax is correct, but generates INNER JOIN operations. Such joins suppress rows that don't match the join criterion.
You need to adopt the LEFT JOIN syntax. Without refactoring your entire query, I will say that you should change
FROM a,
(select something from z) AS b
WHERE a.value=b.value
to
FROM a
LEFT JOIN (select something from z) AS b ON a.value=b.value
Also, beware, you may encounter the character-string length limit in GROUP_CONCAT(). Read this:
MySQL and GROUP_CONCAT() maximum length
Use "left join".
Example:
create table opt (oid int,name varchar(100));
insert into opt values (1,'opt1');
insert into opt values (2,'opt2');
insert into opt values (3,'opt3');
create table optcom (oid int,com varchar(100));
insert into optcom values (1,'opt1_1');
insert into optcom values (1,'opt1_2');
insert into optcom values (3,'opt3_1');
When using "simple join":
select opt.*,optcom.* from opt join optcom on opt.oid=optcom.oid;
+------+------+------+--------+
| oid | name | oid | com |
+------+------+------+--------+
| 1 | opt1 | 1 | opt1_1 |
| 1 | opt1 | 1 | opt1_2 |
| 3 | opt3 | 3 | opt3_1 |
+------+------+------+--------+
When "left join":
select opt.*,optcom.* from opt left join optcom on opt.oid=optcom.oid;
+------+------+------+--------+
| oid | name | oid | com |
+------+------+------+--------+
| 1 | opt1 | 1 | opt1_1 |
| 1 | opt1 | 1 | opt1_2 |
| 2 | opt2 | NULL | NULL |
| 3 | opt3 | 3 | opt3_1 |
+------+------+------+--------+
To follow up on the above responses, the SQL amended to use outer joins:-
SELECT CONCAT('{', '"qid":"', q.q_id, '", "qt":"', q.q_title,'", "op":[', GROUP_CONCAT('{"oi":"', o.op_id, '", "ot":"', o.opt_value, '", ', oc_list, '}' ORDER BY o.opt_upvotes DESC), ']}') AS r
FROM options o
LEFT OUTER JOIN questions q
ON q.q_id = o.option_to
LEFT OUTER JOIN
(
SELECT o.op_id AS ocid,
CONCAT('"oc":[', GROUP_CONCAT('{"oci":"', oc.opt_com_id, '", "occ":"', oc.opt_com_value, '"}' ORDER BY oc.opt_com_added_at), ']') AS oc_list
FROM options o
INNER JOIN opt_comments oc
ON oc.opt_com_to=o.op_id
GROUP BY o.op_id
) r2
ON o.op_id = r2.ocid
GROUP BY q.q_id
ORDER BY q.q_added_at DESC
LIMIT 3;
Looking at this I am unsure about the join to the sub query. This appears to be bringing back an encoded string, but beyond the actual join nothing from this sub query is actually used.
As such I am unsure if that sub query is just being used to narrow down the rows returned (in which case joining against it using an INNER JOIN would be appropriate - and you may as well not bring back the encoded string), or if you have posted a cut down version of the query that you have been trying to debug.

MySQL create columns out of rows on the fly

My issue is that I have a table apidata that holds multiple rows of data for each domain. So when I query apidata I naturally get multiple rows as a result. Is there any way to turn those rows into columns? I ask because I'm already using a query to pull the domain data (page title, URL, top level domain, ip address etc) and I need to add the api data with it. I believe I have to do this in two queries but I would love to at least have one row per domain to make the query and loop as fast a possible.
So the question is, can I create columns out of rows on the fly?
Heres a SQL Fiddle => http://sqlfiddle.com/#!2/8e408/4
(Note, I didnt put the whole database in the fiddle just the tables that effect the query. If you think somethings missing that you need, let me know.)
Tool_Runs (id_sha is the main lookup value for tool runs)
| ID | ID_SHA |
+----+------------------------------------------+
| 1 | 68300DF58B2A8A6E098CB0B3D1A9AE80BBE5897A |
Domains (Run_id is FK to tool_runs.id)
| ID | RUN_ID |
+----+--------+
| 1 | 1 |
API Data
| ID | DOMAIN_ID | EXPORT_COLUMN | COLUMN_TITLE | VALUE |
+----+-----------+------------------+-------------------+-------+
| 1 | 1 | referringDomains | Referring Domains | 10 |
+----+-----------+------------------+-------------------+-------+
| 2 | 1 | linkCount | Backlink Count | 55 |
Heres my query now:
SELECT a.domain_id, a.export_column, a.column_title, a.value
FROM apidata AS a WHERE domain_id IN
(
SELECT d.id FROM tool_runs AS t
JOIN domains AS d ON d.run_id = t.id
WHERE id_sha = '68300DF58B2A8A6E098CB0B3D1A9AE80BBE5897A'
)
ORDER BY a.domain_id
And what I get is:
| DOMAIN_ID | EXPORT_COLUMN | COLUMN_TITLE | VALUE |
+-----------+------------------+-------------------+----------+
| 1 | referringDomains | Referring Domains | 10 |
+-----------+------------------+-------------------+----------+
| 1 | linkCount | Backlink Count | 55 |
But what I want is
| DOMAIN_ID | referringDomains | referringDomains_TITLE | linkCount | linkCount_TITLE |
+-----------+------------------+------------------------+-----------+-----------------+
| 1 | 10 | Referring Domains | 55 | Backlink Count |
What you are trying to is to pivot the table rows into columns. Unfortunately MySQL doesn't have a native pivot table operator, but you can use the CASE expression to do so:
SELECT
a.Domain_id,
MAX(CASE WHEN a.export_column = 'referringDomains' THEN a.value END) AS referringDomains,
MAX(CASE WHEN a.export_column = 'referringDomains' THEN a.column_title END) AS referringDomains_TITLE,
MAX(CASE WHEN a.export_column = 'linkCount' THEN a.value END) AS linkCount,
MAX(CASE WHEN a.export_column = 'linkCount' THEN a.column_title END) AS linkCount_TITLE
FROM apidata AS a
WHERE domain_id IN
(
SELECT d.id FROM tool_runs AS t
JOIN domains AS d ON d.run_id = t.id
WHERE id_sha = '68300DF58B2A8A6E098CB0B3D1A9AE80BBE5897A'
)
GROUP BY a.domain_id;
Updated SQL Fiddle Demo
Note that: If you want to do so for all the values in the export_column, you have to write a CASE expression for each value. But you can do that using dynamic sql like this:
SET #ecvalues = NULL;
SET #ectitles = NULL;
SET #sql = NULL;
SELECT
GROUP_CONCAT(DISTINCT CONCAT('MAX(IF(a.export_column = ''',
a.export_column, ''', a.value , NULL)) AS ', '''', a.export_column , '''')
) INTO #ecvalues
FROM apidata a;
SELECT
GROUP_CONCAT(DISTINCT CONCAT('MAX(IF(a.export_column = ''',
a.export_column, ''', column_title , NULL)) AS ', '''', CONCAT(a.export_column , '_Titles'), '''')
) INTO #ectitles
FROM apidata a;
SET #sql = CONCAT('SELECT
a.Domain_id, ', #ectitles , ',', #ecvalues, '
FROM apidata AS a
WHERE domain_id IN
(
SELECT d.id FROM tool_runs AS t
JOIN domains AS d ON d.run_id = t.id
WHERE id_sha = ''68300DF58B2A8A6E098CB0B3D1A9AE80BBE5897A''
)
GROUP BY a.domain_id;');
prepare stmt
FROM #sql;
execute stmt;
You can put that query inside a stored procedure.
Updated SQL Fiddle Demo
Just as a complement to the #MahmoudGamal answer you should know that for any new registry (EXPORT_COLUMN) you will have to add a new case statement.
So in order to do it dynamic you can create a procedure as described on this post at dba.stackexchange.
How to transpose/convert rows as columns in mysql
It shows how to do it dynamically.
If you want columns, go ahead and pivot as the example above. If you only want a single string, for some reporting reason, go ahead and do:
SELECT group_concat(CONCAT_WS(' ',a.domain_id, a.value, a.column_title, a.export_column, 'next row string separator'))
FROM apidata AS a WHERE domain_id IN
(
SELECT d.id FROM tool_runs AS t
JOIN domains AS d ON d.run_id = t.id
WHERE id_sha = '68300DF58B2A8A6E098CB0B3D1A9AE80BBE5897A'
)
ORDER BY a.domain_id