MySQL JSON_OBJECT instead of GROUP_CONCAT - mysql

I have the following sql query that works fine using GROUP_CONCAT:
SELECT orders.created_at,products.title, o_p.qty AS qty,
(SELECT GROUP_CONCAT(features.title,"\:", o_p_f.value, features.unit) FROM order_product_features AS o_p_f
LEFT JOIN product_features ON product_features.id = o_p_f.product_feature_id
LEFT JOIN features ON o_p_f.product_feature_id = product_features.id
AND features.id = product_features.feature_id
WHERE o_p_f.order_product_id = o_p.id
) AS prop
FROM orders
LEFT JOIN order_products AS o_p ON o_p.order_id = orders.id
LEFT JOIN products ON products.id = o_p.product_id
The above query returns result looks like the following screen shot:
Now, I want to replace GROUP_CONCAT with JSON_OBJECT or in other words, I want to have prop field to be JSON object. I have tried the following:
SELECT orders.created_at,products.title, o_p.qty AS qty,
JSON_OBJECT((SELECT GROUP_CONCAT("title",features.title,"value", o_p_f.value, 'unit',features.unit) FROM order_product_features AS o_p_f
LEFT JOIN product_features ON product_features.id = o_p_f.product_feature_id
LEFT JOIN features ON o_p_f.product_feature_id = product_features.id
AND features.id = product_features.feature_id
WHERE o_p_f.order_product_id = o_p.id
)) AS prop
FROM orders
LEFT JOIN order_products AS o_p ON o_p.order_id = orders.id
LEFT JOIN products ON products.id = o_p.product_id
However, the above query returns error:
1582 - Incorrect parameter count in the call to native function 'JSON_OBJECT'

On ≥ 5.7.22
JSON_OBJECTAGG(key, value) is more likely what you're after.
mysql> SELECT o_id, attribute, value FROM t3;
+------+-----------+-------+
| o_id | attribute | value |
+------+-----------+-------+
| 2 | color | red |
| 2 | fabric | silk |
| 3 | color | green |
| 3 | shape | square|
+------+-----------+-------+
4 rows in set (0.00 sec)
mysql> SELECT o_id, JSON_OBJECTAGG(attribute, value) FROM t3 GROUP BY o_id;
+------+----------------------------------------+
| o_id | JSON_OBJECTAGG(attribute, name) |
+------+----------------------------------------+
| 2 | {"color": "red", "fabric": "silk"} |
| 3 | {"color": "green", "shape": "square"} |
+------+----------------------------------------+
1 row in set (0.00 sec)
But as you don't provide your table structure I can't help you with the query.
It seem your handling of unit will grant you some extra work, as aggregate function use Key=>Value and will not take a third argument...
On ≥ 5.7
You'll have to make a bit of hand craft:
SELECT
o_id,
CONCAT(
'{',
GROUP_CONCAT(
TRIM(
LEADING '{' FROM TRIM(
TRAILING '}' FROM JSON_OBJECT(
`attribute`,
`value`
)
)
)
),
'}'
) json
FROM t3
GROUP BY o_id
≥ 5.5
Without any JSON function:
SELECT
o_id,
CONCAT(
'{',
GROUP_CONCAT(
CONCAT(
'"',
`attribute`,
'":"',
`value`,
'"'
)
),
'}'
) json
FROM t3
GROUP BY o_id

Related

Concatenate non empty CONCAT_GROUP to parent column in MySQL

I need to get the list of categories from store_cat, with the child COUNT from store_item (amount of products) and GROUP_CONCAT from store_cat_attributes (list of attributes). The thing is, using CONCAT function I need to attach the GROUP_CONCAT value with name column in the parent table (store_cat), and that's where it gets tricky.
This works fine:
SELECT
store_cat.id_cat AS id,
store_cat.name AS name,
GROUP_CONCAT(store_cat_attribute.name SEPARATOR ", ") AS attributes,
COUNT(store_item.id_item) AS products,
store_cat.flg_public AS flg_public
FROM store_cat
LEFT JOIN store_item ON store_item.id_cat = store_cat.id_cat
LEFT JOIN store_cat_attribute ON store_cat_attribute.id_cat = store_cat.id_cat
WHERE store_cat.id_store = 1
GROUP BY store_cat.id_cat
ORDER BY name
But this is what I would actually need. The problem is that, when I execute this query, the store_cat.name value shows an empty value when there are no attributes:
SELECT
store_cat.id_cat AS id,
CONCAT(store_cat.name, " (", GROUP_CONCAT(store_cat_attribute.name SEPARATOR ", "), ")") AS name,
COUNT(store_item.id_item) AS products,
store_cat.flg_public AS flg_public
FROM store_cat
LEFT JOIN store_item ON store_item.id_cat = store_cat.id_cat
LEFT JOIN store_cat_attribute ON store_cat_attribute.id_cat = store_cat.id_cat
WHERE store_cat.id_store = 1
GROUP BY store_cat.id_cat ORDER BY name
Basically, the idea is that the store_cat.name column should contain the attributes list with CONCAT and GROUP_CONCAT, just like this:
Comidas
Correas (S, M, L, XL)
Juguetes
Medicinas
Here's the current SQLfiddle. By the way, something is off with the attributes order in the current GROUP_CONCAT. It is displaying (XL, S, M, L) instead of (S, M, L, XL).
Problems to solve:
Use GROUP_CONCAT to concatenate the attributes to the category name only when there are attributes.
Use the store_cat_attributes.position to set the order for the GROUP_CONCAT values.
Any ideas? Thanks!
The following expression should return the results that you expect :
CONCAT(
store_cat.name,
IFNULL(
CONCAT(
' (',
GROUP_CONCAT(
store_cat_attribute.name
ORDER BY store_cat_attribute.position
SEPARATOR ', '
),
')'
),
''
)
) AS name
Basically, this just tries to GROUP_CONCAT() the attributes, and if the result is NULL then it turns the attribute list to an empty string. Please note that GROUP_CONCAT support ORDER BY.
I also fixed the GROUP BY clause : in non-ancient versions of MySQL, all non-aggregared columns must appear in the where clause (you are missing store_cat.name).
Demo on DB Fiddle with your sample data :
SELECT
store_cat.id_cat AS id,
CONCAT(
store_cat.name,
IFNULL(
CONCAT(
' (',
GROUP_CONCAT(store_cat_attribute.name ORDER BY store_cat_attribute.position SEPARATOR ', '),
')'
),
''
)
) AS name,
COUNT(store_item.id_item) AS products,
store_cat.flg_public AS flg_public
FROM
store_cat
LEFT JOIN store_item ON store_item.id_cat = store_cat.id_cat
LEFT JOIN store_cat_attribute ON store_cat_attribute.id_cat = store_cat.id_cat
WHERE store_cat.id_store = 1
GROUP BY store_cat.id_cat, store_cat.name
ORDER BY name;
| id | flg_public | name | products |
| --- | ---------- | --------------------- | -------- |
| 3 | 1 | Comidas | 0 |
| 2 | 1 | Correas (S, M, L, XL) | 4 |
| 1 | 1 | Juguetes | 2 |
| 4 | | Medicinas | 0 |

MYSQL CONCAT + GROUP_CONCAT + LEFT OUTER JOIN

I'm trying to make a link between 2 tables on mySQL but, i think it's a little bit harder than i thought.
I have 3 tables
* One which registers my rules informations
* One which registers my transfers informations
* One which make the pivot between the two first.
CREATE TABLE `rules` (
`id` int,
`Name` varchar(10)
);
INSERT INTO `rules` (`id`, `name`) VALUES
(1,'a'),
(2,'b'),
(3,'c'),
(4,'d');
CREATE TABLE `pivot` (
`id_rule` int,
`id_transfert` int
);
INSERT INTO `pivot` (`id_rule`, `id_transfert`) VALUES
(1,1),
(1,2),
(2,1),
(2,2),
(2,3);
CREATE TABLE `transferts` (
`id` int,
`aeroport` varchar(50),
`station` varchar(50)
);
INSERT INTO `transferts` (`id`, `aeroport`,`station`) VALUES
(1,'GVA','Flaine'),
(2,'GNB','La Tania'),
(3,'GNB','Flaine');
What i'm trying to do is to get all my rules with a column which gather all linked transfers as a JSON string. Like below
------------------------------------------------------------------------
| id | name | transferts |
------------------------------------------------------------------------
| 1 | a | {"GVA": "Flaine"} |
------------------------------------------------------------------------
| 2 | b | {"GVA": "Flaine", "GNB": "Flaine", "La Tania"} |
------------------------------------------------------------------------
What i do actually is this :
SELECT
rule.id, rule.name,GROUP_CONCAT(stations.transferts SEPARATOR ",") as transferts
FROM
rules rule
LEFT OUTER JOIN
pivot pivot
on
(pivot.id_rule = rule.id)
LEFT OUTER JOIN
(
SELECT id,
CONCAT(aeroport, ":",
GROUP_CONCAT(station)
) AS transferts
FROM transferts
GROUP BY aeroport
) stations
on
(pivot.id_transfert = stations.id)
GROUP BY
rule.id
But this is returning me a "null" value. I don't see what i'm doing wrong.
Is there someone who can help me please ?
FYI, I was inspired by this link
MySQL: GROUP_CONCAT with LEFT JOIN
With a MySQL version prior to 5.7.22 you can't use the JSON built-in functions.
You'll have to use a few imbricated GROUP_CONCAT subqueries to obtain your JSON string.
As told in the comments, your expected JSON string is not valid. The following answer will differ from your expected result, to fix this issue.
I suggest you proceed with a first query to get a column with the "aeroport" names, and another column with the associated stations formatted as a list, for each couple of "rule.id + aeroport_name".
This gives the following query:
mysql> select rules.id, name, concat ('"', aeroport, '":') as aeroport_name, group_concat('"', station, '"') as station_list
-> from rules
-> inner join pivot on rules.id = pivot.id_rule
-> inner join transferts on pivot.id_transfert = transferts.id
-> group by rules.id, aeroport_name;
+------+------+---------------+---------------------+
| id | name | aeroport_name | station_list |
+------+------+---------------+---------------------+
| 1 | a | "GNB": | "La Tania" |
| 1 | a | "GVA": | "Flaine" |
| 2 | b | "GNB": | "La Tania","Flaine" |
| 2 | b | "GVA": | "Flaine" |
+------+------+---------------+---------------------+
4 rows in set (0,00 sec)
Then, we are going to use this query as a subquery to associate each "station_list" to its given aeroport, in a rule id context, within a single string.
This give the following encapsulation:
mysql> select id, name, group_concat(aeroport_name, '[', station_list, ']') as aeroport_list
-> from (
-> select rules.id, name, concat ('"', aeroport, '":') as aeroport_name, group_concat('"', station, '"') as station_list
-> from rules
-> inner join pivot on rules.id = pivot.id_rule
-> inner join transferts on pivot.id_transfert = transferts.id
-> group by rules.id, aeroport_name
-> ) as isolated group by id;
+------+------+----------------------------------------------+
| id | name | aeroport_list |
+------+------+----------------------------------------------+
| 1 | a | "GNB":["La Tania"],"GVA":["Flaine"] |
| 2 | b | "GNB":["La Tania","Flaine"],"GVA":["Flaine"] |
+------+------+----------------------------------------------+
2 rows in set (0,00 sec)
And finally, we can now add the final "{}" encapsulation to our string, by adding a top level query over this:
mysql> select id, name, concat('{', aeroport_list, '}') as conf
-> from (
-> select id, name, group_concat(aeroport_name, '[', station_list, ']') as aeroport_list
-> from (
-> select rules.id, name, concat ('"', aeroport, '":') as aeroport_name, group_concat('"', station, '"') as station_list
-> from rules
-> inner join pivot on rules.id = pivot.id_rule
-> inner join transferts on pivot.id_transfert = transferts.id
-> group by rules.id, aeroport_name
-> ) as isolated group by id
-> ) as full_list;
+------+------+------------------------------------------------+
| id | name | conf |
+------+------+------------------------------------------------+
| 1 | a | {"GNB":["La Tania"],"GVA":["Flaine"]} |
| 2 | b | {"GNB":["Flaine","La Tania"],"GVA":["Flaine"]} |
+------+------+------------------------------------------------+
2 rows in set (0,01 sec)

MySql query on 3 tables not returning all rows

I have 3 tables: questions, options, comments_to_options(opt_comments).
I want to write a query that returns in each row the following values, concatenated:
A question, all options to it, all comments to each option.
My query is:
select
concat('{', '"qid":"', q.q_id, '", "qt":"', q.q_title,
'", "op":[', group_concat('{"oi":"', o.op_id, '", "ot":"', o.opt_value, '", ', oc_list, '}'
order by o.opt_upvotes desc), ']}')
as r
from questions q, options o,
(select o.op_id as ocid, concat('"oc":[', group_concat('{"oci":"', oc.opt_com_id, '", "occ":"', oc.opt_com_value, '"}'
order by oc.opt_com_added_at), ']')
as oc_list
from options o, opt_comments oc
where oc.opt_com_to=o.op_id
group by o.op_id)
as r2
where o.op_id=r2.ocid
and q.q_id=o.option_to
group by q.q_id
order by q.q_added_at desc
limit 3;
But the above query gives only those options that have at least one comment to them.
How should I modify?
You are using the old JOIN syntax with comma-separated lists of tables and subqueries. That syntax is correct, but generates INNER JOIN operations. Such joins suppress rows that don't match the join criterion.
You need to adopt the LEFT JOIN syntax. Without refactoring your entire query, I will say that you should change
FROM a,
(select something from z) AS b
WHERE a.value=b.value
to
FROM a
LEFT JOIN (select something from z) AS b ON a.value=b.value
Also, beware, you may encounter the character-string length limit in GROUP_CONCAT(). Read this:
MySQL and GROUP_CONCAT() maximum length
Use "left join".
Example:
create table opt (oid int,name varchar(100));
insert into opt values (1,'opt1');
insert into opt values (2,'opt2');
insert into opt values (3,'opt3');
create table optcom (oid int,com varchar(100));
insert into optcom values (1,'opt1_1');
insert into optcom values (1,'opt1_2');
insert into optcom values (3,'opt3_1');
When using "simple join":
select opt.*,optcom.* from opt join optcom on opt.oid=optcom.oid;
+------+------+------+--------+
| oid | name | oid | com |
+------+------+------+--------+
| 1 | opt1 | 1 | opt1_1 |
| 1 | opt1 | 1 | opt1_2 |
| 3 | opt3 | 3 | opt3_1 |
+------+------+------+--------+
When "left join":
select opt.*,optcom.* from opt left join optcom on opt.oid=optcom.oid;
+------+------+------+--------+
| oid | name | oid | com |
+------+------+------+--------+
| 1 | opt1 | 1 | opt1_1 |
| 1 | opt1 | 1 | opt1_2 |
| 2 | opt2 | NULL | NULL |
| 3 | opt3 | 3 | opt3_1 |
+------+------+------+--------+
To follow up on the above responses, the SQL amended to use outer joins:-
SELECT CONCAT('{', '"qid":"', q.q_id, '", "qt":"', q.q_title,'", "op":[', GROUP_CONCAT('{"oi":"', o.op_id, '", "ot":"', o.opt_value, '", ', oc_list, '}' ORDER BY o.opt_upvotes DESC), ']}') AS r
FROM options o
LEFT OUTER JOIN questions q
ON q.q_id = o.option_to
LEFT OUTER JOIN
(
SELECT o.op_id AS ocid,
CONCAT('"oc":[', GROUP_CONCAT('{"oci":"', oc.opt_com_id, '", "occ":"', oc.opt_com_value, '"}' ORDER BY oc.opt_com_added_at), ']') AS oc_list
FROM options o
INNER JOIN opt_comments oc
ON oc.opt_com_to=o.op_id
GROUP BY o.op_id
) r2
ON o.op_id = r2.ocid
GROUP BY q.q_id
ORDER BY q.q_added_at DESC
LIMIT 3;
Looking at this I am unsure about the join to the sub query. This appears to be bringing back an encoded string, but beyond the actual join nothing from this sub query is actually used.
As such I am unsure if that sub query is just being used to narrow down the rows returned (in which case joining against it using an INNER JOIN would be appropriate - and you may as well not bring back the encoded string), or if you have posted a cut down version of the query that you have been trying to debug.

MySQL create columns out of rows on the fly

My issue is that I have a table apidata that holds multiple rows of data for each domain. So when I query apidata I naturally get multiple rows as a result. Is there any way to turn those rows into columns? I ask because I'm already using a query to pull the domain data (page title, URL, top level domain, ip address etc) and I need to add the api data with it. I believe I have to do this in two queries but I would love to at least have one row per domain to make the query and loop as fast a possible.
So the question is, can I create columns out of rows on the fly?
Heres a SQL Fiddle => http://sqlfiddle.com/#!2/8e408/4
(Note, I didnt put the whole database in the fiddle just the tables that effect the query. If you think somethings missing that you need, let me know.)
Tool_Runs (id_sha is the main lookup value for tool runs)
| ID | ID_SHA |
+----+------------------------------------------+
| 1 | 68300DF58B2A8A6E098CB0B3D1A9AE80BBE5897A |
Domains (Run_id is FK to tool_runs.id)
| ID | RUN_ID |
+----+--------+
| 1 | 1 |
API Data
| ID | DOMAIN_ID | EXPORT_COLUMN | COLUMN_TITLE | VALUE |
+----+-----------+------------------+-------------------+-------+
| 1 | 1 | referringDomains | Referring Domains | 10 |
+----+-----------+------------------+-------------------+-------+
| 2 | 1 | linkCount | Backlink Count | 55 |
Heres my query now:
SELECT a.domain_id, a.export_column, a.column_title, a.value
FROM apidata AS a WHERE domain_id IN
(
SELECT d.id FROM tool_runs AS t
JOIN domains AS d ON d.run_id = t.id
WHERE id_sha = '68300DF58B2A8A6E098CB0B3D1A9AE80BBE5897A'
)
ORDER BY a.domain_id
And what I get is:
| DOMAIN_ID | EXPORT_COLUMN | COLUMN_TITLE | VALUE |
+-----------+------------------+-------------------+----------+
| 1 | referringDomains | Referring Domains | 10 |
+-----------+------------------+-------------------+----------+
| 1 | linkCount | Backlink Count | 55 |
But what I want is
| DOMAIN_ID | referringDomains | referringDomains_TITLE | linkCount | linkCount_TITLE |
+-----------+------------------+------------------------+-----------+-----------------+
| 1 | 10 | Referring Domains | 55 | Backlink Count |
What you are trying to is to pivot the table rows into columns. Unfortunately MySQL doesn't have a native pivot table operator, but you can use the CASE expression to do so:
SELECT
a.Domain_id,
MAX(CASE WHEN a.export_column = 'referringDomains' THEN a.value END) AS referringDomains,
MAX(CASE WHEN a.export_column = 'referringDomains' THEN a.column_title END) AS referringDomains_TITLE,
MAX(CASE WHEN a.export_column = 'linkCount' THEN a.value END) AS linkCount,
MAX(CASE WHEN a.export_column = 'linkCount' THEN a.column_title END) AS linkCount_TITLE
FROM apidata AS a
WHERE domain_id IN
(
SELECT d.id FROM tool_runs AS t
JOIN domains AS d ON d.run_id = t.id
WHERE id_sha = '68300DF58B2A8A6E098CB0B3D1A9AE80BBE5897A'
)
GROUP BY a.domain_id;
Updated SQL Fiddle Demo
Note that: If you want to do so for all the values in the export_column, you have to write a CASE expression for each value. But you can do that using dynamic sql like this:
SET #ecvalues = NULL;
SET #ectitles = NULL;
SET #sql = NULL;
SELECT
GROUP_CONCAT(DISTINCT CONCAT('MAX(IF(a.export_column = ''',
a.export_column, ''', a.value , NULL)) AS ', '''', a.export_column , '''')
) INTO #ecvalues
FROM apidata a;
SELECT
GROUP_CONCAT(DISTINCT CONCAT('MAX(IF(a.export_column = ''',
a.export_column, ''', column_title , NULL)) AS ', '''', CONCAT(a.export_column , '_Titles'), '''')
) INTO #ectitles
FROM apidata a;
SET #sql = CONCAT('SELECT
a.Domain_id, ', #ectitles , ',', #ecvalues, '
FROM apidata AS a
WHERE domain_id IN
(
SELECT d.id FROM tool_runs AS t
JOIN domains AS d ON d.run_id = t.id
WHERE id_sha = ''68300DF58B2A8A6E098CB0B3D1A9AE80BBE5897A''
)
GROUP BY a.domain_id;');
prepare stmt
FROM #sql;
execute stmt;
You can put that query inside a stored procedure.
Updated SQL Fiddle Demo
Just as a complement to the #MahmoudGamal answer you should know that for any new registry (EXPORT_COLUMN) you will have to add a new case statement.
So in order to do it dynamic you can create a procedure as described on this post at dba.stackexchange.
How to transpose/convert rows as columns in mysql
It shows how to do it dynamically.
If you want columns, go ahead and pivot as the example above. If you only want a single string, for some reporting reason, go ahead and do:
SELECT group_concat(CONCAT_WS(' ',a.domain_id, a.value, a.column_title, a.export_column, 'next row string separator'))
FROM apidata AS a WHERE domain_id IN
(
SELECT d.id FROM tool_runs AS t
JOIN domains AS d ON d.run_id = t.id
WHERE id_sha = '68300DF58B2A8A6E098CB0B3D1A9AE80BBE5897A'
)
ORDER BY a.domain_id

Refactor of SQL Statement

The following is working as expected. It shows the default CPM and CPC for the country if it is missing from the original table.
I am using a temporary table and I will like to know if it can be done without using temp table.
mysql> create temporary table country_list (country_name varchar(100));
Query OK, 0 rows affected (0.00 sec)
mysql> insert into country_list values ('IN'), ('SA');
Query OK, 2 rows affected (0.00 sec)
Records: 2 Duplicates: 0 Warnings: 0
select dt.country_name as country, dt.operator,
if (dt.cpm is null, (SELECT cpm FROM ox_bidding_configuration where country = '' and operator = ''), dt.cpm) as updated_cpm,
if (dt.cpc is null, (SELECT cpc FROM ox_bidding_configuration where country = '' and operator = ''), dt.cpc) as updated_cpc,
if (dt.cpa is null, (SELECT cpa FROM ox_bidding_configuration where country = '' and operator = ''), dt.cpa) as updated_cpa
from (
select a.country_name, b.operator, cpm, cpc, cpa from country_list as a left join ox_bidding_configuration as b on a.country_name = b.country group by b.country, b.operator, cpm, cpc, cpa) as dt;
+---------+----------+-------------+-------------+-------------+
| country | operator | updated_cpm | updated_cpc | updated_cpa |
+---------+----------+-------------+-------------+-------------+
| SA | NULL | 1.0000 | 1.0000 | 1.0000 |
| IN | | 2.0000 | 2.0000 | 2.0000 |
| IN | abcd | 11.0000 | 23.0000 | 4.0000 |
+---------+----------+-------------+-------------+-------------+
The country SA is not in the primary table. So I can not use self left joins. I will like to know if there is a better way.
For starters, from where I'm standing you're using the (temporary) country_list table as the PRIMARY table and match that against the ox_bidding_configuration where possible (left outer join); hence your result-set only contains the countries SA & IN.
Although I'm not 100% sure this syntax works on MYSQL, being used to TSQL I would write your query as :
SELECT dt.country_name as country,
dt.operator,
COALESCE(dt.cpm, def.cpm) as updated_cpm,
COALESCE(dt.cpc, def.cpc) as updated_cpd,
COALESCE(dt.cpa, def.cpa) as updated_cpa
FROM (SELECT DISTINCT a.country_name,
b.operator,
b.cpm,
b.cpc,
b.cpa
FROM country_list a
LEFT OUTER JOIN ox_bidding_configuration as b
ON b.country = a.country_name ) dt
LEFT OUTER JOIN ox_bidding_configuration def
ON def.country = ''
AND def.operator = ''
IMHO this is more clear, and probably it's slightly faster too as the default config only needs to be fetched once.
To get rid of the country_list table, you could convert it into an in-line query, but frankly I don't see the benefit of it.
Something along the lines of below :
SELECT dt.country_name as country,
dt.operator,
COALESCE(dt.cpm, def.cpm) as updated_cpm,
COALESCE(dt.cpc, def.cpc) as updated_cpd,
COALESCE(dt.cpa, def.cpa) as updated_cpa
FROM (SELECT DISTINCT a.country_name,
b.operator,
b.cpm,
b.cpc,
b.cpa
FROM (SELECT 'SA' AS country_name
UNION ALL
SELECT 'IN') a
LEFT OUTER JOIN ox_bidding_configuration as b
ON b.country = a.country_name ) dt
LEFT OUTER JOIN ox_bidding_configuration def
ON def.country = ''
AND def.operator = ''