select distinct values from json arrays in mysql database - mysql

I have table like this
my_table
id my_json
1 ['1','2','3']
2 ['2']
3 ['2','3']
...
12000 ....
I want to find all distinct values in json's arrays
like this
result
'1'
'2'
'3'
I have this code but i need split values to rows
set #items = (SELECT
GROUP_CONCAT(
REPLACE(REPLACE(lower(my_json), ']', ''), '[', '')
SEPARATOR ','
)
FROM my_table);
SELECT CONCAT ('[',#items,']') AS jarray;
result is
[1,2,3]
probably somebody have ideas?

You can use json_table:
select json_arrayagg(t2.v) from (select distinct t1.v from tbl t
cross join json_table(t.my_json, '$[*]' columns( v int path '$')) t1) t2
See fiddle.

Related

how to convert json data to specific alias and with comma seperated values in mysql

i have one table of product_attributes that is below:-
id attr_details
1 {"Manufacturer":"Lennovo","Warranty":"6 months"}
2 {"Manufacturer":"HP","Warranty":"6 months"}
3 {"Manufacturer":"DEll","Warranty":"12 months","Type":"DDR"}
4 {"Size":"36","Color":"Red","Material":"Fabric"}
My attr_details column data stored in json. My expected Output is like below:-
label values
Manufacturer Lennovo,HP,DEll
Warranty 6 months,12 months
Type DDR
Size 36
Color Red
Material Fabric
The attr_details json is not predefined it can be any userdefined input json. So can anyone help me how to achieve this output.
Try this:
SET #sql = NULL;
SELECT GROUP_CONCAT(CONCAT("SELECT '",colname,":' AS 'Label', GROUP_CONCAT(val)
FROM (SELECT JSON_UNQUOTE(JSON_EXTRACT(attr_details,'$.", colname,"')) AS 'val'
FROM mytable /*you can add the WHERE condition in here*/ GROUP BY val) A
GROUP BY Label") SEPARATOR " UNION ")
INTO #sql
FROM
(WITH RECURSIVE data AS (
SELECT attr_details,JSON_VALUE(JSON_KEYS(attr_details), '$[0]') AS colname, 0 AS idx FROM mytable
UNION
SELECT attr_details,JSON_VALUE(JSON_KEYS(attr_details), CONCAT('$[', d.idx + 1, ']'))
AS colname, d.idx + 1 AS idx FROM data AS d
WHERE d.idx < JSON_LENGTH(JSON_KEYS(attr_details)) - 1
) SELECT colname
FROM data
GROUP BY colname) V;
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
This is the closest I can get. The WITH RECURSIVE .. part I use to list out all the keys as single row value each. That one I was referring to a query from the comment section in the MariaDB documentation. Then I construct the query using combination of CONCAT + GROUP_CONCAT. Lastly, I used prepared statement to execute the query. Here is the final output of #sql:
SELECT 'Color:' AS 'Label', GROUP_CONCAT(val)
FROM (
SELECT JSON_UNQUOTE(JSON_EXTRACT(attr_details,'$.Color')) AS 'val'
FROM mytable GROUP BY val) A
GROUP BY Label UNION
SELECT 'Manufacturer:' AS 'Label', GROUP_CONCAT(val)
FROM (
SELECT JSON_UNQUOTE(JSON_EXTRACT(attr_details,'$.Manufacturer')) AS 'val'
FROM mytable GROUP BY val) A
GROUP BY Label UNION
SELECT 'Material:' AS 'Label', GROUP_CONCAT(val)
FROM (
SELECT JSON_UNQUOTE(JSON_EXTRACT(attr_details,'$.Material')) AS 'val'
FROM mytable GROUP BY val) A
GROUP BY Label UNION
SELECT 'Size:' AS 'Label', GROUP_CONCAT(val)
FROM (
SELECT JSON_UNQUOTE(JSON_EXTRACT(attr_details,'$.Size')) AS 'val'
FROM mytable GROUP BY val) A
GROUP BY Label UNION
SELECT 'Type:' AS 'Label', GROUP_CONCAT(val)
FROM (
SELECT JSON_UNQUOTE(JSON_EXTRACT(attr_details,'$.Type')) AS 'val'
FROM mytable GROUP BY val) A
GROUP BY Label UNION
SELECT 'Warranty:' AS 'Label', GROUP_CONCAT(val)
FROM (
SELECT JSON_UNQUOTE(JSON_EXTRACT(attr_details,'$.Warranty')) AS 'val'
FROM mytable GROUP BY val) A
GROUP BY Label
Demo fiddle
WITH cte AS (
SELECT DISTINCT jsontable.label
FROM test
CROSS JOIN JSON_TABLE( JSON_KEYS(test.attr_details),
'$[*]' COLUMNS ( label VARCHAR(255) PATH '$' )) jsontable
)
SELECT cte.label, GROUP_CONCAT(DISTINCT JSON_EXTRACT(test.attr_details, CONCAT('$.', cte.label))) `values`
FROM cte
CROSS JOIN test
GROUP BY label;
https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=ab92deb1dc20eef0f090f841ce2c6cd0

Extract Values from key and array on JSON

How could I extract the values in keys and array on json
json
{
"key1": "US",
"key2": "545644566",
"car": ["HONDA","TOYOTA","FORD"]
}
The resulted expected:
key
value
key1
US
key2
545644566
car
HONDA
car
TOYOTA
car
FORD
I tried using:
SELECT JSON_EXTRACT(u.json, CONCAT('$.','"',g.field_name,'"')),g.*
FROM table_json u
LEFT JOIN table_column g
ON JSON_CONTAINS(JSON_KEYS(u.json), JSON_QUOTE(g.field_name), '$')
You can use a dictionary table such as information_schema.tables along with iterating index values for some variables in order to generate rows for the elements of the arrays. The following code also works for non-array elements without need of a conditional. Apply JSON_KEYS() function to derive the key elements in the first subquery, and then use JSON_EXTRACT() for the respective indexes to derive the corresponding values of the arrays in the last subquery such as
WITH t1 AS
(
SELECT #i := #i + 1 AS i,
JSON_UNQUOTE(JSON_EXTRACT(kys, CONCAT('$[', #i - 1, ']'))) AS `key`,
JSON_EXTRACT(json,
CONCAT('$.',
JSON_EXTRACT(kys, CONCAT('$[', #i - 1, ']'))
)
) AS value, kys
FROM t
JOIN (SELECT #i := 0, JSON_KEYS(json) AS kys FROM t) AS k
JOIN information_schema.tables
)
SELECT #k := IF(#j=`key`,#k + 1,0) AS `index`,
#j := `key` AS `key`,
JSON_UNQUOTE(JSON_EXTRACT(value, CONCAT('$[', #k, ']'))) AS value
FROM t1
JOIN (SELECT #k := 0 ) AS k
LEFT JOIN information_schema.tables
ON #k < JSON_LENGTH(value) - 1
WHERE value IS NOT NULL
Demo

How to remove duplicates

I have two tables
test
and test1
test
id formula
1 A12+C32+D+X
2 K/Y
test1
id Code
6 A12
7 C32
100 A1
10 D
12 X
13 K
14 Y
How can I update formula(text formlas) filed in the table test to get
id formula
1 [6]+[7]+[10]+[12]
2 [13]/[14]
Itry the following script sqlfield but it doesn't rerun the correct result . It returns
RFORMULA
[6]+C32+D+X//need to remove it
[13]/Y//need to remove it
[13]/[14]//the best result
[100]2+[7]+[10]+[12]
[13]/Y//need to remove it
K/[14]//need to remove it
[6]+[7]+[10]+[12]//the best result
I am working on it for hours , any idea ?
Try by using the following:
SQL Fiddle
WITH a AS
( SELECT DISTINCT A.id group_id,
Split.A.value('.', 'VARCHAR(100)') AS Data
FROM
(SELECT id,
CAST ('<M>' + REPLACE(REPLACE(REPLACE(REPLACE(formula, '/', '/</M><M>'), '+','+</M><M>'), '-','+</M><M>'),'*','+</M><M>') + '</M>' AS XML) AS Data
FROM test
) AS A CROSS APPLY Data.nodes ('/M') AS Split(a)
),
b AS
(SELECT group_id,
REPLACE(LEFT(Data, LEN(Data)-1), t.[Code], concat(t.[id],RIGHT(Data,1))) yy,
REPLACE(Data, t.[Code], t.[id])uu
FROM a
LEFT JOIN test1 t
ON t.Code =LEFT(Data, LEN(Data)-1)
OR (t.Code=Data)
)
SELECT group_id,
REPLACE(REPLACE(REPLACE(REPLACE(stuff(
(SELECT uu FROM b WHERE newtable.group_id=group_id FOR XML PATH('')
), 1,1,''), 'uu', ''), '</>', ''),'<', ''), '>', '')
FROM b newtable
GROUP BY group_id
Here, I have also specified multiplication and subtraction, in case they appear in your expression column. You can edit it accordingly if there exist some other ones:
CAST ('<M>' + REPLACE(REPLACE(REPLACE(REPLACE(formula, '/', '/</M><M>'), '+','+</M><M>'), '-','+</M><M>'),'*','+</M><M>') + '</M>' AS XML) AS Data

FIND distinct value from comma separated single string

I have a table like this and I want to select distinct value in single column:
ID Value
1 13245|43558
2 45961|5051
3 43558| 5059
I need output Like this:
13245,43558,45961,5051,5059
try that:
select group_concat(distinct trim(substring_index(substring_index(value, '|', n.n), '|', -1)) separator ',' ) as `values`
from table1 t
cross join (select 1 as n union all select 2 ) n
order by `values`
DEMO HERE
Try this query :
SELECT GROUP_CONCAT(distinct(Value)) from table;

Mysql group_concat with sums also inside

I have a table of different attributes I want to sum, and then group concatenate them into a JSON string to make it easier to send over network. Here's a simplified table:
t1
type amount
'atr1' 10
'atr2' 10
'atr1' 17
'atr3' 20
'atr3' 4
I tried something like
select concat('{',
group_concat(
(select concat('"', type, '":', sum(amount)) from t1 group by type)
),
'}')
but failed.
I want to end up with '{"atr1":27,"atr2":10,"atr3":24}'
Try this query -
SELECT CONCAT('{', GROUP_CONCAT(c1), '}') FROM (
SELECT CONCAT('"', type, '":', SUM(amount)) c1 FROM t1 GROUP BY type
) t
something like
select
group_concat(concat('"', type, '":', TheSum))
FROM
(
SELECT SUM(amount) AS TheSum,type
FROM t1
GROUP BY type
) T