How to create json tree from postgresql table - json

I have the table with hierarchical data. I want to aggregate the data from one table to a complex json. The elements should be grouped by section column. And nested json should be grouped by subsection column
----------------------------------------------
| id | section | subsection | subsubsection |
----------------------------------------------
| 111 | s_1 | null | null | // root section
----------------------------------------------
| 222 | s_1 | ss_2 | null | // root subsection
----------------------------------------------
| 333 | s_1 | ss_2 | sss_3 |
----------------------------------------------
| 444 | s_1 | ss_2 | sss_4 |
----------------------------------------------
| 555 | s_2 | null | null | // root section
----------------------------------------------
| 666 | s_2 | ss_3 | null | // root subsection
----------------------------------------------
I want to create json trees and save them as view
{
"id": 111,
"section": "s_1",
"subsections": [
{
"id": 222,
"subsection": "ss_2",
"subsections": [
{
"id": 333,
"subsection": "sss_3"
},
{
"id": 444,
"subsection": "sss_4"
}
]
}
]
}
What I tried to do:
create or replace view my_view(
name
) as
SELECT
(
SELECT
json_build_object(
'section', a.section,
'subsections', a.sections
)
FROM (SELECT b.section,
json_agg(
json_build_object(
'subsection', b.subsection,
'subsubsections', b.subsections
)
) AS sections
FROM (SELECT c.section,
c.subsection,
json_agg(
json_build_object(
'subsubsection', c.subsubsection
)
) AS subsections
FROM table AS c
GROUP BY section, subsection
) AS b
GROUP BY b.section) AS a
) AS name;
But in my solution I can't and an id to each tree node. How to add the each node id to result tree?

I recreated your case with:
create table test (id int, section text, subsection text, subsubsection text);
insert into test values (111,'s_1', null, null);
insert into test values (222,'s_1', 's_2', null);
insert into test values (333,'s_1', 's_2', 's_3');
insert into test values (444,'s_1', 's_2', 's_4');
insert into test values (555,'s_5', null, null);
insert into test values (666,'s_5', 's_6', null);
Note: I changed slightly the names of the sections to be unique.
Result (Ghost is null)
id | section | subsection | subsubsection
-----+---------+------------+---------------
111 | s_1 | πŸ‘» | πŸ‘»
222 | s_1 | s_2 | πŸ‘»
333 | s_1 | s_2 | s_3
444 | s_1 | s_2 | s_4
555 | s_5 | πŸ‘» | πŸ‘»
666 | s_5 | s_6 | πŸ‘»
(6 rows)
Then I approached the problem in phases:
Get the list of subsubsections and related sections and subsections
SELECT c.section,
c.subsection,
c.subsubsection,
json_build_object(
'id',c.id,
'subsubsection', c.subsubsection
) AS subsubsections
FROM test AS c
WHERE c.subsubsection is not null
Which gives
section | subsection | subsubsection | subsubsections
---------+------------+---------------+---------------------------------------
s_1 | s_2 | s_3 | {"id" : 333, "subsubsection" : "s_3"}
s_1 | s_2 | s_4 | {"id" : 444, "subsubsection" : "s_4"}
(2 rows)
Then, build subsections by joining the query above with a query specific at subsections level
SELECT c.section,
c.subsection,
json_build_object(
'id',c.id,
'subsection', c.subsection,
'subsubsection', json_agg(subsubsections.subsubsections)
) AS subsections
FROM test AS c
left outer join subsubsections on c.section = subsubsections.section and c.subsection = subsubsections.subsection
WHERE c.subsection is not null and c.subsubsection is null
group by c.section,
c.subsection,
c.id
Note: the subsubsections table is the alias to the previous table
Result
section | subsection | subsections
---------+------------+--------------------------------------------------------------------------------------------------------------------------------------
s_1 | s_2 | {"id" : 222, "subsection" : "s_2", "subsubsection" : [{"id" : 333, "subsubsection" : "s_3"}, {"id" : 444, "subsubsection" : "s_4"}]}
s_5 | s_6 | {"id" : 666, "subsection" : "s_6", "subsubsection" : [null]}
(2 rows)
Last, a query at section level, joining the results of the subsection above
SELECT c.section,
json_build_object(
'id',c.id,
'section', c.section,
'subsection', json_agg(subsections.subsections)
) AS subsections
FROM test AS c
left outer join subsections on c.section = subsections.section
WHERE c.subsection is null
group by c.section,
c.id
The whole query is
WITH subsubsections as (
SELECT c.section,
c.subsection,
c.subsubsection,
json_build_object(
'id',c.id,
'subsubsection', c.subsubsection
) AS subsubsections
FROM test AS c
WHERE c.subsection is not null and c.subsubsection is not null
)
, subsections as (
SELECT c.section,
c.subsection,
json_build_object(
'id',c.id,
'subsection', c.subsection,
'subsubsection', json_agg(subsubsections.subsubsections)
) AS subsections
FROM test AS c
left outer join subsubsections on c.section = subsubsections.section and c.subsection = subsubsections.subsection
WHERE c.subsection is not null and c.subsubsection is null
group by c.section,
c.subsection,
c.id
)
SELECT c.section,
json_build_object(
'id',c.id,
'section', c.section,
'subsection', json_agg(subsections.subsections)
) AS subsections
FROM test AS c
left outer join subsections on c.section = subsections.section
WHERE c.subsection is null
group by c.section,
c.id
;
And the result is
section | subsections
---------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
s_1 | {"id" : 111, "section" : "s_1", "subsection" : [{"id" : 222, "subsection" : "s_2", "subsubsection" : [{"id" : 333, "subsubsection" : "s_3"}, {"id" : 444, "subsubsection" : "s_4"}]}]}
s_5 | {"id" : 555, "section" : "s_5", "subsection" : [{"id" : 666, "subsection" : "s_6", "subsubsection" : [null]}]}
(2 rows)
Note, this will work only if section and subsection names are "unique" per subtree.

Related

How to search a JSON in MySQL dB to satisfy the required condition?

Assume I have a data list in the MySQL dB as below:
+----+------------+-------------------------------------------+
| id | index_date | token_json |
+----+------------+-------------------------------------------+
| 0 | 20200902_0 | [{"tk": [1, 2], "amt": [20, 49]}] |
| 1 | 20200902_1 | [{"tk": [4, 3], "amt": [10, 39]} |
| 2 | 20200902_2 | [{"tk": [7, 4], "amt": [12, 29]} |
| 3 | 20200902_3 | [{"tk": [8, 7, 6], "amt": [13, 19, 19]} |
| 4 | 20200902_4 | [{"tk": [9, 6], "amt": [25, 59] |
+----+------------+-------------------------------------------+
The token_json field is actually a JSON string stored. Now, I would like to get the subset of this data with criteria "sub_token" > 5.
The output subset should looks like below:
+----+------------+-------------------------------------------+
| id | index_date | token_json |
+----+------------+-------------------------------------------+
| 2 | 20200902_2 | [{"tk": [7, 4], "amt": [12, 29]} |
| 3 | 20200902_3 | [{"tk": [8, 7, 6], "amt": [13, 19, 19]} |
| 4 | 20200902_4 | [{"tk": [9, 6], "amt": [25, 59] |
+----+------------+-------------------------------------------+
I try the command below but doesn't work.
SELECT * from my_table
WHERE JSON_EXTRACT(token_json, '$.tk') > 5;
Can anyone guide me how to get such subset?
My MySQL version is 5.7.19-17-57-log
If your DB's version is 8+, then you can use JSON_TABLE() function as
WITH t(arr, id, token_json) AS
(
SELECT JSON_EXTRACT(token_json, '$[*].tk'), id, token_json FROM my_table t
)
SELECT id, token_json
FROM t
WHERE EXISTS
( SELECT 1
FROM t AS t2
JOIN JSON_TABLE( CAST( arr AS JSON ), "$[*]"
COLUMNS(
val INT PATH "$"
)
) js
WHERE t2.token_json = t.token_json
AND val > 5 )
Update : Considering your DB version is 5.7 , you can use such a method as below :
SELECT id, token_json
FROM my_table
WHERE id IN
(SELECT id
FROM (SELECT JSON_EXTRACT(JSON_EXTRACT(token_json, '$[*].tk'),
CONCAT('$[', i, ']')) AS val,
t.*
FROM my_table t
JOIN (SELECT #i := #i + 1 AS i
FROM information_schema.tables i
JOIN (SELECT #i := -1) AS iter
WHERE #i < (SELECT MAX(JSON_LENGTH(token_json))
FROM my_table) - 1) AS t) AS tt
WHERE val > 5)
Demo
Update 2 : If you converted values into this format(as mentioned within the last comment) ;
+----+------------+---------------------------------------------+
| id | index_date | token_json |
+----+------------+---------------------------------------------+
| 0 | 20200902_0 | {"tk": [1,2], "amt": [20,49]} |
| 1 | 20200902_1 | {"tk": [4,3,2], "amt": [10,39,19]} |
| 2 | 20200902_2 | {"tk": [7,4], "amt": [12,29]} |
| 3 | 20200902_3 | {"tk": [8,7,6], "amt": [13,19,19]} |
| 4 | 20200902_4 | {"tk": [9,6], "amt": [25,59]} |
+----+------------+---------------------------------------------+
then you can use( for 5.7 ) :
SELECT id, token_json
FROM my_table
WHERE id IN
(SELECT id
FROM (SELECT JSON_EXTRACT(JSON_EXTRACT(token_json, '$.tk'),
CONCAT('$[', i, ']')) AS elm,
t.*
FROM my_table t
JOIN (SELECT #i := #i + 1 AS i
FROM information_schema.tables i
JOIN (SELECT #i := -1) AS iter
WHERE #i < (SELECT MAX(JSON_LENGTH(token_json))
FROM my_table) - 1) AS t) AS tt
WHERE elm > 5)
you can use( for 8.0 ) :
WITH t(arr, id, token_json) AS
(
SELECT JSON_EXTRACT(token_json, '$.tk') AS arr, id, token_json FROM my_table t
)
SELECT id, token_json
FROM t
WHERE EXISTS
( SELECT 1
FROM t AS t2
JOIN JSON_TABLE( CAST( arr AS JSON ), "$[*]"
COLUMNS(
val INT PATH "$"
)
) js
WHERE t2.token_json = t.token_json
AND val > 5 )
Demo

Convert fields with empty or null values to JSON data on MYSQL

I have a table with several fields that I want to compile in just one json field. The problem is there several null and emtpy string values in those columns and y don't want to put them in the json, just store the relevant values. I want to do this with a minimum of queries as possible.
Here is my current approach:
select
json_remove(
json_remove(json, json_search(json, 'one', 'null')),
json_search(json, 'all', '')
) result
from (
select
json_object(
'tag_1', coalesce(tag_1, 'null'),
'tag_2', coalesce(tag_2, 'null'),
'tag_3', coalesce(tag_3, 'null')
) json
from leads
) l2;
But the problem is the json_search output is incompatible with the json_remove input. ΒΏAny ideas?
Here's some example data:
-------------------------
| tag_1 | tag_2 | tag_3 |
-------------------------
| x | | null |
| | y | z |
-------------------------
And what I spect as a result:
--------------------------------
| result |
--------------------------------
| {'tag_1': 'x'} |
| {'tag_2': 'y', 'tag_3': 'z'} |
--------------------------------
Thanks.
I get the solution...
If somebody is interested in the solution....
With this data:
CREATE TABLE t (
id INT(11) unsigned not null auto_increment,
`tag_1` VARCHAR(255),
`tag_2` VARCHAR(255),
`tag_3` VARCHAR(255),
primary key (id)
);
INSERT INTO t
(`tag_1`, `tag_2`, `tag_3`)
VALUES
('x', '', null),
('x','x', null),
('x', '', 'x'),
('x','x','x'),
(null, null, 'x')
;
Here is the query:
select id, json_objectagg(field, val) as JSON from (
select id, 'tag_1' field, tag_1 val from t union
select id, 'tag_2' field, tag_2 val from t union
select id, 'tag_3' field, tag_3 val from t
) sub where val is not null and val != ''
group by sub.id;
The subquery pivots the data to use JSON_OBJECTAGG
id field val
1 tag_1 x
2 tag_1 x
3 tag_1 x
4 tag_1 x
5 tag_1 null
1 tag_2 ''
2 tag_2 x
...
And then the grouping with json_objectagg does the trick!
| id | JSON |
| --- | ------------------------------------------ |
| 1 | {"tag_1": "x"} |
| 2 | {"tag_1": "x", "tag_2": "x"} |
| 3 | {"tag_1": "x", "tag_3": "x"} |
| 4 | {"tag_1": "x", "tag_2": "x", "tag_3": "x"} |
| 5 | {"tag_3": "x"} |
Here is the DB Fiddle
Thanks a lot to #Raimond Nijland for his comment, it put me on track! :)

How to put JSON into a column data if sub-query returns more than 1 row in MySQL

I want to select portfolio of user with the same query that i used for select users.
Here the example that i want.
User Table
----------------------------------------------------------------------
UID NAME USERNAME EMAIL PASSWORD STATUS
----------------------------------------------------------------------
1 Manoj manoj a#a.com ******** 1
2 Test U testing b#a.com ******** 1
3 Company user c#a.com ******** 1
4 Agency company d#a.com ******** 1
User table in my database
Portfolio Table
-----------------------------------
PID UID TITLE STATUS
-----------------------------------
1 1 title 1 1
2 1 title 2 1
3 1 title 3 1
4 2 title 1 1
Portfolio Table in my database
Result I want
----------------------------------
UID USERNAME PORTFOLIO
----------------------------------
1 manoj JSON OBJECT OF PID (1,2,3)
2 testing JOSN OBJECT OF PID (4)
3 user NULL
4 company NULL
Currently I'm trying to use
SELECT u.uid,u.username,(SELECT * FROM portfolio p WHERE u.UID=p.UID) as portfolio FROM users u
Is there any solution for this problem?
A bit convoluted, but you could create JSON objects for each row, concatenate them using GROUP_CONCAT and cast the result (wrapped in [] to make it an array) to JSON;
SELECT u.uid, u.username,
CASE WHEN p.uid IS NULL
THEN NULL
ELSE CAST(CONCAT('[',
GROUP_CONCAT(JSON_OBJECT('pid', p.pid,
'title', p.title,
'status', p.status)),
']') AS JSON) END portfolios
FROM user u
LEFT JOIN portfolio p
ON u.uid=p.uid
WHERE p.status = 1
GROUP BY u.uid, u.username;
...which gives...
+------+----------+-----------------------------------------------------------------------------------------------------------------------------------------+
| 1 | manoj | [{"pid": 1, "title": "title 1", "status": 1}, {"pid": 2, "title": "title 2", "status": 1}, {"pid": 3, "title": "title 3", "status": 1}] |
| 2 | testing | [{"pid": 4, "title": "title 1", "status": 1}] |
| 3 | user | NULL |
| 4 | company | NULL |
+------+----------+-----------------------------------------------------------------------------------------------------------------------------------------+
If you're using an older MySQL without JSON support, you could build it as a string;
SELECT u.uid, u.username,
CASE WHEN p.uid IS NULL
THEN NULL
ELSE CONCAT('[',
GROUP_CONCAT(CONCAT('{ "pid":',p.pid,',"title":"', REPLACE(p.title, '"', '\\"'),
'","status":',p.status, ' }')), ']') END portfolios
FROM user u
LEFT JOIN portfolio p
ON u.uid=p.uid AND p.status=1
GROUP BY u.uid, u.username;
...which will give you...
+------+----------+------------------------------------------------------------------------------------------------------------------------------+
| uid | username | portfolios |
+------+----------+------------------------------------------------------------------------------------------------------------------------------+
| 1 | manoj | [{ "pid":2,"title":"title 2","status":1 },{ "pid":3,"title":"title 3","status":1 },{ "pid":1,"title":"title 1","status":1 }] |
| 2 | testing | [{ "pid":4,"title":"title 1","status":1 }] |
| 3 | user | NULL |
| 4 | company | NULL |
+------+----------+------------------------------------------------------------------------------------------------------------------------------+
MySQL 8: With JSON_OBJECTAGG and JSON_OBJECT functions
SELECT
u.user_id,
u.username,
CASE WHEN p.user_id IS NULL
THEN NULL
ELSE
JSON_OBJECTAGG(
IF(p.portfolio_id IS NULL, '', p.portfolio_id ),
JSON_OBJECT('title', p.title, 'status', p.status)) END portfolios
FROM user AS u
LEFT JOIN portfolio AS p ON u.user_id = p.user_id AND p.status = 1
GROUP BY u.user_id;
Result:
+---------+----------+--------------------------------------------------------------------------------------------------------------------------+
| user_id | username | portfolios |
+---------+----------+--------------------------------------------------------------------------------------------------------------------------+
| 1 | manoj | {"1": {"title": "title 1", "status": 1}, "2": {"title": "title 2", "status": 1}, "3": {"title": "title 3", "status": 1}} |
| 2 | testing | {"4": {"title": "title 1", "status": 1}} |
| 3 | user | NULL |
| 4 | company | NULL |
+---------+----------+--------------------------------------------------------------------------------------------------------------------------+
as of 5.7 use
JSON_OBJECT('key1', 1, 'key2', 'abc');
Its way less complicated than multiple concat statements and has the same effect.
Use group_concat
select UID,USERNAME,concat('JSON OBJECT OF PID (',group_concat(PID),')') PORTFOLIO
from User u join Portfolio p
on u.UID=p.UID group by UID
or try this :-
SELECT u.UID,NAME,concat('JSON OBJECT OF PID (',group_concat(pid),')')
FROM `User` u LEFT join Portfolio p
on u.UID=p.UID group by u.UID

SQL JOIN : Prefix fields with table name

I have the following tables
CREATE TABLE `constraints` (
`id` int(11),
`name` varchar(64),
`type` varchar(64)
);
CREATE TABLE `groups` (
`id` int(11),
`name` varchar(64)
);
CREATE TABLE `constraints_to_group` (
`groupid` int(11),
`constraintid` int(11)
);
With the following data :
INSERT INTO `groups` (`id`, `name`) VALUES
(1, 'group1'),
(2, 'group2');
INSERT INTO `constraints` (`id`, `name`, `type`) VALUES
(1, 'cons1', 'eq'),
(2, 'cons2', 'inf');
INSERT INTO `constraints_to_group` (`groupid`, `constraintid`) VALUES
(1, 1),
(1, 2),
(2, 2);
I want to get all constraints for all groups, so I do the following :
SELECT groups.*, t.* FROM groups
LEFT JOIN
(SELECT * FROM constraints
LEFT JOIN constraints_to_group
ON constraints.id=constraints_to_group.constraintid) as t
ON t.groupid=groups.id
And get the following result :
id| name | id | name type groupid constraintid
-----------------------------------------------------
1 | group1 | 1 | cons1 | eq | 1 | 1
1 | group1 | 2 | cons2 | inf | 1 | 2
2 | group2 | 2 | cons2 | inf | 2 | 2
What I'd like to get :
group_id | group_name | cons_id | cons_name | cons_type | groupid | constraintid
-------------------------------------------------------------------------------------
1 | group1 | 1 | cons1 | eq | 1 | 1
1 | group1 | 2 | cons2 | inf | 1 | 2
2 | group2 | 2 | cons2 | inf | 2 | 2
This is an example, in my real case my tables have much more columns so using the SELECT groups.name as group_name, ... would lead to queries very hard to maintains.
Try this way
SELECT groups.id as group_id, groups.name as group_name ,
t.id as cons_id, t.name as cons_name, t.type as cons_type,
a.groupid , a.constraintid
FROM constraints_to_group as a
JOIN groups on groups.id=a.groupid
JOIN constraints as t on t.id=a.constraintid
The only difference I see are the names of the columns? Use for that mather an AS-statement.
SELECT
groups.id AS group_id,
groups.name AS group_name,
t.id AS cons_id,
t.name AS cons_name,
t.groupid, t.constraintid
FROM groups
LEFT JOIN
(SELECT * FROM constraints
LEFT JOIN constraints_to_group
ON constraints.id=constraints_to_group.constraintid) as t
ON t.groupid=groups.id
Besides, a better join-construction is:
SELECT G.id AS group_id,
G.name AS group_name,
CG.id AS cons_id,
CG.name AS cons_name,
C.groupid, C.constraintid
FROM constraints_to_group CG
LEFT JOIN constraints C
ON CG.constraintid = C.id
LEFT JOIN groups G
ON CG.groupid = G.id;
Possible duplicate of this issue

Building a mysql query

I have such a base structure (short version)
Table Product:
product_id, name, status
1, Product 1, 1
2, Product 2, 1
3, Product 3, 1
4, Product 4, 1
99, Product 99, 1
Table Box:
box_id, name
1, Box 1
Table Product_To_Box:
box_id, product_id
1, 1
1, 2
1, 3
1, 4
1, 99
Table Url: (id = product_id from the table 'product')
url_id, id, url, language_id
1, 1, wp.pl, 1
2, 1, wp.pl, 2
3, 2, google.pl, 1
Table Language:
language_id, name
1, English
2, Polish
There is no problem when I need to download all the products from the table 'product', which is not present in table 'url' and are assigned a 'box_id'.
I do this query
SELECT p.product_id AS id FROM product p LEFT JOIN product_to_box p2b ON (p.product_id = p2b.product_id) WHERE NOT EXISTS (SELECT u.id FROM url u WHERE p.product_id = u.id) AND p2b.box_id = '1'
Return 3 products:
Product 3
Product 4
Product 99
However, he needs a query that will return the items as above but additionally also the products that have no connection to each language from the table 'language'
Query should return 4 products:
Product 2 - return the product because the table 'url' no entry for language_id = 2
Product 3
Product 4
Product 99
How would add another language to the table 'language', eg. Vietnam (language_id = 3) and Table 'url' would look like this
1, 1, wp.pl, 1
2, 1, wp.pl, 2
3, 2, google.pl, 1
3, 2, google.pl, 2
3, 3, google.com, 1
3, 3, google.com, 2
3, 4, onet.pl, 1
3, 4, onet.pl, 2
3, 99, interia.pl, 1
3, 99, interia.pl, 2
This query should return all products because the table 'url' there is no reference to language_id = 3
Product 1
Product 2
Product 3
Product 4
Product 99
So I think the problem with you query is the sub query "NOT EXISTS (SELECT u.id FROM url u WHERE p.product_id = u.id)". Currently your SQL query doesn't even look at the language table, so it doesn't matter if there is matching language_ids. However, if you want join all the tables together and show all product_to_box.id = 1 then the following query would work:
SELECT p.product_id AS id, url_id, url, l.name FROM product p LEFT JOIN product_to_box p2b ON (p.product_id = p2b.product_id)
LEFT JOIN url ON (p.product_id = url.id) LEFT JOIN language l ON (url.language_id = l.language_id)
WHERE p2b.box_id = '1';
This outputs the following:
+------+--------+-----------+---------+
| id | url_id | url | name |
+------+--------+-----------+---------+
| 1 | 1 | wp.pl | English |
| 1 | 2 | wp.pl | Polish |
| 2 | 3 | google.pl | English |
| 3 | NULL | NULL | NULL |
| 4 | NULL | NULL | NULL |
| 99 | NULL | NULL | NULL |
+------+--------+-----------+---------+
6 rows in set (0.00 sec)
The below SQL will show all product_id from the table 'product', that is not found in the table 'url' and do not have assigned all language_id from table 'language'. It checks to see if url.id or language_id is null.
SELECT p.product_id AS id, url_id, url, l.name FROM product p LEFT JOIN product_to_box p2b ON (p.product_id = p2b.product_id)
LEFT JOIN url ON (p.product_id = url.id) LEFT JOIN language l ON (url.language_id = l.language_id)
WHERE url.id is null or l.language_id is null;
+------+--------+------+------+
| id | url_id | url | name |
+------+--------+------+------+
| 3 | NULL | NULL | NULL |
| 4 | NULL | NULL | NULL |
| 99 | NULL | NULL | NULL |
+------+--------+------+------+
3 rows in set (0.00 sec)