Postgres - Convert adjacency list to nested JSON object - json

I have a table with this data in Postgres and I am having a hard time to convert this in to a JSON object.
node_id parent_node name
------- ----------- ----
1 node1
2 1 node2
3 1 node3
4 2 node4
5 2 node5
6 2 node6
7 3 node7
8 3 node8
How do I convert it like this?
{
name: 'node1'
childs: [
{
name: 'node2',
childs: [
{
name: 'node4',
childs: []
},
{
name: 'node5',
childs: []
},
{
name: 'node6',
childs: []
}
]
},
...
]
}
Any suggestion will help. Thanks

Using WITH RECURSIVE (https://www.postgresql.org/docs/current/static/queries-with.html) and JSON Functions (https://www.postgresql.org/docs/current/static/functions-json.html) I build this solution:
db<>fiddle
The core functionality:
WITH RECURSIVE tree(node_id, ancestor, child, path, json) AS (
SELECT
t1.node_id,
NULL::int,
t2.node_id,
'{children}'::text[] ||
(row_number() OVER (PARTITION BY t1.node_id ORDER BY t2.node_id) - 1)::text,-- C
jsonb_build_object('name', t2.name, 'children', array_to_json(ARRAY[]::int[])) -- B
FROM test t1
LEFT JOIN test t2 ON t1.node_id = t2.parent_node -- A
WHERE t1.parent_node IS NULL
UNION
SELECT
t1.node_id,
t1.parent_node,
t2.node_id,
tree.path || '{children}' || (row_number() OVER (PARTITION BY t1.node_id ORDER BY t2.node_id) - 1)::text,
jsonb_build_object('name', t2.name, 'children', array_to_json(ARRAY[]::int[]))
FROM test t1
LEFT JOIN test t2 ON t1.node_id = t2.parent_node
INNER JOIN tree ON (t1.node_id = tree.child)
WHERE t1.parent_node = tree.node_id -- D
)
SELECT -- E
child as node_id, path, json
FROM tree
WHERE child IS NOT NULL ORDER BY path
Every WITH RECURSIVE contains a start SELECT and a recursion part (the second SELECT) combined by a UNION.
A: Joining the table agains itself for finding the children of a node_id.
B: Building the json object for the child which can be inserted into its parent
C: Building the path where the child object has to be inserted (from root). The window function row_number() (https://www.postgresql.org/docs/current/static/tutorial-window.html) generates the index of the child within the children array of the parent.
D: The recursion part works as the initial part with one difference: It's not searching for the root element but for the element which has the parent node of the last recursion.
E: Executing the recursion and filtering all elements without any children gives this result:
node_id path json
2 children,0 {"name": "node2", "children": []}
4 children,0,children,0 {"name": "node4", "children": []}
5 children,0,children,1 {"name": "node5", "children": []}
6 children,0,children,2 {"name": "node6", "children": []}
3 children,1 {"name": "node3", "children": []}
7 children,1,children,0 {"name": "node7", "children": []}
8 children,1,children,1 {"name": "node8", "children": []}
Though I found no way to add all children elements in the recursion (the origin json is no global variable; so it always knows the changes of the direct ancestors, not their siblings), I had to iterate the rows in a seconds step.
That's why I build the function. In there I can do the iteration for a global variable. With the function jsonb_insert I am inserting all calculated elements into a root json object - using the calculated path.
CREATE OR REPLACE FUNCTION json_tree() RETURNS jsonb AS $$
DECLARE
_json_output jsonb;
_temprow record;
BEGIN
SELECT
jsonb_build_object('name', name, 'children', array_to_json(ARRAY[]::int[]))
INTO _json_output
FROM test
WHERE parent_node IS NULL;
FOR _temprow IN
/* Query above */
LOOP
SELECT jsonb_insert(_json_output, _temprow.path, _temprow.json) INTO _json_output;
END LOOP;
RETURN _json_output;
END;
$$ LANGUAGE plpgsql;
Last step is calling the function and make the JSON more readable (jsonb_pretty())
{
"name": "node1",
"children": [{
"name": "node2",
"children": [{
"name": "node4",
"children": []
},
{
"name": "node5",
"children": []
},
{
"name": "node6",
"children": []
}]
},
{
"name": "node3",
"children": [{
"name": "node7",
"children": []
},
{
"name": "node8",
"children": []
}]
}]
}
I am sure it is possible to optimize the query but for a sketch it works.

Related

How to map nested array items with N1QL?

I have documents in a bucket called blocks in the following format:
{
"random_field": 1,
"transactions": [{
"id": "CCCCC",
"inputs": [{
"tx_id": "AAAAA",
"index": 0
},{
"tx_id": "BBBBB",
"index": 1
}]
}]
}
{
"transactions": [{
"id": "AAAAA",
"outputs": [{
"field1": "value123",
"field2": "value456"
},{
"field1": "ignore",
"field2": "ignore"
}]
}]
}
{
"transactions": [{
"id": "BBBBB",
"outputs": [{
"field1": "ignored",
"field2": "ignored"
},{
"field1": "value999",
"field2": "value888"
}]
}]
}
and I need to map the inputs from the first document to the corresponding outputs of the second and third documents. The way to do it manually is to, for each input, find a transaction with id equal to the input's tx_id, and then get the item from the outputs array based on the index of the input. To exemplify, this is the object I would like to return in this scenario:
{
"random_field": 1,
"transactions": [{
"id": "CCCCC",
"inputs": [{
"tx_id": "AAAAA",
"index": 0,
"output": {
"field1": "value123",
"field2": "value456"
}
},{
"tx_id": "BBBBB",
"index": 1,
"output": {
"field1": "value999",
"field2": "value888"
}
}]
}]
}
I managed to come up with the following query:
SELECT b.random_field,
b.transactions -- how to map this?
FROM blocks b
UNNEST b.transactions t
UNNEST t.inputs input
JOIN blocks `source` ON (ANY tx IN `source`.transactions SATISFIES tx.`id` = input.tx_id END)
UNNEST `source`.transactions source_tx
UNNEST source_tx.outputs o
WHERE (ANY tx IN b.transactions SATISFIES tx.`id` = 'AAAAA' END) LIMIT 1;
I suppose there should be a way to map b.transactions.inputs by using source_tx.outputs, but I couldn't find how.
I came across this other answer, but I don't really understand how it applies to my scenario. Maybe it does, but I am very new to Couchbase, so I am very much lost: How to map array values in one document to another and display in result
Basically you want inline some other document into current document using condition.
Instead of JOINs+ GROUPS use subquery expressions + correlated subqueries. (b.*, "abc" AS transactions, selects all fields of b and adds transactions (if already exist overwrite else adds)
CREATE INDEX ix1 ON blocks (ALL ARRAY FOR ot.id FOR ot IN transactions END);
SELECT b.*,
(SELECT t.*,
(SELECT i.*,
(SELECT RAW ot
FROM blocks AS o
UNNEST o.transactions AS ot
UNNEST ot.outputs AS oto
WHERE i.tx_id = ot.id AND i.`index` = UNNEST_POS(oto))[0] AS output
FROM t.`inputs` AS i) AS inputs
FROM b.transactions AS t) AS transactions
FROM blocks AS b
WHERE ANY tx IN b.transactions SATISFIES tx.`inputs` IS NOT NULL END ;
OR
SELECT b.*,
(SELECT t.*,
(SELECT i.*,
(SELECT RAW ot.outputs[i.`index`]
FROM blocks AS o
UNNEST o.transactions AS ot
WHERE i.tx_id = ot.id
LIMIT 1)[0] AS output
FROM t.`inputs` AS i) AS inputs
FROM b.transactions AS t) AS transactions
FROM blocks AS b
WHERE ANY tx IN b.transactions SATISFIES tx.`inputs` IS NOT NULL END ;

Recursive parent child JSON Postgresql

I have the following recursive PostgreSQL table.
CREATE TABLE public."data" (
id int4 NULL,
parentid int4 NULL,
menu varchar NULL
);
I am trying to create a JSON object from the table.
[
{
"menu": "home",
"children": [
{
"menu": "home 1",
"children": []
},
{
"menu": "home 2",
"children": [
{
"menu": "home 2.1",
"children": []
}
]
}
]
},
{
"menu": "config",
"children": []
}
]
How could I create a hierarchal object like this?
First you should use the jsonb format instead of the json format in postgres, see the documentation here :
In general, most applications should prefer to store JSON data as
jsonb, unless there are quite specialized needs, such as legacy
assumptions about ordering of object keys..
Then, assuming your table is a set of (id, menu) tuples and parentid is the id of the parent of this tuple, you can try :
CREATE VIEW parent_children (parent, children, root, cond) AS
( SELECT jsonb_build_object('menu', p.menu, 'children', '[]' :: jsonb) :: text AS parent
, jsonb_agg(jsonb_build_object('menu', c.menu, 'children', '[]' :: jsonb)) :: text AS children
, array[c.parentid] AS root
, array[c.parentid] AS cond
FROM public.data AS c
LEFT JOIN public.data AS p
ON p.id = c.parentid
GROUP BY c.parentid
) ;
WITH RECURSIVE list(parent, children, root, cond) AS
( SELECT parent, children, root, cond
FROM parent_children
WHERE root = array[NULL] -- start with the root parents
UNION
SELECT p.parent
, replace(p.children, c.parent, replace(c.parent, '[]', c.children))
, p.root
, p.cond || c.cond
FROM list AS p
INNER JOIN parent_children AS c
ON position(c.parent IN p.children) > 0
AND NOT p.cond #> c.root -- condition to avoid circular path
)
SELECT children :: jsonb
FROM list AS l
ORDER BY array_length(cond, 1) DESC
LIMIT 1 ;

How to deal with not existing values using JSON_EXTRACT?

I have a list ob objects. Each object contains several properties. Now I want to make a SELECT statement that gives me a list of a single property values. The simplified list look like this:
[
[
{
"day": "2021-10-01",
"entries": [
{
"name": "Start of competition",
"startTimeDelta": "08:30:00"
}
]
},
{
"day": "2021-10-02",
"entries": [
{
"name": "Start of competition",
"startTimeDelta": "03:30:00"
}
]
},
{
"day": "2021-10-03",
"entries": [
{
"name": "Start of competition"
}
]
}
]
]
The working SELECT is now
SELECT
JSON_EXTRACT(column, '$.days[*].entries[0].startTimeDelta') AS list
FROM table
The returned result is
[
"08:30:00",
"03:30:00"
]
But what I want to get (and also have expected) is
[
"08:30:00",
"03:30:00",
null
]
What can I do or how can I change the SELECT statement so that I also get NULL values in the list?
SELECT startTimeDelta
FROM test
CROSS JOIN JSON_TABLE(val,
'$[*][*].entries[*]' COLUMNS (startTimeDelta TIME PATH '$.startTimeDelta')) jsontable
https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=491f0f978d200a8a8522e3200509460e
Do you also have a working idea for MySQL< 8? – Lars
What is max amount of objects in the array on the 2nd level? – Akina
Well it's usually less than 10 – Lars
SELECT JSON_EXTRACT(val, CONCAT('$[0][', num, '].entries[0].startTimeDelta')) startTimeDelta
FROM test
-- up to 4 - increase if needed
CROSS JOIN (SELECT 0 num UNION SELECT 1 UNION SELECT 2 UNION SELECT 3) nums
WHERE JSON_EXTRACT(val, CONCAT('$[0][', num, '].entries[0]')) IS NOT NULL;
https://www.db-fiddle.com/f/xnCCSTGQXevcpfPH1GAbUo/0

Expand Postgresql Nested Array Json Field

I have a table (log_table) and in this table there is a nested array json field (activities). With using this activities field, I want to normalize my row.
log_table:
- id:long
- activities:json
- date:timestamp
example activities field:
[
{
"actionType":"NOTIFICATION",
"items":null
},
{
"actionType":"MUTATION",
"items":[
{
"id":387015007,
"name":"epic",
"value":{
"currency":"USD",
"amount":1.76
}
},
{
"id":386521039,
"name":"test",
"value":{
"currency":"USD",
"amount":1.76
}
}
]
}
]
As query, I've tried:
select
*
from
log_table l,
json_array_elements(l.activities) elems,
json_array_elements(elems->'items') obj;
With this query, I got error like below:
ERROR: cannot call json_array_elements on a scalar
Is there any suggestion?
The lack of items should be marked as [null], not null. You can use the case expression to correct this, e.g.:
select elems->>'actionType' as action_type, obj
from log_table
cross join jsonb_array_elements(l.activities::jsonb) elems
cross join jsonb_array_elements(case elems->'items' when 'null' then '[null]' else elems->'items' end) obj
action_type | obj
--------------+---------------------------------------------------------------------------------
NOTIFICATION | null
MUTATION | {"id": 387015007, "name": "epic", "value": {"amount": 1.76, "currency": "USD"}}
MUTATION | {"id": 386521039, "name": "test", "value": {"amount": 1.76, "currency": "USD"}}
(3 rows)

How to flatten json into table

I have just realised on my AWS Aurora postgres cluster having functions with temp_tables are not friendly with read replicas. I need to do a re-write (using CTEs) - anyway.... How do I take a json object with arrays nested and flatten them to a table like so:
{
"data": [
{
"groupName": "TeamA",
"groupCode": "12",
"subGroupCodes": [
"11"
]
},
{
"groupName": "TeamB",
"groupCode": "13",
"subGroupCodes": [
"15", "22"
]
}
]
}
I would like the output table to be:
groupName groupCode subGroupCodes
TeamA 12 11
TeamB 13 15
TeamB 13 22
I know I can get most of the way there with:
SELECT j."groupCode" as int, j."groupName" as pupilgroup_name
FROM json_to_recordset(p_in_filters->'data') j ("groupName" varchar(50), "groupCode" int)
But I just need to get the subGroupCodes as well but unpacking the array and joining to the correct parent groupCodes.
You need to first unnest the array, and then another unnest to get the subgroup codes:
with data (j) as (
values ('{
"data": [
{
"groupName": "TeamA",
"groupCode": "12",
"subGroupCodes": [
"11"
]
},
{
"groupName": "TeamB",
"groupCode": "13",
"subGroupCodes": [
"15", "22"
]
}
]
}'::jsonb)
)
select e ->> 'groupName' as group_name,
e ->> 'groupCode' as code,
sg.*
from data d
cross join lateral jsonb_array_elements(d.j -> 'data') as e(g)
cross join lateral jsonb_array_elements_text(g -> 'subGroupCodes') as sg(subgroup_code)