Destructuring deeply nested json in query - json

Question

I have a database table where one column contains a deeply nested JSON field. Approximately like this:
create temporary table testing (id integer, contents json);
insert into testing values (1, '{
"level1": {
"level2": [
{
"level3": {
"a": {
"value1": 1,
"value2": 2
},
"b": {
"value1": 3,
"value2": 4
}
}
},
{
"level3": {
"d": {
"value1": 5,
"value2": 6
},
"e": {
"value1": 7,
"value2": 8
}
}
}
]
}
}
');
I am trying to get out a result looking like this:
| id | l2_label | data_label | value1 |
+----+----------+------------+--------+
| 1 | level2 | a | 1 |
| 1 | level2 | b | 3 |
| 1 | level2 | d | 5 |
| 1 | level2 | e | 7 |
The top level is always called "level1", but there can be more than one key inside there and "level2" is not a fixed string. Each of these keys contains an array of objects which may have more than the "level3" key, but I'm only looking for "level3". Inside that, "a", "b", "c" could any string. Then I'm looking for one row for every "value1" value.
I've gotten up to the following query:
select id, key as l2_label, json_array_elements(value) from testing, json_each(contents -> 'level1');
which returns
id | l2_label | json_array_elements
----+----------+--------------------------------------
1 | level2 | { +
| | "level3": { +
| | "a": { +
| | "value1": 1,+
| | "value2": 2 +
| | }, +
| | "b": { +
| | "value1": 3,+
| | "value2": 4 +
| | } +
| | } +
| | }
1 | level2 | { +
| | "level3": { +
but I am at a loss at how to unpack the level3 elements now.
My question is firstly how to get to the result I'm looking for, but also advice on how to build a query like this incrementally, since I'm not sure how to operate on that json_array_elements now.

Answer 1

One way we can approach this is by using a Lateral Join, where we compute a given value for an un-nested set of keys.
For example:
WITH getting_lvl3 AS (
SELECT id AS id,
key AS l2_label,
json_array_elements(value) -> 'level3' AS lvl3
FROM testing, json_each(contents -> 'level1')
)
SELECT id,
l2_label,
label,
lvl3 -> label -> 'value1' -- Getting value1 for each key
FROM getting_lvl3
-- Executing this code for every key in level 3
LEFT JOIN LATERAL json_object_keys(lvl3) label ON TRUE
;
The result should look something like:
|id |l2_label|label|value|
|---|--------|-----|-----|
|1 |level2 |a |1 |
|1 |level2 |b |3 |
|1 |level2 |c |5 |
|1 |level2 |d |7 |

Answer 2

If you want to see this incrementally, then try the commented queries under the CTEs to get a picture of what happens with each step:
with testing as (
select 1 as id, '{
"level1": {
"level2": [
{
"level3": {
"a": {
"value1": 1,
"value2": 2
},
"b": {
"value1": 3,
"value2": 4
}
}
},
{
"level3": {
"d": {
"value1": 5,
"value2": 6
},
"e": {
"value1": 7,
"value2": 8
}
}
}
]
}
}
'::json as contents
), keys_lvl_2 as (
select id,
json_object_keys(contents->'level1') as l2_label,
contents->'level1' as contents
from testing
), array_lvl_2 as (
select id, l2_label,
json_array_elements(contents->l2_label) as contents
from keys_lvl_2
), keys_lvl_3 as (
select id, l2_label,
json_object_keys(contents->'level3') as data_label,
contents->'level3' as contents
from array_lvl_2
)
-- select * from keys_lvl_2;
-- select * from array_lvl_2;
-- select * from keys_lvl_3;
select id, l2_label, data_label,
contents->data_label->>'value1' as value1
from keys_lvl_3;