Select hierarchical JSON as table in PostgreSQL - json

I'm trying to select this hierarchical JSON as a table in PostgreSQL
The JSON script:
'{"NODES":[{"DESC_D":"fam","SEQ":"1","ID":"2304500","NODES":[{"DESC_D":"test 1","SEQ":"2.1","ID":"5214","NODES":[{"DESC_D":"test 1.1","SEQ":"3.1","ID":"999"}]},{"DESC_D":"test 2","SEQ":"2.2","ID":"74542"}]}]}'
The output I'm trying to get (click this link)
This is a solution a friend wrote.
It does the required but in a complex way, is there a simpler solution?
WITH RECURSIVE CTE(SEQ, DESC_D, ID, PARENT_ID, NODES) AS (
SELECT json_extract_path_text(e.element, 'SEQ') SEQ,
json_extract_path_text(e.element, 'DESC_D') DESC_D,
json_extract_path_text(e.element, 'ID') ID,
NULL PARENT_ID,
json_extract_path(e.element, 'NODES') NODES
FROM json_each('{"NODES":[{"DESC_D":"fam","SEQ":"1","ID":"2304500","NODES":[{"DESC_D":"test 1","SEQ":"2.1","ID":"5214","NODES":[{"DESC_D":"test 1.1","SEQ":"3.1","ID":"999"}]},{"DESC_D":"test 2","SEQ":"2.2","ID":"74542"}]}]}'::JSON) a(KEY, val)
CROSS JOIN LATERAL json_array_elements(a.val) e(element)
WHERE json_typeof(a.val) = 'array'
UNION ALL
SELECT json_extract_path_text(e.element, 'SEQ') SEQ,
json_extract_path_text(e.element, 'DESC_D') DESC_D,
json_extract_path_text(e.element, 'ID') ID,
r.ID PARENT_ID,
json_extract_path(e.element, 'NODES') NODES
FROM CTE r
CROSS JOIN LATERAL json_array_elements(r.NODES) e(element)
)
SELECT DISTINCT ON (ID) *
FROM CTE;

you can use the jsonpath type and functions and language :
SELECT DISTINCT
child->>'SEQ' AS seq
, child->>'DESC_D' AS desc_d
, child->>'ID' AS id
, parent->>'ID' AS parent_id
, child->>'NODES' AS nodes
FROM jsonb_path_query('{"NODES":[{"DESC_D":"fam","SEQ":"1","ID":"2304500","NODES":[{"DESC_D":"test 1","SEQ":"2.1","ID":"5214","NODES":[{"DESC_D":"test 1.1","SEQ":"3.1","ID":"999"}]},{"DESC_D":"test 2","SEQ":"2.2","ID":"74542"}]}]}'
, '$.** ? (#.NODES.type() == "array")') AS parent
CROSS JOIN LATERAL jsonb_array_elements(parent->'NODES') AS child
ORDER BY seq
Result :
seq
desc_d
id
parent_id
nodes
1
fam
2304500
null
[{"ID": "5214", "SEQ": "2.1", "NODES": [{"ID": "999", "SEQ": "3.1", "DESC_D": "test 1.1"}], "DESC_D": "test 1"}, {"ID": "74542", "SEQ": "2.2", "DESC_D": "test 2"}]
2.1
test 1
5214
2304500
[{"ID": "999", "SEQ": "3.1", "DESC_D": "test 1.1"}]
2.2
test 2
74542
2304500
null
3.1
test 1.1
999
5214
null
see dbfiddle

Your friend's answer is correct (as is Edouard's). A couple of things could perhaps make the recursive query easier to read. You can use the ->> operator, which basically executes the json_extract_path_text function. You can also omit the CROSS JOIN LATERAL because any set-returning function implies that join type when following the FROM with a comma. In your case, IDs are unique, so DISTINCT ON (ID) isn't needed, but if your JSON could have duplicate IDs, you'll need to put it back in.
WITH RECURSIVE CTE(SEQ, DESC_D, ID, PARENT_ID, NODES) AS (
SELECT e.element ->> 'SEQ' SEQ,
e.element ->> 'DESC_D' DESC_D,
e.element ->> 'ID' ID,
NULL PARENT_ID,
e.element -> 'NODES' NODES
FROM json_each('{"NODES":[{"DESC_D":"fam","SEQ":"1","ID":"2304500","NODES":[{"DESC_D":"test 1","SEQ":"2.1","ID":"5214","NODES":[{"DESC_D":"test 1.1","SEQ":"3.1","ID":"999"}]},{"DESC_D":"test 2","SEQ":"2.2","ID":"74542"}]}]}'::JSON) a(KEY, val),
json_array_elements(a.val) e(element)
WHERE json_typeof(a.val) = 'array'
UNION ALL
SELECT e.element ->> 'SEQ' SEQ,
e.element ->> 'DESC_D' DESC_D,
e.element ->> 'ID' ID,
r.ID PARENT_ID,
e.element -> 'NODES' NODES
FROM CTE r,
json_array_elements(r.NODES) e(element)
)
SELECT *
FROM CTE;
To make it easier to understand, just remember that the part about UNION ALL is always your starting point, and the part after is digging down. Just look at each part separately to help make sense of it.

Related

What is the analogous function to DBMS_XMLGEN.getxml(query) for json

I know that I can use json_arrayagg. For instance:
WITH
ta
AS
(SELECT 1 a, 2 b FROM DUAL
UNION ALL
SELECT 11, 22 FROM DUAL)
SELECT JSON_ARRAYagg( json_object(ta.a,ta.b) )
FROM ta;
[{"a":1,"b":2},{"a":11,"b":22}]
But I have to name every column.
SELECT XMLTYPE.createXML (DBMS_XMLGEN.getxml ('select 2 as a from dual')) FROM DUAL;
is more convenient. You don't you give the name of the column.
Is there a similare way to do that.
I've tried that too.
select JSON_ARRAY(DBMS_XMLGEN.getxml ('select 2 as a from dual')) from dual
But as expected
>["<?xml version=\"1.0\"?>\n<ROWSET>\n <ROW>\n <A>2</A>\n </ROW>\n</ROWSET>\n"]
code
From 19c, you can do json_object (*) and this will infer the attribute names from the column names:
with ta as (
select 1 a, 2 b from dual
union all
select 11, 22 from dual
)
select json_arrayagg (
json_object( * )
)
from ta;
JSON_ARRAYAGG(JSON_OBJECT(*))
-------------------------------
[{"A":1,"B":2},{"A":11,"B":22}]

Pre populate Sql data using recursion on each combination

I am trying to fill db pre populated data from sql script where I have two type of constants or enums.
Platform: DG, NK
Department : KK, TG, LO, NP, UI, BG, ED, CC.
Task: To generate a sequential number using procedural loop and for each combination using above value we need to generate key and put in data base with count or sequence value.
Database columns:
id(auto generated), count, category_key, status
Now single row would be one combination which is formed using this pattern,
Department_Platform_SequenceNumber :: example => KK_DG_1,....KK_DG_10000, KK_NK_1,....KK_NK_10000
This is for 10k entries for 10k sequence of each combinations. It follows for other as well.
Approach:
WITH RECURSIVE
number AS ( SELECT 1 number
UNION ALL
SELECT number + 1 FROM cte WHERE number < 10000 ),
platform AS ( SELECT 'DG' platform
UNION ALL
SELECT 'NK' ),
department AS ( SELECT 'KK' department
UNION ALL
SELECT 'TG'
UNION ALL
SELECT 'LO'
UNION ALL
SELECT 'NP'
UNION ALL
SELECT 'UI'
UNION ALL
SELECT 'BG'
UNION ALL
SELECT 'ED'
UNION ALL
SELECT 'CC' )
INSERT INTO counter_key
SELECT null, number, CONCAT_WS('_', department, platform, number), 1
FROM department
CROSS JOIN platform
CROSS JOIN number;
Problem: Getting syntactical error::
Error Code: 1064. You have an error in your SQL syntax; check the
manual that corresponds to your MySQL server version for the right
syntax to use near 'INSERT INTO counter_key SELECT null, number,
CONCAT_WS('_', department, platfor' at line 23
Please help me resolve this since I have been struggling to resolve this.
There are 2 problems in query
cte does not exist in first Recursive Block.
Place the Insert INTO statement before recursive block
So your final query should be like below
insert into counter_key
WITH RECURSIVE
cte AS ( SELECT 1 number
UNION ALL
SELECT number + 1 FROM cte WHERE number < 2 ),
platform AS ( SELECT 'DG' platform
UNION ALL
SELECT 'NK' ),
department AS ( SELECT 'KK' department
UNION ALL
SELECT 'TG'
UNION ALL
SELECT 'LO'
UNION ALL
SELECT 'NP'
UNION ALL
SELECT 'UI'
UNION ALL
SELECT 'BG'
UNION ALL
SELECT 'ED'
UNION ALL
SELECT 'CC' )
SELECT null, number, CONCAT_WS('_', platform, department), 1
FROM department
CROSS JOIN platform
CROSS JOIN cte
order by 3,2;
DEMO

Get all jsonb values in one column for usage in to_tsvector

I got a table jsonb table t_1, that contains column column_1 with jsonb type and i need to convert in to table with string, that is concatination of jsonb values
Here is a script
CREATE TABLE t_1 (
ID serial NOT NULL PRIMARY KEY,
column_1 jsonb NOT NULL
);
INSERT INTO t_1 (column_1)
VALUES
(
'{ "01": "Lily Bush", "03": "prod2uct"}'
),
(
'{ "41": "Josh William", "12": "product7"}'
),
(
'{ "07": "Mary Clark", "items" : "product2"}'
);
I tried this:
SELECT distinct jsonb_array_elements_text(column_1 -> jsonb_object_keys(column_1)) AS Actor FROM t_1
But this return 'unable to extract elements from scalar'
I tried this:
SELECT tiket, string_agg(column_2, ', ') as list FROM(
SELECT column_1 ->> jsonb_object_keys(column_1) as column_2, id as tiket FROM t_1 ) as foo1
GROUP BY tiket
but here is inner select
How can i get all jsonb values in one column without inner select, something like in first query?
I need it to use in to_tsvector
I need to use it in
setweight(
to_tsvector('simple', column_with_json::text),
'A'
)
But column_with_json::text not what i need, i need to get values, without keys
Any examples?
Use jsonb_each_text() in a lateral join:
SELECT id, string_agg(value, ', ') AS list
FROM t_1
CROSS JOIN jsonb_each_text(column_1)
GROUP BY id
id | list
----+------------------------
1 | Lily Bush, prod2uct
2 | product7, Josh William
3 | Mary Clark, product2
(3 rows)
If you want to use the resulting aggregated value in a condition, use HAVING clause, e.g.:
SELECT id, string_agg(value, ', ') AS list
FROM t_1
CROSS JOIN jsonb_each_text(column_1)
GROUP BY id
HAVING string_agg(value, ', ') LIKE '%Lily%'
or a derived table and WHERE clause in a wrapper query:
SELECT *
FROM (
SELECT id, string_agg(value, ', ') AS list
FROM t_1
CROSS JOIN jsonb_each_text(column_1)
GROUP BY id
) s
WHERE list LIKE '%Lily%'
Both approaches are basically equivalent, in typical cases they generate the same query plan. In both the aggregation is calculated only once.

How can I get the count of multiple table rows storing in different variables using single query?

I have 4 different tables and I want to get the count rows of each tables and get the count(*) on 4 different variables.
The query that I have used is:
SELECT count(*) as count1 FROM `table1`
UNION ALL
SELECT count(*) as count2 FROM `table2`
UNION ALL
SELECT count(*) as count3 FROM `table3`
UNION ALL
SELECT count(*) as count4 FROM `table4`
The query gives me the count of each rows but all the value gets stored on the same variable name.
[
{
"count1": 16171
},
{
"count1": 5928
},
{
"count1": 33318
},
{
"count1": 5877
}
]
Is there any way i could get the output as;
[
{
"count1": 16171
},
{
"count2": 5928
},
{
"count3": 33318
},
{
"count4": 5877
}
]
Thanks in advance.
UNION ALL unions rows. It sounds like you just want to list each of the counts in the select list. Try this:
SELECT
(SELECT COUNT(*) FROM `table1`) AS count1,
(SELECT COUNT(*) FROM `table2`) AS count2,
(SELECT COUNT(*) FROM `table3`) AS count3,
(SELECT COUNT(*) FROM `table4`) AS count4
;
Below is for BigQuery Standard SQL
#standardSQL
SELECT
table_id,
row_count
FROM `yourProject.yourDataset.__TABLES__`
WHERE table_id in ('table1', 'table2', 'table3', 'table4')
This approach allows you to get (for free!!) much more than just rows count - see example below
#standardSQL
SELECT table_id,
DATE(TIMESTAMP_MILLIS(creation_time)) AS creation_date,
DATE(TIMESTAMP_MILLIS(last_modified_time)) AS last_modified_date,
row_count,
size_bytes,
CASE
WHEN type = 1 THEN 'table'
WHEN type = 2 THEN 'view'
WHEN type = 3 THEN 'external'
ELSE '?'
END AS type,
TIMESTAMP_MILLIS(creation_time) AS creation_time,
TIMESTAMP_MILLIS(last_modified_time) AS last_modified_time,
dataset_id,
project_id
FROM `bigquery-public-data.baseball.__TABLES__`

How to get all used values in columns in mysql?

I have a MySQL table with columns and values like this:
Column "A": 1, 5, 3, 2, 3, 1, 4, 5, ...
Column "B": 11, 15, 10, 12, 13, 14, 15, 10, 11, ...
Column "C": .... etc.
There is multiple columns in table with repeating numeric values.
I want to find out unique values in each column. So for column "A" it would return 1,2,3,4,5.
Currently I am using this query for one column:
SELECT concat(A) FROM table GROUP BY A;
But I don't know how to do it for multiple columns
If it's a small enough set of values, you can use the GROUP_CONCAT aggregate function, with the DISTINCT keyword
For example:
SELECT GROUP_CONCAT(DISTINCT a ORDER BY a) AS a_values
, GROUP_CONCAT(DISTINCT b ORDER BY b) AS b_values
, GROUP_CONCAT(DISTINCT c ORDER BY c) AS c_values
FROM mytable
The length of the string returned by GROUP_CONCAT is limited by the max_group_concat_len variable (it's in the reference) and I think the max_allowed_packet also comes into play.
Compare the length of the string returned to max_group_concat_len to see if it's shorter, to know that the return string hasn't been silently truncated.
If you want to combine all of those values together, into a single distinct list, you could do something like this:
SELECT GROUP_CONCAT(DISTINCT val ORDER BY val) AS col_values
FROM ( SELECT a AS val FROM mytable
UNION
SELECT b FROM mytable
UNION
SELECT c FROM mytable
) v
EDIT
I was confused by the use of the CONCAT function in the query, and misread the specification. The queries above return a single row, and returns a result that looks EXACTLY like what OP specified:
1,2,3,4,5,...
If we want to return each value on a separate row, a result that looks like this:
val
---
1
2
3
4
5
Then the query from Tim3880's answer does that, but the outer query isn't really necessary.
I'd want to add an ORDER BY, and actually write the query like this:
(SELECT a AS val FROM mytable)
UNION
(SELECT b AS val FROM mytable)
UNION
(SELECT c AS val FROM mytable)
ORDER BY 1
EDIT
Added SQL Fiddle showing how I interpret the specification (table, columns, exemplar values), and results from SQL statements above... one statement returning distinct values as individual rows (query immediately above), and a statement returning a comma separated list (the first query in my answer.)
SQL Fiddle Example HERE http://sqlfiddle.com/#!9/3d61c/1
If we want to identify which column(s) a value appears in
SELECT v.val
, GROUP_CONCAT(DISTINCT v.col ORDER BY v.col) AS in_cols
, MAX(v.col='a') AS in_col_a
, MAX(v.col='b') AS in_col_b
, MAX(v.col='c') AS in_col_c
FROM (
SELECT a AS val, 'a' AS col FROM mytable
UNION
SELECT b AS val, 'b' AS col FROM mytable
UNION
SELECT c AS val, 'c' AS col FROM mytable
) v
GROUP BY v.val
ORDER BY v.val
If your query works for A, then you can do it for A, B, C using this:
SELECT A FROM
(
SELECT A FROM table
UNION
SELECT B FROM table
UNION
SELECT C FROM table
) e
as long as the three columns have compatible types.