Struggling to parse some json.
This is the format, where there are no fixed names/keys - everything is dynamic.
{ "{condition-operator}" : { "{condition-key}" : "{condition-value}" }}
An an example of values:
{
"bool":{"aws:viaawsservice":"true"},
"stringequals":{
"ec2:createaction":[
"CreateSecurityGroup",
"CreateVolume",
"CreateSnapshot",
"RunInstances"
]
}
}
I've managed to extract the 'operator' and 'key' values. (See below)
However, my result for 'values' is problematic.
One value is 'true',
the other ["CreateSecurityGroup","CreateVolume","CreateSnapshot","RunInstances"]
Neither of which I seem able to use or cast as an UNNESTable array.
To be honest, getting woefully lost in what's going on !!
I need to be able to unnest these, to get 1 row per value (so 5 values/rows in total)
Any guidance appreciated !
with cte as (
select '{"bool":{"aws:viaawsservice":"true"},"stringequals":{"ec2:createaction":["CreateSecurityGroup","CreateVolume","CreateSnapshot","RunInstances"]}}'
as sample
)
select
,ct.ct as condition_operator
,map_keys(cast(ct.cb as map<varchar,json>))[1] as condition_key
, map_values(cast(ct.cb as map<varchar,json>))[1] as condition_values
from
cte
CROSS JOIN UNNEST(map_keys(cast(json_parse(cte.sample)as map<varchar,json>)),map_values(cast(json_parse(cte.sample)as map<varchar,json>))) ct(ct,cb)
-- CROSS JOIN UNNEST( ## something here ##) as values(v)
condition_ope.. condition_key condition_values
(string(255)) (string(255)) (json)
bool aws:viaawsservice "true"
stringequals ec2:createaction ["CreateSecurityGroup","CreateVolume","CreateSnapshot","RunInstances"]
You can use try, which results in null in case of failure, and attempt to cast data to array of varchar and fallback to either cast to varchar (which will fail in case of json object in value) or just using json_format:
select ct.ct as condition_operator,
ct_key,
ct_value
from cte
CROSS JOIN UNNEST(
map_keys(cast(json_parse(cte.sample) as map < varchar, json >)),
map_values(cast(json_parse(cte.sample) as map < varchar, json >))
) ct(ct, cb)
CROSS JOIN UNNEST(
map_keys(cast(ct.cb as map < varchar, json >)),
map_values(cast(ct.cb as map < varchar, json >))
) ct1(ct_key, ct_value_json)
CROSS JOIN UNNEST(
coalesce(try(cast(ct_value_json as array < varchar >)),array [ json_format(ct_value_json) ]
)
) ct2(ct_value)
Output:
condition_operator
ct_key
ct_value
bool
aws:viaawsservice
true
stringequals
ec2:createaction
CreateSecurityGroup
stringequals
ec2:createaction
CreateVolume
stringequals
ec2:createaction
CreateSnapshot
stringequals
ec2:createaction
RunInstances
Related
I have JSON stored in a table. The JSON is nested and has the following structure
[
{
"name": "abc",
"ques": [
{
"qId": 100
},
{
"qId": 200
}
]
},{
"name": "xyz",
"ques": [
{
"qId": 100
},
{
"qId": 300
}
]
}
]
Update TABLE_NAME
set COLUMN_NAME = jsonb_set(COLUMN_NAME, '{ques,qId}', '101')
WHERE COLUMN_NAME->>'qId'=100
I am trying to update qId value from JSON. If qId is 100, I want to update it to 101.
1st solution, simple but to be used carefully
You convert your json data into text and you use the replace function :
Update TABLE_NAME
set COLUMN_NAME = replace(COLUMN_NAME :: text,'"qId": 100}', '"qId": 101}') :: jsonb
2nd solution more elegant and more complex
jsonb_set cannot make several replacements in the same jsonb data at the same time. To do so, you need to create your own aggregate based on the jsonb_set function :
CREATE OR REPLACE FUNCTION jsonb_set(x jsonb, y jsonb, path text[], new_value jsonb) RETURNS jsonb LANGUAGE sql AS $$
SELECT jsonb_set(COALESCE(x, y), path, new_value) ; $$ ;
CREATE OR REPLACE AGGREGATE jsonb_set_agg(x jsonb, path text[], new_value jsonb)
( stype = jsonb, sfunc = jsonb_set);
Then you get your result with the following query :
UPDATE TABLE_NAME
SET COLUMN_NAME =
( SELECT jsonb_set_agg(COLUMN_NAME :: jsonb, array[(a.id - 1) :: text, 'ques', (b.id - 1) :: text], jsonb_build_object('qId', 101))
FROM jsonb_path_query(COLUMN_NAME :: jsonb, '$[*]') WITH ORDINALITY AS a(content, id)
CROSS JOIN LATERAL jsonb_path_query(a.content->'ques', '$[*]') WITH ORDINALITY AS b(content, id)
WHERE (b.content)->'qId' = to_jsonb(100)
)
Note that this query is not universal, and it must breakdown the jsonb data according to its structure.
Note that jsonb_array_elements can be used in place of jsonb_path_query, but you will get an error with jsonb_array_elements when the jsonb data is not an array, whereas you won't get any error with jsonb_path_query in lax mode which is the default mode.
Full test results in dbfiddle
You must specify the whole path to the value.
In this case your json is an array so you need to address which element of this array your are trying to modify.
A direct approach (over your example) would be:
jsonb_set(
jsonb_set(
COLUMN_NAME
, '{0,ques,qId}'
, '101'
)
, '{1,ques,qId}'
, '101'
)
Of course, if you want to modify every element of different arrays of different lengths you would need to elaborate this approach disassembling the array to modify every contained element.
Using PostgreSQL 13.4 I have a table with a JSON column in a structure like the following sample:
{
"username": "jsmith",
"location": "United States",
"posts": [
{
"id":"1",
"title":"Welcome",
"newKey":true <----------- insert new key/value pair here
},
{
"id":"4",
"title":"What started it all",
"newKey":true <----------- insert new key/value pair here
}
]
}
For changing keys on the first level, I used a simple query like this
UPDATE
sample_table_json
SET
json = json::jsonb || '{"active": true}';
But this doesn't work for nested objects and objects in an array like in the sample.
How would I insert a key/value pair into a JSON column with nested objects in an array?
You have to use the jsonb_set function while specifying the right path see the manual.
For a single json update :
UPDATE sample_table_json
SET json = jsonb_set( json::jsonb
, '{post,0,active}'
, 'true'
, true
)
For a (very) limited set of json updates :
UPDATE sample_table_json
SET json = jsonb_set(jsonb_set( json::jsonb
, '{post,0,active}'
, 'true'
, true
)
, '{post,1,active}'
, 'true'
, true
)
For a larger set of json updates of the same json data, you can create the "aggregate version" of the jsonb_set function :
CREATE OR REPLACE FUNCTION jsonb_set(x jsonb, y jsonb, p text[], e jsonb, b boolean)
RETURNS jsonb LANGUAGE sql AS $$
SELECT jsonb_set(COALESCE(x,y), p, e, b) ; $$ ;
CREATE OR REPLACE AGGREGATE jsonb_set_agg(x jsonb, p text[], e jsonb, b boolean)
( STYPE = jsonb, SFUNC = jsonb_set) ;
and then use the new aggregate function jsonb_set_agg while iterating on a query result where the path and val fields could be calculated :
SELECT jsonb_set_agg('{"username": "jsmith","location": "United States","posts": [{"id":"1","title":"Welcome"},{"id":"4","title":"What started it all"}]}' :: jsonb
, l.path :: text[]
, to_jsonb(l.val)
, true)
FROM (VALUES ('{posts,0,active}', 'true'), ('{posts,1,active}', 'true')) AS l(path, val) -- this list could be the result of a subquery
This query could finally be used in order to update some data :
WITH list AS
(
SELECT id
, jsonb_set_agg(json :: jsonb
, l.path :: text[]
, to_jsonb(l.val)
, true) AS res
FROM sample_table_json
CROSS JOIN (VALUES ('{posts,0,active}', 'true'), ('{posts,1,active}', 'true')) AS l(path, val)
GROUP BY id
)
UPDATE sample_table_json AS t
SET json = l.res
FROM list AS l
WHERE t.id = l.id
see the test result in dbfiddle
It became a bit complicated. Loop through the array, add the new key/value pair to each array element and re-aggregate the array, then rebuild the whole object.
with t(j) as
(
values ('{
"username": "jsmith",
"location": "United States",
"posts": [
{
"id":"1", "title":"Welcome", "newKey":true
},
{
"id":"4", "title":"What started it all", "newKey":true
}]
}'::jsonb)
)
select j ||
jsonb_build_object
(
'posts',
(select jsonb_agg(je||'{"active":true}') from jsonb_array_elements(j->'posts') je)
)
from t;
I have a JSON like this (see the test setup below)
{
"dt" :
[
{
"values" :
[
{
"key" : "a"
},
{
"key" : "b"
}
]
}
]
}
and it is straightforeward to parse the inner array as it has keys as follows
SELECT tab.id,
jt.*
FROM parse_json_array tab,
json_table(data, '$.dt[*]'
COLUMNS (NESTED PATH '$.values[*]' COLUMNS(
key PATH '$.key' )
)) AS "JT"
where tab.id = 1;
which returns
ID, KEY
--------
1 a
1 b
But if the inner array has no keys, how could I addapt the path in NESTED PATH?
{
"dt" :
[
{
"values" :
[
"a",
"b"
]
}
]
}
All my try such as key PATH '$.*' or key PATH '*' return null or syntax error.
Note I do not need a solution, that parse both variants, but it would be of course a bonus;)
I'm on XE 18.4.0.0.0
Test data
create table parse_json_array
(id int primary key,
data CLOB constraint c1 check(data is JSON)
);
insert into parse_json_array (id, data) values (1, '{ "dt" : [ {"values" : [{"key" : "a"} , {"key" : "b" } ]} ] }');
insert into parse_json_array (id, data) values (2, '{ "dt" : [ {"values" : [ "a" , "b" ]}] }');
This will give you the id and the values within the nested array, when it's just an array of scalars rather than objects.
SELECT tab.id,
jt.*
FROM parse_json_array tab,
json_table(data, '$.dt[*].values[*]'
COLUMNS key PATH '$' )
AS "JT"
where tab.id = 2;
Storing JSON in both formats, and even more so, asking for a solution that works for both, doesn't make a lot of sense; the JSON structure is different. It's like asking for a SQL SELECT query that works for two different tables with different column sets.
If you need a solution with nested path (perhaps because you must pick out additional bits of data, which you did not share with us), you can do something like this (which is what Padders suggested in a comment):
SELECT tab.id,
jt.*
FROM parse_json_array tab,
json_table(data, '$.dt[*]' columns(
nested path '$.values[*]'
COLUMNS (key PATH '$' )) )
AS "JT"
where tab.id = 2;
EDIT:
To get values both from object members and from scalar members of the nested array, you can do something like this. Use nvl(k_token, token) if you just need the value and don't need to know if it comes from an array of objects or an array of scalars. Note that this solution will work even if you have objects and scalars mixed together in the same JSON (in the same nested array).
select p.id, j.k_token, j.token
from parse_json_array p,
json_table(data, '$.dt[*].values[*]'
columns( k_token path '$.key',
token path '$'
)
) j
;
So here is my JSON column in my Postgres DB:
{
"objekt_art": {
"86": {
"code": "86",
"bezeichnung_de": "Kino",
"bezeichnung_fr": "Cinéma",
"bezeichnung_it": "Cinema",
"bezeichnung_en": null,
"kurz_bezeichnung_de": "Kino",
"relevant_fuer_berechnung_steuerquote": true
},
"27": {
"code": "27",
"bezeichnung_de": "Kiosk",
"bezeichnung_fr": "Kiosque",
"bezeichnung_it": "Chiosco",
"bezeichnung_en": null,
"kurz_bezeichnung_de": "Kiosk",
"relevant_fuer_berechnung_steuerquote": true
}
}
}
I need to be able to query the bezechnung_de for example where code = 86.
The number of code i can pass from another table.
How can i for example make a query with two columns. One with the number and the second with bezeichnung_de.
Like this:
Code Bez
86 Kino
Sample data structure and sample table for join data: dbfiddle
select
je.value -> 'code' as "Code",
je.value -> 'bezeichnung_de' as "Bez"
from
test t
cross join jsonb_each((data::jsonb ->> 'objekt_art')::jsonb) je
-- In table test_join I insert value 86 for join record
inner join test_join tj on je.key::int = tj.json_id
As you know the code, this is fairly easy:
select t.the_column -> 'objekt_art' -> '86' ->> 'code' as code,
t.the_column -> 'objekt_art' -> '86' ->> 'bezeichnung_de' as bez
from the_table t
where ...
The value '86' can be a parameter. The first expression to select the code isn't really needed though, as you could replace it with the constant value (=parameter) directly.
If the "outer" JSON key isn't the same value as the code value, you could use something like this:
select o.value ->> 'code' as code,
o.value ->> 'bezeichnung_de' as bez
from the_table t
cross join jsonb_each(t.the_column -> 'objekt_art') o(key, value)
where o.key = '86'
and ... other conditions ...
If you are using Postgres 13 or later, this can also be written as a JSON path expression:
select a.item ->> 'code' as code,
a.item ->> 'bezeichnung_de' as bez
from (
select jsonb_path_query_first(t.the_column, '$.objekt_art.* ? (#.code == "86")') as item
from the_table t
where ....
) a
All examples assume that the column is defined with the data jsonb which it should be. If it's not you need to cast it: the_column::jsonb
I've got MySQL table with JSON field, where I store data in such a format.
{
"fields": {
"1": {
"s": "y"
},
"2": {
"s": "n"
}
}
}
I need to obtain the keys in fields, e.g. 1 or 2 given the value of s.
Example query:
create table mytable ( mycol json );
insert into mytable set mycol = '{"fields": {"1": {"s": "y"},"2": {"s": "n"}}}';
select j.* from mytable, JSON_TABLE(mycol,
'$.fields.*' COLUMNS (
json_key VARCHAR(10) PATH '$',
s VARCHAR(10) PATH '$.s'
)
) AS j where j.s = 'y';
gives:
# json_key, s
null, y
I would expect to get
# json_key, s
1, y
Is it possible to get that data somehow?
I don't need the results in row / table format. I would be happy to get the comma separated list of IDs (json_keys) meeting my criterium.
EDIT:
I was also thinking about getting the paths using JSON_SEARCH and passing that to JSON_EXTRACT, this was achieved here: Combining JSON_SEARCH and JSON_EXTRACT get me: "Invalid JSON path expression."
Unfortunately the difference is that I would need to use JSON_SEARCH in all mode, as I need all results. In such a mode JSON_SEARCH returns list of paths, where as JSON_EXTRACT accepts list of arguments.
Try FOR ORDINALITY (see 12.17.6 JSON Table Functions), this type enumerates rows in the COLUMNS clause:
SELECT
JSON_UNQUOTE(
JSON_EXTRACT(
JSON_KEYS(`mycol` ->> '$.fields'),
CONCAT('$[', `j`.`row` - 1, ']')
)
) `json_key`,
`j`.`s`
FROM
`mytable`,
JSON_TABLE(
`mycol`,
'$.fields.*' COLUMNS (
`row` FOR ORDINALITY,
`s` VARCHAR(10) PATH '$.s'
)
) `j`
WHERE
`j`.`s` = 'y';
See dbfiddle.