What would be the right way of getting Ajax, i.e. the value for the last occurence for key child1Dob1, from a json field that has a data structure that looks like the below,
{
"data": {
"data": {
"data": {
"child1Dob1": "Andy"
},
"child1Dob1": "Bob"
},
"child1Dob1": "Rick"
},
"child1Dob1": "Ajax"
}
The below query was an attempt from a similar question but i am getting a null value, so obviously i am missing something.
SELECT JSON_EXTRACT(`containerValue`,CONCAT("$.data[",JSON_LENGTH(`containerValue` ->> '$.data')-1,"]")) from myTable where containerKey = 'theContainer';
For CREATE TABLE test (data JSON):
WITH RECURSIVE
cte AS (
SELECT data, data -> '$.data' subdata
FROM test
UNION ALL
SELECT subdata, subdata -> '$.data'
FROM cte
WHERE subdata IS NOT NULL
)
SELECT data ->> '$.child1Dob1'
FROM cte
WHERE subdata IS NULL;
Related
I have the following JSON:
{
"rewards": {
"reward_1": {
"type": "type 1",
"amount": "amount 1"
},
"reward_2": {
"type": "type 2",
"amount": "amount 2"
},
"reward_3": {
"type": "type 3",
"amount": "amount 3"
},
"reward_4": {
"type": "type 4",
"amount": "amount 4"
}
}
}
This JSON is dynamic and I don't necessarily know how many rewards it will get, here it's 4 but it can be 2 or 8 etc.
I want to write a query in Big Query that will parse those values dynamically without knowing how many of them exist, and then split them into column, like this:
Thank you!
Hope these are helpful.
since a JSON data is dynamic, first step is to find a max reward sequence. (I've used a regular expression and max_reward UDF.)
and then, extract each reward from a json rewards field in an iterative way.
lastly, make the result to be a wide form using PIVOT query.
If you want a more generic solution, you need to use BigQuery dynamic SQL to generate PIVOT columns. I've hard-coded them in the query.
('reward_1', 'reward_2', 'reward_3', 'reward_4')
query:
CREATE TEMP TABLE sample AS
SELECT 1 AS id, '{"rewards": { "reward_1": { ... ' AS json -- put your json here
UNION ALL
SELECT 2 AS id, '{"rewards": { "reward_1": { ... ' AS json -- put your another json here
;
CREATE TEMP FUNCTION extract_reward(json STRING, seq INT64)
RETURNS STRUCT<type STRING, amount STRING>
LANGUAGE js AS """
return JSON.parse(json)['reward_' + seq];
""";
CREATE TEMP FUNCTION max_reward(arr ARRAY<STRING>) AS ((
SELECT MAX(CAST(v AS INT64)) FROM UNNEST(arr) v
));
SELECT * FROM (
SELECT id,
'reward_' || seq AS reward,
extract_reward(FORMAT('%t', JSON_QUERY(json, '$.rewards')), seq) AS value
FROM sample, UNNEST(GENERATE_ARRAY(1, max_reward(REGEXP_EXTRACT_ALL(json, r'"reward_([0-9]+)"')))) seq
) PIVOT (ANY_VALUE(value) FOR reward IN ('reward_1', 'reward_2', 'reward_3', 'reward_4'));
output:
▶ Split a reward STRUCT column into separate columns
SELECT * FROM (
SELECT id,
'reward_' || seq || '_' || IF (offset = 0, 'type', 'amount') AS reward,
value
FROM sample,
UNNEST(GENERATE_ARRAY(1, max_reward(REGEXP_EXTRACT_ALL(json, r'"reward_([0-9]+)"')))) seq,
UNNEST([extract_reward(FORMAT('%t', JSON_QUERY(json, '$.rewards')), seq)]) pair,
UNNEST([pair.type, pair.amount]) value WITH OFFSET
) PIVOT (ANY_VALUE(value) FOR reward IN ('reward_1_type', 'reward_2_type', 'reward_3_type', 'reward_4_type', 'reward_1_amount', 'reward_2_amount', 'reward_3_amount', 'reward_4_amount'));
output:
I am using JSON_MODIFY to build complex JSON. Moving from MySQL I am struggling with the JSON functions provided by SQL Server. The issue I'm having is that SQL Server seems to construct all JSON objects in an array. There is the WITHOUT_ARRAY_WRAPPER mechanism, which seems like it should do what I want, however; there are two undesirable consequences.
It only returns one result depending on how it is used
The result is a single string with escape characters
I have constructed a simple query which illustrates my needs and the issue.
QUERY 1
SELECT JSON_MODIFY(
JSON_QUERY('{"definitions": {"id": "INT", "name": "VARCHAR(23)"}}'),
'append $.data',
(
SELECT * FROM (
SELECT 1 AS id, '123abc' AS "name" UNION
SELECT 2 AS id, '234bcd' AS "name"
) AS "data"
FOR JSON PATH, WITHOUT_ARRAY_WRAPPER
)
) AS "data";
OUTPUT 1
{
"definitions":{
"id":"INT",
"name":"VARCHAR(23)"
},
"data":[
"{\"id\":1,\"name\":\"123abc\"},{\"id\":2,\"name\":\"234bcd\"}"
]
}
QUERY 2
SELECT JSON_MODIFY(
JSON_QUERY('{"definitions": {"id": "INT", "name": "VARCHAR(23)"}}'),
'append $.data',
(
SELECT * FROM (
SELECT 1 AS id, '123abc' AS "name" UNION
SELECT 2 AS id, '234bcd' AS "name"
) AS "data"
FOR JSON PATH
)
) AS "data";
OUTPUT 2
{
"definitions":{
"id":"INT",
"name":"VARCHAR(23)"
},
"data":[
[
{"id":1, "name":"123abc"},
{"id":2, "name":"234bcd"}
]
]
}
QUERY 1
The data object is an array (which is expected), but the problem is what is in the array... A single string with escape characters.
QUERY 2
The data object is an array, which contains an array. In order to access the actual array of data, I would use something like for each obj in data[0].... The problem this poses is, for anyone consuming the JSON object, I would have to tell them:
"In this particular object the data element is an array of
arrays--You'll want to use the first and only the first
element to access the actual array of data."
I've naively tried many different combinations of JSON_MODIFY, JSON_QUERY, and CONCAT to no avail. How can I properly use JSON_MODIFY to get the following output, without the double array in data?
{
"definitions":{
"id":"INT",
"name":"VARCHAR(23)"
},
"data":[
{"id":1, "name":"123abc"},
{"id":2, "name":"234bcd"}
]
}
You are over-thinking this by trying to JSON_MODIFY an existing object.
Construct the definitions and data properties that you need, inside a subquery if necessary.
Then use FOR JSON a second time to get the outer object.
SELECT
definitions = JSON_QUERY('{"id": "INT", "name": "VARCHAR(23)"}'),
data =
(
SELECT id, name
FROM (VALUES
(1, '123abc'),
(2, '234bcd')
) v(id, name)
FOR JSON PATH
)
FOR JSON PATH;
SQL Fiddle
By trial and error, I found the solution.
Removed the append keyword from the path parameter in the JSON_MODIFY statement
Removed the WITHOUT_ARRAY_WRAPPER parameter from the FOR JSON statement.
Now the results are as expected and I don't need to explain to any consumers to "Just use data[0]"
The Query
SELECT JSON_MODIFY(
JSON_QUERY('{"definitions": {"id": "INT", "name": "VARCHAR(23)"}}'),
'$.data',
(
SELECT * FROM (
SELECT 1 AS id, '123abc' AS "name" UNION
SELECT 2 AS id, '234bcd' AS "name"
) AS "data"
FOR JSON PATH
)
) AS "data";
Produces the following output
{
"definitions":{
"id":"INT",
"name":"VARCHAR(23)"
},
"data":[
{"id":1, "name":"123abc"},
{"id":2, "name":"234bcd"}
]
}
I'm trying to migrate Oracle 12c queries to Postgres11.5.
Here is the json:
{
"cost": [{
"spent": [{
"ID": "HR",
"spentamount": {
"amount": 2000.0,
"country": "US"
}
}]
}],
"time": [{
"spent": [{
"ID": "HR",
"spentamount": {
"amount": 308.91,
"country": "US"
}
}]
}]
}
Here is the query that has to be migrated to Postgres 11.5:
select js.*
from P_P_J r,
json_table(r.P_D_J, '$.*[*]'
COLUMNS(NESTED PATH '$.spent[*]'
COLUMNS(
ID VARCHAR2(100 CHAR) PATH '$.ID',
amount NUMBER(10,4) PATH '$.spentamount.amount',
country VARCHAR2(100 CHAR) PATH '$.spentamount.country'))
) js
The result:
ID, amount, country
HR, 2000.0,US
HR,308.91,US
I have two questions here:
What does $.*[*] mean?
How can we migrate this query in Postgres so that it directly looks at 'spent' instead of navigating 'cost'->'spent' or 'time'->'spent'
There is no direct replacement for json_table in Postgres. You will have to combine several calls to explode the JSON structure.
You didn't show us your expected output, but as far as I can tell, the following should do the same:
select e.item ->> 'ID' as id,
(e.item #>> '{spentamount, amount}')::numeric as amount,
e.item #>> '{spentamount, country}' as country
from p_p_j r
cross join jsonb_each(r.p_d_j) as a(key, val)
cross join lateral (
select *
from jsonb_array_elements(a.val)
where jsonb_typeof(a.val) = 'array'
) as s(element)
cross join jsonb_array_elements(s.element -> 'spent') as e(item)
;
The JSON path expression '$.*[*] means: iterate over all top-level keys, then iterate over all array elements found in there and the nested path '$.spent[*]' then again iterates over all array elements in there. These steps are reflected in the three JSON function calls that are needed to get there.
With Postgres 12, this would be a bit easier as this can be done with a single call to jsonb_path_query() which also use a JSON Path to access the elements using a very similar JSON path expression:
select e.item ->> 'ID' as id,
(e.item #>> '{spentamount, amount}')::numeric as amount,
e.item #>> '{spentamount, country}' as country
from p_p_j r
cross join jsonb_path_query(r.p_d_j, '$.*[*].spent[*]') as e(item)
;
Online example
We are exploring the JSON feature in SQL Sever and for one of the scenarios we want to come up with a SQL which can return a JSON like below
[
{
"field": {
"uuid": "uuid-field-1"
},
"value": {
"uuid": "uuid-value" //value is an object
}
},
{
"field": {
"uuid": "uuid-field-2"
},
"value": "1". //value is simple integer
}
... more rows
]
The value field can be a simple integer/string or a nested object.
We are able to come up with a table which looks like below:
field.uuid | value.uuid | value|
------------|---------- | -----|
uuid-field-1| value-uuid | null |
uuid-field-2| null | 1 |
... more rows
But as soon as we apply for json path, it fails saying
Property 'value' cannot be generated in JSON output due to a conflict with another column name or alias. Use different names and aliases for each column in SELECT list.
Is it possible to do it somehow generate this? The value will either be in the value.uuid or value not both?
Note: We are open to possibility of if we can convert each row to individual JSON and add all of them in an array.
select
json_query((select v.[field.uuid] as 'uuid' for json path, without_array_wrapper)) as 'field',
value as 'value',
json_query((select v.[value.uuid] as 'uuid' where v.[value.uuid] is not null for json path, without_array_wrapper)) as 'value'
from
(
values
('uuid-field-1', 'value-uuid1', null),
('uuid-field-2', null, 2),
('uuid-field-3', 'value-uuid3', null),
('uuid-field-4', null, 4)
) as v([field.uuid], [value.uuid], value)
for json auto;--, without_array_wrapper;
The reason for this error is that (as is mentioned in the documentation) ... FOR JSON PATH clause uses the column alias or column name to determine the key name in the JSON output. If an alias contains dots, the PATH option creates nested objects. In your case value.uuid and value both generate a key with name value.
I can suggest an approach (probably not the best one), which uses JSON_MODIFY() to generate the expected JSON from an empty JSON array:
Table:
CREATE TABLE Data (
[field.uuid] varchar(100),
[value.uuid] varchar(100),
[value] int
)
INSERT INTO Data
([field.uuid], [value.uuid], [value])
VALUES
('uuid-field-1', 'value-uuid', NULL),
('uuid-field-2', NULL, 1),
('uuid-field-3', NULL, 3),
('uuid-field-4', NULL, 4)
Statement:
DECLARE #json nvarchar(max) = N'[]'
SELECT #json = JSON_MODIFY(
#json,
'append $',
JSON_QUERY(
CASE
WHEN [value.uuid] IS NOT NULL THEN (SELECT d.[field.uuid], [value.uuid] FOR JSON PATH, WITHOUT_ARRAY_WRAPPER)
WHEN [value] IS NOT NULL THEN (SELECT d.[field.uuid], [value] FOR JSON PATH, WITHOUT_ARRAY_WRAPPER)
END
)
)
FROM Data d
SELECT #json
Result:
[
{
"field":{
"uuid":"uuid-field-1"
},
"value":{
"uuid":"value-uuid"
}
},
{
"field":{
"uuid":"uuid-field-2"
},
"value":1
},
{
"field":{
"uuid":"uuid-field-3"
},
"value":3
},
{
"field":{
"uuid":"uuid-field-4"
},
"value":4
}
]
In my postgres database I have json that looks similar to this:
{
"myArray": [
{
"myValue": 1
},
{
"myValue": 2
},
{
"myValue": 3
}
]
}
Now I want to rename myValue to otherValue. I can't be sure about the length of the array! Preferably I would like to use something like set_jsonb with a wildcard as the array index, but that does not seem to be supported. So what is the nicest solution?
You have to decompose a whole jsonb object, modify individual elements and build the object back.
The custom function will be helpful:
create or replace function jsonb_change_keys_in_array(arr jsonb, old_key text, new_key text)
returns jsonb language sql as $$
select jsonb_agg(case
when value->old_key is null then value
else value- old_key || jsonb_build_object(new_key, value->old_key)
end)
from jsonb_array_elements(arr)
$$;
Use:
with my_table (id, data) as (
values(1,
'{
"myArray": [
{
"myValue": 1
},
{
"myValue": 2
},
{
"myValue": 3
}
]
}'::jsonb)
)
select
id,
jsonb_build_object(
'myArray',
jsonb_change_keys_in_array(data->'myArray', 'myValue', 'otherValue')
)
from my_table;
id | jsonb_build_object
----+------------------------------------------------------------------------
1 | {"myArray": [{"otherValue": 1}, {"otherValue": 2}, {"otherValue": 3}]}
(1 row)
Using json functions are definitely the most elegant, but you can get by on using character replacement. Cast the json(b) as text, perform the replace, then change it back to json(b). In this example I included the quotes and colon to help the text replace target the json keys without conflict with values.
CREATE TABLE mytable ( id INT, data JSONB );
INSERT INTO mytable VALUES (1, '{"myArray": [{"myValue": 1},{"myValue": 2},{"myValue": 3}]}');
INSERT INTO mytable VALUES (2, '{"myArray": [{"myValue": 4},{"myValue": 5},{"myValue": 6}]}');
SELECT * FROM mytable;
UPDATE mytable
SET data = REPLACE(data :: TEXT, '"myValue":', '"otherValue":') :: JSONB;
SELECT * FROM mytable;
http://sqlfiddle.com/#!17/1c28a/9/4