I'm trying to migrate Oracle 12c queries to Postgres11.5.
Here is the json:
{
"cost": [{
"spent": [{
"ID": "HR",
"spentamount": {
"amount": 2000.0,
"country": "US"
}
}]
}],
"time": [{
"spent": [{
"ID": "HR",
"spentamount": {
"amount": 308.91,
"country": "US"
}
}]
}]
}
Here is the query that has to be migrated to Postgres 11.5:
select js.*
from P_P_J r,
json_table(r.P_D_J, '$.*[*]'
COLUMNS(NESTED PATH '$.spent[*]'
COLUMNS(
ID VARCHAR2(100 CHAR) PATH '$.ID',
amount NUMBER(10,4) PATH '$.spentamount.amount',
country VARCHAR2(100 CHAR) PATH '$.spentamount.country'))
) js
The result:
ID, amount, country
HR, 2000.0,US
HR,308.91,US
I have two questions here:
What does $.*[*] mean?
How can we migrate this query in Postgres so that it directly looks at 'spent' instead of navigating 'cost'->'spent' or 'time'->'spent'
There is no direct replacement for json_table in Postgres. You will have to combine several calls to explode the JSON structure.
You didn't show us your expected output, but as far as I can tell, the following should do the same:
select e.item ->> 'ID' as id,
(e.item #>> '{spentamount, amount}')::numeric as amount,
e.item #>> '{spentamount, country}' as country
from p_p_j r
cross join jsonb_each(r.p_d_j) as a(key, val)
cross join lateral (
select *
from jsonb_array_elements(a.val)
where jsonb_typeof(a.val) = 'array'
) as s(element)
cross join jsonb_array_elements(s.element -> 'spent') as e(item)
;
The JSON path expression '$.*[*] means: iterate over all top-level keys, then iterate over all array elements found in there and the nested path '$.spent[*]' then again iterates over all array elements in there. These steps are reflected in the three JSON function calls that are needed to get there.
With Postgres 12, this would be a bit easier as this can be done with a single call to jsonb_path_query() which also use a JSON Path to access the elements using a very similar JSON path expression:
select e.item ->> 'ID' as id,
(e.item #>> '{spentamount, amount}')::numeric as amount,
e.item #>> '{spentamount, country}' as country
from p_p_j r
cross join jsonb_path_query(r.p_d_j, '$.*[*].spent[*]') as e(item)
;
Online example
Related
I'm trying to sum the contents of a json array in a mysql database, below is the JSON format and the query I'm running. Is there something wrong with it?
// Options JSON Format:
[
{
"optionId": 1,
"optionName": "With Meat",
"optionPrice": 2000
},
{
"optionId": 2,
"optionName": "With Veggies",
"optionPrice": 0
}
]
// Query:
SELECT id, SUM(options->'$[*].optionPrice') FROM table_order_items GROUP BY id;
The result is 0, when it should be 2000
While this query:
SELECT id, options->'$[*].optionPrice' FROM table_order_items;
correctly returns [2000,0]
You need the function JSON_TABLE() to extract the prices:
SELECT t.id,
SUM(j.price) AS total
FROM table_order_items t
JOIN JSON_TABLE(
t.options,
'$[*]' COLUMNS(price INT PATH '$.optionPrice')
) j
GROUP BY t.id;
See the demo.
I am using JSON_MODIFY to build complex JSON. Moving from MySQL I am struggling with the JSON functions provided by SQL Server. The issue I'm having is that SQL Server seems to construct all JSON objects in an array. There is the WITHOUT_ARRAY_WRAPPER mechanism, which seems like it should do what I want, however; there are two undesirable consequences.
It only returns one result depending on how it is used
The result is a single string with escape characters
I have constructed a simple query which illustrates my needs and the issue.
QUERY 1
SELECT JSON_MODIFY(
JSON_QUERY('{"definitions": {"id": "INT", "name": "VARCHAR(23)"}}'),
'append $.data',
(
SELECT * FROM (
SELECT 1 AS id, '123abc' AS "name" UNION
SELECT 2 AS id, '234bcd' AS "name"
) AS "data"
FOR JSON PATH, WITHOUT_ARRAY_WRAPPER
)
) AS "data";
OUTPUT 1
{
"definitions":{
"id":"INT",
"name":"VARCHAR(23)"
},
"data":[
"{\"id\":1,\"name\":\"123abc\"},{\"id\":2,\"name\":\"234bcd\"}"
]
}
QUERY 2
SELECT JSON_MODIFY(
JSON_QUERY('{"definitions": {"id": "INT", "name": "VARCHAR(23)"}}'),
'append $.data',
(
SELECT * FROM (
SELECT 1 AS id, '123abc' AS "name" UNION
SELECT 2 AS id, '234bcd' AS "name"
) AS "data"
FOR JSON PATH
)
) AS "data";
OUTPUT 2
{
"definitions":{
"id":"INT",
"name":"VARCHAR(23)"
},
"data":[
[
{"id":1, "name":"123abc"},
{"id":2, "name":"234bcd"}
]
]
}
QUERY 1
The data object is an array (which is expected), but the problem is what is in the array... A single string with escape characters.
QUERY 2
The data object is an array, which contains an array. In order to access the actual array of data, I would use something like for each obj in data[0].... The problem this poses is, for anyone consuming the JSON object, I would have to tell them:
"In this particular object the data element is an array of
arrays--You'll want to use the first and only the first
element to access the actual array of data."
I've naively tried many different combinations of JSON_MODIFY, JSON_QUERY, and CONCAT to no avail. How can I properly use JSON_MODIFY to get the following output, without the double array in data?
{
"definitions":{
"id":"INT",
"name":"VARCHAR(23)"
},
"data":[
{"id":1, "name":"123abc"},
{"id":2, "name":"234bcd"}
]
}
You are over-thinking this by trying to JSON_MODIFY an existing object.
Construct the definitions and data properties that you need, inside a subquery if necessary.
Then use FOR JSON a second time to get the outer object.
SELECT
definitions = JSON_QUERY('{"id": "INT", "name": "VARCHAR(23)"}'),
data =
(
SELECT id, name
FROM (VALUES
(1, '123abc'),
(2, '234bcd')
) v(id, name)
FOR JSON PATH
)
FOR JSON PATH;
SQL Fiddle
By trial and error, I found the solution.
Removed the append keyword from the path parameter in the JSON_MODIFY statement
Removed the WITHOUT_ARRAY_WRAPPER parameter from the FOR JSON statement.
Now the results are as expected and I don't need to explain to any consumers to "Just use data[0]"
The Query
SELECT JSON_MODIFY(
JSON_QUERY('{"definitions": {"id": "INT", "name": "VARCHAR(23)"}}'),
'$.data',
(
SELECT * FROM (
SELECT 1 AS id, '123abc' AS "name" UNION
SELECT 2 AS id, '234bcd' AS "name"
) AS "data"
FOR JSON PATH
)
) AS "data";
Produces the following output
{
"definitions":{
"id":"INT",
"name":"VARCHAR(23)"
},
"data":[
{"id":1, "name":"123abc"},
{"id":2, "name":"234bcd"}
]
}
Given the following two table columns jsonb type:
dividend_actual
{
"dividends": [
{
"amount": "2.9800",
"balanceDate": "2020-06-30T00:00:00Z"
},
{
"amount": "4.3100",
"balanceDate": "2019-06-30T00:00:00Z"
}
],
"lastUpdated": "2020-11-16T14:50:51.289649512Z",
"providerUpdateDate": "2020-11-16T00:00:00Z"
}
dividend_forecast
{
"dividends": [
{
"amount": "2.3035",
"balanceDate": "2021-06-01T00:00:00Z"
},
{
"amount": "3.0452",
"balanceDate": "2022-06-01T00:00:00Z"
},
{
"amount": "3.1845",
"balanceDate": "2023-06-01T00:00:00Z"
}
],
"lastForecasted": "2020-11-13T00:00:00Z",
"providerUpdateDate": "2020-11-16T00:00:00Z"
}
I would like to merge both dividends arrays from dividend_actual and dividend_forecast, but before merging them I want to add an extra field (forecast) on every single object.
I did try the following:
SELECT
dividends
FROM
stock_financial AS f
INNER JOIN instrument AS i ON i.id = f.instrument_id,
jsonb_array_elements(
(f.dividend_forecast->'dividends' || jsonb '{"forecast": true}') ||
(f.dividend_actual->'dividends' || jsonb '{"forecast": false}')
) AS dividends
WHERE
i.symbol = 'ASX_CBA'
ORDER BY
dividends ->>'balanceDate' DESC;
The above query gives me the following results:
{"forecast":true}
{"forecast":false}
{"amount":"3.1845","balanceDate":"2023-06-01T00:00:00Z"}
{"amount":"3.0452","balanceDate":"2022-06-01T00:00:00Z"}
{"amount":"2.3035","balanceDate":"2021-06-01T00:00:00Z"}
{"amount":"2.9800","balanceDate":"2020-06-30T00:00:00Z"}
{"amount":"4.3100","balanceDate":"2019-06-30T00:00:00Z"}
But what I need instead is the following output:
{"amount":"3.1845","balanceDate":"2023-06-01T00:00:00Z","forecast":true}
{"amount":"3.0452","balanceDate":"2022-06-01T00:00:00Z","forecast":true}
{"amount":"2.3035","balanceDate":"2021-06-01T00:00:00Z","forecast":true}
{"amount":"2.9800","balanceDate":"2020-06-30T00:00:00Z","forecast":false}
{"amount":"4.3100","balanceDate":"2019-06-30T00:00:00Z","forecast":false}
It turns out that it is not possible to update multiple jsons objects within a json array in a single operation by default.
To be able to do that a Postgres function needs to be created:
-- the params are the same as in aforementioned `jsonb_set`
CREATE OR REPLACE FUNCTION update_json_array_elements(target jsonb, path text[], new_value jsonb)
RETURNS jsonb language sql AS $$
-- aggregate the jsonb from parts created in LATERAL
SELECT jsonb_agg(updated_jsonb)
-- split the target array to individual objects...
FROM jsonb_array_elements(target) individual_object,
-- operate on each object and apply jsonb_set to it. The results are aggregated in SELECT
LATERAL jsonb_set(individual_object, path, new_value) updated_jsonb
$$;
The above function was suggested by kubak in this answer: https://stackoverflow.com/a/53712268/782390
Combined with this query:
SELECT
dividends
FROM
stock_financial AS f
INNER JOIN instrument AS i ON i.id = f.instrument_id,
jsonb_array_elements(
update_json_array_elements(f.dividend_forecast->'dividends', '{forecast}', 'true') ||
update_json_array_elements(f.dividend_actual->'dividends', '{forecast}', 'false')
) AS dividends
WHERE
i.symbol = 'ASX_CBA'
ORDER BY
dividends ->>'balanceDate' DESC;
I then get the following output, that it is exactly what I need:
{"amount":"3.1845","forecast":true,"balanceDate":"2023-06-01T00:00:00Z"}
{"amount":"3.0452","forecast":true,"balanceDate":"2022-06-01T00:00:00Z"}
{"amount":"2.3035","forecast":true,"balanceDate":"2021-06-01T00:00:00Z"}
{"amount":"2.9800","forecast":false,"balanceDate":"2020-06-30T00:00:00Z"}
{"amount":"4.3100","forecast":false,"balanceDate":"2019-06-30T00:00:00Z"}
I'm trying to migrate Oracle 12c queries to Postgres11.5.
Here is the json:
{
"cost": [{
"spent": [{
"ID": "HR",
"spentamount": {
"amount": 2000.0,
"country": "US"
}
}]
}],
"time": [{
"spent": [{
"ID": "HR",
"spentamount": {
"amount": 308.91,
"country": "US"
}
}]
}]
}
Here is the query that has to be migrated to Postgres 11.5:
select js.*
from P_P_J r,
json_table(r.P_D_J, '$.*[*]'
COLUMNS(NESTED PATH '$.spent[*]'
COLUMNS(
ID VARCHAR2(100 CHAR) PATH '$.ID',
amount NUMBER(10,4) PATH '$.spentamount.amount',
country VARCHAR2(100 CHAR) PATH '$.spentamount.country'))
) js
The result:
ID, amount, country
HR, 2000.0,US
HR,308.91,US
I have two questions here:
What does $.*[*] mean?
How can we migrate this query in Postgres so that it directly looks at 'spent' instead of navigating 'cost'->'spent' or 'time'->'spent'
There is no direct replacement for json_table in Postgres. You will have to combine several calls to explode the JSON structure.
You didn't show us your expected output, but as far as I can tell, the following should do the same:
select e.item ->> 'ID' as id,
(e.item #>> '{spentamount, amount}')::numeric as amount,
e.item #>> '{spentamount, country}' as country
from p_p_j r
cross join jsonb_each(r.p_d_j) as a(key, val)
cross join lateral (
select *
from jsonb_array_elements(a.val)
where jsonb_typeof(a.val) = 'array'
) as s(element)
cross join jsonb_array_elements(s.element -> 'spent') as e(item)
;
The JSON path expression '$.*[*] means: iterate over all top-level keys, then iterate over all array elements found in there and the nested path '$.spent[*]' then again iterates over all array elements in there. These steps are reflected in the three JSON function calls that are needed to get there.
With Postgres 12, this would be a bit easier as this can be done with a single call to jsonb_path_query() which also use a JSON Path to access the elements using a very similar JSON path expression:
select e.item ->> 'ID' as id,
(e.item #>> '{spentamount, amount}')::numeric as amount,
e.item #>> '{spentamount, country}' as country
from p_p_j r
cross join jsonb_path_query(r.p_d_j, '$.*[*].spent[*]') as e(item)
;
Online example
I want that MySQL will return a result set in a form of JSON array.
A worked query:
SELECT CONCAT(
'[',
GROUP_CONCAT(
JSON_OBJECT(
'ProductId', tblproducts.ProductId,
'ProductName', tblproducts.ProductName
)
),
']') As Products
FROM tblproducts;
This query returns the following JSON array:
[
{
"ProductId": "1",
"ProductName": "Milk"
},
{
"ProductId": "2",
"ProductName": "Bread"
}
]
Although this code works, I have a strong feeling that building a JSON array with GROUP_CONCAT and CONCAT is kind of workaround. If to use a JSON_ARRAY, a result set will consist of JSON arrays for each JSON_OBJECT.
Is there any native way to get a single JSON array with all JSON_OBJECT in the result set?
JSON_ARRAYAGG() could be what you're looking for.
SELECT JSON_ARRAYAGG(
JSON_OBJECT(
'ProductId', tblproducts.ProductId,
'ProductName', tblproducts.ProductName
)
) FROM tblproducts;
Here's an example: https://www.db-fiddle.com/f/uQ9UC7MDZM4gncNjViTsKw/0