Flatten arrays of string - couchbase

I have the following document
{
"agentId": "agent2",
"date": "2022-08-30",
"metaData": {
"documentCreationDate": "2022-08-30T15:49:21Z",
"documentVersion": "1.0",
"expiry": 1662479361
},
"emailInteractions": [
"0c99ea2a-c235-4c5a-a0bd-aeffba559bca",
"12846a9d-7cc1-4755-b527-cd8aee9d2de4"
],
"voiceInteractions": [
"1c99ea2a-c235-4c5a-a0bd-aeffba559bca",
"22846a9d-7cc1-4755-b527-cd8aee9d2de4"
]
}
And I'm trying to retrieve a list of ids found in emailInteraction and/or voiceInteraction, I'm currently doing the following
SELECT ARRAY_UNION(IFMISSINGORNULL(emailInteractions, []), IFMISSINGORNULL(voiceInteractions, [])) AS ids
FROM `agent-activities`
WHERE agentId = "agent2"
My issue is that it returns this :
[
[
"12846a9d-7cc1-4755-b527-cd8aee9d2de4",
"22846a9d-7cc1-4755-b527-cd8aee9d2de4",
"0c99ea2a-c235-4c5a-a0bd-aeffba559bca",
"1c99ea2a-c235-4c5a-a0bd-aeffba559bca"
]
]
and it bothers me because I need a plain array of strings and not an array of array of strings, because I need to use this list of ids in a where clause in another query with something along the lines of :
WHERE lst.interactionId IN (SELECT raw ARRAY_UNION(IFMISSINGORNULL(emailInteractions, []), IFMISSINGORNULL(voiceInteractions, [])) AS ids
FROM `agent-activities`
WHERE agentId = $agentId)
FYI my main query is this :
SELECT lst.*
FROM (
SELECT iv.id AS interventionId,
vi.direction,
vi.channel,
vi.startDate AS startDate,
vi.id AS interactionId,
vi.customerProfileId
FROM `voice-interactions` AS vi
UNNEST vi.distributions AS dv
UNNEST dv.interventions AS iv
UNION
SELECT ie.id AS interventionId,
ei.direction,
ei.channel,
ei.startDate AS startDate,
ei.id AS interactionId,
ei.customerProfileId
FROM `email-interactions` AS ei
UNNEST ei.distributions AS de
UNNEST de.interventions AS ie) AS lst
WHERE lst.interactionId IN (
SELECT raw ARRAY_UNION(IFMISSINGORNULL(emailInteractions, []), IFMISSINGORNULL(voiceInteractions, [])) AS ids
FROM `agent-activities`
WHERE agentId = $agentId)
ORDER BY startDate ASC
LIMIT $limit
OFFSET $offset
I could use some help to either flatten my array of array into an array somehow or find a better solution than this where clause with the 'property in subquery'

WITH interactionIds AS ( SELECT DISTINCT RAW u
FROM `agent-activities` AS a
UNNEST ARRAY_CONCAT(IFMISSINGORNULL(a.emailInteractions, []), IFMISSINGORNULL(a.voiceInteractions, [])) AS u
WHERE a.agentId = $agentId)
)
SELECT lst.*
FROM (
SELECT iv.id AS interventionId,
vi.direction,
vi.channel,
vi.startDate AS startDate,
vi.id AS interactionId,
vi.customerProfileId
FROM `voice-interactions` AS vi
UNNEST vi.distributions AS dv
UNNEST dv.interventions AS iv
UNION
SELECT ie.id AS interventionId,
ei.direction,
ei.channel,
ei.startDate AS startDate,
ei.id AS interactionId,
ei.customerProfileId
FROM `email-interactions` AS ei
UNNEST ei.distributions AS de
UNNEST de.interventions AS ie) AS lst
WHERE lst.interactionId IN interactionIds
ORDER BY startDate ASC
LIMIT $limit
OFFSET $offset;
Also you can use ARRAY_DISTINCT(ARRAY_FLATTEN((subquery),1))
NOTE: In function argument you need () around subquery treat it as expression

Related

How to Convert Oracle to Postgresql [duplicate]

I'm trying to migrate Oracle 12c queries to Postgres11.5.
Here is the json:
{
"cost": [{
"spent": [{
"ID": "HR",
"spentamount": {
"amount": 2000.0,
"country": "US"
}
}]
}],
"time": [{
"spent": [{
"ID": "HR",
"spentamount": {
"amount": 308.91,
"country": "US"
}
}]
}]
}
Here is the query that has to be migrated to Postgres 11.5:
select js.*
from P_P_J r,
json_table(r.P_D_J, '$.*[*]'
COLUMNS(NESTED PATH '$.spent[*]'
COLUMNS(
ID VARCHAR2(100 CHAR) PATH '$.ID',
amount NUMBER(10,4) PATH '$.spentamount.amount',
country VARCHAR2(100 CHAR) PATH '$.spentamount.country'))
) js
The result:
ID, amount, country
HR, 2000.0,US
HR,308.91,US
I have two questions here:
What does $.*[*] mean?
How can we migrate this query in Postgres so that it directly looks at 'spent' instead of navigating 'cost'->'spent' or 'time'->'spent'
There is no direct replacement for json_table in Postgres. You will have to combine several calls to explode the JSON structure.
You didn't show us your expected output, but as far as I can tell, the following should do the same:
select e.item ->> 'ID' as id,
(e.item #>> '{spentamount, amount}')::numeric as amount,
e.item #>> '{spentamount, country}' as country
from p_p_j r
cross join jsonb_each(r.p_d_j) as a(key, val)
cross join lateral (
select *
from jsonb_array_elements(a.val)
where jsonb_typeof(a.val) = 'array'
) as s(element)
cross join jsonb_array_elements(s.element -> 'spent') as e(item)
;
The JSON path expression '$.*[*] means: iterate over all top-level keys, then iterate over all array elements found in there and the nested path '$.spent[*]' then again iterates over all array elements in there. These steps are reflected in the three JSON function calls that are needed to get there.
With Postgres 12, this would be a bit easier as this can be done with a single call to jsonb_path_query() which also use a JSON Path to access the elements using a very similar JSON path expression:
select e.item ->> 'ID' as id,
(e.item #>> '{spentamount, amount}')::numeric as amount,
e.item #>> '{spentamount, country}' as country
from p_p_j r
cross join jsonb_path_query(r.p_d_j, '$.*[*].spent[*]') as e(item)
;
Online example

JSON EXTRACT SUM returning 0 instead of correct value

I'm trying to sum the contents of a json array in a mysql database, below is the JSON format and the query I'm running. Is there something wrong with it?
// Options JSON Format:
[
{
"optionId": 1,
"optionName": "With Meat",
"optionPrice": 2000
},
{
"optionId": 2,
"optionName": "With Veggies",
"optionPrice": 0
}
]
// Query:
SELECT id, SUM(options->'$[*].optionPrice') FROM table_order_items GROUP BY id;
The result is 0, when it should be 2000
While this query:
SELECT id, options->'$[*].optionPrice' FROM table_order_items;
correctly returns [2000,0]
You need the function JSON_TABLE() to extract the prices:
SELECT t.id,
SUM(j.price) AS total
FROM table_order_items t
JOIN JSON_TABLE(
t.options,
'$[*]' COLUMNS(price INT PATH '$.optionPrice')
) j
GROUP BY t.id;
See the demo.

How To Merge Multiple PostgreSQL JSON Arrays That Are The Result Of Another JSON Array Update On All Of Its Elements?

Given the following two table columns jsonb type:
dividend_actual
{
"dividends": [
{
"amount": "2.9800",
"balanceDate": "2020-06-30T00:00:00Z"
},
{
"amount": "4.3100",
"balanceDate": "2019-06-30T00:00:00Z"
}
],
"lastUpdated": "2020-11-16T14:50:51.289649512Z",
"providerUpdateDate": "2020-11-16T00:00:00Z"
}
dividend_forecast
{
"dividends": [
{
"amount": "2.3035",
"balanceDate": "2021-06-01T00:00:00Z"
},
{
"amount": "3.0452",
"balanceDate": "2022-06-01T00:00:00Z"
},
{
"amount": "3.1845",
"balanceDate": "2023-06-01T00:00:00Z"
}
],
"lastForecasted": "2020-11-13T00:00:00Z",
"providerUpdateDate": "2020-11-16T00:00:00Z"
}
I would like to merge both dividends arrays from dividend_actual and dividend_forecast, but before merging them I want to add an extra field (forecast) on every single object.
I did try the following:
SELECT
dividends
FROM
stock_financial AS f
INNER JOIN instrument AS i ON i.id = f.instrument_id,
jsonb_array_elements(
(f.dividend_forecast->'dividends' || jsonb '{"forecast": true}') ||
(f.dividend_actual->'dividends' || jsonb '{"forecast": false}')
) AS dividends
WHERE
i.symbol = 'ASX_CBA'
ORDER BY
dividends ->>'balanceDate' DESC;
The above query gives me the following results:
{"forecast":true}
{"forecast":false}
{"amount":"3.1845","balanceDate":"2023-06-01T00:00:00Z"}
{"amount":"3.0452","balanceDate":"2022-06-01T00:00:00Z"}
{"amount":"2.3035","balanceDate":"2021-06-01T00:00:00Z"}
{"amount":"2.9800","balanceDate":"2020-06-30T00:00:00Z"}
{"amount":"4.3100","balanceDate":"2019-06-30T00:00:00Z"}
But what I need instead is the following output:
{"amount":"3.1845","balanceDate":"2023-06-01T00:00:00Z","forecast":true}
{"amount":"3.0452","balanceDate":"2022-06-01T00:00:00Z","forecast":true}
{"amount":"2.3035","balanceDate":"2021-06-01T00:00:00Z","forecast":true}
{"amount":"2.9800","balanceDate":"2020-06-30T00:00:00Z","forecast":false}
{"amount":"4.3100","balanceDate":"2019-06-30T00:00:00Z","forecast":false}
It turns out that it is not possible to update multiple jsons objects within a json array in a single operation by default.
To be able to do that a Postgres function needs to be created:
-- the params are the same as in aforementioned `jsonb_set`
CREATE OR REPLACE FUNCTION update_json_array_elements(target jsonb, path text[], new_value jsonb)
RETURNS jsonb language sql AS $$
-- aggregate the jsonb from parts created in LATERAL
SELECT jsonb_agg(updated_jsonb)
-- split the target array to individual objects...
FROM jsonb_array_elements(target) individual_object,
-- operate on each object and apply jsonb_set to it. The results are aggregated in SELECT
LATERAL jsonb_set(individual_object, path, new_value) updated_jsonb
$$;
The above function was suggested by kubak in this answer: https://stackoverflow.com/a/53712268/782390
Combined with this query:
SELECT
dividends
FROM
stock_financial AS f
INNER JOIN instrument AS i ON i.id = f.instrument_id,
jsonb_array_elements(
update_json_array_elements(f.dividend_forecast->'dividends', '{forecast}', 'true') ||
update_json_array_elements(f.dividend_actual->'dividends', '{forecast}', 'false')
) AS dividends
WHERE
i.symbol = 'ASX_CBA'
ORDER BY
dividends ->>'balanceDate' DESC;
I then get the following output, that it is exactly what I need:
{"amount":"3.1845","forecast":true,"balanceDate":"2023-06-01T00:00:00Z"}
{"amount":"3.0452","forecast":true,"balanceDate":"2022-06-01T00:00:00Z"}
{"amount":"2.3035","forecast":true,"balanceDate":"2021-06-01T00:00:00Z"}
{"amount":"2.9800","forecast":false,"balanceDate":"2020-06-30T00:00:00Z"}
{"amount":"4.3100","forecast":false,"balanceDate":"2019-06-30T00:00:00Z"}

Oracle JSON_TABLE to PostgreSQL - how to search from the second hierarchical key in a JSON column

I'm trying to migrate Oracle 12c queries to Postgres11.5.
Here is the json:
{
"cost": [{
"spent": [{
"ID": "HR",
"spentamount": {
"amount": 2000.0,
"country": "US"
}
}]
}],
"time": [{
"spent": [{
"ID": "HR",
"spentamount": {
"amount": 308.91,
"country": "US"
}
}]
}]
}
Here is the query that has to be migrated to Postgres 11.5:
select js.*
from P_P_J r,
json_table(r.P_D_J, '$.*[*]'
COLUMNS(NESTED PATH '$.spent[*]'
COLUMNS(
ID VARCHAR2(100 CHAR) PATH '$.ID',
amount NUMBER(10,4) PATH '$.spentamount.amount',
country VARCHAR2(100 CHAR) PATH '$.spentamount.country'))
) js
The result:
ID, amount, country
HR, 2000.0,US
HR,308.91,US
I have two questions here:
What does $.*[*] mean?
How can we migrate this query in Postgres so that it directly looks at 'spent' instead of navigating 'cost'->'spent' or 'time'->'spent'
There is no direct replacement for json_table in Postgres. You will have to combine several calls to explode the JSON structure.
You didn't show us your expected output, but as far as I can tell, the following should do the same:
select e.item ->> 'ID' as id,
(e.item #>> '{spentamount, amount}')::numeric as amount,
e.item #>> '{spentamount, country}' as country
from p_p_j r
cross join jsonb_each(r.p_d_j) as a(key, val)
cross join lateral (
select *
from jsonb_array_elements(a.val)
where jsonb_typeof(a.val) = 'array'
) as s(element)
cross join jsonb_array_elements(s.element -> 'spent') as e(item)
;
The JSON path expression '$.*[*] means: iterate over all top-level keys, then iterate over all array elements found in there and the nested path '$.spent[*]' then again iterates over all array elements in there. These steps are reflected in the three JSON function calls that are needed to get there.
With Postgres 12, this would be a bit easier as this can be done with a single call to jsonb_path_query() which also use a JSON Path to access the elements using a very similar JSON path expression:
select e.item ->> 'ID' as id,
(e.item #>> '{spentamount, amount}')::numeric as amount,
e.item #>> '{spentamount, country}' as country
from p_p_j r
cross join jsonb_path_query(r.p_d_j, '$.*[*].spent[*]') as e(item)
;
Online example

Correctly return column as JSON Array in MySQL after using CONCAT, GROUP_CONCAT and JSON_OBJECT

I'm using MySQL in a Node.JS API, so I need to get data from the database as JSON objects/arrays.
I'm trying to get a JSON Array nested within the result JSON as one of the values, so this is my current query:
SELECT
l.id AS id, l.description AS description, l.parent AS parent,
(
SELECT CONCAT(
'[',
GROUP_CONCAT(
JSON_OBJECT(
'id', a.id, 'description', a.description,
'ip', a.ip, 'lastmovementdetected', a.lastmovementdetected
)
),
']'
)
FROM airconditioners AS a WHERE location = l.id
) AS airconditioners
FROM locations as l`
However, this is the query result (actual output is an array of these JSON objects):
{
"id": 1,
"description": "Meu quarto",
"parent": 0,
"airconditioners": "[{\"id\": 1, \"ip\": \"192.168.137.96\", \"description\": \"Ar-condicionado\", \"lastmovementdetected\": null},{\"id\": 2, \"ip\": \"192.168.0.1\", \"description\": \"Ar-condicionado\", \"lastmovementdetected\": null},{\"id\": 3, \"ip\": \"192.168.0.1\", \"description\": \"Ar-condicionado\", \"lastmovementdetected\": null}]"
}
SQL is returning the JSON Array as a String and it's also escaping the double quotes from within the JSON.
This is the expected return:
"id": 1,
"description": "Meu quarto",
"parent": 0,
"airconditioners": [
{
"id":1,
"ip":"192.168.137.96",
"description":"Ar-condicionado",
"lastmovementdetected":null
},
{
"id":2,
"ip":"192.168.0.1",
"description":"Ar-condicionado",
"lastmovementdetected":null
},
{
"id":3,
"ip":"192.168.0.1",
"description":"Ar-condicionado",
"lastmovementdetected":null
}
]
Can this be done using a SQL query only? Or I'll have to treat the result before sending the response on the API?
I've tried surrounding the column with a CAST((SELECT...) AS JSON) AS airconditioners and also putting JSON_UNQUOTE() in many places, with no success whatsoever.
EDIT
I couldn't get to a conclusion whether MySQL is compatible with what I want or not. But, for instance, I'm using the following Javascript code to work around it:
Object.keys(result).forEach(key => {
let airconditioners = result[key].airconditioners;
if(airconditioners == null) {
// If the airconditioner field is null, then we replace it with an empty array
result[key].airconditioners = [];
} else {
result[key].airconditioners = JSON.parse(airconditioners);
}
});
use JSON_EXTRACT then get result as you expect
SELECT
l.id AS id, l.description AS description, l.parent AS parent,
(
SELECT JSON_EXTRACT( IFNULL(
CONCAT(
'[',
GROUP_CONCAT(
JSON_OBJECT(
'id', a.id, 'description', a.description,
'ip', a.ip, 'lastmovementdetected', a.lastmovementdetected
)
),
']'
)
,'[]'),'$')
FROM airconditioners AS a WHERE location = l.id
) AS airconditioners
FROM locations as l`