How to map nested array items with N1QL? - couchbase

I have documents in a bucket called blocks in the following format:
{
"random_field": 1,
"transactions": [{
"id": "CCCCC",
"inputs": [{
"tx_id": "AAAAA",
"index": 0
},{
"tx_id": "BBBBB",
"index": 1
}]
}]
}
{
"transactions": [{
"id": "AAAAA",
"outputs": [{
"field1": "value123",
"field2": "value456"
},{
"field1": "ignore",
"field2": "ignore"
}]
}]
}
{
"transactions": [{
"id": "BBBBB",
"outputs": [{
"field1": "ignored",
"field2": "ignored"
},{
"field1": "value999",
"field2": "value888"
}]
}]
}
and I need to map the inputs from the first document to the corresponding outputs of the second and third documents. The way to do it manually is to, for each input, find a transaction with id equal to the input's tx_id, and then get the item from the outputs array based on the index of the input. To exemplify, this is the object I would like to return in this scenario:
{
"random_field": 1,
"transactions": [{
"id": "CCCCC",
"inputs": [{
"tx_id": "AAAAA",
"index": 0,
"output": {
"field1": "value123",
"field2": "value456"
}
},{
"tx_id": "BBBBB",
"index": 1,
"output": {
"field1": "value999",
"field2": "value888"
}
}]
}]
}
I managed to come up with the following query:
SELECT b.random_field,
b.transactions -- how to map this?
FROM blocks b
UNNEST b.transactions t
UNNEST t.inputs input
JOIN blocks `source` ON (ANY tx IN `source`.transactions SATISFIES tx.`id` = input.tx_id END)
UNNEST `source`.transactions source_tx
UNNEST source_tx.outputs o
WHERE (ANY tx IN b.transactions SATISFIES tx.`id` = 'AAAAA' END) LIMIT 1;
I suppose there should be a way to map b.transactions.inputs by using source_tx.outputs, but I couldn't find how.
I came across this other answer, but I don't really understand how it applies to my scenario. Maybe it does, but I am very new to Couchbase, so I am very much lost: How to map array values in one document to another and display in result

Basically you want inline some other document into current document using condition.
Instead of JOINs+ GROUPS use subquery expressions + correlated subqueries. (b.*, "abc" AS transactions, selects all fields of b and adds transactions (if already exist overwrite else adds)
CREATE INDEX ix1 ON blocks (ALL ARRAY FOR ot.id FOR ot IN transactions END);
SELECT b.*,
(SELECT t.*,
(SELECT i.*,
(SELECT RAW ot
FROM blocks AS o
UNNEST o.transactions AS ot
UNNEST ot.outputs AS oto
WHERE i.tx_id = ot.id AND i.`index` = UNNEST_POS(oto))[0] AS output
FROM t.`inputs` AS i) AS inputs
FROM b.transactions AS t) AS transactions
FROM blocks AS b
WHERE ANY tx IN b.transactions SATISFIES tx.`inputs` IS NOT NULL END ;
OR
SELECT b.*,
(SELECT t.*,
(SELECT i.*,
(SELECT RAW ot.outputs[i.`index`]
FROM blocks AS o
UNNEST o.transactions AS ot
WHERE i.tx_id = ot.id
LIMIT 1)[0] AS output
FROM t.`inputs` AS i) AS inputs
FROM b.transactions AS t) AS transactions
FROM blocks AS b
WHERE ANY tx IN b.transactions SATISFIES tx.`inputs` IS NOT NULL END ;

Related

How to get a specific object in an JSON array in MySQL?

I have a JSON column "jobs" that looks like this:
[
{
"id": "1",
"done": "100",
"target": "100",
"startDate": "123123132",
"lastAction": "123123132",
"status": "0"
},
{
"id": "2",
"done": "10",
"target": "20",
"startDate": "2312321",
"lastAction": "2312321",
"status": "1"
}
]
I want to filter the array by object key values. For example: To find all items that have target > done, status != 0 and lastAction is yesterday to get response like this:
[
{
"id": "1",
"done": "19",
"target": "100",
"startDate": "123123132",
"lastAction": "123123132",
"status": "0"
}
]
I know I can extract the data to a JSON_TABLE() to do the filtering but I don't get the original object back(unless I recreate it back) and the solution is not dynamic.
Can this kind of array filtering can really be done in MySQL?
SELECT JSON_PRETTY(JSON_EXTRACT(jobs.jobs, CONCAT('$[', j.rownum-1, ']'))) AS object
FROM jobs
CROSS JOIN JSON_TABLE(
jobs.jobs, '$[*]' COLUMNS(
rownum for ordinality,
done int path '$.done',
target int path '$.target',
status int path '$.status'
)
) as j
WHERE j.target > j.done AND j.status != 0;
You also mentioned a condition on lastAction, but the example values you gave are not valid dates, so I'll leave that enhancement to you. The example above demonstrates the technique.
Yes it is possible to do it using the JSON_EXTRACT and JSON_SEARCH functions.
Let's say your table is named tbl_Jobs and the jobs column is of type JSON.
SELECT * FROM tbl_Jobs
WHERE JSON_EXTRACT(jobs, "$[*].target") = JSON_EXTRACT(jobs, "$[*].done")
AND JSON_EXTRACT(jobs, "$[*].status") != 0
AND JSON_SEARCH(jobs, 'one', DATE_SUB(CURDATE(), INTERVAL 1 DAY), NULL, "$[*].lastAction") IS NOT NULL

How to deal with not existing values using JSON_EXTRACT?

I have a list ob objects. Each object contains several properties. Now I want to make a SELECT statement that gives me a list of a single property values. The simplified list look like this:
[
[
{
"day": "2021-10-01",
"entries": [
{
"name": "Start of competition",
"startTimeDelta": "08:30:00"
}
]
},
{
"day": "2021-10-02",
"entries": [
{
"name": "Start of competition",
"startTimeDelta": "03:30:00"
}
]
},
{
"day": "2021-10-03",
"entries": [
{
"name": "Start of competition"
}
]
}
]
]
The working SELECT is now
SELECT
JSON_EXTRACT(column, '$.days[*].entries[0].startTimeDelta') AS list
FROM table
The returned result is
[
"08:30:00",
"03:30:00"
]
But what I want to get (and also have expected) is
[
"08:30:00",
"03:30:00",
null
]
What can I do or how can I change the SELECT statement so that I also get NULL values in the list?
SELECT startTimeDelta
FROM test
CROSS JOIN JSON_TABLE(val,
'$[*][*].entries[*]' COLUMNS (startTimeDelta TIME PATH '$.startTimeDelta')) jsontable
https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=491f0f978d200a8a8522e3200509460e
Do you also have a working idea for MySQL< 8? – Lars
What is max amount of objects in the array on the 2nd level? – Akina
Well it's usually less than 10 – Lars
SELECT JSON_EXTRACT(val, CONCAT('$[0][', num, '].entries[0].startTimeDelta')) startTimeDelta
FROM test
-- up to 4 - increase if needed
CROSS JOIN (SELECT 0 num UNION SELECT 1 UNION SELECT 2 UNION SELECT 3) nums
WHERE JSON_EXTRACT(val, CONCAT('$[0][', num, '].entries[0]')) IS NOT NULL;
https://www.db-fiddle.com/f/xnCCSTGQXevcpfPH1GAbUo/0

Expand Postgresql Nested Array Json Field

I have a table (log_table) and in this table there is a nested array json field (activities). With using this activities field, I want to normalize my row.
log_table:
- id:long
- activities:json
- date:timestamp
example activities field:
[
{
"actionType":"NOTIFICATION",
"items":null
},
{
"actionType":"MUTATION",
"items":[
{
"id":387015007,
"name":"epic",
"value":{
"currency":"USD",
"amount":1.76
}
},
{
"id":386521039,
"name":"test",
"value":{
"currency":"USD",
"amount":1.76
}
}
]
}
]
As query, I've tried:
select
*
from
log_table l,
json_array_elements(l.activities) elems,
json_array_elements(elems->'items') obj;
With this query, I got error like below:
ERROR: cannot call json_array_elements on a scalar
Is there any suggestion?
The lack of items should be marked as [null], not null. You can use the case expression to correct this, e.g.:
select elems->>'actionType' as action_type, obj
from log_table
cross join jsonb_array_elements(l.activities::jsonb) elems
cross join jsonb_array_elements(case elems->'items' when 'null' then '[null]' else elems->'items' end) obj
action_type | obj
--------------+---------------------------------------------------------------------------------
NOTIFICATION | null
MUTATION | {"id": 387015007, "name": "epic", "value": {"amount": 1.76, "currency": "USD"}}
MUTATION | {"id": 386521039, "name": "test", "value": {"amount": 1.76, "currency": "USD"}}
(3 rows)

N1QL Check if the array contains id

I have a document called
player::id
for each player. Where id is the player's id (auto-incremented).
How can I run search operations on the array below such as checking the id's or count? This array is stored in a player's save document.
"inventory": {
"0": {
"count": 1,
"id": 6
},
"1": {
"count": 1,
"id": 13
},
"2": {
"count": 1,
"id": 142
},
"3": {
"count": 1,
"id": 144
}
},
There is no ARRAY in the object you have posted.
if you want search id 13 is present in the document and get the corresponding count you can use OBJECT_PAIRS() function which convert dynamic object into ARRAY described https://docs.couchbase.com/server/current/n1ql/n1ql-language-reference/objectfun.html
SELECT op.val.id, op.val.count, op.name AS pos
FROM default AS d
UNNEST OBJECT_PAIRS(d.inventory) AS op
WHERE op.val.id = 13
OR
SELECT d.*
FROM default AS d
WHERE ANY op IN OBJECT_PAIRS(d.inventory) SATISFIES op.val.id = 13 END;

Postgres - Convert adjacency list to nested JSON object

I have a table with this data in Postgres and I am having a hard time to convert this in to a JSON object.
node_id parent_node name
------- ----------- ----
1 node1
2 1 node2
3 1 node3
4 2 node4
5 2 node5
6 2 node6
7 3 node7
8 3 node8
How do I convert it like this?
{
name: 'node1'
childs: [
{
name: 'node2',
childs: [
{
name: 'node4',
childs: []
},
{
name: 'node5',
childs: []
},
{
name: 'node6',
childs: []
}
]
},
...
]
}
Any suggestion will help. Thanks
Using WITH RECURSIVE (https://www.postgresql.org/docs/current/static/queries-with.html) and JSON Functions (https://www.postgresql.org/docs/current/static/functions-json.html) I build this solution:
db<>fiddle
The core functionality:
WITH RECURSIVE tree(node_id, ancestor, child, path, json) AS (
SELECT
t1.node_id,
NULL::int,
t2.node_id,
'{children}'::text[] ||
(row_number() OVER (PARTITION BY t1.node_id ORDER BY t2.node_id) - 1)::text,-- C
jsonb_build_object('name', t2.name, 'children', array_to_json(ARRAY[]::int[])) -- B
FROM test t1
LEFT JOIN test t2 ON t1.node_id = t2.parent_node -- A
WHERE t1.parent_node IS NULL
UNION
SELECT
t1.node_id,
t1.parent_node,
t2.node_id,
tree.path || '{children}' || (row_number() OVER (PARTITION BY t1.node_id ORDER BY t2.node_id) - 1)::text,
jsonb_build_object('name', t2.name, 'children', array_to_json(ARRAY[]::int[]))
FROM test t1
LEFT JOIN test t2 ON t1.node_id = t2.parent_node
INNER JOIN tree ON (t1.node_id = tree.child)
WHERE t1.parent_node = tree.node_id -- D
)
SELECT -- E
child as node_id, path, json
FROM tree
WHERE child IS NOT NULL ORDER BY path
Every WITH RECURSIVE contains a start SELECT and a recursion part (the second SELECT) combined by a UNION.
A: Joining the table agains itself for finding the children of a node_id.
B: Building the json object for the child which can be inserted into its parent
C: Building the path where the child object has to be inserted (from root). The window function row_number() (https://www.postgresql.org/docs/current/static/tutorial-window.html) generates the index of the child within the children array of the parent.
D: The recursion part works as the initial part with one difference: It's not searching for the root element but for the element which has the parent node of the last recursion.
E: Executing the recursion and filtering all elements without any children gives this result:
node_id path json
2 children,0 {"name": "node2", "children": []}
4 children,0,children,0 {"name": "node4", "children": []}
5 children,0,children,1 {"name": "node5", "children": []}
6 children,0,children,2 {"name": "node6", "children": []}
3 children,1 {"name": "node3", "children": []}
7 children,1,children,0 {"name": "node7", "children": []}
8 children,1,children,1 {"name": "node8", "children": []}
Though I found no way to add all children elements in the recursion (the origin json is no global variable; so it always knows the changes of the direct ancestors, not their siblings), I had to iterate the rows in a seconds step.
That's why I build the function. In there I can do the iteration for a global variable. With the function jsonb_insert I am inserting all calculated elements into a root json object - using the calculated path.
CREATE OR REPLACE FUNCTION json_tree() RETURNS jsonb AS $$
DECLARE
_json_output jsonb;
_temprow record;
BEGIN
SELECT
jsonb_build_object('name', name, 'children', array_to_json(ARRAY[]::int[]))
INTO _json_output
FROM test
WHERE parent_node IS NULL;
FOR _temprow IN
/* Query above */
LOOP
SELECT jsonb_insert(_json_output, _temprow.path, _temprow.json) INTO _json_output;
END LOOP;
RETURN _json_output;
END;
$$ LANGUAGE plpgsql;
Last step is calling the function and make the JSON more readable (jsonb_pretty())
{
"name": "node1",
"children": [{
"name": "node2",
"children": [{
"name": "node4",
"children": []
},
{
"name": "node5",
"children": []
},
{
"name": "node6",
"children": []
}]
},
{
"name": "node3",
"children": [{
"name": "node7",
"children": []
},
{
"name": "node8",
"children": []
}]
}]
}
I am sure it is possible to optimize the query but for a sketch it works.