I have JSON like below
{
"_type": "Usage",
"bids": [
{
"bid": 0.015,
"clicks": 13,
"endTime": "2016-06-01T00:31:39.000Z",
},
{
"bid": 0.023,
"clicks": 1731,
"endTime": "2016-06-01T03:08:30.000Z",
},
{
"bid": 0.025,
"clicks": 604,
"endTime": null,
}
],
"campaignId": 2229,
}
I want to multiplication of clicks and bid inside of bids But when I run below query its result is null ARRAY_SUM( ( bids[*].clicks )*(bids[*].bid))
select campaignId,ARRAY_MAX(bids[*].startTime) date,
ARRAY_SUM(bids[*].clicks) clicks,ARRAY_SUM((bids[*].clicks)*(bids[*].bid)) total
from Default where _type="DailyCampaignUsage";
But Total coming NULL
You want to multiply individual clicks/bid values for each element of the array. Note that bids[].clicks produces an array of clicks values, and '' is multiplication operator for basic/scalar data types. So, try this:
SELECT campaignId,ARRAY_MAX(bids[*].endTime) date,
ARRAY_SUM(bids[*].clicks) clicks,
ARRAY_SUM(ARRAY x.clicks * x.bid FOR x IN bids END) total
FROM default
WHERE _type="DailyCampaignUsage";
-prasad
Related
I have a JSON column "jobs" that looks like this:
[
{
"id": "1",
"done": "100",
"target": "100",
"startDate": "123123132",
"lastAction": "123123132",
"status": "0"
},
{
"id": "2",
"done": "10",
"target": "20",
"startDate": "2312321",
"lastAction": "2312321",
"status": "1"
}
]
I want to filter the array by object key values. For example: To find all items that have target > done, status != 0 and lastAction is yesterday to get response like this:
[
{
"id": "1",
"done": "19",
"target": "100",
"startDate": "123123132",
"lastAction": "123123132",
"status": "0"
}
]
I know I can extract the data to a JSON_TABLE() to do the filtering but I don't get the original object back(unless I recreate it back) and the solution is not dynamic.
Can this kind of array filtering can really be done in MySQL?
SELECT JSON_PRETTY(JSON_EXTRACT(jobs.jobs, CONCAT('$[', j.rownum-1, ']'))) AS object
FROM jobs
CROSS JOIN JSON_TABLE(
jobs.jobs, '$[*]' COLUMNS(
rownum for ordinality,
done int path '$.done',
target int path '$.target',
status int path '$.status'
)
) as j
WHERE j.target > j.done AND j.status != 0;
You also mentioned a condition on lastAction, but the example values you gave are not valid dates, so I'll leave that enhancement to you. The example above demonstrates the technique.
Yes it is possible to do it using the JSON_EXTRACT and JSON_SEARCH functions.
Let's say your table is named tbl_Jobs and the jobs column is of type JSON.
SELECT * FROM tbl_Jobs
WHERE JSON_EXTRACT(jobs, "$[*].target") = JSON_EXTRACT(jobs, "$[*].done")
AND JSON_EXTRACT(jobs, "$[*].status") != 0
AND JSON_SEARCH(jobs, 'one', DATE_SUB(CURDATE(), INTERVAL 1 DAY), NULL, "$[*].lastAction") IS NOT NULL
I'm trying read data from a json document stored on azure data lake storage from an Azure SQL database using the below query. I tried a couple of ways and nothing seems to bring the data back. I tried using this '$.data[0].AccID' in OPENJSON for instance and brings back only the first record from multiple arrays but, not sure how to bring back all the data.
json:
[
{
"data": [
{
"AccID": 1234,
"CustID": 456,
"Total": 1234.1234,
"OrderDate": "2022-12-01"
},
{
"AccID": 5678,
"CustID": 890,
"Total": 5678.5678,
"OrderDate": "2022-12-01"
}
],
"count": 2
},
{
"data": [
{
"AccID": 1234,
"CustID": 456,
"Total": 100.0,
"OrderDate": "2021-12-01"
},
{
"AccID": 5678,
"CustID": 890,
"Total": 200.0,
"OrderDate": "2021-12-01"
},
{
"AccID": 8900,
"CustID": 235,
"Total": 300.0,
"OrderDate": "2021-12-01"
}
],
"count": 3
}
]
Query:
SELECT *
FROM OPENROWSET (
BULK 'blobpath/file.json',
DATA_SOURCE = 'adls',
SINGLE_CLOB
) AS [data]
CROSS APPLY OPENJSON (X.BulkColumn, '$.value')
WITH (
AccID int,
CustID int,
Total float,
OrderDate date)
I created storage account and uploaded Json file into container,
reference image.
input .json:
[
{
"data": [
{
"AccID": 1234,
"CustID": 456,
"Total": 1234.1234,
"OrderDate": "2022-12-01"
},
{
"AccID": 5678,
"CustID": 890,
"Total": 5678.5678,
"OrderDate": "2022-12-01"
}
],
"count": 2
},
{
"data": [
{
"AccID": 1234,
"CustID": 456,
"Total": 100.0,
"OrderDate": "2021-12-01"
},
{
"AccID": 5678,
"CustID": 890,
"Total": 200.0,
"OrderDate": "2021-12-01"
},
{
"AccID": 8900,
"CustID": 235,
"Total": 300.0,
"OrderDate": "2021-12-01"
}
],
"count": 3
}
]
I generated sas token and created masterkey and data source. I created a table in sql with following columns
Create table data1( ACCID varchar(100),
CustID varchar(100),
Total float(100),
OrderDate date,
count int )
Insert data into that table using below code:
INSERT INTO data1
SELECT ACCID,CustID,Total,OrderDate,count FROM OPENROWSET(
BULK 'jsonfile path',
DATA_SOURCE = 'your data source'
SINGLE_CLOB
) AS DataFile
cross apply openjson(BulkColumn)
WITH (
AccID varchar(100) '$.data[0].AccID',
CustID varchar(100) '$.data[0].CustID',
Total float '$.data[0].Total',
OrderDate date '$.data[0].OrderDate',
count int '$.count'
)
INSERT INTO data1
SELECT ACCID,CustID,Total,OrderDate,count FROM OPENROWSET(
BULK 'jsonfile path',
DATA_SOURCE = 'your data source'
SINGLE_CLOB
) AS DataFile
cross apply openjson(BulkColumn)
WITH (
AccID varchar(100) '$.data[1].AccID',
CustID varchar(100) '$.data[1].CustID',
Total float '$.data[1].Total',
OrderDate date '$.data[1].OrderDate',
count int '$.count'
)
INSERT INTO data1
SELECT ACCID,CustID,Total,OrderDate,count FROM OPENROWSET(
BULK 'jsonfile path',
DATA_SOURCE = 'your data source'
SINGLE_CLOB
) AS DataFile
cross apply openjson(BulkColumn)
WITH (
AccID varchar(100) '$.data[2].AccID',
CustID varchar(100) '$.data[2].CustID',
Total float '$.data[2].Total',
OrderDate date '$.data[2].OrderDate',
count int '$.count'
)
Data is inserted successfully.
I retrieved the data of table I got as below
I deleted the Null values row using below code
Image for reference:
delete from data1 where ACCID is Null
I retrieved the data of table order by count according to the json using below code
select * from data1 order by count
Output:
In this way I retrieve all the data from Json document.
I have documents in a bucket called blocks in the following format:
{
"random_field": 1,
"transactions": [{
"id": "CCCCC",
"inputs": [{
"tx_id": "AAAAA",
"index": 0
},{
"tx_id": "BBBBB",
"index": 1
}]
}]
}
{
"transactions": [{
"id": "AAAAA",
"outputs": [{
"field1": "value123",
"field2": "value456"
},{
"field1": "ignore",
"field2": "ignore"
}]
}]
}
{
"transactions": [{
"id": "BBBBB",
"outputs": [{
"field1": "ignored",
"field2": "ignored"
},{
"field1": "value999",
"field2": "value888"
}]
}]
}
and I need to map the inputs from the first document to the corresponding outputs of the second and third documents. The way to do it manually is to, for each input, find a transaction with id equal to the input's tx_id, and then get the item from the outputs array based on the index of the input. To exemplify, this is the object I would like to return in this scenario:
{
"random_field": 1,
"transactions": [{
"id": "CCCCC",
"inputs": [{
"tx_id": "AAAAA",
"index": 0,
"output": {
"field1": "value123",
"field2": "value456"
}
},{
"tx_id": "BBBBB",
"index": 1,
"output": {
"field1": "value999",
"field2": "value888"
}
}]
}]
}
I managed to come up with the following query:
SELECT b.random_field,
b.transactions -- how to map this?
FROM blocks b
UNNEST b.transactions t
UNNEST t.inputs input
JOIN blocks `source` ON (ANY tx IN `source`.transactions SATISFIES tx.`id` = input.tx_id END)
UNNEST `source`.transactions source_tx
UNNEST source_tx.outputs o
WHERE (ANY tx IN b.transactions SATISFIES tx.`id` = 'AAAAA' END) LIMIT 1;
I suppose there should be a way to map b.transactions.inputs by using source_tx.outputs, but I couldn't find how.
I came across this other answer, but I don't really understand how it applies to my scenario. Maybe it does, but I am very new to Couchbase, so I am very much lost: How to map array values in one document to another and display in result
Basically you want inline some other document into current document using condition.
Instead of JOINs+ GROUPS use subquery expressions + correlated subqueries. (b.*, "abc" AS transactions, selects all fields of b and adds transactions (if already exist overwrite else adds)
CREATE INDEX ix1 ON blocks (ALL ARRAY FOR ot.id FOR ot IN transactions END);
SELECT b.*,
(SELECT t.*,
(SELECT i.*,
(SELECT RAW ot
FROM blocks AS o
UNNEST o.transactions AS ot
UNNEST ot.outputs AS oto
WHERE i.tx_id = ot.id AND i.`index` = UNNEST_POS(oto))[0] AS output
FROM t.`inputs` AS i) AS inputs
FROM b.transactions AS t) AS transactions
FROM blocks AS b
WHERE ANY tx IN b.transactions SATISFIES tx.`inputs` IS NOT NULL END ;
OR
SELECT b.*,
(SELECT t.*,
(SELECT i.*,
(SELECT RAW ot.outputs[i.`index`]
FROM blocks AS o
UNNEST o.transactions AS ot
WHERE i.tx_id = ot.id
LIMIT 1)[0] AS output
FROM t.`inputs` AS i) AS inputs
FROM b.transactions AS t) AS transactions
FROM blocks AS b
WHERE ANY tx IN b.transactions SATISFIES tx.`inputs` IS NOT NULL END ;
SELECT JSON_query([json], '$') from mytable
Returns fine the contents of [json] field
SELECT JSON_query([json], '$.Guid') from mytable
Returns null
SELECT JSON_query([json], '$.Guid[1]') from mytable
Returns null
I've also now tried:
SELECT JSON_query([json], '$[1].Guid')
SELECT JSON_query([json], '$[2].Guid')
SELECT JSON_query([json], '$[3].Guid')
SELECT JSON_query([json], '$[4].Guid')
and they all return null
So I'm stuck as to figuring out how create the path to get to the info. Maybe SQL Server json_query can't handle the null as the first array?
Below is the string that is stored inside of the [json] field in the database.
[
null,
{
"Round": 1,
"Guid": "15f4fe9d-403c-4820-8e35-8a8c8d78c33b",
"Team": "2",
"PlayerNumber": "78"
},
{
"Round": 1,
"Guid": "8e91596b-cc33-4ce7-bfc0-ac3d1dc5eb67",
"Team": "2",
"PlayerNumber": "54"
},
{
"Round": 1,
"Guid": "f53cd74b-ed5f-47b3-aab5-2f3790f3cd34",
"Team": "1",
"PlayerNumber": "23"
},
{
"Round": 1,
"Guid": "30297678-f2cf-4b95-a789-a25947a4d4e6",
"Team": "1",
"PlayerNumber": "11"
}
]
You need to follow the comments below your question. I'll just summarize them:
Probably the most appropriate approach in your case is to use OPENJSON() with explicit schema (the WITH clause).
JSON_QUERY() extracts a JSON object or a JSON array from a JSON string and returns NULL. If the path points to a scalar JSON value, the function returns NULL in lax mode and an error in strictmode. The stored JSON doesn't have a $.Guid key, so NULL is the actual result from the SELECT JSON_query([json], '$.Guid') FROM mytable statement.
The following statements provide a working solution to your problem:
Table:
SELECT *
INTO Data
FROM (VALUES
(N'[
null,
{
"Round": 1,
"Guid": "15f4fe9d-403c-4820-8e35-8a8c8d78c33b",
"Team": "2",
"PlayerNumber": "78",
"TheProblem": "doesn''t"
},
{
"Round": 1,
"Guid": "8e91596b-cc33-4ce7-bfc0-ac3d1dc5eb67",
"Team": "2",
"PlayerNumber": "54"
},
{
"Round": 1,
"Guid": "f53cd74b-ed5f-47b3-aab5-2f3790f3cd34",
"Team": "1",
"PlayerNumber": "23"
},
{
"Round": 1,
"Guid": "30297678-f2cf-4b95-a789-a25947a4d4e6",
"Team": "1",
"PlayerNumber": "11"
}
]')
) v (Json)
Statements:
SELECT j.Guid
FROM Data d
OUTER APPLY OPENJSON(d.Json) WITH (
Guid uniqueidentifier '$.Guid',
Round int '$.Round',
Team nvarchar(1) '$.Team',
PlayerNumber nvarchar(2) '$.PlayerNumber'
) j
SELECT JSON_VALUE(j.[value], '$.Guid')
FROM Data d
OUTER APPLY OPENJSON(d.Json) j
Result:
Guid
------------------------------------
15f4fe9d-403c-4820-8e35-8a8c8d78c33b
8e91596b-cc33-4ce7-bfc0-ac3d1dc5eb67
f53cd74b-ed5f-47b3-aab5-2f3790f3cd34
30297678-f2cf-4b95-a789-a25947a4d4e6
How to get aggregate SUM(amount) from "refunds" array in postgres json select
Following is my data schema and structure:
Table Name: transactions
Column name: data
{
"id": "tran_6ac25129951962e99f28fa488993",
"amount": 1200,
"origin_amount": 3900,
"status": "partial_refunded",
"description": "Subscription#sub_a67d59efb2bcbf73485a ",
"livemode": false,
"refunds": [
{
"id": "refund_ee4192ffb6d2caa490a1",
"amount": 1200,
"status": "refunded",
"created_at": 1426412340,
"updated_at": 1426412340,
},
{
"id": "refund_0e4a34e4ee7281d369df",
"amount": 1500,
"status": "refunded",
"created_at": 1426412353,
"updated_at": 1426412353,
}
]
}
Out put should be: 1200+1500 = 2700
Output
|---------
|total
|---------
|2700
Please provide global solution and not with static data
This should work on 9.3+
WITH x AS( SELECT
'{
"id": "tran_6ac25129951962e99f28fa488993",
"amount": 1200,
"origin_amount": 3900,
"status": "partial_refunded",
"description": "Subscription#sub_a67d59efb2bcbf73485a ",
"livemode": false,
"refunds": [
{
"id": "refund_ee4192ffb6d2caa490a1",
"amount": 1200,
"status": "refunded",
"created_at": 1426412340,
"updated_at": 1426412340
},
{
"id": "refund_0e4a34e4ee7281d369df",
"amount": 1500,
"status": "refunded",
"created_at": 1426412353,
"updated_at": 1426412353
}
]
}'::json as y),
refunds AS(
SELECT json_array_elements(y->'refunds') as j FROM x)
SELECT sum((j->>'amount')::int) FROM refunds;
WITH AllRefunds AS ( SELECT jsonb_array_elements(data->'refunds') AS refund FROM transactions)
SELECT SUM( CAST ( refund ->> 'amount' AS INTEGER )) FROM AllRefunds;
If you need to know how the query is built:
1.
WITH AllRefunds AS ( SELECT jsonb_array_elements(data->'refunds') FROM transactions)
SELECT * FROM AllRefunds;
This selects all elements as JSON objects (done via ->) from the array refunds that were found in transactions table and stores it in a new table AllRefunds. This new table only consists of one unnamed column.
2.
WITH AllRefunds AS ( SELECT jsonb_array_elements(data->'refunds') AS refund FROM transactions)
SELECT * FROM AllRefunds;
Here the added (second) AS renames the currently unnamed column inside AllRefunds to refund
3.
WITH AllRefunds AS ( SELECT jsonb_array_elements(data->'refunds') AS refund FROM transactions)
SELECT SUM( CAST ( refund ->> 'amount' AS INTEGER )) FROM AllRefunds;
Our array entries are JSON objects. So we return the field amount as a simple string with ->> that we then cast to Integers and SUM all entries up.