reading JSON values .. in SQL 2012 - json

I have a SQL Server 2012 table with a column as per:
ResponseJSON varchar(max)
It contains text values like
{
"partNumber": 1,
"partTotal": 1,
"payeeLocationId": "ABC00011",
"remittanceAdviceId": "77592",
"paymentInfo": {
"accountInfo": {
"accountName": "ABC Hospital",
"xyzNumber": "",
"xyzCode": ""
},
"depositAmount": "1234",
"paymentReference": "ERA 1234"
},
"paymentRun": {
"payerName": "ABC",
"runDate": "2022-12-05"
},
"claimSummary": [
{
"benefit": "5555",
"channelCode": "ABC",
"claimId": "1234",
"lodgementDate": "2022-02-14",
"transactionId": "xpxpxpxpxxp",
"accountReferenceId": "12345678"
}
]
}
I wondered how to read the remittanceAdviceId value of 77592 (in this case) out of this JSON column data ..
The remittanceAdviceId may be varying size in length .. e.g. 1,2,3,4,5,6,7 etc digits
I considered something like :
SELECT remittanceAdviceId = CASE
WHEN E.RequestJSON IS NOT NULL AND
CHARINDEX('"remittanceAdviceId"', E.RequestJSON, 0) > 0 THEN
SUBSTRING(E.RequestJSON,
CHARINDEX('"remittanceAdviceId"', E.RequestJSON, 0) + 22,
5)
ELSE
NULL
END
but this isn't quite right as value may be other than 5 digits ..

Assuming upgrading to the latest version of SQL isn't in the cards right now, here's a simple approach using SUBSTRING and CHARINDEX:
DECLARE #json varchar(2000) = '{
"partNumber": 1,
"partTotal": 1,
"payeeLocationId": "ABC00011",
"remittanceAdviceId": "77592",
"paymentInfo": {
"accountInfo": {
"accountName": "ABC Hospital",
"xyzNumber": "",
"xyzCode": ""
},
"depositAmount": "1234",
"paymentReference": "ERA 1234"
},
"paymentRun": {
"payerName": "ABC",
"runDate": "2022-12-05"
},
"claimSummary": [
{
"benefit": "5555",
"channelCode": "ABC",
"claimId": "1234",
"lodgementDate": "2022-02-14",
"transactionId": "xpxpxpxpxxp",
"accountReferenceId": "12345678"
}
]
}';
SELECT SUBSTRING (
#json
, CHARINDEX ( '"remittanceAdviceId": "', #json, 0 ) + 23
, CHARINDEX ( '",', #json, CHARINDEX ( '"remittanceAdviceId": "', #json, 0 ) ) - CHARINDEX ( '"remittanceAdviceId": "', #json, 0 ) - 23
) AS remittanceAdviceId;
RETURNS
+--------------------+
| remittanceAdviceId |
+--------------------+
| 77592 |
+--------------------+
NOTES
Assumes valid JSON with quoted values.
There is no need to specify a length for the remittance id. It
will get parsed accordingly.
UPDATE
Now that you know you can use the native JSON feature in SQL, the simplest way to extract a single value from JSON is:
SELECT JSON_VALUE ( #json, '$.remittanceAdviceId' ) AS remittanceAdviceId;
RETURNS
+--------------------+
| remittanceAdviceId |
+--------------------+
| 77592 |
+--------------------+

Related

Azure Synapse Analytics Json Flatten

I'm new to Azure Synapse and currently have the following problem:
I get a json that looks like the following:
{
"2022-02-01":[
{
"shiftId": ,
"employeeId": ,
"duration": ""
},
{
"shiftId": ,
"employeeId": ,
"duration": ""
}
],
"2022-02-02": [
{
"shiftId": ,
"employeeId": ,
"duration": ""
}
],
"2022-02-03": [
{
"shiftId": ,
"employeeId": ,
"duration": ""
},
{
"shiftId": ,
"employeeId": ,
"duration": ""
}
],
"2022-02-4": []
}
Now I would like to convert this so that I get it in a view. I have already tried with a dataflow as array of documents but I get an error.
"Malformed records are detected in schema inference. Parse Mode: FAILFAST"
I want something like:
date shiftId employeeId duration
___________|_________|____________|_________
2022-02-01 | 1234 | 345345 | 420
2022-02-01 | 2345 | 345345 | 124
2022-02-02 | 5345 | 123567 | 424
2022-02-03 | 5675 | 987542 | 123
2022-02-03 | 9456 | 234466 | 754
Azure Synapse Analytics, dedicated SQL pools are actually very capable with JSON, supporting OPENJSON and JSON_VALUE, so you could just use a Stored Procedure with the JSON as a parameter. A simple exmaple:
SELECT
k.[key] AS [shiftDate],
JSON_VALUE( d.[value], '$.shiftId' ) shiftId,
JSON_VALUE( d.[value], '$.employeeId' ) employeeId,
JSON_VALUE( d.[value], '$.duration' ) duration
FROM OPENJSON( #json, '$' ) k
CROSS APPLY OPENJSON( k.value, '$' ) d;
The full code:
DECLARE #json NVARCHAR(MAX) = '{
"2022-02-01": [
{
"shiftId": 1234,
"employeeId": 345345,
"duration": 420
},
{
"shiftId": 2345,
"employeeId": 345345,
"duration": 124
}
],
"2022-02-02": [
{
"shiftId": 5345,
"employeeId": 123567,
"duration": 424
}
],
"2022-02-03": [
{
"shiftId": 5675,
"employeeId": 987542,
"duration": 123
},
{
"shiftId": 9456,
"employeeId": 234466,
"duration": 754
}
]
}'
SELECT
k.[key] AS [shiftDate],
JSON_VALUE( d.[value], '$.shiftId' ) shiftId,
JSON_VALUE( d.[value], '$.employeeId' ) employeeId,
JSON_VALUE( d.[value], '$.duration' ) duration
FROM OPENJSON( #json, '$' ) k
CROSS APPLY OPENJSON( k.value, '$' ) d;
My results:
You could use a Synapse Notebook or Mapping Data Flows if you wanted something more dynamic.

Update a nested value from a Json field

Consider this table:
DROP TABLE IF EXISTS `example`;
CREATE TABLE `example` (
`id` int NOT NULL AUTO_INCREMENT,
`content` json NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3;
And these rows:
INSERT INTO example(content)
VALUES (
'[ { "date": "1617210148", "name": "John", "status": "0" },
{ "date": "1617210148", "name": "Jack", "status": "0" },
{ "date": "1617210148", "name": "Henry", "status": "0" }]'
);
I'd like to update the value of the key status where name = Jack to 1
The result would be:
{ "date": "1617210148", "name": "Jack", "status": "1" }
How can I do this using JSON_REPLACE() or JSON_SET() in a SQL query (I'm aiming for a partial update since I'm using MySQL 8.0.25)?
This is very awkward, nearly impossible with MySQL's JSON functions.
You can use JSON_REPLACE() or JSON_SET(), but both require that you know the path to the field you want to change. So in this case, we can see that the array element is $[1] but if you didn't know that, you couldn't use this solution.
mysql> select json_pretty(json_replace(content, '$[1].status', '1')) as j
from example\G
*************************** 1. row ***************************
j: [
{
"date": "1617210148",
"name": "John",
"status": "0"
},
{
"date": "1617210148",
"name": "Jack",
"status": "1"
},
{
"date": "1617210148",
"name": "Henry",
"status": "0"
}
]
The question like yours has come up before on Stack Overflow, for example JSON update single value in MySQL table. The solution in that case depends on you knowing which array element your pseudo-record exists in.
You can get the path to a JSON element with JSON_SEARCH(), but you can only search by value, not by a key/value pair. If "Jack" occurs in some other field, that would also be found.
mysql> select json_unquote(json_search(content, 'one', 'Jack')) as path from example;
+-----------+
| path |
+-----------+
| $[1].name |
+-----------+
To search for a key/value pair, you need to use JSON_TABLE() and that requires you upgrade to MySQL 8.0. And that doesn't tell you the path to the element, it only allows you to return a specific row out of the array.
mysql> select j.* from example cross join json_table(content, '$[*]' columns(
date int unsigned path '$.date',
name varchar(10) path '$.name',
status int path '$.status')
) as j where name = 'Jack';
+------------+------+--------+
| date | name | status |
+------------+------+--------+
| 1617210148 | Jack | 0 |
+------------+------+--------+
Here's a trick: You can extract the name field, and that turns into an array of those values:
mysql> select json_extract(content, '$[*].name') as a from example;
+---------------------------+
| a |
+---------------------------+
| ["John", "Jack", "Henry"] |
+---------------------------+
Then you can search that array to get the array position:
mysql> select json_search(json_extract(content, '$[*].name'), 'one', 'Jack') as root from example;
+--------+
| root |
+--------+
| "$[1]" |
+--------+
Some unquoting and adding .status and you can get the full path to the field you want to update:
mysql> select concat(json_unquote(json_search(json_extract(content, '$[*].name'), 'one', 'Jack')), '.status') as path from example;
+-------------+
| path |
+-------------+
| $[1].status |
+-------------+
Now use it in a JSON_SET() call:
mysql> select json_pretty(
json_set(content,
concat(json_unquote(json_search(json_extract(content, '$[*].name'), 'one', 'Jack')), '.status'),
'1')) as newcontent
from example\G
*************************** 1. row ***************************
newcontent: [
{
"date": "1617210148",
"name": "John",
"status": "0"
},
{
"date": "1617210148",
"name": "Jack",
"status": "1"
},
{
"date": "1617210148",
"name": "Henry",
"status": "0"
}
]
Using this in an UPDATE looks like this:
mysql> update example set content = json_set(content, concat(json_unquote(json_search(json_extract(content, '$[*].name'), 'one', 'Jack')), '.status'), '1');
That's a long way to go. Now compare how difficult that is to:
UPDATE content SET status = 1 WHERE name = 'Jack';
Storing data in a JSON document when you eventually want to use SQL expressions to search or update individual fields within the JSON document is a costly mistake. It increases the complexity of any code you write to do it, and the developer who needs to take over maintenance of your code after you have moved on to another project will curse your name.

Parse Nested JSON into SQL Table

I am trying to get a JSON file parsed into a usable format so I can insert it into a SQL table.
The JSON file I have is heavily nested (and I can't get the vendor to change it at this point), and uses the same name at different levels.
I have used the following code, to start off, but it is the multi sections and potentially multiple accounts etc that has me stumped. I know I will probably need to iterate through somehow, but just not sure where to begin.
DECLARE #JSON VARCHAR(MAX)
SELECT #JSON = BulkColumn
FROM OPENROWSET
(BULK 'C:\Users\joshu\Downloads\Cashflow.JSON', SINGLE_CLOB)
AS j
If (ISJSON(#JSON)=1)
Select * FROM OPENJSON (#JSON,'$.data')
with
(
[id] nvarchar(50),
[title] nvarchar(50),
[sections] nvarchar(max) as json
) data_Structure
cross apply openjson(data_structure.sections, '$')
with (
[income] nvarchar(max) as json
) data2
--Income is one type, there should be a loop here
cross apply openjson(data2.income, '$')
Which is getting my down the tree. The last data points that I want to collect are based on this bit of code
DECLARE #JSON VARCHAR(MAX)
SELECT #JSON = BulkColumn
FROM OPENROWSET
(BULK 'C:\Users\joshu\Downloads\Cashflow.JSON', SINGLE_CLOB)
AS j
If (ISJSON(#JSON)=1)
Select * FROM OPENJSON (#JSON,'$.data.sections.income.sections.tracker_1193.sections.tracker_1193_income.rows."b5cfd1ce-bb7f-4f5c-a6b4-12b469ff0b9d".data."2017-06"')
with
(
[date] nvarchar(50),
[value] decimal(18,2))
A sample of the JSON is here
"data": {
"id": "cashflow",
"title": "Cashflow Report",
"sections": {
"income": {
"id": "income",
"title": "Income",
"sections": {
"tracker_1193": {
"id": "tracker_1193",
"title": "xxxxxxx",
"sections": {
"tracker_1193_income": {
"id": "tracker_1193_income",
"title": "Income",
"sections": null,
"rows": {
"b5cfd1ce-bb7f-4f5c-a6b4-12b469ff0b9d": {
"account_id": "b5cfd1ce-bb7f-4f5c-a6b4-12b469ff0b9d",
"account_name": "Bobby Calf Sales",
"data": {
"2017-06": {
"date": "2017-06",
"value": 0
},
"2017-09": {
"date": "2017-09",
"value": 4801.36
},
"2017-12": {
"date": "2017-12",
"value": 1997.33
Now the fun part.
The income section is the most complicated;
The First "section" in the JSON data is one of 13.
The Second "section" is variable, as in there could be 1, or 12.
The Third "section is fixed to 3 (income, costs, gross-profit)
The rest is more straight forward
{
"data": {
"id": "cashflow",
"title": "Cashflow Report",
"sections": {
//other data ahead of this
"operating_expenses": {
"id": "operating_expenses",
"title": "Operating Expenses",
"sections": {
"operating_expenses_animal_health_animal": {
"id": "operating_expenses_animal_health_animal",
"title": "Animal Health",
"sections": null,
"rows": {
"0de82545-be93-4fb5-9d20-fa076af48e40": {
"account_id": "0de82545-be93-4fb5-9d20-fa076af48e40",
"account_name": "Animal Health - Minerals",
"data": {
"2019-07": {
"date": "2019-07",
"value": 5827.93
}
}
},
"9ba329a6-f77e-4779-9d79-28dd20465b9c": {
"account_id": "9ba329a6-f77e-4779-9d79-28dd20465b9c",
"account_name": "Animal Health - Other",
"data": {
"2019-07": {
"date": "2019-07",
"value": 663.73
}
}
},
"4f406965-3355-4968-a5ba-519d9706f329": {
"account_id": "4f406965-3355-4968-a5ba-519d9706f329",
"account_name": "Animal Health - Treatments",
"data": {
"2019-07": {
"date": "2019-07",
"value": 8670.1
}
}
},
"79c8ab89-22a2-4c5c-b591-0a3d95a4a95b": {
"account_id": "79c8ab89-22a2-4c5c-b591-0a3d95a4a95b",
"account_name": "Animal Health - Vet",
"data": {
"2019-07": {
"date": "2019-07",
"value": 7645.18
}
}
}
},
"totals": {
"2019-07": {
"date": "2019-07",
"value": 22806.94
}
}
},
Because of the nature of this data, I haven't sorted my SQL table structure yet, but I am imagining it to be something along the lines of the below:
That's about it. I am stuck, need some help/guidance so anything you can do to assist is greatly appreciated
I've made an attempt with your JSON to traverse its data without having to define explicit keys. This only handles the "Income" portion, however, it should get you moving in the right direction to extract your data into SQL server. Note that given "rows" can have multiple values, some data is duplicated.
DECLARE #data nvarchar(MAX) = '{"data":{"id":"cashflow","title":"Cashflow Report","sections":{"income":{"id":"income","title":"Income","sections":{"tracker_1193":{"id":"tracker_1193","title":"xxxxxxx","sections":{"tracker_1193_income":{"id":"tracker_1193_income","title":"Income","sections":null,"rows":{"b5cfd1ce-bb7f-4f5c-a6b4-12b469ff0b9d":{"account_id":"b5cfd1ce-bb7f-4f5c-a6b4-12b469ff0b9d","account_name":"Bobby Calf Sales","data":{"2017-06":{"date":"2017-06","value":0},"2017-09":{"date":"2017-09","value":4801.36},"2017-12":{"date":"2017-12","value":1997.33}}}}}}}}}}}}';
SELECT
section_id,
section_title,
tracker_id,
tracker_title,
string_id,
string_title,
account_id,
account_title,
[period],
period_value
FROM OPENJSON ( #data, '$.data.sections.income' )
WITH (
section_id varchar(50) '$.id',
section_title varchar(50) '$.title',
sections nvarchar(max) '$.sections' AS JSON
) AS dat
OUTER APPLY (
SELECT * FROM OPENJSON ( dat.sections ) AS a
CROSS APPLY (
SELECT * FROM OPENJSON ( a.value )
WITH (
tracker_id varchar(50) '$.id',
tracker_title varchar(50) '$.title',
tracker_income nvarchar(max) '$.sections' AS json
)
) AS b
CROSS APPLY (
SELECT [key] AS income_key, [value] AS income_value FROM OPENJSON ( b.tracker_income )
) AS c
CROSS APPLY (
SELECT * FROM OPENJSON ( c.income_value )
WITH (
string_id varchar(50) '$.id',
string_title varchar(50) '$.title',
income_rows nvarchar(max) '$.rows' AS json
)
) AS d
CROSS APPLY (
SELECT [key] AS account_key, [value] AS account_value FROM OPENJSON ( d.income_rows )
) e
CROSS APPLY (
SELECT * FROM OPENJSON ( e.account_value )
WITH (
account_id varchar(255) '$.account_id',
account_title varchar(50) '$.account_name',
account_data nvarchar(max) '$.data' AS json
)
) f
CROSS APPLY (
SELECT [key] AS [period], JSON_VALUE ( [value], '$.value' ) AS period_value FROM OPENJSON ( f.account_data )
) g
) AS Income;
Returns
+------------+---------------+--------------+---------------+---------------------+--------------+--------------------------------------+------------------+---------+--------------+
| section_id | section_title | tracker_id | tracker_title | string_id | string_title | account_id | account_title | period | period_value |
+------------+---------------+--------------+---------------+---------------------+--------------+--------------------------------------+------------------+---------+--------------+
| income | Income | tracker_1193 | xxxxxxx | tracker_1193_income | Income | b5cfd1ce-bb7f-4f5c-a6b4-12b469ff0b9d | Bobby Calf Sales | 2017-06 | 0 |
| income | Income | tracker_1193 | xxxxxxx | tracker_1193_income | Income | b5cfd1ce-bb7f-4f5c-a6b4-12b469ff0b9d | Bobby Calf Sales | 2017-09 | 4801.36 |
| income | Income | tracker_1193 | xxxxxxx | tracker_1193_income | Income | b5cfd1ce-bb7f-4f5c-a6b4-12b469ff0b9d | Bobby Calf Sales | 2017-12 | 1997.33 |
+------------+---------------+--------------+---------------+---------------------+--------------+--------------------------------------+------------------+---------+--------------+

how to add key-value pair in json root node and convert it into table using SQL server

I have table people and it's maintain Four column which is Name ,TagName ,Value , Location.
I want to convert the tagname and value in json with name and location column as rootnode (Name & location same for multiple records)
Need output as :
{
"{"Name":"EMP1","Location":"mumbai"}": [
{
"TagName": "1",
"Value": "844.17769999999996"
},
{
"TagName": "abc",
"Value": "837.43679999999995"
},
{
"TagName": "pqr",
"Value": "0"
},
{
"TagName": "XYZ",
"Value": "1049.2429999999999"
}
]
}
please check the below query, In which I am trying to create json string using json path but stuck in root node.
SELECT TagName
,Value
FROM dbo.people
FOR JSON PATH, ROOT('')---
when I convert the above json into tabular format, required output as :
Name | Location |TagName| Value
EMP1 | Mumbai |1 | 844.17769999999996|
EMP1 | Mumbai |abc | 837.43679999999995|
.....
Your expected output is not a valid JSON, but you are probably looking for something like this:
Table:
CREATE TABLE People (
[Name] varchar(10),
[Location] varchar(50),
[TagName] varchar(3),
[Value] numeric(20, 14)
)
INSERT INTO People ([Name], [Location], [TagName], [Value])
VALUES
('EMP1', 'Mumbai', '1', 844.17769999999996),
('EMP1', 'Mumbai', 'abc', 837.43679999999995),
('EMP2', 'Mumbai', 'abc', 837.43679999999995)
Statement:
SELECT DISTINCT p.[Name], p.[Location], c.Items
FROM People p
CROSS APPLY (
SELECT [TagName], [Value]
FROM People
WHERE [Name] = p.[Name] AND [Location] = p.[Location]
FOR JSON AUTO
) c (Items)
FOR JSON PATH
Result:
[
{
"Name":"EMP1",
"Location":"Mumbai",
"Items":[
{
"TagName":"1",
"Value":844.17769999999996
},
{
"TagName":"abc",
"Value":837.43679999999995
}
]
},
{
"Name":"EMP2",
"Location":"Mumbai",
"Items":[
{
"TagName":"abc",
"Value":837.43679999999995
}
]
}
]
If you want to parse the generated JSON, you need to use OPENJSON() twice:
Generated JSON:
DECLARE #json varchar(max) = N'[
{
"Name":"EMP1",
"Location":"Mumbai",
"Items":[
{
"TagName":"1",
"Value":844.17769999999996
},
{
"TagName":"abc",
"Value":837.43679999999995
}
]
},
{
"Name":"EMP2",
"Location":"Mumbai",
"Items":[
{
"TagName":"abc",
"Value":837.43679999999995
}
]
}
]'
Statement:
SELECT j1.Name, j1.Location, j2.TagName, j2.Value
FROM OPENJSON(#json) WITH (
[Name] varchar(10) '$.Name',
[Location] varchar(50) '$.Location',
[Items] nvarchar(max) '$.Items' AS JSON
) j1
OUTER APPLY OPENJSON(j1.Items) WITH (
[TagName] varchar(3) '$.TagName',
[Value] numeric(20, 14) '$.Value'
) j2

How to query expanded json content in db2

Since version 11, DB2 supports json functions and json value query. I was trying to follow the documentation but could not figure out how to expand a json object into multiple rows. Would like to know if it's possible at all in DB2.
Example:
The JSON object looks like this:
{
"content":{
"key_1": {
"text": "sample",
"label": "l1"
},
"key_2": {
"text": "sample2",
"label": "l2"
},
...
}
}
and the table looks like this:
|id|json|
|--|----|
|1 | '{...}'|
|2 | '{...}'|
I want to query this table:
|id|text|label|
|--|----|-----|
|1 |sample|l1|
|1 |sample2|l2|
...
I tried
SELECT JSON_VALUE(json, 'strict $.content.*.text' RETURNING CHAR(10)) AS text,
JSON_VALUE(json, 'strict $.content.*.label' RETURNING CHAR(10)) AS label
FROM table
but I got NULL for all the rows.
Any insights are appreciated. Thank you!
I belive you need to do something like this
SELECT I, T.*
FROM
TABLE(VALUES('{
"content":{
"key_1": {
"text": "sample",
"label": "l1"
},
"key_2": {
"text": "sample2",
"label": "l2"
}
}
}')) AS D(JSON)
, TABLE(VALUES(1),(2),(3)) AS A(I)
, JSON_TABLE(D.JSON, 'strict $'
COLUMNS(
TEXT VARCHAR(80) PATH '$.content.key_' || i || '.text'
, LABEL VARCHAR(80) PATH '$.content.key_' || i || '.label'
)
ERROR ON ERROR) AS T
WHERE T.TEXT is NOT NULL
which will give you
I|TEXT |LABEL
-|-------|------
1|sample |l1
2|sample2|l2
The A table in the above will need as many rows as you have key_ entries in the JSON