Parsing a JSON to meet minimum requirements inside a stored procedure - json

I have been looking at what I believe to every single page on SQL Server and half stackoverflow, and I can't find a proper solution to this
Our challenge, is to deal with an exiting application that send/receive JSON form SQL Server. So we have to build a STRONG JSON architecture on SQL Server.
We need to validate the format of the JSON (legacy system has its own standard) so messages are in exact expected format.
The thing is, JSON functions are not so advance as XML, and seems there is no way to validate a schema in SQL Server.
We tried with sp_prepare and sp_execute, but that does not seem to work.
We tested something like this:
Declare #ptSQL1 int;
Exec sp_prepare #ptSQL1 output,
N'#P1 nvarchar(128), #json NVARCHAR(1000) ',
N' SELECT *
INTO temp_tblPersons
FROM OPENJSON (#json, ''$.root'')
WITH (
Cname NVARCHAR(100) ''strict$.FirstName'',
Csurname NVARCHAR(100) ''lax$.surname''
) as J
where Csurname like #P1';
DECLARE #json7 NVARCHAR(1000)
SET #json7 = N'{
"root": [
{ "FirstName": "Charles" , "surname":"perez" },
{ "FirstName": "Jade" , "surname":"pelaz" },
{ "FirstName": "Jim" , "surname":"alvarez" },
{ "FirstName": "Luke" , "surname":"alonso" },
{ "FirstName": "Ken"}
]
}'
IF (#ptSQL1 = 0) PRINT 'THE SUPPLY JSON IS NOT VALID'
ELSE Exec sp_execute #ptSQL1, N'a%', #json7;
but does not meet the sp_prepare/execute behavior.
Our intention it to validate a minimum schema before proceed to process the data, and if the schema doesn't meet the standard, return an ERROR.
How can this be accomplished?
(not sure where we read the #ptSQL1 = 0, but I believe to read somewhere)

Our intention it to validate a minimum schema before proceed to
process the data, and if the schema doesn't meet the standard, return
an ERROR.
The JSON must be parsed in order to validate the schema. A prepare doesn't actually execute the query in order to parse the JSON document, plus sp_prepare and sp_execute are internal API system stored procedures not intended to be called directly in T-SQL.
Although one can't currently validate JSON schema in T-SQL (without writing a custom SQLCLR assembly), you could just use TRY/CATCH and handle errors. The example below handles JSON errors differently but I would personally just THROW all errors and handle specific ones in the app code.
DECLARE #json NVARCHAR(1000);
DECLARE #P1 NVARCHAR(128) = 'a%';
SET #json = N'{
"root": [
{ "FirstName": "Charles" , "surname":"perez" },
{ "FirstName": "Jade" , "surname":"pelaz" },
{ "FirstName": "Jim" , "surname":"alvarez" },
{ "FirstName": "Luke" , "surname":"alonso" },
{ "FirstName": "Ken"}
]
}';
BEGIN TRY
SELECT *
INTO temp_tblPersons
FROM OPENJSON (#json, '$.root')
WITH (
Cname NVARCHAR(100) 'strict$.FirstName',
Csurname NVARCHAR(100) 'lax$.surname'
) as J
where Csurname like #P1;
END TRY
BEGIN CATCH
DROP TABLE IF EXISTS temp_tblPersons;
IF ERROR_MESSAGE() LIKE N'%JSON%'
BEGIN
PRINT 'THE SUPPLY JSON IS NOT VALID';
END
ELSE
BEGIN
THROW;
END;
END CATCH;

Related

Using OPENJSON in SQL Server to parse a Non-Array Object

I'm using SQL Server v15, called from a .NET application.
A website I'm using (not mine - I don't control the data) has a JSON dataset formatted strangely. Instead of being an array like:
[{"id":"1","Name":"Charlie"},{"id":"2","Name"="Sally"}]
It's an object with each element named as its ID:
{"1":{"id":"1","Name":"Charlie"}, "2":{"id":"2","Name"="Sally"}}
I know how to use the OPENJSON to read data from an array, but is it possible to have it parse this format? Or is my best bet to have a script loop through the objects one at a time?
Please try the following solution.
SQL
DECLARE #json NVARCHAR(MAX) =
N'{
"1": {
"id": "1",
"Name": "Charlie"
},
"2": {
"id": "2",
"Name": "Sally"
}
}';
SELECT rs.*
FROM OPENJSON (#json) AS seq
CROSS APPLY OPENJSON(seq.value)
WITH
(
[id] INT '$.id'
, [Name] VARCHAR(20) '$.Name'
) AS rs;
Output
id
Name
1
Charlie
2
Sally

The argument 2 of the "JSON_VALUE or JSON_QUERY" must be a string literal

I am passing this JSON to a stored procedure in SQL Server
{
"individual": [
{
"app_id": 1057029,
"size": 2
},
{
"app_id": 1057053,
"size": 3
},
{
"app_id": 1057048,
"size": 1
}
]
}
In the stored procedure I am extracting values of app_id and size as under
SET #len = JSON_VALUE(#json, CONCAT('$.individual[', #i, '].size'));
SET #appId = JSON_VALUE(#json, CONCAT('$.individual[', #i, '].app_id'));
(Here i is index variable incrementing in a loop)
This works perfect on Microsoft SQL Server 2017 (version 14.0.1000.169)
But on Microsoft SQL Server 2016 (version 13.0.4604.0) I am getting error:
JSON_Value error: The argument 2 of the "JSON_VALUE or JSON_QUERY"
must be a string literal
Please note this is not duplicate as I already have referred questions below on SO but still didn't get solution.
JSON_Value error: The argument 2 of the "JSON_VALUE or JSON_QUERY" must be a string literal
SQL Sever 2016 - Inconsistent Behavior - The argument 2 of the "JSON_VALUE or JSON_QUERY" must be a string literal
Update
Why this is not duplicate question?
None of the other questions discusses the issue in clear precise way. They have mentioned specifics instead. Rather in this question, I have mentioned use of variable #i which is precisely causing this error. JSON_VALUE works in MSSQL2016 but it doesn't support variables as second param, that precisely is the problem here. As a workaround, we've to use OPENJSON but again writing OPENJSON query to get record of particular index from json is tricky. None of the answers to other similar questions discusses this clearly. I am going to write an answer in some time demonstrating it all.
This example demonstrates the use of OpenJson for your JSON and desired query. I've never used OpenJson but found the Microsoft documentation more than adequate. In the last SELECT in this example, the app_id and size are columns in a table with each value pair as a row. Now you don't have to loop through an array; you now have a standard table to work with.
DECLARE #json nvarchar(max) = '{
"individual": [
{
"app_id": 1057029,
"size": 2
},
{
"app_id": 1057053,
"size": 3
},
{
"app_id": 1057048,
"size": 1
}
]
}';
SELECT * FROM OpenJson(#json);
SELECT * FROM OpenJson(#json, '$.individual');
SELECT * FROM OPENJSON(#json, '$.individual')
WITH (
app_id int,
size int
) as apps
;
The output:

SQL Server OPENROWSET not pulling JSON Data

I am trying so save the JSON file Data 'WhitelistedOrigins' in a SQL table however I keep receiving NULL Entries.
I have used the same method to import attributes into a table and this has worked before although the JSON was formatted differently
JSON
"Client": {
"whiteListedOrigins": [
"file://",
"https://mobile.gtent.eu",
"https://mobile.assists.co.uk",
"https://valueadds3.active.eu",
"https://flash3.active.eu",
"https://valueadds3.assists.co.uk"
]
}
SQL
DECLARE #JSON VARCHAR(MAX)
SELECT #JSON = BulkColumn
FROM OPENROWSET
(BULK 'C:\config.json', SINGLE_CLOB)
AS A
UPDATE dbo.CommonBaseline
SET CommonBaseline.whiteListedOrigins= whiteListedOrigins
FROM OPENJSON (#JSON, '$.Client')
WITH (
whiteListedOrigins Varchar (MAX))
RESULT
You need to use OPENJSON() with explicit schema and AS JSON option in a column definition.
If you want to return a nested JSON fragment from a JSON property, you
have to provide the AS JSON flag. Without this option, if the property
can't be found, OPENJSON returns a NULL value instead of the
referenced JSON object or array, or it returns a run-time error in
strict mode.
Statement:
DECLARE #Json nvarchar(max) = N'{
"Client": {
"whiteListedOrigins": [
"file://",
"https://mobile.gtent.eu",
"https://mobile.assists.co.uk",
"https://valueadds3.active.eu",
"https://flash3.active.eu",
"https://valueadds3.assists.co.uk"
]
}
}'
SELECT *
FROM OPENJSON (#JSON, '$.Client') WITH (
whiteListedOrigins nvarchar (MAX) AS JSON
)
Output:
--------------------
whiteListedOrigins
--------------------
[
"file://",
"https://mobile.gtent.eu",
"https://mobile.assists.co.uk",
"https://valueadds3.active.eu",
"https://flash3.active.eu",
"https://valueadds3.assists.co.uk"
]
Notes:
Your JSON (without surrounding { and }) is not valid.

Parse unknown JSON path in TSQL with openjson and/or json_value

I have a incoming data structure that looks like this:
declare #json nvarchar(max) = '{
"action": "edit",
"data": {
"2077-09-02": {
"Description": "some stuff",
"EffectDate": "2077-1-1"
}
}
}';
To give you a long story short, I think TSQL hates this json structure, because no matter what I have tried, I can't get to any values other than "action".
The {data} object contains another object, {2077-09-02}. "2077-09-02" will always be different. I can't rely on what that date will be.
This works:
select json_value(#json, '$.action');
None of this works when trying to get to the other values.
select json_value(#json, '$.data'); --returns null
select json_value(#json, '$.data[0]'); --returns null
select json_value(#json, 'lax $.data.[2077-09-02].Description');
--JSON path is not properly formatted. Unexpected character '[' is found at position 11.
select json_value(#json, 'lax $.data.2077-09-02.Description');
--JSON path is not properly formatted. Unexpected character '2' is found at position 11.
How do I get to the other values? Is the JSON not perfect enough for TSQL?
It is never a good idea to use the declarative part of a text based container as data. The "2077-09-02" is a valid json key, but hard to query.
You can try this:
declare #json nvarchar(max) = '{
"action": "edit",
"data": {
"2077-09-02": {
"Description": "some stuff",
"EffectDate": "2077-1-1"
}
}
}';
SELECT A.[action]
,B.[key] AS DateValue
,C.*
FROM OPENJSON(#json)
WITH([action] NVARCHAR(100)
,[data] NVARCHAR(MAX) AS JSON) A
CROSS APPLY OPENJSON(A.[data]) B
CROSS APPLY OPENJSON(B.[value])
WITH (Description NVARCHAR(100)
,EffectDate DATE) C;
The result
action DateValue Description EffectDate
edit 2077-09-02 some stuff 2077-01-01
The idea:
The first OPENJSON will return the action and the data.
I use a WITH clause to tell the engine, that action is a simple value, while data is nested JSON
The next OPENJSON dives into data
We can now use B.[key] to get the json key's value
Now we need another OPENJSON to dive into the columns within data.
However: If this JSON is under your control I'd suggest to change its structure.
Use double quotes instead of []. JSON Path uses JavaScript's conventions where a string is surrounded by double quotes. The documentation's example contains this path $."first name".
In this case :
select json_value(#json,'$.data."2077-09-02".Description');
Returns :
some stuff
As for the other calls, JSON_VALUE can only return scalar values, not objects. You need to use JSON_QUERY to extract JSON objects, eg :
select json_query(#json,'$.data."2077-09-02"');
Returns :
{
"Description": "some stuff",
"EffectDate": "2077-1-1"
}

SQL Server - JSON Handling Unknown First Element

I am capturing JSON data from a number of services sources to a table in SQL Server; where it is queued for processing. Each of the data sources has a different initial element to identify the type. A fictional example is below.
I can query the content of the file easily enough using json_value (example below) or json_query. What I'm struggling to figure out is identifying which type of file i have in order to commence processing. The identifier is the name of first element "AddressDetails" in the example below. Is there a way in SQL's json querying to get the name of the first element out? Or do I need to push that up to the service that is receiving the data and save the type with the json data?
declare #test table (jsondata nvarchar(max))
insert into #test
select '{
"AddressDetails": {
"type": 1,
"address": {
"Street1": "A Street",
"Street2": "",
"town": "Bristol",
"county": "Avon",
"country": "England"
}
}
} '
select
json_value(jsondata, '$.AddressDetails."type"'),
json_value(jsondata, '$.AddressDetails.address."Street1"'),
json_value(jsondata, '$.AddressDetails.address."Street2"'),
json_value(jsondata, '$.AddressDetails.address."town"'),
json_value(jsondata, '$.AddressDetails.address."county"'),
json_value(jsondata, '$.AddressDetails.address."country"')
from #test
Suppose your structure will remain the same, except the first identifier
you can use the following dynamic part:
UPDATE
SET NOCOUNT ON
IF OBJECT_ID ('tempdb..##Test') IS NOT NULL DROP TABLE ##Test
CREATE TABLE ##TEST (jsondata nvarchar(max))
DECLARE #Cmd NVARCHAR(MAX)=''
insert into ##TEST
select '{
"NameOfIdentifier": {
"type": 1,
"address": {
"Street1": "A Street",
"Street2": "",
"town": "Bristol",
"county": "Avon",
"country": "England"
}
}
}'
SELECT #Cmd+= 'SELECT DISTINCT '''+[Key]+''' as Identifier FROM ##TEST
CROSS APPLY OPENJSON(jsondata ,'''+'$.'+[Key]+''');'
FROM ##TEST
CROSS APPLY OPENJSON (jsondata )
EXEC sp_executesql #Cmd
DROP TABLE ##TEST
you'll get the following result:
Let me know if this helps