I have a table with an VARCHAR(MAX) column which stores JSON key value pairs.
The JSON document schema is simply a different number of key value pairs, no nesting, nor arrays in there.
I wish to build a query, which gives back the JSON in a tabular format,
which is easy with the named elements (see WITH clause below):
DECLARE #MYJSONTABLE TABLE (ID INT IDENTITY NOT NULL PRIMARY KEY, MYDATA NVARCHAR(max) null)
INSERT INTO #MYJSONTABLE
(
MYDATA
)
VALUES
(N'{"id": 2, "info": "some info", "age": 25}'),
(N'{"id": 5, "info": "other info", "dob": "2005-11-04T12:00:00"}')
SELECT p.ID,MYDATA.*
FROM #MYJSONTABLE p
CROSS APPLY
OPENJSON(p.MYDATA)
WITH (
id INT 'strict $.id',
info NVARCHAR(50) '$.info',
age INT,
dateOfBirth DATETIME2 '$.dob'
) AS MYDATA
While the output is exactly what I want,
my issue with the above solution, that I don't know the key names in the JSON document neither, and how many are there, but still wish to return them all in the same tabular format.
If I omit the WITH clause above, the query do return all key value pairs, but the output goes "vertical" and each key in the JSON generates a new row.
Could the above query be modified to be dynamic, and return all key value pairs without explicitly specifying the JSON key names?
Perhaps something like this will work for you.
This uses a CTE to get the DISTINCT key's from your JSON. Then string aggregation to create a dynamic statement, which you can see from the PRINT statement.
Note that for your sample data, the column dob is not returned because it is outside of the initial JSON defined. If the first right brace (}) is removed, the column appears.
DECLARE #SQL nvarchar(MAX),
#CRLF nchar(2) = NCHAR(13) + NCHAR(10);
DECLARE #Delimiter nvarchar(50) = N',' + #CRLF + N' ';
WITH Keys AS(
SELECT DISTINCT J.[key]
FROM dbo.YourTable YT
CROSS APPLY OPENJSON(YT.JsonColumn) J)
SELECT #SQL = N'SELECT YT.ID,' + #CRLF +
N' J.*' + #CRLF +
N'FROM dbo.YourTable YT' + #CRLF +
N' CROSS APPLY OPENJSON(YT.JsonColumn)' + #CRLF +
N' WITH(' +
STRING_AGG(QUOTENAME(K.[key]) + N' nvarchar(100)', #Delimiter) + N') J;'
FROM Keys K;
PRINT #SQL;
EXEC sys.sp_executesql #SQL;
Note, this will not work with a table variable, unless you create a table type and then pass the TYPE as a parameter to sys.sp_executesql. This is why the above assumes a real table.
Related
We have a base table which has custom column_values column
Table_Structure:
User_Dimension-
Userid,username,addresss,Custom_value
Userid is the primary key and customer can map the fields present in the file using our UI.
If any of the columns present in the files doesn't fit in the column present in our base tables, we will create a custom column and store it in the form of json.
Userid,username,addresss,Custom_value
234,AK4140,BANGLORE,{"Pin:"522413","State":"Maharastra"}
The data will be stored as mentioned in the above in a staging table
Note: for the table User_Dimension there can be data from multiple files so my custom values are different for each file and that information is stored in a meta data table.
We are using SCD Type 1 for dimension tables
The problem is to merge JSON column.
Consider this scenario:
User_Dimension
Userid,username,addresss,Service_Type,User_Type,Custom_value
234,ak4140,banglore,null,null,{"Pin:"522413","State":"Maharastra"}
The above entry was present in my user_dimension from File1
Now I need to push below value to my table from File2
Userid,username,addresss,Service_Type,User_Type,Custom_value
234,NULL,NULL,Customer,DVV,{"Birthdate:"19-09-1995","State":"Karnataka"}
I am merging both the values based on the Userid.
The problem is Custom_Value column. From the above entries I need to update this column as shown here:
Userid,username,addresss,Service_Type,User_Type,Custom_value
234,ak4140,banglore,Customer,DVV,{"Pin:"522413","State":"Karnataka","Birthdate":"19-09-1995"}
I wrote a function which can perhaps be used to merge two json files:
create or alter function dbo.FN_JSON_MERGE(#pJson1 NVARCHAR(MAX), #pJson2 NVARCHAR(MAX))
RETURNS NVARCHAR(MAX)
AS
BEGIN
-- Get keys...
declare #t table ([key] nvarchar(max) collate database_default, [value] nvarchar(max) collate database_default, row_id int identity)
insert into #t
select [key], [value]
from OPENJSON(#pJson1)
-- Merge values from #pjson2
update t
set value = oj.value
from #t t
inner join OPENJSON(#pJson2) oj
ON oj.[key] collate database_default = t.[key] collate database_default
insert into #t
select [key], [value]
from OPENJSON(#pJson2) o
where not exists(
select 1
from #t t2
where t2.[key] collate database_default = o.[key] collate database_default
)
-- Finally generate new json...
set #pJson2 = ''
select #pJson2 = #pJson2 + ',' + '"' + [key] + '": "' + (value) + '"'
from #t
order by [row_id]
return '{' + stuff(#pJson2, 1, 1, '') + '}'
END
Test code:
select dbo.FN_JSON_MERGE('{"Pin":"522413","State":"Maharastra"}'
, '{"Birthdate":"19-09-1995","State":"Karnataka"}')
-- returns {"Pin": "522413","State": "Karnataka","Birthdate": "19-09-1995"}
But there are lot of BUTs. It might not handle very long strings / strings with quotes or other more weird jsons etc.
Also, it doesn't always keep the same attribute order as original json.
It's likely to be very slow.
Finally, it doesn't handle if you want to merge data from 3 files.
Right now, the second argument values always overwrite first.
But maybe it can be of some use. You can always create a procedure which does this for better performance.
I'm hoping to build an optimized data JSON structure that only includes data, no names. I'll included the names in another JSON.
For example
[["1", "William"],["2", "Dylan"]]
I'm looking at "for json auto", running a query like this.
declare #t table(id int, name varchar(20))
insert into #t (id, name) values( 1, 'William')
insert into #t (id, name) values( 2, 'Dylan')
declare #result as varchar(max)
select id, name from #t for json auto
However it includes the names with every value.
[{"id":1,"name":"William"},{"id":2,"name":"Dylan"}]
Is there a way to instruct SQL Server to omit the names and just return a string array?
I'll need to update a couple hundred queries, so I'm hoping for an answer that doesn't require too much modification on a basic query.
Unfortunately, SQL Server does not support the JSON_AGG function or similar. You can hack it with STRING_AGG and STRING_ESCAPE.
You can either do this with a single aggregation and concatenating the row together
SELECT '[' + STRING_AGG(CONCAT(
'["',
id,
'","',
STRING_ESCAPE(name, 'json'),
'"]'
), ',') + ']'
FROM #t t;
Or with a nested aggregation, aggregating first each row in an unpivoted subquery, then all rows together
SELECT '[' + STRING_AGG('[' + j.json + ']', ',') + ']'
FROM #t t
CROSS APPLY (
SELECT STRING_AGG('"' + STRING_ESCAPE(value, 'json') + '"', ',')
FROM (VALUES
(CAST(id AS nvarchar(max))),
(name)
) v(value)
) j(json);
db<>fiddle
I've assumed columns are not nullable. Nullable columns will need special handling, I leave it as an exercise to the reader.
Not all that different from Charlie's but uses CONCAT_WS to remove some of the explicit " characters:
SELECT [json] = '['
+ STRING_AGG('["' + CONCAT_WS('", "', id,
STRING_ESCAPE(COALESCE(name,''), N'JSON'))
+ '"]', ',') + ']'
FROM #t;
Output (after adding a 3rd row, values(3, NULL):
json
[["1", "William"],["2", "Dylan"],["3", ""]]
Example db<>fiddle
If you want the literal string null with no quotes:
SELECT [json] = '['
+ STRING_AGG('['
+ CONCAT_WS(', ', CONCAT('"', id, '"'),
COALESCE('"' + STRING_ESCAPE(name, N'JSON') + '"', 'null'))
+ ']', ',') + ']'
FROM #t;
Output:
json
[["1", "William"],["2", "Dylan"],["3", null]]
Example db<>fiddle
If you don't want the NULL value to present a column in the JSON, just remove the COALESCE:
SELECT [json] = '['
+ STRING_AGG('["' + CONCAT_WS('", "', id,
STRING_ESCAPE(name, N'JSON'))
+ '"]', ',') + ']'
FROM #t;
Output:
json
[["1", "William"],["2", "Dylan"],["3"]]
Example db<>fiddle
If you don't want that row present in the JSON at all, just filter it out:
FROM #t WHERE name IS NOT NULL;
If that column doesn't allow NULLs, state it explicitly so we don't have to guess (probably doesn't hurt to confirm id is unique, either):
declare #t table(id int UNIQUE, name varchar(20) NOT NULL);
I have a json array in my table. It contains an array. I can create, append or make my array NULL. Inside my stored procedure but I don't see any way to pop value from array. Apparently JSON_Modify may have solution as you can update key as well as Single value but how can I use it to modify my array?
--My Array
Declare #json = '{"array":[123,456]}'
Desired results after update:
'{"array":[123]}'
Please note that array contain int values. Which are my sub department id. All values are (supposed to be) unique.
You could use:
DECLARE #json NVARCHAR(MAX) = '{"array":[123,456]}';
WITH cte AS (
SELECT *, MAX([key]) OVER() AS m_key
FROM OPENJSON(#json, '$.array') s
)
SELECT JSON_QUERY('[' + IIF(MAX(m_key) = 0, '', STRING_AGG(value,',')
WITHIN GROUP (ORDER BY [key])) + ']','$') AS array
FROM cte
WHERE [key] != m_key OR m_key = 0
FOR JSON AUTO, WITHOUT_ARRAY_WRAPPER;
Output:
{"array":[123]}
DBFiddle Demo SQL Server 2017
As I was in hurry I solved my problem following way, but I would really recommend not to use it. Please see answer above by #lad2025.
DECLARE #json VARCHAR(MAX)
=(SELECT jsonDept
FROM tblEmployee
WHERE tblEmployeeID = #empid)
DECLARE #newjson VARCHAR(MAX)= (
SELECT LEFT(subdept, LEN(subdept)-1)
FROM (
SELECT Distinct value + ', ' FROM OPENJSON(#json,'$.array') Where value <> #subdeptid
FOR XML PATH ('')
) t (subdept))
UPDATE tblEmployee SET jsonDept = '{"array":['+ #newjson +']}' WHERE tblEmployeeID = #empid
I am storing ids in comma separated string.
e.g
1,2,3,4
How can I store this in JSON in the column and should be able to insert delete any particular value?
Thanks
Part of the following answer comes from here, so all credits go there: https://stackoverflow.com/a/37844117/2695832
Here's a solution that enables you to store your string values in a JSON array in a table column. However, the should be able to insert delete any particular value part of your question is not totally clear to me.
DECLARE #source VARCHAR(20);
SET #source = '1,2,3,4';
DECLARE #values TABLE
(
[Id] VARCHAR(20)
);
INSERT INTO #values
(
[Id]
)
SELECT
value
FROM [STRING_SPLIT](#source, ',')
WHERE RTRIM(value) <> '';
INSERT INTO #values ([Id]) VALUES ('5')
DELETE FROM #values WHERE Id = 2
SELECT
JSON_QUERY('[' + STUFF(( SELECT ',' + '"' + Id + '"'
FROM #values FOR XML PATH('')),1,1,'') + ']' ) ids
FOR JSON PATH , WITHOUT_ARRAY_WRAPPER
This produces the following JSON object:
{"ids":["1","3","4","5"]}
The code might need some tweaking to completely match your needs since you're probably not using a table variable and also maybe want to use a numeric data type for your id values.
I am trying to create a stored that will accept two values, a column name and a value. It will then check if there is a record that exists for the passed in column name with the passed in value. I've tried the following
CREATE PROCEDURE p_HasActiveReference
#pi_colName varchar(100)
,#pi_colValue varchar(100)
AS
BEGIN
SET NOCOUNT ON
declare #sql varchar(1000)
set #sql = 'IF EXISTS(SELECT TOP 1 p.PaymentId
FROM Payments p
WHERE ' + #pi_colName + ' = ' + #pi_colValue + 'AND Active = 1)
SELECT ''True'' AS RETVAL
ELSE
SELECT ''False'' AS RETVAL'
exec(#sql)
END
However, I always get this error
Conversion failed when converting the varchar value 'InternalOr' to data type int.
When I call the procedure with the following
p_HasActiveReference 'InternalOrgCode', '10110'
The internalOrgCode column is of value varchar(10)
I am not a SQL expert, so I am not even sure if what I need to achieve is even possible using that technique!
Thanks!
At least one issue: you should be surrounding your string value with single quotes, and to escape those inside a string you need to double them up:
WHERE ' + #pi_colName + ' = ''' + #pi_colValue + ''' AND ...
You also may want to declare your #sql variable as something bigger than 100 characters! Looks like your string is getting truncated.
If the possible values for #pi_colName are finite, the data type is always string, and the columns are collation compatible, you could do something like this and avoid dynamic SQL:
SELECT ...
WHERE CASE #pi_colName
WHEN 'col1' THEN col1
WHEN 'col2' THEN col2
END = #pi_ColValue;