Merging JSON objects in SQL Server - json

Question

We have a base table which has custom column_values column
Table_Structure:
User_Dimension-
Userid,username,addresss,Custom_value
Userid is the primary key and customer can map the fields present in the file using our UI.
If any of the columns present in the files doesn't fit in the column present in our base tables, we will create a custom column and store it in the form of json.
Userid,username,addresss,Custom_value
234,AK4140,BANGLORE,{"Pin:"522413","State":"Maharastra"}
The data will be stored as mentioned in the above in a staging table
Note: for the table User_Dimension there can be data from multiple files so my custom values are different for each file and that information is stored in a meta data table.
We are using SCD Type 1 for dimension tables
The problem is to merge JSON column.
Consider this scenario:
User_Dimension
Userid,username,addresss,Service_Type,User_Type,Custom_value
234,ak4140,banglore,null,null,{"Pin:"522413","State":"Maharastra"}
The above entry was present in my user_dimension from File1
Now I need to push below value to my table from File2
Userid,username,addresss,Service_Type,User_Type,Custom_value
234,NULL,NULL,Customer,DVV,{"Birthdate:"19-09-1995","State":"Karnataka"}
I am merging both the values based on the Userid.
The problem is Custom_Value column. From the above entries I need to update this column as shown here:
Userid,username,addresss,Service_Type,User_Type,Custom_value
234,ak4140,banglore,Customer,DVV,{"Pin:"522413","State":"Karnataka","Birthdate":"19-09-1995"}

Answer 1

I wrote a function which can perhaps be used to merge two json files:
create or alter function dbo.FN_JSON_MERGE(#pJson1 NVARCHAR(MAX), #pJson2 NVARCHAR(MAX))
RETURNS NVARCHAR(MAX)
AS
BEGIN
-- Get keys...
declare #t table ([key] nvarchar(max) collate database_default, [value] nvarchar(max) collate database_default, row_id int identity)
insert into #t
select [key], [value]
from OPENJSON(#pJson1)
-- Merge values from #pjson2
update t
set value = oj.value
from #t t
inner join OPENJSON(#pJson2) oj
ON oj.[key] collate database_default = t.[key] collate database_default
insert into #t
select [key], [value]
from OPENJSON(#pJson2) o
where not exists(
select 1
from #t t2
where t2.[key] collate database_default = o.[key] collate database_default
)
-- Finally generate new json...
set #pJson2 = ''
select #pJson2 = #pJson2 + ',' + '"' + [key] + '": "' + (value) + '"'
from #t
order by [row_id]
return '{' + stuff(#pJson2, 1, 1, '') + '}'
END
Test code:
select dbo.FN_JSON_MERGE('{"Pin":"522413","State":"Maharastra"}'
, '{"Birthdate":"19-09-1995","State":"Karnataka"}')
-- returns {"Pin": "522413","State": "Karnataka","Birthdate": "19-09-1995"}
But there are lot of BUTs. It might not handle very long strings / strings with quotes or other more weird jsons etc.
Also, it doesn't always keep the same attribute order as original json.
It's likely to be very slow.
Finally, it doesn't handle if you want to merge data from 3 files.
Right now, the second argument values always overwrite first.
But maybe it can be of some use. You can always create a procedure which does this for better performance.