SQL JSON array sort - mysql

record in DB:
id info
0 [{"name":"a", "time":"2017-9-25 17:20:21"},{"name":"b", "time":"2017-9-25 23:23:41"},{"name":"c", "time":"2017-9-25 12:56:78"}]
my goal is to sort json array column info base on time, like:
id info
0 [{"name":"c", "time":"2017-9-25 12:56:78"},{"name":"a", "time":"2017-9-25 17:20:21"},{"name":"b", "time":"2017-9-25 23:23:41"},]
I use sparkSQL, having no clue

You can do this by converting the json array into a sql result set, extract the sorting column, and finally convert it back to a json array:
DECLARE #json NVARCHAR(MAX);
SET #json = '[
{"name":"a", "time":"2017-09-25 17:20:21"},
{"name":"b", "time":"2017-09-25 23:23:41"},
{"name":"c", "time":"2017-09-25 12:56:59"}
]';
WITH T AS (
SELECT [Value] AS array_element
, TRY_CAST(JSON_VALUE(Value, 'strict $.time') AS DATETIME) AS sorting
FROM OPENJSON(#json, 'strict $')
)
SELECT STRING_AGG(T.array_element, ',') WITHIN GROUP (ORDER BY sorting)
FROM T
Notice:
I changed the sample data slightly, due to invalid months and seconds.
The STRING_AGG() function is only available from SQL 2017/Azure SQL Database. For older versions, use the classic "FOR XML PATH" method, which I will leave as an exercise to the reader.
If you want to apply it to a full sql table, use CROSS APPLY as follows:
DECLARE #json NVARCHAR(MAX);
SET #json = '[
{"name":"a", "time":"2017-09-25 17:20:21"},
{"name":"b", "time":"2017-09-25 23:23:41"},
{"name":"c", "time":"2017-9-25 12:56:59"}
]';
WITH dat AS (
SELECT * FROM (VALUES (1,#json), (2,#json)) AS T(id, info)
)
, T AS (
SELECT id, [Value] AS array_element
, TRY_CAST(JSON_VALUE(Value, 'strict $.time') AS DATETIME) AS sorting
FROM dat
CROSS APPLY OPENJSON(info, 'strict $')
)
SELECT id
, STRING_AGG(T.array_element, ',') WITHIN GROUP (ORDER BY sorting) AS info
FROM T
GROUP BY id

What I Suggest
Alternate Database Storage System
I would not recommend storing data in this manner. It simply makes your data less accessible and malleable as you are experiencing now. If you store your data like this:
id name time
0 a 2017-9-25 17:20:21
0 b 2017-9-25 23:23:41
0 c 2017-9-25 12:56:71
1 ... ...
Then you can select the data in time order using the ORDER BY method at the end of a select query. For example:
SELECT name, time FROM table_name where id=0 ORDER BY time asc;
If you have more columns in this table that you did not show, you may need a another table to efficiently store the information, but the performance benefits of foreign keys and joins between these types of tables would outweigh having all the data in one table as inconvenient JSON arrays.

Related

In a SQL Server table how do I filter records based on JSON search on a column having JSON values

I am facing a challenge while filtering records in a SQL Server 2017 table which has a VARCHAR column having JSON type values:
Sample table rows with JSON column values:
Row # 1. {"Department":["QA"]}
Row # 2. {"Department":["DEV","QA"]}
Row # 3. {"Group":["Group 2","Group 12"],"Cluster":[Cluster 11"],"Vertical":
["XYZ"],"Department":["QAT"]}
Row # 4. {"Group":["Group 20"],"Cluster":[Cluster 11"],"Vertical":["XYZ"],"Department":["QAT"]}
Now I need to filter records from this table based on an input parameter which can be in the following format:
Sample JSON input parameter to query:
1. `'{"Department":["QA"]}'` -> This should return Row # 1 as well as Row # 2.
2. `'{"Group":["Group 2"]}'` -> This should return only Row # 3.
So the search should be like if the column value contains "any available json tag with any matching value" then return those matching records.
Note - This is exactly similar to PostgreSQL jsonb as shown below:
PostgreSQL filter clause:
TableName.JSONColumnName #> '{"Department":["QA"]}'::jsonb
By researching on internet I found OPENJSON capability that is available in SQL Server which works as below.
OPENJSON sample example:
SELECT * FROM
tbl_Name UA
CROSS APPLY OPENJSON(UA.JSONColumnTags)
WITH ([Department] NVARCHAR(500) '$.Department', [Market] NVARCHAR(300) '$.Market', [Group] NVARCHAR(300) '$.Group'
) AS OT
WHERE
OT.Department in ('X','Y','Z')
and OT.Market in ('A','B','C')
But the problem with this approach is that if in future there is a need to support any new tag in JSON (like 'Area'), that will also need to be added to every stored procedure where this logic is implemented.
Is there any existing SQL Server 2017 capability I am missing or any dynamic way to implement the same?
Only thing I could think of as an option when using OPENJSON would be break down your search string into its key value pair, break down your table that is storing the json you want to search into its key value pair and join.
There would be limitations to be aware of:
This solution would not work with nested arrays in your json
The search would be OR not AND. Meaning if I passed in multiple "Department" I was searching for, like '{"Department":["QA", "DEV"]}', it would return the rows with either of the values, not those that only contained both.
Here's a working example:
DECLARE #TestData TABLE
(
[TestData] NVARCHAR(MAX)
);
--Load Test Data
INSERT INTO #TestData (
[TestData]
)
VALUES ( '{"Department":["QA"]}' )
, ( '{"Department":["DEV","QA"]}' )
, ( '{"Group":["Group 2","Group 12"],"Cluster":["Cluster 11"],"Vertical": ["XYZ"],"Department":["QAT"]}' )
, ( '{"Group":["Group 20"],"Cluster":["Cluster 11"],"Vertical":["XYZ"],"Department":["QAT"]}' );
--Here is the value we are searching for
DECLARE #SeachJson NVARCHAR(MAX) = '{"Department":["QA"]}';
DECLARE #SearchJson TABLE
(
[Key] NVARCHAR(MAX)
, [Value] NVARCHAR(MAX)
);
--Load the search value into a temp table as its key\value pair.
INSERT INTO #SearchJson (
[Key]
, [Value]
)
SELECT [a].[Key]
, [b].[Value]
FROM OPENJSON(#SeachJson) [a]
CROSS APPLY OPENJSON([a].[Value]) [b];
--Break down TestData into its key\value pair and then join back to the search table.
SELECT [TestData].[TestData]
FROM (
SELECT [a].[TestData]
, [b].[Key]
, [c].[Value]
FROM #TestData [a]
CROSS APPLY OPENJSON([a].[TestData]) [b]
CROSS APPLY OPENJSON([b].[Value]) [c]
) AS [TestData]
INNER JOIN #SearchJson [srch]
ON [srch].[Key] COLLATE DATABASE_DEFAULT = [TestData].[Key]
AND [srch].[Value] = [TestData].[Value];
Which gives you the following results:
TestData
-----------------------------
{"Department":["QA"]}
{"Department":["DEV","QA"]}

Recursive CTEs to pull BOM (Bill of Material)

I need some assistance or pointer on CTEs.
I am trying to extract Bill of Material and I have used CTEs query. The query works good and it pulls all the data. My struggle is there are lot of parts where the parts has new version on different levels and I want to grab new versions only. Currently my query grabs everything. I have a version column.
I tried few different things like trying to utilize max function within CTEs but I got an error saying group by, having cannot be part of recursive ctes.
Also, I tried using subquery but I didnt get the right result.
WITH BOM (
Parent
,Child
,Qty
,Childrev
,LEVEL
,sort
)
AS (
SELECT Parent
,cast(RTRIM(Child) AS NVARCHAR(max))
,Qty
,Childrev
,0 AS LEVEL
,cast(RTRIM(Child) AS NVARCHAR(max))
FROM Bomtable
UNION ALL
SELECT BOM.Parent
,cast(RTRIM(Bomtable.Child) AS NVARCHAR(max))
,Bomtable.Qty
,BOM.Childrev
,LEVEL + 1
,CAST(BOM.Sort + '..... ' + RTRIM(Bomtable.Child) AS NVARCHAR(max))
FROM BOM
INNER JOIN Bomtable ON Bomtable.Parent = BOM.Child
WHERE BOM.Parent = main product
ORDER BY SORT
)
I know I do not fully understand your data model. However, try replacing your BOM and BomTable tables with a derived table like this which will give you one row for each Child record with the greatest Childrev value without using a GROUP BY.
SELECT *
FROM (
SELECT *
, ROW_NUMBER() OVER (PARTITION BY Child ORDER BY Childrev DESC) AS ROW_NBR
FROM BOM
) AS x
WHERE x.ROW_NBR = 1;
Here is the documentation for the the OVER Clause.
Noel

convert query output into json format in sql server

I want to display the below query output into JSON Format(Required output format)
select ApplicationID ApplicationID ,Roleid Roles from UserRoles where userid=11 and applicationid is not null
Output
ApplicationID Roles
1 1
1 5
3 5
i want to display below format i am using sql server 2016
Required output Format:
[{"ApplicationID":1,"Roles":[1,5]},{"ApplicationID":3,"Roles":[5]}]
The closest you get with pure T-SQL-JSON-support will be this:
DECLARE #tbl TABLE(ApplicationID INT, Roles INT);
INSERT INTO #tbl VALUES
(1,1)
,(1,5)
,(3,5);
SELECT t.ApplicationID
,Roles.Roles AS Roles
FROM #tbl t
INNER JOIN #tbl Roles ON Roles.ApplicationID=t.ApplicationID
GROUP BY t.ApplicationID,Roles.Roles
FOR JSON AUTO;
The result
[{"ApplicationID":1
,"Roles":[{"Roles":1}
,{"Roles":5}]
}
,{"ApplicationID":3
,"Roles":[{"Roles":5}]}
]
In AUTO mode the engine will "see" the join and pack this into an array of objects.
Regretfully FOR JSON does not support naked arrays ([1,2,3]). You will always get arrays of objects like [{Prop:Val},{Prop:Val},{Prop:Val}]...
But you can out-trick this with a correlated sub-query and some string aggregation (We need JSON_QUERY() to avoid quotes around the array):
SELECT t.ApplicationID
,JSON_QUERY('[' + STUFF((
SELECT CONCAT(',',Roles)
FROM #tbl t2
WHERE t2.ApplicationID=t.ApplicationID
FOR XML PATH('')
),1,1,'') + ']') AS Roles
FROM #tbl t
GROUP BY t.ApplicationID
FOR JSON PATH;
The result
[{"ApplicationID":1,"Roles":[1,5]}
,{"ApplicationID":3,"Roles":[5]}]
Can you use STRING_AGG() (v2017+)? In this case you can simplify the sub-select.
As I remember:
select ApplicationID ApplicationID ,Roleid Roles from UserRoles where userid=11 and applicationid is not null
FOR JSON PATH, ROOT("UserRoles")
If you are using SQL Server 2016+, you can use FOR JSON statement
SELECT
ApplicationID ApplicationID,
Roleid Roles
FROM UserRoles
WHERE userid = 11 AND applicationid IS NOT NULL
FOR JSON AUTO
Take a look into this: Format Query Results as JSON with FOR JSON

PostgreSQL json aggregation issue

How with Postgres aggregation function merge one object as element in array field in parent object.
What need
Sector
Project
Result
My SQL request.
select row_to_json(t)
from (
select id, data,
(
select array_to_json(array_agg(row_to_json(p)))
from (
select id, data
from public."Project"
where (s.data -ยป 'projectId') :: UUID = id
) p
) as projects
from public."Sector" s
) t;
It don't work, because projects is null. But need, unwind data field and merge projectId in data with Project table. Like unwind and lookup in MongoDB.

Spliting Json data into array<string> columns

I have many json arrays stored in a table like this:
{"p_id":
{"id_type":"XXX","id":"ABC111"},
"r_ids":[
{"id_type":"HAWARE_ABCDA1","id":"dfe234fhgt"},
{"id_type":"HAWARE_CDFE2","id":"sgteth5673"}
]
}
My requirement is to get data in below format:
p_id , p_id_type ,r_ids (array string), r_id_type (array string)
Ex: XXX,ABC111,[dfe234fhgt,sgteth5673],[HAWARE_ABCDA1,HAWARE_CDFE2]
I am able to get the whole set in exploded format but how to generate array
My current query:
select p_id
,p_id_type
,get_json_object(c.qqqq,'$.id') as r_id
,get_json_object(c.qqqq,'$.id_type') as r_id_type
from
(
select p_id
,p_id_type
,qqqq
from
(
select
get_json_object(a.main_pk,'$.id_type') as p_id_type
,get_json_object(a.main_pk,'$.id') as p_id
,split(regexp_replace(regexp_replace(a.r_ids,'\\}\\,\\{','\\}\\;\\{'),'\\[|\\]',''),'\\;') as yyyy
from
(
select
get_json_object(json_string,'$.p_id') as main_pk
,get_json_object(json_string, '$.r_ids') as r_ids
from sample_table limit 10
) a
) b lateral view explode(b.yyyy) yyyy_exploded as qqqq
)c
Can anyone help me what wrong I am doing? Any suggestions will be appreciated.
If you use JsonSerDe, it will be more easy to solve complex data types.
I am giving here small example, you can solve by using this:
CREATE TABLE table_json (
p_id struct<id_type:string,
id:string,
r_ids:array<struct<id_type:string,
id:string>>>
)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe';
LOAD DATA LOCAL INPATH '<path>/your_file.json'
OVERWRITE INTO TABLE table_json;