I have a JSON structure stored in a MySql table. Now, months later, I have a requirement to join to pieces found deep in the bowels of this JSON string.
{
"id": "000689ba-9891-4d82-ad13-a7b96dc08ec4",
"type": "emp",
"firstName": "Brooke",
"facilities": {
"0001": [{
"id": 1,
"code": "125",
"name": "DSL",
"type": "MGMTSupport",
"payRate": 18}],
"0002": [
{
"id": 1,
"code": "100",
"name": "Server",
"type": "FOH",
"payRate": 8
}, {
"id": 2,
"code": "320",
"name": "NCFOHTrainer",
"type": "NCHourlyTraining",
"payRate": 14.5
}]
},
"permissions": ["read:availability", "..."],
"primaryJobCode": "150",
"primaryPayRate": 9,
"primaryFacility": "0260"
}
The big question is: How do I shape this as a query in MySql when the facilities do not follow a single key/value pattern? i.e.: the key to the first entry is the facilityId so I cannot use a path like '$.0001' and the dictionary value is an array so how do I path that correctly?
select id as EmployeeId
, companyId as cpkEmployeeId
, json_table( `data`
, '$.facilities[*]' COLUMNS( facilityId VARCHAR(10) PATH '$.????'
, NESTED PATH '??? $[*] ???' COLUMNS ( code VARCHAR(10) PATH '$.code'
, payRate DECIMAL(8,4) PATH '$.payRate') facilities
from employee
;
Yea - the above does not work. Any help appreciated.
Desired output?
[Other columns from the table] plus facilityId, code & payrate.
A single row in the native table could produce something like:
id | companyId | facilityId | code | payRate
--------+-----------+------------+------+---------
1 | 324337 | 0001 | 125 | 18.0000
1 | 324337 | 0002 | 100 | 8.0000
1 | 324337 | 0002 | 320 | 14.5000
WITH
cte AS (
SELECT test.id,
test.value,
jsontable.facilityId,
CONCAT('$.facilities."', jsontable.facilityId, '"') path
FROM test
CROSS JOIN JSON_TABLE(JSON_KEYS(test.value, '$.facilities'),
'$[*]' COLUMNS (facilityId CHAR(4) PATH '$')) jsontable
)
SELECT cte.id,
cte.facilityId,
jsontable.code,
jsontable.payRate
FROM cte
CROSS JOIN JSON_TABLE(JSON_EXTRACT(cte.value, cte.path),
'$[*]' COLUMNS (code CHAR(3) PATH '$.code',
payRate DECIMAL(6, 4) PATH '$.payRate')) jsontable
https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=18ab7b6f181b61fb53f88a6de6e049be
Related
Here's the JSON that I have in the MySQL table:
{
"qmSubsegmentFlags": [
{
"text": "Accuracy",
"id": 1,
"children": [
{
"text": "Mistranslation",
"id": 2,
"children": [
{
"text": "Terminology",
"id": 3
}
]
},
{
"text": "Omission",
"id": 4
},
{
"text": "Addition",
"id": 5
},
{
"text": "Untranslated",
"id": 6
}
]
}
]
}
So you see, id's and text can be in different levels. I only have the value of the id and have to get the text for this with SQL. Below a JSON path that works for the first level, but not for all levels. Neither does it work in MySQL, since MySQL only supports limited filtering.
$.qmSubsegmentFlags[?(#.id==1)].text
A little SQL that goes into the right direction, but also does not solve the issue.
SELECT JSON_EXTRACT(t.qmSubsegmentFlags, '$.qmSubsegmentFlags[*].text') AS lqa FROM translate5.LEK_task AS t where id = 3719;
I used to be a programmer, but after 4 years of project management, my skills are dead. But I need a proof of concept until monday. Any help would be great!
Here's a solution tested on MySQL 8.0.32.
WITH RECURSIVE tree AS (
SELECT j.id, j.text, j.children
FROM LEK_task CROSS JOIN JSON_TABLE(
JSON_EXTRACT(qmSubsegmentFlags, '$.qmSubsegmentFlags'),
'$[*]' COLUMNS (
text TEXT PATH '$.text',
id INT PATH '$.id',
children JSON PATH '$.children'
)
) AS j
WHERE LEK_task.id = 3719
UNION ALL
SELECT j.id, j.text, j.children
FROM tree CROSS JOIN JSON_TABLE(
children,
'$[*]' COLUMNS(
text TEXT PATH '$.text',
id INT PATH '$.id',
children JSON PATH '$.children'
)
) AS j
)
SELECT id, text FROM tree;
Result:
+------+----------------+
| id | text |
+------+----------------+
| 1 | Accuracy |
| 2 | Mistranslation |
| 4 | Omission |
| 5 | Addition |
| 6 | Untranslated |
| 3 | Terminology |
+------+----------------+
You can add a WHERE id = 1 to the last line of the query to get the row for a specific id.
I have a JSON variable that looks like this (the real one is more complex):
DECLARE #myJson VARCHAR(3000) = '{
"CustomerId": "123456",
"Orders": [{
"OrderId": "852",
"OrderManifests": [{
"ShippedProductId": 884,
"ProductId": 884
}, {
"ShippedProductId": 951,
"ProductId": 2564
}
]
}, {
"OrderId": "5681",
"OrderManifests": [{
"ShippedProductId": 198,
"ProductId": 4681
}, {
"ShippedProductId": 8188,
"ProductId": 8188
}, {
"ShippedProductId": 144,
"ProductId": 8487
}
]
}
]
}'
In the end, I need to know if any of the ShippedProductId values match their corresponding ProductId (in the same JSON object).
I started in by trying to get a list of all the OrderManifests. But while this will get me the array of orders:
SELECT JSON_QUERY(#myJson, '$.Orders')
I can't seem to find a way to get a list of all the OrderManifests across all the entries in the Orders array. This does not work:
SELECT JSON_QUERY(#myJson, '$.Orders.OrderManifests')
Is there a way to do a Select Many kind of query to get all the OrderManifests in the Orders array?
Use OPENJSON and CROSS APPLY to drill down into your objects.
This should do it for you:
SELECT j.CustomerId,o.OrderId, m.ShippedProductId, m.ProductId
FROM OPENJSON(#myJson)
WITH (
CustomerId NVARCHAR(1000),
Orders NVARCHAR(MAX) AS JSON
) j
CROSS APPLY OPENJSON(j.Orders)
WITH (
OrderId NVARCHAR(1000),
OrderManifests NVARCHAR(MAX) AS JSON
) o
CROSS APPLY OPENJSON(o.OrderManifests)
WITH (
ShippedProductId INT,
ProductId int
) m
WHERE m.ShippedProductId = m.ProductId;
This query returns:
CustomerId | OrderId | ShipedProductId | ProductId
------------+-----------+-------------------+-------------
123456 | 852 | 884 | 884
------------+-----------+-------------------+-------------
123456 | 5681 | 8188 | 8188
I am trying to get a JSON file parsed into a usable format so I can insert it into a SQL table.
The JSON file I have is heavily nested (and I can't get the vendor to change it at this point), and uses the same name at different levels.
I have used the following code, to start off, but it is the multi sections and potentially multiple accounts etc that has me stumped. I know I will probably need to iterate through somehow, but just not sure where to begin.
DECLARE #JSON VARCHAR(MAX)
SELECT #JSON = BulkColumn
FROM OPENROWSET
(BULK 'C:\Users\joshu\Downloads\Cashflow.JSON', SINGLE_CLOB)
AS j
If (ISJSON(#JSON)=1)
Select * FROM OPENJSON (#JSON,'$.data')
with
(
[id] nvarchar(50),
[title] nvarchar(50),
[sections] nvarchar(max) as json
) data_Structure
cross apply openjson(data_structure.sections, '$')
with (
[income] nvarchar(max) as json
) data2
--Income is one type, there should be a loop here
cross apply openjson(data2.income, '$')
Which is getting my down the tree. The last data points that I want to collect are based on this bit of code
DECLARE #JSON VARCHAR(MAX)
SELECT #JSON = BulkColumn
FROM OPENROWSET
(BULK 'C:\Users\joshu\Downloads\Cashflow.JSON', SINGLE_CLOB)
AS j
If (ISJSON(#JSON)=1)
Select * FROM OPENJSON (#JSON,'$.data.sections.income.sections.tracker_1193.sections.tracker_1193_income.rows."b5cfd1ce-bb7f-4f5c-a6b4-12b469ff0b9d".data."2017-06"')
with
(
[date] nvarchar(50),
[value] decimal(18,2))
A sample of the JSON is here
"data": {
"id": "cashflow",
"title": "Cashflow Report",
"sections": {
"income": {
"id": "income",
"title": "Income",
"sections": {
"tracker_1193": {
"id": "tracker_1193",
"title": "xxxxxxx",
"sections": {
"tracker_1193_income": {
"id": "tracker_1193_income",
"title": "Income",
"sections": null,
"rows": {
"b5cfd1ce-bb7f-4f5c-a6b4-12b469ff0b9d": {
"account_id": "b5cfd1ce-bb7f-4f5c-a6b4-12b469ff0b9d",
"account_name": "Bobby Calf Sales",
"data": {
"2017-06": {
"date": "2017-06",
"value": 0
},
"2017-09": {
"date": "2017-09",
"value": 4801.36
},
"2017-12": {
"date": "2017-12",
"value": 1997.33
Now the fun part.
The income section is the most complicated;
The First "section" in the JSON data is one of 13.
The Second "section" is variable, as in there could be 1, or 12.
The Third "section is fixed to 3 (income, costs, gross-profit)
The rest is more straight forward
{
"data": {
"id": "cashflow",
"title": "Cashflow Report",
"sections": {
//other data ahead of this
"operating_expenses": {
"id": "operating_expenses",
"title": "Operating Expenses",
"sections": {
"operating_expenses_animal_health_animal": {
"id": "operating_expenses_animal_health_animal",
"title": "Animal Health",
"sections": null,
"rows": {
"0de82545-be93-4fb5-9d20-fa076af48e40": {
"account_id": "0de82545-be93-4fb5-9d20-fa076af48e40",
"account_name": "Animal Health - Minerals",
"data": {
"2019-07": {
"date": "2019-07",
"value": 5827.93
}
}
},
"9ba329a6-f77e-4779-9d79-28dd20465b9c": {
"account_id": "9ba329a6-f77e-4779-9d79-28dd20465b9c",
"account_name": "Animal Health - Other",
"data": {
"2019-07": {
"date": "2019-07",
"value": 663.73
}
}
},
"4f406965-3355-4968-a5ba-519d9706f329": {
"account_id": "4f406965-3355-4968-a5ba-519d9706f329",
"account_name": "Animal Health - Treatments",
"data": {
"2019-07": {
"date": "2019-07",
"value": 8670.1
}
}
},
"79c8ab89-22a2-4c5c-b591-0a3d95a4a95b": {
"account_id": "79c8ab89-22a2-4c5c-b591-0a3d95a4a95b",
"account_name": "Animal Health - Vet",
"data": {
"2019-07": {
"date": "2019-07",
"value": 7645.18
}
}
}
},
"totals": {
"2019-07": {
"date": "2019-07",
"value": 22806.94
}
}
},
Because of the nature of this data, I haven't sorted my SQL table structure yet, but I am imagining it to be something along the lines of the below:
That's about it. I am stuck, need some help/guidance so anything you can do to assist is greatly appreciated
I've made an attempt with your JSON to traverse its data without having to define explicit keys. This only handles the "Income" portion, however, it should get you moving in the right direction to extract your data into SQL server. Note that given "rows" can have multiple values, some data is duplicated.
DECLARE #data nvarchar(MAX) = '{"data":{"id":"cashflow","title":"Cashflow Report","sections":{"income":{"id":"income","title":"Income","sections":{"tracker_1193":{"id":"tracker_1193","title":"xxxxxxx","sections":{"tracker_1193_income":{"id":"tracker_1193_income","title":"Income","sections":null,"rows":{"b5cfd1ce-bb7f-4f5c-a6b4-12b469ff0b9d":{"account_id":"b5cfd1ce-bb7f-4f5c-a6b4-12b469ff0b9d","account_name":"Bobby Calf Sales","data":{"2017-06":{"date":"2017-06","value":0},"2017-09":{"date":"2017-09","value":4801.36},"2017-12":{"date":"2017-12","value":1997.33}}}}}}}}}}}}';
SELECT
section_id,
section_title,
tracker_id,
tracker_title,
string_id,
string_title,
account_id,
account_title,
[period],
period_value
FROM OPENJSON ( #data, '$.data.sections.income' )
WITH (
section_id varchar(50) '$.id',
section_title varchar(50) '$.title',
sections nvarchar(max) '$.sections' AS JSON
) AS dat
OUTER APPLY (
SELECT * FROM OPENJSON ( dat.sections ) AS a
CROSS APPLY (
SELECT * FROM OPENJSON ( a.value )
WITH (
tracker_id varchar(50) '$.id',
tracker_title varchar(50) '$.title',
tracker_income nvarchar(max) '$.sections' AS json
)
) AS b
CROSS APPLY (
SELECT [key] AS income_key, [value] AS income_value FROM OPENJSON ( b.tracker_income )
) AS c
CROSS APPLY (
SELECT * FROM OPENJSON ( c.income_value )
WITH (
string_id varchar(50) '$.id',
string_title varchar(50) '$.title',
income_rows nvarchar(max) '$.rows' AS json
)
) AS d
CROSS APPLY (
SELECT [key] AS account_key, [value] AS account_value FROM OPENJSON ( d.income_rows )
) e
CROSS APPLY (
SELECT * FROM OPENJSON ( e.account_value )
WITH (
account_id varchar(255) '$.account_id',
account_title varchar(50) '$.account_name',
account_data nvarchar(max) '$.data' AS json
)
) f
CROSS APPLY (
SELECT [key] AS [period], JSON_VALUE ( [value], '$.value' ) AS period_value FROM OPENJSON ( f.account_data )
) g
) AS Income;
Returns
+------------+---------------+--------------+---------------+---------------------+--------------+--------------------------------------+------------------+---------+--------------+
| section_id | section_title | tracker_id | tracker_title | string_id | string_title | account_id | account_title | period | period_value |
+------------+---------------+--------------+---------------+---------------------+--------------+--------------------------------------+------------------+---------+--------------+
| income | Income | tracker_1193 | xxxxxxx | tracker_1193_income | Income | b5cfd1ce-bb7f-4f5c-a6b4-12b469ff0b9d | Bobby Calf Sales | 2017-06 | 0 |
| income | Income | tracker_1193 | xxxxxxx | tracker_1193_income | Income | b5cfd1ce-bb7f-4f5c-a6b4-12b469ff0b9d | Bobby Calf Sales | 2017-09 | 4801.36 |
| income | Income | tracker_1193 | xxxxxxx | tracker_1193_income | Income | b5cfd1ce-bb7f-4f5c-a6b4-12b469ff0b9d | Bobby Calf Sales | 2017-12 | 1997.33 |
+------------+---------------+--------------+---------------+---------------------+--------------+--------------------------------------+------------------+---------+--------------+
I'm trying to write a query against following table
create table employees (id int identity(1,1) primary key, content nvarchar(max));
insert into employees (content)
values (
N'{
"name": "John Doe",
"skills": [
{
"language": "SQL",
"years": 3
},
{
"language": "C#",
"years": 4
}
]
}'),
(
N'{
"name": "Alice Smith",
"skills": [
{
"language": "JS",
"years": 5
}
]
}
')
;
What I'm up to is to end up with the following result
+ id + name + language + years +
| 1 | John Doe | SQL | 3 |
| 2 | John Doe | C# | 4 |
| 3 | Alice Smith | JS | 3 |
Here is what I came up with
SELECT * FROM
OPENJSON(JSON_QUERY((SELECT content FROM employees), '$.skills'))
WITH (
Language VARCHAR(50) '$.language',
Years int '$.years'
)
You can check it out in dbfiddle
https://dbfiddle.uk/?rdbms=sqlserver_2019&fiddle=8ed626ed8441ea6ccc62b8feec99c227
It works when content column contains a single row, but fails when there is more.
Subquery returned more than 1 value. This is not permitted when the subquery
follows =, !=, <, <= , >, >= or when the subquery is used as an expression.
I understand what it means, yet I cannot figure out how to get to it in different way.
I've been trying with OPENJSON and JSON_QUERY
The reason for this error is the JSON_QUERY((SELECT content FROM employees), '$.skills') part of the statement, because JSON_QUERY() expects a JSON string. To solve this error you may try the following statement:
SELECT e.id, j1.name, j2.language, j2.years
FROM employees e
OUTER APPLY OPENJSON(e.content) WITH (
name nvarchar(100) '$.name',
skills nvarchar(max) '$.skills' AS JSON
) j1
OUTER APPLY OPENJSON(j1.skills) WITH (
language nvarchar(100) '$.language',
years int '$.years'
) j2
Result:
id name language years
1 John Doe SQL 3
1 John Doe C# 4
2 Alice Smith JS 5
I'm using MYSql server 8.0.17.
I want to get record with uId= 'UR000001' and also with VIEW = 'Y' from the security column(Shown in Table).
Viewid Security
VW0000000002 {"security": [{"uId": "UR000001", "edit": "N", "view": "Y"}, {"uId": "UR000002", "edit": "N", "view": "Y"}]}
VW0000000013 {"security": [{"uId": "UR000001", "edit": "N", "view": "N"}, {"uId": "UR000002", "edit": "N", "view": "Y"}]}
VW0000000014 {"security": [{"uId": "UR000001", "edit": "N", "view": "Y"}, {"uId": "UR000002", "edit": "N", "view": "Y"}]}
JSON_SEARCH function searches all array elements of the record that I don't want.
Here is the query that I had tried but it returns the result with all matching (uID='UR000001' OR View='Y')
SELECT viewid,
Json_search(`security`, 'one', 'UR000001', NULL, '$.security[*].uId'),
Json_search(`security`, 'one', 'Y', NULL, '$.security[*].view')
FROM vw_viewmaster
WHERE Json_search(`security`, 'one', 'UR000001', NULL, '$.security[*].uId')
AND Json_search(`security`, 'one', 'Y', NULL, '$.security[*].view');
Actual Result:(uID='UR000001' OR View='Y')
VW0000000002 "$.security[0].uId" "$.security[0].view"
VW0000000013 "$.security[0].uId" "$.security[1].view"
VW0000000014 "$.security[0].uId" "$.security[0].view"
Expected result:(uID='UR000001' AND View='Y')
VW0000000002 "$.security[0].uId" "$.security[0].view"
VW0000000014 "$.security[0].uId" "$.security[0].view"
In MySQL 8.0, you can use handy JSON function json_table() to convert a json array to rows. You can then search the resultset.
The following query gives you all viewids whose at least one array element with attribute uId is equal to 'UR000001' and attribute view is 'Y':
select v.viewid
from vw_viewmaster v
where exists (
select 1
from json_table(
v.security -> '$.security',
'$[*]'
columns(
uid varchar(50) path '$.uId',
edit varchar(1) path '$.edit',
view varchar(1) path '$.view'
)
) x
where x.uid = 'UR000001' and x.view = 'Y'
);
For your dataset, this produces:
| viewid |
| ------------ |
| VW0000000002 |
| VW0000000014 |
If you want the details of the matching array object(s), then:
select v.viewid, x.*
from vw_viewmaster v
cross join json_table(
v.security -> '$.security',
'$[*]'
columns(
rowid for ordinality,
uid varchar(50) path '$.uId',
edit varchar(1) path '$.edit',
view varchar(1) path '$.view'
)
) x
where x.uid = 'UR000001' and x.view = 'Y'
As a bonus, rowid gives you the index of the matching object in the JSON array (the first object has index 1).
This yields:
| viewid | rowid | uid | edit | view |
| ------------ | ----- | -------- | ---- | ---- |
| VW0000000002 | 1 | UR000001 | N | Y |
| VW0000000014 | 1 | UR000001 | N | Y |
However please note that if more than one object in the array that satisfies the conditions, the above query would generate more than one row per row in the original table (this is why I used exists in the first query).