I'm convinced this must be answered somewhere but for the life of me I just can't seem to find anything no matter how much I change my search phrases.
I need to select data from two completely independent tables and export the information to JSON. In this case, they're both 1 record in each table.
If I select just 1 at a time and export to JSON, they're 1 record, but when I join the two single records in SQL and then export to JSON, they're 1 record arrays.
Just 1 record SQL Input:
DECLARE #Json nvarchar(max) =
(
SELECT 'Data1' AS [Data1], 'Data2' AS [Data2]
FOR JSON PATH
, INCLUDE_NULL_VALUES
, WITHOUT_ARRAY_WRAPPER
);
SELECT #Json;
GO
Just 1 record JSON Output (note there's no array):
{
"Data1": "Data1",
"Data2": "Data2"
}
2 record SQL Input:
DECLARE #Json nvarchar(max) =
(
SELECT
(
SELECT 'Data1' AS [Data1], 'Data2' AS [Data2]
FOR JSON PATH
, INCLUDE_NULL_VALUES
) AS [Part1]
,
(
SELECT 'Text1' AS [Text1], 'Text2' AS [Text2]
FOR JSON PATH
, INCLUDE_NULL_VALUES
) AS [Part2]
FOR JSON PATH
, WITHOUT_ARRAY_WRAPPER
);
SELECT #Json;
GO
2 record JSON Output (note the inclusion of arrays):
{
"Part1": [
{
"Data1": "Data1",
"Data2": "Data2"
}
],
"Part2": [
{
"Text1": "Text1",
"Text2": "Text2"
}
]
}
I "think" that WITHOUT_ARRAY_WRAPPER is the correct attribute to add which will resolve this but as soon as I add that, I get the entire record as a string:
{
"Part1": "{\"Data1\":\"Data1\",\"Data2\":\"Data2\"}",
"Part2": "{\"Text1\":\"Text1\",\"Text2\":\"Text2\"}"
}
I understand that there's text manipulation methods I can use to get this to work, but I'm hoping for a clean SQL > JSON statement.
I'm currently working on SQL Server 2016 but I can if necessary get a 2017 or 2019 server. Not sure if later SQL handles this better or if it's just my query that needs optimisation.
Edit: My desired output is:
{
"Part1": {
"Data1": "Data1",
"Data2": "Data2"
},
"Part2": {
"Text1": "Text1",
"Text2": "Text2"
}
}
According to the accepted answer of FOR JSON PATH. how to not use escape characters on SQL Server's forum on MSDN:
FOR JSON will escape any text unless if it is generated as JSON result by some JSON function/query. In your example, FOR JSON cannot know do you really want raw JSON or you are just sending some free text that looks like JSON.
Properly defined JSON is generated with FOR JSON (unless if it has WITHOUT_ARRAY_WRAPPER option) or JSON_QUERY. If you wrap your JSON literal with JSON_QUERY it will not be escaped.
This answer got me to try the following code:
DECLARE #Json nvarchar(max) =
(
SELECT
JSON_QUERY((
SELECT 'Data1' AS [Data1], 'Data2' AS [Data2]
FOR JSON PATH
, INCLUDE_NULL_VALUES
, WITHOUT_ARRAY_WRAPPER
)) AS [Part1]
,
JSON_QUERY((
SELECT 'Text1' AS [Text1], 'Text2' AS [Text2]
FOR JSON PATH
, INCLUDE_NULL_VALUES
, WITHOUT_ARRAY_WRAPPER
)) AS [Part2]
FOR JSON PATH
, WITHOUT_ARRAY_WRAPPER
);
SELECT #Json;
As as it turns out - this is working like a charm. Results:
{
"Part1": {
"Data1": "Data1",
"Data2": "Data2"
},
"Part2": {
"Text1": "Text1",
"Text2": "Text2"
}
}
DB<>Fiddle
Update
Look what I found buried in official documentation:
To avoid automatic escaping, provide newValue by using the JSON_QUERY function. JSON_MODIFY knows that the value returned by JSON_MODIFY is properly formatted JSON, so it doesn't escape the value.
Related
I have some columns in my Oracle database that contains json and to extract it's data in a query, I use REGEXP_SUBSTR.
In the following example, value is a column in the table DOSSIER that contains json. The regex extract the value of the property client.reference in that json
SELECT REGEXP_SUBSTR(value, '"client"(.*?)"reference":"([^"]+)"', 1, 1, NULL, 2) FROM DOSSIER;
So if the json looks like this :
[...],
"client": {
"someproperty":"123",
"someobject": {
[...]
},
"reference":"ABCD",
"someotherproperty":"456"
},
[...]
The SQL query will return ABDC.
My problem is that some json have multiple instance of "client", for example :
[...],
"contract": {
"client":"Name of the client",
"supplier": {
"reference":"EFGH"
}
},
[...],
"client": {
"someproperty":"123",
"someobject": {
[...]
},
"reference":"ABCD",
"someotherproperty":"456"
},
[...]
You get the issue, now the SQL query will return EFGH, which is the supplier's reference.
How can I make sure that "reference" is contained in a json object "client" ?
EDIT : I'm on Oracle 11g so I can't use the JSON API and I would like to avoid using third-party package
Assuming you are using Oracle 12c or later then you should NOT use regular expressions and should use Oracle's JSON functions.
If you have the table and data:
CREATE TABLE table_name ( value CLOB CHECK ( value IS JSON ) );
INSERT INTO table_name (
value
) VALUES (
'{
"contract": {
"client":"Name of the client",
"supplier": {
"reference":"EFGH"
}
},
"client": {
"someproperty":"123",
"someobject": {},
"reference":"ABCD",
"someotherproperty":"456"
}
}'
);
Then you can use the query:
SELECT JSON_VALUE( value, '$.client.reference' ) AS reference
FROM table_name;
Which outputs:
REFERENCE
ABCD
db<>fiddle here
If you are using Oracle 11 or earlier then you could use the third-party PLJSON package to parse JSON in PL/SQL. For example, this question.
Or enable Java within the database and then use CREATE JAVA (or the loadjava utility) to add a Java class that can parse JSON to the database and then wrap it in an Oracle function and use that.
I faced similar issue recently. If "reference" is a property that is only present inside "client" object, this will solve:
SELECT reference FROM (
SELECT DISTINCT
REGEXP_SUBSTR(
DBMS_LOB.SUBSTR(
value,
4000
),
'"reference":"(.+?)"',
1, 1, 'c', 1) reference
FROM DOSSIER
) WHERE reference IS NOT null;
You can also try to adapt the regex to your need.
Edit:
In my case, column type is CLOB and that's why I use DBMS_LOB.SUBSTR function there. You can remove this function and pass column directly in REGEXP_SUBSTR.
I have the following JSON in a SQL field in a table:
{
"type": "info",
"date": "2019/11/12 14:28:51",
"state": {
"6ee8587f-3b8c-4e5c-89a9-9f04752607f0": {
"state": "open",
"color": "#0000ff"
}
},
...
}
I query this in MS SQL using the folloing:
SELECT
JSON_VALUE(json_data, '$.type') AS msg_type
,JSON_VALUE(json_data, '$."date"') AS event_date
,JSON_QUERY(json_data, '$.state."6ee8587f-3b8c-4e5c-89a9-9f04752607f0".state') AS json_state
,JSON_QUERY(json_data, '$.state."6ee8587f-3b8c-4e5c-89a9-9f04752607f0".color') AS json_color
FROM
[dbo].[tbl_json_dump]
To get the date (a reserved word) back I have to put the the field name in like $."date"
I cannot seem to get the data back for the state or color fields and I think it has to do with that it is nested under "6ee8587f-3b8c-4e5c-89a9-9f04752607f0" because when I query :
JSON_QUERY(json_data, '$.state."6ee8587f-3b8c-4e5c-89a9-9f04752607f0"') AS json_state
I get the object back -
{"state":"open","color":"#0000ff"}
but using
JSON_QUERY(json_data, '$.state."6ee8587f-3b8c-4e5c-89a9-9f04752607f0".state') AS json_state
it is not working
Any suggestions on what I'm doing wrong??
Just replace JSON_QUERY with JSON_VALUE since you're interested in getting the value.
JSON_QUERY is supposed to return a JSON fragment and designed to work on objects and arrays, not values.
Salman A already provided the answer. Just to add a few points.
JSON_VALUE() - Extracts a Scalar value
JSON_QUERY() - Extracts an object or an array from a JSON string.
If you see the syntax , JSON_QUERY ( expression [ , path ] ) & JSON_VALUE ( expression , path ) , both are more or less except the [] square brackets for path and it means optional. It is because JSON_QUERY() can extract whole JSON field if required.
And on the return types,
JSON_VALUE() returns a JSON fragment of type nvarchar(max)
JSON_QUERY() returns a single text value of type nvarchar(4000)
Overall comparison
DECLARE #data NVARCHAR(4000)
SET #data=N'{
"type": "info",
"date": "2019/11/12 14:28:51",
"state": {
"6ee8587f-3b8c-4e5c-89a9-9f04752607f0": {
"state": "open",
"color": "#0000ff"
}
},
}'
SELECT
JSON_VALUE(#data,'$.state."6ee8587f-3b8c-4e5c-89a9-9f04752607f0"') AS 'JSON_VALUE_FAILED',
JSON_QUERY(#data,'$.state."6ee8587f-3b8c-4e5c-89a9-9f04752607f0"') AS 'JSON_QUERY_SUCCEED',
JSON_VALUE(#data,'$.state."6ee8587f-3b8c-4e5c-89a9-9f04752607f0".state') AS 'JSON_VALUE_SUCCEED',
JSON_QUERY(#data,'$.state."6ee8587f-3b8c-4e5c-89a9-9f04752607f0".state') AS 'JSON_QUERY_SUCCEED';
Check Output here
You may try with another possible approach (more complicated), which parses all nested JSON objects.
Table:
CREATE TABLE Data (
JsonData nvarchar(max)
)
INSERT INTO Data
(JsonData)
VALUES
(N'{
"type": "info",
"date": "2019/11/12 14:28:51",
"state": {
"6ee8587f-3b8c-4e5c-89a9-9f04752607f0": {
"state": "open",
"color": "#0000ff"
},
"6ee8587f-3b8c-4e5c-89a9-9f04752607f1": {
"state": "open",
"color": "#0000ff"
}
}
}')
Statement:
SELECT
j1.[type], j1.[date], j2.[key], j3.state, j3.color
FROM Data d
CROSS APPLY OPENJSON(d.JsonData) WITH (
[type] nvarchar(100) '$.type',
[date] datetime '$.date',
[state] nvarchar(max) '$.state' AS JSON
) j1
CROSS APPLY OPENJSON(j1.state) j2
CROSS APPLY OPENJSON(j2.[value]) WITH (
state nvarchar(10) '$.state',
color nvarchar(10) '$.color'
) j3
Result:
type date key state color
info 12/11/2019 14:28:51 6ee8587f-3b8c-4e5c-89a9-9f04752607f0 open #0000ff
info 12/11/2019 14:28:51 6ee8587f-3b8c-4e5c-89a9-9f04752607f1 open #0000ff
Notes:
If the input JSON has only one key "6ee8587f-3b8c-4e5c-89a9-9f04752607f0" in the "state" JSON object, you may get the value with JSON_VALUE() using the correct path $.state."6ee8587f-3b8c-4e5c-89a9-9f04752607f0".state.
I have a incoming data structure that looks like this:
declare #json nvarchar(max) = '{
"action": "edit",
"data": {
"2077-09-02": {
"Description": "some stuff",
"EffectDate": "2077-1-1"
}
}
}';
To give you a long story short, I think TSQL hates this json structure, because no matter what I have tried, I can't get to any values other than "action".
The {data} object contains another object, {2077-09-02}. "2077-09-02" will always be different. I can't rely on what that date will be.
This works:
select json_value(#json, '$.action');
None of this works when trying to get to the other values.
select json_value(#json, '$.data'); --returns null
select json_value(#json, '$.data[0]'); --returns null
select json_value(#json, 'lax $.data.[2077-09-02].Description');
--JSON path is not properly formatted. Unexpected character '[' is found at position 11.
select json_value(#json, 'lax $.data.2077-09-02.Description');
--JSON path is not properly formatted. Unexpected character '2' is found at position 11.
How do I get to the other values? Is the JSON not perfect enough for TSQL?
It is never a good idea to use the declarative part of a text based container as data. The "2077-09-02" is a valid json key, but hard to query.
You can try this:
declare #json nvarchar(max) = '{
"action": "edit",
"data": {
"2077-09-02": {
"Description": "some stuff",
"EffectDate": "2077-1-1"
}
}
}';
SELECT A.[action]
,B.[key] AS DateValue
,C.*
FROM OPENJSON(#json)
WITH([action] NVARCHAR(100)
,[data] NVARCHAR(MAX) AS JSON) A
CROSS APPLY OPENJSON(A.[data]) B
CROSS APPLY OPENJSON(B.[value])
WITH (Description NVARCHAR(100)
,EffectDate DATE) C;
The result
action DateValue Description EffectDate
edit 2077-09-02 some stuff 2077-01-01
The idea:
The first OPENJSON will return the action and the data.
I use a WITH clause to tell the engine, that action is a simple value, while data is nested JSON
The next OPENJSON dives into data
We can now use B.[key] to get the json key's value
Now we need another OPENJSON to dive into the columns within data.
However: If this JSON is under your control I'd suggest to change its structure.
Use double quotes instead of []. JSON Path uses JavaScript's conventions where a string is surrounded by double quotes. The documentation's example contains this path $."first name".
In this case :
select json_value(#json,'$.data."2077-09-02".Description');
Returns :
some stuff
As for the other calls, JSON_VALUE can only return scalar values, not objects. You need to use JSON_QUERY to extract JSON objects, eg :
select json_query(#json,'$.data."2077-09-02"');
Returns :
{
"Description": "some stuff",
"EffectDate": "2077-1-1"
}
I'm starting to fiddle out how to handle JSON in MSSQL 2016+
I simply created a table having a ID (int) and a JSON (nvarchar) column.
Here are my queries to show the issue:
First query just returns the relational table result, nice and as expected.
SELECT * FROM WS_Test
-- Results:
1 { "name": "thomas" }
2 { "name": "peter" }
Second query returns just the json column as "JSON" created my MSSQL.
Not nice, because it outputs the json column content as string and not as parsed JSON.
SELECT json FROM WS_Test FOR JSON PATH
-- Results:
[{"json":"{ \"name\": \"thomas\" }"},{"json":"{ \"name\": \"peter\" }"}]
Third query gives me two result rows with json column content as parsed JSON, good.
SELECT JSON_QUERY(json, '$') as json FROM WS_Test
-- Results:
{ "name": "thomas" }
{ "name": "peter" }
Fourth query gives me the json column contents as ONE (!) JSON object, perfectly parsed.
SELECT JSON_QUERY(json, '$') as json FROM WS_Test FOR JSON PATH
-- Results:
[{"json":{ "name": "thomas" }},{"json":{ "name": "peter" }}]
BUT:
I don't want to have the "json" property containing the json column content in each array object of example four. I just want ONE array containing the column contents, not less, not more. Like this:
[
{
"name": "peter"
},
{
"name": "thomas"
}
]
How can I archive this with just T-SQL? Is this even possible?
The FOR JSON clause will always include the column names - however, you can simply concatenate all the values in your json column into a single result, and then add the square brackets around that.
First, create and populate sample table (Please save us this step in your future questions):
CREATE TABLE WS_Test
(
Id int,
Json nvarchar(1000)
);
INSERT INTO WS_Test(Id, Json) VALUES
(1, '{ "name": "thomas" }'),
(2, '{ "name": "peter" }');
For SQL Server 2017 or higher, use the built in string_agg function:
SELECT '[' + STRING_AGG(Json, ',') + ']' As Result
FROM WS_Test
For lower versions, you can use for xml path with stuff to get the same result as the string_agg:
SELECT STUFF(
(
SELECT ',' + Json
FROM WS_Test
FOR XML PATH('')
), 1, 1, '[')+ ']' As Result
The result for both of these queries will be this:
Result
[{ "name": "thomas" },{ "name": "peter" }]
You can see a live demo on DB<>Fiddle
I have a column of text type be contain JSON value.
{
"customer": [
{
"details": {
"customer1": {
"name": "john",
"addresses": {
"address1": {
"line1": "xyz",
"line2": "pqr"
},
"address2": {
"line1": "abc",
"line2": "efg"
}
}
}
"customer2": {
"name": "robin",
"addresses": {
"address1": null
}
}
}
}
]
}
How can I extract 'address1' JSON field of column with query?
First I am trying to fetch JSON value then I will go with parsing.
SELECT JSON customer from text_column;
With my query, I get following error.
com.datastax.driver.core.exceptions.SyntaxError: line 1:12 no viable
alternative at input 'customer' (SELECT [JSON] customer...)
com.datastax.driver.core.exceptions.SyntaxError: line 1:12 no viable
alternative at input 'customer' (SELECT [JSON] customer...)
Cassandra version 2.1.13
You can't use SELECT JSON in Cassandra v2.1.x CQL v3.2.x
For Cassandra v2.1.x CQL v3.2.x :
The only supported operation after SELECT are :
DISTINCT
COUNT (*)
COUNT (1)
column_name AS new_name
WRITETIME (column_name)
TTL (column_name)
dateOf(), now(), minTimeuuid(), maxTimeuuid(), unixTimestampOf(), typeAsBlob() and blobAsType()
In Cassandra v2.2.x CQL v3.3.x Introduce : SELECT JSON
With SELECT statements, the new JSON keyword can be used to return each row as a single JSON encoded map. The remainder of the SELECT statment behavior is the same.
The result map keys are the same as the column names in a normal result set. For example, a statement like “SELECT JSON a, ttl(b) FROM ...” would result in a map with keys "a" and "ttl(b)". However, this is one notable exception: for symmetry with INSERT JSON behavior, case-sensitive column names with upper-case letters will be surrounded with double quotes. For example, “SELECT JSON myColumn FROM ...” would result in a map key "\"myColumn\"" (note the escaped quotes).
The map values will JSON-encoded representations (as described below) of the result set values.
If your Cassandra version is 2.1x and below, you can use the Python-based approach.
Write a python script using Cassandra-Python API
Here you have to get your row first and then use python json's loads method, which will convert your json text column value into JSON object which will be dict in Python. Then you can play around with Python dictionaries and extract your required nested keys. See the below code snippet.
from cassandra.cluster import Cluster
from cassandra.auth import PlainTextAuthProvider
import json
if __name__ == '__main__':
auth_provider = PlainTextAuthProvider(username='xxxx', password='xxxx')
cluster = Cluster(['0.0.0.0'],
port=9042, auth_provider=auth_provider)
session = cluster.connect("keyspace_name")
print("session created successfully")
rows = session.execute('select * from user limit 10')
for user_row in rows:
customer_dict = json.loads(user_row.customer)
print(customer_dict().keys()