JSON Oracle SQL parsing / unnest embedded JSON data in escaped form - json

Here is my JSON stored in a CLOB column:
select upJSON from myLocations;
{"values":[
{"nameValuePairs":{"upJSON":"{\"mResults\":[0.0,0.0],\"mProvider\":\"fused\",\"mDistance\":0.0,\"mAltitude\":0.0}","id":"1","updated":"2015-03-30 20:28:51"}},
{"nameValuePairs":{"upJSON":"{\"mResults\":[0.0,0.0],\"mProvider\":\"FINDME\",\"mDistance\":0.0,\"mAltitude\":22.2}","id":"2","updated":"2015-03-30 20:28:53"}},
{"nameValuePairs":{"upJSON":"{\"mResults\":[0.0,0.0],\"mProvider\":\"fused\",\"mDistance\":0.0,\"mAltitude\":0.0}","id":"3","updated":"2015-03-30 20:28:55"}},
{"nameValuePairs":{"upJSON":"{\"mResults\":[0.0,0.0],\"mProvider\":\"fused\",\"mDistance\":0.0,\"mAltitude\":0.0}","id":"4","updated":"2015-03-30 20:28:57"}}
]}
(I have inserted newlines for clarity)
Please: What is the SQL (or PL/SQL) needed to select just the value of mProvider, mAltitude, and the id from the 2nd "nameValuePairs"
(= "FINDME" and 22.2 and "2") in the example above)
??

Since you're using 12c you have access to the native JSON parsing (as long as your CLOB column has an is json check constraint).
Some good background is available at:
https://docs.oracle.com/database/121/ADXDB/json.htm#ADXDB6371
If you're JSON looks something like this:
{
"values": [
{
"nameValuePairs": {
"upJSON": {
"mResults": [
"0.0",
"0.0"
],
"mProvider": "fused",
"mDistance": "0.0",
"mAltitude": "22.2"
},
"id": "1",
"updated": "2015-03-30 20:28:51"
}
},
...
...
Although, when I put your snippet from the question into JSONLint it returns:
{
"values": [
{
"nameValuePairs": {
"upJSON": "{\"mResults\":[0.0,0.0],\"mProvider\":\"fused\",\"mDistance\":0.0,\"mAltitude\":0.0}",
"id": "1",
"updated": "2015-03-30 20:28:51"
}
},
{
"nameValuePairs": {
"upJSON": "{\"mResults\":[0.0,0.0],\"mProvider\":\"FINDME\",\"mDistance\":0.0,\"mAltitude\":22.2}",
"id": "2",
"updated": "2015-03-30 20:28:53"
}
},
Something like the following might get you started:
select
upJSON.values
from
myLocations
where
json_value(upJSON, '$nameValuePairs.id' returning varchar2 error on error) = '2';
If you want to limit the query to a single ID, you'll need to add a full-text or function-based index to the JSON column.

https://odieweblog.wordpress.com/2015/04/12/json_table-chaining/#comment-1025
with tmp as (
SELECT /*+ no_merge */ d.*
FROM ulocations ul,
json_table(ul.upjson, '$'
columns(
NESTED PATH '$.values[*].nameValuePairs'
COLUMNS (
updated VARCHAR2(19 CHAR) PATH '$.updated'
, id varchar2(9 char) path '$._id'
, upJSON VARCHAR2(2000 CHAR) PATH '$.upJSON'
)) ) d
--where d.id = '0'
)
select t.updated
, t.id
, jt2.*
from tmp t
, json_table(t.upJSON, '$'
columns mProvider varchar2(5) path '$.mProvider'
, mLongitude number path '$.mLongitude'
) jt2
;

Related

How to get a specific object in an JSON array in MySQL?

I have a JSON column "jobs" that looks like this:
[
{
"id": "1",
"done": "100",
"target": "100",
"startDate": "123123132",
"lastAction": "123123132",
"status": "0"
},
{
"id": "2",
"done": "10",
"target": "20",
"startDate": "2312321",
"lastAction": "2312321",
"status": "1"
}
]
I want to filter the array by object key values. For example: To find all items that have target > done, status != 0 and lastAction is yesterday to get response like this:
[
{
"id": "1",
"done": "19",
"target": "100",
"startDate": "123123132",
"lastAction": "123123132",
"status": "0"
}
]
I know I can extract the data to a JSON_TABLE() to do the filtering but I don't get the original object back(unless I recreate it back) and the solution is not dynamic.
Can this kind of array filtering can really be done in MySQL?
SELECT JSON_PRETTY(JSON_EXTRACT(jobs.jobs, CONCAT('$[', j.rownum-1, ']'))) AS object
FROM jobs
CROSS JOIN JSON_TABLE(
jobs.jobs, '$[*]' COLUMNS(
rownum for ordinality,
done int path '$.done',
target int path '$.target',
status int path '$.status'
)
) as j
WHERE j.target > j.done AND j.status != 0;
You also mentioned a condition on lastAction, but the example values you gave are not valid dates, so I'll leave that enhancement to you. The example above demonstrates the technique.
Yes it is possible to do it using the JSON_EXTRACT and JSON_SEARCH functions.
Let's say your table is named tbl_Jobs and the jobs column is of type JSON.
SELECT * FROM tbl_Jobs
WHERE JSON_EXTRACT(jobs, "$[*].target") = JSON_EXTRACT(jobs, "$[*].done")
AND JSON_EXTRACT(jobs, "$[*].status") != 0
AND JSON_SEARCH(jobs, 'one', DATE_SUB(CURDATE(), INTERVAL 1 DAY), NULL, "$[*].lastAction") IS NOT NULL

Retrieve JSON from sql

My json format in one of the sql columns "jsoncol" in the table "jsontable" is like below.
Kindly help me to get this data using JSON_QUERY or JSON_VALUE
Please pay attention to the brackets and double quotes in the key value pairs...
{
"Company": [
{
"Info": {
"Address": "123"
},
"Name": "ABC",
"Id": 999
},
{
"Info": {
"Address": "456"
},
"Name": "XYZ",
"Id": 888
}
]
}
I am trying to retrieve all the company names using sql query. Thanks in advance
You can use:
SELECT j.name
FROM table_name t
CROSS APPLY JSON_TABLE(
t.value,
'$.Company[*]'
COLUMNS(
name VARCHAR2(200) PATH '$.Name'
)
) j
Which, for the sample data:
CREATE TABLE table_name (
value CLOB CHECK (value IS JSON)
);
INSERT INTO table_name (value)
VALUES ('{
"Company": [
{
"Info": {
"Address": "123"
},
"Name": "ABC",
"Id": 999
},
{
"Info": {
"Address": "456"
},
"Name": "XYZ",
"Id": 888
}
]
}');
Outputs:
NAME
ABC
XYZ
db<>fiddle here
You can easily use JSON_TABLE() function for this case rather provided the DB version is at least 12.1.0.2 such as
SELECT name
FROM jsontable,
JSON_TABLE(jsoncol,
'$' COLUMNS(NESTED PATH '$."Company"[*]'
COLUMNS(name VARCHAR2 PATH '$."Name"')))
Demo

Convert the contents of a SQL Server Column into a JSON Format

I'm having a SQL Server Table with a column named 'Filter'.
Below are the SQL Server Scripts to create a sample table:
CREATE TABLE dbo.TestJSON
(
TestJSONID INT IDENTITY(1,1),
[Filter] NVARCHAR(4000)
)
INSERT INTO dbo.TestJSON ([Filter])
VALUES ('$WYTS IN (''Control'', ''Machine'', ''Power'', ''DSP'', ''NM'', ''Digital'', ''AI'')')
Now my target is to convert the contents of the column Filter into the following JSON Format:
"conditions":{
"condition":"AND",
"rules":[
{
"condition":"AND",
"operator":"IN",
"value":[
"Control",
"Machine",
"Power",
"DSP",
"NM",
"Digital",
"AI"
],
"type":"string"
}
]
}
How can I achieve this?
Any help is going to be highly appreciated.
Thanks in advance. :)
Here's one option
Example
Select [conditions.condition]='AND'
,[conditions.rules] = json_query(
(Select condition='AND'
,operator ='IN'
,value = json_query('['+replace(stuff(stuff(Filter,charindex(')',Filter),len(Filter),''),1,charindex('(',Filter),''),'''','"')+']')
,type = 'string'
For JSON Path )
)
From TestJSON
For JSON Path,Without_Array_Wrapper
Results
{
"conditions": {
"condition": "AND",
"rules": [
{
"condition": "AND",
"operator": "IN",
"value": [
"Control",
"Machine",
"Power",
"DSP",
"NM",
"Digital",
"AI"
],
"type": "string"
}
]
}
}
If By Chance You Need to Escape the String
Select [conditions.condition]='AND'
,[conditions.rules] = json_query(
(Select condition='AND'
,operator ='IN'
,value = json_query('['+replace(stuff(stuff(B.S,charindex(')',B.S),len(B.S),''),1,charindex('(',B.S),''),'''','"')+']')
,type = 'string'
For JSON Path )
)
From TestJSON A
Cross Apply ( values ( string_escape(Filter,'JSON') ) ) B(S)
For JSON Path,Without_Array_Wrapper

SQL query to return Json including array

I have a relational database (SQL Server) with normal data.
I am trying to build something with this format:
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"id": "1",
"properties": { "address": "2" },
"geometry": {
"type": "Point",
"coordinates": [36.33456, 59.523456]
}
},
{
"type": "Feature",
"id": "2",
"properties": { "address": "151" },
"geometry": {
"type": "Point",
"coordinates": [36.33456, 59.523456]
}
}]
}
So far I have written this query:
select top 10
'Feature' as [type],
m.Id as id, m.Address as 'properties.address',
'Point' as 'geometry.type',
'[' + m.location + ']' as 'geometry.coordinates'
from
Buildings m
where
m.Location is not null
and m.Location <> ''
for json path, root('features')
But what I receive in action is like:
{
"features": [{
"type": "Feature",
"id": 250343,
"properties": {
"address": "there"
},
"geometry": {
"type": "Point",
"coordinates": "[5714843008,3363769468.235179]"
}
}, {
"type": "Feature",
"id": 266306,
"properties": {
"address": "here"
},
"geometry": {
"type": "Point",
"coordinates": "[36.38449104993326,59.48238372802735]"
}
}}
How can I add "type": "FeatureCollection", before the root?
I want the coordinate part to hold an array of 2 numbers, but in my current code it is a string holding the array. How can I achieve an array?
It's difficult without test data, but I think that you can build the expected JSОN output using the following statement. You need one more FOR JSON PATH (to generate the outer JSON object) and a JSON_QUERY call (to return a JSON array of scalar values instead of text holding the array):
Table:
CREATE TABLE Buildings (
Id int,
Address varchar(100),
Location varchar(100)
)
INSERT INTO Buildings (Id, Address, Location)
VALUES
(250343, 'there', '5714843008,3363769468.235179'),
(266306, 'here', '36.38449104993326,59.48238372802735')
Statement:
SELECT
[type] = 'FeatureCollection',
[features] = JSON_QUERY((
select top 10
'Feature' as [type],
m.Id as id, m.Address as 'properties.address',
'Point' as 'geometry.type',
JSON_QUERY('[' + m.location + ']') as 'geometry.coordinates'
from
Buildings m
where
m.Location is not null
and m.Location <> ''
for json path
))
FOR JSON PATH, WITHOUT_ARRAY_WRAPPER
Result:
{
"type":"FeatureCollection",
"features":[
{
"type":"Feature",
"id":250343,
"properties":{
"address":"there"
},
"geometry":{
"type":"Point",
"coordinates":[
5714843008,
3363769468.235179
]
}
},
{
"type":"Feature",
"id":266306,
"properties":{
"address":"here"
},
"geometry":{
"type":"Point",
"coordinates":[
36.38449104993326,
59.48238372802735
]
}
}
]
}
I figured a way to build custom JSON using STUFF (GROUP_CONCAT in MySQL) and XML.
See if this works for you:
1 Answer to question 1 - Note, this is a TWO-STEP PROCESS:
a) VARIABLE TO HOLD QUERY DATA
DECLARE #QUERY_DATA varchar(8000)
b) CUSTOM QUERY TO FETCH QUERY DATA FORMATTED FOR INNER JSON METADATA (all in one line)
SET #QUERY_DATA = (
SELECT STUFF(
(
SELECT ',' + CONCAT('{"type": "Feature","id": ', m.Id, ',"properties": {"address": "', m.Address, '"},"geometry": {"type": "Point","coordinates": [', m.location, ']}}'
)
FROM
Buildings m
WHERE
m.Location is not null
and m.Location <> ''
FOR XML PATH(''), TYPE).value('.', 'NVARCHAR(MAX)'), 1, 1, '')
+)
c) FINAL OUTPUT - Concatenate / Join your JSON
SELECT CONCAT('{ ''type'':''FeatureCollection''','','',',''FEATURES'': [ ', #QUERY_DATA, ' ] }')
2 - ANSWER TO QUESTION 2 with CONSOLIDATED solution using separated X and Y coordinates as two INTEGER Type elements
{''X'':', CAST(LEFT( m.location, CHARINDEX(',', m.location)-1) as int), ',''Y'':', cast(RIGHT( m.location, LEN( m.location)-CHARINDEX(',', m.location)) as int),'}
Therefore, when we add this together, your new query would look like the following:
a) VARIABLE TO HOLD QUERY DATA
+DECLARE #QUERY_DATA varchar(8000)
b) CUSTOM QUERY TO FETCH QUERY DATA FORMATTED FOR INNER JSON METADATA (all in one line), including splitting coordinates into two INTEGER Type elements
SET #QUERY_DATA = (
SELECT STUFF(
(
SELECT ',' + CONCAT('{"type": "Feature","id": ', m.Id, ',"properties": {"address": "', m.Address, '"},"geometry": {"type": "Point","coordinates": [ {''X'':', CAST(LEFT( m.location, CHARINDEX(',', m.location)-1) as int), ',''Y'':', cast(RIGHT( m.location, LEN( m.location)-CHARINDEX(',', m.location)) as int),'}]}}'
)
FROM
Buildings m
WHERE
m.Location is not null
and m.Location <> ''
FOR XML PATH(''), TYPE).value('.', 'NVARCHAR(MAX)'), 1, 1, '')
+)
c) FINAL OUTPUT - Concatenate / Join your OUTER and INNER JSON
SELECT CONCAT('{ ''type'':''FeatureCollection''','','',',''FEATURES'': [ ', #QUERY_DATA, ' ] }')

Issue in JOIN query in apache drill

File stored in Hive:
[
{
"occupation": "guitarist",
"fav_game": "football",
"name": "d1"
},
{
"occupation": "dancer",
"fav_game": "chess",
"name": "k1"
},
{
"occupation": "traveller",
"fav_game": "cricket",
"name": "p1"
},
{
"occupation": "drummer",
"fav_game": "archery",
"name": "d2"
},
{
"occupation": "farmer",
"fav_game": "cricket",
"name": "k2"
},
{
"occupation": "singer",
"fav_game": "football",
"name": "s1"
}
]
CSV file in hadoop:
name,age,city
d1,23,delhi
k1,23,indore
p1,23,blore
d2,25,delhi
k2,30,delhi
s1,25,delhi
I queried them individually, it's working fine. Then, I tried join query:
select * from hdfs.`/demo/distribution.csv` d join hive.demo.`user_details` u on d.name = u.name
I got the following issue:
org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: DrillRuntimeException: Join only supports implicit casts between 1. Numeric data 2. Varchar, Varbinary data 3. Date, Timestamp data Left type: INT, Right type: VARCHAR. Add explicit casts to avoid this error Fragment 0:0 [Error Id: b01db9c8-fb35-4ef8-a1c0-31b68ff7ae8d on IMPETUS-DSRV03.IMPETUS.CO.IN:31010]
Please refer this https://drill.apache.org/docs/data-type-conversion/
We need to do explicit typecasting to deal with such scenario.
Consider we have a JSON file employee.json and a csv file sample.csv. In order to query on both at the same time , in one query we need to do type casting.
0: jdbc:drill:zk=local> select emp.employee_id, dept.department_description, phy.columns[2], phy.columns[3] FROM cp.`employee.json` emp , cp.`department.json` dept, dfs.`/tmp/sample.csv` phy where CAST(emp.employee_id AS INT) = CAST(phy.columns[0] AS INT) and emp.department_id = dept.department_id;
Here we are typecasting CAST(emp.employee_id AS INT) = CAST(phy.columns[0] AS INT) so that equality does not fail.
Refer this for more detail:- http://www.devinline.com/2015/11/apache-drill-setup-and-SQL-query-execution.html#multiple_src
You need to cast even though by default it has taken varchar. Try this:
select * from hdfs.`/demo/distribution.csv` d join hive.demo.`user_details` u on cast(d.name as VARCHAR) = cast(u.name as VARCHAR)
But you cannot refer to column name directly from csv. you need to consider columns[0] for name.