How to convert json to FoxPro cursor - json

File contains invoice list in json format like:
{
"status": "OK", "statusCode": 200, "messages": null,
"data": [{
"payment_type": "banktransfer", "fine": "0.200000", "quote_id": null, "order_id": null, "prepayment_id": null, "credited_invoices": [],
"interested_party_address_id": 279, "project_id": 875, "currency": "EUR", "owner_id": 3, "date": "2019-03-15", "deadline": "2019-03-20",
},
{
"payment_type": "banktransfer", "fine": "0.30000", "quote_id": null, "order_id": null, "prepayment_id": null, "credited_invoices": [],
"interested_party_address_id": 79, "project_id": 85, "currency": "EUR", "owner_id": 3, "date": "2019-04-15", "deadline": "2019-43-20",
}
.... more same type elements
]
}
How to convert it to FoxPRo cursor ?
Cursor should contain payment_type, fine, quote_id etc columns.
I tried
https://archive.codeplex.com/?p=qdfoxjson
and
http://www.sweetpotatosoftware.com/blog/index.php/2008/12/19/visual-foxpro-json-class-update/
but it looks like they require json to be in different format than my json.
Andrus

First thing is your JSON is invalid. There shouldn't be a hanging comma after the "deadline" date values.
The second thing is that there is no mechanism to tell VFP how to map elements to fields. So you have to do it yourself.
To do this I would use the njJson library, specifically nfJsonRead.
Assuming your JSON is in file 'test.json' and you have nfJsonRead.prg then for example:
Close All
Clear All
Clear
Set Procedure To nfJsonRead additive
* -- Add the other fields as appropriate.
Create Cursor mydata (payment_type c(20), fine n(12, 2))
loJson = nfjsonread(FileToStr("test.json"))
For each loData in loJson.data
Insert into mydata values (loData.payment_type, Val(loData.fine))
EndFor

Just a quick something I came up with for you. If I had more stuff in similar format, I would have done some cursor/table structure of "looking for begin delimiter", "looking for end delimiter", "final data type", etc... then looping through for each part. Here, I actually am showing how each part is being extracted and how to get one data row at a time.
** Example to put into a string called "json", no other context to json
TEXT TO json noshow
{
"status": "OK", "statusCode": 200, "messages": null,
"data": [{1
"payment_type": "banktransfer", "fine": "0.200000", "quote_id": null, "order_id": null, "prepayment_id": null, "credited_invoices": [],
"interested_party_address_id": 279, "project_id": 875, "currency": "EUR", "owner_id": 3, "date": "2019-03-15", "deadline": "2019-03-20",
},
{2
"payment_type": "banktransfer", "fine": "0.30000", "quote_id": null, "order_id": null, "prepayment_id": null, "credited_invoices": [],
"interested_party_address_id": 79, "project_id": 85, "currency": "EUR", "owner_id": 3, "date": "2019-04-15", "deadline": "2019-43-20",
}
.... more same type elements
]
}
ENDTEXT
** Strip the "Status" component ang get just the Data
json = SUBSTR( json, ATC( ["data":], json ))
CREATE CURSOR C_ImportRecs;
( PayType c(20),;
Fine n(10,6),;
QuoteID i,;
OrderID i,;
PrepayID i,;
Invoices c(20),;
IntPartyAddrID i,;
ProjectID i,;
TransCurrency c(3),;
OwnerID i,;
DatePart c(15),;
TransDate d,;
DeadPart c(15),;
Deadline d )
** scatter all fields to an object which will have fields by same name
SCATTER NAME newRec
** start counter to 1
nRow = 1
DO while .t.
** Look within the json string for the "nRow" instance of data starting / ending with { / }
oneData = STREXTRACT( json, "{", "}", nRow )
** if no more entries, get out
IF EMPTY( ALLTRIM( oneData ))
exit
ENDIF
** if memory variables are same name as cursor/table columns, simplifies insert
newRec.PayType = STREXTRACT( oneData, ["payment_type": "], ["] )
newRec.Fine = VAL( STREXTRACT( oneData, ["fine": "], ["] ))
** not including closing quote as ending delimiter since null allowed, looking for the "," after the IDs
newRec.QuoteID = INT( VAL( STREXTRACT( oneData, ["quote_id": ], [,] ) ))
newRec.OrderID = INT( VAL( STREXTRACT( oneData, ["order_id": ], [,] ) ))
newRec.PrepayID = INT( VAL( STREXTRACT( oneData, ["prepayment_id": ], [,] ) ))
newRec.Invoices = STREXTRACT( oneData, ["credited_invoices": ], [,] )
newRec.IntPartyAddrID = INT( VAL( STREXTRACT( oneData, ["interested_party_address_id": ], [,] ) ))
newRec.ProjectID = INT( VAL( STREXTRACT( oneData, ["project_id": ], [,] ) ))
** Currency is reserved word.
newRec.TransCurrency = STREXTRACT( oneData, ["currency": "], ["] )
newRec.OwnerID = INT( VAL( STREXTRACT( oneData, ["owner_id": ], [,] ) ))
** Date is reserved word
newRec.DatePart = ""
newRec.DatePart = STREXTRACT( oneData, ["date": "], ["] )
** Nice for macro substitution. Take incoming date format of ex: "2019-03-15"
** and change to "2019,03,15" This is because the DATE() function expects
** 3 parameters for YEAR, MONTH, DAY
tryDate = STRTRAN( newRec.DatePart, "-", "," )
** Now, by using macro via "&" forces the line below to immediately run as if
** run as if it was DATE( 2019,03,15) and create an actual date value
TRY
newRec.TransDate = DATE( &tryDate )
CATCH
newRec.TransDate = CTOD( "" )
ENDTRY
** same for deadline
newRec.DeadPart = STREXTRACT( oneData, ["deadline": "], ["] )
tryDate = STRTRAN( newRec.DeadPart, "-", "," )
TRY
** your second record has a month 43 which is an invalid month
newRec.Deadline = DATE( &tryDate )
CATCH
newRec.Deadline = CTOD( "" )
ENDTRY
** Since the cursor "C_ImportRecs" is already the active focus,
** we can just insert from memory variables
INSERT INTO C_ImportRecs FROM NAME newRec
** try if any more records
nRow = nRow + 1
ENDDO
BROWSE NORMAL NOCAPTIONS NOWAIT

Related

Convert the contents of a SQL Server Column into a JSON Format

I'm having a SQL Server Table with a column named 'Filter'.
Below are the SQL Server Scripts to create a sample table:
CREATE TABLE dbo.TestJSON
(
TestJSONID INT IDENTITY(1,1),
[Filter] NVARCHAR(4000)
)
INSERT INTO dbo.TestJSON ([Filter])
VALUES ('$WYTS IN (''Control'', ''Machine'', ''Power'', ''DSP'', ''NM'', ''Digital'', ''AI'')')
Now my target is to convert the contents of the column Filter into the following JSON Format:
"conditions":{
"condition":"AND",
"rules":[
{
"condition":"AND",
"operator":"IN",
"value":[
"Control",
"Machine",
"Power",
"DSP",
"NM",
"Digital",
"AI"
],
"type":"string"
}
]
}
How can I achieve this?
Any help is going to be highly appreciated.
Thanks in advance. :)
Here's one option
Example
Select [conditions.condition]='AND'
,[conditions.rules] = json_query(
(Select condition='AND'
,operator ='IN'
,value = json_query('['+replace(stuff(stuff(Filter,charindex(')',Filter),len(Filter),''),1,charindex('(',Filter),''),'''','"')+']')
,type = 'string'
For JSON Path )
)
From TestJSON
For JSON Path,Without_Array_Wrapper
Results
{
"conditions": {
"condition": "AND",
"rules": [
{
"condition": "AND",
"operator": "IN",
"value": [
"Control",
"Machine",
"Power",
"DSP",
"NM",
"Digital",
"AI"
],
"type": "string"
}
]
}
}
If By Chance You Need to Escape the String
Select [conditions.condition]='AND'
,[conditions.rules] = json_query(
(Select condition='AND'
,operator ='IN'
,value = json_query('['+replace(stuff(stuff(B.S,charindex(')',B.S),len(B.S),''),1,charindex('(',B.S),''),'''','"')+']')
,type = 'string'
For JSON Path )
)
From TestJSON A
Cross Apply ( values ( string_escape(Filter,'JSON') ) ) B(S)
For JSON Path,Without_Array_Wrapper

Remove elements from a JSON array, with no key

I'm looking for a way that I can remove elements from a JSON array in SQL Server, I tried using JSON_MODIFY/OPENPATH, but I can't seem to get the path parameter correct.
Here is an example of what I want to do is convert:
[{"FieldName":"OrderStatusTypeId","FieldType":"Int32","ValueBefore":"8","ValueAfter":"10","Action":2},{"FieldName":"PatientReferenceNumber","FieldType":"String","ValueBefore":"","ValueAfter":"","Action":2}]
To be:
[{"FieldName":"OrderStatusTypeId","FieldType":"Int32","ValueBefore":"8","ValueAfter":"10","Action":2}]
Basically, I want to remove any element of the array, whereby the Action is 2 and the ValueBefore and the ValueAfter fields are the same.
Here is what I'm attempting at the moment, as a test, but I keep getting the error below:
DECLARE #JSONData AS NVARCHAR(4000)
SET #JSONData = N'[{"FieldName":"OrderStatusTypeId","FieldType":"Int32","ValueBefore":"8","ValueAfter":"10","Action":2},{"FieldName":"PatientReferenceNumber","FieldType":"String","ValueBefore":"","ValueAfter":"","Action":2},{"FieldName":"PoNumber","FieldType":"String","ValueBefore":"","ValueAfter":"","Action":2},{"FieldName":"ReferringPhysicianName","FieldType":"String","ValueBefore":"","ValueAfter":"","Action":2},{"FieldName":"InsuranceProvider","FieldType":"String","ValueBefore":"","ValueAfter":"","Action":2},{"FieldName":"TreatmentId","FieldType":"String","ValueBefore":"","ValueAfter":"","Action":2},{"FieldName":"RowVersion[6]","FieldType":"Byte","ValueBefore":"10","ValueAfter":"115","Action":2},{"FieldName":"OrderStatusType.Id","FieldType":"Int32","ValueBefore":"8","ValueAfter":"10","Action":2},{"FieldName":"OrderStatusType.Description","FieldType":"String","ValueBefore":"Delivered","ValueAfter":"Failed Logistics","Action":2},{"FieldName":"OrderStatusType.IsSelectable","FieldType":"Boolean","ValueBefore":"False","ValueAfter":"True","Action":2}]'
SELECT #JSONData = JSON_MODIFY(#JSONData, '$',
JSON_QUERY(
(
SELECT *
FROM OPENJSON(#JSONData, '$') WITH (
FieldName nvarchar(1000) '$.FieldName',
FieldType nvarchar(1000) '$.FieldType',
ValueBefore nvarchar(1000) '$.ValueBefore',
ValueAfter nvarchar(1000) '$.ValueAfter',
Action int '$.Action'
)
WHERE Action <> 2 AND ValueBefore <> ValueAfter
FOR JSON PATH
)
)
)
Error:
Msg 13619, Level 16, State 1, Line 4
Unsupported JSON path found in argument 2 of JSON_MODIFY.
The expected output I'm looking for is:
[{"FieldName":"OrderStatusTypeId","FieldType":"Int32","ValueBefore":"8","ValueAfter":"10","Action":2},{"FieldName":"RowVersion[6]","FieldType":"Byte","ValueBefore":"10","ValueAfter":"115","Action":2},{"FieldName":"OrderStatusType.Id","FieldType":"Int32","ValueBefore":"8","ValueAfter":"10","Action":2},{"FieldName":"OrderStatusType.Description","FieldType":"String","ValueBefore":"Delivered","ValueAfter":"Failed Logistics","Action":2},{"FieldName":"OrderStatusType.IsSelectable","FieldType":"Boolean","ValueBefore":"False","ValueAfter":"True","Action":2}]
How can I work out the correct JSON Path value, all the examples I seem to find online don't have an array as the root element of the JSON string.
Note, the order of the elements isn't important.
You don't have to use JSON_MODIFY here. You can just select the data you want as a table, filter it, then re-encode it as JSON.
DECLARE #JSONData AS NVARCHAR(4000)
SET #JSONData = N'[{"FieldName":"OrderStatusTypeId","FieldType":"Int32","ValueBefore":"8","ValueAfter":"10","Action":2},{"FieldName":"PatientReferenceNumber","FieldType":"String","ValueBefore":"","ValueAfter":"","Action":2},{"FieldName":"PoNumber","FieldType":"String","ValueBefore":"","ValueAfter":"","Action":2},{"FieldName":"ReferringPhysicianName","FieldType":"String","ValueBefore":"","ValueAfter":"","Action":2},{"FieldName":"InsuranceProvider","FieldType":"String","ValueBefore":"","ValueAfter":"","Action":2},{"FieldName":"TreatmentId","FieldType":"String","ValueBefore":"","ValueAfter":"","Action":2},{"FieldName":"RowVersion[6]","FieldType":"Byte","ValueBefore":"10","ValueAfter":"115","Action":2},{"FieldName":"OrderStatusType.Id","FieldType":"Int32","ValueBefore":"8","ValueAfter":"10","Action":2},{"FieldName":"OrderStatusType.Description","FieldType":"String","ValueBefore":"Delivered","ValueAfter":"Failed Logistics","Action":2},{"FieldName":"OrderStatusType.IsSelectable","FieldType":"Boolean","ValueBefore":"False","ValueAfter":"True","Action":2}]'
set #JSONData =
(
SELECT *
FROM OPENJSON(#JSONData, '$') WITH (
FieldName nvarchar(1000) '$.FieldName',
FieldType nvarchar(1000) '$.FieldType',
ValueBefore nvarchar(1000) '$.ValueBefore',
ValueAfter nvarchar(1000) '$.ValueAfter',
Action int '$.Action'
)
WHERE not (Action = 2 and ValueBefore = ValueAfter)
FOR JSON PATH
)
and the JSON is
[
{
"FieldName": "OrderStatusTypeId",
"FieldType": "Int32",
"ValueBefore": "8",
"ValueAfter": "10",
"Action": 2
},
{
"FieldName": "RowVersion[6]",
"FieldType": "Byte",
"ValueBefore": "10",
"ValueAfter": "115",
"Action": 2
},
{
"FieldName": "OrderStatusType.Id",
"FieldType": "Int32",
"ValueBefore": "8",
"ValueAfter": "10",
"Action": 2
},
{
"FieldName": "OrderStatusType.Description",
"FieldType": "String",
"ValueBefore": "Delivered",
"ValueAfter": "Failed Logistics",
"Action": 2
},
{
"FieldName": "OrderStatusType.IsSelectable",
"FieldType": "Boolean",
"ValueBefore": "False",
"ValueAfter": "True",
"Action": 2
}
]

SQL Error: Subquery returned more than 1 value while JSON parsing

My database has json (ExtraData) persisted in it. I want to read the json and extract some file names stored there. In this json there are some image filenames persisted:
"UploadedPictures": [
{
"Type": 0,
"Filename": "9d349bf8222c4fb6a5fe883dbcd9b7b8.jpeg",
"Description": "Image 1"
},
{
"Type": 0,
"Filename": "b2d296a4ee6c49ce80bad90339bdeec1.jpeg",
"Description": "Image 2"
},
{
"Type": 0,
"Filename": "d088f43c1a8043cd82533f544dbc7b21.jpeg",
"Description": "Image 3"
}
],
I want to read these filenames, by I get the following error:
Subquery returned more than 1 value. This is not permitted when the subquery follows =, !=, <, <= , >, >= or when the subquery is used as an expression.
My SQL query:
/****** Script for SelectTopNRows command from SSMS ******/
SELECT TOP (1000)
[ID]
,[CreatedAt]
,[PhoneNumber]
,[FirstName]
,[LastName]
,[HouseNumber]
,[City]
,[PostalCode]
,[Street]
,[ExtraData]
,(SELECT T.Filename
FROM OpenJson(JSON_QUERY([ExtraData], '$.UploadedPictures'))
WITH (Filename varchar(200) '$.Filename') T ) AS FileName
FROM
[LD].[dbo].[Database]
However when I do a TOP(1) I get the first file name:
/****** Script for SelectTopNRows command from SSMS ******/
SELECT TOP (1000)
[ID]
,[CreatedAt]
,[PhoneNumber]
,[FirstName]
,[LastName]
,[HouseNumber]
,[City]
,[PostalCode]
,[Street]
,[ExtraData]
,(SELECT TOP(1) T.Filename
FROM OpenJson(JSON_QUERY([ExtraData], '$.UploadedPictures'))
WITH (Filename varchar(200) '$.Filename') T) AS FileName
FROM
[LD].[dbo].[Database]
How can I get all the file names?
If you want to parse the stored JSON and extract the filenames, you need a different statement:
Table:
CREATE TABLE Data (ExtraData varchar(max))
INSERT INTO Data (ExtraData) VALUES ('{"UploadedPictures": [
{
"Type": 0,
"Filename": "9d349bf8222c4fb6a5fe883dbcd9b7b8.jpeg",
"Description": "Image 1"
},
{
"Type": 0,
"Filename": "b2d296a4ee6c49ce80bad90339bdeec1.jpeg",
"Description": "Image 2"
},
{
"Type": 0,
"Filename": "d088f43c1a8043cd82533f544dbc7b21.jpeg",
"Description": "Image 3"
}
]}')
Statement to parse the stored JSON:
SELECT
-- Include additional columns
-- d.*,
j.*
FROM Data d
CROSS APPLY OPENJSON(d.ExtraData, '$.UploadedPictures') WITH (
Filename varchar(200) '$.Filename'
) j
Result:
Filename
9d349bf8222c4fb6a5fe883dbcd9b7b8.jpeg
b2d296a4ee6c49ce80bad90339bdeec1.jpeg
d088f43c1a8043cd82533f544dbc7b21.jpeg
If you want to concatenate the filenames, for SQL Server 2016, you need to use FOR XML PATH:
SELECT
--d.*,
a.Filenames
FROM Data d
CROSS APPLY (SELECT STUFF(
(
SELECT CONCAT(',', Filename)
FROM OPENJSON(d.ExtraData, '$.UploadedPictures') WITH (
Filename varchar(200) '$.Filename'
)
FOR XML PATH(''), TYPE
).value('.', 'varchar(max)'), 1, 1, '')
) a (Filenames)

What is the best way to import selective properties from JSON into a sql server table

I have a json file which contains data I need to import into a table, the problem is I only want the Latitude, Longitude and preferredGazzeteerName properties.
Here is how my json data looks like :
[
{
"MRGID": 2375,
"gazetteerSource": "Nomenclature des espaces maritimes/List of maritime areas",
"placeType": "Strait",
"latitude": 51.036666666667,
"longitude": 1.5486111111111,
"minLatitude": 49.71788333,
"minLongitude": 0.238905863,
"maxLatitude": 51.78156033,
"maxLongitude": 2.744909289,
"precision": 21000,
"preferredGazetteerName": "Dover Strait",
"preferredGazetteerNameLang": "English",
"status": "standard",
"accepted": 2375
},
{
"MRGID": 2376,
"gazetteerSource": "The Times comprehensive atlas of the world. 10th ed. Times Books: London, UK. ISBN 0-7230-0792-6. 67, 220, 124 plates pp.,",
"placeType": "Strait",
"latitude": 54.604722222222,
"longitude": 11.220833333333,
"minLatitude": null,
"minLongitude": null,
"maxLatitude": null,
"maxLongitude": null,
"precision": 40000,
"preferredGazetteerName": "Femer Baelt",
"preferredGazetteerNameLang": "English",
"status": "standard",
"accepted": 2376
}]
and table
The preferredGazetteerName values will be inserted at strait name column.
Here is one approach where we split the JSON string with a delimiter of '{' then it just becomes a matter of string manipulation
Example
Declare #S varchar(max) ='... your JSON String ...'
Select Name = left(Name,charindex(',',Name+',')-1)
,Lat = try_convert(float,left(Lat,charindex(',',Lat+',')-1))
,Lng = try_convert(float,left(Lng,charindex(',',Lng+',')-1))
From (
Select Name = ltrim(replace(substring(RetVal,nullif(charindex('"preferredGazetteerName"',RetVal),0)+25,75),'"',''))
,Lat = ltrim(substring(RetVal,nullif(charindex('"latitude"',RetVal),0) +11,25))
,Lng = ltrim(substring(RetVal,nullif(charindex('"longitude"',RetVal),0)+12,25))
From [dbo].[tvf-Str-Parse](#S,'{')
) A
Where Name is not null
Returns
Name Lat Lng
Dover Strait 51.036666666667 1.5486111111111
Femer Baelt 54.604722222222 11.220833333333
The Split/Parse TVF if Interested
CREATE FUNCTION [dbo].[tvf-Str-Parse] (#String varchar(max),#Delimiter varchar(10))
Returns Table
As
Return (
Select RetSeq = Row_Number() over (Order By (Select null))
,RetVal = LTrim(RTrim(B.i.value('(./text())[1]', 'varchar(max)')))
From (Select x = Cast('<x>' + replace((Select replace(#String,#Delimiter,'§§Split§§') as [*] For XML Path('')),'§§Split§§','</x><x>')+'</x>' as xml).query('.')) as A
Cross Apply x.nodes('x') AS B(i)
);

JSON Oracle SQL parsing / unnest embedded JSON data in escaped form

Here is my JSON stored in a CLOB column:
select upJSON from myLocations;
{"values":[
{"nameValuePairs":{"upJSON":"{\"mResults\":[0.0,0.0],\"mProvider\":\"fused\",\"mDistance\":0.0,\"mAltitude\":0.0}","id":"1","updated":"2015-03-30 20:28:51"}},
{"nameValuePairs":{"upJSON":"{\"mResults\":[0.0,0.0],\"mProvider\":\"FINDME\",\"mDistance\":0.0,\"mAltitude\":22.2}","id":"2","updated":"2015-03-30 20:28:53"}},
{"nameValuePairs":{"upJSON":"{\"mResults\":[0.0,0.0],\"mProvider\":\"fused\",\"mDistance\":0.0,\"mAltitude\":0.0}","id":"3","updated":"2015-03-30 20:28:55"}},
{"nameValuePairs":{"upJSON":"{\"mResults\":[0.0,0.0],\"mProvider\":\"fused\",\"mDistance\":0.0,\"mAltitude\":0.0}","id":"4","updated":"2015-03-30 20:28:57"}}
]}
(I have inserted newlines for clarity)
Please: What is the SQL (or PL/SQL) needed to select just the value of mProvider, mAltitude, and the id from the 2nd "nameValuePairs"
(= "FINDME" and 22.2 and "2") in the example above)
??
Since you're using 12c you have access to the native JSON parsing (as long as your CLOB column has an is json check constraint).
Some good background is available at:
https://docs.oracle.com/database/121/ADXDB/json.htm#ADXDB6371
If you're JSON looks something like this:
{
"values": [
{
"nameValuePairs": {
"upJSON": {
"mResults": [
"0.0",
"0.0"
],
"mProvider": "fused",
"mDistance": "0.0",
"mAltitude": "22.2"
},
"id": "1",
"updated": "2015-03-30 20:28:51"
}
},
...
...
Although, when I put your snippet from the question into JSONLint it returns:
{
"values": [
{
"nameValuePairs": {
"upJSON": "{\"mResults\":[0.0,0.0],\"mProvider\":\"fused\",\"mDistance\":0.0,\"mAltitude\":0.0}",
"id": "1",
"updated": "2015-03-30 20:28:51"
}
},
{
"nameValuePairs": {
"upJSON": "{\"mResults\":[0.0,0.0],\"mProvider\":\"FINDME\",\"mDistance\":0.0,\"mAltitude\":22.2}",
"id": "2",
"updated": "2015-03-30 20:28:53"
}
},
Something like the following might get you started:
select
upJSON.values
from
myLocations
where
json_value(upJSON, '$nameValuePairs.id' returning varchar2 error on error) = '2';
If you want to limit the query to a single ID, you'll need to add a full-text or function-based index to the JSON column.
https://odieweblog.wordpress.com/2015/04/12/json_table-chaining/#comment-1025
with tmp as (
SELECT /*+ no_merge */ d.*
FROM ulocations ul,
json_table(ul.upjson, '$'
columns(
NESTED PATH '$.values[*].nameValuePairs'
COLUMNS (
updated VARCHAR2(19 CHAR) PATH '$.updated'
, id varchar2(9 char) path '$._id'
, upJSON VARCHAR2(2000 CHAR) PATH '$.upJSON'
)) ) d
--where d.id = '0'
)
select t.updated
, t.id
, jt2.*
from tmp t
, json_table(t.upJSON, '$'
columns mProvider varchar2(5) path '$.mProvider'
, mLongitude number path '$.mLongitude'
) jt2
;