I have data and series of validation (Business logic) before I can use it further. Like following :-
{
"site" : "UK",
"domain" : "xyz.UK",
"currency" : "DOLLAR",
"type" : "unlimited",
"limit" : ""
}
{
"site" : "UK",
"domain" : "xyz.UK",
"currency" : "EURO",
"type" : "limited",
"limit" : "100"
}
{
"site" : "US",
"domain" : "xyz.COM",
"currency" : "DOLLAR",
"type" : "limited",
"limit" : ""
}
Validations FLOW is like
Domain validation
Site = us >> domain = .com
Site = uk >> domain = .uk
Currency validation
Site = us >> currency = DOLLAR
Site = uk >> currency = EURO
Type validation
Site = us >> type = "only limited"
Site = us >> type = "limited/unlimited"
type = limited >> limit > 0
type = unlimited >> limit == null
So only second JSON will pass the validations. First one have wrong Currency and 3 one limit is null.
More fields will keep on adding and the validations will keep on increasing. I dont want to put the validation in JAVA code
as in I have to change the code again and again. Is there any other way to save the validations/rules and there sequence in DB and
fetch the validation/rules from DB to validate the data. If I need to change or add more I can modify DB.
Assuming that the financial data set is available as a Java bean (and that there is a typo US/UK), the third validation can be written as:
rule "check type"
when
Data( site == "US" && type == "limited ||
site == "UK" && type == "limited || == "unlimited,
type == "limited" && limit != null && limit > 0 ||
type == "unlimited" && limit == null )
then
// Data's type and limit is OK
end
This rule would "pass" the check. For determining what is wrong you'll need more rules, each of which pinpoints some fault.
Related
I have been looking for over 4 days now but I havent been able to find much support on code for lua based json schema compiler. Mainly I have been dealing with
ljsonschema (https://github.com/jdesgats/ljsonschema)
rjson (https://luarocks.org/modules/romaboy/rjson)
But either of the above have not been straight forward to use.
After dealing with issues on the luarocks, I finally got ljsonschema working but the JSON syntax looks different than normal JSON structure - For ex: equals in place of semi colon, no double quotes for key names etc.
ljsonschema supports
{ type = 'object', properties = {
foo = { type = 'string' },
bar = { type = 'number' },},}
I require :
{ "type" : "object",
"properties" : {
"foo" : { "type" : "string" },
"bar" : { "type" : "number" }}}
With rjson there is an issue with the installation location itself. Though the installation goes fine, it is never able to find the .so file while running the lua code. Plus there is not much development support that I could find.
Please help point in the right direction, in case I am missing something.
I have the json schema & a sample json, I just need a lua code to help write a program around it.
This is to write a custom JSON Validation Plugin for Kong CE.
UPDATED:
I would like the below code to work with ljsonschema:
local jsonschema = require 'jsonschema'
-- Note: do cache the result of schema compilation as this is a quite
-- expensive process
local myvalidator = jsonschema.generate_validator{
"type" : "object",
"properties" : {
"foo" : { "type" : "string" },
"bar" : { "type" : "number" }
}
}
print(myvalidator { "foo":"hello", "bar":42 })
But I get the error : '}' expected (to close '{' at line 5) near ':'
it looks like the argument to generate_validator and myvalidator are lua tables, not raw json strings. You'll want to parse the json first:
> jsonschema = require 'jsonschema'
> dkjson = require('dkjson')
> schema = [[
>> { "type" : "object",
>> "properties" : {
>> "foo" : { "type" : "string" },
>> "bar" : { "type" : "number" }}}
>> ]]
> s = dkjson.decode(schema)
> myvalidator = jsonschema.generate_validator(s)
>
> json = '{ "foo": "bar", "bar": 42 }'
> print(myvalidator(json))
false wrong type: expected object, got string
> print(myvalidator(dkjson.decode(json)))
true
Ok, I think rapidjason came to be helpful:
Refer the link
Here is a sample working code :
local rapidjson = require('rapidjson')
function readAll(file)
local f = assert(io.open(file, "rb"))
local content = f:read("*all")
f:close()
return content
end
local jsonContent = readAll("sampleJson.txt")
local sampleSchema = readAll("sampleSchema.txt")
local sd = rapidjson.SchemaDocument(sampleSchema)
local validator = rapidjson.SchemaValidator(sd)
local d = rapidjson.Document(jsonContent)
local ok, message = validator:validate(d)
if ok then
print("json OK")
else
print(message)
end
I have this JSON (I don't give you the whole thing because it's freaking long but you don't need the rest.)
cve" : {
"data_type" : "CVE",
"data_format" : "MITRE",
"data_version" : "4.0",
"CVE_data_meta" : {
"ID" : "CVE-2018-9991",
"ASSIGNER" : "cve#mitre.org"
},
"affects" : {
"vendor" : {
"vendor_data" : [ {
"vendor_name" : "frog_cms_project",
"product" : {
"product_data" : [ {
"product_name" : "frog_cms",
"version" : {
"version_data" : [ {
"version_value" : "0.9.5"
} ]
}
} ]
}
} ]
}
},
What I want to do is to print the vendor name of this cve.
So, what I did is :
with open("nvdcve-1.0-2018.json", "r") as file:
data = json.load(file)
increment = 0
number_cve = data["CVE_data_numberOfCVEs"]
while increment < int(number_cve):
print (data['CVE_Items'][increment]['cve']['CVE_data_meta']['ID'])
print (',')
print (data['CVE_Items'][increment]['cve']['affects']['vendor']['vendor_data'][0]['vendor_name'])
print ("\n")
increment +=
The reason I did a while is because in the JSON file, there is a lot of CVEs, this is why I did data['CVE_Items'][increment]['cve'] (and this part works fine, the line `print (data['CVE_Items'][increment]['cve']['CVE_data_meta']['ID'] is working well).
My error is in the print (data['CVE_Items'][increment]['cve']['affects']['vendor']['vendor_data'][0]['vendor_name']) line, python returns a list index out of range error.
But if I'm reading this JSON well, vendor_data is an array of 1 column so vendor_name is the ['vendor_data'][0]['vendor_name'] isn't it ?
The only way to parse the vendor_name i found is :
for value in data['CVE_Items'][a]['cve']['affects']['vendor']['vendor_data']:
print (value['vendor_name'])
instead of print (data['CVE_Items'][increment]['cve']['affects']['vendor']['vendor_data'][0]['vendor_name'])
And doing a for just for one iteration is pretty disgusting :s, but at least, value is the data['CVE_Items'][a]['cve']['affects']['vendor']['vendor_data'][0] that I wanted....
Anyone knows something about it ?
Make sure every CVE_Item has an vender_data.
Example:
with open("nvdcve-1.0-2018.json", "r") as file:
data = json.load(file)
increment = 0
number_cve = data["CVE_data_numberOfCVEs"]
while increment < int(number_cve):
print (data['CVE_Items'][increment]['cve']['CVE_data_meta']['ID'])
print (',')
if (len(data['CVE_Items'][increment]['cve']['affects']['vendor']['vendor_data']) > 0) :
print (data['CVE_Items'][increment]['cve']['affects']['vendor']['vendor_data'][0]['vendor_name'])
print ("\n")
increment +=
Thanks to Ron Nabuurs' answer i found that all my vendor_data does not always have a vendor_name. So it is why the for works and not the print.
(the for check if the object is non null, else it stops).
So what I did is :
try:
print (data['CVE_Items'][increment]['cve']['affects']['vendor']['vendor_data'][0]['vendor_name'])
print (',')
except:
pass
In my system from json-call http://192.168.1.6:8080/json.htm?type=devices&rid=89 I get the output below.
{
"ActTime" : 1501360852,
"ServerTime" : "2017-07-29 22:40:52",
"Sunrise" : "05:50",
"Sunset" : "21:28",
"result" : [
{
"AddjMulti" : 1.0,
"AddjMulti2" : 1.0,
"AddjValue" : 0.0,
"AddjValue2" : 0.0,
"BatteryLevel" : 255,
"CustomImage" : 0,
"Data" : "73 Lux",
"Description" : "",
"Favorite" : 1,
"HardwareID" : 4,
"HardwareName" : "Dummies",
"HardwareType" : "Dummy (Does nothing, use for virtual switches only)",
"HardwareTypeVal" : 15,
"HaveTimeout" : true,
"ID" : "82089",
"LastUpdate" : "2017-07-29 21:16:22",
"Name" : "ESP8266C_Licht1",
"Notifications" : "false",
"PlanID" : "0",
"PlanIDs" : [ 0 ],
"Protected" : false,
"ShowNotifications" : true,
"SignalLevel" : "-",
"SubType" : "Lux",
"Timers" : "false",
"Type" : "Lux",
"TypeImg" : "lux",
"Unit" : 1,
"Used" : 1,
"XOffset" : "0",
"YOffset" : "0",
"idx" : "89"
}
],
"status" : "OK",
"title" : "Devices"
}
'Automating' such call by the below scripttime lua-script is aimed at extraction of specific information, to be further used in applications.
The first 11 lines run without problems, but further extraction of the information is a problem.
I have tried various scriptlines to get a solution for A), B), C), D) and E), but either they generate error-report, or they don't give results: see further below for the 'best' trial-scriptlines and related results.
To avoid misunderstanding: those dashed commentlines in the script just below this question with A), B), C), D) and E) are only describing the desired actions/functions and are in no way meant as scriptlines!
Question:
Help requested in the form of better applicable scriptlines for A) till E) in the trialscript at the end of this message, or hints where to find applicable example scriptlines.
-- Lua-script to determine staleness, time-out and value for data from Json-call
print('Start of Timeout-script')
commandArray = {}
TimeOutLimit = 10 -- Allowed timeout in seconds
json = (loadfile "/home/pi/domoticz/scripts/lua/JSON.lua")() -- For Linux
-- json = (loadfile "D:\\Domoticz\\scripts\\lua\\json.lua")() -- For Windows
-- Line 07
local content=assert(io.popen('curl "http://192.168.1.6:8080/json.htm?type=devices&rid=89"')) -- notice double quotes
local list = content:read('*all')
content:close()
local jsonList = json:decode(list)
-- Line 12 Next scriptlines describe desired actions
-- A) Extract ServerTime as numeric value (not as string)
-- B) Extract LastUpdate as numeric value (not as string)
-- Staleness = ServerTime - LastUpdate
-- C) Extract HaveTimeout as boolean (not as string)
-- If HaveTimeout and (Staleness > TimeOutlimit) then
-- Print('TimeOutLimit exceeded by ' .. (Staleness - TimeOutLimit) .. 'seconds')
-- End
-- D) Extract textstring from Type or Data
-- E) Extract numeric value from Data
print('End of Timeout-script')
return commandArray
For lines 11 etc, the following trial-scriptlines gave 'best' results (= no errors):
-- Line 11
-- local Servertime = json:decode(ServerTime)
-- print('Servertime : '..Servertime)
-- Line 14
-- CheckTimeOut =jsonValue.result[1].HaveTimeout -- value from "HaveTimeout", inside "result" bloc number 1 (even if it's the only one)
CurrentServerTime =jsonValue.Servertime -- value from "ServerTime"
CurrentLastUpdate = jsonValue.result[1].LastUpdate
CurrentData = jsonValue.result[1].Data
-- Line 19
print('TimeOut : '..CheckTimeOut)
print('Servertime : '..CurrentServerTime)
print('LastUpdate : '..CurrentLastUpdate)
print('Data-content : '..CurrentData)
print('End of Timeout-script')
return commandArray
Results:
Without dashes before the lines 12 and 13, respectively 15, then the following error reports:
660: nil passed to JSON:decode()
lua:15: attempt to index global 'jsonValue' (a nil value)
With dashes before lines 12, 13 and 15 for the trial-scriptlines shown above, according to the log no errors exist (as demonstrated by the 2 prints)
2017-07-31 16:30:02.520 LUA: Start of Timeout-script
2017-07-31 16:30:02.563 LUA: End of Timeout-script
But why no print-results in the log from Lines 20 till 23?
Not having those print-results makes it difficult to determine next steps in data-extraction, to achieve the objectives described under A) till E).
;-) Error reports generally are more useful information than "no errors, but no results"
Without downloading content over the network, the working script looks like this:
local json = require("json")
local jj = [[
{
"ActTime" : 1501360852,
"ServerTime" : "2017-07-29 22:40:52",
"Sunrise" : "05:50",
"Sunset" : "21:28",
"result" : [
{
"AddjMulti" : 1.0,
"AddjMulti2" : 1.0,
"AddjValue" : 0.0,
"AddjValue2" : 0.0,
"BatteryLevel" : 255,
"CustomImage" : 0,
"Data" : "73 Lux",
"Description" : "",
"Favorite" : 1,
"HardwareID" : 4,
"HardwareName" : "Dummies",
"HardwareType" : "Dummy (Does nothing, use for virtual switches only)",
"HardwareTypeVal" : 15,
"HaveTimeout" : true,
"ID" : "82089",
"LastUpdate" : "2017-07-29 21:16:22",
"Name" : "ESP8266C_Licht1",
"Notifications" : "false",
"PlanID" : "0",
"PlanIDs" : [ 0 ],
"Protected" : false,
"ShowNotifications" : true,
"SignalLevel" : "-",
"SubType" : "Lux",
"Timers" : "false",
"Type" : "Lux",
"TypeImg" : "lux",
"Unit" : 1,
"Used" : 1,
"XOffset" : "0",
"YOffset" : "0",
"idx" : "89"
}
],
"status" : "OK",
"title" : "Devices"
}
]]
print('Start of Timeout-script')
local jsonValue = json.decode(jj)
CheckTimeOut =jsonValue.result[1].HaveTimeout
CurrentServerTime =jsonValue.Servertime
CurrentLastUpdate = jsonValue.result[1].LastUpdate
CurrentData = jsonValue.result[1].Data
print('TimeOut : '.. (CheckTimeOut and "true" or "false") )
print('Servertime : '.. (CurrentServerTime or "nil") )
print('LastUpdate : '.. (CurrentLastUpdate or "nil") )
print('Data-content : '.. (CurrentData or "nil") )
print('End of Timeout-script')
result:
Start of Timeout-script
TimeOut : true
Servertime : nil
LastUpdate : 2017-07-29 21:16:22
Data-content : 73 Lux
End of Timeout-script
I'm currently learning python and I'm familiar (still beginner) with json
my goal is to have one json with many lookup list that could have duplicated value but different key
instead of having duplication of list, I'm trying to find a way to only have one copy and then reusing, i have made htis simple example;
import json
json_enum1 = '{"01" : "ab", "02" : "cd"}'
json_enum2 = '{"01" : "zz", "02" : "xx"}'
json_string = '{"val1": null, "val2": null, "val3": null}'
parsed_json = json.loads(json_string)
parsed_enum1 = json.loads(json_enum1)
parsed_enum2 = json.loads(json_enum2)
parsed_json['val1'] = parsed_enum1
parsed_json['val2'] = parsed_enum2
parsed_json['val3'] = parsed_enum1
print(parsed_json)
print(parsed_json['val1']['01'])
print(parsed_json['val2']['01'])
print(parsed_json['val3']['02'])
result
{u'val3': {u'02': u'cd', u'01': u'ab'}, u'val2': {u'02': u'xx', u'01':
u'zz'}, u'val1': {u'02': u'cd', u'01': u'ab'}}
ab
zz
cd
I could also do that:
import json
json_string = '{"val1": {"01" : "ab", "02" : "cd"}, "val2": {"01" : "zz", "02" : "xx"}, "val3": {"01" : "ab", "02" : "cd"}}'
parsed_json = json.loads(json_string)
print(parsed_json)
print(parsed_json['val1']['01'])
print(parsed_json['val2']['01'])
print(parsed_json['val3']['02'])
which give the same result but now if json_enum1 change, i need to change it twice
and this is a very small example, the real data is way bigger
my question is; is there a better way of doing what I'm describing/showing?
So I'm trying to pull data from a JSON string (as seen below). When I decode the JSON using the code below, and then attempt to index the duration text, I get a nil return. I have tried everything and nothing seems to work.
Here is the Google Distance Matrix API JSON:
{
"destination_addresses" : [ "San Francisco, CA, USA" ],
"origin_addresses" : [ "Seattle, WA, USA" ],
"rows" : [
{
"elements" : [
{
"distance" : {
"text" : "1,299 km",
"value" : 1299026
},
"duration" : {
"text" : "12 hours 18 mins",
"value" : 44303
},
"status" : "OK"
}]
}],
"status" : "OK"
}
And here is my code:
local json = require ("json")
local http = require("socket.http")
local myNewData1 = {}
local SaveData1 = function (event)
distanceReturn = ""
distance = ""
local URL1 = "http://maps.googleapis.com/maps/api/distancematrix/json?origins=Seattle&destinations=San+Francisco&mode=driving&&sensor=false"
local response1 = http.request(URL1)
local data2 = json.decode(response1)
if response1 == nil then
native.showAlert( "Data is nill", { "OK"})
print("Error1")
distanceReturn = "Error1"
elseif data2 == nill then
distanceReturn = "Error2"
native.showAlert( "Data is nill", { "OK"})
print("Error2")
else
for i = 1, #data2 do
print("Working")
print(data2[i].rows)
for j = 1, #data2[i].rows, 1 do
print("\t" .. data2[i].rows[j])
for k = 1, #data2[i].rows[k].elements, 1 do
print("\t" .. data2[i].rows[j].elements[k])
for g = 1, #data2[i].rows[k].elements[k].duration, 1 do
print("\t" .. data2[i].rows[k].elements[k].duration[g])
for f = 1, #data2[i].rows[k].elements[k].duration[g].text, 1 do
print("\t" .. data2[i].rows[k].elements[k].duration[g].text)
distance = data2[i].rows[k].elements[k].duration[g].text
distanceReturn = data2[i].rows[k].elements[k].duration[g].text
end
end
end
end
end
end
timer.performWithDelay (100, SaveData1, 999999)
Your loops are not correct. Try this shorter solution.
Replace all your "for i = 1, #data2 do" loop for this one below:
print("Working")
for i,row in ipairs(data2.rows) do
for j,element in ipairs(row.elements) do
print(element.duration.text)
end
end
This question was solved on Corona Forums by Rob Miracle (http://forums.coronalabs.com/topic/47319-parsing-json-from-google-distance-matrix-api/?hl=print_r#entry244400). The solution is simple:
"JSON and Lua tables are almost identical data structures. In this case your table data2 has top level entries:
data2.destination_addresses
data2.origin_addresses
data2.rows
data2.status
Now data2.rows is another table that is indexed by numbers (the [] brackets) but here is only one of them, but its still an array entry:
data.rows[1]
Then inside of it is another numerically indexed table called elements.
So far to get to the element they are (again there is only one of them
data2.rows[1].elements[1]
then it's just accessing the remaining elements:
data2.rows[1].elements[1].distance.text
data2.rows[1].elements[1].distance.value
data2.rows[1].elements[1].duration.text
data2.rows[1].elements[1].duration.value
There is a great table printing function called print_r which can be found in the community code which is great for dumping tables like this to see their structure."