jsonschema Required properties inoperative with $ref - json

I am going to write json schema to verify tree data.
Schema consisting of top root and block below.
There may be another block below the block.
Schema for validation.
schema = {
"$schema": "http://json-schema.org/draft-04/schema",
"$ref": "#/definitions/root",
"definitions":{
"root": {
"properties": {
"name": {
"type": "string"
},
"children": {
"type": "array",
"items": [
{"$ref":"#/definitions/block"}
]
}
},
"required": ["name", "children"]
},
"block": {
"properties": {
"name": {
"type": "string"
},
"children": {
"type": "array",
"items": [
{"$ref":"#/definitions/block"}
]
}
},
"required": ["name"]
}
}
}
Below is incorrect data for testing. The last name properties do not exist.
{
"name": "group8",
"children": [
{
"name": "group7",
"children": [
{
"name": "group6",
"children": [
{
"name": "group5",
"children": [
{ ###### wrong
"children": []
}
]
}
]
}
]
}
]
}
This data validates well, but it doesn't work on a slightly complex tree.
# Error: ValidationError: file /home/gulliver/.local/lib/python2.7/site-packages/jsonschema/validators.py line 934: 'name' is a required property #
{
"name": "group8",
"children": [
{
"name": "group7",
"children": [
{
"name": "group6",
"children": [
{
"name": "group12",
"children": [
{
"name": "group11",
"children": [
{
"name": "group10",
"children": []
}
]
}
]
},
{
"name": "group9",
"children": [
{
"name": "group5",
"children": [
{ ####### wrong
"children": []
}
]
}
]
}
]
}
]
},
{
"name": "group13",
"children": [
{
"name": "null1",
"children": []
}
]
}
]
}
It does not work when the data at the bottom of the tree is invalid.
My guess is that the branch splits and this happens, does anyone know why or how to fix it?
I tested using python and jsonschema.

When items is an array, it applies the subschema values to the same index location in the array in the instance.
For example, where you define...
"items": [
{"$ref":"#/definitions/block"}
]
only the first item in the array will be tested. It has nothing to do with deep nesting. For example, the follwing data is valid according to your schema...
{
"name": "group8",
"children": [
{
"name": "group7"
},
{
"something": "else",
"Not": "name"
}
]
}
(Live demo: https://jsonschema.dev/s/etFGE)
If you modify your use of items, then it will work like you expect:
"items": {"$ref":"#/definitions/block"}
(do this for both uses)
Live demo: https://jsonschema.dev/s/rk1OD

Related

scala ujson.read() returns ujson.Obj

I am trying to read a json string using Li Haoyi's ujson. This is the string:
{
"dataflows": [
{
"name": "test",
"sources": [
{
"name": "person_inputs",
"path": "/data/input/events/person/*",
"format": "JSON"
}
],
"transformations": [
{
"name": "validation",
"type": "validate_fields",
"params": {
"input": "person_inputs",
"validations": [
{
"field": "office",
"validations": [
"notEmpty"
]
},
{
"field": "age",
"validations": [
"notNull"
]
}
]
}
},
{
"name": "ok_with_date",
"type": "add_fields",
"params": {
"input": "validation_ok",
"addFields": [
{
"name": "dt",
"function": "current_timestamp"
}
]
}
}
],
"sinks": [
{
"input": "ok_with_date",
"name": "raw-ok",
"paths": [
"/data/output/events/person"
],
"format": "JSON",
"saveMode": "OVERWRITE"
},
{
"input": "validation_ko",
"name": "raw-ko",
"paths": [
"/data/output/discards/person"
],
"format": "JSON",
"saveMode": "OVERWRITE"
}
]
}
]
}
And this is how I read it:
val j = os.read(os.pwd/RelPath("src/main/scala/metadata.json"))
val jsonData = ujson.read(j)
But, the return type is ujson.Obj, and not Arr(ArrayBuffer(Obj), as expected, such that when I try to get jsonData(0), what I get is json.Value$InvalidData: Expected ujson.Arr.
I am asking this question because I have tried to use the ujson object to create a upickle object, but I cannot, and I suspect it is because of this initial error.
Any ideas of why this happens? Any help would be greatly appreciated! Thanks in advance!!
The outer element of your JSON is not an array, it is an object with a single element dataflows whose value is an array. Try jsonData("dataflows")(0).

JSON Schema with Nested Objects with different properties

The entire JSON file is rather large so I've only taken out the subsection I've had an issue with.
{
"diagrams": {
"5f759d15cd046720c28531dd": {
"_id": "5f759d15cd046720c28531dd",
"offsetX": 320,
"offsetY": 42,
"zoom": 80,
"modified": 1604279356,
"nodes": {
"5f9f5c3ccd046720c28531e4": {
"nodeID": "5f9f5c3ccd046720c28531e4",
"type": "start",
"coords": [
360,
120
],
"data": {
"name": "Start",
"color": "standard",
"ports": [
{
"type": "",
"target": "5f9f5c3ccd046720c28531e6"
}
],
"steps": []
}
},
"5f9f5c3ccd046720c28531e5": {
"nodeID": "5f9f5c3ccd046720c28531e5",
"type": "block",
"coords": [
760,
120
],
"data": {
"name": "Help Message",
"color": "standard",
"steps": [
"5f9f5c3ccd046720c28531e6",
"5f9f5c3ccd046720c28531e7"
]
}
},
"5f9f5c3ccd046720c28531e6": {
"nodeID": "5f9f5c3ccd046720c28531e6",
"type": "speak",
"data": {
"randomize": false,
"dialogs": [
{
"voice": "Alexa",
"content": "You said help. Do you want to continue?"
}
],
"ports": [
{
"type": "",
"target": "5f9f5c3ccd046720c28531e7"
}
]
}
},
"5f9f5c3ccd046720c28531e7": {
"nodeID": "5f9f5c3ccd046720c28531e7",
"type": "interaction",
"data": {
"name": "Choice",
"else": {
"type": "path",
"randomize": false,
"reprompts": []
},
"choices": [
{
"intent": "",
"mappings": []
},
{
"intent": "",
"mappings": []
}
],
"reprompt": null,
"ports": [
{
"type": "else",
"target": null
},
{
"type": "",
"target": null
},
{
"type": "",
"target": "5f9f5c3ccd046720c28531e9"
}
]
}
},
"5f9f5c3ccd046720c28531e8": {
"nodeID": "5f9f5c3ccd046720c28531e8",
"type": "block",
"coords": [
1170,
260
],
"data": {
"name": "Exit",
"color": "standard",
"steps": [
"5f9f5c3ccd046720c28531e9"
]
}
},
"5f9f5c3ccd046720c28531e9": {
"nodeID": "5f9f5c3ccd046720c28531e9",
"type": "exit",
"data": {
"ports": []
}
}
},
"children": [],
"creatorID": 42661,
"variables": [],
"name": "Help Flow",
"versionID": "5f759d15cd046720c28531db"
}
}
}
The Current JSON Schema Definition I have is:
{
"$schema":"http://json-schema.org/schema#",
"type":"object",
"properties":{
"diagrams":{
"type":"object"
}
},
"required":[
"diagrams",
]
}
The problem I am having is that within diagrams contains multiple objects with a random string as the name e.g "5f759d15cd046720c28531dd".
Then within that object there are properties such as (_id, offsetX) which I want to express as well as a nodes object, which again contains multiple objects with arbitrary names e.g ("5f9f5c3ccd046720c28531e4", "5f9f5c3ccd046720c28531e5", ...) which have a unique node definition where some nodes have different properties to other nodes (nodeID, type, data vs nodeID, type, data, coords).
My question is with all these arbitrary things such as random names as well as different properties per each node. How do I turn it into 1 JSON schema definition which covers all the cases of how a diagram/node can be made.
You can do this with additionalProperties or patternProperties.
additionalProperties applies to any property that isn't declared in properties or patternProperties.
{
"type": "object",
"additionalProperties": {
"type": "object",
"properties": {
"_id": { ... },
"offsetX": { ... },
...
}
}
}
Your property names appear to always be hex numbers. If you want to enforce that those property names are always hex numbers, you can use patternProperties. Any property that matches the regex must conform to that schema.
{
"type": "object",
"patternProperties": {
"^[0-9a-f]{24}$": {
"type": "object",
"properties": {
"_id": { ... },
"offsetX": { ... },
...
}
}
},
"additionalProperties": false
}

How can I create a hierarchical json response in FLASK

I have a single table in database like database table. I want to search a child from database and return a hierarchical JSON to a front end in order to create a tree. How can I do that in FLASK.
My expected JSON for mat should be like expected JSON
Since you have tagged your question with flask, this post assumes you are using Python as well. To format your database values in JSON string, you can query the db and then use recursion:
import sqlite3, collections
d = list(sqlite3.connect('file.db').cursor().execute("select * from values"))
def get_tree(vals):
_d = collections.defaultdict(list)
for a, *b in vals:
_d[a].append(b)
return [{'name':a, **({} if not (c:=list(filter(None, b))) else {'children':get_tree(b)})} for a, b in _d.items()]
import json
print(json.dumps(get_tree(d), indent=4))
Output:
[
{
"name": "AA",
"children": [
{
"name": "BB",
"children": [
{
"name": "EE",
"children": [
{
"name": "JJ",
"children": [
{
"name": "EEV"
},
{
"name": "FFW"
}
]
},
{
"name": "KK",
"children": [
{
"name": "HHX"
}
]
}
]
}
]
},
{
"name": "CC",
"children": [
{
"name": "FF",
"children": [
{
"name": "LL",
"children": [
{
"name": "QQY"
}
]
},
{
"name": "MM",
"children": [
{
"name": "RRV"
}
]
}
]
},
{
"name": "GG",
"children": [
{
"name": "NN",
"children": [
{
"name": "SSW"
}
]
}
]
}
]
},
{
"name": "DD",
"children": [
{
"name": "HH",
"children": [
{
"name": "OO",
"children": [
{
"name": "TTZ"
}
]
}
]
},
{
"name": "II",
"children": [
{
"name": "PP",
"children": [
{
"name": "UUW"
}
]
}
]
}
]
}
]
}
]

JSON is not getting validated based on schema

I must be missing something here, but the below JSON is not getting validated against the schema.
For example, the required attribute from the Java/JavaScript object is never getting enforced as per the schema. (FYI- Every language object may have other attributes or nested object)
However, if I completely remove the definition and directly put under array items each separately, then it validates.
I want to use 'definitions' and gets validated.The reason I have to put all the object in definitions and later I may have to put different language object in oneOf/allOf for other certain validation check.
Online check:
Schema and JSON
JSON
{
"languages": [
{
"lang": "Java",
"trainer": "Peter"
},
{
"lang": "JavaScript",
"enrolled": "42",
"available": "5"
}
]
}
and the Schema
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"required": [
"languages"
],
"properties": {
"languages": {
"type": "array",
"minItems": 1,
"items": {
"type": "object"
},
"anyOf": [
{
"$ref": "#/definitions/Java"
},
{
"$ref": "#/definitions/JavaScript"
}
]
}
},
"definitions": {
"Java": {
"required": [
"trainer"
],
"properties": {
"lang": {
"enum": [
"Java"
]
},
"trainer": {
"type": "string"
}
}
},
"JavaScript": {
"required": [
"enrolled",
"available"
],
"properties": {
"lang": {
"enum": [
"JavaScript"
]
},
"enrolled": {
"type": "string"
},
"available": {
"type": "string"
}
}
}
}
}
The fixed schema and it works now
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"required": [
"languages"
],
"properties": {
"languages": {
"type": "array",
"minItems": 1,
"items": {
"type": "object",
"anyOf": [
{
"$ref": "#/definitions/Java"
},
{
"$ref": "#/definitions/JavaScript"
}
]
}
}
},
"definitions": {
"Java": {
"required": [
"trainer"
],
"properties": {
"lang": {
"enum": [
"Java"
]
},
"trainer": {
"type": "string"
}
}
},
"JavaScript": {
"required": [
"enrolled",
"available"
],
"properties": {
"lang": {
"enum": [
"JavaScript"
]
},
"enrolled": {
"type": "string"
},
"available": {
"type": "string"
}
}
}
}
}
I think that the "array" definition is not correct. The objects that can be added in the array must be refereed in the "items", after you define the type of the items. Something like this:
"languages": {
"type": "array",
"minItems": 1,
"items": {
"type": "object",
"anyOf": [
{"$ref": "#/definitions/Java"},
{"$ref": "#/definitions/JavaScript"}
]
}
}
I have fixed myself
What I did: I have placed the json object blocks under items section of array.

Traversing through nested json in scala play

I'm using scala play and am attempting to traverse a json tree in order to validate that specific name values have specific children with specific name values. I have the following Json in the form of a JsObject:
{ "name": "user", "children": [ { "name": "$a", "children": [ { "name": "foo", "children": [ ] }, { "name": "fooBar", "children": [ { "name": "$a", "children": [ { "name": "subFoobar1", "children": [ ] }, { "name": "subFoobar2", "children": [ { "name": "TEST", "children": [ ] } ] }, { "name": "subFoobar3", "children": [ ] } ] } ] }, { "name": "bar", "children": [ { "name": "$a", "children": [ ] }, { "name": "$c", "children": [ ] }, { "name": "$b", "children": [ ] } ] }, { "name": "barFoo", "children": [ ] } ] } ] }
Ideally I would use nested for loops to traverse but the JsObject structure is preventing me from accessing the underlying values when attempting traverse. I have also attempted mapping the JsObject to a map of type [Map[String,Map[String,Any]]] but I am getting invalid cast compiler errors.
Any tips on how I can traverse and validate the name value at each level would be appreciated. I would preferably like to use the play json library
Issue was in the case class I was attempting to use. I wasn't accounting for the recursive nature of my Json structure
case class ActorTree(name : String, children:Seq[ActorTree] )