I'd like to be able to write JSON schema code that allows one property's value to be dependent on the value of another property.
More specifically, I have two questions A and B. Question B's answer can only be not null when question A has a specific answer. If question A does not have that answer, then the value to question B must be null.
E.g.
A: Do you like cars? Yes/No
B: What is your favourite car?
Question B can only be answered if the answer to question A is "Yes", otherwise it must be left null.
After some research I have found this Stack Overflow thread, which describes the enum and if-then-else approaches to answering this question. The enum is very close to what I need and is defined as below:
{
"type": "object",
"properties": {
"foo": { "enum": ["bar", "baz"] },
"bar": { "type": "string" },
"baz": { "type": "string" }
},
"anyOf": [
{
"properties": {
"foo": { "enum": ["bar"] }
},
"required": ["bar"]
},
{
"properties": {
"foo": { "enum": ["baz"] }
},
"required": ["baz"]
}
]
}
In the above, when the value of Foo is "Bar", then the Bar property is required. Likewise with the value of "Baz". However instead of making the property required, I want to be able to change the type of the property from null to string. Or do something to be able to make the answer to B valid.
Any thoughts on this?
Did you consider
not defining type of B upfront
using "dependencies" keyword for your schema or making appropriate definition containing answer 'Yes' for question A
and defining question B type only as a result of such dependency?
Let's take your gist:
"questionA": {
"type": "object",
"properties": {
"answer": {
"type": "string",
"minLength": 1,
"enum": ["Yes", "No"]
}
}
}
"questionB": {
"type": "object",
"properties": {
"answer": {
"type": null,
}
}
}
"questions": {
"type": "object",
"properties": {
"A": {"$ref": "#/definitions/questionA"},
"B": {"$ref": "#/definitions/questionB"}
},
"if": {
"properties" : {
"A": {"enum": ["Yes"]}
}
},
"then": {
"B": //Type = string and min length = 1 <-- Unsure what to put here to change the type of QuestionB
}
If I understand correctly your question, the effect you want to achieve is:
If the respondent likes cars, ask him about favourite car and grab answer, else don't bother with the favourite car (and preferrably force the answer to be null).
As Relequestual correctly ponted out in his comment, JSON Schema makes it hard to 'redefine' type. Moreover, each if-then-else content must be a valid schema on it's own.
In order to achieve this effect, you may want to consider following approach:
Define questionA as enum, as you did
Leave property for questionB undefined upfront
Define two possible schemas than can work as property definition for questionB and can be used as a result of dependency
Make use od proper questionB definition in relation to value of questionA
Some sample schema (draft07 compliant) which solves your case is listed below. Also some explanations provided below the schema.
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type" : "object",
"propertyNames" : {
"enum" : [
"questionA",
"questionB",
]
},
"properties" : {
"questionA" : { "$ref" : "#/questionA" },
"questionB" : { "$ref" : "#/questionB" },
},
"dependencies" : {
"questionA" : {
"$ref" : "#/definitions/valid-combinations-of-qA-qB"
}
},
"definitions" : {
"does-like-cars":{
"properties" : {
"questionA" : {
"properties" : {
"answer" : { "enum" : ["Yes","y"] }
}
},
"questionB" : {
"properties" : {
"answer" : {
"$comment" : "Here #/questionB/answer becomes a type:string...",
"$ref" : "#/questionB/definitions/answer-def/string"
}
}
}
},
"required" : ["questionB"]
},
"doesnt-like-cars" :{
"properties" : {
"questionA" : {
"properties" : {
"answer" : { "enum" : ["No","n"] }
}
},
"questionB" : {
"properties" : {
"answer" : {
"$comment" : "Here #/questionB/answer becomes a type:null...",
"$ref" : "#/questionB/definitions/answer-def/null"
}
}
}
}
},
"valid-combinations-of-qA-qB" : {
"anyOf" : [
{ "$ref" : "#/definitions/doesnt-like-cars" },
{ "$ref" : "#/definitions/does-like-cars" }
]
},
},
"examples" : [
{
"questionA" : {
"answer" : "Yes",
},
"questionB" : {
"answer" : "Ass-kicking roadster",
},
},
{
"questionA" : {
"answer" : "No",
},
"questionB" : {
"answer" : null,
},
},
{
},
],
"questionA" : {
"$id" : "#/questionA",
"type" : "object",
"propertyNames" : {
"enum" : ["answer"]
},
"properties" : {
"answer" : {"$ref" : "#/questionA/definitions/answer-def"}
},
"definitions" : {
"answer-def" : {
"$comment" : "Consider using pattern instead of enum if case insensitiveness is required",
"type" : "string",
"enum" : ["Yes", "y", "No", "n"]
}
}
},
"questionB" : {
"$id" : "#/questionB",
"$comment" : "Please note no properties definitions here aside from propertyNames",
"type" : "object",
"propertyNames" : {
"enum" : ["answer"]
},
"definitions" : {
"answer-def" : {
"string" : {
"type" : "string",
"minLength" : 1,
},
"null" : {
"type" : "null"
}
}
}
},
}
Why so complicated?
Because your gist made it so ;-) And more seriously, this is because:
In your gist you define both questions as objects. There might be
valid reason behind it, so I kept it that way (however whenever flat
list of properties could be used, like "questionA-answer",
"questionB-answer" I'd prefer it so to keep schema rules less nested,
thus more readable and easy to create).
It seems from your question and gist, that this is important for you
that "questionB/answer" is null instead of not being validated
against/ignored when it's not relevant, thus I kept it so
Step by step
Questions as objects
Please note, that I've created separate subschemas for "questionA" and "questionB". This is my personal preference and nothing stops you from getting everything inside "definitions" schema of main schema, however I do it usually that way because:
it's easier to split large schema into multiple smaller files after you make everything work like it should (encourages re-use of sub-schemas and helps to structur models in programming languages if someone gets idea to build their data model after my schema)
keeps object schemas/sub-schemas properly encapsulated and well, relative referencing is usually more clear to the reader as well
helps viewing complex schemas in editors that handle JSON syntax
The "propertyNames"
Since we're working here on "type" : "object" I used "propertyNames" keyword to define schema for allowed property names (since classess in programming languages usually have static sets of properties). Try to enter in each object a property outside of this set - schema valdiation fails. This prevents garbage in your objects. Should it be not desired behaviour, just remove "propertyNames" schemas from each object.
"questionB" - where does the trick with changing type sits?
The trick is: do not define property type and other relevant schema rules upfront. Please note how there's no "properties" schema in "questionB" schema. Instead, I used "definitions" to prepare two possible definitions of "answer" property inside "questionB" object. I will use them depending on "questionA" answer value.
"examples" section?
Some objects, that should illustrate how schema works. Play around with answer values, presence of properties etc. Please note, that an empty object will also pass validation, as no property is required (as in your gist) and there's only one dependency - if "questionA" appears, a "questionB" must show up as well.
Ok, ok, top to bottom now, please
Sure.
So the main schema can have two properties:
questionA (an object containing property "answer")
questionB (an object containing property "answer")
Is "#/questionA" required? -> No, at least based on your gist.
Is "questionB" required? -> Only if "#/questionA" appears. To add insult to injury :-) the type and allowed values of "#/questionB/answer" strictly depend on the value of "#/questionA/answer".
--> I can safely pre-define main object, foundation for questions objects and will need to define dependency
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type" : "object",
"propertyNames" : {
"enum" : [
"questionA",
"questionB",
]
},
"properties" : {
"questionA" : { "$ref" : "#/questionA" },
"questionB" : { "$ref" : "#/questionB" },
},
"dependencies" : {
"questionA" : {
"$comment" : "when questionA prop appears in validated entity, do something to enforce questionB to be what it wants to be! (like Lady Gaga in Machette...)"
}
},
"questionA" : {
"$id" : "#/questionA",
"type" : "object",
"propertyNames" : {
"enum" : ["answer"]
},
},
"questionB" : {
"$id" : "#/questionB",
"type" : "object",
"propertyNames" : {
"enum" : ["answer"]
},
},
}
Please note I am conciously setting relative base reference via "$id" keyword for question sub-schemas to be able to split schema into multiple smaller files and also for read-ability.
--> I can safely pre-define "questionA/answer" property: type, allowed values etc.
"questionA" : {
"$id" : "#/questionA",
"type" : "object",
"propertyNames" : {
"enum" : ["answer"]
},
"properties" : {
"answer" : {"$ref" : "#/questionA/definitions/answer-def"}
},
"definitions" : {
"answer-def" : {
"$comment" : "Consider using pattern instead of enum if case insensitiveness is required",
"type" : "string",
"enum" : ["Yes", "y", "No", "n"]
}
}
},
Note: I used "definitions" to, well, define schema for specific property. Just in case I'd need to re-use that definition somewhere else... (yep, paranoid about that I am)
--> I can't safely pre-define "#/questionB/answer" property as mentioned above and must do the "trick" part in "#/questionB" sub-schema
"questionB" : {
"$id" : "#/questionB",
"$comment" : "Please note no properties definitions here aside from propertyNames",
"type" : "object",
"propertyNames" : {
"enum" : ["answer"]
},
"definitions" : {
"answer-def" : {
"string" : {
"type" : "string",
"minLength" : 1,
},
"null" : {
"type" : "null"
}
}
}
},
NOTE: See "#/definitions/answer-def"? There are two sub-nodes to that, "#/definitions/answer-def/string" and "#/definitions/answer-def/null" . I wasn't entirely sure how I'll do it at the moment, yet I knew I definitely will need that capability of juggle with "#/questionB/answer" property schema in the end.
--> I must define rules for valid combinations of both answers and since "#/questionB/answer" must be always present; I'm doing that in main schema, which uses questions sub-schemas as it's a cap over them that logically makes a good place to define such rule.
"definitions" : {
"does-like-cars":{
"properties" : {
"questionA" : {
"properties" : {
"answer" : { "enum" : ["Yes","y"] }
}
},
"questionB" : {
"properties" : {
"answer" : {
"$comment" : "Here #/questionB/answer becomes a type:string...",
"$ref" : "#/questionB/definitions/answer-def/string"
}
}
}
},
"required" : ["questionB"]
},
"doesnt-like-cars" :{
"properties" : {
"questionA" : {
"properties" : {
"answer" : { "enum" : ["No","n"] }
}
},
"questionB" : {
"properties" : {
"answer" : {
"$comment" : "Here #/questionB/answer becomes a type:null...",
"$ref" : "#/questionB/definitions/answer-def/null"
}
}
}
}
},
"valid-combinations-of-qA-qB" : {
"anyOf" : [
{ "$ref" : "#/definitions/doesnt-like-cars" },
{ "$ref" : "#/definitions/does-like-cars" }
]
},
So there are those, who like cars - I basically define allowed values of "#/questionA/answer" and relevant definition of property of "#/questionB/answer". Since this is the schema, both sets must match to fulfill this definition. Please note I marked "questionB" property key as required in order to not validate JSON that contains only "questionA" property key against schema.
I did similar thing for those, who don't like cars (how one cannot like cars?! Wicked times...) and at the end I said in "valid-combinations-of-qA-qB": It's either or, people. Either you like cars and give me the answer or you don't like cars and the answer must be null. "XOR" ("oneOf") comes to mind automatically but since I've defined like cars AND answer and doesn't like cars AND answer = null as a complete schemas, logical OR is completely sufficient -> "anyOf".
At the end the finishing touch was to use that rule in "dependencies" section of main schema, which translates to: if "questionA" appears in validated instance, either... or...
"dependencies" : {
"questionA" : {
"$ref" : "#/definitions/valid-combinations-of-qA-qB"
}
},
Hope it clarifies and helps with your case.
Open questions
Why not use object "answers" with properties reflecting each question answer, with key identifying the question? It could simplify a bit rules and references with regards to dependencies between answers (less typing, yep, I'm a lazy lad).
Why "#/questionB/answer" must be null instead just ignoring it if "#/questionA/answer" : { "enum" : ["No"] } ?
Recommended reading
See "Understanding JSON Schema" : https://json-schema.org/understanding-json-schema/index.html
Some basic examples: https://json-schema.org/learn/
JSON schema validation reference: https://json-schema.org/latest/json-schema-validation.html
A lot of StackOverflow Q&A provides nice insight in how to manage different cases with JSON Schema.
Also it might be helpful at occasion to check for relative JSON Pointers RFC.
I cannot find examples in the Elastic manual on nested objects on how to modify fields and nested objects of documents using RESTful commands in Kibana Sense. I am looking for something similar to Solrs atomic updates here, which allow to update specific fields of documents.
How do RESTful commands in Kibana Sense look like that accomplish this? The only related info in the manual I can find is on Partial Updates to Documents, but I do not know how this can be applied for this use case.
For example, straight from the Elastic docs:
PUT my_index
{
"mappings": {
"my_type": {
"properties": {
"user": {
"type": "nested"
}
}
}
}
}
PUT my_index/my_type/1
{
"group" : "fans",
"user" : [
{
"first" : "John",
"last" : "Smith"
},
{
"first" : "Alice",
"last" : "White"
}
]
}
How can I delete an entry in the nested object, so that the document "1" looks like:
{
"group" : "fans",
"user" : [
{
"first" : "John",
"last" : "Smith"
}
]
}
How can I add an entry in the nested object, so that the document "1" looks like:
{
"group" : "fans",
"user" : [
{
"first" : "John",
"last" : "Smith"
},
{
"first" : "Alice",
"last" : "White"
},
{
"first" : "Peter",
"last" : "Parker"
}
]
}
You will have to use scripted updates unless you want to fetch all nested objects then add / remove items and re-index them all which is the previous answer proposed. However if you have a lot of nested documents you should be doing partial updates / additions and deletes. It is much quicker from data transfer and indexing point of view.
Here is a good article how to do scripted updates in general:
https://iridakos.com/programming/2019/05/02/add-update-delete-elasticsearch-nested-objects
Unless I misunderstand your ask, you just post the updated document version to the same document id each time you want.
To delete a nested document (or any field):
PUT my_index/my_type/1
{
"group" : "fans",
"user" : [
{
"first" : "Alice",
"last" : "White"
}
]
}
To add a user, add it to the list:
PUT my_index/my_type/1
{
"group" : "fans",
"user" : [
{
"first" : "Alice",
"last" : "White"
},
{
"first" : "Peter",
"last" : "Parker"
}
]
}
Note: Documents in elasticsearch are immutable. Making a change to a single field causes the entire document to be re-indexed. Nested documents are always re-indexed with the parent document so if you change a field in the parent the nested document is also re-indexed. This can be a performance issue if the nested documents are large and the parents have frequent changes.
For this specific use case, you must use a scripted update. In javascript the call will look something like:
const documentUpdateInstructions = {
index: "index-name",
id: "document-id",
body: {
script: {
lang: "painless",
source: `ctx._source.myNestedObject.removeIf(object -> object.username == params.username);`,
params: {
username: "my_username"
},
},
},
};
await client.update(documentUpdateInstructions);
This takes a document in the form of
document._source = {
...
"myNestedObject": [
{
"username": "my_username",
...
},
{
"username": "not_my_username",
...
}
]
}
and deletes the object inside myNestedObject who's username matches the username provided (in this case my_username). The resulting document will be:
document._source = {
...
"myNestedObject": [
{
"username": "not_my_username",
...
}
]
}
Could you help me, how I can validate "keys"(ex. "1","2","3") of list items in following json:
{
"list" : {
"1" : {
"element1" : "1",
"element2" : "2"
},
"2" : {
"element1" : "1",
"element2" : "2"
},
....
"512" : {
"element1" : "1",
"element2" : "2"
}
}
}
Please give me common examples to validate keys in json, too.
JSON Schema has three ways to constraint the property names of an object.
properties
The properties keyword allows you to set an exact match for property names.
{
"type": "object",
"properties": {
"1": { ... },
"2": { ... },
...
}
}
patternProperties
The patternProperties keyword constrains any property name that matches a regular expression to validate against a given schema.
{
"type": "object",
"patternProperties": {
"[1-9][0-9]*": { ... }
}
}
propertyNames
The propertyNames keyword constraints property names to match the given schema.
{
"type": "object",
"propertyNames": {
"pattern": "[1-9][0-9]*"
}
}
Edit 12-19-2018 Improve answer, add missing patternProperties, add new (since original answer) option propertyNames.
I am writing a software that can read the JSON data from a file. The file contains "person" - an object whose value is an array of objects. I am planning to use a JSON schema validating libraries to validate the contents instead of writing the code myself. What is the correct schema that conforms to JSON Schema Draf-4 which represents the below data?
{
"person" : [
{
"name" : "aaa",
"age" : 10
},
{
"name" : "ddd",
"age" : 11
},
{
"name" : "ccc",
"age" : 12
}
]
}
The schema that wrote down is given below. I am not sure whether it is correct or is there any other form?
{
"person" : {
"type" : "object",
"properties" : {
"type" : "array",
"items" : {
"type" : "object",
"properties" : {
"name" : {"type" : "string"},
"age" : {"type" : "integer"}
}
}
}
}
}
You actually only have one line in the wrong place, but that one line breaks the whole schema. "person" is a property of the object and thus must be under the properties keyword. By putting "person" at the top, JSON Schema interprets it as a keyword instead of a property name. Since there is no person keyword, JSON Schema ignores it and everything below it. Therefore, it is the same as validating against the empty schema {} which places no restrictions on what a JSON document can contain. Any valid JSON is valid against the empty schema.
{
"type" : "object",
"properties" : {
"person" : {
"type" : "array",
"items": {
"type" : "object",
"properties" : {
"name" : {"type" : "string"}
"age" : {"type" : "integer"}
}
}
}
}
}
By the way, there are several online JSON Schema testing tools out there that can help you out when crafting your schemas. This one is my goto http://jsonschemalint.com/draft4/#
Also, here is a great JSON Schema reference that might help you out as well: https://spacetelescope.github.io/understanding-json-schema/
How can I hint the type of embedded objects in JSON Schema, analogous to xsi:type in XML Schema?
Example schema document:
{
"type": "storeRequest",
"properties": {
"txid": {
"description": "Transaction ID to prevent double committing",
"type": "integer"
},
"objects": {
"description": "Objects to store",
"type": "array"
"items": {
"type": "object"
},
"minItems": 1,
"uniqueItems": true
},
},
"required": ["txid", "objects"]
}
This is a request the client sends to the server to store multiple objects in the database. Now how can I recursively validate the content of objects when it can contain more than one type of object. (Plymorphism, really).
There is not an equivalent to xsi:type in JSON-schema AFAIK.
Perhaps the most JSON-schema idiomatic way to hint the existence of types would be the explicit definition of types as schemas and referencing them through $ref:
{
"properties" : {
"wheels" : {
"type" : "array",
"items" : "$ref" : "#/definitions/wheel"
}
}
"definitions" : {
"wheel" : {
"type" : "object"
}
}
}
Another way could be to give a hint through enums :
{
"definitions" : {
"vehicle" : {
"properties" : {
"type" : {
"enum" : ["car", "bike", "plane"]
}
}
},
"plane" : {
"properties" : {
"type" : {
"enum" : "plane"
}
}
"allOf" : ["$ref" : "#/definitions/vehicle"]
}
}
}
Finally you can also add whatever tag you can process to a JSON-schema and follow your conventions.
Be aware that you are not going to find an equivalent translation between typical object oriented programming languages (java, C#) inheritance semantics and JSON-schema.