JSON Schema Conditionals - Different Behaviour Between IDEs - json

I have two private IDE plugins that are generating JSON schemas to validate things. They are comprised of a bunch of definitions that are conditionally used with pattern regexs. The problem occurs in that the schema has different behaviours in IntelliJ and VS Code.
The schema is similar to below, however reduced and redacted:
{
"$schema": "http://json-schema.org/draft-07/schema",
"type": "array",
"additionalItems": true,
"definitions": {
"meat": {
"$id": "#/definitions/meat",
"type": "object",
"properties": {
"cut": {
"$id": "#/definitions/meat/cut",
"title": "cut",
"type": "string"
},
"fillet": {
"$id": "#/definitions/meat/fillet",
"title": "fillet"
},
"bones": {
"$id": "#/definitions/meat/bones",
"title": "bones",
"type": "string"
}
},
"required": [
"cut",
"fillet"
],
"additionalProperties": true
},
"fruit": {
"$id": "#/definitions/fruit",
"type": "object",
"properties": {
"pips": {
"$id": "#/definitions/fruit/pips",
"title": "pips",
"type": "string"
},
"stone": {
"$id": "#/definitions/fruit/stone",
"title": "stone"
},
"citrus": {
"$id": "#/definitions/fruit/citrus",
"title": "citrus",
"type": "string"
}
},
"required": [
"stone",
"pips"
],
"additionalProperties": true
}
},
"items": {
"$id": "#/items",
"type": "object",
"properties": {
"name": {
"$id": "#/items/name",
"type": "string",
"title": "Name"
},
"food": {
"$id": "#/items/food",
"type": "string"
}
},
"allOf": [
{
"if": {
"properties": {
"food": {
"pattern": "meat"
}
}
},
"then": {
"properties": {
"variants": {
"$ref": "#/definitions/meat"
}
}
}
},
{
"if": {
"properties": {
"food": {
"pattern": "fruit"
}
}
},
"then": {
"properties": {
"variants": {
"$ref": "#/definitions/fruit"
}
}
}
}
],
"additionalProperties": true
}
With YAML structure:
- name: orange
food: fruit
variants:
pips: true
So with food being fruit it will match, validate and suggest for that schema, and require stone and pips.
This works in VS Code. IntelliJ however, requires stone and pips correctly, however suggests keys from meat's properties.
This made me question if I have the correct implementation here - should I be using allOf, anyOf or oneOf here? I only need to match one. I also tried adding else: false, however that didn't change anything, and I still get suggestions for meat. When originally writing it, I think I found someone saying allOf will evaluate correctly as the unmatched patterns won't stop the merging of subschemas.
What I'm trying to identify is, is this an issue with my schema implementation, or a bug in IntelliJ's parsing of conditions? Or perhaps a bug in VS Code's parsing that has let me get away with faulty schemas.

Related

JSON Schema - How to apply conditionals based on boolean and further restrict with additionalProperties: false?

Ansible picked up support for jsonschema, and I have a variety of use cases for it, but most all of them require conditionals. I have been reading the doc's for jsonschema and draft-7 if-then-else, and tried several permutations, and seemingly only get half of my target handled. Hoping someone can help pave the way with a practical example for me that I can deconstruct.
Two possibilities based on the bool in "send_notifications":
{
"capture_state": "value here",
"send_notifications": true,
"user_email_list": ["email#something.com", "email2#something.com"],
"host_info": [
{
"ansible_host": "192.20.xxx.xxx",
"ECRTicket": "1103035"
},
.... continued
]
}
or
{
"capture_state": "value here",
"send_notifications": false
}
Essentially, only allow, and require the properties shown in the examples based on the bool in "send_notifications", and unconditionally fail if its neither of these two schemas. I'll likely improve this more with patterns as well, but just these keys and types are a great start.
I had initially generated the following base schema to use as a reference to manipulate and add the conditionals, and perhaps it is something in this schema that conflicts with using conditionals and additionalProperties:false, but being new to jsonschema it's not immediately obvious to me, so reaching out for some assistance so I can put this to use more often. Any help would be appreciated, thank you for your time, new to posting on stack, if I can improve my question please let me know.
{
"$schema": "http://json-schema.org/draft-07/schema",
"type": "object",
"required": [
"capture_state",
"send_notifications",
"user_email_list",
"host_info"
],
"properties": {
"capture_state": {
"type": "string"
},
"send_notifications": {
"type": "boolean"
},
"user_email_list": {
"type": "array",
"items": {
"anyOf": [
{
"type": "string"
}
]
}
},
"host_info": {
"type": "array",
"items": {
"anyOf": [
{
"type": "object",
"required": [
"ansible_host",
"ECRTicket"
],
"properties": {
"ansible_host": {
"type": "string"
},
"ECRTicket": {
"type": "string"
}
},
"additionalProperties": false
}
]
}
}
},
"additionalProperties": false
}
Generally, you're going to have a hard time whenever you choose to restrict unknown properties. I'll assume you have a good reason and including it is worth it.
There are a couple ways to do this. I'm presenting it this way because it produces the best messages when a JSON instance fails schema validation.
{
"$schema": "http://json-schema.org/draft-07/schema",
"type": "object",
"properties": {
"capture_state": { "type": "string" },
"send_notifications": { "type": "boolean" },
},
"required": ["capture_state", "send_notifications"],
"if": {
"properties": {
"send_notifications": { "const": true },
},
"required": ["send_notifications"]
},
"then": {
"properties": {
"capture_state": true,
"send_notifications": true,
"user_email_list": {
"type": "array",
"items": { "type": "string" }
},
"host_info": {
"type": "array",
"items": {
"type": "object",
"properties": {
"ansible_host": { "type": "string" },
"ECRTicket": { "type": "string" }
},
"required": ["ansible_host", "ECRTicket"],
"additionalProperties": false
}
}
},
"required": ["user_email_list", "host_info"],
"additionalProperties": false
},
"else": {
"properties": {
"capture_state": true,
"send_notifications": true
},
"additionalProperties": false
}
}
additionalProperties only recognizes properties declared in the same sub-schema it appears in. That means you need to include some dummy properties entries so it knows about the properties that are declared elsewhere in the schema.
I assume ansible doesn't support more recent versions of JSON Schema than draft-07? If it does, you can use unevaluatedProperties instead of additionalProperties to avoid jumping through some of the hoops you have to with additionalProperties.
{
"$schema": "http://json-schema.org/draft-07/schema",
"type": "object",
"properties": {
"capture_state": { "type": "string" },
"send_notifications": { "type": "boolean" },
},
"required": ["capture_state", "send_notifications"],
"unevaluatedProperties": false,
"if": {
"properties": {
"send_notifications": { "const": true },
},
"required": ["send_notifications"]
},
"then": {
"properties": {
"user_email_list": {
"type": "array",
"items": { "type": "string" }
},
"host_info": {
"type": "array",
"items": {
"type": "object",
"properties": {
"ansible_host": { "type": "string" },
"ECRTicket": { "type": "string" }
},
"required": ["ansible_host", "ECRTicket"],
"additionalProperties": false
}
}
},
"required": ["user_email_list", "host_info"]
}
}
If anyone at Ansible happens to see this, draft 2020-12 includes unevaluatedProperties and will get a lot of implementation support as it is also included in OpenAPI 3.1.

Conditionally Merging JSON Schema Properties

I'm trying to create a JSON schema to validate YAML for some VSCode intellisense. What I'm trying to do is choose the correct subschema to use for a property in the main schema based on an adjacent key's value.
Some JSON examples:
[
{
"name": "doesntmatter",
"matchMe": "stringToMatch:123whatever",
"mergeMe": {
"key1": "value1",
"key2": "value2"
}
}
]
[
{
"name": "doesntmatter",
"matchMe": "anotherStringToMatch:123whatever",
"mergeMe": {
"anotherKey": "valueSomething",
"anotherKey2": "cheese"
}
}
]
So I need to choose the correct schemas for the mergeMe objects based on the substring match of matchMe. After following a bunch of answers, I'm at a point where I can either make it match multiple, and error my linter, or match none, but an online validator says it's ok (except nothing matches as the required fields aren't triggering).
I moved my sub-schemas to be merged into definitions to reference them, and then used an if/then to match. That worked with one, but then I tried to expand it to do the tree matching, and I can't get that to work. Someone said that I should wrap my if/thens in an allOf (I'm not sure why that would work since surely not all of them would match?). Changing it to an anyOf makes none of them match and I get no intellisense. Nor do I really understand why I should wrap single if/thens or thens in allOfs.
The idea is that based on the pattern it uses a definitions schema to match the mergeMe property, but the conditional logic isn't quite right. Thinned schema below:
{
"$schema": "http://json-schema.org/draft-07/schema",
"$id": "http://example.com/example.json",
"type": "array",
"title": "The root schema",
"description": "The root schema comprises the entire JSON document.",
"default": [],
"additionalItems": true,
"definitions": {
"stringToMatch": {
"$id": "#/definitions/stringToMatch",
"type": "object",
"properties": {
"key1": {
"type": "string"
}
},
"required": [
"key1"
],
"additionalProperties": true
},
"anotherStringToMatch": {
"$id": "#/definitions/anotherStringToMatch",
"type": "object",
"properties": {
"key2": {
"type": "string"
}
},
"required": [
"key2"
],
"additionalProperties": true
}
},
"items": {
"$id": "#/items",
"type": "object",
"title": "main schema",
"description": "An explanation about the purpose of this instance.",
"default": {},
"examples": [],
"required": [
"name",
"matchMe",
"mergeMe"
],
"properties": {
"name": {
"$id": "#/items/name",
"type": "string",
"title": "The name schema",
"description": "An explanation about the purpose of this instance.",
"default": "",
"examples": []
},
"matchMe": {
"$id": "#/items/matchMe",
"type": "string",
"title": "The matchMe schema",
"description": "An explanation about the purpose of this instance.",
"default": "",
"examples": []
}
},
"allOf": [
{
"if": {
"properties": {
"matchMe": {
"pattern": "^stringToMatch:[0-9.]+"
}
}
},
"then": {
"allOf": [
{
"type": "object",
"properties": {
"mergeMe": {
"$ref": "#/definitions/stringToMatch"
}
}
}
]
}
},
{
"if": {
"properties": {
"gear": {
"pattern": "^anotherStringToMatch:[0-9.]+"
}
}
},
"then": {
"allOf": [
{
"type": "object",
"properties": {
"mergeMe": {
"$ref": "#/definitions/anotherStringToMatch"
}
}
}
]
}
}
],
"additionalProperties": true
}
}
What I want in JS would look something like
const schema = { name, matchMe }
if (matchMe == "string1") schema.mergeMe = ...subschema1;
else if (...)
else if (...)
but I just can't really work it out. Can someone help?
Edit: jsonschema.dev playground - the idea being if I specify the food as prefixed by "fruit" I have to give it "pips" and "berry", whereas if I specify "vegetable" I have to give it a totally differet schema, and they don't overlap.
https://jsonschema.dev/s/pHzGo
This actually ended up being a bug in the VSCode YAML extension that was ingesting my schema, causing the if blocks to not evaluate, and has been raised, fixed and released.

Restrict enum values defined in a separate JSONschema

I have a simple enums only schema separately to the payload schema say like this
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "demo::common::Husbandry",
"properties": {
"Anilmals": {
"$ref": "#/definitions/Anilmals"
},
"Breeds": {
"$ref": "#/definitions/Breeds"
}
},
"definitions": {
"Anilmals": {
"type": "string",
"description": "Any animal",
"minLength": 1,
"maxLength": 30,
"enum": [
"Dog",
"Cat",
"Bear",
"Human"
]
},
"Breeds": {
"type": "string",
"description": "Any Breed",
"minLength": 1,
"maxLength": 30,
"enum": [
"Poodle",
"Cheshire",
"Polar",
"Trump"
]
}
}
}
If I/payload producer was to send out a stricter schema over this to express and code additional rules without violating the original schema, Example - When Animal chosen was "Cat" they can only choose "Cheshire" as a Breed and so on how would I code the stricter schema? Note the original schema is not changeable and readonly. I may need to use anyOf or oneOf but can't seem to find a good example.
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"additionalProperties": false,
"properties": {
"ReferenceToLocalSchema": {
"$ref": "#/definitions/LocalType"
},
"ReferenceToExternalSchema": {
"$ref": "masterh-husbandry.json#/properties"
}
},
"definitions": {
"LocalType": {
"type": "object",
"additionalProperties": false,
"properties": {
"no-write": {
"type": "boolean",
"default": false
}
}
}
}
}

JSON Schema if-else condition complex scenario

{
"policyHolder": {
"fullName": "A"
},
"traveller": [
{
"fullName": "B",
"relationship": "Spouse"
},
{
"fullName": "A",
"relationship": "My Self"
}
]
}
In above json, I want to validate that
if "relationship" = "My Self" then fullName must match the fullName in policyHolder
A field relationship must exist in traveller array, else json is invalid
I have tried to create a json schema with if-else, allOf, etc. but nothing works which can do these validations but not able to.
Please help!!
Schema:
{
"type": "object",
"required": [
"policyHolder",
"traveller",
],
"properties": {
"policyHolder": {
"$id": "#/properties/policyHolder",
"type": "object",
"required": [
"fullName"
],
"properties": {
"fullName": {
"$id": "#/properties/policyHolder/properties/fullName",
"type": "string",
}
}
},
"traveller": {
"$id": "#/properties/traveller",
"type": "array",
"minItems": 1,
"items": {
"$id": "#/properties/traveller/items",
"type": "object",
"properties": {
"fullName": {
"$ref": "#/properties/policyHolder/properties/fullName"
},
"relationship": {
"$id": "#/properties/traveller/items/properties/relationship",
"type": "string",
}
},
"required": [
"fullName",
"relationship"
],
}
}
}
}```
It's your first requirement that you're going to have the most trouble with. JSON Schema doesn't support validation of data against data elsewhere in the instance. It's a highly discussed topic, but nothing has been adopted yet. I suggest you verify this with a little code.
For the second, I would suggest you extract some of your subschemas into definitions rather than trying to muck about with IDs. IDs are typically more beneficial if you're referencing them from other documents or if you use short (like single-word) IDs. Defining the ID as its location in the document is redundant; most processors will handle this automatically.
{
"type": "object",
"required": [
"policyHolder",
"traveller",
],
"definitions": {
"person": {
"type": "object"
"properties": {
"fullName": {"type": "string"}
},
"required": ["fullName"]
},
"relationship": { "enum": [ ... ] } // list possible relationships
},
"properties": {
"policyHolder": { "$ref": "#/definitions/person" },
"traveller": {
"type": "array",
"minItems": 1,
"items": {
"allOf": [
{ "$ref": "#/definitions/person" },
{
"properties": {
"relationship": { "$ref": "#/definitions/relationship" }
},
"required": ["relationship"]
}
]
}
}
}
}
(I extracted the relationship into its own enum definition, but this is really optional. You can leave it inline, or even an unrestricted string if you don't have a defined set of relationships.)
This can't currently be done with JSON Schema. All JSON Schema keywords can only operate on one value at a time. There's a proposal for adding a $data keyword that would enable doing this kind of validation, but I don't think it's likely to be adopted. $data would work like $ref except it references the JSON being validated rather than referencing the schema.
Here's what how you would solve your problem with $data.
{
"type": "object",
"properties": {
"policyHolder": {
"type": "object",
"properties": {
"fullName": { "type": "string" }
}
},
"traveler": {
"type": "array",
"items": {
"type": "object",
"properties": {
"fullName": { "type": "string" },
"relationship": { "type": "string" }
},
"if": {
"properties": {
"relationship": { "const": "My Self" }
}
},
"then": {
"properties": {
"fullName": { "const": { "$data": "#/policyHolder/fullName" } }
}
}
}
}
}
}
Without $data, you will have to do this validation in code or change your data structure so that it isn't necessary.

JSON validation against JSON schemas: Why is this obvious JSON data not failing validation

This JSON file should fail validation but it does not. Someone tell me why.
Plug the below json data and schema into this web site, validate,
http://json-schema-validator.herokuapp.com
and I get the same results in the Mule Validate JSON Schema. Its obviously does not comply with the schema (i added some fields, I misspelled some fields, the date-time value is not a real date time) but yet it does not fail it. Can someone tell me why?
JSON Schema:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"id": "http://hud.gov/ocio/xsd/esb/serviceauditingframework/2.0#",
"definitions": {
"serviceAuditLogData": {
"type": "object",
"title": "serviceAuditLogData",
"required": [
"serviceRequestTimestamp",
"sourceSystem"
],
"properties": {
"auditId": {
"type": "string"
},
"serviceRequestTimestamp": {
"type": "string",
"format": "date-time"
},
"serviceProvider": {
"type": "string"
},
"serviceProviderVersion": {
"type": "string"
},
"serviceProviderTimestamp": {
"type": "string",
"format": "date-time"
},
"eventType": {
"type": "string"
},
"eventDetail": {
"type": "string"
},
"hostName": {
"type": "string"
},
"sourceSystem": {
"type": "string"
},
"authenticationId": {
"type": "string"
},
"endUserId": {
"type": "string"
},
"inputData": {
"type": "string"
}
},
"propertiesOrder": [
"auditId",
"serviceRequestTimestamp",
"serviceProvider",
"serviceProviderVersion",
"serviceProviderTimestamp",
"eventType",
"eventDetail",
"hostName",
"sourceSystem",
"authenticationId",
"endUserId",
"inputData"
]
}
}
}
JSON Data
{
"serviceAuditLogData": {
"junk":"asdfasdf",
"serviceRequestTimestamp": "2004-09-29T12:58:31.470Z",
"serviceProvider": "FLQS",
"serviceProviderVersion": "v1.0.1",
"audit_id": "17f24136-2494-4bf8-9d3b-9baafaae0cc9",
"serviceProviderTimestamp": "2012-11-04T21:44:57.997Z",
"eventType": "Query Pool",
"eventDetail": "default pool",
"hostName": "esb-d-srv1.",
"sourceSystem": "LRS",
"authenticationId": "EsbLrsAccount",
"endUserId": "H574857",
"inputData": "L234234234, L32453462345, L23452346"
}
}
It does not fail because your schema does not enforce any constraint. Notice that definitions is not a jsonschema keyword that constraints validation. It is normally used to place sub-schemas that are re-used in other parts of the schema definition. Thus, to start with, you should change the definitions keyword for properties.
Another common misunderstanding with jsonschema is related to the properties keyword. Let's take the following example:
{
"type" : "object",
"properties" : {
"key1" : {
"type" : "string"
}
}
}
You must read it as: json must be an object, and in the case that it contains a key equal to key1, its value must be a string. According to that the following two json objects are valid:
{
"key2":12
}
And:
{
"key1":"sdf"
}
Finally, related to date-time format, you must check the section 6 of RFC3339 to be sure you have a valid date-time. And in any case, the implementation of formats is not compulsory in jsonschema validators.
Thanks #jruizaranguren I also learned that I needed to place
"additionalProperties": false, and "required": to make sure that whats' being passed in the API is what's expected.
The below is how I solved my problem.
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"definitions": {
"serviceAuditLogData": {
"type": "object",
"additionalProperties": false,
"required": [
"auditCorrelationId",
"serviceRequestTimestamp",
"serviceProvider",
"serviceProviderVersion",
"serviceProviderTimestamp",
"eventType",
"hostName",
"sourceSystem",
"authenticationId"
],
"properties": {
"auditCorrelationId": {
"type": "string"
},
"serviceRequestTimestamp": {
"type": "string",
"format": "date-time"
},
"serviceProvider": {
"type": "string"
},
"serviceProviderVersion": {
"type": "string"
},
"serviceProviderTimestamp": {
"type": "string",
"format": "date-time"
},
"eventType": {
"type": "string"
},
"eventDetail": {
"type": "string"
},
"hostName": {
"type": "string"
},
"sourceSystem": {
"type": "string"
},
"authenticationId": {
"type": "string"
},
"endUserId": {
"type": "string"
},
"inputData": {
"type": "string"
}
}
}
},
"additionalProperties": false,
"required": [
"serviceAuditLogData"
],
"properties": {
"serviceAuditLogData": {
"$ref": "#/definitions/serviceAuditLogData"
}
}
}