I have searched SO and JSON schema documentation for a few days now, but I'm struggling to get my JSON Schema references to work across multiple files in the same directory.
My root schema would be an oneOf object:
{
"$schema": "http://json-schema.org/draft-04/schema",
"id": "http://localhost/json-editor/schema/function.json",
"title": "Function",
"oneOf": [
{
"$ref": "./fn_md5.json"
},
{
"$ref": "fn_sha1.json"
}
]
}
Now after reading the docs and most questions here, I'd assume both ./fn_md5.json and fn_sha1.json would be resolved to http://localhost/json-editor/schema/fn_....json (where the ... represents their respective name, of course). However, the resolver keeps looking into http://localhost/json-editor/fn_....json, which appears to be relative to the application's url (JDorn's JSON Editor running at my http://localhost/json-editor/).
Am I understanding the schema and id wrong, or could it be an error with the editor's URL resolver?
Thanks!
for find the relative references you need use $id like this:
{
"$schema": "http://json-schema.org/draft-04/schema",
"id": "http://localhost/json-editor/schema/function.json",
"$id":"https://www.baseurl.com/"
"title": "Function",
"oneOf": [
{
"$id": "./fn_md5.json"
},
{
"$id": "fn_sha1.json"
}
]
}
source: https://datatracker.ietf.org/doc/html/draft-handrews-json-schema-01#section-8.2.4
Related
My issue
I need to validate a json file from a Json Schema.
I have a main schema and this schema should load an external schema file to validate some part of the json.
For different reasons I can't merge them. I do need 2 schema file.
I am using a $ref statement as proposed in this doc:
https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/test-json?view=powershell-7.2&WT.mc_id=DT-MVP-5004831
but whatever I try I have this error:
Cannot parse the JSON schema.
What I need
What is the right schema syntax to do this in my case.
Test case
I have this directories on my disk:
.
test.ps1
----/jsons
params.json
----/schemas
securities.json
storages.json
I run validation from this pretty simple PowerShell (test.ps1):
# read json param
$testFile = Get-Content "./jsons/params.json" -Encoding UTF8 | convertfrom-json -Depth 50
$testee = $testFile.parameters | convertto-json
# read json schema
$schemasFile = (Get-ChildItem -Path "./schemas/storages.json").FullName
# test json from schema
$result = $testee | Test-Json -SchemaFile $schemasFile
$result
My json (params.json):
{
"parameters": {
"storages": [
{
"comment": "Very important storage",
"name": "fdlmsto",
"securities": [
{
"comment": "Critical rule",
"kind": "MSI"
}
]
}
]
}
}
The main schema (storages.json):
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"additionalProperties": false,
"required": [
"storages"
],
"properties": {
"storages": {
"$id": "#/properties/storages",
"type": "array",
"additionalProperties": false,
"items": [
{
"$id": "#/properties/storages/items",
"type": "object",
"additionalProperties": false,
"required": [
"name",
"comment"
],
"properties": {
"$ref": "#/schemas/securities.json",
"name": {
"type": "string",
"$id": "#/properties/storages/items/properties/name"
},
"comment": {
"type": "string",
"$id": "#/properties/storages/items/properties/comment"
}
}
}
]
}
}
}
and the child schema (securities.json):
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"additionalProperties": false,
"required": [
"securities"
],
"properties": {
"securities": {
"$id": "#/properties/securities",
"type": "array",
"additionalProperties": false,
"items": {
"$id": "/properties/securities/items",
"anyOf": [
{
"$id": "#/properties/securities/items/anyOf/0",
"type": "object",
"additionalItems": false,
"additionalProperties": false,
"required": [
"kind",
"comment"
],
"properties": {
"comment": {
"type": "string",
"$id": "#/properties/securities/items/anyOf/0/properties/comment"
},
"kind": {
"type": "string",
"$id": "#/properties/securities/items/anyOf/0/properties/kind"
}
}
}
]
}
}
}
}
What I tested
I tested several syntaxe for $ref:
#/schemas/securities.json
./schemas/securities.json
/schemas/securities.json
....
I also tried to set the schema in the repository of test.ps1
I checked that this is the securities section that don't work
Thank you
I edit the question to leave what worked for me:
Change storages.json like this:
"properties": {
"securities": {
"$ref": "securities1.json#/securities"
},
"name": {
"type": "string"
},
"comment": {
"type": "string"
}
}
Change securities.json as:
{
"securities": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": [
"kind",
"comment"
],
"properties": {
"comment": {
"type": "string"
},
"kind": {
"type": "string"
}
}
}
}
}
And of course I applied all changed suggested in the post
It appears that the powershell cmdlet is powered by NJsonSchema, which at this stage doesn't support JSON Schema draft 2020-12.
There are several issues I see.
storages.json
"$schema": "https://json-schema.org/draft/2020-12/schema" declares this is a draft 2020-12 schema. As your schema doesn't use any 2020-12 features, I'd suggest using draft 7 as it'll definitely be supported by NJsonSchema.
Your schema is declaring itself to be the meta-schema with "$id": "https://json-schema.org/draft/2020-12/schema". The meta-schema is a special schema that validates other schemas. What you probably want here is just storages.json, or even better, a full URI like https://examp.le/schemas/storages.json. It's just an identifier though; it doesn't need to be downloadable.
Remove the internal $ids. They're only specifying relative location and not providing any more detail. This kind of thing is generally calculated at runtime.
Your $ref is wrong. #/schemas/securities.json is going to try to search for this location inside the current schema. You need a full URI here, and you'll want to declare that same URI as $id in securities.json as well. This may be what is producing the error.
At #/properties/storages, you have "type": "array" and "additionalProperties": false. Maybe you meant additionalItems? additionalProperties only works on objects. (I also see this in securities.json at #/properties/securities.)
Your use of items is an array. This will only validate the first item. I think you're wanting to validate all the items. If so, just use the schema as the value for items; don't wrap it in an array. (also seen in securities.json)
securities.json
This schema uses draft 4, which is quite old and a bit incompatible with later drafts. Specifically, I see it's using $id in several places, which replaced draft 4's id in draft 6. Since draft 4 doesn't recognize $id, it'll probably ignore them. Moreover, $ref-ing between drafts isn't something that's required (or perhaps even addressed) by the specification. (I added some tests to the test suite around this only about a week ago.) As with storages.json I'd suggest using draft 7 here.
As mentioned above, you'll want to add an $id to the root so that the $ref in storages.json can identify this schema properly.
As with storages.json, remove the interal $ids; they're not providing any additional function.
You use an anyOf but only declare a single schema. You don't need this. Just move the keywords in that subschema up to where the allOf is.
{
"type": "object",
"allOf": [ { ... } ]
}
// becomes
{
"type": "object",
...
}
Sorry this sounds like a pile-on. I realize you might be new with JSON Schema. The learning curve can be pretty steep.
Instead of the documentation provided by MS, I'd use Understanding JSON Schema for how to write schemas. Use the MS docs only for how to use the cmdlet.
Just to add one point to the excellent answer above: the array form of items is not valid in 2020-12 anyway - it was replaced with prefixItems.
But the advice to use draft-07 throughout is the right advice, as is the advice to use the schema form of items anyway.
I have the following use-case I try to solve with JSON schemas.
I have a generic JSON data schema for, for example, a user. Here is an example of the user.schema.json file.
{
"type": "object",
"definitions": {},
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"name": {
"type": "string",
"minLength": 1
},
"email": {
"type": "string",
"minLength": 1
},
"locale": {
"type": "string",
"minLength": 1
},
"active": {
"type": "boolean",
"default": true
},
"password": {
"type": "string",
"minLength": 8
},
"roles": {
"type": "array",
"items": {
"type": "string",
"minLength": 1
}
}
}
}
Now I have 2 different kinds of requests:
- POST: Add a user
- PATCH: Update user data.
In 1 case, I can send this data structure, with 3 required fields, while in case of a patch each field is optional.
So I get the post request file: post-user.schema.json:
{
"$schema": "http://json-schema.org/draft-07/schema#",
"$ref": "user.schema.json",
"required": [
"name",
"password",
"email"
]
}
And for my patch (path-user.schema.json:
{
"$schema": "http://json-schema.org/draft-07/schema#",
"$ref": "user.schema.json"
}
Now the issue that I am having is that my POST schema also marks a user like:
{
"name": "NoPassword",
"email": "nopassword#moba.nl",
"roles": []
}
Which is missing the required password field, as a valid JSON schema.
Apparently, this is not the way to assign required fields to a referenced data structure. I have tried to use google to see what I can find on the subject regarding this using searches like:
[ how to assign required field to referenced schema's ]
and I tried to obtain this info from the documentation.
I have no luck.
My questions now are:
A. Is it possible to assign required fields to a $referenced json schema data object.
B. If this is possible how to do it
C. If this is not possible, what would be a good way to approach this.
Any help is much appreciated.
Using $ref results in all other properties in the object being ignored, so you need to wrap your use of $ref.
Let's take a look at the spec:
An object schema with a "$ref" property MUST be interpreted as a
"$ref" reference. The value of the "$ref" property MUST be a URI
Reference. Resolved against the current URI base, it identifies the
URI of a schema to use. All other properties in a "$ref" object MUST
be ignored.
https://datatracker.ietf.org/doc/html/draft-handrews-json-schema-01#section-8.3
Then consider the schema you included in your question:
{
"$schema": "http://json-schema.org/draft-07/schema#",
"$ref": "user.schema.json",
"required": [
"name",
"password",
"email"
]
}
Reading the spec, you can see why required will be ignored.
Originally $ref was only designed to replace a WHOLE object, not ADD to the conditions for the object.
What you want is for multiple schemas to be applied to the instance. To do this, you use allOf.
{
"$schema": "http://json-schema.org/draft-07/schema#",
"allOf": [
{
"$ref": "user.schema.json"
},
{
"required": [
"name",
"password",
"email"
]
}
]
}
I loaded this schema into a demo for you to test at https://jsonschema.dev - although it doesn't support references yet, so I transcluded the reference, but the validation will work the same.
From draft-8 onwards, $ref will behave as you expect, as it becomes an applicator keyword rather than a keyword with special behaviours, meaning other keywords in the same object will not need to be ignored.
I have a large library of JSONSchemas and I'd like to structure it as follows:
my_object.json # a canonical definition of my_object
create_my_object_response.json # a response to a request to create a particular my_object
In this setup, my_object.json would define a general format of my_object, while create_my_object_response.json would expect some particular values for the fields of my_object. I'd like to structure the create_my_object_response.json schema like this:
{
"type": "object",
"definitions": {},
"$schema": "http://json-schema.org/draft-06/schema#",
"allOf": [
{ "$ref": "my_object.json#" },
{
"type": "object",
"properties": {
"id": { "const": 2 },
"name": { "const": "A Specific Name" }
}
}
]
}
my_object.json contains both the id and name properties but doesn't specify the const values. I tried this setup but it didn't seem to work. How can I represent this? Is it even possible?
I managed to figure this out, so posting the answer here for posterity.
The construction I had was actually right, but instead of "const": 2 I switched to "enum": [2]. I think this had to do with the version of the schema I was using.
I need to validate several JSON files against a schema in Golang.
I have been able to achieve it by using gojsonschema, that is really a straight forward library.
However, the problem I'm facing right now is that I have been given with schemas that have dependencies to another schemas and haven't found the way to load all the schemas that I need. Therefore, my validations always fail.
This is my main schema:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"$ref": "#/definitions/List",
"definitions": {
"List": {
"type": "array",
"items": {
"$ref": "#/definitions/Item"
}
},
"Item": {
"description": "An item ....",
"type": "object",
"additionalProperties": false,
"properties": {
"property01": {
"description": "The property01 code.",
"$ref": "./CommonTypes.json#/definitions/Type01Definition"
}
},
"required": [
"property01"
]
}
}
}
And, I have another one with common types:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"definitions": {
"Type01Definition": {
"description": "The definition for the type 01",
"type": "string",
"pattern": "^[A-Z0-9]{3}$"
}
}
}
Is there a way to load several schemas using that library? Or is there any other Golang library that allows to achieve that?
The way to refer to a file using $ref is to specify the absolute path of the file using a URL scheme. If you change the $ref to look like "$ref" : "file:///home/user/directory/CommonTypes.json#/definitions/Type01Definition, your example will work as expected.
If you need a bit more flexibility you can either try gojsonschema's NewReferenceLoaderFilesystem or switch to a different Golang library https://github.com/santhosh-tekuri/jsonschema. That library allows you to add custom resources so you can load several schemas at once.
Is it possible to define a regex once and re-use it? I have a few pretty complex regexes which I would like to use as the pattern for the value of a large number of properties of various different object in my schema. Doing Copy paste of this looks like asking for trouble further down the line, but I can't seem to find a suitable re-use example anywhere.
Cut down schema which illustrates what I want to do.
{
"$schema": "http://json-schema.org/draft-04/schema#",
"patterns": {
"fqdn_or_ipaddress": "(?=^.{4,253}$)(^((?!-)[a-zA-Z0-9-]{1,63}(?<!-)\\.)+[a-zA-Z]{2,63}$)||(((?:^[0-9])(?:(?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2}))(?![0-9])$)|(^\\*$))",
},
"properties": {
"server_hostname" : {
"type":"string",
"pattern": {"#ref", "#/patterns/address"},
},
"proxy_hostname" : {
"type":"string",
"pattern": {"#ref", "#/patterns/address"},
}
}
}
Doesn't validate here http://www.jsonschemavalidator.net/ because "pattern" is not a string. Is this a hole in the re-use. I've looked at patternProperties, but that seems to solve completely different use case.
You can only $ref a schema. You would need to do something like this.
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"server_hostname" : {
"$ref": "#/definitions/fqdn_or_ipaddress",
"description": "The server hostname"
},
"proxy_hostname" : {
"allOf": [{ "$ref": "#/definitions/fqdn_or_ipaddress" }],
"description": "The proxy hostname"
}
},
"definitions": {
"fqdn_or_ipaddress": {
"type": "string",
"pattern": "(?=^.{4,253}$)(^((?!-)[a-zA-Z0-9-]{1,63}(?<!-)\\.)+[a-zA-Z]{2,63}$)||(((?:^[0-9])(?:(?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2}))(?![0-9])$)|(^\\*$))"
}
}
}
EDIT
I added two examples of how to extend from a $ref. In the first, you can just add the description. It will be ignored, but it is not an error. Since description is just a meta-data keyword, this shouldn't be a problem.
In the second example, you can use allOf to wrap the $ref and you can add whatever keywords you need (even non-meta data keywords).
This answer highlights key structure for reusing the defined pattern. However, the example does not validate as:
The definition is given as part of the object reference with "definitions" whereas the "$ref" path is referencing a "patterns" definition that is nowhere to be found.
The pattern definition key is provided as "fqdn_or_ipaddress" whereas the paths in the properties refer to a "fqn_or_ipaddress" which is not defined.
Following is the corrected sample that passes the validation in the JSON schema validator:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"server_hostname" : {
"$ref": "#/definitions/fqdn_or_ipaddress",
"description": "The server hostname"
},
"proxy_hostname" : {
"allOf": [{ "$ref": "#/definitions/fqdn_or_ipaddress" }],
"description": "The proxy hostname"
}
},
"definitions": {
"fqdn_or_ipaddress": {
"type": "string",
"fqdn_or_ipaddress": "(?=^.{4,253}$)(^((?!-)[a-zA-Z0-9-]{1,63}(?<!-)\\.)+[a-zA-Z]{2,63}$)||(((?:^[0-9])(?:(?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2}))(?![0-9])$)|(^\\*$))"
}
}
}