Background
I'm building an API with Fastify and it's my first time using JSON schema validation. The idea is that it will both make the server code more efficient and help our developers as they learn how to consume my API.
Problem
I'm trying to validate a route that allows the client to query kittens by name only. A successful formed query would look /kittens?name=fluffykins.
My schema for this route looks like this:
{
querystring: {
type: 'object',
name: { type: 'string' },
}
}
Question
How can I make my schema validator accept only queries on name and reject other queries like /kittens?age=1? My preference is for the schema validator to handle it independently of my controller code and for it to also support queries that we may add in the future.
Thanks!
As is typical of when I post a question to SO, I find an answer myself shortly after. The following is what worked for me but I'm still interested to hear if there are other better ways of doing this!
{
querystring: {
type: 'object',
properties: {
name: { type: 'string' }
},
anyOf: [
{
required: [ 'name' ]
}
],
},
}
I'm not quite sure what you are trying to do with the anyOf, so I might be missing something, but I believe this is what you want (if you are using draft-06 or later):
{
"type": "object",
"required": ["name"],
"propertyNames": {"enum": ["name"]},
"properties": {
"name": {"type": "string"}
}
}
The propertyNames ensures that name is the only acceptable property. You can also do this by setting "additoinalProperties": false instead (if you are useing draft-04 you have to do this as it does not support propertyNames). But doing that can cause unexpected problems when you try to combine schemas, so if you can use draft-06 propertyNames is more flexible.
Here's the draft-04 version:
{
"type": "object",
"required": ["name"],
"properties": {
"name": {"type": "string"}
},
"additionalProperties": false
}
Related
Consider an example in
https://github.com/json-schema-org/JSON-Schema-Test-Suite/blob/master/tests/draft6/ref.json#L414
the original schema is:
{
"allOf": [{
"$ref": "http://localhost:1234/bar#foo"
}],
"definitions": {
"A": {
"$id": "http://localhost:1234/bar#foo",
"type": "integer"
}
}
}
which is valid.
For this schema S, if I want to create a new schema by adding a combining schema outside this schema, like: {"not":S}, which is:
{
"not": {
"allOf": [{
"$ref": "http://localhost:1234/bar#foo"
}],
"definitions": {
"A": {
"$id": "http://localhost:1234/bar#foo",
"type": "integer"
}
}
}
}
with the Location-independent identifier with absolute URI, also use $id. But it is invalid.
The error message(using https://www.jsonschemavalidator.net/):
Error parsing schema
Message:
Error when resolving schema reference 'http://localhost:1234/bar#foo'. Path 'not.allOf[0]', line 3, position 20.
A similar example is:
{
"not": {
"allOf": [{
"$ref": "#foo"
}],
"definitions": {
"A": {
"$id": "#foo",
"type": "integer"
}
}
}
}
with the Location-independent identifier, is also invalid.
I cannot figure out why it is invalid after adding a parent schema if there are $ref and $id at the same time...
Here {"not":S} is just one possibility, also can consider {"anyOf":[{S}]}.
I know that $id declares a base URI against which $ref URI-references are resolved.
But what is the problem with the above schemas?
And how should I correct them?
I will so appreciate it if someone helps me out...
You are correct here, and your schema is correct. I would call this a bug in the implementation.
Your schema is valid for JSON Schema draft-07, but not JSON Schema draft 2019-09 or above. For 2019-09 and above, your $id value can't contain a non-empty fragment.
You can see the correct and valid behaviour on another web based validator, a web based version of the HyperJump validator: https://json-schema.hyperjump.io
This web based implementation defaults to latest (2020-12), but you can specify a draft version using $schema (for example "$schema": "http://json-schema.org/draft-07/schema#").
To see this working on HyperJump, either modify your schema to remove fragments in the identifier URIs, or specify that you're using draft-07.
Wondering if I can create a "dynamic mapping" within an elasticsearch index. The problem I am trying to solve is the following: I have a schema that has an attribute that contains an object that can differ greatly between records. I would like to mirror this data within elasticsearch if possible but believe that automatic mapping may get in the way.
Imagine a scenario where I have a schema like the following:
{
name: string
origin: string
payload: object // can be of any type / schema
}
Is it possible to create a mapping that supports this? I do not need to query the records by this payload attribute, but it would be great if I can.
Note that I have checked the documentation but am confused on if what elastic calls dynamic mapping is what I am looking for.
It's certainly possible to specify which queryable fields you expect the payload to contain and what those fields' mappings should be.
Let's say each doc will include the fields payload.livemode and payload.created_at. If these are the only two fields you'll want to perform queries on, and you'd like to disable dynamic, index-time mappings autogenerated by Elasticsearch for the rest of the fields, you can use dynamic templates like so:
PUT my-payload-index
{
"mappings": {
"dynamic_templates": [
{
"variable_payload": {
"path_match": "payload",
"mapping": {
"type": "object",
"dynamic": false,
"properties": {
"created_at": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss"
},
"livemode": {
"type": "boolean"
}
}
}
}
}
],
"properties": {
"name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"origin": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
}
}
}
}
Then, as you ingest your docs:
POST my-payload-index/_doc
{
"name": "abc",
"origin": "web.dev",
"payload": {
"created_at": "2021-04-05 08:00:00",
"livemode": false,
"abc":"def"
}
}
POST my-payload-index/_doc
{
"name": "abc",
"origin": "web.dev",
"payload": {
"created_at": "2021-04-05 08:00:00",
"livemode": true,
"modified_at": "2021-04-05 09:00:00"
}
}
and verify with
GET my-payload-index/_mapping
no new mappings will be generated for the fields payload.abc nor payload.modified_at.
Not only that — the new fields will also be ignored, as per the documentation:
These fields will not be indexed or searchable, but will still appear in the _source field of returned hits.
Side note: if fields are neither stored nor searchable, they're effectively the opposite of enabled.
The Big Picture
Working with variable contents of a single, top-level object is quite standard. Take for instance the stripe event object — each event has an id, an api_version and a few other shared params. Then there's the data object that's analogous to your payload field.
Now, all is fine, until you need to aggregate on the contents of your payload. See, since the content is variable, so are the data paths / accessors. But wildcards in aggregation paths don't work in Elasticsearch. Scripts do but are onerous to maintain.
Back to stripe. They partially solved it through what they call polymorphic, typed hashes — as discussed in their blog on API design:
A pretty neat approach that's worth emulating.
P.S. I discuss dynamic templates in more detail in the chapter "Mapping Automation" of my ES Handbook.
I have an array of options. Each item in the array will have text and a boolean value isAnswer I am trying to validate in a way that only one of the items can be and must be marked true. Anything else should be invalid. Via two items are true or 0 are true should fail. I have been playing around with oneOf as that seems to make the most sense however it always validates successful.
First off, is this possible to validate?
Secondly am I on the right track?
Thanks for any help you can offer
"question": {
"title": "Question",
"type": "object",
"properties": {
"options": {
"title": "Options",
"type": "array",
"minItems": 2,
"maxItems": 10,
"items": {
"title": "Option",
"type": "object",
"properties": {
"isAnswer": {
"title": "Answer",
"type": "boolean",
"format": "checkbox",
"default": false
},
"text": {
"title": "Choice Text",
"type": "string"
},
},
"oneOf": [
{
"properties": {
"isAnswer": true
}
}
]
}
}
}
}
this question is almost identical to How to enforce only one property value to true in an array (JSON Schema) - check the answer to that one.
this is a little bit different because you have a maxItems - this opens up an ugly option of brute-forcing the possible combinations.
I'm going to pretend your maxItems is 3 instead of 10 to cut down on verbosity:
definitions:
correctAnswer:
{properties: {isAnwser: {const: true}}}
incorrectAnswer:
{properties: {isAnwser: {const: false}}}
oneOf:
- items: [{'$ref': '#/definitions/correctAnswer'}, {'$ref': '#/definitions/incorrectAnswer'}, {'$ref': '#/definitions/incorrectAnswer'}]
- items: [{'$ref': '#/definitions/incorrectAnswer'}, {'$ref': '#/definitions/correctAnswer'}, {'$ref': '#/definitions/incorrectAnswer'}]
- items: [{'$ref': '#/definitions/incorrectAnswer'}, {'$ref': '#/definitions/incorrectAnswer'}, {'$ref': '#/definitions/correctAnswer'}]
ugly and unmaintainable! better to write this requirement in your code, unless / until you can use 2019-09.
other notes:
oneOf checks that one of a set of schemas validates against the instance, not whether one element of an array instance validates against a schema.
you have "properties": {"isAnswer": true} - what you want there is "properties": {"isAnswer": {"const": true}}. what you have will use the true schema which matches any instance. const matches an instance equal to its value.
This kind of test is called "business logic" or "data consistency validation" which falls outside the scope of JSON Schema (and most validation tools, such as XML Schema, RELAX NG, and the like). See Scope of JSON Schema Validation for more information on this.
It may be technically possible to write a schema that produces the results you're looking for, by crafting different schemas, one for each possible correct answer. Similar solutions are used to validate different classes of objects, where a "category" or "type" field determines how to validate all the other properties.
However, JSON Schema does not support comparing one value with another in the general.
As far as the schema layout goes, your schema is acceptable; but I would consider specifying the answer separately from the questions:
{
"options": [
{ "text": "Butaful" },
{ "text": "Bueatful" },
{ "text": "Beautiful" },
{ "text": "Beeyoutiful" }
],
"answer": "Beautiful"
}
This adds some redundancy, and supports freeform answers.
I'm trying to create a JSON schema for an existing JSON file that looks something like this:
{
"variable": {
"name": "age",
"type": "integer"
}
}
In the schema, I want to ensure the type property has the value string or integer:
{
"variable": {
"name": "string",
"type": {
"type": "string",
"enum": ["string", "integer"]
}
}
}
Unfortunately it blows up with message: ValidationError {is not any of [subschema 0]....
I've read that there are "no reserved words" in JSON schema, so I assume a type of type is valid, assuming I declare it correctly?
The accepted answer from jruizaranguren doesn't actually answer the question.
The problem is that given JSON (not JSON schema, JSON data) that has a field named "type", it's hard to write a JSON schema that doesn't choke.
Imagine that you have an existing JSON data feed (data, not schema) that contains:
"ids": [ { "type": "SSN", "value": "123-45-6789" },
{ "type": "pay", "value": "8675309" } ]
What I've found in trying to work through the same problem is that instead of putting
"properties": {
"type": { <======= validation chokes on this
"type": "string"
}
you can put
"patternProperties": {
"^type$": {
"type": "string"
}
but I'm still working through how to mark it as a required field. It may not be possible.
I think, based on looking at the "schema" in the original question, that JSON schemas have evolved quite a lot since then - but this is still a problem. There may be a better solution.
According to the specification, in the Valid typessection for type:
The value of this keyword MUST be either a string or an array. If it is an array, elements of the array MUST be strings and MUST be unique.
String values MUST be one of the seven primitive types defined by the core specification.
Later, in Conditions for successful validation:
An instance matches successfully if its primitive type is one of the types defined by keyword. Recall: "number" includes "integer".
In your case:
{
"variable": {
"name": "string",
"type": ["string", "integer"]
}
}
In the app we're developing, we create all the JSON at the server side using dinamically generated configs (JSON objects). We use that for stores (and other stuff, like GUIs), with a dinamically generated list of its data fields.
With a JSON like this:
{
"proxy": {
"type": "rest",
"url": "/feature/163",
"timeout": 600000
},
"baseParams": {
"node": "163"
},
"fields": [{"name": "id", "type": "int" },
{"name": "iconCls", "type": "auto"},
{"name": "text","type": "string"
},{ "name": "name", "type": "auto"}
],
"xtype": "jsonstore",
"autoLoad": true,
"autoDestroy": true
}, ...
Ext will gently create an "implicit model" with which I'll be able to work with, load it on forms, save it, delete it, etc.
What I want is to specify through a JSON config not the fields, but the model itself. Is this possible?
Something like:
{
model: {
name: 'MiClass',
extends: 'Ext.data.Model',
"proxy": {
"type": "rest",
"url": "/feature/163",
"timeout": 600000},
etc... }
"autoLoad": true,
"autoDestroy": true
}, ...
That way I would be able to create a whole JSON from the server without having to glue stuff using JS statements on the client side.
Best regards,
I don't see why not. The syntax to create a model class is similar to that of store and components:
Ext.define('MyApp.model.MyClass', {
extend:'Ext.data.Model',
fields:[..]
});
So if you take this apart you could call Ext.define(className,config);
where className is a string and config is a JSON object and both are generated on the server.
There's no way to achieve what I want.
The only way you can do it is by means of defining the fields of the Ext.data.Store and have it to generate the implicit model by using the fields configuration.