JSON Schema - Foreign Key (FK) Validation - json

I have the JSON schemas below:
Schema A:
{
"$schema": "http://json-schema.org/draft-04/hyper-schema#",
"title": "A",
"type": "object",
"properties": {
"id": {
"type": "string"
},
"name": {
"type": "string"
},
"entityBId": {
"type": "string"
}
},
"required": [
"name",
"entityBId"
],
"links": [
{
"rel": "B",
"href": "myapi/EntityB?id={entityBId}"
}
]
}
Schema B :
{
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "B",
"type": "object",
"properties": {
"id": {
"type": "string"
},
"name": {
"type": "string"
}
},
"required": [
"name"
]
}
I'm trying to figure out if there is a way to run something like a integrity check based in a JSON Schema with external links/references.
For instance: When I receive an Object A with a entityBId = 1, I'd like to fetch this entity B running a GET in the endpoint declared in the links href and check if there is a valid object from the received id.
It will run like a deep validation and can be useful in a scenario without a defined DB schema.

As suggested by Raj Kamal, I made my own "link validation".
There is a function that looks for link attribute on schemas and validate the externals references directly on database and throw an exception if a valid reference is not found.

Related

Kafka cp-server fails message validation on a broker side

I need to validate messages on a brokers side.
I run cp-server (merely ran cp-all-in-one compose file).
created a topic
set confluent.value.schema.validation to true
registered a schema (JSON)
produced a message
It always fails. Why validation fails? Should I change configuration?
Schema:
{
"$id": "http://example.com/models/data-item-definition.json",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"description": "test data item 1",
"properties": {
"array_val": {
"items": {
"type": "string"
},
"type": [
"array",
"null"
]
},
"int_val": {
"type": "integer"
},
"string_val": {
"type": "string"
}
},
"required": [
"string_val",
"int_val"
],
"title": "data item",
"type": "object"
}
Message:
{
"string_val": "text",
"int_val": 10,
"array_val": ["one", "two", "three"]
}
An issue is that if confluent.value.schema.validation is true then to produce a message we need to send a magic byte and a schema ID at the beginning.
See Wire Format.

Conditional References in a JSON Schema

I want to write a single file JSON schema definition with several sub schemas that I can combine, depending on the payload.
The following schema validates, that my schema is working with my sample JSON response. (The response object has a wrong type for payload.role to make sure the schema catches this mistake!)
For clarity, I reduce it on the most important parts. A full working example can be found here: https://www.jsonschemavalidator.net/s/3KAaXjtg
Schema
{
"$schema": "http://json-schema.org/draft-07/schema",
"$id": "http://example.com/baseSchema.json",
"type": "object",
"required": [
"payload"
],
"properties": {
"payload": {
"$id": "#/properties/payload",
"type": "object",
// reference the right schema depending on the payload child key
// if `payload.user` reference `userSchema.json`
// if `payload.users` reference `usersSchema.json`
// if `payload.*` reference `*Schema.json`
"$ref": "userSchema.json"
}
},
"definitions": {
"user": {
"$id": "http://example.com/userSchema.json",
"type": "object",
"required": [
"user"
],
"properties": {
"user": {
"type": "object",
"$ref": "userProperties.json"
}
}
},
"users": {
"$id": "http://example.com/usersSchema.json",
"type": "object",
"required": [
"users"
],
"properties": {
"users": {
"type": "array",
"items": {
"$ref": "userProperties.json"
}
}
}
},
"userProperties": {
"$id": "http://example.com/userProperties.json",
"type": "object",
"properties": {
"firstName": {
"$id": "#/properties/payload/properties/user/properties/firstName",
"type": "string"
}
}
}
}
}
Response
{
"status": {
"code": 200,
"description": "User retrieved successfully."
},
"payload": {
"user": {
"firstName": "Joe",
"lastName": "Doe",
"role": "3", // for testing reasons, this is the wrong type!
"email": "doe#example.com",
"customerID": "",
"projects": [
"AIXG5mEg6QLl9rhVSE6m",
"Bs1bHiOIqKclwwis3CNf",
"NC2OUGVZXU35FA7iwRn4"
],
"status": "Status",
"id": "c555BSZnKLdHSRYqrU5hqiQo733j13"
}
}
}
So I've got a baseSchema.json that matches this response:
{
"status": {},
"payload": {}
}
payload gets extended by a certain key like payload.user = {} or payload.foo = {} and depending on that key, I want to extend schema with one of my definitions.
The following part only works for the key user:
"properties": {
"payload": {
"$id": "#/properties/payload",
"type": "object",
// reference the right schema depending on the payload child key
// if `payload.user` reference `userSchema.json`
// if `payload.users` reference `usersSchema.json`
// if `payload.*` reference `*Schema.json`
"$ref": "userSchema.json"
}
},
I failed to setup any conditions (with allOf, if, else), that would reference the correct sub-schema, based on the payload key.
Any hints and help to solve that is appreciated.
Schema and link to demo at the end... Let's look at how we got there...
In JSON Schema draft-07 and previous, you can't use $ref alongside other keywords. Other keywords are ignored. (In your schema http://example.com/userSchema.json you had type next to $ref). Fortunatly this didn't cause you any problems as you declare the type in the referenced schema. (You CAN do this with 2019-09 or above.)
The values for the keywords if, then, and else are schemas.
For the then subschema to be applied to your instance location, the if schema must come back as valid. If it fails, the else subschema value will be applied.
Our if condition checks for the presense of a specific key.
If the key exists, THEN apply the schema which references the correct schema.
Because you want the conditions to be mutually exclusive, you need to wrap the multiple conditions in a oneOf, and add else: false to the conditional checks. false as a schema makes validation fail.
Let me know if you want any further clarification on any of the above.
Demo: https://jsonschema.dev/s/HLniL
{
"$schema": "http://json-schema.org/draft-07/schema",
"$id": "http://example.com/baseSchema.json",
"type": "object",
"required": [
"payload"
],
"properties": {
"payload": {
"$id": "#/properties/payload",
"type": "object",
"oneOf": [
{
"if": {
"required": [
"user"
]
},
"then": {
"$ref": "userSchema.json"
},
"else": false
},
{
"if": {
"required": [
"users"
]
},
"then": {
"$ref": "usersSchema.json"
},
"else": false
}
]
}
},
"definitions": {
"user": {
"$id": "http://example.com/userSchema.json",
"type": "object",
"required": [
"user"
],
"properties": {
"user": {
"$ref": "userProperties.json"
}
}
},
"users": {
"$id": "http://example.com/usersSchema.json",
"type": "object",
"required": [
"users"
],
"properties": {
"users": {
"type": "array",
"items": {
"$ref": "userProperties.json"
}
}
}
},
"userProperties": {
"$id": "http://example.com/userProperties.json",
"type": "object",
"properties": {
"firstName": {
"type": "string"
}
}
}
}
}

Unique value for a property validation using json schema

I have a JSON object like:
{
"result": [
{
"name" : "abc",
"email": "abc.test#mail.com"
},
{
"name": "def",
"email": "def.test#mail.com"
},
{
"name": "xyz",
"email": "abc.test#mail.com"
}
]
}
and schema for this:
{
"definitions": {},
"$schema": "http://json-schema.org/draft-07/schema#",
"$id": "https://example.com/object1607582431.json",
"title": "Root",
"type": "object",
"required": [
"result"
],
"properties": {
"result": {
"$id": "#root/result",
"title": "Result",
"type": "array",
"default": [],
"uniqueItems": true,
"items": {
"$id": "#root/result/items",
"title": "Items",
"type": "object",
"required": [
"name",
"email"
],
"properties": {
"name": {
"$id": "#root/result/items/name",
"title": "Name",
"type": "string"
},
"email": {
"$id": "#root/result/items/email",
"title": "Email",
"type": "string"
}
}
}
}
}
}
I am looking for an option to check uniqueness for email irrespective of name. How I can validate that every email should be unique?
You can't. There are no keywords that let you compare one particular data value against another, other than uniqueItems, which compares an array element in toto against another.
The JsonSchema specification does not currently support this.
You can see the active GitHub issue here: https://github.com/json-schema-org/json-schema-vocabularies/issues/22
However, there are various extensions of JsonSchema that do validate unique fields within lists of objects.
If you happen to be using Python you can use the package (I created) JsonVL. It can be installed with pip install jsonvl and then run with jsonvl data.json schema.json.
Code examples in the GitHub repo: https://github.com/gregorybchris/jsonvl

Validation of presence of certain keys in an EAV payload depending on value of another field in the payload using json schema validator

I have following schema:
{
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Bulk Create Entity",
"title": "Bulk Create Entity",
"type": "object",
"properties": {
"idempotence_key": {
"description": "Idempotence Key",
"type": "string"
},
"requested_by": {
"description": "Requested By",
"type": "string"
},
"updated_by_process_id": {
"description": "Process id which is creating this entity.",
"type": "string"
},
"entity_creation_requests": {
"description": "Entity creation requests.",
"type": "array",
"minItems": 1,
"items": {
"title": "Entity Creation Request",
"type": "object",
"properties": {
"type": {
"description": "Entity Type",
"type": "string"
},
"taxonomies": {
"description": "Entity Taxonomies",
"type": "array",
"items": {
"title": "Taxonomy",
"type": "object",
"properties": {
"name": {
"description": "Taxonomy Name",
"type": "string"
},
"value": {
"description": "Taxonomy Value",
"type": "string"
}
}
}
}
}
}
}
},
"required": [
"idempotence_key",
"requested_by",
"updated_by_process_id",
"entity_creation_requests"
]
}
Here, the root level payload is an object which has a key "entity_creation_requests" which is an array of objects which in turn have an array property "taxonomies" which contains a list of key-value pairs.
Now depending on the "type" of the request under "entity_creation_requests", I want to validate the presence of certain keys in the taxonomies list.
For example, for the creation request of type "product", I want keys "MRP", "seller" etc. to be present in the taxonomies list.
Can we achieve this using JSON schema validator?
{Here is something I have created: https://codebeautify.org/jsonviewer/cb80c728}
Are there any other alternatives?
I am using this in an spring boot application (Java).

Create MongoDB collection with standard JSON schema

I want to create a MongoDB collection using an JSON schema file.
Suppose the JSON file address.schema.json contain address information schema (this file is one of the Json-schema.org's examples):
{
"$id": "https://example.com/address.schema.json",
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "An address similar to http://microformats.org/wiki/h-card",
"type": "object",
"properties": {
"post-office-box": {
"type": "string"
},
"extended-address": {
"type": "string"
},
"street-address": {
"type": "string"
},
"locality": {
"type": "string"
},
"region": {
"type": "string"
},
"postal-code": {
"type": "string"
},
"country-name": {
"type": "string"
}
},
"required": [ "locality", "region", "country-name" ],
"dependencies": {
"post-office-box": [ "street-address" ],
"extended-address": [ "street-address" ]
}
}
What is the MongoDB command, such as mongoimport or db.createCollection to create a MongoDB collection using the above schema?
It can be nice if I can use the file directly in MongoDB with the need of changing the file format manually.
I wonder if the JSON schema format is a standard one, why do I need to change it to adopt it for MongoDB. Why MongoDB does not have this functionality built-in?
You can create it via createCollection command or shell helper:
db.createCollection(
"mycollection",
{validator:{$jsonSchema:{
"description": "An address similar to http://microformats.org/wiki/h-card",
"type": "object",
"properties": {
"post-office-box": { "type": "string" },
"extended-address": { "type": "string" },
"street-address": { "type": "string" },
"locality": { "type": "string" },
"region": { "type": "string" },
"postal-code": { "type": "string" },
"country-name": { "type": "string" }
},
"required": [ "locality", "region", "country-name" ],
"dependencies": {
"post-office-box": [ "street-address" ],
"extended-address": [ "street-address" ]
}
}}})
You only need to specify bsonType instead of type if you want to use a type that exists in bson but not in generic json schema. You do have to remove the lines $id and $schema as those are not supported by MongoDB JSON schema support (documented here)
The only option which you can use is adding jsonSchema validator during collection creating: https://docs.mongodb.com/manual/reference/operator/query/jsonSchema/#document-validator. It will mean that any document which you will insert/update in your collection will have to match with the provided schema