Use draft-7 json schema CLOSED_CONTENT_MODEL and BACKWARD compatibility and confluent schema registry 7.2.1-post.
deps:
implementation 'com.github.victools:jsonschema-generator:4.26.0'
implementation 'io.confluent:kafka-schema-registry:7.2.1'
schema example
{
"type" : "object",
"properties" : {
"name" : {
"type" : "string",
"description" : "String"
},
"timeDescription" : {
"type" : "string",
"description" : "String"
},
},
"required" : [ "name"],
"additionalProperties" : false
}
I am trying to find incompatibilities between schema via
{{base_url}}/compatibility/subjects/subject/versions/latest?verbose=true
(timeDescription is absent but not required)
{
"type" : "object",
"properties" : {
"name" : {
"type" : "string",
"description" : "String"
},
},
"required" : [ "name"],
"additionalProperties" : false
}
and see
```json
{
"is_compatible": false,
"messages": [
"Found incompatible change: Difference{jsonPath='#/properties/timeDescription', type=PROPERTY_REMOVED_FROM_CLOSED_CONTENT_MODEL}"
]
}
How to disable incompatibilities for case when optional field exists in one schema and absent in another?
Thanks for your answers.
It is not possible to enable or disable compatibilities. You might need select different Schema type.
Related
I am trying to validate this specific schema:
{
"messages": [
{
"name": "test msg",
"id": "0x100",
"signals": {
"0": {"name": "Engine RPM", "bit_length": 16},
"16": {"name": "Gear", "bit_length": 3},
"19": {"name": "Battery Voltage", "bit_length": 5}
}
}
]
}
I am using the python from jsonschema import Draft4Validator to validate this schema... however I am not sure how to continue.
This is my current schema validation so far:
{
"$schema" : "https://json-schema.org/schema#",
"type" : "object",
"properties" :
{
"messages" :
{
"type" : "array",
"items" :
{
"properties" :
{
"name" :
{
"type" : "string"
},
"id" :
{
"type" : "string"
},
"signals" :
{
"type" : "object"
},
"properties" :
{
}
},
"required": ["name", "id", "signals"]
}
}
}
}
The problem I am facing is I am not sure how to deal with the objects that are in the "signals" key as they start with a string and are NOT consistent ("0", "16", "19")... How could I go about validating this by ensuring the type is always a string, disregarding whether or not the string is consistent.
Thanks to all of those who reply in advance.
I was able to accomplish this by doing the following:
{
"$schema" : "https://json-schema.org/schema#",
"type" : "object",
"properties" :
{
"messages" :
{
"type" : "array",
"items" :
{
"properties" :
{
"name" :
{
"type" : "string"
},
"id" :
{
"type" : "string"
},
"signals" :
{
"type" : "object"
},
"properties" :
{
"start_bit" :
{
"type" : "object",
"properties" :
{
"name" :
{
"type" : "string"
},
"bit_length" :
{
"type" : "string"
},
"factor" :
{
"type" : "string"
},
"offset" :
{
"type" : "string"
}
},
"required" : ["name", "bit_length", "factor", "offset"]
}
}
},
"required": ["name", "id", "signals"]
}
}
}
}
To "avoid" having to keep the string consistent, in the validator file, I can put any string (obviously it makes more sense to name the string what is represents, in my case "start_bit") and then by NOT having it be required.
How can I declare multiple schemas and let a validator select one depending on a property?
For example I would like this json to be validated against second schema for type2
{
"id2": 1,
"name": "A green door",
"price": 12.50,
"tags": ["home", "green"],
"schemaToValidate" : "type2"
}
A first schema definition for object of type1:
{
"$schema": "http://json-schema.org/draft-06/schema#",
"title": "Product",
"description": "A product from Acme's catalog",
"type": "object",
"required": ["id1"],
"schemaForType" : "type1"
}
A second schema definition for object of type2:
{
"$schema": "http://json-schema.org/draft-06/schema#",
"title": "Product",
"description": "A product from Acme's catalog",
"type": "object",
"required": ["id2"],
"schemaForType" : "type2"
}
I tried to find a sample for this at http://json-schema.org/example1.html but didn't realized if is even possible.
It's not possible to have multiple schema depending on a type but there is an alternative: change your example to use oneOf (also simplified to be easier on the schema):
{
"Type2" : {
"id2": 1
}
}
And with a schema that work on one of Type1 and Type2 can be added:
{
"description" : "schema validating Type1 and Type2",
"type" : "object",
"oneOf" : [{
"properties" : {
"Type1" : {
"type": "object",
"properties" : {
"id1" : {
"type" : "integer"
}
},
"additionalProperties": false
}
},
"required" : [ "Type1" ]
}, {
"properties" : {
"Type2" : {
"type": "object",
"properties" : {
"id2" : {
"type" : "integer"
}
},
"additionalProperties" : false
}
},
"required" : ["Type2"]
}
]
}
Another example of AnyOf
I would like to know what the proper avro schema would be for some json to avro conversion that is in this format:
{"entryDate": "2018-01-26T12:00:40.930"}
My schema:
{
"type" : "record",
"name" : "schema",
"fields" : [{
"name" : "entryDate",
"type" : ["null", {
"type" : "long",
"logicalType" : "timestamp-micros"
}],
"default" : null
}]
}
I keep getting
`'Cannot convert field entryDate: Cannot resolve union:
"2018-01-26T12:00:40.930"
not in
["null",{"type":"long","logicalType":"timestamp-millis"}]'`
It was a silly mistake...obviously I was storing the timestamp value as a string so the avro schema needed a string instead of long for type.
ie.
{
"type" : "record",
"name" : "schema",
"fields" : [{
"name" : "entryDate",
"type" : ["null", {
"type" : `**"long"**`,
"logicalType" : "timestamp-micros"
}],
"default" : null
}]
}
should be
{
"type" : "record",
"name" : "schema",
"fields" : [{
"name" : "entryDate",
"type" : ["null", {
"type" : `**"string"**`,
"logicalType" : "timestamp-micros"
}],
"default" : null
}]
}
doh!
I am trying to figure out how oneOf works by building a schema which validates two different object types. For example a person (firstname, lastname, sport) and vehicles (type, cost).
Here are some sample objects:
{"firstName":"John", "lastName":"Doe", "sport": "football"}
{"vehicle":"car", "price":20000}
The question is what have I done wrongly and how can I fix it. Here is the schema:
{
"description": "schema validating people and vehicles",
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"required": [ "oneOf" ],
"properties": { "oneOf": [
{
"firstName": {"type": "string"},
"lastName": {"type": "string"},
"sport": {"type": "string"}
},
{
"vehicle": {"type": "string"},
"price":{"type": "integer"}
}
]
}
}
When I try to validate it in this parser:
https://json-schema-validator.herokuapp.com/
I get the following error:
[ {
"level" : "fatal",
"message" : "invalid JSON Schema, cannot continue\nSyntax errors:\n[ {\n \"level\" : \"error\",\n \"schema\" : {\n \"loadingURI\" : \"#\",\n \"pointer\" : \"/properties/oneOf\"\n },\n \"domain\" : \"syntax\",\n \"message\" : \"JSON value is of type array, not a JSON Schema (expected an object)\",\n \"found\" : \"array\"\n} ]",
"info" : "other messages follow (if any)"
}, {
"level" : "error",
"schema" : {
"loadingURI" : "#",
"pointer" : "/properties/oneOf"
},
"domain" : "syntax",
"message" : "JSON value is of type array, not a JSON Schema (expected an object)",
"found" : "array"
} ]
Try this:
{
"description" : "schema validating people and vehicles",
"type" : "object",
"oneOf" : [
{
"type" : "object",
"properties" : {
"firstName" : {
"type" : "string"
},
"lastName" : {
"type" : "string"
},
"sport" : {
"type" : "string"
}
}
},
{
"type" : "object",
"properties" : {
"vehicle" : {
"type" : "string"
},
"price" : {
"type" : "integer"
}
},
"additionalProperties":false
}
]
}
oneOf need to be used inside a schema to work.
Inside properties, it's like another property called "oneOf" without the effect you want.
Is there a standard approach to specifying a property to be a dictionary or map keyed by string with a value type T specified somewhere else in the schema?
For example, suppose you want to model a user's favorite movies where the key type is the name of the movie and the value type is some set of attributes about the movie (year made, budget, gross income, etc.)
I imagine you could model first a MovieDataPair as a type with name property and a value property containing the desired attributes. Then the map would be an array of those. But, then you would need a special unique constraint that ensured any movie name only appeared once.
Is there something in json schema to support this, or a standard pattern used for it?
If not built in support in json schema, what about other schema solutions?
After some study I've come up with the following answer:
The best way to see this in action is to find some examples. It
happens that there are several examples of this in the draft04 schema
itself (definitions, properties, patternProperties,...) and they
usually follow the same pattern.
For example, the definitions property of the draft04 schema defines what
should appear in a schema at the definitions property. Here is the
subschema associated with the definitions property:
"definitions": {
"type": "object",
"additionalProperties": { "$ref": "#" },
"default": {}
},
This says the entry at "#/definitions/" must be an object. The fact
that it is a json object means it will have unique keys itself. Now
for the values in the object, that is what additionalProperties is
designed to describe. In this case it says that the value of each
property must itself conform to the root of the schema "#". What this
means is that each value in the definitions property object of a valid json schema
object must also be a schema.
If this were typed like C++ it might look like:
std::map< std::string, Schema > definitions;
Effectively a map with a string key can be thought of as like a json
object with a structured value type. So, to create your own:
std::map< std::string, T >
First define the schema for T. For example:
"definitions" : {
"movie" : {
"properties": {
"title" : { "type" : "string" },
"year_made" : { "type" : "integer" },
"rating" : { "type" : "integer" }
}
}
}
For the value type T stored, decide if you want to allow any
properties, as long as these specified properties are typed as
specified above. If you only want these properties, add
"additionalProperties" : false
"definitions" : {
"movie" : {
"additionalProperties" : false,
"properties": {
"title" : { "type" : "string" },
"year_made" : { "type" : "integer" },
"rating" : { "type" : "integer" }
}
}
}
Also decide if you actually require all of the properties to be
present for the movie to be valid. If so, add a required entry.
"definitions" : {
"movie" : {
"additionalProperties": false,
"required" : [ "title", "year_made", "rating" ],
"properties": {
"title" : { "type" : "string" },
"year_made" : { "type" : "integer" },
"rating" : { "type" : "integer" }
}
},
Now the shape T for movie is defined. Create a definition for
the collection, or map of movies referencing the movie schema
defined as was done by definitions in the draft schema. Note: in
the "movie_map" additionalProperties has a different meaning than
that of "movie". In the case of "movie" it is a boolean false
which indicates no additional properties beyond what is listed in
properties. In the case of "movie_map" it means - if there are
additional properties, they must look like this schema. But,
since no properties have been specified in movie_map it really means
all properties in the object instance must conform to #/definitions/movie. Now all
values in a "movie_map" will look like the defined movie schema.
{
"definitions" : {
"movie" : {
"additionalProperties": false,
"required" : [ "title", "year_made", "rating" ],
"properties": {
"title" : { "type" : "string" },
"year_made" : { "type" : "integer" },
"rating" : { "type" : "integer" }
}
},
"movie_map" : {
"type": "object",
"additionalProperties": { "$ref": "#/definitions/movie" },
"default": {}
}
}
}
Now use the defined schema movie_map somewhere within the schema:
{
"title" : "movie data",
"additionalProperties" : false,
"required" : [ "movies" ],
"properties" : {
"movies" : { "$ref" : "#/definitions/movie_map" }
},
"definitions" : {
"movie" : {
"additionalProperties": false,
"required" : [ "title", "year_made", "rating" ],
"properties": {
"title" : { "type" : "string" },
"year_made" : { "type" : "integer" },
"rating" : { "type" : "integer" }
}
},
"movie_map" : {
"type": "object",
"additionalProperties": { "$ref": "#/definitions/movie" },
"default": {}
}
}
}
Here is a sample object, which can be thought of as a map, of movies
that validates against the schema:
{
"movies" : {
"the mission" : {
"title":"The Mission",
"year_made":1986,
"rating":5
},
"troll 2" : {
"title":"Troll 2",
"year_made":1990,
"rating":2
}
}
}
If I wanted to model a structure for users favorites movies (remind Json Schema is intended for structure validation) I would make something like:
{
"description":"moviesFan",
"properties": [
"favoriteMovies": {
"type":"array",
"uniqueItems":True
"allOf": [{ "$ref": "#/definitions/movie" }]
}
],
"definitions": {
"movie": {
"type": "object",
"properties": {
"yearMade": {}
...
}
}
}
Does it make sense to you?
Here's my way to support for map. Hope to help.
{
"type": "object",
"title": "map data",
"required": [
"map"
],
"properties": {
"sOnePurRecord": {
"title": "map",
"additionalProperties": false,
"properties": {
"mapItem": {
"type": "object",
"maxProperties": 10,
"minProperties": 1,
"patternProperties": {
"^[a-zA-Z0-9]{5,20}$": {
"$ref": "#/definitions/value"
}
},
"additionalProperties": {
"$ref": "#/definitions/value"
}
}
},
"required": [
"mapItem"
]
}
},
"definitions": {
"value": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"id": {
"type": "integer"
}
}
}
}
}