I have this little issue. I am building API tests with postman.
One of my tests want to validate a Json response.
This is the kind of response that I have received:
{
"comuni": [
{
"istat": "015002",
"code": "A010",
"comune": "ABBIATEGRASSO",
"provincia": "MI",
"cap": "20081",
"latitude": 45.393036,
"longitude": 8.919824,
"soppresso": false,
"regione": "Lombardia",
"parte_italia": "nord",
"is_provincia": 0,
"nome_provincia": "Milano"
},
...
...
]};
So I receive an array of objects like this one above.
This is the test that I wrote:
var schema = {
"comuni" :
[
{
"istat" : {
"type" : "Integer"
},
"code" : {
"type" : "string"
},
"comune" : {
"type" : "string"
},
"provincia" : {
"type" : "string"
},
"cap" : {
"type" : "integer"
},
"latitude" : {
"type": "Number"
},
"longitude" : {
"type": "Number"
},
"soppresso": {
"tyoe" : "boolean"
},
"regione" : {
"type" : "string"
},
"parte_italia": {
"type": "string"
},
"is_provincia": {
"type": "integer"
},
"nome_provincia": {
"type": "string"
}
}]
}
pm.test("JSON schema validation", function() {
var paperwork = pm.response.json();
var result = tv4.validate(paperwork, schema, false, true);
if (result !== true) {
console.log('Schema validation failed:', tv4.error);
}
/*console.log(tv4.error.dataPath);*/
pm.expect(result).to.be.true;
console.log(JSON.stringify(result));
});
But the test fails:
Schema validation failed: unknown property (not in schema)
Obviously I am doing something wrong with the schema, but I do not understand what.
Your schema is incorrect. It should be like this.
{
"description": "Any validation failures are shown in the right-hand Messages pane.",
"type": "object",
"properties": {
"foo": {
"type": "number"
},
"bar": {
"type": "string",
"enum": [
"a",
"b",
"c"
]
}
}
}
And data should look like,
{
"foo": 12345,
"bar": "a"
}
Refer below link for more examples, like Array/Objects etc.
Related
I'm new to Json Schema validation. I think the validation should fail, but it passes. Not sure why the If/then is not forcing the required field. I believe I formatted the If/Then correctly.
JSON:
{
"name": "Battery Wear",
"triggerAlert": {
"trigger": "When",
"timeSpan": 50,
"timeSpanMeasure": "Hours"
}
}
SCHEMA:
{
"$schema": "http://json-schema.org/draft-07/schema",
"type": "object",
"required": [
"name",
"triggerAlert"
],
"properties": {
"name": {
"type": "string"
},
"triggerAlert": {
"type": "object",
"required": ["trigger"],
"properties": {
"trigger": {
"type": "string",
"enum": ["Always","When"]
},
"numberOfEvents": {
"type": "integer"
},
"timeSpan": {
"type": "integer"
},
"timeSpanMeasure": {
"type": "string"
}
},
"if": { "properties": {"trigger": {"enum": ["When"]} } },
"then": {
"required": [
"numberOfEvents",
"timeSpan",
"timeSpanMeasure"
]
}
}
}
}
Depending on the implementation you are using, you may not have support for these conditionals. if/then/else were only added in specification draft 7.
The schema is correct; the expected error result is:
{
"errors" : [
{
"error" : "missing property: numberOfEvents",
"instanceLocation" : "/triggerAlert",
"keywordLocation" : "/properties/triggerAlert/then/required"
},
{
"error" : "subschema is not valid",
"instanceLocation" : "/triggerAlert",
"keywordLocation" : "/properties/triggerAlert/then"
},
{
"error" : "not all properties are valid",
"instanceLocation" : "",
"keywordLocation" : "/properties"
}
],
"valid" : false
}
I'm trying to validate a required property with if statement in JsonSchema, but it isn't informing me the property's error in detail.
The validation has been made correctly, but the error don't specify the property and which validation failed.
It works for required properties in the root level of an object, but when I specify a required property inside an object, it just warn that the json doesn't match and specify the then or else schema's path.
Schema example:
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"required": [
"name",
"partners"
],
"properties": {
"partners": {
"type": "array",
"items": {
"type": "object",
"if": {
"properties": {
"juridical": {
"type": "object"
},
"natural": {
"type": "null"
}
}
},
"then": {
"properties": {
"juridical": {
"required": [ "tradeName" ],
"properties": {
"tradeName": {
"type": "string"
}
}
}
}
},
"else": {
"properties": {
"natural": {
"required": ["name"],
"properties": {
"name": {
"type": "string"
}
}
}
}
}
}
}
}
}
Json example:
{
"name": "Joao",
"partners": [
{
"juridical": null,
"natural": {
}
},
{
"juridical": {
"tradeName": ""
},
"natural": null
}
]
}
It should warns that the first partner have the "name" required (Required properties are missing from object: name.), instead it only tell me: JSON does not match schema from 'else'..
With a simple schema like this it works as expected:
Schema example:
{
"if": { "properties": { "power": { "minimum": 9000 } } },
"then": { "required": [ "disbelief" ] },
"else": { "required": [ "confidence" ] }
}
Json example:
{ "power": 10000 }
I'm using JsonSchemaValidator.net to verify the results.
Basically, JSON document either validates against JSON schema or not. This logic goes down through all sub-schemas and conditionals.
The content of error messages depends on specific implementation of JSON Schema validator. The one you use comes from a specific provider. As pointed by Relequestal, you cannot expect specific type of error message handling from specific implementation, unless it's what the provider documentation describes.
How about filing a suggestion to authors of the validator you use about extending messages for if-then-else case, feeding in your case?
Alternative approach: As I understand, your goal is to get as much specific error information as possible with this specific validator. It is what it is, so an alternative schema might fit the goal. As JSON Schema itself is a JSON document, thus you may consider a workaround by naming nodes in Schema in some consistent manner and using one of logical operators ("anyOf" (logical OR), "allOf" (logical AND), "oneOf" (logical XOR) ) instead of if-then-else.
Please note: schema based validator, in case of "allOf", "anyOf", "oneOf" is expected to run through all schemas until the logical condition is satisfied.
"allOf" - will check always if JSON doc validates against all schemas
(AND)
"anyOf" - will check if JSON doc validates at least against 1 schema
(OR, so some validator implementations might stop checking after
first positive result as it's sufficient to evaluate check against
"anyOf" to true),
"oneOf" - will check always if JSON doc validates exactly against
one of enlisted schemas by checking against all of them (XOR)
(see: https://json-schema.org/draft-07/json-schema-validation.html#rfc.section.6.7.1 )
Thus if validated instance doesn't match schemas in above cases, the validator implementation may produce some 'false positives' in terms of error messages, as it will enlist issues encountered vs all schemas. It simply can't read our minds and guess what we meant by providing specific JSON doc, so it throws all on us to judge and decide.
One of many solutions could be to define variants of "juridical" and "natural" and combine them logically into schemas as in your case it seems you expect:
either juridical is an object (+relevant constraints) AND natural is not an object or juridical is not an object and natural is an object
(+relevant constraints).
Alternative schema (please note the "examples" section containing some test JSON documents):
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"required": [
"name",
"partners"
],
"properties": {
"partners": {
"type": "array",
"items": {
"type": "object",
"anyOf" : [
{"$ref" : "#/definitions/juridical-is-object-and-natural-is-null"},
{"$ref" : "#/definitions/juridical-is-not-an-object-and-natural-is-an-object"}
],
"required" : ["natural","juridical"]
}
}
},
"examples" : [
{
"name": "Joao",
"partners": [
{
"juridical": null,
"natural": {
}
},
{
"juridical": {
"tradeName": ""
},
"natural": null
}
]
},
{
"name": "Joao",
"partners": [
{
"juridical": null,
"natural": {
"name" : ""
}
},
{
"juridical": {
"tradeName": ""
},
"natural": null
}
]
},
{
"name": "Joao",
"partners": [
{
"juridical": null,
"natural": {
}
},
{
"juridical": {
"tradeName": ""
},
"natural": null
},
{
"juridical" : [],
"natural" : {}
}
]
}
],
"definitions" : {
"natural" : {
"is-object" : {
"type" : "object",
"required": ["name"],
"properties": {
"name": {
"type": "string"
}
}
},
"is-not-an-object" : {
"not" : { "type" : "object" }
},
},
"juridical" : {
"is-object" : {
"type" : "object",
"required": ["tradeName"],
"properties": {
"name": {
"type": "string"
}
}
},
"is-not-an-object" : {
"not" : { "type" : "object" }
},
},
"juridical-is-object-and-natural-is-null" : {
"properties" : {
"natural" : {"$ref" : "#/definitions/natural/is-not-an-object"},
"juridical" : {"$ref" : "#/definitions/juridical/is-object"}
},
},
"juridical-is-not-an-object-and-natural-is-an-object" : {
"properties" : {
"natural" : {"$ref" : "#/definitions/natural/is-object"},
"juridical" : {"$ref" : "#/definitions/juridical/is-not-an-object"}
}
},
}
}
Notes:
"not" : { schema } error message might be confusing for casual users, but it conforms to the spec: https://json-schema.org/draft-07/json-schema-validation.html#rfc.section.6.7.4
Update
As explained in comments, you are after error details. Given the constraints of the selected tool in terms of if-then-else error details for more complex schemas, did you try to reshape schema using different keywords to trigger as less overhead messages as possible?
Alternative schema 2
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"required": [
"name",
"partners"
],
"properties": {
"partners": {
"type": "array",
"items" : {
"properties" : {
"natural" : {
"if" : { "type" : "object" },
"then" : { "required" : ["name"] },
"dependencies" : {
"name" : {
"properties" : {
"name" : {"type" : "string"}
}
}
}
},
"juridical" : {
"if" : { "type" : "object" },
"then" : { "required" : ["tradeName"] },
"dependencies" : {
"tradeName" : {
"propertyNames" : {
"enum" : ["tradeName"]
},
"properties" : {
"tradeName" : {"type" : "string"}
}
}
}
}
},
"anyOf" : [
{"$ref" : "#/definitions/natural-is-null-juridical-is-an-object"},
{"$ref" : "#/definitions/natural-is-an-object-juridical-is-null"}
]
}
}
},
"definitions" : {
"natural-is-null-juridical-is-an-object" : {
"properties" : {
"natural" : { "type": "null"},
"juridical" : { "type" : "object"}
}
},
"natural-is-an-object-juridical-is-null" : {
"properties" : {
"natural" : { "type": "object"},
"juridical" : { "type" : "null"}
}
},
},
"examples" : [
{
"name": "Joao",
"partners": [
{
"juridical": null,
"natural": {
}
},
{
"juridical": {
"tradeName": "",
},
"natural": null
},
{
"juridical" : {},
"natural" : {}
},
{
"juridical" : null,
"natural" : null
}
]
},
]
}
I generated the following JSON:
{
"someString" : "example",
"obj1" : {
"opt1" : 1,
"opt2" : 1,
"opt3" : "aaa"
},
"obj2" : {
"opt1" : 55,
"opt2" : 55,
"opt3" : "bbb"
}
}
and there will be more of objects(obj1, obj2, obj3, obj4, ...) with same data type (opt1, opt2, opt3)
now I want to create schema for this but i don't know how to combine all this objects in schema.
EDIT:
I created schema:
root: {
"type" : "object",
"oneOf" : [
{
"properties" : {
"someString" : { "type" : "string" }
},
"patternProperties" : { "^.*$" : { "$ref" : "./schemas/myPatternProperties.json#" } },
"additionalProperties" : false }
}
]
}
and myPatternProperties.json looks:
{
"type" : "object",
"properties" : {
"opt1" : { "type" : "number" },
"opt2" : { "type" : "number" },
"opt3" : { "type" : "string" },
}
"required" : [ "opt1", "opt2", "opt3" ]
}
Is there anything wrong because, my generated JSON is still not recognized as this schema type.
As I understand your problem is to describe object with a lot of properties with the same type and some naming rules. To solve that you must specify patternProperties section
{
"patternProperties": {
"^(/[^/]+)+$": { "$ref": "http://some.site.somewhere/entry-schema#" }
}
that construction specify pattern to match for properties. Example how to use patternProperties Read more at specification
UPDATE
Actually full scheme must be something like that
{
"$schema": "http://json-schema.org/draft-06/schema#",
"type": "object",
"properties": {
"someString": {
"type": "string"
}
},
"patternProperties": {
"^obj([0-9]+)$": {
"$ref": "#/definitions/objEntity"
}
},
"additionalProperties": false,
"required": [ "someString" ],
"definitions": {
"objEntity": {
"type": "object",
"properties": {
"opt1": { "type": "number" },
"opt2": { "type": "number" },
"opt3": { "type": "string" }
},
"required": ["opt1", "opt2", "opt3"]
}
}
}
Of course you can split that scheme to more than 1 file, and change links to type definitions.
I have an issue in Kibana with the field value explained in the following lines. I'll try to explain the situation.
I'm sending dynamoDB streams to Lambda then to Kenesis Firehouse and finally from Firehose to Elasticsearch. I'm using Kibana to visualize data and here is where I have the issue.
Lets say that I'm sending this JSON to DynamoDB:
{
"id": "identificator",
"timestamp": "2017-05-09T06:38:00.337Z",
"value": 33,
"units": "units",
"description": "This is the description",
"machine": {
"brand": "brand",
"application": "application"
}
}
In Lambda I receive the following:
{
"data": {
"M": {
"machine": {
"M": {
"application": {
"S": "application"
},
"brand": {
"S": "band"
}
}
},
"description": {
"S": "This is the description"
},
"id": {
"S": "identificator"
},
"units": {
"S": "units"
},
"value": {
"N": "33"
},
"_msgid": {
"S": "85209b75.f51ee8"
},
"timestamp": {
"S": "2017-05-09T06:38:00.337Z"
}
}
},
"id": {
"S": "85209b75.f51ee8"
}
}
If I forward this last JSON to Kinesis Firehose, when in Kibana I configure the index pattern, it recognizes the "timestamp" automatically (and that's great). The problem here, is that the field "value" is like a string and it is not recognized.
I tried to modify the JSON and then send it again to Firehose but then Kibana doesn't recognizes the "timestamp":
{
"data": {
"machine": {
"application": "application",
"brand": "brand"
},
"description": "This is the description",
"id": "identificator",
"units": "KWh",
"value": 33,
"_msgid": "85209b75.f51ee8",
"timestamp": "2017-05-09T06:38:00.337Z"
},
"id": "85209b75.f51ee8"
}
I would like to know how could I send this data and Kibana recognizes the "timestamp" and "value" fields.
This is an example of the code that I'm using in lambda:
var AWS = require('aws-sdk');
var unmarshalJson = require('dynamodb-marshaler').unmarshalJson;
var firehose = new AWS.Firehose();
exports.lambda_handler = function(event, context) {
var record = JSON.stringify(event.Records[0].dynamodb.NewImage);
console.log("[INFO]:"+JSON.stringify(event.Records[0].dynamodb.NewImage));
var params = {
DeliveryStreamName: 'DeliveryStreamName',
Record:{
Data: record
}
};
firehose.putRecord(params, function(err, data) {
if (err) console.log(err, err.stack); // an error occurred
else console.log(JSON.stringify(data)); // successful response
context.done();
});
};
I solved it creating the index mapping by myself instead of let Kinesis Firehose create it. And declare the "timestamp" attribute as { "type" : "date" } and the "value" attibute as { "type" : "float" }
For instance for this type of JSON:
{
"data": {
"timestamp": "2017-05-09T11:30:41.484Z",
"tag": "tag",
"value": 33,
"units": "units",
"type": "type",
"machine":{
"name": "name",
"type": "type",
"company": "company"
}
},
"id": "85209b75.f51ee8"
}
I created manually the following elasticsearch index and mapping:
PUT /index
{
"settings" : {
"number_of_shards" : 2
},
"mappings" : {
"type" : {
"properties" : {
"data" : {
"properties" : {
"machine":{
"properties": {
"name": { "type" : "text" },
"type": { "type" : "text" },
"company": { "type" : "text" }
}
},
"timestamp": { "type" : "date" },
"tag" : { "type" : "text" },
"value": { "type" : "float" },
"description": { "type" : "text" },
"units": { "type" : "text" },
"type" : { "type" : "text" },
"_msgid": { "type" : "text" }
}
},
"id": { "type" : "text" }
}
}
}
}
So, to solve it, the better solution I think that in lambda you have to check if the index mapping exist and if not create it by yourself.
In below json example - "Vendor" is mandatory and should have some value (minLength=1), however when Mode= Ground, it always returns as blank, hence a general json schema is failing for below json response.
Json response looks like below:
{
"Res": {
"Mem": [
{
"Mode": "Flight",
"Vendor": "YP",
"MemNum": "222"
},
{
"Mode": "Ground",
"Vendor": "",
"MemNum": "242"
}
]
}
}
In json response, both "Mode" will be returned always.
Is it possible to create customized schema where i can specify:
When "Mode"="Ground", validate with schema1
for all others "Mode" take reference from schema2 for validation.
I have tried below but unable to get desired results:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"required": [
"Res"
],
"properties": {
"Res": {
"type": "object",
"required": [
"Mem"
],
"properties": {
"Mem": {
"type": "array",
"allOf": [
{
"$ref": "#/definitions/Flight"
},
{
"$ref": "#/definitions/Ground"
}
]
}
}
}
},
"definitions": {
"Flight": {
"properties": {
"Mode": {
"enum": [
"Flight"
]
},
"VendorCode": {
"type": "string",
"minLength": 1,
"optional": false
},
"MemNum": {
"type": "string",
"minLength": 1,
"optional": false
}
}
},
"Ground": {
"properties": {
"Mode": {
"enum": [
"Ground"
]
},
"VendorCode": {
"type": "string"
},
"MemNum": {
"type": "string",
"minLength": 1,
"optional": false
}
}
}
}
}
Is there any other concept in json schema which can be useful in achieving desired result?
It is a quite awkward usecase but you can hack it together with "allOf" , "anyOf" and "not" .
Schema:
{
"oneOf" : [
{
"allOf" : [
{
"$ref" : "#/definitions/ModeIsGround"
},
{
// schema1 comes here
}
]
},
{
"allOf" : [
{
"not" : {
"$ref" : "#/definitions/ModeIsGround"
}
},
{
// schema2 comes here
}
]
}
],
"definitions" : {
"ModeIsGround" : {
"properties" : {
"Mode" : {
"enum" : ["Ground"]
}
}
}
}
}
You didn't tell in your question what is schema1 and schema2, and also your pasted schema is malformed, so I refer to these simply as schema1 and schema2, commented.
Explanation: so at the top level you say that the subject should match one of the subschemas. The first subschema says that the subject should match "ModeIsGround" and also schema1 (this is the case that you explanied as "When "Mode"="Ground", validate with schema1") . The second subschema says that the subject shouldn't match "ModeIsGround" but should match schema2 (this is the "for all others "Mode" take reference from schema2 for validation" part).
These 2 subschemas combined with "oneOf" will do what you want.
The "Mode"="Ground" predicate is expressed via the single-element enum type.
I hope it helps.