Mule json schema validation - json

I am trying to use Validate JSON schema component in Mule flow and I am getting com.fasterxml.jackson.core.JsonParseException for the json that is passed. Below is the json schema, json sample and code of Mule flow. Can you please point me where am I doing mistake?
Json schema:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "Product",
"description": "A product from Acme's catalog",
"type": "object",
"properties": {
"id": {
"description": "The unique identifier for a product",
"type": "integer"
},
"name": {
"description": "Name of the product",
"type": "string"
},
"price": {
"type": "number",
"minimum": 0,
"exclusiveMinimum": true
}
},
"required": ["id", "name", "price"]
}
Json passed to POST method:
[
{
"id": 2,
"name": "An ice sculpture",
"price": 12.50,
},
{
"id": 3,
"name": "A blue mouse",
"price": 25.50,
}
]
Error :
Root Exception stack trace:
com.fasterxml.jackson.core.JsonParseException: Unexpected character ('}' (code 125)): was expecting double-quote to start field name
at [Source: java.io.InputStreamReader#6e7f030; line: 6, column: 5]
at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1419)
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:508)
Mule Flow:
<http:listener-config name="HTTP_Listener_Configuration" host="0.0.0.0" port="8081" doc:name="HTTP Listener Configuration"/>
<flow name="jsonschemavalidationFlow">
<http:listener config-ref="HTTP_Listener_Configuration" path="/" allowedMethods="POST" doc:name="HTTP"/>
<json:validate-schema schemaLocation="jsonschema.json" doc:name="Validate JSON Schema"/>
<logger message="#[payload]" level="INFO" doc:name="Logger"/>
</flow>

There are few errors in both the JSON and the Schema as follows:-
There is an extra comma, after "price": 12.50
So the Valid JSON will be :-
[
{
"id": 2,
"name": "An ice sculpture",
"price": 12.50
},
{
"id": 3,
"name": "A blue mouse",
"price": 25.50
}
]
In JSON Schema file jsonschema.json There are two errors:-
You need to put "type": "array" instead of "type": "object"
and next is "exclusiveMinimum": true seems to be invalid with this JSON input...
So, the correct working JSON schema will be :-
{
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "Product",
"description": "A product from Acme's catalog",
"type": "array",
"properties": {
"id": {
"description": "The unique identifier for a product",
"type": "integer"
},
"name": {
"description": "Name of the product",
"type": "string"
},
"price": {
"type": "number",
"minimum": 0
}
},
"required": ["id", "name", "price"]
}
Try it.. it will work fine :)

The json u r passing is not proper. It has an extra comma ',' after price: 12.50. Even after second price element also an extra comma is added.
Just remove it and it works fine.

Related

How to a build a JSON file from a CSV using an existing JSON schema format?

I have a JSON schema and a CSV file. The CSV file has 2,511 rows and one header row (2,512 rows total). Each row has 43 columns. I was able to convert the CSV to a JSON using one of the myriad of online converters, but the result is what I believed is termed a 'flat JSON file'.
Here is the CSV header row:
F1,F2,F3.1.F1,F3.1.F2,F3.1.F3,F3.1.F4,...F3.10.F1,F3.10.F2,F3.10.F3,F3.10.F4,F4
Here is my JSON schema:
{
"$schema": "http://json-schema.org/schema#",
"$id": "./.schema.json",
"title": "",
"description": "",
"type": "object",
"properties": {
"F1": {
"description": "",
"type": "string"
},
"F2": {
"description": "",
"type": "string"
},
"F3": {
"description": "",
"type": "array",
"items": {
"description": "",
"type": "object",
"properties": {
"F3.F1": {
"description": "",
"type": "string"
},
"F3.F2": {
"description": "",
"type": "string"
},
"F3.F3": {
"description": "",
"type": "string"
},
"F3.F4": {
"description": "",
"type": "string"
}
},
"required": [
"F3.F1",
"F3.F2",
"F3.F3",
"F3.F4"
]
},
"numItems": 10,
"unique": false
},
"F4": {
"description": "",
"type": "string"
}
},
"required": [
"F1",
"F2",
"F3",
"F4"
],
"additionalProperties": false
}
From the CSV->JSON conversion, my JSON file looks like:
[
{
"F1": 2429546524130460000,
"F2": 2429519276857919500,
"F3.1.F1": 2428316170619109000,
"F3.1.F2": 0.0690932185744956,
"F3.1.F3": 2.6355498567408557,
"F3.1.F4": 0.4369495787854096,
...
"F3.10.F1": 2429415922764859400,
"F3.10.F2": 0.15328371980044203,
"F3.10.F3": 2.677944208300451,
"F3.10.F4": 0.31036472544281585,
"F4": 0.16889514829995647
},
... //repeated 2,509 times
{
"F1": 1143081876266241000,
"F2": 1143588785487818100,
"F3.1.F1": 1141377392726037800,
"F3.1.F2": 1.332366799133926,
"F3.1.F3": 0.24878185970548322,
"F3.1.F4": 1.560443994684636,
...
"F3.10.F1": "XXX",
"F3.10.F2": "XXX",
"F3.10.F3": "XXX",
"F3.10.F4": "XXX",
"F4": 2.2916768389567497
}
]
Clearly, making the necessary changes 2,511 times is impractical, so I am hoping there is a method to make the changes automatically. I can code, but I could not find any specific solutions anywhere to go from a CSV to a JSON with the JSON output matching a specific JSON schema. Preferably, I would like a solution that is not restricted to just converting this one set of data to this one specific format, i.e., a general solution that could be used with a different CSV and different JSON schema.

How to validate a field against a parent object in a recursive JSON schema

I have a simple JSON schema that's used to define a tree of objects that are labeled with an Object Identifier. These objects are recursive and look identical, so the JSON schema just uses a "$ref": "#".
What I would like to do is validate that the actual numeric OID structure is constructed correctly. Right now I'm validating that it starts with the right prefix (1.2.3), but if the subsequent structure is screwed up (e.g. the parent is 1.2.3.6 but someone adds children of 1.2.3.6.1 and 1.2.3.2.2) it won't be caught.
I'm not sure how I can reference the oid field in the parent object to check against the child. Can someone point me in the right direction?
My schema:
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://my-site/my-schema.json",
"title": "My JSON Schema",
"description": "A fancy JSON schema for a tree of objects",
"type": "object",
"properties": {
"oid": {
"description": "The OID",
"type": "string",
"pattern": "^1.2.3"
},
"description": {
"description": "A description",
"type": "string"
},
"children": {
"description": "Children",
"type": [
"array",
"null"
],
"items": {
"$ref": "#"
}
}
},
"required": [
"oid",
"description",
"children"
]
}
Example data:
{
"oid": "1.2.3",
"description": "Parent",
"children": [
{
"oid": "1.2.3.1",
"description": "Group 1",
"children": [
{
"oid": "1.2.3.1.1",
"description": "Group 1.1",
"children": [
{
"oid": "1.2.3.1.1.1",
"description": "Item 1.1.1",
"children": null
}
]
},
{
"oid": "1.2.3.1.2",
"description": "Group 1.2",
"children": [
{
"oid": "1.2.3.1.2.1",
"description": "Item 1.2.1",
"children": null
},
{
"oid": "1.2.3.1.2.2",
"description": "Item 1.2.2",
"children": null
}
]
}
]
}
]
}

How to check for matching properties using a JSON schema?

I'm making a Discord bot in Python that reacts to certain keywords in messages and my script uses a JSON file that might for example look like this:
{
"characters": {
"john": {
"name": "Johnny",
"hex_colour": "0xC61B1B"
},
"marc": {
"name": "Marcus",
"hex_colour": "0x8AC0FF"
}
},
"reactions": [
{
"keywords": ["Hi", "Hello"],
"quotes": {
"john": ["Hello my name is Johnny"]
"marc": [
"Hi there. I'm Marcus.",
"Not now, I'm looking for John, have you seen him?"
]
}
},
{
"keywords": ["Bye"],
"quotes": {
"john": ["See you later!"]
}
}
]
}
As you can see the structure is quite complex, so I decide to make a schema for it so vscode could point out any issues. It currently looks like this:
{
"title": "Quotes data",
"description": "Quotes bot data file",
"type": "object",
"properties": {
"characters": {
"title": "Character collection",
"description": "A collection of the available characters.",
"type": "object",
"additionalProperties":{
"title": "Character tag",
"description": "A tag that represents the character as a short string",
"type": "object",
"properties": {
"name": {
"title": "Character name",
"description": "The name of the character.",
"type": "string"
},
"hex_colour": {
"title": "Character colour",
"description": "The hex code of the colour in 0x000000 format.",
"type": "string"
},
"image_url": {
"title": "Character image",
"description": "An url to an image of the character.",
"type": "string"
}
},
"required": ["name"],
"additionalProperties": false
}
},
"reactions": {
"title": "Reaction collection",
"description": "A list of the reactions",
"type": "array",
"items": {
"type": "object",
"properties": {
"keywords": {
"title": "Keyword list",
"description": "A list of keywords for the bot to react to.",
"type": "array",
"items": {
"title": "Keyword",
"type": "string",
"minItems": 1,
"uniqueItems": true
}
},
"quotes": {
"title": "Quotes collection",
"description": "A collection of characters with quotes.",
"type": "object",
"additionalProperties":{
"title":"Character tag",
"description": "A tag that matches a character above.",
"type": "array",
"items": {
"title": "Quote",
"description": "A quote to react with.",
"type": "string",
"minItems": 1,
"uniqueItems": true
}
}
}
},
"required": ["keywords", "quotes"],
"additionalProperties": false
}
}
},
"required": ["characters", "reactions"],
"additionalProperties": true,
"allowTrailingCommas": true
}
So for the properties of the "characters" object, any property name is allowed, therefore I use "additionalProperties". Later on, for the "quotes" object, I did the same. But in this case, not any property should be allowed. Only those that match one of the properties of the "characters" object is allowed. Is there any way to make the schema check for matching properties?
This is a very specific semantic check and not possible using the standard specification of JSON Schema. Validation can't access an arbitrary set of property names and also can't look up the validation path. You would need to add this as application code.

Avro Schema format Exception - “SecurityClassification” is not a defined name

I'm trying to use this avro schema
{
"type": "record",
"name": "ComplianceEntity",
"namespace": "com.linkedin.events.metadata",
"fields": [
{
"name": "fieldPath",
"type": "string"
},
{
"name": "complianceDataType",
"type": {
"type": "enum",
"name": "ComplianceDataType",
"symbols": [
"NONE",
"MEMBER_ID"
],
"symbolDocs": {
"NONE": "None of the following types apply",
"MEMBER_ID": "ID for LinkedIn members"
}
}
},
{
"name": "complianceDataTypeUrn",
"type": [
"null",
"string"
],
"default": null
},
{
"name": "fieldFormat",
"type": [
"null",
{
"type": "enum",
"name": "FieldFormat",
"symbols": [
"NUMERIC"
],
"symbolDocs": {
"NUMERIC": "Numerical format, 12345"
},
"doc": "The field format"
}
]
},
{
"name": "securityClassification",
"type": "SecurityClassification"
},
{
"name": "valuePattern",
"default": null,
"type": [
"null",
"string"
]
}
]
}
To generate and avro file using the avro-tools:
java -jar ./avro-tools-1.8.2.jar compile schema ComplianceEntity.avsc .
But I am getting the following error message:
Exception in thread "main" org.apache.avro.SchemaParseException: "SecurityClassification" is not a defined name. The type of the "securityClassification" field must be a defined name or a {"type": ...} expression.
Could anyone tell, why SecurityClassification is not identified as a defined name?
You are using it as type of your field, however you are not defining it properly like for complianceDataType, that's the reason why you are getting the avro exception
{
"name": "securityClassification",
"type": "SecurityClassification"
}
Make sure that if you have more than 1 Schema, you pass all of them, especially dependency schemas. It is supported from AVRO 1.5.3 https://issues.apache.org/jira/browse/AVRO-877.
java -jar ./avro-tools-1.8.2.jar compile schema SecurityClassification.avsc ComplianceEntity.avsc .

Format the pretty JSON object

I'm using Node.js to pretty-print a JSON object.
This line
obj = JSON.stringify(obj, null, 1);
results in:
{
"name": "Member",
"type": "object",
"properties": {
"Id": {
"type": "GUID",
"description": "Unique identifier"
},
"Name": {
"type": "string",
"description": "Members name"
}
}
}
But I want it to look this way:
{
"name": "Member",
"type": "object",
"properties": {
"Id": {"type": "GUID", "description": "Unique identifier"},
"Name": {"type": "string", "description": "Members name"}
}
}
How can I do this?
Use JSON.stringify, then use some regular expressions on the resulting string.
First strip the top level braces, then replace every newline within a {...} group with blank.