JSON: Is there an equivalent of Schematron for JSON and JSON Schema? (That is, a JSON technology to express co-constraints) - json

Here is a JSON instance showing the start-time and end-time for a meeting:
{
"start time": "2015-02-19T08:00:00Z",
"end time": "2015-02-19T09:00:00Z"
}
I can specify the structure of that instance using JSON Schema: the instance must contain an object with a "start time" property and an "end time" property and each property must be a date-time formatted string. See below for the JSON schema. But what I cannot specify is this: the meeting must start before it ends. That is, the value of "start time" must be less than the value of "end time". Some people call this data dependency a co-constraint. In the XML world there is a wonderful, simple technology for expressing co-constraints: Schematron. I am wondering if there is an equivalent technology in the JSON world? What would you use to declaratively describe the relationship between the value of "start time" and "end time"? (Note: writing code in some programming language is not what I mean by "declaratively describe the relationships". I am seeking a declarative means to describe the data dependencies that are present in JSON documents, not procedural code.)
{
"$schema": "http://json-schema.org/draft-04/schema#",
"definitions": {
"meeting": {
"type": "object",
"properties": {
"start time": { "type": "string", "format": "date-time"},
"end time": { "type": "string", "format": "date-time"}
},
"required": [ "start time", "end time" ],
"additionalProperties": false
}
},
"$ref": "#/definitions/meeting"
}

Yes.There is a JSON Semantic Validator based on Schematron available at:
https://www.npmjs.com/package/jsontron
It implements 'schema', 'phase', 'rule', 'assert' and reporting features of Schematron.
Here is when the original example of start time and end time was run through the validator:
good_time.json file contents:
{
"starttime": "2015-02-19T08:00:00Z",
"endtime": "2015-02-19T09:00:00Z"
}
bad_time.json file contents:
{
"starttime": "2015-02-19T09:00:00Z",
"endtime": "2015-02-19T08:00:00Z"
}
Schematron Rules file meeting-times-rules.json snippet:
"rule":[
{
"context": "$",
"assert":[
{
"id":"start_stop_meeting_chec",
"test":"jp.query(contextNode, '$..starttime') < jp.query(contextNode, '$..endtime')",
"message": "Meeting cannot end before it starts"
}
]
}
]
When ran with correct example:
$jsontron\bin>node JSONValidator -i ./good_time.json -r ./meeting-times-rules.json
The output was:
Starting Semantic Validation .........
Parsing Pattern: Meetingtimings
1 Pattern(s) Requested. 1 Pattern(s) Processed. 0 Pattern(s) Ignored.
**** THIS INSTANCE IS SEMANTICALLY VALID ****
Completed Semantic Validation .........
When ran with bad data example. The output was:
$jsontron\bin>node JSONValidator -i ./bad_time.json -r ./meeting-times-rules.json
Starting Semantic Validation .........
Parsing Pattern: Meetingtimings
1 Pattern(s) Requested. 1 Pattern(s) Processed. 0 Pattern(s) Ignored.
**** THIS INSTANCE CONTAINS SEMANTIC VALIDATION ISSUES. PLEASE SEE FULL REPORT BY ENABLING DEBUG WITH -d OPTION ****
Completed Semantic Validation .........
The message with debug options was:
...validation failed...
message: 'Meeting cannot end before it starts'

Sadly, the answer is no. JSON Schema allows you to validate the structure, and permitted values, but there are no mechanisms for validating sets of values, a'la Schematron.
The simplest way to solve this is to have another script in the pipeline which runs these kinds of checks.

There is an implementation in Oxygen JSON Editor that allows you to validate JSON documents against Schematron.
https://www.oxygenxml.com/doc/versions/22.0/ug-editor/topics/json-validating-documents-against-schema.html
The Schematron rules are expressed using XPath expressions, and the problems are reported in the JSON documents.
<!-- The 'genre' property should be none but one of the items declared in 'literatureGenres' property -->
<sch:rule context="genre">
<sch:let name="genre" value="text()"/>
<sch:let name="literatureGenres" value="//literatureGenres/text()"/>
<sch:assert test="normalize-space($genre) = $literatureGenres">
Wrong genre: '<sch:value-of select="$genre"/>'. See the 'literatureGenres' property for the permitted ones.
</sch:assert>
</sch:rule>
https://www.slideshare.net/nottavy/schematron-for-nonxml-languages

The json-schema.org website lists quite a few implementations.

Related

How to perform validation of json files with cuelang?

I am evaluating cuelang.org to define data constraints, such as type or value checks on untrusted json payloads submitted to azure devops pipelines, before further processing.
When testing my definitions against invalid data I get no errors returned.
I want to have both type and value checks in place:
rules.cue:
#LoggingLinuxVM: {
resource_id: #NonEmptyString
version: =~"2020-07-21"
type: =~"Linux"
properties: #LoggingPropertiesLinuxVM
}
#LoggingPropertiesLinuxVM: {
CustomLogFiles: [...#NonEmptyString]
}
#NonEmptyString: string & !=""
invalid payload (missing type)
{
"resource_id": "/subscription/.../...",
"version": "2020-07-21",
"properties": {
"custom_log_files": [
"/var/log/dmesg",
"/var/log/messages"
]
}
}
when attempting to validate this with: cue vet payload.json rules.cue I expect to see validation errors, however none are raised and the exit status is 0 . Has anyone used cuelang in a similar fashion and can see where I am going wrong?
Turns out you have to provide the --schema flag to operate in the intended way. For this example, if the json were in payload.json the full command would be:
cue vet --schema '#LoggingLinuxVM' payload.json rules.cue

The name cannot contain any of the following symbols: '[, ], .'.'. for Compose in Azure Logic App

I have below JSON which I need to update inside a logic app
{
"name": "SampleDoc",
"type": "123",
"properties": {
"GP.Test": "M1",
"MG.Test": "C1"
}
}
I have used following setProperty syntax: -
#setProperty(variables('ResponseBody'),'properties', setProperty(variables('ResponseBody')['properties'], 'test','abc'),
setProperty(variables('ResponseBody')['properties'], 'GP.Test','M2'))
My desired JSON output should be
{
"name": "SampleDoc",
"type": "123",
"properties": {
"GP.Test": "M2",
"MG.Test": "C1"
}
}
But when I am running this, I am getting this error: -
InvalidTemplate. Unable to process template language expressions in action 'Compose' inputs at line '1' and column '2617': 'The provided property name 'GP.Test' has these invalid characters '.'. The name cannot contain any of the following symbols: '[, ], .'.'.
Could anyone suggest if we can handle '.' inside compose or any other way for achieving this?
Yes that is correct behavior that is occurring in the logic apps. The reason is that you have used the set property function to set the value of the GP.Test property. When working with the expressions in logic apps, the '.' operator is reserved operator and will be used to access sub properties etc of the expressions, functions etc. Hence you get the error. The solution to this is actually simple, you use the compose action directly without using the set property. Sample screenshot below.
Or if you want complex transformations, then using the liquid transformations through the integration account is the way to go

$filter on schema extensions microsoft graph doesn't support 'contains'

I added a schema extension to users in my org, to keep track of training a user has taken. Since lists are not supported I am trying to store this as a comma separated string, as follows:
{
"id": "voctestextension",
"description": "voc test extension",
"targetTypes": ["User"],
"properties": [
{
"name": "trainings",
"type": "String"
}
]
}
Now, while trying to fetch the users who have taken training 'X' I am making the below call:
https://graph.microsoft.com/v1.0/users?$filter=contains(extrw7rtbc9_voctestextension/trainings, 'Azure'), $select=extrw7rtbc9_voctestextension,displayName
This doesn't give the correct response, but throws this error:
{
"error": {
"code": "Request_UnsupportedQuery",
"message": "Unsupported Query.",
"innerError": {
"request-id": "dc3fda19-6464-43d9-95ce-54a0567bf5a9",
"date": "2018-03-15T09:14:30"
}
}
}
From different forum answers, I understand that contains is not supported. Can you suggest a better way to track this info in the user's profile?
Contains is not supported. You need to use startswith or add multiple properties like training1, training2, training3.. and then use filter with ORs and EQs.
https://graph.microsoft.com/v1.0/users?$filter(extrw7rtbc9_voctestextension/trainign1 eq 'Azure' or extrw7rtbc9_voctestextension/trainign2 eq 'Azure')
Kepaas solution is working (almost) perfectly, but the "=" is missing after "$filter":
https://graph.microsoft.com/v1.0/users?$filter=(extrw7rtbc9_voctestextension/trainign1 eq 'Azure' or extrw7rtbc9_voctestextension/trainign2 eq 'Azure')

i18next failing to load translation file: "there is a typo"

I've created translation file, validated it at jsonlint, ensured the translation file was located at /locales/translation-en.json.
I consistently get the error,
There is a typo in: locales/translation-en.json
I'm stumped ... here's the translation json I have.
{
"tab":{
"legionella":"LEGIONELLA",
"logbook":"LOGBOOK"
},
"representative":{
"tag":"Representative: __rep__ — Phone: __phone__ — ",
"email":"Click here to email your rep"
},
"portlet":{
"contacts":{
"title":"Contacts",
"type":"Contact<br>Type",
"name":"Contact<br>Name",
"phone":"Phone<br>Number",
"type_context_1":"Owner",
"type_context_2":"Maintenance",
"type_context_3":"Other"
},
"samples":{
"title":"Legionella Samples",
"sampleDate":"Sample<br>Date",
"transmitForm":"Transmittal<br>Form",
"certOfAnalysis":"Certificate<br>of Analysis",
"concentration":"Concentration<br>(UFC/L)",
"correctAction":"Corrective<br>Action",
"range_context_1":"Interference",
"range_context_2":"Less than 10,000 UFC/L",
"range_context_3":"Between 10,000 to 1,000,000 UFC/L",
"range_context_4":"Greater than 1,000,000 UFC/L"
},
"serviceReports":{
"title":"Service Reports",
"date":"Report<br>Date"
},
"maintenance":{
"title":"Maintenance Programs",
"popup":"Create New Maintenance Program",
"type":"Program<br>Type",
"date":"Effective<br>Date",
"document":"Program<br>Document",
"type_context_1":"Water Treatment",
"type_context_2":"Mechanical",
"type_context_3":"Schematic",
"type_context_4":"O&M Manual",
"popup_type":"Type",
"popup_date":"Effective Date",
"popup_document":"Document",
"popup_save":"Save Maintenance Program"
},
"history":{
"title":"System History",
"popup":"Create New System History Entry",
"date":"Event<br>Date",
"type":"Event<br>Type",
"details":"Event<br>Details",
"type_context_1":"Breakage",
"type_context_2":"Repair",
"type_context_3":"Decontamination",
"type_context_4":"Replacement"
},
"reminders":{
"title":"Reminders",
"date":"Date",
"description":"Description"
},
"emails":{
"title":"Emails",
"date":"Date",
"subject":"Subject",
"recipient":"Recipient"
}
},
"common":{
"view":"View",
"registryList":"Registry: ",
"signout":"Sign Out"
}
}
So, I found the issue. I didn't give ALL the details required in the original question. What I failed to let everyone know was that the json file resided in Netsuite. Netsuite doesn't like serving up .json files. When I converted it to .json.txt, all was well in the world. Thanks!
Just in the event someone missed the obvious..
JSON, unlike Javascipt, requires the keys be quoted.
Valid JSON:
{ "foo": "bar" }
Invalid JSON:
{ foo: "bar" }
Since this question pops high in Google when search for "i18next there is a typo in", here's what you can do.
Use a JSON validation tool. If it's not valid, correct your mistakes and try again.

json schema date-time does not check correctly

I have a JSON and a JSON-schema
JSON:
{
"aaa": "4000-02-01 00:00:00"
}
JSON-schema:
{
"$schema": "http://json-schema.org/draft-04/schema",
"type": "object",
"properties": {
"aaa": {
"type": "string",
"format": "date-time"
}
}, "required": ["aaa"]
}
The JSON gets validated by the JSON-schema. However if I change the field aaa to "bla" the schema does not notice that it is not a date-time any longer.
Did I miss anything in the schema?
For Python's jsonschema library, specify the format checker when calling validate:
jsonschema.validate(data, schema, format_checker=jsonschema.FormatChecker())
To validate a date-time format, the strict-rfc3339 package should be installed.
See Validating Formats.
Validation with "format" is optional. This is partly because schema authors are allowed to completely make up new formats, so expecting all formats to be validated is not reasonable.
Your library should (if it is decent) have a way to register custom validators for particular formats. For example, the tv4 validation library (in JavaScript) has the tv4.addFormat() method:
tv4.addFormat('date-time', function (data) {
return isValidDate(data);
});
Once you've done this, then "format": "date-time" in the schema should validate dates correctly.
It is highly likely that the implementation of JSON schema validation that you're using is requiring the T separator between the date and time components. This is a staple of the RFC3339 spec and ISO8601 which it is based upon. While both have provisions for omitting the T, they both make it something that can be done by agreement, rather then a mandatory thing to support. (Go figure.)
Also, RFC3339 does require that you include either a time zone offset or a Z to indicate UTC. This locks it down to a particular moment in time, rather than a human representation of one in some unknown time zone. Since you have required neither, that's likely while it has failed validation.
From the JSON Schema spec:
7.3.1.2. Validation
A string instance is valid against this attribute if it is a valid date representation as defined by RFC 3339, section 5.6 [RFC3339].
I found a workaround by using this library. It checks the content of the field in javascript code:
function isValidDate(datestring) {
var format = d3.time.format("%Y-%m-%d %H:%M:%S");
var date = format.parse(datestring);
if (date) {
return true;
}
return false;
}
In this example, if you change "aaa" to "bla" you are adding a new property that is not defined and hence the schema will only validate "aaa". If you do not want to add properties I believe you should add "additionalProperties": false. Example below:
Here the schema states that "aaa" is the only property and its values should be in date-time.
{
"$schema": "http://json-schema.org/draft-04/schema",
"type": "object",
"additionalProperties": false,
"properties": {
"aaa": {
"type": "string",
"format": "date-time"
}
}, "required": ["aaa"]
}
You can change the source code for the python jsonschema module.
Find the datetime-related code, at jsonschema/_format.py func is_date_time(instance). Like this, about line 204 - 225, for version 2.6.0:
try:
import strict_rfc3339
except ImportError:
try:
import isodate
except ImportError:
pass
else:
#_checks_drafts("date-time", raises=(ValueError, isodate.ISO8601Error))
def is_datetime(instance):
if not isinstance(instance, str_types):
return True
return isodate.parse_datetime(instance)
else:
#_checks_drafts("date-time")
def is_datetime(instance):
if not isinstance(instance, str_types):
return True
return strict_rfc3339.validate_rfc3339(instance)
Comment out the above and paste this, or replace the _check_drafts function above with this:
#_checks_drafts("date-time")
def is_datetime(instance):
if not isinstance(instance, str_types):
return True
try:
datetime.datetime.strptime(instance, "%Y-%m-%d %H:%M:%S")
except ValueError:
return False
return True