Proper validation of a referenced file when editing OpenAPI json in IntelliJ - json

I'm editing an OpenAPI JSON spec in IntelliJ. The automatic validation and code completion work very nicely.
The OpenAPI version used is 3.0.3, which IntelliJ detects correctly. It seems that it uses "openapi30.json" internally for validation, and all is good.
However, the file is getting very large and it's time to move some commonly-used models out of it using $ref.
This is where things break. The main spec looks like this (snippet):
{
"openapi": "3.0.3",
"info": {
"title": "Cars REST API",
"description": "Calls, Responses and DTOs for REST",
"version": "1.0.0"
},
"components": {
"schemas": {
"car": {
"$ref": "car.json"
},
"car-group": {
"$ref": "car-group.json"
}
And when editing it, IntelliJ recognizes it as "openapi30".
However, the referenced documents are not recognized. For example, the car.json file looks like this:
{
"car": {
"required": [
"id",
"name"
],
"properties": {
"id": {
"type": "integer",
"format": "int64"
},
"name": {
"type": "string"
},
"tag": {
"type": "string"
}
}
}
}
And it's recognized simply as a JSON document, not an OpenAPI one, so there is no proper validation and no code completion, etc.
How does one tell IntelliJ that the file is part of an OpenAPI specification, to be validated as such? One should think this could be inferred from begin $ref'ed from the main spec, but this doesn't work.
Trying to add a $schema value in the referenced file had no effect (and probably isn't in line with the OpenAPI spec anyway).
Manually selecting the OpenAPI 3.0 file type for car.json is not helpful, because then validation (rightly) fails - as it doesn't have the top-level structure required (info, openapi, paths).
Perhaps some specific JSON schema mapping needs to be added in IntelliJ preferences? If that's the case, it would be actually a sub-schema or some tag in the main OpenAPI spec, how can that be done?
IntelliJ version is: IntelliJ IDEA 2021.3.2 (Ultimate Edition)
Any help would be greatly appreciated.

Ron!
Such functionality is not yet supported. Please vote for https://youtrack.jetbrains.com/issue/IDEA-284305

Related

Making Valid JSON-LD Schema & Object Instance

I'm trying to create a graph database for describing power systems and this is my first time using JSON-LD.
Steps so far:
Created a schema according to the following guide
Saved the schema to this url
Attempted to use JSON-LD playground to create and validate and instance of a power-station object described in the schema
The issue I'm facing is that my schema seems to be accepted (changing it creates an error) but no output is shown and no explanation is provided. Any help to work out what I'm missing would be much appreciated.
Schema
{
"$id": "https://osuked.github.io/Power-Station-Dictionary/power-station",
"type": "object",
"properties": {
"capacity": { "type": "float" },
"fuel-type": { "type": "string" }
},
"required": ["capacity", "fuel-type"]
}
Example object instance
{
"#context": "https://osuked.github.io/Power-Station-Dictionary/power-station.json",
"#id": "http://osuked.github.io/Drax",
"capacity": 12,
"fuel-type": "wind"
}
Check the definition of the context (https://json-ld.org/spec/latest/json-ld/#the-context). It must specify how your properties (capacity and fuel-type) map to RDF properties. You can either provide the definitions inline or point to a URL that contains this mapping (e.g. https://schema.org/docs/jsonldcontext.jsonld).
In your example, you provide the URL https://osuked.github.io/Power-Station-Dictionary/power-station.json. It contains a JSON document but does not contain the JSON-LD context, because JSON Schema != JSON LD (you might find this article helpful to understand the difference: https://dashjoin.medium.com/json-schema-schema-org-json-ld-whats-the-difference-e30d7315686a).
Therefore, the JSON LD playground does not show an error (no property definitions found) but also shows no parsed triples.
To fix this, you can try using "#context": "http://schema.org/", you can define a namespace prefix p,
{
"#context": {
"p": "https://osuked.github.io/Power-Station-Dictionary/power-station/"
},
"#id": "http://osuked.github.io/Drax",
"p:capacity": 12,
"p:fuel-type": "wind"
}
or define the properties individually:
{
"#context": {
"capacity": "https://osuked.github.io/Power-Station-Dictionary/power-station/capacity",
"fuel-type": "https://osuked.github.io/Power-Station-Dictionary/power-station/fuel-type"
},
"#id": "http://osuked.github.io/Drax",
"capacity": 12,
"fuel-type": "wind"
}

When using the combining schema as the parent schema: Could not resolve schema reference '#foo'. Path 'not.allOf[0]'

Consider an example in
https://github.com/json-schema-org/JSON-Schema-Test-Suite/blob/master/tests/draft6/ref.json#L414
the original schema is:
{
"allOf": [{
"$ref": "http://localhost:1234/bar#foo"
}],
"definitions": {
"A": {
"$id": "http://localhost:1234/bar#foo",
"type": "integer"
}
}
}
which is valid.
For this schema S, if I want to create a new schema by adding a combining schema outside this schema, like: {"not":S}, which is:
{
"not": {
"allOf": [{
"$ref": "http://localhost:1234/bar#foo"
}],
"definitions": {
"A": {
"$id": "http://localhost:1234/bar#foo",
"type": "integer"
}
}
}
}
with the Location-independent identifier with absolute URI, also use $id. But it is invalid.
The error message(using https://www.jsonschemavalidator.net/):
Error parsing schema
Message:
Error when resolving schema reference 'http://localhost:1234/bar#foo'. Path 'not.allOf[0]', line 3, position 20.
A similar example is:
{
"not": {
"allOf": [{
"$ref": "#foo"
}],
"definitions": {
"A": {
"$id": "#foo",
"type": "integer"
}
}
}
}
with the Location-independent identifier, is also invalid.
I cannot figure out why it is invalid after adding a parent schema if there are $ref and $id at the same time...
Here {"not":S} is just one possibility, also can consider {"anyOf":[{S}]}.
I know that $id declares a base URI against which $ref URI-references are resolved.
But what is the problem with the above schemas?
And how should I correct them?
I will so appreciate it if someone helps me out...
You are correct here, and your schema is correct. I would call this a bug in the implementation.
Your schema is valid for JSON Schema draft-07, but not JSON Schema draft 2019-09 or above. For 2019-09 and above, your $id value can't contain a non-empty fragment.
You can see the correct and valid behaviour on another web based validator, a web based version of the HyperJump validator: https://json-schema.hyperjump.io
This web based implementation defaults to latest (2020-12), but you can specify a draft version using $schema (for example "$schema": "http://json-schema.org/draft-07/schema#").
To see this working on HyperJump, either modify your schema to remove fragments in the identifier URIs, or specify that you're using draft-07.

MalformedURLException when using "$ref" in json schema

I have a json schema which refers to another json schema present in another folder using "$ref" (relative path) and i get a "MalformedURLException".
{
"$schema": "http://json-schema.org/draft-07/schema#",
"$ref": "#/definitions/Base",
"definitions": {
"Base": {
"type": "object",
"additionalProperties": false,
"properties": {
"event": {
"$ref": "com/artifacts/click/ClickSchema.json"
},
"arrival_timestamp": {
"type": "integer",
"minimum": 0.0
}
},
"title": "Base"
}
}
}
And the click schema in another folder is as follows:
{
"$schema": "http://json-schema.org/draft-07/schema#",
"$id": "com/artifacts/click/ClickSchema.json",
"Event": {
"type": "object",
"additionalProperties": false,
"properties": {
"sourceName": {
"type": "string"
}
}
}
}
Can someone please help. I am using this schema validator.
A JSON Schema knows nothing about where it sits in a file, and nothing about the other fils in a folder, by default.
It looks like the library you're using recognises this, and suggests you use a special reference protocol (classpath) to target other files in a folder with ease:
https://github.com/everit-org/json-schema#loading-from-the-classpath
As your schemas grow you will want to split that up into multiple
source files and wire them with "$ref" references. If you want to
store the schemas on the classpath (instead of eg. serving them
through HTTP) then the recommended way is to use the classpath:
protocol to make the schemas reference each other.
This isn't something defined by JSON Schema.
The more common approach is to load in all the schemas you intend to use, and allow for local resolution where you have the files already. The library you're using also supports this: https://github.com/everit-org/json-schema#registering-schemas-by-uri
Sometimes it is useful to work with preloaded schemas, to which we
assign an arbitary URI (maybe an uuid) instead of loading the schema
through a URL. This can be done by assigning the schemas to a URI with
the #registerSchemaByURI() method of the schema loader. Example:
SchemaLoader schemaLoader = SchemaLoader.builder()
.registerSchemaByURI(new URI("urn:uuid:a773c7a2-1a13-4f6a-a70d-694befe0ce63"), aJSONObject)
.registerSchemaByURI(new URI("http://example.org"), otherJSONObject)
.schemaJson(jsonSchema)
.resolutionScope("classpath://my/schemas/directory/")
.build();
There are additional considerations if you intend for your schemas to be used by others. If that's the case, do comment, and I'll expand.

swagger-codegen: errors with \d+ patterns and type: "array"

I'm a newbie to Swagger. I've used the swagger servlet to generate my swagger.json file from our REST API Java classes. The swagger.json file shows swagger 2.0 (I assume this is the 2.0 schema version). There was nothing fancy in the source files, just #Api and a few #ApiOperation annotations.
Then I tried using swagger-codegen-cli (both version 2.1.4 and 2.1.6-SNAPSHOT, the latest) to generate HTML output from the JSON file. I got the following results on both:
reading from dsm_swagger.json
[main] ERROR io.swagger.codegen.DefaultCodegen - unexpected missing property for name suppressed
[main] WARN io.swagger.codegen.DefaultCodegen - skipping invalid property {
"type" : "array"
}
writing file /home/combs/dsm_swagger/./index.html
So I get an output file, but any types that are flagged as lists of objects are not handled correctly. These do appear to be valid 2.0 constructs.
I'm also getting Jackson errors about invalid escape characters because it sees
"pattern": "\d+"
in the file. I can work around the \d by using [0-9], but assume it should be handled as is.
Has anybody seen these particular issues and know if they're either fixed or there is a workaround in swagger-codegen or the source file? Is swagger-codegen actually handling v2.0 specs correctly? Any pointers to up to date info or code would be appreciated!
EDIT:
As mentioned in a comment, by using "#JsonIgnore" and "#JsonProperty" in appropriate places and upgrading to V1.5.6 of swagger-core, I got around the issues with invalid property and type "array" messages. Here's an example of the issue with \d:
"/v1/admins/{adminId}": {
"put": {
"tags": [
"admins"
],
"summary": "Update information about a particular admin, given its ID. The update information is passed in the POST body.",
"description": "Longer notes about what this does",
"operationId": "updateUser",
"consumes": [
"application/json"
],
"produces": [
"application/json"
],
"parameters": [
{
"name": "adminId",
"in": "path",
"required": true,
"type": "integer",
"pattern": "\d+",
"format": "int64"
},
{
"in": "body",
"name": "body",
"required": false,
"schema": {
"$ref": "#/definitions/UserUpdateInfo"
}
}
],
"responses": {
"200": {
"description": "successful operation",
"schema": {
"$ref": "#/definitions/UserInfo"
}
}
}
}
},
This is the exact output of swagger-core, and yet swagger-codegen fails with the following:
combs#dcombs-lap:~/dsm_swagger$ gen_file
reading from dsm_swagger.json
reading from dsm_swagger.json
com.fasterxml.jackson.core.JsonParseException: Unrecognized character escape 'd' (code 100)
at [Source: dsm_swagger.json; line: 411, column: 27]
at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1419)
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:508)
at com.fasterxml.jackson.core.base.ParserMinimalBase._handleUnrecognizedCharacterEscape(ParserMinimalBase.java:485)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._decodeEscaped(UTF8StreamJsonParser.java:2924)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishString2(UTF8StreamJsonParser.java:2209)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishString(UTF8StreamJsonParser.java:2165)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.getText(UTF8StreamJsonParser.java:279)
at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:224)
at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer.deserializeArray(JsonNodeDeserializer.java:262)
at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:221)
at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:218)
at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:218)
at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:218)
at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:62)
at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:14)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3066)
at com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:1861)
at io.swagger.parser.SwaggerCompatConverter.readResourceListing(SwaggerCompatConverter.java:139)
at io.swagger.parser.SwaggerCompatConverter.read(SwaggerCompatConverter.java:74)
at io.swagger.parser.SwaggerParser.read(SwaggerParser.java:73)
at io.swagger.codegen.config.CodegenConfigurator.toClientOptInput(CodegenConfigurator.java:317)
at io.swagger.codegen.cmd.Generate.run(Generate.java:186)
at io.swagger.codegen.SwaggerCodegen.main(SwaggerCodegen.java:35)
Exception in thread "main" java.lang.RuntimeException: missing swagger input or config!
at io.swagger.codegen.DefaultGenerator.generate(DefaultGenerator.java:89)
at io.swagger.codegen.cmd.Generate.run(Generate.java:188)
at io.swagger.codegen.SwaggerCodegen.main(SwaggerCodegen.java:35)
combs#dcombs-lap:~/dsm_swagger$

Can you put comments in Avro JSON schema files?

I'm writing my first Avro schema, which uses JSON as the schema language. I know you cannot put comments into plain JSON, but I'm wondering if the Avro tool allows comments. E.g. Perhaps it strips them (like a preprocessor) before parsing the JSON.
Edit: I'm using the C++ Avro toolchain
Yes, but it is limited. In the schema, Avro data types 'record', 'enum', and 'fixed' allow for a 'doc' field that contains an arbitrary documentation string. For example:
{"type": "record", "name": "test.Weather",
"doc": "A weather reading.",
"fields": [
{"name": "station", "type": "string", "order": "ignore"},
{"name": "time", "type": "long"},
{"name": "temp", "type": "int"}
]
}
From the official Avro spec:
doc: a JSON string providing documentation to the user of this schema (optional).
https://avro.apache.org/docs/current/spec.html#schema_record
An example:
https://github.com/apache/avro/blob/33d495840c896b693b7f37b5ec786ac1acacd3b4/share/test/schemas/weather.avsc#L2
Yes, you can use C comments in an Avro JSON schema : /* something */ or // something Avro tools ignores these expressions during the parsing.
EDIT: It only works with the Java API.
According to the current (1.9.2) Avro specification it's allowed to put in extra attributes, that are not defined, as metadata:
This allows you add comments like this:
{
"type": "record",
"name": "test",
"comment": "This is a comment",
"//": "This is also a comment",
"TODO": "As per this comment we should remember to fix this schema" ,
"fields" : [
{
"name": "a", "type": "long"
},
{
"name": "b", "type": "string"
}
]
}
No, it can't in the C++ nor the C# version (as of 1.7.5). If you look at the code they just shove the JSON into the JSON parser without any comment preprocessing - bizarre programming style. Documentation and language support appears to be pretty sloppy...