I want to have a single schema file with many definitions.
I then want to validate messages against different definitions within that schema.
Is there a way of doing this with a JSON Schema?
I'm trying two NodeJS validators to see which works best:
https://github.com/geraintluff/tv4
and https://github.com/tdegrunt/jsonschema
Apologies if this is not logically possible - I'm new to JSON Schema.
Cross-posted to https://github.com/geraintluff/tv4/issues/170 and https://github.com/tdegrunt/jsonschema/issues/94
I found what I needed in the API section.
tv4.addSchema() and tv4.getSchema(...#subschema_id')
Related
I want to convert incoming JSON data from Kafka into a dataframe.
I am using structured streaming with Scala 2.12
Most people add a hard coded schema, but if the json can have additional fields, it requires changing the code base every-time, which is tedious.
One approach is to write it into a file and infer it with but I rather avoid doing that.
Is there any other way to approach this problem?
Edit: Found a way to turn a json string into a dataframe but cant extract it from the stream source, it is possible to extract it?
One way is to store the schema itself in the message headers (not in the key or value).
Though, this increases message size, it will be easy to parse the JSON value without the need for any external resource like a file or a schema registry.
New messages can have new schemas while at the same time old messages can still be processed using their old schema itself, because the schema is within the message itself.
Alternatively, you can version the schemas and include an id for every schema in the message headers (or) a magic byte in the key or value and infer the schema from there.
This approach is followed by Confluent Schema registry. It allows you to basically go through different versions of same schema and see how your schema has evolved over time.
Read the data as string and then convert it to map[string,String], this way you can process the any json without even knowing its schema
based on JavaTechnical answer , the best approach would be to use a schema registry and
avro data instead of json, there is no going around hardcoding a schema (for now).
include your schema name and id as a header and use them to read the schema from the schema registry.
use the from_avro fucntion to turn that data into a df!
Say I want to validate a YAML file against a JSON schema in Intellij IDEA. The file's structure would be like:
foo:
command: touch /tmp/a.txt # I know I don't need this but it's an example
bar:
command: echo "Hello World!" > /tmp/a.txt
baz:
command: cat /tmp/a.txt
dependencies:
- foo
- bar
So the property names can be any string, but the dependencies should only be keys/property names of the root object. Ideally I would specify an enum, but this question suggests it's not possible Use object property keys as enum in JSON schema (unless the answer is obsolete).
Still, I have noticed that when you write a schema in Intellij and you add a "required" = [...] it autocompletes the required fields with the property names of the "property" object (even though it doesn't use them to validate, but close enough for my purpose). I have checked out the schema for it http://json-schema.org/draft-07/schema# but haven't been able to understand how it does that.
Is there a way that I can define my schema so Intellij autocompletes based on another properties' keys like it does when you define a schema?
There is nothing in the schema itself that indicates possible values from data. There's actually no requirement that items in the required array also be defined in properties.
This sort of functionality is defined by the IDE only.
IntelliJ IDEA documents the ability to add custom schemas:
Besides schemas from JSON Schema Store, IntelliJ IDEA lets you
configure and use custom schemas from other storages. You can download
the required schema and store it under the project root or specify the
URL of the resource so IntelliJ IDEA can download the schema
automatically.
To configure a custom JSON Schema:
In the Settings/Preferences dialog ⌘,, go to Languages and Frameworks
| Schemas and DTDs | JSON Schema Mappings.
https://www.jetbrains.com/help/idea/json.html#ws_json_schema_add_custom
It also details later how to make the intelesense provide a rich preview:
Using HTML descriptions in JSON schema #
By default, IntelliJ IDEA escapes HTML characters when displaying
documentation for JSON schema definitions in documentation popups. To
get nice looking documentation with rich HTML markup, store the HTML
description in the x-intellij-html-description extension property
instead of description.
https://www.jetbrains.com/help/idea/json.html#ws_json_show_doc_in_html
However,
autocompletes based on another properties' keys
sounds like custom functionality specifically designed for writing JSON Schema. JSON Schema itself cannot reference data dynamically like that (which I assume is what you were thinking).
I am talking to an API that uses RAML. I am both downloading and uploading data to it.
When uploading data I had previously downloaded, I got a JSON Schema violation error, saying some of the parameters in the JSON objects were not allowed.
I realize the data i received is not following the JSON schema.
Is this kind of behavior a violation against the RAML principles?
What columns you GET and what columns you are allowed to POST may not be the same. For example, if you GET a user, it might include a user_id column, but you usually won't be allowed to write to it!
It's difficult to give a fuller answer without seeing the RAML spec document.
It's totally possible and valid to define different schemas for each HTTP method.
I'm looking for a Json Schema that generates a Json schema. that sounds a little confusing but if I can try to simplify it I'm looking for a Json schema that out lines the rules to make a Json schema and outputs them.
You're looking for the meta-schemas. From JSONSchema:
Meta-schemas
The meta-schemas are the schemas which define the JSON Schema and Hyper-> Schema formats.
Core/Validation Meta-Schema: Used for schemas written for pure validation.
Hyper Meta-Schema: Used for schemas written for validation and hyper-linking.
Taken from: http://json-schema.org/documentation.html
I used xml earlier and learning JSON now. To restrict the user to enter garbage values for form fields such as address etc, in XML we can define a complex object in schema file. But using JSON How can do that? JSON accepts any data from the user. Could any one please provide me ideas or an example code?
Thanks in Advance.
it is possible to do validation with schemas in JSON :
http://en.wikipedia.org/wiki/JSON#JSON_Schema
depending on what you need to validate :
http://json-schema.org/latest/json-schema-validation.html
Online tool for testing :
http://www.jsonschema.net/