I have discovery rule, which returns JSON document:
{
"data":[
{"SIZE":9556},
{"SIZE_DIFFERENCE":0.00502302218501465},
{"DUPLICATES":0},
{"TODAY_ZERO_CLPRICE":9556},
{"LISTED_SYMBOLS":true}
]
}
Can I assign values of this JSON objects to Item prototypes? Or handle it in triggers.
Like "If SIZE < 1 Warning will appear"
Thank you
The JSON document in the question is not very valid for low-level discovery.
In that JSON, the data element has five objects, each with distinct attributes. Something like that would be more appropriate (note the LLD macro syntax):
{
"data":[
{
"{#SIZE}":9556,
"{#SIZE_DIFFERENCE}":0.00502302218501465,
"{#DUPLICATES}":0,
"{#TODAY_ZERO_CLPRICE}":9556,
"{#LISTED_SYMBOLS}":true
}
]
}
If you wish to create items with fixed values, you could probably create calculated items with a constant expression, like so:
{#SIZE}
However, a better approach would be to create trapper items during LLD and send those values separately.
Please see official documentation on low-level discovery and trapper items for more information.
Related
I'm consuming a Kafka topic published by another team (so I have very limited influence over the message format). The message has a field that holds an ARRAY of STRUCTS (an array of objects), but if the array has only one value then it just holds that STRUCT (no array, just an object). I'm trying to transform the message using Confluent KSQL. Unfortunately, I cannot figure out how to do this.
For example:
{ "field": {...} } <-- STRUCT (single element)
{ "field": [ {...}, {...} ] } <-- ARRAY (multiple elements)
{ "field": [ {...}, {...}, {...} ] <-- ARRAY (multiple elements)
If I configure the field in my message schema as a STRUCT then all messages with multiple values error. If I configure the field in my message schema as an ARRAY then all messages with a single value error. I could create two streams and merge them, but then my error log will be polluted with irrelevant errors.
I've tried capturing this field as a STRING/VARCHAR which is fine and I can split the messages into two streams. If I do this, then I can parse the single value messages and extract the data I need, but I cannot figure out how to parse the multivalue messages. None of the KSQL JSON functions seem to allow parsing of JSON Arrays out of JSON Strings. I can use EXTRACTJSONFIELD() to extract a particular element of the array, but not all of the elements.
Am I missing something? Is there any way to handle this reasonably?
In my experience, this is one use-case where KSQL just doesn't work. You would need to use Kafka Streams or a plain consumer to deserialize the event as a generic JSON type, then check object.get("field").isArray() or isObject(), and handle accordingly.
Even if you used a UDF in KSQL, the STREAM definition would be required to know ahead of time if you have field ARRAY<?> or field STRUCT<...>
I finally solved this in a roundabout way...
First, I created an initial stream reading the transaction as a stream of bytes using KAFKA format instead of JSON format. This allows me to put a filter conditional filter on the data so I can fork the stream into a version for the single (STRUCT) variation and a version for the multiple (ARRAY) variation.
The initial stream looks like:
CREATE OR REPLACE STREAM `my-topic-stream` (
id STRING KEY,
data BYTES
)
WITH (
KAFKA_TOPIC='my-topic',
VALUE_FORMAT='KAFKA'
);
Forking that stream looks like this with a second for a multiple version filtering for IS NOT NULL:
CREATE OR REPLACE STREAM `my-single-stream`
WITH (
kafka_topic='my-single-topic'
) AS
SELECT *
FROM `my-topic-stream`
WHERE JSON_ARRAY_LENGTH(EXTRACTJSONFIELD(FROM_BYTES(data, 'utf8'), '$.field')) IS NULL;
At this point I can create a schema for both variations, explode field, and merge the two streams back together. I don't know if this can be refined to be more efficient, but this successfully processes the transactions as I wanted.
The json data structure for jstree is define in https://github.com/vakata/jstree, here is an example
[ { "text" : "Root node", "children" : [ "Child node 1", "Child node 2" ] } ]
Notably it says
The children key can be used to add children to the branch, it should
be an array
However later on in section Populating the tree using AJAX and lazy loading nodes it shows to use set children to false to indicate when a child has not be processed
[{
"id":1,"text":"Root node","children":[
{"id":2,"text":"Child node 1","children":true},
{"id":3,"text":"Child node 2"}
]
}]
So here we see children used as both as an array and as a boolean
I am using jstree as an example because this is where I encountered the issue, but my question is really a general json question. My question is this, is it valid JSON for the same element in json to be two different types (an array and a boolean)
Structure wise, both are valid JSON packets. This is okay, as JSON is somewhat less stricter than XML(with a XSD or a DTD). As per: https://www.w3schools.com/js/js_json_objects.asp,
JSON objects are surrounded by curly braces {}.
JSON objects are written in key/value pairs.
Keys must be strings, and values must be a valid JSON data type (string, number, object, array, boolean or null).
Keys and values are separated by a colon.
Each key/value pair is separated by a comma.
Having said that, if the sender is allowed to send such JSONs, only caveat is that server side will have to handle this discrepancy upon receiving such different packets. This is a bad-looking-contract, and hence server might need to do extra work to manage it. Server side handling of such incoming JSON packets can become tricky.
See: How do I create JSON data structure when element can be different types in for use by
You could validate whether a JSON is okay or not at https://jsonlint.com/
See more about JSON in this answer: https://stackoverflow.com/a/4862511/945214
It is valid Json. JSON RFC 8259 defines a general syntax but it contains nothing that would allow a tool to identify that two equally named entries are meant to describe the same conceptual thing.
The need to have a criteria to check two JSON structures for instance equality has been one motivation to create something like Json Schema.
I also think it is not too unusual for javascript to provide this kind of mixed data. Sometimes it might help to explicitly convert the javascript object to JSON. Like in JSON.stringify(testObject)
A thing for json validation
https://www.npmjs.com/package/json-validation
https://davidwalsh.name/json-validation.
For a sample JSON data which looks like this -
{
"children":{
"Alice":{...},
"Jamie":{...},
"Bob":{...}
// Any new child with a given unique name will be added to this object
},
childrenOrder:["Alice", "Bob", "Jamie"]
}
In the corresponding JSON Schema, I am trying to limit the valid values in "childrenOrder" array to be from the run time children keys.
I didn't see any means of referring to runtime dynamic values in the official JSON Schema documentation (http://json-schema.org/documentation.html).
Is this even possible at the moment?
For the sake of brevity I omitted JSON Schema code. I can add it if folks think it is needed to address the question.
Thanks in advance.
No it is not possible using the current JSON Schema specification. However, there is a proposal for the next version of JSON Schema that could change that.
https://github.com/json-schema/json-schema/wiki/%24data-(v5-proposal)
I am working on a (.NET) REST API which is returning some JSON data. The consumer of the API is an embedded client. We have been trying to establish the structure of the JSON we will be working with. The format the embedded client wants to use is something I have not seen before in working with JSON. I suggested that it is no "typical" JSON. I was met with the question "Where is 'typical' JSON format documented"?
As an example of JSON I "typically" see:
{
"item" : {
"users": [ ... list of user objects ... ],
"times": [ ... list of time objects ...],
}
}
An example of the non-typical JSON:
{
"item" : [
{
"users": [ ... list of user objects ... ]
},
{
"times": [ ... list of time objects ...]
},
]
}
In the second example, item contains an array of objects, which each contain a property whose value is an array of entities. This is valid JSON. However, I have not encountered another instance of JSON that is structured this way when it is not an arbitrary array of objects but is in fact a set list of properties on the "item" object.
In searching json.org, stackoverflow.com and other places on the interwebs I have not found any guidelines on why the structure of JSON follows the "typical" example above rather than the second example.
Can you provide links to documentation that would provide recommendations for one format or the other above?
Not a link, but just straightforward answer: Items are either indexed (0, 1, 2, ...) or keyed (users, times). No matter what software you use, you can get at indexed or keyed data equally easily and quickly. But not with what you call "non-typical" JSON: To get at the users, I have to iterate through the array and find one dictionary that has a key "users". But there might be two or more dictionaries with that key. So what am I supposed to do then? If you use JSON schema, the "non-typical" JSON is impossible to check. In iOS, in the typical case I write
NSArray* users = itemDict [#"users"];
For the non-typical JSON I have to write
NSArray* users = nil;
for (NSDictionary* dict in itemArray)
if (dict [#"users"] != nil)
users = dict [#"users"];
but that still has no error checking for multiple dicts with the key "users". Which is an error that in the first case isn't even possible. So just tell them what the are asking for is rubbish and creates nothing but unnecessary work. For other software, you probably have the same problems.
I am using JSON for defining some configuration files and I want to validate them with json scheme. My problem is that I want to ensure that keys of some object should be subset of items of an array defined in same JSON:
ex:
Valid:
{
"files": ["file1", "file2"],
"filelocations": {
"file1": "/etc/globalconfigs/file1.conf",
"file2": "/usr/bin/file2.sh"
}
}
Invalid (otherkey is not in files):
{
"files": ["file1", "file2"],
"filelocations": {
"file1": "/etc/globalconfigs/file1.conf",
"otherkey": "/usr/bin/file2.sh"
}
}
etc. What I want is to ensure that keys of filelocations are found in files array.
Although in this example, we can change the structure of JSON by combining keys-values so that there is no need to have this kind of constraint, in my case I can't change JSON like this, so it is nice to have a validation mechanism for this.
How can I achieve this?
You cannot achieve this with JSON Schema, there are no combination of keywords which can guarantee this.
If you are adventurous (and I can even code that for you), you can use my JSON Schema API and code a custom keyword to fit your needs, however. It is doable.