How to get structured JSON using KSQL? - json

There are two tables in the database. For example: Layouts(Id, VenueId, Description) and Venues(Id, Adress, Phone). Layout table has reference key to Venues.
There are also two topics in Kafka accordingly tables. How can I send into my output topic JSON like this:
{
"Id": "1",
"Description": "LayoutDesc",
"Venue": {
"Id": 5,
"Adress": "VenueAdress",
"Description": "VenueDesc"
}
}

As of 5.1.1 you can't construct nested objects in KSQL (you can only read them). There is an open issue for this, please do upvote/comment on it: https://github.com/confluentinc/ksql/issues/2147

Related

Is it okay to use a flexible json attribute type? array vs object based on the amount of items?

i'm trying to integrate our application with a payment providers. There API uses both Arrays and Objects on the same json property.
Example:
When getting the Shoppingcart with cart-items, the response will be like this when there is one cart-item:
GET /cart/{cart-identifier}
{
"cart_identifier": 1,
"items": {
"product_identifier": 2,
"amount": 1
}
}
When there are 2 items in the cart, the response will be like this.
{
"cart_identifier": 1,
"items": [
{
"product_identifier": 2,
"amount": 1
},
{
"product_identifier": 3,
"amount": 1
}
]
}
To me, this does not make sense, but does anyone know what the JSON specification says of this? And are there any good reasons to do it this way?
Ps: If you have some good blog posts that are related to this then please send me a message.
As far as the JSON specification is concerned, both forms are correct. You may have problems if you try to map the JSON to a Java class though. This depends on what parser you're using.

Is it possible to create graphs taking data from json in Zabbix?

Would it be possible, in any way, to create json code that zabbix can understand and recreate on a graph?
Eg:
I have this json:
{
"response:" {
"success": true,
"server": {
"name": "Test Server",
"alive": true,
"users": 25
}
}
}
And I would like to have a simple graph where I can see the value of users.
I might be asking a nonsense here but I was reading about the URL element and it looks like it is possible but couldn't find any type template or any info on how to send the data.
Create a Zabbix trapper item and send such values with the zabbix_sender. The values will be processed as any normal item values by Zabbix, and graphs will be available as well.

Grails Json Views "model" key conflicts with model keyword

I am using the rest-api profile for a Grails app and have the following in one of my json views (_event.gson):
model {
Event event
}
json g.render(event, [excludes: ['product']]) {
product {
id event.product.id
name event.product.name
model event.product.model
}
}
In short, a Product belongs to an Event. By default, I would get the product key with an id inside it as json. I wanted to add more fields to that.
So I used excludes so I could define the fields that would appear under the embedded json document detailing the product. My goal is to have the following as json:
{
"id": 123,
...,
"product": {
"id": 545434,
"name": "Something Cool",
"model": "MZX 1234"
}
}
The last fields -- model -- is not appearing. It seems it is being confused with the model keyword that is used in the very first line of my _event.gson file. Is there any way around this? I tried adding quotes to "model" but it still does not work.
The issue has been reported as a bug: http://github.com/grails/grails-views/issues/45

Mongo DB query of complex json structure

Say I have a json structure like so:
{
"A":{
"name":"dog",
"foo":"bar",
"array":[
{"name":"one"},
{"name":"two"}
]
},
"B":{
"name":"cat",
"foo":"bar",
"array":[
{"name":"one"},
{"name":"three"}
]
}
}
I want to be able to do two things.
1: Query for any "name":* within "A.array".
2: Query for any "name":"one" within "*.array".
That is, any object within a specific document's array, and any specific object within any document's array.
I hope I have used proper terminology here, I am just starting to familiarize myself with a lot of these concepts. I have tried searching for an answer but am having trouble finding something like my case.
Thanks.
EDIT:
Since I still haven't really made progress towards this, I'll just explain what I'm trying to do: I want to use the "AllSets" dataset (after I trim it down below 16mb) available on mtgjson.com. I am having problems getting mongo to play nicely though.
In an effort to try and learn what's going on, I have downloaded one set: http://mtgjson.com/json/OGW.json.
Here is a photo of its structure laid out:
I am unable to even get mongo to return an object from within the cards array using:
"find({cards: {$elemMatch: {name:"Deceiver of Form"}}})"
"find({"cards.name":"Deceiver of Form"})"
When I run either of the commands above it just returns the entire document to me.
You could use the positional projection $ operator to limit the contents of an array. For example, if you have a single document like below:
{
"block": "Battle for Zendikar",
"booster": "...",
"translations": "...",
"cards": [
{
"name": "Deceiver of Form",
"power": "8"
},
{
"name": "Eldrazi Mimic",
"power": "2"
},
{
"name": "Kozilek, the Great Distortion",
"power": "12"
}
]
}
You can query for a card name matching "Deceiver of Form", and limit fields to return only the matching array card element(s) using:
> db.collection.find({"cards.name":"Deceiver of Form"}, {"cards.$":1})
{
"_id": ObjectId("..."),
"cards": [
{
"name": "Deceiver of Form",
"power": "8"
}
]
}
Having said the above, I think you should re-consider your data model. MongoDB is a document-oriented database. A record in MongoDB is a document, so having a single record in a database does not bring out the potential of the database i.e. similar to storing all data in a single row in a table.
You should try storing the 'cards' into a collection instead. Where each document is a single card, (depending on your use case) you could add a reference to another collection containing the deck information. i.e: block, type, releaseDate, etc. For example:
// a document in cards collection:
{
"name": "Deceiver of Form",
"power": "8",
"deck_id": 1
}
// a document in decks collection:
{
"deck_id": 1,
"releaseDate": "2016-01-22",
"type": "expansion"
}
For different types of data model designs and examples, please see Data Model Design.

Issue with Cloud Datastore backup in BigQuery

I use an App Enginge Datastore backup file and create a BigQuery table. The issue I face is all the JSON values are treated as 'Flattened strings' by default.
I couldn't access the repeated string value for example as below. Value is for column: qoption
[{
"optionId": 0,
"optionTitle": "All inclusive",
"optionImageUrl": "http://sampleurl",
"masterCatInfo": 95680,
"brInfo": 56502428160,
"category": "",
"tags": ["Holiday"]
}, {
"optionId": 1,
"optionTitle": "Self catered",
"optionImageUrl": "http://sampleurl1",
"masterCatInfo": 520280,
"brId": 56598160,
"category": "",
"tags": ["Holiday"]
}]
Is it possibe to again recreate existing table as in JSON format , ideally through BQ CLI, so that I can access table qoption.optionId, qoption.optionTitle,etc
Take a look at Nested and Repeated Data. Basically you have to manually setup your bigquery schema with a nested data schema. Once that is done and your data is imported you should be able to use your nested properties.
Alternatively big query can parse your json ad-hoc.