JSON structuring a groupchat - json

I'm going to implement a groupchat into my app by using Firebase. I were thinking of 2 different structures of saving data in JSON.
First structure:
Second structure:
I would like to achieve a fast query and i would like it to parse a small amount of data each time. What structure should i go for and are there maybe a better alternative then these 2?

The first solution is clearly not viable, as you would have a hard time finding all messages belonging to a given group.
The second solution is ok if every time you will query a given group's node, you are going to need also all of its messages, which is probably not what you want.
It is hard, of course, to advise on data structure without more info on your use-case, the queries you are going to make, etc., but a rather standard approach would be:
{
"users": {
"$userId"": {
// user data
}
},
"groups": {
"$groupId": {
// group data
}
},
"group_users": {
"$groupId": {
"$userId": true
// separation of list of users from the group is useful
// if you are going to query the group node not needing its full list of users
}
},
"group_messages": {
"$groupId": {
"$messageId": {
// message data
}
}
}
}

Related

Better JSON representation for GET ALL REST API response

What is the better JSON representation for GET ALL REST API response? With Option 1, it is easier to get the details of any particular employee. With Option 2, we need to filter the complete array to get any particular employee details. So, Option 1 seems to be better option. Are there any other scenarios where Option 2 can be better?
Option 1:
{
"data":{
"1234":{
"name":"ABC",
"country":"US",
"mobile":"999999999"
},
"1235":{
"name":"BDE",
"country":"IND",
"mobile":"999999998"
}
}
}
Option 2:
{
"data":[
{
"empId":"1234",
"name":"ABC",
"country":"US",
"mobile":"999999999"
},
{
"empId":"1235",
"name":"BDE",
"country":"IND",
"mobile":"999999998"
}
]
}
For option 1, since data is a JSON object, the order is not preserved and you cannot iterate over it in order. This may or may not be an issue depending on your use case. If this data is being consumed by a front-end app and you want to display the employees in order, then option 2 would be more suitable. If ordering doesn't matter, go with option 1.

JSON structure for message payload

I am sending a message to a third party and the payload looks like as below:
{
"keys": {
"salesCaseId": 1000449
},
"attributes": {
"assetDeliveryDetails": {
"registrationNumber": "PXS3388",
"vinNumber": "DW1265",
"deliveryDate": "16-04-2020"
}
}
}
"Keys" and "attributes" are common for all the messages between us.
Question: Because I have only one nested JSON inside attributes which is assetDeliveryDetails is it necessary to have this nested JSON property? or I can have the other 3 attributes one level higher as below :
{
"keys": {
"salesCaseId": 1000449
},
"attributes": {
"assetRegistrationNumber": "PXS3388",
"assetVinNumber": "DW1265",
"assetDeliveryDate": "16-04-2020"
}
}
Does it makes sense to group because I can logically group them together or is it an overhead because I need to create equivalent DTO classes as well?
Generally speaking you want any data structure (whether that's JSON, XML, an Object model, etc.) to be as simple as possible to fulfill your requirements. If the second model (i.e. without assetDeliveryDetails) fulfills your requirements then I recommend you go with that. Unnecessary complexity is not desirable as it will be more difficult to understand and take more resources to parse..

Returning a specific JSON object using Express

So let's say I've got a massive JSON file, and the general structure is roughly like so:
{
"apples": { complex object },
"oranges": { complex object },
"grapes": { complex object }
}
Is there some way to specifically target an object to return while using express? As in, say, if someone made a simple get request to my server, it'd return specifically the given object(s). I know the syntax and concept is completely wrong in this instance but for lack of a better way to say it, something like...
let testData = 'testdata.json';
app.get('/thing', res => {
res.json(testData.oranges);
}
I know you can return the entire file, but that adds a good amount of loading time in this instance, and is impractical in this particular case.
Or, alternatively - would it be better to have node parse the JSON file and split it into an apples.json, oranges.json, etc files to use? Trying to understand A, the best practice for doing something like this, and B, the most effective way to translate this into a practical application for a medium sized project.
Any thoughts or advice along this line - even if it's a library recommendation - would be greatly appreciated.
It should work if you make a POST request caring the payload of the specific 'thing', and then returning an object based on that thing. Example:
let testData = {
"apples": { complex object },
"oranges": { complex object },
"grapes": { complex object }
};
app.post('/route', (req, res) => {
thing = req.body.thing;
res.json(testData[thing]);
}
This is a GET request for some data and essentially since the JSON file can be used as a key/value store to query for the desired response data.
Assuming the query parameter for specifying the desired key for the object to return is part then the following example would work:
const testData = require('./testdata.json');
app.get('/thing', (req, res) => res.json(testdata[req.query.part]);
Querying for /thing?part=apples would return testdata.apples in the response.

JSON Deserialization on Talend

Trying to figuring out how to deserialize this kind of json in talend components :
{
"ryan#toofr.com": {
"confidence":119,"email":"ryan#toofr.com","default":20
},
"rbuckley#toofr.com": {
"confidence":20,"email":"rbuckley#toofr.com","default":15
},
"ryan.buckley#toofr.com": {
"confidence":18,"email":"ryan.buckley#toofr.com","default":16
},
"ryanbuckley#toofr.com": {
"confidence":17,"email":"ryanbuckley#toofr.com","default":17
},
"ryan_buckley#toofr.com": {
"confidence":16,"email":"ryan_buckley#toofr.com","default":18
},
"ryan-buckley#toofr.com": {
"confidence":15,"email":"ryan-buckley#toofr.com","default":19
},
"ryanb#toofr.com": {
"confidence":14,"email":"ryanb#toofr.com","default":14
},
"buckley#toofr.com": {
"confidence":13,"email":"buckley#toofr.com","default":13
}
}
This JSON comes from the Toofr API where documentation can be found here .
Here the actual sitation :
For each line retreived in the database, I call the API and I got this (the first name, the last name and the company change everytime.
Does anyone know how to modify the tExtractJSONField (or use smthing else) to show the results in tLogRow (for each line in the database) ?
Thank you in advance !
EDIT 1:
Here's my tExtractJSONfields :
When using tExtractJSONFields with XPath, you need
1) a valid XPath loop point
2) valid XPath mapping to your structure relative to the loop path
Also, when using XPath with Talend, every value needs a key. The key cannot change if you want to loop over it. Meaning this is invalid:
{
"ryan#toofr.com": {
"confidence":119,"email":"ryan#toofr.com","default":20
},
"rbuckley#toofr.com": {
"confidence":20,"email":"rbuckley#toofr.com","default":15
},
but this structure would be valid:
{
"contact": {
"confidence":119,"email":"ryan#toofr.com","default":20
},
"contact": {
"confidence":20,"email":"rbuckley#toofr.com","default":15
},
So with the correct data the loop point might be /contact.
Then the mapping for Confidence would be confidence (the name from the JSON), the mapping for Email would be email and vice versa for default.
EDIT
JSONPath has a few disadvantages, one of them being you cannot go higher up in the hierarchy. You can try finding out the correct query with jsonpath.com
The loop expression could be $.*. I am not sure if that will satisfy your need, though - it has been a while since I've been using JSONPath in Talend because of the downsides.
I have been ingesting some complex json structures and did this via minimal json libraries, and tjava components within talend.

Design RESTful API using HAL - serialize model relationships

I'm relatively new to REST but I've been doing my homework on how RESTful should be. Now I'm trying to create a RESTful api implementing a JSON+HAL serializer for my models which have relationships with other models.
Example models in python:
class Category(Model):
name = CharField()
parent = ManyToOneField(Category)
categories = OneToManyField(Category)
products = ManyToManyField(Product)
class Product(Model):
name = CharField()
price = DecimalField()
related = ManyToManyField(Product)
categories = ManyToManyField(Category)
lets suppose we have a category "catalog" with a sub-category "food" with products "burger" and "hot-dog" which are both related.
First question. Categories and products should be resources so they need an URI, should I implement an uri field in my model and store it in the DB or somehow calculate it at runtime, what about multiple identifiers(URIs)?
Second question. Discoverability, In Hal format what should "GET /" and diferent nodes return to make the api easily self discoverable.
{
"_links":{
"self":{
"href":"/"
},
"categories":[
{
"href":"/catalog"
}
]
}
}
Third question. Add as properties, embed or link. Example "GET /catalog/food":
{
"_links":{
"self":{
"href":"/catalog/food"
}
},
"name":"food",
"parent":"/catalog",
"categories":[],
"products":[
"/products/burger",
"/products/hot-dog"
]
}
{
"_links":{
"self":{
"href":"/catalog/food"
},
"parent":{
"href":"/catalog"
},
"categories":[
],
"products":[
{
"href":"/products/burger"
},
{
"href":"/products/hot-dog"
}
]
},
"name":"food"
}
{
"_links":{
"self":{
"href":"/catalog/food"
}
},
"name":"food",
"_embedded":{
"parent":{
"_links":{
"self":{
"href":"/catalog"
}
},
"name":"catalog",
...
},
"categories":[
],
"products":[
{
"_links":{
"self":{
"href":"/products/burger"
}
},
"name":"burger",
...
},
{
"_links":{
"self":{
"href":"/products/hot-dog"
}
},
"name":"hot-dog",
...
}
]
}
}
Fourth question. How deep should I go when returning structures. Example "GET /catalog
{
"_links":{
"self":{
"href":"/catalog"
}
},
"name":"catalog",
"parent":null,
"categories":[
{
"name":"food",
"parent":{...},
"categories":[],
"products":[
{
"name":"burger",
"price":"",
"categories":[...],
"related":[...]
},
{
"name":"hot-dog",
"price":"",
"categories":[...],
"related":[...]
}
]
}
],
"products": []
}
About 1st question: I wouldn't store the URIs in the DB. You can easily calculate them inside your controller in runtime, and it's of controller's responsibility to care about URIs. This way you keep your model and your API decoupled, and should you decide to change the API structure in the future, you won't need to update your whole database with the new URIs.
About multiple identifiers, I'm not sure what the question is, but again, in my opinion, it has nothing to do with the DB, it's the routing and the controllers who should care about how to deal with any URIs.
About 2nd question: First of all, as a side note: I would keep the word categories as part of the URI. For example, I 'd have http://domain.com/api/categories/catalog/food. This way, you make your API more descriptive and more "hackable", meaning that user should be able to remove the /catalog/food part and expect to receive a collection with all the available categories.
Now, about what GET should return to allow discoverability: I think it's already being made clear from your URI structure. When user hits GET /categories he expects to get a list with the categories (the name and URI for each, to keep it lightweight), and when he follows one of the URIs like GET /categories/catalog he should receive the resource catalog which is a category. Likewise, when he wants to GET /products/burger, he should receive a product resource with all the attributes you have in your model. You may want to check this example about the structure of your responses.
About 3rd question: Again, the same example can help you form the structure. I think your 2nd way of response is closer to that, but I would also add a name field, not only the href.
About 4th question: When the GET request expects a collection of resources (like GET /categories) I would suggest providing only the necessary for each resource, that is, the name and the URI for each and only when user follows the desired URI, he can receive the rest information.
In your example, catalog is a resource, so on GET /categories/catalog I would include of course the name of the resource (catalog) and its self link, and for parent, sub-categories and products that are related to it, I would just provide the name and the URI for each, to keep it light. But: This was a general thought about designing APIs. In your actual problem, you should decide depending on your specific business problem. I mean, if your API is about a restaurant menu with categories and dishes, you may want to include the price, or a small description even when responding not for the actual product but for a collection of products, because probably for your users, that's an important information. So generally, provide all the necessary info (you only know what are these for your problem) when responding about a list of resources, and provide all the details of the resource when responding about a specific resource.
I would store something in the DB and calculate the URI at Runtime. That way if you move boxes it's not static.
Create a 'bookmark' page. The page we created was just a list of links with their rels. I believe HAL defines that specifically. The bookmark page was the only page other pages needed to know about
Not sure about this
How deep you go is up to you. There is a big debate now at my place of work for fine grain vs course grain. I'm going to do fine grain with small resource to keep the api simple, but then use the Expand-ability concept. It's a combination ofthe idea of composite resources defined on pg 35 of Subbu’s REST book and the expand concept used by Netflix. http://developer.netflix.com/docs/REST_API_Conventions