I am sending a message to a third party and the payload looks like as below:
{
"keys": {
"salesCaseId": 1000449
},
"attributes": {
"assetDeliveryDetails": {
"registrationNumber": "PXS3388",
"vinNumber": "DW1265",
"deliveryDate": "16-04-2020"
}
}
}
"Keys" and "attributes" are common for all the messages between us.
Question: Because I have only one nested JSON inside attributes which is assetDeliveryDetails is it necessary to have this nested JSON property? or I can have the other 3 attributes one level higher as below :
{
"keys": {
"salesCaseId": 1000449
},
"attributes": {
"assetRegistrationNumber": "PXS3388",
"assetVinNumber": "DW1265",
"assetDeliveryDate": "16-04-2020"
}
}
Does it makes sense to group because I can logically group them together or is it an overhead because I need to create equivalent DTO classes as well?
Generally speaking you want any data structure (whether that's JSON, XML, an Object model, etc.) to be as simple as possible to fulfill your requirements. If the second model (i.e. without assetDeliveryDetails) fulfills your requirements then I recommend you go with that. Unnecessary complexity is not desirable as it will be more difficult to understand and take more resources to parse..
Related
What is the better JSON representation for GET ALL REST API response? With Option 1, it is easier to get the details of any particular employee. With Option 2, we need to filter the complete array to get any particular employee details. So, Option 1 seems to be better option. Are there any other scenarios where Option 2 can be better?
Option 1:
{
"data":{
"1234":{
"name":"ABC",
"country":"US",
"mobile":"999999999"
},
"1235":{
"name":"BDE",
"country":"IND",
"mobile":"999999998"
}
}
}
Option 2:
{
"data":[
{
"empId":"1234",
"name":"ABC",
"country":"US",
"mobile":"999999999"
},
{
"empId":"1235",
"name":"BDE",
"country":"IND",
"mobile":"999999998"
}
]
}
For option 1, since data is a JSON object, the order is not preserved and you cannot iterate over it in order. This may or may not be an issue depending on your use case. If this data is being consumed by a front-end app and you want to display the employees in order, then option 2 would be more suitable. If ordering doesn't matter, go with option 1.
This issue is a bit tricky to describe so bear with me and please ask questions if I am missing anything...
Say you have a json object that defines a list of features, each feature has a the same three properties but has a property that has an entirely different structure. For example:
{
features: [
{
id: "feature-a",
enabled: true,
configurationData: {
featureAConfigPropertyA: {
somePrperty: "whatever",
anotherProperty: true
},
featureAConfigPropertyB: "some string"
}
},
{
id: "feature-b",
enabled: true,
configurationData: {
featureBConfigArrayPropertyA: ["some string"],
featureBConfigPropertyB: [
{
"id": "some string",
"name": "some string",
"description": "some string",
"enabled": true
}
]
}
}
]
}
The actual structure of each feature is irrelevant. I am just trying to express this via json schema whereby the structure of configurationData for each feature is dependent on or dictated by the feature id value of its parent.
EDIT: I guess technically it doesnt need to be dependent on so long as either structure of configurationData is valid schema for that property on the feature schema itself. Also, the types in configurationData arent arbitrary, they would always be one of the two types for a given feature in this example.
This however needs to be structured in a way that can be expressed via Formly as I am using this to generate forms. In this case it would be an array of ObjectFieldTypes, one for feature a and one for feature b, which would enumerate the three properties and provide Input field types, until it got to configurationData at which point it would use an ObjectFieldType again, which would now be different for each field type.
The issue here is that 1) I'm not sure how to express this in json schema and 2) I can't use things like patternProperties with formly because the properties have to be explicitly defined in the json schema in order for formly to render the field types for each property. Although patternProperties would technically be valid schema in this case, if the schema doesn't define those properties, then the model in the valueChanges observable on the FormGroup just excludes them entirely. So I would end up with:
{
features:[
{
id: "feature-a",
enabled: true,
configurationData: { }
},
{
id: "feature-b",
enabled: true,
configurationData: { }
}
]
}
I have tried the if then else construct, but I cant tell if the schema is wrong or if formly just doesn't support this. I made a stack blitz for this below:
https://stackblitz.com/edit/angular-g45ydm?file=src%2Fassets%2Fjson-schema%2Fif_then.json
Trying to figuring out how to deserialize this kind of json in talend components :
{
"ryan#toofr.com": {
"confidence":119,"email":"ryan#toofr.com","default":20
},
"rbuckley#toofr.com": {
"confidence":20,"email":"rbuckley#toofr.com","default":15
},
"ryan.buckley#toofr.com": {
"confidence":18,"email":"ryan.buckley#toofr.com","default":16
},
"ryanbuckley#toofr.com": {
"confidence":17,"email":"ryanbuckley#toofr.com","default":17
},
"ryan_buckley#toofr.com": {
"confidence":16,"email":"ryan_buckley#toofr.com","default":18
},
"ryan-buckley#toofr.com": {
"confidence":15,"email":"ryan-buckley#toofr.com","default":19
},
"ryanb#toofr.com": {
"confidence":14,"email":"ryanb#toofr.com","default":14
},
"buckley#toofr.com": {
"confidence":13,"email":"buckley#toofr.com","default":13
}
}
This JSON comes from the Toofr API where documentation can be found here .
Here the actual sitation :
For each line retreived in the database, I call the API and I got this (the first name, the last name and the company change everytime.
Does anyone know how to modify the tExtractJSONField (or use smthing else) to show the results in tLogRow (for each line in the database) ?
Thank you in advance !
EDIT 1:
Here's my tExtractJSONfields :
When using tExtractJSONFields with XPath, you need
1) a valid XPath loop point
2) valid XPath mapping to your structure relative to the loop path
Also, when using XPath with Talend, every value needs a key. The key cannot change if you want to loop over it. Meaning this is invalid:
{
"ryan#toofr.com": {
"confidence":119,"email":"ryan#toofr.com","default":20
},
"rbuckley#toofr.com": {
"confidence":20,"email":"rbuckley#toofr.com","default":15
},
but this structure would be valid:
{
"contact": {
"confidence":119,"email":"ryan#toofr.com","default":20
},
"contact": {
"confidence":20,"email":"rbuckley#toofr.com","default":15
},
So with the correct data the loop point might be /contact.
Then the mapping for Confidence would be confidence (the name from the JSON), the mapping for Email would be email and vice versa for default.
EDIT
JSONPath has a few disadvantages, one of them being you cannot go higher up in the hierarchy. You can try finding out the correct query with jsonpath.com
The loop expression could be $.*. I am not sure if that will satisfy your need, though - it has been a while since I've been using JSONPath in Talend because of the downsides.
I have been ingesting some complex json structures and did this via minimal json libraries, and tjava components within talend.
I'm relatively new to REST but I've been doing my homework on how RESTful should be. Now I'm trying to create a RESTful api implementing a JSON+HAL serializer for my models which have relationships with other models.
Example models in python:
class Category(Model):
name = CharField()
parent = ManyToOneField(Category)
categories = OneToManyField(Category)
products = ManyToManyField(Product)
class Product(Model):
name = CharField()
price = DecimalField()
related = ManyToManyField(Product)
categories = ManyToManyField(Category)
lets suppose we have a category "catalog" with a sub-category "food" with products "burger" and "hot-dog" which are both related.
First question. Categories and products should be resources so they need an URI, should I implement an uri field in my model and store it in the DB or somehow calculate it at runtime, what about multiple identifiers(URIs)?
Second question. Discoverability, In Hal format what should "GET /" and diferent nodes return to make the api easily self discoverable.
{
"_links":{
"self":{
"href":"/"
},
"categories":[
{
"href":"/catalog"
}
]
}
}
Third question. Add as properties, embed or link. Example "GET /catalog/food":
{
"_links":{
"self":{
"href":"/catalog/food"
}
},
"name":"food",
"parent":"/catalog",
"categories":[],
"products":[
"/products/burger",
"/products/hot-dog"
]
}
{
"_links":{
"self":{
"href":"/catalog/food"
},
"parent":{
"href":"/catalog"
},
"categories":[
],
"products":[
{
"href":"/products/burger"
},
{
"href":"/products/hot-dog"
}
]
},
"name":"food"
}
{
"_links":{
"self":{
"href":"/catalog/food"
}
},
"name":"food",
"_embedded":{
"parent":{
"_links":{
"self":{
"href":"/catalog"
}
},
"name":"catalog",
...
},
"categories":[
],
"products":[
{
"_links":{
"self":{
"href":"/products/burger"
}
},
"name":"burger",
...
},
{
"_links":{
"self":{
"href":"/products/hot-dog"
}
},
"name":"hot-dog",
...
}
]
}
}
Fourth question. How deep should I go when returning structures. Example "GET /catalog
{
"_links":{
"self":{
"href":"/catalog"
}
},
"name":"catalog",
"parent":null,
"categories":[
{
"name":"food",
"parent":{...},
"categories":[],
"products":[
{
"name":"burger",
"price":"",
"categories":[...],
"related":[...]
},
{
"name":"hot-dog",
"price":"",
"categories":[...],
"related":[...]
}
]
}
],
"products": []
}
About 1st question: I wouldn't store the URIs in the DB. You can easily calculate them inside your controller in runtime, and it's of controller's responsibility to care about URIs. This way you keep your model and your API decoupled, and should you decide to change the API structure in the future, you won't need to update your whole database with the new URIs.
About multiple identifiers, I'm not sure what the question is, but again, in my opinion, it has nothing to do with the DB, it's the routing and the controllers who should care about how to deal with any URIs.
About 2nd question: First of all, as a side note: I would keep the word categories as part of the URI. For example, I 'd have http://domain.com/api/categories/catalog/food. This way, you make your API more descriptive and more "hackable", meaning that user should be able to remove the /catalog/food part and expect to receive a collection with all the available categories.
Now, about what GET should return to allow discoverability: I think it's already being made clear from your URI structure. When user hits GET /categories he expects to get a list with the categories (the name and URI for each, to keep it lightweight), and when he follows one of the URIs like GET /categories/catalog he should receive the resource catalog which is a category. Likewise, when he wants to GET /products/burger, he should receive a product resource with all the attributes you have in your model. You may want to check this example about the structure of your responses.
About 3rd question: Again, the same example can help you form the structure. I think your 2nd way of response is closer to that, but I would also add a name field, not only the href.
About 4th question: When the GET request expects a collection of resources (like GET /categories) I would suggest providing only the necessary for each resource, that is, the name and the URI for each and only when user follows the desired URI, he can receive the rest information.
In your example, catalog is a resource, so on GET /categories/catalog I would include of course the name of the resource (catalog) and its self link, and for parent, sub-categories and products that are related to it, I would just provide the name and the URI for each, to keep it light. But: This was a general thought about designing APIs. In your actual problem, you should decide depending on your specific business problem. I mean, if your API is about a restaurant menu with categories and dishes, you may want to include the price, or a small description even when responding not for the actual product but for a collection of products, because probably for your users, that's an important information. So generally, provide all the necessary info (you only know what are these for your problem) when responding about a list of resources, and provide all the details of the resource when responding about a specific resource.
I would store something in the DB and calculate the URI at Runtime. That way if you move boxes it's not static.
Create a 'bookmark' page. The page we created was just a list of links with their rels. I believe HAL defines that specifically. The bookmark page was the only page other pages needed to know about
Not sure about this
How deep you go is up to you. There is a big debate now at my place of work for fine grain vs course grain. I'm going to do fine grain with small resource to keep the api simple, but then use the Expand-ability concept. It's a combination ofthe idea of composite resources defined on pg 35 of Subbu’s REST book and the expand concept used by Netflix. http://developer.netflix.com/docs/REST_API_Conventions
I'm going to implement a groupchat into my app by using Firebase. I were thinking of 2 different structures of saving data in JSON.
First structure:
Second structure:
I would like to achieve a fast query and i would like it to parse a small amount of data each time. What structure should i go for and are there maybe a better alternative then these 2?
The first solution is clearly not viable, as you would have a hard time finding all messages belonging to a given group.
The second solution is ok if every time you will query a given group's node, you are going to need also all of its messages, which is probably not what you want.
It is hard, of course, to advise on data structure without more info on your use-case, the queries you are going to make, etc., but a rather standard approach would be:
{
"users": {
"$userId"": {
// user data
}
},
"groups": {
"$groupId": {
// group data
}
},
"group_users": {
"$groupId": {
"$userId": true
// separation of list of users from the group is useful
// if you are going to query the group node not needing its full list of users
}
},
"group_messages": {
"$groupId": {
"$messageId": {
// message data
}
}
}
}