Json - optional subdocument - json

I´m getting a json from an application with a couple of nested subdocuments. Some of those documents are optional and not present all the time. I´m wondering if there is a best practice how to handel this.
e.g. (The document is just an example, the real one looks differnt but I can´t post it, the Example is copied from: How to represent sub-documents in JSON array as Java Collection using Jackson?): The Adreess subdocument is not present in every document I receive.
{
"attributes": {
"type": "Lead",
"url": "/services/data/v30.0/sobjects/Lead/00Qi000000Jr44XEAR"
},
"Id": "00Qi000000Jr44XEAR",
"Name": "Kristen Akin",
"Address": {
"city": null,
"country": "USA",
"state": "CA",
"stateCode": null,
"street": null
},
"Phone": "(434) 369-3100"
}
Currently I´m receiving the data in the worst possible way I can imagine with a differnt type, which is like:
{
"attributes": {
"type": "Lead",
"url": "/services/data/v30.0/sobjects/Lead/00Qi000000Jr44XEAR"
},
"Id": "00Qi000000Jr44XEAR",
"Name": "Kristen Akin",
"Address": "",
"Phone": "(434) 369-3100"
}
I want to suggest better ways and I´m wondering whats the best one?
Leaving the adress subdocument out completely
receiving "Adress: null"
receiving Adress: {}
receiving Adress: {"city": null, "country": null, ...}
anything else
Personally I would go with Nr. 3 because I still get a (sub)document and can treat it the usual way. Does anythin speak against it or are there any best practices for this situation?
Thanks in advance.
Best regards.

Go with 3.
Leaving the adress subdocument out completely
Would work for many deserialization tools, but it is hard to identify the structure and identify if something is missing on debugging easily
receiving "Adress: null"
Would work for many deserialization tools, but it is not a good practice to deliver null for more complex attributes like arrays or objects. You cannot identify, that this is a complex object easily.
receiving Adress: {}
It is a good practice to deliver empty arrays if they are empty and empty objects, if they are empty. You can identify that there could be a complex object but it is not available here. Please go with this solution
receiving Adress: {"city": null, "country": null, ...}
Don't do this. It gives you more details for the complex object, but you cannot identify easily if the address was not added on purpose or if the API partner sends incomplete address data by accident or if incomplete data is valid on their side.

I always differentiate between values which are:
set but empty: We usually interpret these values as valid values which are intended to be empty, like an empty address book, which may contain no entries at all.
undefined: usually this is an optional value. The application has to handle if it needs the data from somewhere else.
null: setting a value intentionally to null means to invalidate the value. We often use this to reset the data. In case of the address book means: there is no address book at all, even no empty one.
I would prefer these options:
1.: if it is left out, it is undefined and means that it is up to the application to handle undefined values. Especially for optional values, you should be aware of handling undefined values.
3.: if it is empty, you still have a valid address book, but an empty one, which makes the handling in code easier.
What I would avoid:
4.: You get an valid address with invalid data, so you have to deep-check if the address is usable, which increases the efforts on validation, so I would not use this option.
5.: changing the data type to "" is also bad because for typed languages it will make it hard to parse because it expects an object but receives a string.

Related

What is the best practice in REST-Api, to pass structured data or key-value pair?

I have a data-structure similar to the given below, which I am supposed to process. I am designing an API which should accept a POST request similar to the one given below. (ignore the headers, etc)
{
"Name" : "Johny English",
"Id": "534dsf",
"Message":[
{
"Header":"Country of origin",
"Value":"England"
},
{
"Header":"Nature of work",
"Value":"Secret Agent/Spy"
}
]
}
Some how I do not feel, its a correct way to pass/accept data. Here I am talking about structured data vs. Key-Value pair. While I can extract the fields ("Name", "Id") directly to an object attributes, but for Key-Value pairs, I need to loop through the collection and compare with strings (eg. "Nature of Work") to extract values.
I searched few sites, looking for any best practices, could not reach into any conclusion. Is there any best practice, suggestions etc.
I don't think you are going to find any firm, evidence based arguments against including a list of key value pairs in your message schema. But that's the sort of thing to look for - people writing about message schema design, and how to design messages to support change, and so on.
As a practical matter, there's not a whole lot of difference
{
"Name" : "Johny English",
"Id": "534dsf",
"Message":[
{
"Header":"Country of origin",
"Value":"England"
},
{
"Header":"Nature of work",
"Value":"Secret Agent/Spy"
}
]
}
or
{
"Name" : "Johny English",
"Id": "534dsf",
"Message": {
"Country of origin": "England",
"Nature of work": "Secret Agent/Spy"
}
}
In the early days of the world wide web, "everything" is key value pairs, because it was easy to describe a collection of key value pairs in such a way that a general-purpose component, like a web browser, could work with it (ie, definitions of HTML forms). It got the job done.
It's usually good to structure your response data the same as what you'd expect the input of the corresponding POST, PUT, and PATCH endpoints to be. This allows record alteration to not require the consuming entity to transform the data first. So in that context, arrays of objects with "name"/"value" fields is much easier to write input validation for.

Pact Consumer / Provider based in data type and not in data value

We are currently using Pact-Broker in our Spring Boot application with really good results for our integration tests.
Our tests using Pact-Broker are base in a call to a REST API and comparing the response with the value in our provider, always using JSON format.
Our problem is that the values to compare are in a DB where the data is changing quite often, which make us update the tests really often.
Do you know if it is possible to just validate by the data type?
What we would like to try is to validate that the JSON is properly formed and the data type match, for example, if our REST API gives this output:
[
{
"action": "VIEW",
"id": 1,
"module": "A",
"section": "pendingList",
"state": null
},
{
"action": "VIEW",
"id": 2,
"module": "B",
"section": "finished",
"state": null
}
}
]
For example, what we would like to validate from the previous output is the following:
The JSON is well formed.
All the keys / value pair exists based in the model.
The value match a specific data type, for example, that the key action exist in all the entries and contains a string data type.
Do you know if this is possible to be accomplished with Pact-Broker? I was searching in the documentation but I did not found any example of how to do it.
Thanks a lot in advance.
Best regards.
Absolutely! The first 2 things Pact will always do without any extra work.
What you are talking about is referred to as flexible matching [1]. You don't want to match the value, but the type (or a regex). Given you are using Spring Boot, you may want to look at the various matchers available for Pact JVM [2].
I'm not sure if you meant it, but just for clarity, Pact and Pact Broker are separate things. Pact is the Open Source contract-testing framework, and Pact Broker [3] is a tool to help share and collaborate on those contracts with the team.
[1] https://docs.pact.io/getting_started/matching
[2] https://github.com/DiUS/pact-jvm/tree/master/consumer/pact-jvm-consumer#dsl-matching-methods
[3] https://github.com/pact-foundation/pact_broker/

Structuring json data in GET call query parameters

I'm trying to pass a list of the following objects as query params to a GET call to my Java service:
{
"id": "123456",
"country": "US",
"locale": "en_us"
}
As a url, this would like like
GET endpoint.com/entity?id1=123456&country1=US&locale1=en_us&id2=...
What's the best way to handle this as a service? If I'm passing potentially 15 of these objects, is there a concise way to take in these parameters and convert them to Java objects on the server side?
I imagine with a URL like this, the service controller would have a lot of #QueryParams...
Create the entire dataset as JSON array, e.g.
[
{
"id": "123456",
"country": "US",
"locale": "en_us"
},
{
"id": "7890",
"country": "UK",
"locale": "en_gb"
}
]
base64 encode it and pass it as a parameter, e.g.
GET endpoint.com/entity?set=BASE64_ENCODED_DATASET
then decode on the server and parse the JSON array into Java objects using perhaps Spring Boot.
Based on the valid URL size comment (although 2000 is usable), you could put the data in a header instead, which can be from 8-16kb depending on the server. GETting multiple resources at once is going to involve compromise somewhere in the design.
As Base64 can contain +/= you can url encode it too although I haven't found the need to do this in practice when using this technique in SAML.
Another approach would be to compromise on searching via country and locale specific IDs:
GET endpoint.com/entity/{country}/{locale}/{id_csv}
so you would search like this:
GET endpoint.com/entity/US/en_us/123456,0349,23421
your backend handles (if using Spring) as #PathParam for {country} and {locale} and it splits {id_csv} to get the list of IDs for that country/locale combination.
To get another country/locale search:
GET endpoint.com/entity/UK/en_gb/7890,234,123232
URLs are much smaller but you can't query the entire dataset in one go as you need to query based on country/locale each time.
It looks like your GET is getting multiple resources from the server. I'd consider refactoring to GET 1 resource from the server per GET request. If this causes performance issues, consider using HTTP caching.

Return a field as object or as primitive type in JSON in a REST API?

Currently I'm working on a REST API with an object that has a status. Should I return the status as a string or as an object?
When is it smart to change from field being a primitive type to a field being an object?
[
{
"id": 1
"name": "Hello"
"status": "active"
},
{
"id": 1
"name": "Hello"
"status": {
"id": 0
"name": "active"
}
}
]
In terms of extensibility I would suggest going for and object.
Using an object also adds the advantage of being able to split responsibility in terms of identifying (via f.e. an id field) and describing (via f.e. a name or description field), in your case, a status.
Adding i18n as a possible necessity, an object would also have to carry a string as identifier.
All these things are not possible with simple primitives. Conclusion: go for an object.
Other interesting remarks are given here.
It depends on what you need to pass.
If you only want to distinguish between different states and have all other related information (strings, translations, images) on the client either way, you might only want to send a simple integer value and use an enum on the client side. This reduces the data to the smallest amount.
If you have data that changes within one status on the server side, you need an object to pass everything else.
But best practice here would be to reduce data as much as possible.

Explaining JSON (structure) .. to a business user

Suppose you have some data you would want business users to contribute to, which will end up being represented as JSON. Data represents a piece of business logic your program knows how to handle.
As expected, JSON has nested sections, data has categorizations, some custom rules may optionally be introduced etc.
It so happens that you already a vision of what "a perfect" JSON should look like. That JSON is your starting point.
Question:
Is there a way one can take a (reasonably complex) JSON and present it in a (non-JSON) format, that would be easy for a non-technical person to understand?
If possible, could you provide an example?
What do you think of this?
http://www.codeproject.com/script/Articles/ArticleVersion.aspx?aid=90357&av=126401
Or, make your own using Ext JS for the visualization part. After all, JSON is a lingua franca on the web these days.
Apart from that, you could use XML instead of JSON, given that there are more "wizard" type tools for XML.
And finally, if when you say "business users" you mean "people who are going to laugh at you when you show them code," you should stop thinking about this as "How do I make people in suits edit JSON" and start thinking about it as "How do I make a GUI that makes sense to people, and I'll make it spit out JSON later."
Show them as key, value pairs. If your value has sub sections then show them as drill downs/tree structure. An HTML mockup which parses a JSON object in your system would help in the understanding.
Picked this example from JSON site
{
"name": "Jack (\"Bee\") Nimble",
"format": {
"type": "rect",
"width": 1920,
"height": 1080,
"interlace": false,
"frame rate": 24
}
}
Name,format would be the tree nodes.