I am attempting to consume an API that I do not have control over which is somewhat poorly documentented and somewhat inconsistent. This means that sometimes, the API returns a different type than what is documented or what you would normally see. For this example, we'll look at a case when an array was returned in a place where I would normally see a string. That makes a crappy API, but my real problem is: How can I more easily track those things down? Right now, the errors look something like this:
No usable value for identifier
Do not know how to convert JArray(List(JString(3c8723eceb1a), JString(cba8849e7a2f))) into class java.lang.String
After deciphering the problem (why JValue::toString doesn't emit a JSON string is utterly perplexing to me), I can figure out the API returned an array when I made my case class only able to deal with Strings. Great. My issue is that finding this discrepancy between my object model and the contents of the JSON seems significantly more difficult than it should be.
Currently, this is my workflow for hunting down decoding errors:
Hope bad data has some sort of identifying marker. If this is not true, then it is way more guesswork and you will have to repeat the following steps for each entry that looks like the bad bits.
Go through the troubles of converting the JArray(List(JString(...), ...)) from the error message into valid JSON, hoping that I encode JSON the same way at the API endpoint I got the data from does. If this is not true, then I use a JSON formatter (jq) to format all data consistently.
Locate the place in the source data where the decoding error originates from.
Backtrack through arrays and objects to discover how I need to change my object model to more accurately represent what data is coming back to me from the API.
Some background: I'm coming from C++, where I rolled my own JSON deserialization framework for this purpose. The equivalent error when using the library I built is:
Error decoding value at result.taskInstances[914].subtasks[5].identifier: expected std::string but found array value (["3c8723eceb1a","cba8849e7a2f"]) at 1:4084564
This is my process when using my hand-rolled library:
Look at the expected type (std::string) compared with the data that was actually found (["3c8723eceb1a","cba8849e7a2f"]) and alter my data model for the path for the data in the source (result.taskInstances[914].subtasks[5].identifier)
As you can see, I get to jump immediately to the problem that I actually have.
My question is: Is there a way to more quickly debug inconsistencies between my data model and the results I'm getting back from the API?
I'm using json4s-native_2.10 version 3.2.8.
A simplified example:
{ "property": ["3c8723eceb1a", "cba8849e7a2f"] }
Does not mesh with Scala class:
case class Thing(property: String)
The best solution would be to use Try http://www.scala-lang.org/api/current/#scala.util.Try in Scala, but unfortunately json4s API cannot.
So, I think you should use Scala Option type http://www.scala-lang.org/api/current/#scala.Option .
In Scala, and more generally in functional languages, Options are used to represent an object that can be there or not (like à nil value).
For handle parsing failures, you can use parse(str).toOption, which is a function that return an Option[JValue], and you can doing a pattern matching on the resulting value.
For handling extraction of data extraction into case classes, you can use extractOpt function, to do pattern matching on the value.
You can read this answer : https://stackoverflow.com/a/15944506/2330361
Related
Chrome Developer Console is console.loging this:
Your JSON sent is>> {yourVariable: "nothing yet"}
So i know the value "nothing yet" in the {yourVariable: "nothing yet"} JSON object is a string. But how do I know the type of the key yourVariable?
Is there a way how to find that out using the Chrome console only?
All object keys are strings with quotes or without quotes. Try this way to see it. May be you are confused with console print because console print it without quotes and we usually write with quotes.
var jsonObj = {person:"me","age":"30", 123:"123"};
Object.keys(jsonObj).forEach(function(key){
console.log(typeof key)}
);
I have a slightly different take on this question which does not invalidate the comments or answer that you received, but is worth considering.
Because you are talking about JSON, there is no intrinsic datatype in your example. As stated on the JSON.org page:
JSON (JavaScript Object Notation) is a lightweight data-interchange
format. It is easy for humans to read and write. It is easy for
machines to parse and generate.
The point is, that there is a difference between JSON, which is a representation of javascript objects, arrays etc., and variables of those types in javascript.
If you remind yourself that JSON is a form of serialization it makes a lot more sense. Javascript objects for example, can include functions, but a javascript function is not a portable thing, so in rendering some JSON from a javascript object, it is up to the language creating the JSON to do whatever it it needs to in order to convert the data it needs to represent, and that may include simplification and in many cases removal of elements that are incompatible JSON.
The other thing to keep in mind, is that all modern languages have functions or libraries that can parse JSON and turn them into variables or objects that work in those languages. In doing so, they have arguments that can completely change the way the JSON is converted back into instance variables.
For example, in PHP you can choose to have the JSON create one or more PHP objects, or an array of php variables.
In summary, JSON doesn't have variables with datatypes at all. It is a representation of data, that is portable across languages, but the languages must decode the JSON and create objects or variables that are valid in their own runtime.
Is it advisable or not in a RESTful web service to use JSON literal values (string / number) as input parameter in the payload or in the response body?
If I have an endpoint PUT /mytodolist is it OK for it to accept a JSON string literal value "Take out the rubbish" in the request payload (with Content-Type=application/json) or should it accept a JSON object instead ({"value":"Take out the rubbish"})?
Similarly, is it fine for GET /mytodolist/1 to return "Take out the rubbish" in the response body or should it return a proper JSON object {"value":"Take out the rubbish"}
Spring MVC to makes implementing and testing such endpoints easy, however clients have flagged this as non standard or hard to implement. In my point of view JSON literals are JSON, but not JSON objects, so I'd say it is fine. I have found no recommendations using Google.
EDIT 1: Clafirication
The question is entirely about the 'standard', if it allows this or not.
I understand the problem with the extensibility, but one can never design a fully extensible interface IMHO. If changes need to be done, we can try extending what we have in a backwards compatible way, but there will come a time when it becomes messy and an other approach is required - which is commonly handled by versioning the API in one way or another. I find it a fair point even though, because using literals as request/response body immediately becomes inextensible, while coming up with a reasonable one-attribute JSON object does not.
It is also understood that some frameworks have problems with handling JSON literals, this is the origin of this question. The tool I used happened to support this, so I thought this was all right, but the front-end library did not.
Still, what I am intending to find out right now, is if using JSON literals is according to the de-facto standard (even if it is a cornercase) or not.
I would recommend to use JSON object always. One reason is that for Content-Type application/json people expect something staring with "{" and not all frameworks will handle json literals properly. Second reason is that probably you will add some additional attributes to you list item (due date, category, priority, etc). And then you'll break backward compatibility, by adding new field.
It may be acceptable in the context of your example, but keep in mind that unambiguous interfaces are easier to use and that will encourage adoption.
For example, your interface could interpret "Take out he rubbish" as the same as {task:"take out the rubbish"}, but once you add additional properties (eg "when" or "who") the meaning of a solitary string in the request becomes ambiguous. It's inevitable that you'll add support for new properties as your interface matures.
I have java library that runs webservices and these return a response in XML. The webservices all revolve around giving a list of details about items. Recently, changes were made to allow the services to return JSON by simply converting the XML to JSON. When looking at the responses, I saw they're not as easy to parse as I thought. For example, a webservice that returns details about items.
If there are no items, the returned JSON is as follows:
{"ItemResponse":""}
If there is 1 item, the response is as follows (now itemResponse has a object as value instead of a string):
{"ItemResponse":{"Items":{"Name":"Item1","Cost":"$5"}}}
If there two or more items, the response is (now items has an array as value instead of an object):
{"ItemResponse":{"Items":[{"Name":"Item1","Cost":"$5"},{"Name":"Item2","Cost":"$3"}]}}
To parse these you need several if/else which I think are clunky.
Would it be an improvement if the responses were:
0 items: []
1 item: [{"Name":"Item1","Cost":"$5"}]
2 items: [{"Name":"Item1","Cost":"$5"},{"Name":"Item2","Cost":"$3"}]
This way there is always an array, and it contains the itemdata. An extra wrapper object is possible:
0 items: {"Items":[]}
1 item: {"Items":[{"Name":"Item1","Cost":"$5"}]}
2 items: {"Items":[{"Name":"Item1","Cost":"$5"},{"Name":"Item2","Cost":"$3"}]}
I'm not experienced in JSON so my question is, if you were a developer having to use these webservices, how would you expect the JSON resonse to be formatted? Is it better to always return a consistent array, even if there are no items or is this usually not important? Or is an array not enough and do you really expect a wrapper object around the array?
What are conventions/standards regarding this?
Don't switch result types, always return an array if there are more items possible. Do not mix, for 1 item an object for more an array. That's not a good idea.
Another best practise is that you should version your API. Use something like yoursite.com/api/v1/endpoint. If you don't do this and you change the response of your API. All your client apps will break. So keep this in mind together with documentation. (I've seen this happen a lot in the past..)
As a developer I personally like your second approach, but again it's a preference. There is no standard for this.
There are several reasons to use json:
much more dense and compact: thus data sent is less
in javascript you can directly access those properties without parsing anything. this means you could convert it into an object read the attributes (often used for AJAX)
also in java you usually don't need to parse the json by yourself - there are several nice libs like www.json.org/java/index.html
if you need to know how json is build ... use google ... there tons of infos.
To your actual questions:
for webservices you often could choose between xml and json as a "consumer" try:
https://maps.googleapis.com/maps/api/place/textsearch/json
and
https://maps.googleapis.com/maps/api/place/textsearch/xml
there is no need to format json visually - is it not meant for reading like xml
if your response doesn't have a result, json-service often still is giving a response text - look again at the upper google map links - those are including a response status which makes sense as it is a service.
Nevertheless it's the question if it is worth converting from xml to json if there isn't a specific requirement. As Dieter mentioned: it depends on who is already using this service and how they are consumed ... which means the surrounding environment is very important.
I'm using RestSharp. I was using XML as the transport encoding for my data, but then I had problems with that, so I switched to using JSON. Now I'm having problems with that too!
All was well until I tried to pass an object containing a byte array (*). Now I get a de-serialization error complaining about "object has no parameterless constructor". (The JSON returned by the server looks Kosher - it's just not being correctly de-serialized by RestSharp).
I see I'm not the only one having problems. Is there no solution other than the baby-out-with-the-bathwater approach suggested in that post?
(*) I had tested it with a small hand-coded byte array early on in my development, just to check that it worked. It did work then, but doesn't work now. I don't know if that's due to the size of the array, the "characters" in the array, or what. Dammit, this has been such a time-sink!
For anyone else struggling with this, I ended up simply swapping out the JSON formatter in favour of JSON.NET. That works.
How can one check to see if a string is properly formatted in json? I am running into situations where I receive a partial json string, and then when trying to decode the erroneous string with Lua's json.decode, my application crashes.
Thanks
Note that there is a difference between crash and crash in Lua.
If you see segfaults, throw away your JSON library and use something else (there are plenty).
If you see a Lua error, just wrap JSON processing code in pcall or xpcall. (But, better, again, throw away your library, and pick a better one).