It might be a frequent question but i cannot figure out how to prevent errors in my parsing when the script can't find a property...
in XML was easy because even the empty properties were like <location/>
but now if location is not available JSON paser cant find it and it results in errors...OR
it may happen the json has different property or a children lost its father..... so for instance if you need to extract the LocalityName is no more under SubAdministrativeArea but under AddressLine...
any of you have any experience about? what the best way to solve it and to parse it correctly?
While answering one of your other question I had written the following javascript code to obtain the lat and lng from the JSON returned by the maps api without any validations for zero results.
$.getJSON("getjson.php?address="+address,
function(data){
lat=data.results[0].geometry.location.lat;
lng=data.results[0].geometry.location.lng;
//.....map initialization code
}
);
Now if I were to validate for zero results, I'd modify the code in the following way
$.getJSON("getjson.php?address="+address,
function(data){
if (data.result.length>0) {
for (count=0;count<data.result.length;count++){
lat=data.results[count].geometry.location.lat;
lng=data.results[count].geometry.location.lng;
//.....map initialization code
}
}
}
);
As you can see parsing JSON comes naturally to javascript and to many other languages for it resolves down to arrays/lists and objects/dictionary/hash.
If I get this right and you are using a library to convert to json like gson try to construct some kind of object array like arrayList in java and then convert to json so every object you retrieve later in javascript is distinct and thus clear during debugging.Also if you don't use firebug give it try as it shows json data clearly.cheers
Related
I am using Blueprism to make an API Call. The response is a block of json. I need to extract the conversationId from:
{"results":[{"group":{"queueId":"aad701ad-56db-452e-8b70-aa9abd6046c7","mediaType":"email"},"data":[{"metric":"oWaiting","stats":{"count":1},"truncated":false,"observations":[{"observationDate":"2022-01-20T11:19:04.882Z","conversationId":"116b9f91-bf82-4275-9cdc-c405068b4cba","sessionId":"f97de11e-eb99-4781-ae13-33a9e5b6c3f0","routingPriority":0,"direction":"inbound","addressFrom":"e.mc#gmail.ie","addressTo":"info#gmail.ie","requestedRoutings":["Standard"]}]}]}]}
I am using this regex:
^.*? with conversationId ([a-f0-9]+)
but it is not working. Is this the best approach? Is there a better way to do this?
.*conversationId":"([^"]+)".*
will save the conversationId into capture group 1.
If you are using perl, you could do this:
s/.*conversationId":"([^"]+)".*/\1/
this works for your example but it's probably not going to scale well to different input JSON messages. As others have mentioned, the right way to do this is to parse the string into a native JSON object and then extract the field using it's methods.
Ultimately I was going about this the wrong way. I should have been parsing the json and not trying to extract a particular string.
Blueprism has a json object which completes this very function.
https://digitalexchange.blueprism.com/dx/entry/3439/solution/utility---json
I have java library that runs webservices and these return a response in XML. The webservices all revolve around giving a list of details about items. Recently, changes were made to allow the services to return JSON by simply converting the XML to JSON. When looking at the responses, I saw they're not as easy to parse as I thought. For example, a webservice that returns details about items.
If there are no items, the returned JSON is as follows:
{"ItemResponse":""}
If there is 1 item, the response is as follows (now itemResponse has a object as value instead of a string):
{"ItemResponse":{"Items":{"Name":"Item1","Cost":"$5"}}}
If there two or more items, the response is (now items has an array as value instead of an object):
{"ItemResponse":{"Items":[{"Name":"Item1","Cost":"$5"},{"Name":"Item2","Cost":"$3"}]}}
To parse these you need several if/else which I think are clunky.
Would it be an improvement if the responses were:
0 items: []
1 item: [{"Name":"Item1","Cost":"$5"}]
2 items: [{"Name":"Item1","Cost":"$5"},{"Name":"Item2","Cost":"$3"}]
This way there is always an array, and it contains the itemdata. An extra wrapper object is possible:
0 items: {"Items":[]}
1 item: {"Items":[{"Name":"Item1","Cost":"$5"}]}
2 items: {"Items":[{"Name":"Item1","Cost":"$5"},{"Name":"Item2","Cost":"$3"}]}
I'm not experienced in JSON so my question is, if you were a developer having to use these webservices, how would you expect the JSON resonse to be formatted? Is it better to always return a consistent array, even if there are no items or is this usually not important? Or is an array not enough and do you really expect a wrapper object around the array?
What are conventions/standards regarding this?
Don't switch result types, always return an array if there are more items possible. Do not mix, for 1 item an object for more an array. That's not a good idea.
Another best practise is that you should version your API. Use something like yoursite.com/api/v1/endpoint. If you don't do this and you change the response of your API. All your client apps will break. So keep this in mind together with documentation. (I've seen this happen a lot in the past..)
As a developer I personally like your second approach, but again it's a preference. There is no standard for this.
There are several reasons to use json:
much more dense and compact: thus data sent is less
in javascript you can directly access those properties without parsing anything. this means you could convert it into an object read the attributes (often used for AJAX)
also in java you usually don't need to parse the json by yourself - there are several nice libs like www.json.org/java/index.html
if you need to know how json is build ... use google ... there tons of infos.
To your actual questions:
for webservices you often could choose between xml and json as a "consumer" try:
https://maps.googleapis.com/maps/api/place/textsearch/json
and
https://maps.googleapis.com/maps/api/place/textsearch/xml
there is no need to format json visually - is it not meant for reading like xml
if your response doesn't have a result, json-service often still is giving a response text - look again at the upper google map links - those are including a response status which makes sense as it is a service.
Nevertheless it's the question if it is worth converting from xml to json if there isn't a specific requirement. As Dieter mentioned: it depends on who is already using this service and how they are consumed ... which means the surrounding environment is very important.
I am attempting to consume an API that I do not have control over which is somewhat poorly documentented and somewhat inconsistent. This means that sometimes, the API returns a different type than what is documented or what you would normally see. For this example, we'll look at a case when an array was returned in a place where I would normally see a string. That makes a crappy API, but my real problem is: How can I more easily track those things down? Right now, the errors look something like this:
No usable value for identifier
Do not know how to convert JArray(List(JString(3c8723eceb1a), JString(cba8849e7a2f))) into class java.lang.String
After deciphering the problem (why JValue::toString doesn't emit a JSON string is utterly perplexing to me), I can figure out the API returned an array when I made my case class only able to deal with Strings. Great. My issue is that finding this discrepancy between my object model and the contents of the JSON seems significantly more difficult than it should be.
Currently, this is my workflow for hunting down decoding errors:
Hope bad data has some sort of identifying marker. If this is not true, then it is way more guesswork and you will have to repeat the following steps for each entry that looks like the bad bits.
Go through the troubles of converting the JArray(List(JString(...), ...)) from the error message into valid JSON, hoping that I encode JSON the same way at the API endpoint I got the data from does. If this is not true, then I use a JSON formatter (jq) to format all data consistently.
Locate the place in the source data where the decoding error originates from.
Backtrack through arrays and objects to discover how I need to change my object model to more accurately represent what data is coming back to me from the API.
Some background: I'm coming from C++, where I rolled my own JSON deserialization framework for this purpose. The equivalent error when using the library I built is:
Error decoding value at result.taskInstances[914].subtasks[5].identifier: expected std::string but found array value (["3c8723eceb1a","cba8849e7a2f"]) at 1:4084564
This is my process when using my hand-rolled library:
Look at the expected type (std::string) compared with the data that was actually found (["3c8723eceb1a","cba8849e7a2f"]) and alter my data model for the path for the data in the source (result.taskInstances[914].subtasks[5].identifier)
As you can see, I get to jump immediately to the problem that I actually have.
My question is: Is there a way to more quickly debug inconsistencies between my data model and the results I'm getting back from the API?
I'm using json4s-native_2.10 version 3.2.8.
A simplified example:
{ "property": ["3c8723eceb1a", "cba8849e7a2f"] }
Does not mesh with Scala class:
case class Thing(property: String)
The best solution would be to use Try http://www.scala-lang.org/api/current/#scala.util.Try in Scala, but unfortunately json4s API cannot.
So, I think you should use Scala Option type http://www.scala-lang.org/api/current/#scala.Option .
In Scala, and more generally in functional languages, Options are used to represent an object that can be there or not (like à nil value).
For handle parsing failures, you can use parse(str).toOption, which is a function that return an Option[JValue], and you can doing a pattern matching on the resulting value.
For handling extraction of data extraction into case classes, you can use extractOpt function, to do pattern matching on the value.
You can read this answer : https://stackoverflow.com/a/15944506/2330361
I am loading data from a mongodb collection to a mysql table through Kettle transformation.
First I extract them using MongodbInput and then I use json input step.
But since json input step has very low performance, I wanted to replace it with a
javacript script.
I am a beginner in Javascript and even though i tried somethings, the kettle javascript script is not recognizing any keywords.
can anyone give me sample code to convert Json data to different columns using javascript?
To solve your problem you need to see three aspects:
Reading from MongoDB
Reading from JSON
Reading from (probably) String
Reading from MongoDB Except if you changed the interface, MongoDB returns not JSON but BSON files (~binary JSON). You need to see the MongoDB documentation about reading and writing BSON: probably something like BSON.to() and BSON.from() but I don't know it by heart.
Reading from JSON Once you have your BSON in JSON format, you can read it using JSON.stringify() which returns a String.
Reading from (probably) String If you want to use the capabilities of JSON (why else would you use JSON?), you also want to use JSON.parse() which returns a JSON object.
My experience is that to send a JSON object from one step to the other, using a String is not a bad idea, i.e. at the end of a JavaScript step, you write your JSON object to a String and at the beginning of the next JavaScript step (can be further down the stream) you parse it back to JSON to work with it.
I hope this answers your question.
PS: writing JavaScript steps requires you to learn JavaScript. You don't have to be a master, but the basics are required. There is no way around it.
you could use the json input step to get the values of this json and put in common rows
I am using jQuery.get/post methods in my Struts2 application.
See the following code. This URL return "success" and "error". It will hit a method and return a string (the struts2). I have made JSPs that contatin these strings in them.
But I want to change this approach. If I have to return a list of objects. What I will do. How a method that return a list of objects will be handeled in jQuery. How JSON will help me? I need a pointer on it.
I hope my point is understood.
jQuery.post("register_user.action" , jQuery("#user_form").serialize(),
function(data){
if (data == "success"){
jQuery("#success").dialog("open");
}else if (data == "error"){
jQuery("#error").dialog("open");
}
});
It's difficult to answer the question "How will JSON help you?" without specific context of what you are doing but I can offer up the generic answer. JSON stands for javascript object notation. It has many uses but relative to your question, it allows you to send a serialized representation of your data from the server down to your client without the "bloat" that comes with XML. Another advantage of using JSON over XML is modern browsers have built in support for it so you don't have to worry about parsing data.
You access the data like properties of a class.
Taking your example, you could return an object (I typically use anonymous objects in scenarios like this) with a single BOOL property called 'success' serialized to JSON. You could then avoid string comparisons in your javascript. In my personal opinion, the code becomes much cleaner.
jQuery.post("register_user.action" , jQuery("#user_form").serialize(),
function(data){
if (data.success){
jQuery("#success").dialog("open");
}else{
jQuery("#error").dialog("open");
}
});
This would be the most basic of examples. JSON becomes much more powerful if you return a serialized list of data from the server to the client. Let's say you send a list of comments in JSON. You could then use jquery templating to bind that list against a template and vuala, you can quickly display that day in your format of choice without manually iterating through all items.
As I mentioned, it's hard to be to specific as I don't know exactly what you are looking to accomplish but I hope this helps.