Accessing nedb results - json

The question is: how to access multiple documents returned from a nedb 'find' command.
Understanding some insight I gained using nedb in nodejs.
Regarding docs returned via nedb find and accessing key value pair:
Thought this would work: docs.current.temp_f but it doesn't.
This works: docs[0].current.temp_f does work.
Background and detail:
I'm new to JavaScript, nodejs, nedb, and have never used mongodb, but understand nedb has a similar API.
The reason for my post is: I use this site a lot, but could not find an answer to my problem. I finally figured it out myself, and am posting it here so I can return a contribution to the community.
I'm accessing Wunderground using their API. Works great. I then pushed the results into nedb using the nedb API's 'insert' command. There are multiple examples of this available, here and elsewhere.
Using the nedb API's find command, I was able to retrieve my data. I could print it out using console.log(). Again, multiple examples here and elsewhere for this.
Wunderground returns a JSON object (if requested in this format), in the form {response:{...some data}, current_observation:{...some data...temp_f...some more data}}, which can be inserted into the nedb database.
Using the nedb API 'find' command (e.g.,
db.find({},function(err,docs) {console.log(docs);} I was able to print out my data. But my desire was to just obtain a few key:value pairs, not the whole structure. I thought that something like docs.current_observation.temp_f would do the trick, but this was always 'undefined' and caused an error.
What I didn't understand, because I'm so new in this area, is that nedb is returning a "set" of data; an array of JSON objects in this case, since I was inserting JSON objects. I never saw this mentioned anywhere on the npmjs.com site for nedb. So I wanted to alert others like myself that lacked this knowledge.
Here's what ultimately worked to obtain the key:value information:
console.log("temp_f: ", docs[0].current_observation.temp_f);
where I'm accessing the array element 0.
Of course, this would probably end up in something like a for/in loop.
I would be interested in seeing any additional ideas on retrieving key:values from the result set returned, if you know of any.
//Thought this would work, but it doesn't
docs.current_observation.temp_f
//but this does, where x is a value to record
docs[x].current_observation.temp_f

Simple answer is use mongo's query language with find method to return an array of matching documents then manipulate the array of objects to get the values you need. Use for of when iterating arrays not for in. Alternatively use map, filter, forEach and reduce array methods for manipulating array data.

Related

Jenkins API xpath like functionality for JSON

I am trying to use the jenkins API to retrieve a list of running jobs buildURLs and this works with this the query
https://jenkins.server.com/computer/api/xml?tree=computer[executors[currentExecutable[url]]]&depth=1&xpath=//url&wrapper=buildUrls
By searching for all executors on given jenkins server and then grabbing the urls and wrapping them in a xml buildUrls object
(what I actually want is a count of total running jobs but I can filter in api then get a .size of this once client side)
However the application I am uses only accepts json, although I could convert to json I am wondering if there is a way I can have this return a json object which contains just the buildUrls. Currently if I change return to json the xpath modification is removed (since its xml only)
And I instead get returned the list of all executors and their status
I think my answer lies within Tree but i cant seem to get the result I want
To return a JSON object you can modify the URL as below
http://jenkins.server.com/computer/api/json?tree=computer[executors[currentExecutable[url]]]&depth=1&pretty=true
It will not be possible to get only the build urls you need to iterate through all the executables to get the total running jobs. The reason is that build urls alone cannot give you information where it is running. Hence, unfortunately we need to iterate through each executor.
The output you will get is something like below.
Another way of doing this same thing is parsing via jobs
http://jenkins.server.com/api/json?tree=jobs[name,url,lastBuild[building,timestamp]]&pretty=true
You can check for building parameter. But here too you will not be able to filter output from url diretly in jenkins. Iteration would be required through each and every job.

What are developers their expectations when receiving a JSON response from a server

I have java library that runs webservices and these return a response in XML. The webservices all revolve around giving a list of details about items. Recently, changes were made to allow the services to return JSON by simply converting the XML to JSON. When looking at the responses, I saw they're not as easy to parse as I thought. For example, a webservice that returns details about items.
If there are no items, the returned JSON is as follows:
{"ItemResponse":""}
If there is 1 item, the response is as follows (now itemResponse has a object as value instead of a string):
{"ItemResponse":{"Items":{"Name":"Item1","Cost":"$5"}}}
If there two or more items, the response is (now items has an array as value instead of an object):
{"ItemResponse":{"Items":[{"Name":"Item1","Cost":"$5"},{"Name":"Item2","Cost":"$3"}]}}
To parse these you need several if/else which I think are clunky.
Would it be an improvement if the responses were:
0 items: []
1 item: [{"Name":"Item1","Cost":"$5"}]
2 items: [{"Name":"Item1","Cost":"$5"},{"Name":"Item2","Cost":"$3"}]
This way there is always an array, and it contains the itemdata. An extra wrapper object is possible:
0 items: {"Items":[]}
1 item: {"Items":[{"Name":"Item1","Cost":"$5"}]}
2 items: {"Items":[{"Name":"Item1","Cost":"$5"},{"Name":"Item2","Cost":"$3"}]}
I'm not experienced in JSON so my question is, if you were a developer having to use these webservices, how would you expect the JSON resonse to be formatted? Is it better to always return a consistent array, even if there are no items or is this usually not important? Or is an array not enough and do you really expect a wrapper object around the array?
What are conventions/standards regarding this?
Don't switch result types, always return an array if there are more items possible. Do not mix, for 1 item an object for more an array. That's not a good idea.
Another best practise is that you should version your API. Use something like yoursite.com/api/v1/endpoint. If you don't do this and you change the response of your API. All your client apps will break. So keep this in mind together with documentation. (I've seen this happen a lot in the past..)
As a developer I personally like your second approach, but again it's a preference. There is no standard for this.
There are several reasons to use json:
much more dense and compact: thus data sent is less
in javascript you can directly access those properties without parsing anything. this means you could convert it into an object read the attributes (often used for AJAX)
also in java you usually don't need to parse the json by yourself - there are several nice libs like www.json.org/java/index.html
if you need to know how json is build ... use google ... there tons of infos.
To your actual questions:
for webservices you often could choose between xml and json as a "consumer" try:
https://maps.googleapis.com/maps/api/place/textsearch/json
and
https://maps.googleapis.com/maps/api/place/textsearch/xml
there is no need to format json visually - is it not meant for reading like xml
if your response doesn't have a result, json-service often still is giving a response text - look again at the upper google map links - those are including a response status which makes sense as it is a service.
Nevertheless it's the question if it is worth converting from xml to json if there isn't a specific requirement. As Dieter mentioned: it depends on who is already using this service and how they are consumed ... which means the surrounding environment is very important.

Debugging json4s read deserialization errors

I am attempting to consume an API that I do not have control over which is somewhat poorly documentented and somewhat inconsistent. This means that sometimes, the API returns a different type than what is documented or what you would normally see. For this example, we'll look at a case when an array was returned in a place where I would normally see a string. That makes a crappy API, but my real problem is: How can I more easily track those things down? Right now, the errors look something like this:
No usable value for identifier
Do not know how to convert JArray(List(JString(3c8723eceb1a), JString(cba8849e7a2f))) into class java.lang.String
After deciphering the problem (why JValue::toString doesn't emit a JSON string is utterly perplexing to me), I can figure out the API returned an array when I made my case class only able to deal with Strings. Great. My issue is that finding this discrepancy between my object model and the contents of the JSON seems significantly more difficult than it should be.
Currently, this is my workflow for hunting down decoding errors:
Hope bad data has some sort of identifying marker. If this is not true, then it is way more guesswork and you will have to repeat the following steps for each entry that looks like the bad bits.
Go through the troubles of converting the JArray(List(JString(...), ...)) from the error message into valid JSON, hoping that I encode JSON the same way at the API endpoint I got the data from does. If this is not true, then I use a JSON formatter (jq) to format all data consistently.
Locate the place in the source data where the decoding error originates from.
Backtrack through arrays and objects to discover how I need to change my object model to more accurately represent what data is coming back to me from the API.
Some background: I'm coming from C++, where I rolled my own JSON deserialization framework for this purpose. The equivalent error when using the library I built is:
Error decoding value at result.taskInstances[914].subtasks[5].identifier: expected std::string but found array value (["3c8723eceb1a","cba8849e7a2f"]) at 1:4084564
This is my process when using my hand-rolled library:
Look at the expected type (std::string) compared with the data that was actually found (["3c8723eceb1a","cba8849e7a2f"]) and alter my data model for the path for the data in the source (result.taskInstances[914].subtasks[5].identifier)
As you can see, I get to jump immediately to the problem that I actually have.
My question is: Is there a way to more quickly debug inconsistencies between my data model and the results I'm getting back from the API?
I'm using json4s-native_2.10 version 3.2.8.
A simplified example:
{ "property": ["3c8723eceb1a", "cba8849e7a2f"] }
Does not mesh with Scala class:
case class Thing(property: String)
The best solution would be to use Try http://www.scala-lang.org/api/current/#scala.util.Try in Scala, but unfortunately json4s API cannot.
So, I think you should use Scala Option type http://www.scala-lang.org/api/current/#scala.Option .
In Scala, and more generally in functional languages, Options are used to represent an object that can be there or not (like à nil value).
For handle parsing failures, you can use parse(str).toOption, which is a function that return an Option[JValue], and you can doing a pattern matching on the resulting value.
For handling extraction of data extraction into case classes, you can use extractOpt function, to do pattern matching on the value.
You can read this answer : https://stackoverflow.com/a/15944506/2330361

Export entire Logical Data Model from GoodData

In this https://developer.gooddata.com/article/data-modeling-api there is a logical data model and its corresponding JSON. However, I can't seem to find out how to extract JSON from a logical data model via the REST API. Is there a way to do this other than using the single load interface (which would be very inefficient)?
For the record, my end goal is to make a tool that extracts that JSON (which would be in dev), then post that to the ldm manager2, and then apply the suggested changes through the returned MaQL to production. Any help is greatly appreciated
Currently this works only for Getting or Updating the entire Project. Anyway you can GET all model definition by simple API call. See the documentation:
http://docs.gooddatadrafts.apiary.io/
There is a GET request which is asynchronous. You can build some logic on the top of that on your end. You can get all models, store per datasets information, but at the end you need to POST the "final version" and all updates will be applied.
Let me know if I can help you with anything!
Regards,
JT

questions wcf rest service with webclient

I'm getting confused on a few things in regards to wcf rest.
If you call a login method, should I use a POST or GET? After implementing a POST, I started to find various articles saying you should only use post to update data, and get for retrieving data. Which is the most appropriate method?
If I had to change the Login method from a Post to a Get, how would I call this?
http://....myservice.svc/login/{username}/{passpord} or is there another way to call this?
Note that in my post method, I'm passing and returning data in json format.
I need to create a search function that requires to pass various parameters i.e. list, string, list, etc... I assume in this instance I would have to define GET method, but again how to I pass these list of objects? Convert them to json first and pass them as parameters?
A brief url sample would be great.
Ok, I guess I'll answer my own question based on further finding when researching it and remember that my answer is based on using JSON as parameters. I'm not sure how it would behave if xml was used as I did not try it.
It appears to make more sense to use POST when logging in as you do not want to display the information you are posting via a url. You could encrypt the data and pass it in the url using a GET method... Again I could be wrong, but that's how I interpreted the various articles I read.
Again, in this instance, it appears POST is the best solution if a) you require a large amount of data to be passed to your url and b) if you do not want to show this data to the user. If your query only requires simple parameters (i.e. userid, type, etc...) and you don't mind showing this info, you can use the GET method.
If you require to pass multiple parameters to a function, you should instead pass a single parameter. This parameter should be a single object. This object should be made of all the parameters you wanted to use in the first place, this way, when using the POST method, this object can easily be converted to JSON and it will handle passing multiple parameters through a single object and it will handle numeric, string, list<>, array<>, etc... very nicely.