JSON string pulled via net/http to Hash - json

I'm trying to pull JSON from a Yahoo API to get the conversion rate of USD to SEK. However, I can't seem to get the JSON converted to a Hash, it shows "query" as being the only key since JSON comes in as one string.
The JSON request returns:
{"query":{"count":1,"created":"2016-12-04T13:06:00Z","lang":"en-us","results":{"rate":{"id":"USDSEK","Name":"USD/SEK","Rate":"9.1900","Date":"12/2/2016","Time":"9:59pm","Ask":"9.2000","Bid":"9.1900"}}}}
My code is as follow:
require 'net/http'
require 'json'
url = 'https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20yahoo.finance.xchange%20where%20pair%20in%20(%22USDSEK%22)&format=json&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys&callback='
uri = URI(url)
response = Net::HTTP.get(uri)
json_hash= JSON.parse(response)
puts json_hash["Rate"]
the puts statement returns 'nil'
I've tried following an example from this site, however I do not yield the same results based on the way my data is being brought as his is being statically entered.
*Note I am not using 'ostruct', trying simply with json.
Thank you for any insight.

As you can see, the field you are looking for is into an inner hash. Try
puts json_hash["query"]["results"]["rate"]["Rate"]

Have you check the structure of your json?
{
"query": {
"count": 1,
"created": "2016-12-04T13:06:00Z",
"lang": "en-us",
"results": {
"rate": {
"id": "USDSEK",
"Name": "USD/SEK",
"Rate": "9.1900",
"Date": "12/2/2016",
"Time": "9:59pm",
"Ask": "9.2000",
"Bid": "9.1900"
}
}
}
}
To fetch the rate key you should do something like:
json_hash["query"]["results"]["rate"]
Compare that with json above to understand your problem.

Related

Extract all the JSON data using Kotlin Android Studio

I'm using Volley library to communicate with my API. I'm pretty new to Android and Kotlin and I'm really confused about extracting keys from the following JSON data
{
"message": {
"_id": "60bc7fa7abeedb25643fa692",
"hash": "3a54b415461a63abac1fc6dfa0e140584047bd15358e33a177f9505ed2faa4d4",
"blockchain": "ethereum",
"amount": 5000,
"amount_usd": 13352971,
"from": "d3d69228cb2292f933572399593617f574c70eb1",
"to": "fe9996da73d6bf5252f15024811954ae37ab68be",
"__v": 0
}
}
The volley library returns all of this JSON data in a variable called response and I'm using response.getString("message") to extract the message key but, I don't understand how to extract the internal data such as hash, blockchain, amount, etc.
I'm using the following code to get the JSON data from my backend.
val jsonRequest = JsonObjectRequest(
Request.Method.GET, url, null,
{ response ->
tweet_text.setText(response.getString("message"))
Log.d("resp", response.toString())
},
{
Log.d("err", it.localizedMessage)
})
Any help would be appreciated, Thanks!
I found it, I just used the getJSONObject() method to make it work
val jsonRequest = JsonObjectRequest(
Request.Method.GET, url, null,
{ response ->
val txn = response.getJSONObject("message")
//txn object can be used to extract the internal data
},
{
Log.d("err", it.localizedMessage)
})

How to parse nested JSON, within a string, using Kusto

I have a Python Azure Function that produces custom logging messages when the Function executes. I need to pull out some of the JSON values nested in the logging message.
How can I use Kusto to access the nested JSON within the logging message string?
Example logging message:
Desired values marked with <----------
####### EventGrid trigger processing an event:
{
"id": "long-guid",
"data": {
"api": "FlushWithClose",
"requestId": "long-guid",
"eTag": "long-guid",
"contentType": "application/octet-stream",
"contentLength": 16264, <----------------------
"contentOffset": 0,
"blobType": "BlockBlob",
"blobUrl": "https://function.blob.core.windows.net/parentdir/childdir/file.name",
"url": "https://function.dfs.core.windows.net/parentdir/childdir/file.name", <---- JUST FILE.NAME here
"sequencer": "long-guid",
"identity": "long-guid",
"storageDiagnostics": {
"batchId": "long-guid"
}
},
"topic": "/subscriptions/long-guid/resourceGroups/resourceGroup/providers/Microsoft.Storage/storageAccounts/accountName",
"subject": "/blobServices/default/containers/containerName/blobs/childDir/file.name",
"event_type": "Microsoft.Storage.BlobCreated"
} #######
I imagine it has something to do with the Kusto extend function, but piping in...
| extend parsedMessage = todynamic(message)
| project timestamp, test = parsedMessage["id"]
...yields only an empty test column
message in your specific case isn't a valid JSON payload - as it has the ###... EventGrid trigger processing an event: prefix (and a somewhat similar suffix).
That is why todynamic() isn't able to process it and why you're not able to reference properties in the JSON payload that's included in it.
Ideally, you would change the payload you ingest to be a valid JSON payload, and re-type the target column to dynamic instead of string.
If you can't do that, you can use the substring() function or parse operator to get everything but the aforementioned prefix/suffix, and parse the output of that using todynamic()
though note that doing that each time you query the data bears runtime overhead that could be avoided by following the advice above.

How to parse JsonArray in Scala and writing them in a DataFrame?

Using my Scala HTTP Client I retrieved a response in JSON format from an API GET call.
My end goal is to write this JSON content to an AWS S3 bucket in order to make it available as a table on RedShift running a simple AWS Glue crawler.
My thinking is to parse this JSON message and somehow converting into a Spark DataFrame, so later on I can save it to my preferred S3 location in the format of .csv, .parquet, or whatever.
The JSON file looks like this
{
"response": {
"status": "OK",
"start_element": 0,
"num_elements": 100,
"categories": [
{
"id": 1,
"name": "Airlines",
"is_sensitive": false,
"last_modified": "2010-03-19 17:48:36",
"requires_whitelist_on_external": false,
"requires_whitelist_on_managed": false,
"is_brand_eligible": true,
"requires_whitelist": false,
"whitelist": {
"geos": [],
"countries_and_brands": []
}
},
{
"id": 2,
"name": "Apparel",
"is_sensitive": false,
"last_modified": "2010-03-19 17:48:36",
"requires_whitelist_on_external": false,
"requires_whitelist_on_managed": false,
"is_brand_eligible": true,
"requires_whitelist": false,
"whitelist": {
"geos": [],
"countries_and_brands": []
}
}
],
"count": 148,
"dbg_info": {
"warnings": [],
"version": "1.18.1621",
"output_term": "categories"
}
}
}
The content I would like to map to a Dataframe is the one contained by the "categories" JSON Array.
I have managed to parse the message using json4s.JsonMethods method parse this way:
val parsedJson = parse(request) \\ "categories"
Obtaining the following:
output: org.json4s.JValue = JArray(List(JObject(List((id,JInt(1)), (name,JString(Airlines)), (is_sensitive,JBool(false)), (last_modified,JString(2010-03-19 17:48:36)), (requires_whitelist_on_external,JBool(false)), (requires_whitelist_on_managed,JBool(false)), (is_brand_eligible,JBool(true)), (requires_whitelist,JBool(false)), (whitelist,JObject(List((geos,JArray(List())), (countries_and_brands,JArray(List()))))))), JObject(List((id,JInt(2)), (name,JString(Apparel)), (is_sensitive,JBool(false)), (last_modified,JString(2010-03-19 17:48:36)), (requires_whitelist_on_external,JBool(false)), (requires_whitelist_on_managed,JBool(false)), (is_brand_eligible,JBool(true)), (requires_whitelist,JBool(false)), (whitelist,JObject(List((geos,JArray(List())), (countries_and_brands,JArray(List()))))))))
However, I am completely lost on how to proceed. I have even tried using another library for Scala called uJson:
val json = (ujson.read(request))
val tuples = json("response")("categories").arr /* <-- categories is an array */ .map { item =>
(item("id"), item("name"))
This time I have only parsed two fields for testing, but this shouldn't change much. Hence, I obtained the following structure:
tuples: scala.collection.mutable.ArrayBuffer[(ujson.Value, ujson.Value, ujson.Value, ujson.Value)] = ArrayBuffer((1,"Airlines",false,"2010-03-19 17:48:36"), (2,"Apparel",false,"2010-03-19 17:48:36"))
However, also this time I do not know how to move forward and everything I try results in errors, mostly related to format incompatibility.
Please, feel free to propose any other approach to achieve my goal even if it changes totally my workflow. I rather learn something properly. Thanks
We can use the following code to convert JSON to Spark Dataframe/Dataset
val df00 =
spark.read.option("multiline","true").json(Seq(JSON_OUTPUT).toDS())

How to parse dynamic json reponse and get specific value and pass it as an input to next request

I get .json file as a response from an API and from that file I should parse and fins specific parameter and pass it as an input to the next request, how do I do that using Katalon.
If I say
response = JSON.parse("response.json");
it says it is unable to identify JSON as valid. Can someone help me out with the solution?
Your JSON is invalid, maybe it is a copy-paste issue.
The valid JSON should be
{
"responseStatusCode": "OK",
"data": {
"screenName": "employeeTimeslip",
"screenType": "Redirect",
"searchResultCount": 0,
"rows": [],
"tabs": [],
"searchParams": {
"employeeID": "000092926",
"timeslipNumber": "201900019701"
}
}
}
So, you were missing a "," between "OK" and "data" and two closing curly braces at the end of the file.
You can check JSON files for validity yourself using online JSON validators, for example, this one.
i found a way to read specific parameter from the json response file like below:
val scn = scenario("ClaimSubmission")
.exec(http("request_2")
.post("URL")
.headers(headers_2)
.body(RawFileBody("json file path"))
.check(jsonPath("$..timeslipnumber").find.saveAs("timeslipnumber")))
Timeslip number would be retrieved using : .check(jsonPath("$..timeslipnumber").find.saveAs("timeslipnumber")))

JSON API result format

Folks,
Designing my first API in Node.JS using restify.js. My background is not webapis, pardon my amateur questions. In any case, I would like to have the res.send(data); responses to comply with the http://jsonapi.org/format/ so that my mobile application can start utilizing the api calls. At the moment if you were to call my api, it would return data in the following format:
{"Count":1,"Items":[{"dbsource":{"S":"foo"},"id":{"S":"5002820"},"name":{"S":"fnameblah,lnameblah"},"expiration":{"S":"06/13/2015"},"type":{"S":"bar"}}]}
Actually what you see above is just a return of a DynamoDB Query call.
So the question is... do you use a special library that you can pass data to, which would format and return the data in JSON format. Which in turn you can return it via res.send(data) to the clients, or is it up to us to make 'data' JSON compliant, then return it? At the end of the day we all want the results to look like:
{
"posts": [{
"id": "1",
"title": "Rails is Omakase",
"links": {
"author": "9",
"comments": [ "5", "12", "17", "20" ]
}
}]
}
Thanks!
In server side, stringify JSON object,
//...
res.statusCode = 200;
res.setHeader('Content-Type', 'application/json');
res.send(JSON.stringify(data)); //data is JSON object
res.end();
In client side, parse JSON string accordingly.
EDIT: Corrected response content type.
JSON data from server should be a JSON string
You have to parse it back the JSON format in client.
JSON.parse(string); // return JSON object