I am currently writing an application that queries a third party API. Previously when I have done this, I have done it the correct way and created a struct and unmarshalled the response string into the struct and accessed all the data that way. However that only works if the data structure is consistent.
I have the issue of trying to query an API where the structure is inconsistent. If the request was successful I get the response
{'status': 'ok', 'due_date': '2023-01-01', 'library': 'AIEHA1'}
but if it's unsuccessful, depending on the error type, I get different structures; some examples:
{'status': 'unauthorized', 'error': 'Field Bearer empty'}
{'status': 'not-found', 'error-details': {'type': 'file-not-found', 'file': '/index'}}
Obviously, I can pass this into a generic map but I was wondering what the proper practice is for something like this? Yes the API is terrible, yes sadly I have to use it.
One way of dealing with this is to have a structure containing all possible fields:
type apiResult struct {
Status string
DueDate string
Error string
ErrorDetails ErrorDetail
}
You unmarshal the API response, then process the API result structure to determine the actual return type.
Another way of doing it is to unmarshal into the expected struct, and then unmarshal again into an error struct based on status:
json.Unmarshal(result,&data)
if data.Status=="error" {
json.Unmarshal(result,&errorStruct)
} else if data.Status=="not-found" {
... etc.
}
When the response is not an error, this has the benefit of unmarshalling only once to the target struct.
Usually APIs return HTTP status other than 2xx code when there is an error. If that is the case, you can look at the response code and unmarshal based on that.
Related
How can I parse a JSON response from https://api.twitchinsights.net/v1/bots/online to an array in Go and iterate over every entry?
I dont understand the struct because there are no keys only values...
Can anyone please help and explain how this works?
I've mapped it but then I get something like
map[_total:216 bots:[[anotherttvviewer 67063 1.632071051e+09] [defb 26097 1.632071051e+09] [commanderroot 17531 1.632071048e+09] [apparentlyher 16774 1.63207105e+09]...
But I cant iterate over the map.
Because the API you're working with returns data where it could be a string or a number (in the array of arrays property bots), you'll need to use []interface{} as the type for each element of that array because the empty interface (https://tour.golang.org/methods/14) works for any type at run time.
type response struct {
Bots [][]interface{} `json:"bots"`
Total int `json:"_total"`
}
Then, as you iterate through each item in the slice, you can check its type using reflection.
It would be ideal for the API to return data in a schema where every JSON array element has the same JSON type as every other element in its array. This will be easier to parse, especially using statically typed languages like Go.
For example, the API could return data like:
{
"bots": [
{
"stringProp": "value1",
"numberProps": [
1,
2
]
}
],
"_total": 1
}
Then, you could write a struct representing the API response without using the empty interface:
type bot struct {
StringProp string `json:"stringProp"`
NumberProps []float64 `json:"numberProps"`
}
type response struct {
Bots []bot `json:"bots"`
Total int `json:"_total"`
}
But sometimes you're not in control of the API you're working with, so you need to be willing to parse the data from the response in a more dynamic way. If you do have control of the API, you should consider returning the data this way instead.
So i'm just starting out with typescript and something that I can't find much info on is how to deal with types for api call responses.
Lets say, I make a GET request to an api and it returns a JSON object for example:
{
name: "john",
age: 12
}
in my code, if i wanted to interact with response.name would i have to create an interface like below to use response.name without having a red linter under it?
interface Response {
name: string,
age: number
}
or is there a easier way to do it as some api calls would return JSON > 10 lines long and copying out the whole structure for every type of call seems very troublesome. Another thought I had was to create the interface but only have values I would use instead of the whole structure, but i'm not too sure so any help would be greatly appreciated!
You can define Response as a "common" type, that supports all types of API json response.
interface IResponse {
[key: string]: any
}
Now, you can type response.name without the "red line", also response.not_exist_property is valid.
My recommendation is to define all types for all API response type:
interface GetUserResponse {
name: string,
age: number,
}
for GET /users/:id (example)
You can use this tool to convert a json response to Typescript type.
My professor wants me to write a little tutorial on how to deploy Ballerina services. So I'm trying to learn it. I'm using Version 1.2 and I'm a bit overwhelmed by the concept of taint checking and the variable types...
I'm trying to write a minimal REST-Service with an endpoint that requests json data from another api and then uses that JSON to do stuff.
What's working so far is the following:
service tutorial on new http:Listener(9090) {
// Resource functions are invoked with the HTTP caller and the incoming request as arguments.
resource function getName(http:Caller caller, http:Request req) {
http:Client clientEP = new("https://api.scryfall.com/");
var resp = clientEP->get("/cards/random");
if (resp is http:Response) {
var payload = resp.getJsonPayload();
if (payload is json) {
var result = caller->respond(<#untainted>(payload));
} else {
log:printError("");
}
} else {
log:printError("");
}
}
That responds with the JSON that is returned from https://api.scryfall.com/cards/random
But lets now say, that I want to access a single value from that JSON. e.G. "name".
If I try to access it like this: payload["name"]
I get: invalid operation: type 'json' does not support indexing
I just figured out that it works if I create a map first like that:
map mp = <map>payload;
If I then access mp["name"] it works. BUT WHY? What is the json type good for if you still have to create a map and then cast the payload? And how would I access json inside the json? For example mp["data"][0]... invalid operation: type 'json' does not support indexing again...
And I'm still trying to understand the concept of taint checking....
do I just cast everything that is tainted to <#untainted> after checking the content?
Sometimes I really do not get what the documentation is trying to tell me....
I would recommend you to use Ballerina Swan Lake versions. Swan Lake versions contain enhancements to various language features. Here is a sample code that covers your use case. You can download Swan Lake Alpha2 at https://ballerina.io/
import ballerina/io;
import ballerina/http;
service tutorial on new http:Listener(9090) {
resource function get payload() returns json|error {
http:Client clientEP = check new ("https://api.scryfall.com/");
json payload = <json> check clientEP -> get("/cards/random", targetType = json);
// Processing the json payload
// Here the type of the `payload.name` expression is json | error
// You can eliminate with check: it returns from this resource with this error
json nameField = check payload.name;
io:println(nameField);
// You can traverse the json tree as follows
json standardLegality = check payload.legalities.standard;
io:println(standardLegality);
// colors is an array
// The cast is necessary because `check payload.colors` gives you a json
json colors = <json[]> check payload.colors;
io:println(colors);
// Responding with the complete payload recived from api.scryfall.com
return payload;
}
}
Taint analysis helps you to write code with no security vulnerabilities. However, we've disabled taint analysis in Swan Lake versions. If you want to enable it, you can use the option --taint-check with bal build
Is there a way to marshall a struct to JSON that skips any fields that can not be serialised?
E.G. if I marshal
type aStruct struct {
request *http.Request
name string
}
the JSON representation of
type aStruct struct {
name string
}
would result?
By 'fields that can not be serialised' I mean any field that would cause json.Marshal(..) to return an 'error serializing json: unsupported type' error. Having a look through the json package now with a mind to create another Marshal function that will skip a field it fails to serialise rather than abort and return the error.
==== UPDATE ====
I have created this https://github.com/myles-mcdonnell/jsonx which satisfies my use case. At first I thought this is a terrible way to extend the base json package (it's a copy from 1.6.2 with new behaviour added) but the consensus among my colleagues is that this is the idiomatic way to do this.
I have a document on a mongodb on Heroku. Each object in the document has a system generated object id in the form of
"_id": {
"$oid": "xxxxxxxxxxxxxxxxxxxxxxxx"
}
When I make a query and get the response from the server, I stringify the response using JSON.stringify and I log the object on the server console. When I do this the following gets logged:
this is the response: [{"creator":"al","what[place]":"home","what[time [start]":"22:00","what[time][end]":"","what[details]":"","who[]":["joe","kay","mau"],"_id":"xxxxxxxxxxxxxxxxxxxxxxxx"}]
Right after the full object gets logged, I try to log the id to make sure I can access it... I want to then pass the id to a different object so that I can have a reference to the logged object.
I have this right now:
var stringyfied = JSON.stringify(res);
console.log("this is the response: " + stringyfied);
console.log("id: " + stringyfied._id);
but when the item is logged I get
id: undefined
instead of
id: "xxxxxxxxxxxxxxxxxxxxxxxx"
No matter how I try to access the _id property, I keep getting undefined even though it get printed with the console.log for the full object
I've tried:
stringyfied.id
stringyfied["_id"]
stringyfied["id"]
stringyfied._id.$oid
stringyfied.$oid
you need to use JSON.parse(), cause JSON.stringify() is to convert to string, parse is to get the object. stringifiedid is a string
What is being returned is an array with one object in it.
The way to access the _id is with stringyfied[0]._id. However, it would be cleaner to pull the object out of the array instead.
There are a few ways you can do that. If this query will only ever return one result and that's all you'll want, you can use the findOne method instead. If the query may return more than a single document, then you will want to loop through the returned results.
I also agree with #dariogriffo that you'll need to use JSON.parse() on the stringified JSON variable.