Question:What is the easiest way to test an API response for a binary response?
Context:I have a function that makes an API Call for some data. The response (api_response) of that API call will either be JSON or binary. If JSON, and if it contains percent_complete then the data is not yet ready, and my function uses the percent_complete key:pair value to update a progress bar for the user. If the response is JSON and contains meta, then my data is ready and has returned as a JSON object. If the response is binary, then my data is also ready, however has been returned as .xlsx [binary]. This is the response when the data is not ready and you will see percent_complete is utilized for a progress bar -
{
"data": {
"id": "2768510",
"type": "jobs",
"attributes": {
"job_type": "PORTFOLIO_VIEW_RESULTS",
"started_at": "2022-04-14T16:19:21Z",
"parameters": {
"end_date": "2022-04-14",
"output_type": "json",
"view_id": 304078,
"portfolio_id": 1,
"portfolio_type": "firm",
"start_date": "2022-04-14"
},
"percent_complete": 0.0,
"status": "In Progress"
},
"relationships": {
"creator": {
"links": {
"self": "/v1/jobs/2768510/relationships/creator",
"related": "/v1/jobs/2768510/creator"
},
"data": {
"type": "users",
"id": "731221"
}
}
},
"links": {
"self": "/v1/jobs/2768510"
}
},
"included": []
Current function:The following function continues to call the API every 5-seconds (using the 7-digit code in self, per the above API object) until meta is returned in a JSON object (therefore my data is returned as a JSON object) and returns the JSON object as api_response.Otherwise, the API call continues every 5-seconds and simply uses percent_complete to update a status bar (using enlighten library)
def api_call():
# Calling function containing the JOBS API endpoint for calling, until its RESPONSE == data requested.
endpoint_url = endpoint_initializer()
# Calling function containing API credentials
key, secret, url, group_url = ini_reader()
# Setting variable for use in progress bar, used to reflect API 'percent_complete' key pair value.
BAR_FORMAT = u'{id_value} {percentage:3.0f}%|{bar}| ' u'[{elapsed}<{eta}, {rate:.2f} %/s]'
manager = enlighten.get_manager()
date = dt.datetime.today().strftime("%Y-%m-%d")
print("------------------------------------\n","API URL constructed for:", date, "\n------------------------------------")
print("----------------------------------------------------------------------\n","Addepar Endpoint:", endpoint_url, "\n----------------------------------------------------------------------")
# Setting variable with counter for progress bar.
pbar = manager.counter(total=100, bar_format=BAR_FORMAT)
while True:
response = requests.get(url = endpoint_url, auth = HTTPBasicAuth(key, secret), headers = {"Vendor-firm": "665"})
api_response = json.loads(response.text)
if "meta" not in api_response:
id_value = "id"
res1 = [val[id_value] for key, val in api_response.items() if id_value in val]
id_value = "".join(res1)
percent_value = "percent_complete"
res2 = api_response["data"]["attributes"].get("percent_complete", '')*100
pbar.count = res2
pbar.update(incr=0, id_value=id_value)
time.sleep(5)
elif "meta" in api_response:
pbar.count = 100
pbar.update(incr=0, id_value=id_value)
pbar.close()
return api_response
How would I expand this function to test if the response (api_response) contains binary and if so, then return api_response?
A normal http-server should return an appropriate content-type. Please check:
response.headers['content-type']
Per Markus response, by accessing the Content-Type response.headers, I am able to trinagulate whether the response is binary (Content-Type: application/binary) or JSON (Content-Type: application/vnd.api+json).
Related
This question already has answers here:
Azure Function - Python - ServiceBus Output Binding - Setting Custom Properties
(2 answers)
Closed 8 months ago.
I'm using an Azure Function (Python) to send a message to a Service Bus topic whenever a file lands in blob storage following a similar set up to that outlined here.
In particular, in order to send the message I have this in the JSON file:
{
"type": "serviceBus",
"direction": "out",
"connection": "AzureServiceBusConnectionString",
"name": "msg",
"queueName": "outqueue"
}
and in init.py file I have msg.set(input_msg) where input_msg is a JSON string, the output of doing json.dumps(list(reader)) on a CSV string.
When this message is picked up by the topic and subscriptions it has content type set to text/plain, whilst I'd like this to be application/json as mentioned here.
Is there a way to set this, for instance when I do msg.set, is there a way to specify the content type?
Full code:
init.py
def get_json_content_from_csv(csv_content: str) -> str:
reader = csv.DictReader(io.StringIO(csv_content))
json_content = json.dumps(list(reader))
return json_content
def main(event: func.EventGridEvent, msg: func.Out[str]):
data = event.get_json()
url = data["url"]
input_blob = BlobClient.from_blob_url(url, DefaultAzureCredential())
csv_content = input_blob.download_blob(encoding='UTF-8').readall()
json_content = get_json_content_from_csv(csv_content)
msg.set(json.dumps(json_content))
function.json
{
"bindings": [
{
"type": "eventGridTrigger",
"name": "event",
"direction": "in"
},
{
"type": "serviceBus",
"direction": "out",
"connection": "AzureServiceBus",
"name": "msg",
"topicName": "dev-iris-service-bus-topic"
}
]
}
According to this github issue for the Python SDK:
Cannot set Service Bus Message ContentType - Github Issue
The github issue response points to the docs here to set the contentType property on the message class
https://learn.microsoft.com/en-us/python/api/uamqp/uamqp.message.messageproperties?view=azure-python
I'm using Volley library to communicate with my API. I'm pretty new to Android and Kotlin and I'm really confused about extracting keys from the following JSON data
{
"message": {
"_id": "60bc7fa7abeedb25643fa692",
"hash": "3a54b415461a63abac1fc6dfa0e140584047bd15358e33a177f9505ed2faa4d4",
"blockchain": "ethereum",
"amount": 5000,
"amount_usd": 13352971,
"from": "d3d69228cb2292f933572399593617f574c70eb1",
"to": "fe9996da73d6bf5252f15024811954ae37ab68be",
"__v": 0
}
}
The volley library returns all of this JSON data in a variable called response and I'm using response.getString("message") to extract the message key but, I don't understand how to extract the internal data such as hash, blockchain, amount, etc.
I'm using the following code to get the JSON data from my backend.
val jsonRequest = JsonObjectRequest(
Request.Method.GET, url, null,
{ response ->
tweet_text.setText(response.getString("message"))
Log.d("resp", response.toString())
},
{
Log.d("err", it.localizedMessage)
})
Any help would be appreciated, Thanks!
I found it, I just used the getJSONObject() method to make it work
val jsonRequest = JsonObjectRequest(
Request.Method.GET, url, null,
{ response ->
val txn = response.getJSONObject("message")
//txn object can be used to extract the internal data
},
{
Log.d("err", it.localizedMessage)
})
I am trying to send the following POST request with Matlab webwrite:
POST https://url.to.com/hello/world
HEADERS {"API_KEY": "abc123"}
JSON PAYLOAD
{
"return_type": "hello",
"entities": ["ent1"],
"events": ["legal"],
"fields": [],
"filters": {},
"start_date": "2015-01-01 00:00:00",
"end_date": "2016-01-01 00:00:00",
"format": "csv",
"compressed": false
}
In Matlab, I tried the following:
API_KEY = 'abc123';
url = 'https://url.to.com/hello/world';
options = weboptions(...
'MediaType', 'application/json', ...
'HeaderFields', {...
'API_KEY', API_KEY; ...
'Content-Type' 'application/json'});
payload.('return_type') = 'hello';
payload.('entities') = ['ent1'];
payload.('events') = ['legal'];
payload.('fields') = [];
payload.('filters') = {};
payload.('start_date') = '2015-01-01 00:00:00';
payload.('end_date') = '2016-01-01 00:00:00';
payload.('format') = 'csv';
payload.('compressed') = 'false';
response = webwrite(url, payload, options);
However, this returns the error:
The server returned the status 400 with message "Bad Request" in
response to the request to URL
I tried the request above with Postman and it worked. I have also verified that my Matlab headers are properly setup. So it must be my Matlab setup for the JSON payload part. What is wrong there?
Update 1:
I noticed that when you run jsonencode(payload) that it does not return the desired format. Moreover, the "[ .. ]" gets dropped out. I think that the problem starts there as then the request becomes indeed invalid. So we need a way to incorporate the brackets where necessary.
Found the answer on another forum. The problem was indeed the double brackets. We need to set it as follows:
payload.('entities') = {{'ent1'}};
Read more here: https://nl.mathworks.com/matlabcentral/answers/217716-how-to-pass-single-element-json-arrays-using-webwrite
Im quite new to this, any guidance or help is much appreciated.
I currently have a list of survey response i get form the following call:
import requests
import json
client = requests.session()
headers = {
"Authorization": "bearer %s" % "AccessTOKEN",
"Content-Type": "application/json"
}
HOST = "https://api.surveymonkey.net"
SURVEY_LIST_ENDPOINT = "/v3/surveys/SURVEYID/responses/bulk"
uri = "%s%s" % (HOST, SURVEY_LIST_ENDPOINT)
response = client.get(uri, headers=headers)
response_json = response.json()
print(response_json['data'])
i get a response from as something of the following:
{
"per_page": 50,
"total": 5114,
"data": [
{
"total_time": 40,
"href": "https://api.surveymonkey.net/v3/surveys/surveyID/responses/ID",
"custom_variables": {},
"ip_address": "IP ADDRESS",
"id": "ID",
"logic_path": {},
"date_modified": "2015-12-01T05:31:22+00:00",
"response_status": "completed",
"custom_value": "",
"analyze_url": "http://www.surveymonkey.com/analyze/browse/ID?respondent_id=ID",
"pages": [
{
"id": "220527570",
"questions": [
{
"id": "872991507",
"answers": [
{
"choice_id": "9573882449",
"row_id": "9573882439"
}
]
I would like to get the actually response value e.g "Yes, No, Maybe" from the choice_id?
many many thanks in advance,
Pon
There isn't a direct payload returned from the API right now that has the answer text as well.
You'll need to fetch your survey details first:
SURVEY_DETAIL_ENDPOINT = "/v3/surveys/SURVEYID/details"
uri = "%s%s" % (HOST, SURVEY_DETAIL_ENDPOINT)
response = client.get(uri, headers=headers)
survey_data = response.json()
Then you likely want to loop through the answers to create a lookup dictionary. Roughly something like:
answer_dict = {}
for page in survey_data['pages']:
for question in page['questions']:
# Rows, choices, other, and cols all make up the possible answers
answers = question['answers'].get('rows', [])\
+ question['answers'].get('choices', [])\
+ question['answers'].get('other', [])\
+ question['answers'].get('cols', [])
for answer in answers:
answer_dict[answer['id']] = answer
I know this looks rough with for loops but you're basically just traversing the entire survey. This works because choice IDs are globally unique (i.e. no collisions between columns, rows, choices etc.).
Then you can easily get the entire answer details by doing answer_dict[answer['choice_id']] for any answer in the response.
It wouldn't be a bad idea if the API itself allowed an option to fill in the answer text with the response for you.
What I want is a POST request in kemal where the body has a certain number of keys/values that I want to access and then an arbitrary JSON Object that I just want to stringify and pass on and later parse back to JSON.
My problem is that I apparently can't get the types right.
Think of a potential JSON body like this:
{
"endpoint": "http://example.com",
"interval": 500,
"payload": {
"something": "else",
"more": {
"embedded": 1
}
}
}
Now what I've been trying to do is the following:
require "kemal"
post "/schedule" do |env|
endpoint = env.params.json["endpoint"].as(String)
interval = env.params.json["interval"].as(Int64)
payload = String.from_json(env.params.json["payload"].as(JSON::Any))
# ... move things along
env.response.content_type = "application/json"
{ id: id }.to_json
end
Kemal.run
Now apparently what I seem to be getting when accessing "payload" is something of type Hash(String, JSON::Type), which confuses me a bit.
Any ideas how I'd be able to just get a sub-JSON from the request body, transform it to String and back to JSON?
Updated: payload is a type of JSON::Type. Casting and then calling .to_json does the trick.
require "kemal"
post "/schedule" do |env|
endpoint = env.params.json["endpoint"].as(String)
interval = env.params.json["interval"].as(Int64)
payload = env.params.json["payload"].as(JSON::Type)
env.response.content_type = "application/json"
payload.to_json
end
Kemal.run