I am trying to send the following POST request with Matlab webwrite:
POST https://url.to.com/hello/world
HEADERS {"API_KEY": "abc123"}
JSON PAYLOAD
{
"return_type": "hello",
"entities": ["ent1"],
"events": ["legal"],
"fields": [],
"filters": {},
"start_date": "2015-01-01 00:00:00",
"end_date": "2016-01-01 00:00:00",
"format": "csv",
"compressed": false
}
In Matlab, I tried the following:
API_KEY = 'abc123';
url = 'https://url.to.com/hello/world';
options = weboptions(...
'MediaType', 'application/json', ...
'HeaderFields', {...
'API_KEY', API_KEY; ...
'Content-Type' 'application/json'});
payload.('return_type') = 'hello';
payload.('entities') = ['ent1'];
payload.('events') = ['legal'];
payload.('fields') = [];
payload.('filters') = {};
payload.('start_date') = '2015-01-01 00:00:00';
payload.('end_date') = '2016-01-01 00:00:00';
payload.('format') = 'csv';
payload.('compressed') = 'false';
response = webwrite(url, payload, options);
However, this returns the error:
The server returned the status 400 with message "Bad Request" in
response to the request to URL
I tried the request above with Postman and it worked. I have also verified that my Matlab headers are properly setup. So it must be my Matlab setup for the JSON payload part. What is wrong there?
Update 1:
I noticed that when you run jsonencode(payload) that it does not return the desired format. Moreover, the "[ .. ]" gets dropped out. I think that the problem starts there as then the request becomes indeed invalid. So we need a way to incorporate the brackets where necessary.
Found the answer on another forum. The problem was indeed the double brackets. We need to set it as follows:
payload.('entities') = {{'ent1'}};
Read more here: https://nl.mathworks.com/matlabcentral/answers/217716-how-to-pass-single-element-json-arrays-using-webwrite
Related
Question:What is the easiest way to test an API response for a binary response?
Context:I have a function that makes an API Call for some data. The response (api_response) of that API call will either be JSON or binary. If JSON, and if it contains percent_complete then the data is not yet ready, and my function uses the percent_complete key:pair value to update a progress bar for the user. If the response is JSON and contains meta, then my data is ready and has returned as a JSON object. If the response is binary, then my data is also ready, however has been returned as .xlsx [binary]. This is the response when the data is not ready and you will see percent_complete is utilized for a progress bar -
{
"data": {
"id": "2768510",
"type": "jobs",
"attributes": {
"job_type": "PORTFOLIO_VIEW_RESULTS",
"started_at": "2022-04-14T16:19:21Z",
"parameters": {
"end_date": "2022-04-14",
"output_type": "json",
"view_id": 304078,
"portfolio_id": 1,
"portfolio_type": "firm",
"start_date": "2022-04-14"
},
"percent_complete": 0.0,
"status": "In Progress"
},
"relationships": {
"creator": {
"links": {
"self": "/v1/jobs/2768510/relationships/creator",
"related": "/v1/jobs/2768510/creator"
},
"data": {
"type": "users",
"id": "731221"
}
}
},
"links": {
"self": "/v1/jobs/2768510"
}
},
"included": []
Current function:The following function continues to call the API every 5-seconds (using the 7-digit code in self, per the above API object) until meta is returned in a JSON object (therefore my data is returned as a JSON object) and returns the JSON object as api_response.Otherwise, the API call continues every 5-seconds and simply uses percent_complete to update a status bar (using enlighten library)
def api_call():
# Calling function containing the JOBS API endpoint for calling, until its RESPONSE == data requested.
endpoint_url = endpoint_initializer()
# Calling function containing API credentials
key, secret, url, group_url = ini_reader()
# Setting variable for use in progress bar, used to reflect API 'percent_complete' key pair value.
BAR_FORMAT = u'{id_value} {percentage:3.0f}%|{bar}| ' u'[{elapsed}<{eta}, {rate:.2f} %/s]'
manager = enlighten.get_manager()
date = dt.datetime.today().strftime("%Y-%m-%d")
print("------------------------------------\n","API URL constructed for:", date, "\n------------------------------------")
print("----------------------------------------------------------------------\n","Addepar Endpoint:", endpoint_url, "\n----------------------------------------------------------------------")
# Setting variable with counter for progress bar.
pbar = manager.counter(total=100, bar_format=BAR_FORMAT)
while True:
response = requests.get(url = endpoint_url, auth = HTTPBasicAuth(key, secret), headers = {"Vendor-firm": "665"})
api_response = json.loads(response.text)
if "meta" not in api_response:
id_value = "id"
res1 = [val[id_value] for key, val in api_response.items() if id_value in val]
id_value = "".join(res1)
percent_value = "percent_complete"
res2 = api_response["data"]["attributes"].get("percent_complete", '')*100
pbar.count = res2
pbar.update(incr=0, id_value=id_value)
time.sleep(5)
elif "meta" in api_response:
pbar.count = 100
pbar.update(incr=0, id_value=id_value)
pbar.close()
return api_response
How would I expand this function to test if the response (api_response) contains binary and if so, then return api_response?
A normal http-server should return an appropriate content-type. Please check:
response.headers['content-type']
Per Markus response, by accessing the Content-Type response.headers, I am able to trinagulate whether the response is binary (Content-Type: application/binary) or JSON (Content-Type: application/vnd.api+json).
I am trying to build a script that will take each json object I have and execute a requests.post successfully until it is finished. The problem I feel I may be having is running a successful loop that handles that task for me. It has been awhile since I coded some python so any insights will be helpful. Below is my data and code sets
new_df = {
"id": 1,
"name": "memeone",
"smartphoneWidth": 0,
"isHtmlCompatible": true,
"instancesPerPage": 1,
"isArchived": false
}, {
"id": 1,
"name": "memetwo",
"smartphoneWidth": 0,
"isHtmlCompatible": true,
"instancesPerPage": 1,
"isArchived": false
}
I realize it is not in an a list or within brackets [], but this is the only I can get the data to successfully post in my experience. Do I need to put it in a dataframe?
Below is my code that I am using -
test_df = pd.read_csv('dummy_format_no_id_v4.csv').rename_axis('id')
test_two_df = test_df.reset_index().to_json(orient='records')
test_three_df = test_two_df[1:-1]
new_df = test_three_df
for item in new_df:
try:
username = 'username'
password = 'password'
headers = {"Content-Type": "application/json; charset=UTF-8"}
response = requests.post('https://api.someurl.com/1234/thispath', data=new_df, headers=headers, auth=(username, password))
print(response.text)
except:
print('ERROR')
the issue here is it will post first json object ("name": "memeone") successfully, but won't post the next one ("name":"memetwo")? How can I get it to iterate and also post the next json object? Must it be in a dataframe?
Thank you for any advice in advance. Apologies if my code is bad.
Actually, package requests itself has a json parameter that has been provided, and you can use that. instead of using data or saving it in dataframe, you can use like this :
for item in new_df:
try:
headers = {"Content-Type": "application/json; charset=UTF-8"}
response = requests.post(
f"https://httpbin.org/anything/{new_df}", json=new_df, headers=headers
)
# put break statement to terminate the loop
print(response.text)
break
except Exception as e:
raise Exception(f"Uncaught exception error {e}")
Im quite new to this, any guidance or help is much appreciated.
I currently have a list of survey response i get form the following call:
import requests
import json
client = requests.session()
headers = {
"Authorization": "bearer %s" % "AccessTOKEN",
"Content-Type": "application/json"
}
HOST = "https://api.surveymonkey.net"
SURVEY_LIST_ENDPOINT = "/v3/surveys/SURVEYID/responses/bulk"
uri = "%s%s" % (HOST, SURVEY_LIST_ENDPOINT)
response = client.get(uri, headers=headers)
response_json = response.json()
print(response_json['data'])
i get a response from as something of the following:
{
"per_page": 50,
"total": 5114,
"data": [
{
"total_time": 40,
"href": "https://api.surveymonkey.net/v3/surveys/surveyID/responses/ID",
"custom_variables": {},
"ip_address": "IP ADDRESS",
"id": "ID",
"logic_path": {},
"date_modified": "2015-12-01T05:31:22+00:00",
"response_status": "completed",
"custom_value": "",
"analyze_url": "http://www.surveymonkey.com/analyze/browse/ID?respondent_id=ID",
"pages": [
{
"id": "220527570",
"questions": [
{
"id": "872991507",
"answers": [
{
"choice_id": "9573882449",
"row_id": "9573882439"
}
]
I would like to get the actually response value e.g "Yes, No, Maybe" from the choice_id?
many many thanks in advance,
Pon
There isn't a direct payload returned from the API right now that has the answer text as well.
You'll need to fetch your survey details first:
SURVEY_DETAIL_ENDPOINT = "/v3/surveys/SURVEYID/details"
uri = "%s%s" % (HOST, SURVEY_DETAIL_ENDPOINT)
response = client.get(uri, headers=headers)
survey_data = response.json()
Then you likely want to loop through the answers to create a lookup dictionary. Roughly something like:
answer_dict = {}
for page in survey_data['pages']:
for question in page['questions']:
# Rows, choices, other, and cols all make up the possible answers
answers = question['answers'].get('rows', [])\
+ question['answers'].get('choices', [])\
+ question['answers'].get('other', [])\
+ question['answers'].get('cols', [])
for answer in answers:
answer_dict[answer['id']] = answer
I know this looks rough with for loops but you're basically just traversing the entire survey. This works because choice IDs are globally unique (i.e. no collisions between columns, rows, choices etc.).
Then you can easily get the entire answer details by doing answer_dict[answer['choice_id']] for any answer in the response.
It wouldn't be a bad idea if the API itself allowed an option to fill in the answer text with the response for you.
What I want is a POST request in kemal where the body has a certain number of keys/values that I want to access and then an arbitrary JSON Object that I just want to stringify and pass on and later parse back to JSON.
My problem is that I apparently can't get the types right.
Think of a potential JSON body like this:
{
"endpoint": "http://example.com",
"interval": 500,
"payload": {
"something": "else",
"more": {
"embedded": 1
}
}
}
Now what I've been trying to do is the following:
require "kemal"
post "/schedule" do |env|
endpoint = env.params.json["endpoint"].as(String)
interval = env.params.json["interval"].as(Int64)
payload = String.from_json(env.params.json["payload"].as(JSON::Any))
# ... move things along
env.response.content_type = "application/json"
{ id: id }.to_json
end
Kemal.run
Now apparently what I seem to be getting when accessing "payload" is something of type Hash(String, JSON::Type), which confuses me a bit.
Any ideas how I'd be able to just get a sub-JSON from the request body, transform it to String and back to JSON?
Updated: payload is a type of JSON::Type. Casting and then calling .to_json does the trick.
require "kemal"
post "/schedule" do |env|
endpoint = env.params.json["endpoint"].as(String)
interval = env.params.json["interval"].as(Int64)
payload = env.params.json["payload"].as(JSON::Type)
env.response.content_type = "application/json"
payload.to_json
end
Kemal.run
I am trying to write to Firebase from Google Apps Script.
I try this:
var act="https://SampleChat.firebaseIO-demo.com/users/fred/name.json";
UrlFetchApp.fetch(act,{"method":"post","payload":"test"});
and I get this error:
Request failed for https://SampleChat.firebaseIO-demo.com/users/fred/name.json
returned code 400. Truncated server response:
{ "error" : "Invalid data; couldn't parse JSON object, array, or value.
Perhaps you're using invalid characters in your key names." }...
Question
How do I write to Firebase from GAS? (What am I doing wrong?)
Answer
You must stringify the payload object.
Here is a question and answer example.
I tested it and it does work.
Example
function putToFire() {
var payload = '{ "first": "Foo", "last": "Bar" }';
var options = {
"method": "put",
"payload": payload
};
UrlFetchApp.fetch(
"https://SampleChat.firebaseIO-demo.com/users/fred/name.json",
options
);
}
The Key Trick
Notice the single quotes ' around the payload object?
In the following line:
var payload = '{ "first": "Foo", "last": "Bar" }';
That counterintuitive syntax seems to be the trick to making this work. ;-)
Alternative Syntax for Variable Payloads
If your payload is a variable, you can use JSON.stringify():
var payload=JSON.stringify({"first":"Foo","last":"Bar"});