Get value from ID JSON response surveymonkey API v3 - json

Im quite new to this, any guidance or help is much appreciated.
I currently have a list of survey response i get form the following call:
import requests
import json
client = requests.session()
headers = {
"Authorization": "bearer %s" % "AccessTOKEN",
"Content-Type": "application/json"
}
HOST = "https://api.surveymonkey.net"
SURVEY_LIST_ENDPOINT = "/v3/surveys/SURVEYID/responses/bulk"
uri = "%s%s" % (HOST, SURVEY_LIST_ENDPOINT)
response = client.get(uri, headers=headers)
response_json = response.json()
print(response_json['data'])
i get a response from as something of the following:
{
"per_page": 50,
"total": 5114,
"data": [
{
"total_time": 40,
"href": "https://api.surveymonkey.net/v3/surveys/surveyID/responses/ID",
"custom_variables": {},
"ip_address": "IP ADDRESS",
"id": "ID",
"logic_path": {},
"date_modified": "2015-12-01T05:31:22+00:00",
"response_status": "completed",
"custom_value": "",
"analyze_url": "http://www.surveymonkey.com/analyze/browse/ID?respondent_id=ID",
"pages": [
{
"id": "220527570",
"questions": [
{
"id": "872991507",
"answers": [
{
"choice_id": "9573882449",
"row_id": "9573882439"
}
]
I would like to get the actually response value e.g "Yes, No, Maybe" from the choice_id?
many many thanks in advance,
Pon

There isn't a direct payload returned from the API right now that has the answer text as well.
You'll need to fetch your survey details first:
SURVEY_DETAIL_ENDPOINT = "/v3/surveys/SURVEYID/details"
uri = "%s%s" % (HOST, SURVEY_DETAIL_ENDPOINT)
response = client.get(uri, headers=headers)
survey_data = response.json()
Then you likely want to loop through the answers to create a lookup dictionary. Roughly something like:
answer_dict = {}
for page in survey_data['pages']:
for question in page['questions']:
# Rows, choices, other, and cols all make up the possible answers
answers = question['answers'].get('rows', [])\
+ question['answers'].get('choices', [])\
+ question['answers'].get('other', [])\
+ question['answers'].get('cols', [])
for answer in answers:
answer_dict[answer['id']] = answer
I know this looks rough with for loops but you're basically just traversing the entire survey. This works because choice IDs are globally unique (i.e. no collisions between columns, rows, choices etc.).
Then you can easily get the entire answer details by doing answer_dict[answer['choice_id']] for any answer in the response.
It wouldn't be a bad idea if the API itself allowed an option to fill in the answer text with the response for you.

Related

How to specify content type as application/json while sending message to azure service bus topic using an Azure Function? [duplicate]

This question already has answers here:
Azure Function - Python - ServiceBus Output Binding - Setting Custom Properties
(2 answers)
Closed 8 months ago.
I'm using an Azure Function (Python) to send a message to a Service Bus topic whenever a file lands in blob storage following a similar set up to that outlined here.
In particular, in order to send the message I have this in the JSON file:
{
"type": "serviceBus",
"direction": "out",
"connection": "AzureServiceBusConnectionString",
"name": "msg",
"queueName": "outqueue"
}
and in init.py file I have msg.set(input_msg) where input_msg is a JSON string, the output of doing json.dumps(list(reader)) on a CSV string.
When this message is picked up by the topic and subscriptions it has content type set to text/plain, whilst I'd like this to be application/json as mentioned here.
Is there a way to set this, for instance when I do msg.set, is there a way to specify the content type?
Full code:
init.py
def get_json_content_from_csv(csv_content: str) -> str:
reader = csv.DictReader(io.StringIO(csv_content))
json_content = json.dumps(list(reader))
return json_content
def main(event: func.EventGridEvent, msg: func.Out[str]):
data = event.get_json()
url = data["url"]
input_blob = BlobClient.from_blob_url(url, DefaultAzureCredential())
csv_content = input_blob.download_blob(encoding='UTF-8').readall()
json_content = get_json_content_from_csv(csv_content)
msg.set(json.dumps(json_content))
function.json
{
"bindings": [
{
"type": "eventGridTrigger",
"name": "event",
"direction": "in"
},
{
"type": "serviceBus",
"direction": "out",
"connection": "AzureServiceBus",
"name": "msg",
"topicName": "dev-iris-service-bus-topic"
}
]
}
According to this github issue for the Python SDK:
Cannot set Service Bus Message ContentType - Github Issue
The github issue response points to the docs here to set the contentType property on the message class
https://learn.microsoft.com/en-us/python/api/uamqp/uamqp.message.messageproperties?view=azure-python

How to test an API response for binary

Question:What is the easiest way to test an API response for a binary response?
Context:I have a function that makes an API Call for some data. The response (api_response) of that API call will either be JSON or binary. If JSON, and if it contains percent_complete then the data is not yet ready, and my function uses the percent_complete key:pair value to update a progress bar for the user. If the response is JSON and contains meta, then my data is ready and has returned as a JSON object. If the response is binary, then my data is also ready, however has been returned as .xlsx [binary]. This is the response when the data is not ready and you will see percent_complete is utilized for a progress bar -
{
"data": {
"id": "2768510",
"type": "jobs",
"attributes": {
"job_type": "PORTFOLIO_VIEW_RESULTS",
"started_at": "2022-04-14T16:19:21Z",
"parameters": {
"end_date": "2022-04-14",
"output_type": "json",
"view_id": 304078,
"portfolio_id": 1,
"portfolio_type": "firm",
"start_date": "2022-04-14"
},
"percent_complete": 0.0,
"status": "In Progress"
},
"relationships": {
"creator": {
"links": {
"self": "/v1/jobs/2768510/relationships/creator",
"related": "/v1/jobs/2768510/creator"
},
"data": {
"type": "users",
"id": "731221"
}
}
},
"links": {
"self": "/v1/jobs/2768510"
}
},
"included": []
Current function:The following function continues to call the API every 5-seconds (using the 7-digit code in self, per the above API object) until meta is returned in a JSON object (therefore my data is returned as a JSON object) and returns the JSON object as api_response.Otherwise, the API call continues every 5-seconds and simply uses percent_complete to update a status bar (using enlighten library)
def api_call():
# Calling function containing the JOBS API endpoint for calling, until its RESPONSE == data requested.
endpoint_url = endpoint_initializer()
# Calling function containing API credentials
key, secret, url, group_url = ini_reader()
# Setting variable for use in progress bar, used to reflect API 'percent_complete' key pair value.
BAR_FORMAT = u'{id_value} {percentage:3.0f}%|{bar}| ' u'[{elapsed}<{eta}, {rate:.2f} %/s]'
manager = enlighten.get_manager()
date = dt.datetime.today().strftime("%Y-%m-%d")
print("------------------------------------\n","API URL constructed for:", date, "\n------------------------------------")
print("----------------------------------------------------------------------\n","Addepar Endpoint:", endpoint_url, "\n----------------------------------------------------------------------")
# Setting variable with counter for progress bar.
pbar = manager.counter(total=100, bar_format=BAR_FORMAT)
while True:
response = requests.get(url = endpoint_url, auth = HTTPBasicAuth(key, secret), headers = {"Vendor-firm": "665"})
api_response = json.loads(response.text)
if "meta" not in api_response:
id_value = "id"
res1 = [val[id_value] for key, val in api_response.items() if id_value in val]
id_value = "".join(res1)
percent_value = "percent_complete"
res2 = api_response["data"]["attributes"].get("percent_complete", '')*100
pbar.count = res2
pbar.update(incr=0, id_value=id_value)
time.sleep(5)
elif "meta" in api_response:
pbar.count = 100
pbar.update(incr=0, id_value=id_value)
pbar.close()
return api_response
How would I expand this function to test if the response (api_response) contains binary and if so, then return api_response?
A normal http-server should return an appropriate content-type. Please check:
response.headers['content-type']
Per Markus response, by accessing the Content-Type response.headers, I am able to trinagulate whether the response is binary (Content-Type: application/binary) or JSON (Content-Type: application/vnd.api+json).

How to pass a json object as a query in a get request

I am trying to pass parameters to a url get request that requires one of the parameters be a json object. I am using Google apps script. I get an invalid argument exception. The searchConditions parameter requires the json. I also need to pass some additional parameters after the query to specify the fields to return.
The example in the documentation looks like this:
https://theurl/search?searchConditions=[{ "Param1": "value", "Param2": "value"}]&fields=[ "Field1", "Field2", "etc"]
My code is as follows:
var url = "https:/theurl/search?searchConditions=";
var conditions = {
"Param1":"value",
"Param2":"value"
};
var fields = "&fields=[\'Field1\',\'Field2\',\'etc\']";
var options = {
"method":"Get",
"Accept": "application/json",
"Content-Type": "application/json",
"muteHttpExceptions" : true
};
options.headers = {"Authorization": "Basic xxxxxxxxxxxxx="};
var jsondata = UrlFetchApp.fetch(url + conditions + fields, options);
var data = JSON.parse(jsondata.getContentText())
Update: From Documentation:
Search Incidents
URI
https://trackerbeyond.phaseware.com:443/pro9/api/incident/search
HTTP Verb
GET
Request Parameters
There are two request parameters to pass in as part of the URI; searchConditions and fields.
searchConditions Parameter
This is an array of search conditions. Note that when multiple conditions are defined, all conditions must be met for the Incident to be returned.
This table shows the fields to set within each search condition.
Fieldname Description
ConditionType Possible values: Exact, BeginsWith, Contains, DateRange, IsBlank, IsNotBlank, ArchiveStatus, OpenCloseStatus.
FieldName The name of the field to search against. To get the full list of fields use the Incident Schema API call.
TableName The name of the table that the field exists on. Possible values are z_Incident, UserIncident, z_FullTextMain, Customer, UserCustomer, Contacts ,UserContacts ,CustomerContacts ,UserCustomerContacts ,CustomerProducts, CustomerSupportPackages ,IncidentParts ,IncidentParties ,DepartmentGroupingMembers (for searching on a DepartmentGrouping). If blank then z_Incident will be used.
Note: For the CreateDate and Description fields use z_FullTextMain for the TableName.
Lookup An object that contains two fields; LookupTableName and LookupFieldName.
SearchValue The value to search on.
EndingSearchValue Used when specifying ConditionType of DateRange to supply the ending date.
OpenActive Possible values: All, OpenOrActive, ClosedOrArchived. Please note that this parameter only applies when adding a search condition with ConditionType of OpenCloseStatus.
Example
In this example, the first search condition is for searching on DepartmentID equal to 1. The second condition is for searching on Priority description of "Priority 3". This is an example of using the Lookup field to search on a description for a lookup instead of the id. The third condition is for including both open and closed incidents. By default only open incidents are returned.
[
{
"ConditionType": "Exact",
"FieldName": "DepartmentID",
"TableName": "",
"Lookup": "",
"SearchValue": "1",
"EndingSearchValue": "",
"OpenActive": "OpenOrActive"
},
{
"ConditionType": "Exact",
"FieldName": "PriorityID",
"TableName": "",
"Lookup": {
"LookupTableName": "LU_Priority",
"LookupFieldName": "Description"
},
"SearchValue": "Priority 3",
"EndingSearchValue": "",
"OpenActive": "OpenOrActive"
},
{
"ConditionType": "OpenCloseStatus",
"FieldName": "",
"TableName": "",
"Lookup": "",
"SearchValue": "1",
"EndingSearchValue": "",
"OpenActive": "All"
}
]
fields Parameter
This is an array of fields to return in the result. If this parameter is left blank then a default set of fields will be returned.
Successful call made from the Test this API interface: https://trackerbeyond.phaseware.com:443/pro9/api/incident/search?searchConditions=[ { "ConditionType": "Exact", "FieldName": "StatusID", "SearchValue": "12", "OpenActive": "OpenOrActive" } ]&fields=[ "CustomerName", "CreateDate", "AgentFullName", "ClosedDateTime", "StatusID", "IncidentID", "Description" ]
Modification points:
At GET method, contentType is not required to be used.
When you want to use Accept, please include it in the request header.
Although I'm not sure about the detail of the specification of the API you want to use, from The example in the documentation looks like this: https://theurl/search?searchConditions=[{ "Param1": "value", "Param2": "value"}]&fields=[ "Field1", "Field2", "etc"] in your question, I think that in this case, how about URL encoding the values of query parameter?
When above points are reflected to your script, it becomes as follows.
Sample script:
var url = "https:/theurl/search";
var conditions = [{ "Param1": "value", "Param2": "value" }];
var fields = ["Field1", "Field2", "etc"];
var options = {
"method": "get",
"headers": {
"Authorization": "Basic xxxxxxxxxxxxx=",
"Accept": "application/json",
},
"muteHttpExceptions": true
};
var jsondata = UrlFetchApp.fetch(`${url}?searchConditions=${encodeURIComponent(JSON.stringify(conditions))}&fields=${encodeURIComponent(JSON.stringify(fields))}`, options);
var data = JSON.parse(jsondata.getContentText())
Note:
In this answer, it supposes that the URL of https:/theurl/search and the token of "Basic xxxxxxxxxxxxx=" can be used for the API you want to use. Please be careful this.
If above modification didn't resolve your issue, can you provide the official document of the API you want to use? By this, I would like to confirm it.
References:
encodeURIComponent()
fetch(url, params)

JSON string pulled via net/http to Hash

I'm trying to pull JSON from a Yahoo API to get the conversion rate of USD to SEK. However, I can't seem to get the JSON converted to a Hash, it shows "query" as being the only key since JSON comes in as one string.
The JSON request returns:
{"query":{"count":1,"created":"2016-12-04T13:06:00Z","lang":"en-us","results":{"rate":{"id":"USDSEK","Name":"USD/SEK","Rate":"9.1900","Date":"12/2/2016","Time":"9:59pm","Ask":"9.2000","Bid":"9.1900"}}}}
My code is as follow:
require 'net/http'
require 'json'
url = 'https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20yahoo.finance.xchange%20where%20pair%20in%20(%22USDSEK%22)&format=json&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys&callback='
uri = URI(url)
response = Net::HTTP.get(uri)
json_hash= JSON.parse(response)
puts json_hash["Rate"]
the puts statement returns 'nil'
I've tried following an example from this site, however I do not yield the same results based on the way my data is being brought as his is being statically entered.
*Note I am not using 'ostruct', trying simply with json.
Thank you for any insight.
As you can see, the field you are looking for is into an inner hash. Try
puts json_hash["query"]["results"]["rate"]["Rate"]
Have you check the structure of your json?
{
"query": {
"count": 1,
"created": "2016-12-04T13:06:00Z",
"lang": "en-us",
"results": {
"rate": {
"id": "USDSEK",
"Name": "USD/SEK",
"Rate": "9.1900",
"Date": "12/2/2016",
"Time": "9:59pm",
"Ask": "9.2000",
"Bid": "9.1900"
}
}
}
}
To fetch the rate key you should do something like:
json_hash["query"]["results"]["rate"]
Compare that with json above to understand your problem.

Get rid of Mongo $ signs in JSON

I am building python backend for SPA (Angular) using MongoDB.
Here is what I use: Python 3.4, MongoDB 3, Flask, flask-mongoengine and flask-restful
Now I receive the following JSON from my backend:
[
{
"_id": {
"$oid": "55c737029380f82fbf52eec3"
},
"created_at": {
"$date": 1439129906376
},
"desc": "Description.....",
"title": "This is title"
},
etc...
]
And I want to receive something like that:
[
{
"_id": "55c737029380f82fbf52eec3",
"created_at": 1439129906376,
"desc": "Description.....",
"title": "This is title"
},
etc...
]
My code for now:
from flask import json
from vinnie import app
from flask_restful import Resource, Api
from vinnie.models.movie import Movie
api = Api(app)
class Movies(Resource):
def get(self):
movies = json.loads(Movie.objects().all().to_json())
return movies
api.add_resource(Movies, '/movies')
Model:
import datetime
from vinnie import db
class Movie(db.Document):
created_at = db.DateTimeField(default=datetime.datetime.now, required=True)
title = db.StringField(max_length=255, required=True)
desc = db.StringField(required=True)
def __unicode__(self):
return self.title
What is the best way to format convenient JSON for front-end?
If you are confident you want to get rid of all the similar cases, then you can certainly write code that matches that pattern. For example:
info = [
{
"_id": {
"$oid": "55c737029380f82fbf52eec3"
},
"created_at": {
"$date": 1439129906376
},
"desc": "Description.....",
"title": "This is title"
},
#etc...
]
def fix_array(info):
''' Change out dict items in the following case:
- dict value is another dict
- the sub-dictionary only has one entry
- the key in the subdictionary starts with '$'
In this specific case, one level of indirection
is removed, and the dict value is replaced with
the sub-dict value.
'''
for item in info:
for key, value in item.items():
if not isinstance(value, dict) or len(value) != 1:
continue
(subkey, subvalue), = value.items()
if not subkey.startswith('$'):
continue
item[key] = subvalue
fix_array(info)
print(info)
This will return this:
[{'title': 'This is title', 'created_at': 1439129906376, 'desc': 'Description.....', '_id': '55c737029380f82fbf52eec3'}]
Obviously, reformatting that with JSON is trivial.
I found a neat solution to my problem in flask-restful extension which I use.
It provides fields module.
Flask-RESTful provides an easy way to control what data you actually render in your response. With the fields module, you can use whatever objects (ORM models/custom classes/etc.) you want in your resource. fields also lets you format and filter the response so you don’t have to worry about exposing internal data structures.
It’s also very clear when looking at your code what data will be rendered and how it will be formatted.
Example:
from flask_restful import Resource, fields, marshal_with
resource_fields = {
'name': fields.String,
'address': fields.String,
'date_updated': fields.DateTime(dt_format='rfc822'),
}
class Todo(Resource):
#marshal_with(resource_fields, envelope='resource')
def get(self, **kwargs):
return db_get_todo() # Some function that queries the db
Flask-RESTful Output Fields Documentation