EDIT: Here's a bit more context to how the JSON is received. I'm using the ApiAI API to generate a request to their platform, and they have a method to retrieve it, like this:
# instantiate ApiAI
ai = apiai.ApiAI(CLIENT_ACCESS_TOKEN)
# declare a request obect, fill in in lower lines
request = ai.text_request()
# send ApiAI the request
request.query = "{}".format(textobject.body)
# get response from ApiAI
response = request.getresponse()
response_decode = response.read().decode("utf-8")
response_data = json.loads(response_decode)
I'm coding a webapp in Django and trying to read through a JSON response POSTed to a webhook. The code to read through the JSON, after it has been decoded, is:
if response_data['result']['action'] != "":
Request.objects.create(
request = response_data['result']['resolvedQuery']
)
When I try to run this code, I get this error:
KeyError: 'result'
on the line
if response_data['result']['action'] != "":
I'm confused because it looks to me like 'result' should be a valid key to this JSON that is being read:
{
'id':'65738806-eb8b-4c9a-929f-28dc09d6a333',
'timestamp':'2017-07-10T04:59:46.345Z',
'lang':'en',
'result':{
'source':'agent',
'resolvedQuery':'Foobar',
'action':'Baz'
},
'alternateResult':{
'source':'domains',
'resolvedQuery':'abcdef',
'actionIncomplete':False,
},
'status':{
'code':200,
'errorType':'success'
}
}
Is there another way I should be reading this JSON in my program?
Try:
import JSON
if 'action' in response_data:
parsed_data = json.loads(response_data)
if parsed_data['result']['action'] != "":
Request.objects.create(request = parsed_data['result']['resolvedQuery'])
Thanks for everyone's thoughts. It turned out there was an another error with how I was trying to implement the ApiAI API, and that was causing this error. It now reads through the JSON fine, and I'm using #sasuke's suggestion.
Related
I am trying to build a script that will take each json object I have and execute a requests.post successfully until it is finished. The problem I feel I may be having is running a successful loop that handles that task for me. It has been awhile since I coded some python so any insights will be helpful. Below is my data and code sets
new_df = {
"id": 1,
"name": "memeone",
"smartphoneWidth": 0,
"isHtmlCompatible": true,
"instancesPerPage": 1,
"isArchived": false
}, {
"id": 1,
"name": "memetwo",
"smartphoneWidth": 0,
"isHtmlCompatible": true,
"instancesPerPage": 1,
"isArchived": false
}
I realize it is not in an a list or within brackets [], but this is the only I can get the data to successfully post in my experience. Do I need to put it in a dataframe?
Below is my code that I am using -
test_df = pd.read_csv('dummy_format_no_id_v4.csv').rename_axis('id')
test_two_df = test_df.reset_index().to_json(orient='records')
test_three_df = test_two_df[1:-1]
new_df = test_three_df
for item in new_df:
try:
username = 'username'
password = 'password'
headers = {"Content-Type": "application/json; charset=UTF-8"}
response = requests.post('https://api.someurl.com/1234/thispath', data=new_df, headers=headers, auth=(username, password))
print(response.text)
except:
print('ERROR')
the issue here is it will post first json object ("name": "memeone") successfully, but won't post the next one ("name":"memetwo")? How can I get it to iterate and also post the next json object? Must it be in a dataframe?
Thank you for any advice in advance. Apologies if my code is bad.
Actually, package requests itself has a json parameter that has been provided, and you can use that. instead of using data or saving it in dataframe, you can use like this :
for item in new_df:
try:
headers = {"Content-Type": "application/json; charset=UTF-8"}
response = requests.post(
f"https://httpbin.org/anything/{new_df}", json=new_df, headers=headers
)
# put break statement to terminate the loop
print(response.text)
break
except Exception as e:
raise Exception(f"Uncaught exception error {e}")
POST ing json from javascript to server in Play Framework:
var myJson = {"name": "joe", "age":20};
var obj = JSON.parse(myJson);
$.ajax(jsRoutes.controllers.MyController.create(obj));
Now, I have the javascript router configured fine. If i recieve the obj as a string I can print it out to the console just fine.
routes.conf:
POST /person/add controllers.MyController.createFromAjax(ajax: String)
BUT, I want to write the json to MongoDB using an Async promise which Activator gives the compile time error:
scala.concurrent.Future[play.api.mvc.Result][error] cannot be applied to (String)
I have other routes that take no parameters that receive json using Postman and write it to MongoDB just fine
routes.conf
POST /heartrates/bulk controllers.HRController.createFromJson
If I omit the parameter on the route that receives the json from Ajax instead of using Postman I get a HTTP 400 error in the browser.
POST http://localhost:9000/person/add 400 (Bad Request)
SO, my question is, Ajax needs a parameter but String wont work. Play documentation says json is always received as a String. What am I doing wrong here?
Scala Controller Code taken from Lightbend seed Play.Reactive.MongoDB:
def createBulkFromAjax = Action.async(parse.json) { request =>
val documents = for {
heartRate <- request.body.asOpt[JsArray].toStream
maybeHeartRate <- heartRate.value
validHeartRate <- maybeHeartRate.transform(transformer).asOpt.toList
} yield validHeartRate
for {
heartRate <- hrFuture
multiResult <- heartRate.bulkInsert(documents = documents, ordered = true)
} yield {
Logger.debug(s"Successfully inserted with multiResult: $multiResult")
Created(s"Created ${multiResult.n} heartRate")
}
}
I think you're getting mixed up between the parameters you pass to your Action as part of the jsRoutes call, and parameters that get passed to endpoints (i.e. the query string, query parameters etc).
Play will return a 400 Bad Request if you've declared a non-optional parameter (like you did with ajax: String) and you don't then actually supply it in your request.
While conceptually you are passing obj to your action, it's not as a query parameter - you've declared that your endpoint expects an HTTP POST - so the JSON should be in the HTTP request body. Notice your other endpoints don't take any query parameters.
So step 1 is to fix your routes file (I've renamed your method to match your other existing working one):
POST /person/add controllers.MyController.createFromJson
If you look at the Play documentation for the Javascript reverse router, you'll see that you'll need to set the type (aka HTTP method) if you're doing something other than a GET. So, step 2, here's what your Javascript should look like to achieve a POST:
var myJson = {"name": "joe", "age":20};
var obj = JSON.stringify(myJson);
var r = controllers.MyController.createFromJson;
$.ajax({url: r.url, type: r.type, data: obj });
After those changes you should be good; your controller code looks fine. If you still get 400 Bad Request responses, check that jQuery is setting your Content-Type header correctly - you may need to use the contentType option in the jQuery $.ajax call.
Edit after still getting 400 errors:
I've just noticed that you were using JSON.parse in your Javascript - as per this answer you should be using JSON.stringify to convert an object into something jQuery can send - otherwise it may try to URLEncode the data and/or send the fields as query parameters.
The other thing to look at is whether the JSON you are sending actually agrees with what you're trying to parse it as. I'm not sure if you've provided a simplified version for this question but it looks like you're trying to parse:
{"name": "joe", "age":20}
Using:
request.body.asOpt[JsArray]
Which will always result in a None - you didn't give it an array.
The Answer to ajax javascript routes in Play Framework 2.5 for ReativeMongo:
routes.conf:
GET /javascriptRoutes controllers.HRController.javascriptRoutes
HRController:
def javascriptRoutes = Action { implicit request =>
Ok(
JavaScriptReverseRouter("jsRoutes")(
routes.javascript.HRController.createBulkFromAjax
)
).as("text/javascript")
}
routes.conf:
POST /heartrates/add controllers.HRController.createBulkFromAjax
main.scala.html:
<script type="text/javascript" src="#routes.HRController.javascriptRoutes"></script>
javascript:
var r = jsRoutes.controllers.HRController.createBulkFromAjax();
$.ajax({url: r.url, type: r.type, contentType: "application/json", data: JsonString });
HRController:
def createBulkFromAjax = Action.async(parse.json) { request =>
//Transformation silent in case of failures.
val documents = for {
heartRate <- request.body.asOpt[JsArray].toStream
maybeHeartRate <- heartRate.value
validHeartRate <- maybeHeartRate.transform(transformer).asOpt.toList
} yield validHeartRate
for {
heartRate <- hrFuture
multiResult <- heartRate.bulkInsert(documents = documents, ordered = true)
} yield {
Logger.debug(s"Successfully inserted with multiResult: $multiResult")
Created(s"Created ${multiResult.n} heartRate")
}
}
HRController.createBulkFromAjax was built from a Lightbend activator ui seed example called play.ReactiveMogno
I am relatively new to both JSON and Django forms. And I wonder how Djagno's user_form.errors.as_json() should be used to transfer error messages to client-slde. Right now, I have the following code:
On the server-side. I have:
if form.is_valid():
# some code
else:
return JsonResponse(user_form.errors.as_json(), status = 400, safe = False)
Client:
$.post('/url/', data, function(response){
// Success
}).fail(function(response){
var errors = $.parseJSON($.parseJSON(response.responseText)); // looks stupid
The akward line $.parseJSON($.parseJSON(response.responseText)); proves that I am doing something wrong. Can anyone provide a best-practice code pattern for sending and parsing jsonified form errors ?
The problem is that you are You are converting to JSON twice - once when you call as_json, then again when you use JsonResponse.
You could use HttpResponse with form.errors.as_json():
return HttpResponse(user_form.errors.as_json(), status = 400, content_type='application/json')
Note the warnings in the as_json docs about escaping results to avoid a cross site scripting attack. You should ensure the results are escaped if you use JsonResponse as well.
According to boto3 documentation here: https://boto3.readthedocs.org/en/latest/reference/services/iot-data.html#client the update_thing_shadow method takes the thingName & JSON payload as parameters. Currently it reads:
client = boto3.client('iot-data', region_name='us-east-1')
data = {"state" : { "desired" : { "switch" : "on" }}}
mypayload = json.dumps(data)
response = client.update_thing_shadow(
thingName = 'MyDevice',
payload = b'mypayload'
)
When I use the command line there's no problem but can't seem to get it right from within the lamba function. I've called it with numerous versions of code (json.JSONEncoder, bytearray(), etc..) without any luck. The errors range from syntax to (ForbiddenException) when calling the UpdateThingShadow operation: Bad Request: ClientError. Has anyone had success calling this or a similar method from within a AWS lambda function? Thanks.
This code is working fine for me:
def set_thing_state(thingName, state):
# Change topic, qos and payload
payload = json.dumps({'state': { 'desired': { 'property': state } }})
logger.info("IOT update, thingName:"+thingName+", payload:"+payload)
#payload = {'state': { 'desired': { 'property': state } }}
response = client.update_thing_shadow(
thingName = thingName,
payload = payload
)
logger.info("IOT response: " + str(response))
logger.info("Body:"+response['payload'].read())
def get_thing_state(thingName):
response = client.get_thing_shadow(thingName=thingName)
streamingBody = response["payload"]
jsonState = json.loads(streamingBody.read())
print jsonState
#print jsonState["state"]["reported"]
Good luck
garnaat is right.
Just replace payload = b'mypayload' with payload = mypayload and it should work.
I have written a service which takes a json as input. I am using the website hurl.it to send post requests to check. Below is my code snippet:
class BatchSemanticSimilarityHandler(tornado.web.RequestHandler):
def post(self):
self.set_header('Access-Control-Allow-Origin', '*')
self.set_header('Access-Control-Allow-Credentials', 'true')
self.set_header('Access-Control-Allow-Methods', 'GET, POST, PUT, DELETE, OPTIONS')
self.set_header('Access-Control-Allow-Headers','Origin, Accept, Content-Type, X-Requested-With, X-CSRF-Token')
data = json.loads(self.request.body)
apikey = data["apikey"]
try:
UA = self.request.headers["User-Agent"]
except:
UA = "NA"
if bool(usercoll.find_one({"apikey":apikey})) == True:
sentence = data["sentence"]
sentence_array = data["sentence_array"]
n = data["num_of_results"]
if sentence is None or sentence_array is [] or apikey is None or n is None:
self.set_status(200)
output = {"error":[{"code":334,"message":"Bad Input data"}]}
misscoll.insert({"apitype":"batchsemanticsimilarity","timestamp":datetime.datetime.now(), "ip":self.request.remote_ip, "useragent":UA, "uri":self.request.uri,"apikey":apikey, "output":output, "input":{"s1":sentence,"s2":sentence_array}})
self.write(output)
return
results = nb.get_similar(sentence, sentence_array, apikey, n)
print "results is",results
output = {"similar_sentences": results, 'credits':'ParallelDots'}
hitscoll.insert({"apitype":"batchsemanticsimilarity","timestamp":datetime.datetime.now(), "ip":self.request.remote_ip, "useragent":UA, "uri":self.request.uri,"apikey":apikey, "output":output, "input":{"s1":sentence,"s2":sentence_array}})
self.write(output)
return
else:
rejectcoll.insert({"apitype":"batchsemanticsimilarity","apikey":apikey,"timestamp":datetime.datetime.now(), "ip":self.request.remote_ip, "useragent":UA, "url":self.request.uri})
self.write({"error":[{"code":333,"message": "Bad Authentication data"}]})
return
The json that I am giving as the body of the request is as below:
{
"sentence": "BJP leads in Bengaluru civic body`s poll, all eyes on JD(S)",
"sentence_array": [
"Narendra Modi is the prime minister",
"Sonia Gandhi runs Congress",
"Sachin is a good batsman"
],
"apikey": "DyMe1gSNhvMV1I1b20a7KARYIwuQX5GAQ",
"num_of_results": 2
}
I have verified on jsonlint that this is a valid JSON.
However while sending the request it gives me below error:
ValueError: No JSON object could be decoded
Can anyone please help me sort this out!!
The JSON object that you are passing in POST request is encoded into the url.
JSON library cannot read the encoded data.So you need to decode the url first.
Decoding of url can be done using urlparse library in python.so you need something like this.
post_data=urlparse.parse_qsl(self.request.body)
According to your need of final format to read there are various methods in urlparse.check this
or
As specified in the docs you can override a method to enable JSON parsing
def prepare(self):
if self.request.headers["Content-Type"].startswith("application/json"):
self.json_args = json.loads(self.request.body)
else:
self.json_args = None
check this