I am trying to send a simple JSON message to an Amazon SNS topic in Boto 3. However, I keep getting a _jsonparsefailure in the tag of the message and I only receive the default value. Here is my code:
mess = {'default': 'default', 'this': 'that'}
jmess = json.JSONEncoder().encode(mess)
response = self.boto_client.publish(
TopicArn = self.TopicArn,
MessageStructure = 'json',
Message = jmess
)
I have also tried json.dumps(), which produces the same result.
mess = {'default': 'default', 'this': 'that'}
jmess = json.dumps(mess)
response = self.boto_client.publish(
TopicArn = self.TopicArn,
MessageStructure = 'json',
Message = jmess
)
I seem to be following all of the guidelines set by the documentation, and I'm not getting an exception when I run the script. There are SQS queues that subscribe to the topic, and I am pulling the result data straight from the console.
This is how I fixed it:
message = {"record_id": "my_id", "name": "value"}
json_message = json.dumps({"default":json.dumps(message)})
sns_client.publish("topic_arn", Subject="test", MessageStructure="json", Message=json_message)
SNS expects "default" as the key which contains the message to be published.
It turns out the message needs to look like this:
json.dumps({"default": "my default", "sqs": json.dumps({"this": "that"})})
Amazon has horrible documentation in this regard.
You can also remove the MessageStructure='json'and send just json.dumps({'this':'that'}) if you set the SQS queue to receive just the raw message. This is simply done through the console.
In Boto 3 (I'm using v1.4.7) this is the format:
sns.publish(TopicArn="topic_arn", Message=json.dumps({"this": "that"},ensure_ascii=False))
There isn't any need for the protocol definition, i.e. "default" unless you are delivering different structures per protocol, i.e., JSON for Lambda and HTML for email.
Related
I'm working with a raspberry pi zero and Python to send and recieve sensor data with Azure IoT. I've already created an endpoint and message routing to the storage container. But when I check the JSON-Files in the container, I've got two problems:
The file include various general data which i don't need
My message body is in Base24-format
My message looks like this:
{"EnqueuedTimeUtc":"2021-06-25T13:03:25.7110000Z","Properties":{},"SystemProperties":{"connectionDeviceId":"RaspberryPi","connectionAuthMethod":"{"scope":"device","type":"sas","issuer":"iothub","acceptingIpFilterRule":null}","connectionDeviceGenerationId":"637555519600003402","enqueuedTime":"2021-06-25T13:03:25.7110000Z"},"Body":"eyJ0ZW1wZXJhdHVyZSI6IDI4Ljk1LCAicHJlc3N1cmUiOiA5ODEuMDg2Njk1NDU5MzMyNiwgImh1bWlkaXR5IjogNDYuMjE0ODE3NjkyOTEyODgsICJ0aW1lIjogIjIwMjEtMDYtMjUgMTQ6MDM6MjUuNjMxNzk1In0="}
The body included my sensor data in Base64-format. I've already read about contentType = application/JSON and contentEncoding = UTF-8 so that Azure can work with correct JSON files. But where do i apply these settings? When I apply it to the routing query, I get the following error:
Routing Query Error (The server didn't understand your query. Check your query syntax and try again)
I just want to get the body-message in correct JSON Format.
Thank you all for any kind of help! Since it's my first experience with this kind of stuff, I'm a little helpless.
Zero clue if this helps, but here is my code for sending data from Raspberry Pi Python to AWS - Parse Server using base64/JSON. The only reason I use base64 is to send pictures. You should only have to use JSON to send your other data.
import requests
import random, time
import math
import json
import Adafruit_DHT
import base64
from Adafruit_CCS811 import Adafruit_CCS811
from picamera import PiCamera
from time import sleep
DHT_SENSOR = Adafruit_DHT.DHT22
DHT_PIN =4
ccs = Adafruit_CCS811()
camera = PiCamera()
while True:
time.sleep(5)
camera.start_preview()
sleep(5)
camera.capture('/home/pi/Desktop/image.jpg')
camera.stop_preview()
with open('/home/pi/Desktop/image.jpg', 'rb') as binary_file:
binary_file_data = binary_file.read()
base64_encoded_data = base64.b64encode(binary_file_data)
base64_message = base64_encoded_data.decode('utf-8')
humidity, temperature = Adafruit_DHT.read_retry(DHT_SENSOR, DHT_PIN)
ccs.readData()
parseServer = {
"temp": temperature,
"humid": humidity,
"co2": ccs.geteCO2(),
"pic": base64_message
}
resultJSON = json.dumps(parseServer)
headers = {
'X-Parse-Application-Id': 'myappID',
'Content-Type': 'application/json',
}
data = resultJSON
response =
requests.put('http://1.11.111.1111/parse/classes/Gamefuck/TIuRnws3Ag',
headers=headers, data=data)
print(data)
If you're using the Python SDK for Azure IoT, sending the message as UTF-8 encoded JSON is as easy as setting two properties on your message object. There is a good example here
msg.content_encoding = "utf-8"
msg.content_type = "application/json"
Furthermore, you don't need to change anything in IoT Hub for this. This message setting is a prerequisite to be able to do message routing based on the body of the message.
First timer when it comes to connecting to API. I'm trying to pull data from Toggl using my API token but I can't get credentials working. I tried to replicate the method by Chris Webb (https://blog.crossjoin.co.uk/2014/03/26/working-with-web-services-in-power-query/) but I can't get it working. Here's my M code:
let
Source = Web.Contents(
"https://toggl.com/reports/api/v2/details?workspace_id=xxxxx&client=xxxxxx6&billable=yes&user_agent=xxxxxxx",
[
Query=[ #"filter"="", #"orderBy"=""],
ApiKeyName="api-token"
])
in
Source
After that I'm inputting my API Token into Web API method in Access Web content windows but I get an error that credentials could not be authenticated. Here's Toggl API specification:
https://github.com/toggl/toggl_api_docs/blob/master/reports.md
Web.Contents function receives two parameters: url + options
Inside options, you define the headers and the api_key, and other queryable properties, such as:
let
baseUrl = "https://toggl.com/",
// the token part can vary depending on the requisites of the API
accessToken = "Bearer" & "insert api token here"
options = [
Headers = [Authorization = accessToken, #"Content-Type" =
"application/Json"], RelativePath ="reports/api/v2/details", Query =
[workspace_id=xxxxx, client=xxxxxx6 , billable=yes, user_agent=xxxxxxx]
]
Source = Web.Contents(baseUrl, options)
// since Web.Contents() doesn't parse the binaries it fetches, you must use another
// function to see if the data was retreived, based on the datatype of the data
parsedData = Json.Document(Source)
in
parsedData
The baseUrl is the smallest url that works and never changes;
The RelativePath is the next part of the url before the first "?".
The Query record is where you define all the attributes to query as a record.
This is usually the format, but check the documentation of the API you're querying to see if it is similar.
Can I handle a non-JSON response in node-rest-client POST method?
This is the error and response i'm getting:
response: [PURGED], error: [SyntaxError: Unexpected token P in JSON at
position 0]
Can I somehow avoid getting an error? The function does what is requested.
I suppose creating a custom parser is a solution? I don't understand how to achieve it though.
You need to add a "mimetypes" attribute to your client options.
For example, something like this should enable you to handle XML as well as JSON responses:
const Client = require('node-rest-client').Client;
var client = new Client({mimetypes:{
json:["application/json","application/json;charset=utf-8"],
xml:["application/xml","application/xml;charset=utf-8"]
}});
client.post(...)
I was successful in publishing (POST) a JSON file in Zapier and creating a Storage for it. However, I´d like to access the JSON in Zapier Storage using a Python code run locally. I am able to access the storage with Python3, see that is something written there, but I cannot access the JSON contents.
import urllib
import json
import codecs
reader = codecs.getreader("utf-8")
access_token = "password"
def GetStorage(page_id, access_token):
url = 'https://hooks.zapier.com/url/'
response = urllib.request.urlopen(url)
data = json.load(reader(response))
return data
a=GetStorage(url, access_token)
print(a)
All I get is:
{'attempt': '5a539a49-65eb-44f8-a30e-e171faf7a680',
'id': '1b38d21a-0150-46df-98c1-490a0d04b565',
'request_id': '5a539a49-65eb-44f8-a30e-e171faf7a680',
'status': 'success'}
When in fact I need:
{'Name':'value',
'Address': 'value'
}
Any ideas ?
David here, from the Zapier Platform team.
You're close! hooks.zapier.com is the url we use for incoming webhooks, so we always reply with a 200 and the response body you're seeing.
Instead, use store.zapier.com. You'll also want to make sure to include your secret. A full request URL will look like:
https://store.zapier.com/api/records?secret=test
which will return arbitrary json data:
{
"name": "david",
"job": "programmer"
}
The full docs are in json here: https://store.zapier.com/
EDIT: Here's a bit more context to how the JSON is received. I'm using the ApiAI API to generate a request to their platform, and they have a method to retrieve it, like this:
# instantiate ApiAI
ai = apiai.ApiAI(CLIENT_ACCESS_TOKEN)
# declare a request obect, fill in in lower lines
request = ai.text_request()
# send ApiAI the request
request.query = "{}".format(textobject.body)
# get response from ApiAI
response = request.getresponse()
response_decode = response.read().decode("utf-8")
response_data = json.loads(response_decode)
I'm coding a webapp in Django and trying to read through a JSON response POSTed to a webhook. The code to read through the JSON, after it has been decoded, is:
if response_data['result']['action'] != "":
Request.objects.create(
request = response_data['result']['resolvedQuery']
)
When I try to run this code, I get this error:
KeyError: 'result'
on the line
if response_data['result']['action'] != "":
I'm confused because it looks to me like 'result' should be a valid key to this JSON that is being read:
{
'id':'65738806-eb8b-4c9a-929f-28dc09d6a333',
'timestamp':'2017-07-10T04:59:46.345Z',
'lang':'en',
'result':{
'source':'agent',
'resolvedQuery':'Foobar',
'action':'Baz'
},
'alternateResult':{
'source':'domains',
'resolvedQuery':'abcdef',
'actionIncomplete':False,
},
'status':{
'code':200,
'errorType':'success'
}
}
Is there another way I should be reading this JSON in my program?
Try:
import JSON
if 'action' in response_data:
parsed_data = json.loads(response_data)
if parsed_data['result']['action'] != "":
Request.objects.create(request = parsed_data['result']['resolvedQuery'])
Thanks for everyone's thoughts. It turned out there was an another error with how I was trying to implement the ApiAI API, and that was causing this error. It now reads through the JSON fine, and I'm using #sasuke's suggestion.