I'm working with a raspberry pi zero and Python to send and recieve sensor data with Azure IoT. I've already created an endpoint and message routing to the storage container. But when I check the JSON-Files in the container, I've got two problems:
The file include various general data which i don't need
My message body is in Base24-format
My message looks like this:
{"EnqueuedTimeUtc":"2021-06-25T13:03:25.7110000Z","Properties":{},"SystemProperties":{"connectionDeviceId":"RaspberryPi","connectionAuthMethod":"{"scope":"device","type":"sas","issuer":"iothub","acceptingIpFilterRule":null}","connectionDeviceGenerationId":"637555519600003402","enqueuedTime":"2021-06-25T13:03:25.7110000Z"},"Body":"eyJ0ZW1wZXJhdHVyZSI6IDI4Ljk1LCAicHJlc3N1cmUiOiA5ODEuMDg2Njk1NDU5MzMyNiwgImh1bWlkaXR5IjogNDYuMjE0ODE3NjkyOTEyODgsICJ0aW1lIjogIjIwMjEtMDYtMjUgMTQ6MDM6MjUuNjMxNzk1In0="}
The body included my sensor data in Base64-format. I've already read about contentType = application/JSON and contentEncoding = UTF-8 so that Azure can work with correct JSON files. But where do i apply these settings? When I apply it to the routing query, I get the following error:
Routing Query Error (The server didn't understand your query. Check your query syntax and try again)
I just want to get the body-message in correct JSON Format.
Thank you all for any kind of help! Since it's my first experience with this kind of stuff, I'm a little helpless.
Zero clue if this helps, but here is my code for sending data from Raspberry Pi Python to AWS - Parse Server using base64/JSON. The only reason I use base64 is to send pictures. You should only have to use JSON to send your other data.
import requests
import random, time
import math
import json
import Adafruit_DHT
import base64
from Adafruit_CCS811 import Adafruit_CCS811
from picamera import PiCamera
from time import sleep
DHT_SENSOR = Adafruit_DHT.DHT22
DHT_PIN =4
ccs = Adafruit_CCS811()
camera = PiCamera()
while True:
time.sleep(5)
camera.start_preview()
sleep(5)
camera.capture('/home/pi/Desktop/image.jpg')
camera.stop_preview()
with open('/home/pi/Desktop/image.jpg', 'rb') as binary_file:
binary_file_data = binary_file.read()
base64_encoded_data = base64.b64encode(binary_file_data)
base64_message = base64_encoded_data.decode('utf-8')
humidity, temperature = Adafruit_DHT.read_retry(DHT_SENSOR, DHT_PIN)
ccs.readData()
parseServer = {
"temp": temperature,
"humid": humidity,
"co2": ccs.geteCO2(),
"pic": base64_message
}
resultJSON = json.dumps(parseServer)
headers = {
'X-Parse-Application-Id': 'myappID',
'Content-Type': 'application/json',
}
data = resultJSON
response =
requests.put('http://1.11.111.1111/parse/classes/Gamefuck/TIuRnws3Ag',
headers=headers, data=data)
print(data)
If you're using the Python SDK for Azure IoT, sending the message as UTF-8 encoded JSON is as easy as setting two properties on your message object. There is a good example here
msg.content_encoding = "utf-8"
msg.content_type = "application/json"
Furthermore, you don't need to change anything in IoT Hub for this. This message setting is a prerequisite to be able to do message routing based on the body of the message.
I am making an http request to an API gateway wrapping a lambda.
I have a query string parameter which is like this :
LG - 50" Class (49.5" Diag.) - LED - 2160p - Smart - 4K Ultra HD TV - Black
I tried to encodeURIComponent and pass it to my http request call. I get this error
body: '{"message": "Could not parse request body into json: Unexpected character (\\\'C\\\' (code 67)): was expecting comma to separate OBJECT entries\\n at [Source: [B#5ebb26a9; line: 1, column: 28]"}' }
If I remove the double quotes it works fine.
The error message is thrown while your Lambda function tries to parse it. Can you check API Gateway's CloudWatch logs (easy to check in test invoke console) to make sure that the "Endpoint request body after transformations" is as your Lambda function expects?
I am working with Node-RED in Bluemix for IoT.
How can I parse out the individual pieces of information (like the cmdmsg and the tempr) in a function node so I can use it in other nodes in the flow?
I'm getting an error when I try (see below)
I am receiving the JSON complete message object (from an IoT in Node) that I see in my "debug" node when I set it to look at the complete message object. see the object below.
It appears to me that the JSON is formatted correctly.
I tried putting the following in the function node, but I'm getting an error that says "TypeError: Cannot read property 'tempr' of undefined"
Here is what the function parameter is:
return {payload:msg.payload.d.tempr};
and here is the message object
{
"topic": "iot-2/type/Arduino-tempsensor/id/FFFFFFFFFFFF/evt/status/fmt/json",
"payload": "{\n\"d\": {\n\"myName\": \"Arduino CF\",\n\"cmdmsg\": \"Weekly\",\n\"tempr\": -3,\n}\n}",
"deviceId": "FFFFFFFFFFFF",
"deviceType": "Arduino-tempsensor",
"eventType": "status",
"format": "json",
"_msgid": "ffffffff.55555"
}
note: I obfuscated the device ID (mac address) and msgid
Any ideas on how to parse the data out and why I'm getting an error?
Sorry, but your JSON Payload is completely messed up, it should look like this: {"d": {"myName": "Arduino CF","cmdmsg": "Weekly","tempr": -3}}
You shouldn't see any \or \nin the payload, they look like escape characters from the client side. I also believe that the last comma after the tempr value shouldn't be there for valid JSON.
I am not an Arduino expert but I have experimented with a Raspberry Pi and the Mosquitto client, this is how I can successfully send an event to IoTF:
mosquitto_pub -h <org>.messaging.internetofthings.ibmcloud.com -p 1883 -u "use-token-auth" -P "<token>" -i d:<org>:raspi:raspi2 -t iot-2/evt/message/fmt/json -m {"d":{"text":"Hello World"}}
If the paylod is correct JSON your statement return {payload:msg.payload.d.tempr}; will work.
Have you seen this: http://www.ibm.com/developerworks/cloud/library/cl-bluemix-arduino-iot2/
The JSON string you are showing should be converted to a Javascript object before you try to access its fields. To do that is as simple as wiring the arduino output to a "JSON" node, which does the conversion for you (or throws an error if the string is not valid JSON).
Wire the output of the JSON node to a debug node, if you want to see the structure of the msg object. You can also wire it to a "change" node, if you simply want to replace the msg.payload with the temperature value, for instance. You don't need any custom javascript code in a function node to do simple changes like that.
Here is a sample flow that you can import... the arduino output string is simulated by pasting your payload into a "template" node:
[{"id":"1a79abfe.b8abb4","type":"inject","z":"58c8eb7a.5496c4","name":"send output","topic":"iot-2/type/Arduino-tempsensor/id/FFFFFFFFFFFF/evt/status/fmt/json","payload":"true","payloadType":"bool","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":170,"y":2740,"wires":[["9fc678fb.ae18e8"]]},{"id":"69e91778.e0c6e8","type":"json","z":"58c8eb7a.5496c4","name":"","property":"payload","action":"","pretty":false,"x":390,"y":2800,"wires":[["d066800f.60a9b","cf991eb1.f2a1a"]]},{"id":"9d8d7da2.2a7da","type":"debug","z":"58c8eb7a.5496c4","name":"msg string","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","x":610,"y":2740,"wires":[]},{"id":"9fc678fb.ae18e8","type":"template","z":"58c8eb7a.5496c4","name":"arduino string","field":"payload","fieldType":"msg","format":"json","syntax":"plain","template":"{\n \"d\": {\n \"myName\": \"Arduino CF\",\n \"cmdmsg\": \"Weekly\",\n \"tempr\": -3\n }\n}","output":"str","x":360,"y":2740,"wires":[["69e91778.e0c6e8","9d8d7da2.2a7da"]]},{"id":"d066800f.60a9b","type":"change","z":"58c8eb7a.5496c4","name":"extract tempr","rules":[{"t":"set","p":"payload","pt":"msg","to":"payload.d.tempr","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":450,"y":2860,"wires":[["af730d72.2995a"]]},{"id":"af730d72.2995a","type":"debug","z":"58c8eb7a.5496c4","name":"tempr","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":630,"y":2860,"wires":[]},{"id":"cf991eb1.f2a1a","type":"debug","z":"58c8eb7a.5496c4","name":"msg object","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","x":610,"y":2800,"wires":[]}]
As was noted, your original JSON string was not valid, because of the trailing comma -- but it is ok to have double-quotes and newlines as long as they are escaped with a "\". When the string is parsed, they are stripped out anyway.
I am running into some kind of trouble communicating with a restful service from my .Net WPF application (.Net 4.5), in particular when sending a "PUT" request with some json data.
FYI: The restful service is running under Python Flask.
The method I use the following method to send request to the restful service:
HttpClient http = new HttpClient();
http.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", encodedCredentials);
http.Timeout = TimeSpan.FromSeconds(1);
// Add an Accept header for JSON format.
http.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
HttpContent content = new StringContent(jDataString, Encoding.UTF8, "application/json");
When I submit usual string, all works just fine. But I am having trouble as soon as the string contains line breaks.
Using:
mytring.Replace("\r", "").Replace("\n", "")
works, my string is then accepted by the restful service.
Unfortunately, this is not acceptable, because I want to be able to retrieve line breaks.
I hence tried approaches like:
mytring.Replace("\n", "\\n").Replace("\r", "\\r")
or even with a inside-character to make sure I recognize the pattern:
mytring.Replace("\n", "\\+n").Replace("\r", "\\+r")
In both cases, my parsed string looks fine, but isn't accepted by the restful service.
Below two examples - the first version is accepted, not the second and third...
"XML_FIELD": "<IDs><Id Type=\"System.Int32\" Value=\"7\" /></IDs>"
"XML_FIELD": "<IDs>\r\n<Id Type=\"System.Int32\" Value=\"20\" />\r\n</IDs>"
"XML_FIELD": "<IDs>\+r\+n<Id Type=\"System.Int32\" Value=\"20\" />\+r\+n</IDs>"
Thanks in advance for the help!!
Regards!
Ok, got it...
The issue was coming from "\r\n" character which were coming directly from my DB...
Anyway, change to perform is for SERIALIZATION is
mySerializedString.Replace("\r\n", "\n")
.Replace("\n", "\\n")
.Replace("\r", "\\r")
.Replace("\'", "\\'")
.Replace("\"", "\\\"")
.Replace("\t", "\\t")
.Replace("\b", "\\b")
.Replace("\f", "\\f");
And to de-serialize do the inverse:
myDeSerializedString.Replace("\\n", "\n")
.Replace("\\r", "\r")
.Replace("\\'", "\'")
.Replace("\\\"", "\"")
.Replace("\\t", "\t")
.Replace("\\b", "\b")
.Replace("\\f", "\f");
NOTE: in the process, we loose "\r\n" characters (which are replaced by "\n").
I'm trying to post a json data via Arduino.When ı'm trying to this code.ı will send a json data with QueryString.If ı try this code the server answer me with Wrong QueryString format.Which mean is ı'm connected to server and server got my data.
if (client.connect(server, 80)) {
Serial.println("connected");
// Make a HTTP request:
client.println("POST /URL?query=jsondata HTTP/1.1");
client.println("Host: **.**.**.**");
client.println("Connection: close\r\nContent-Type: application/x-www-form-urlencoded");
client.print("Content-Length: ");
client.println(PostData.length());
client.println();
client.println(PostData);
}
But My Main Plan is send my json data with querystring.If ı Try this code ;
client.println("POST /URL?query={request:{Header:{Username:kullaniciAdi,Password:123456},Item:{Serial:ABC123QWE,Data:100, DateOn:23/11/1986 15:45:24}}} HTTP/1.1");
I get a HTTP Error 400. The request is badly formed.
Anyone Has a any idea?
Yes, your URI contains spaces and may contain other characters to confuse the format of the post request. You need to encode these characters.
As far as I can tell, the Arduino standard libraries do not include any form of urlEncode method, which is common in other languages and libraries, so you will either have to create your own or look for one.
Your resulting code would be something like:
String request = "/URL?query={request:{Header:{Username:kullaniciAdi,Password:123456},Item:{Serial:ABC123QWE,Data:100, DateOn:23/11/1986 15:45:24}}}";
String encRequest = uriEncode(request); // need to write your own method for this...
String post = "POST " + encRequest + " HTTP/1.1");
client.println( post);
Some discussion on creating a uriEncode function is on the Arduino Forum and there also appears to be a working method on hardwarefun.com