arduino ide esp8266 json decode from URL - json

I program esp8266 using the Arduino IDE. I join with the local network via WIFI. Ultimately, I want to download content JSON generated on the local server. I am using the code:
#include <ESP8266WiFi.h>
#include <ArduinoJson.h>
const char* ssid = "NeoRaf";
const char* password = "password";
const char* host = "192.168.1.8";
void setup() {
Serial.begin(115200);
delay(100);
// We start by connecting to a WiFi network
Serial.println();
Serial.println();
Serial.print("Connecting to ");
Serial.println(ssid);
WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
Serial.println("");
Serial.println("WiFi connected");
Serial.println("IP address: ");
Serial.println(WiFi.localIP());
//-----------------------
delay(5000);
Serial.print("connecting to ");
Serial.println(host);
// Use WiFiClient class to create TCP connections
WiFiClient client;
const int httpPort = 8095;
if (!client.connect(host, httpPort)) {
Serial.println("connection failed");
return;
}
// We now create a URI for the request
String url = "/api";
Serial.print("Requesting URL: ");
Serial.println(url);
// This will send the request to the server
client.print("GET " + url + " HTTP/1.1\r\n" +
"Host: " + host + "\r\n" +
"ApiCode: 8273\r\n" +
"Content-Type: application/json\r\n" +
"Connection: close\r\n\r\n");
delay(500);
char c[1024];
// Read all the lines of the reply from server and print them to Serial
while(client.available()){
c[0] = client.read();
//Serial.print(c);
Serial.print(c);
}
StaticJsonBuffer<200> jsonBuffer;
JsonObject& root = jsonBuffer.parseObject(c);
int data = root["lowVersion"];
Serial.println();
Serial.print(data);
Serial.println();
Serial.println("closing connection");
}
void loop()
{}
The result is this:
Connecting to NeoRaf
......
WiFi connected
IP address:
192.168.1.21
connecting to 192.168.1.8
Requesting URL: /api
HTTP/1.1 200 OK
Content-Length: 32
Content-Type: application/json
Server: Microsoft-HTTPAPI/2.0
Access-Control-Allow-Origin: *
Date: Mon, 06 Jun 2016 16:50:48 GMT
Connection: close
{"lowVersion":1,"highVersion":3}
0
closing connection
This line is JSON:
{"lowVersion":1,"highVersion":3}
So it should display 1 and displays 0.
I do not know how to get rid of the header:
HTTP/1.1 200 OK
Content-Length: 32
Content-Type: application/json
Server: Microsoft-HTTPAPI/2.0
Access-Control-Allow-Origin: *
Date: Mon, 06 Jun 2016 16:50:48 GMT
and how to read the contents of JSON ( lowVersion or highVersion)?

You need to detect when the payload starts in the HTTP response. So, when you read data from the client you can skip all HTTP header lines. Incidentally in a JSON response the first { usually signifies the start of the JSON document.
httpbin example
Requesting http://httpbin.org/get yields
{
"args": {},
"headers": {
...
},
"origin": "XXXXXXX",
"url": "http://httpbin.org/get"
}
You can print the "url" field like so:
String json = "";
boolean httpBody = false;
while (client.available()) {
String line = client.readStringUntil('\r');
if (!httpBody && line.charAt(1) == '{') {
httpBody = true;
}
if (httpBody) {
json += line;
}
}
StaticJsonBuffer<400> jsonBuffer;
Serial.println("Got data:");
Serial.println(json);
JsonObject& root = jsonBuffer.parseObject(json);
String data = root["url"];

HTTP headers always end with an empty line, so you just need to read until you find two consecutive line breaks.
Have a look at ArduinoJson's example JsonHttpClient.ino, it uses Stream::find():
// Skip HTTP headers so that we are at the beginning of the response's body
bool skipResponseHeaders() {
// HTTP headers end with an empty line
char endOfHeaders[] = "\r\n\r\n";
client.setTimeout(HTTP_TIMEOUT);
bool ok = client.find(endOfHeaders);
if (!ok) {
Serial.println("No response or invalid response!");
}
return ok;
}

First of all, for handling HTTP requests, you can use a RestClient library rather than writing all the low level requests. It saves a lot of time and is less error-prone.
For example, for a GET request, all you have to do is:
String response = "";
int statusCode = client.get("/", &response);
One good such library with SSL support is written by github user DaKaz.
You can use it for your GET request. The returned response will be without the HTTP header. The function will return the response from the server without the headers.
Now you can use the ArduinoJson Library by bblanchin for decoding the JSON object.
Details can be seen here.
Or you can do plain string manipuation to get the values though it is not a recommended route to take and is prone to errors.

Related

Wrong format of JSON data (Arduino) to .net Azure App service, getting code 415

I would like to ask about sending data from arduino/esp to .net App service running on Azure.
I am sending JSON to API with "list" of measurements (simple JSON, Id and Value).
Over PostMan I don't have any issue, but over Arduino I am facing issue that I receive result code 415.
Data are sent in JSON.
Arduino Code:
#include <ESP8266WiFi.h>
#include <ESP8266HTTPClient.h>
#include <WiFiClient.h>
const char* ssid = "*******";
const char* password = "*******";
//Your Domain name with URL path or IP address with path
const char* serverName = "https://********.azurewebsites.net/api/Measurements/";
unsigned long lastTime = 0;
// Set timer to 5 seconds (5000)
unsigned long timerDelay = 5000;
void setup() {
Serial.begin(115200);
WiFi.begin(ssid, password);
Serial.println("Connecting");
while(WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
Serial.println("");
Serial.print("Connected to WiFi network with IP Address: ");
Serial.println(WiFi.localIP());
Serial.println("Timer set to 5 seconds (timerDelay variable), it will take 5 seconds before publishing the first reading.");
}
void loop() {
//Send an HTTP POST request every 10 minutes
if ((millis() - lastTime) > timerDelay) {
//Check WiFi connection status
if(WiFi.status()== WL_CONNECTED)
{
std::unique_ptr<BearSSL::WiFiClientSecure>client(new BearSSL::WiFiClientSecure);
client->setInsecure();
HTTPClient https;
https.addHeader("Content-Type", "application/json"); // tried even "text/json"
//https.addHeader("Content-Length", "32");
https.addHeader("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
https.addHeader("Host","******.azurewebsites.net");
https.addHeader("Cache-Control","no-cache");
String httpRequestData = "[{\"SensorId\": 1, \"Value\": 77.7}]";
if (https.begin(*client, serverName))
{
Serial.print("[HTTPS] POST...\n");
int httpCode = https.POST(httpRequestData);
if (httpCode > 0)
{
// HTTP header has been send and Server response header has been handled
Serial.printf("[HTTPS] POST... code: %d\n", httpCode);
// file found at server
if (httpCode == HTTP_CODE_OK || httpCode == HTTP_CODE_MOVED_PERMANENTLY)
{
String payload = https.getString();
Serial.println(payload);
}
else
{
Serial.printf("[HTTPS] POST... failed, error: %s\n", https.errorToString(httpCode).c_str());
}
}
else
{
Serial.printf("[HTTPS] POST... failed, error: %s\n", https.errorToString(httpCode).c_str());
}
https.end();
}
else
{
Serial.printf("[HTTPS] Unable to connect\n");
}
}
lastTime = millis();
}
}
Not receiving any error, just response code 415.
The HTTP 415 Unsupported Media Type client error response code indicates that the server refuses to accept the request because the payload
JSON text:
[{"SensorId": 1, "Value": 77.7}]
AppInsight:
PostMan successful request:
body of request in Postman:
And report from AppInsight:
App service:
Over PostMan I am able to reach azure and send data, but over Arduino I am not able to send data (I am able to get data, but not post)
Probably some beginner mistake.
Can someone give me hint?
Do I have wrong format of JSON?

Parsing a JSON document in an HTTP response-Arduino (Unexpected response: HTTP/1.1 301 moved permanently error)

I'm using Arduino's ethernet shield with Arduino mega to parse a JSON document in a HTTP response to read some values in the JSON file.
The JSON address is: https://api.coinbase.com/v2/prices/BTC-USD/spot.
I'm getting this error message:
Connecting...
Connected!
Unexpected response: HTTP/1.1 301 Moved Permanently
Is is that the the Arduino library doesn't have SSL capability, so it can't connect to a secure server? If it is, then I would need your help to resolve this issue.
Or perhaps I need to use another port that's related to https?
Here is my code:
#include <ArduinoJson.h>
#include <Ethernet.h>
#include <SPI.h>
void setup() {
// Initialize Serial port
Serial.begin(9600);
while (!Serial) continue;
// Initialize Ethernet library
byte mac[] = {0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED};
if (!Ethernet.begin(mac)) {
Serial.println(F("Failed to configure Ethernet"));
return;
}
delay(1000);
Serial.println(F("Connecting..."));
// Connect to HTTP server
EthernetClient client;
client.setTimeout(10000);
if (!client.connect("api.coinbase.com", 80)) {
Serial.println(F("Connection failed"));
return;
}
Serial.println(F("Connected!"));
// Send HTTP request
client.println(F("GET /v2/prices/BTC-USD/spot/response.json HTTP/1.0"));
client.println(F("Host: api.coinbase.com"));
client.println(F("Connection: close"));
if (client.println() == 0) {
Serial.println(F("Failed to send request"));
return;
}
// Check HTTP status
char status[32] = {0};
client.readBytesUntil('\r', status, sizeof(status));
// It should be "HTTP/1.0 200 OK" or "HTTP/1.1 200 OK"
if (strcmp(status + 9, "200 OK") != 0) {
Serial.print(F("Unexpected response: "));
Serial.println(status);
return;
}
// Skip HTTP headers
char endOfHeaders[] = "\r\n\r\n";
if (!client.find(endOfHeaders)) {
Serial.println(F("Invalid response"));
return;
// the rest of the code related to parsing that I didn't paste for the sake the argument.
Thanks everybody.

Multiple HTTP GETs in one go on ESP8266

I can't figure out how to send the data in one go. I am asking because sending it simultaneously delays a bit.
I did see an other topic of someone posting a piece of code, but that did not work.
This is a bit of the code:
void handleRoot() {
String s = MAIN_page; // Read HTML contents
server.send(200, "text/html", s); // Send web page
}
void handleFORESTTEMPERATURE() {
String ForrestTemperatureDev = String(ForestTemperature, 1);
server.send(200, "text/plane", ForestTemperatureDev); // Send ADC value only to client ajax request
}
void handleFORESTPRESSURE() {
String ForrestPressureDev = String(ForestPressure);
server.send(200, "text/plane", ForrestPressureDev); // Send ADC value only to client ajax request
}
void handleFORESTHUMIDITY() {
String ForrestHumidityDev = String(ForestHumidity);
server.send(200, "text/plane", ForestHumidityDev); // Send ADC value only to client ajax request
}
server.on("/readFORESTPRESSURE", handleFORESTPRESSURE);
server.on("/readFORESTTEMPERATURE", handleFORESTTEMPERATURE);
server.on("/readFORESTHUMIDITY", handleFORESTHUMIDITY);
server.on("/readFORESTWEATHERSTATUS", handleFORESTWEATHERSTATUS);
The following code would send all three values separated with a ; when \readALL is requested.
The three values can be split on the ; in your receiving application; using JSON would be just a little bit more work.
Note1: Untested, I just typed it in, but you get the idea.
Note2: It's text/plain, not text/plane.
void handleRoot() {
String s = MAIN_page; //Read HTML contents
server.send(200, "text/html", s); //Send web page
}
void handleFORESTTEMPERATURE() {
String ForrestTemperatureDev = String(ForestTemperature, 1);
server.send(200, "text/plain", ForestTemperatureDev); //Send ADC value only to client ajax request
}
void handleFORESTPRESSURE() {
String ForrestPressureDev = String(ForestPressure);
server.send(200, "text/plain", ForrestPressureDev); //Send ADC value only to client ajax request
}
void handleFORESTHUMIDITY() {
String ForrestHumidityDev = String(ForestHumidity);
server.send(200, "text/plain", ForestHumidityDev); //Send ADC value only to client ajax request
}
void handleALL() {
String AllDev = String(ForestTemperature, 1) + ";" + String(ForestPressure) + ";" + String(ForestHumidity);
server.send(200, "text/plain", AllDev); //Send ADC values only to client ajax request
}
server.on("/readFORESTPRESSURE", handleFORESTPRESSURE);
server.on("/readFORESTTEMPERATURE", handleFORESTTEMPERATURE);
server.on("/readFORESTHUMIDITY", handleFORESTHUMIDITY);
server.on("/readFORESTWEATHERSTATUS", handleFORESTWEATHERSTATUS);
server.on("/readALL", handleALL);

StatusCode: 404, ReasonPhrase: 'Not Found', Version: 1.1, Content: System.Net.Http.HttpConnectionResponseContent,

{StatusCode: 404, ReasonPhrase: 'Not Found', Version: 1.1, Content: System.Net.Http.HttpConnectionResponseContent, Headers:
{
Cache-Control: private
Server: Microsoft-IIS/10.0
X-Powered-By: ASP.NET
Date: Sat, 06 Jun 2020 02:26:17 GMT
Connection: close
Content-Type: text/html; charset=utf-8
Content-Length: 5059
}}
when calling a post method in api producing the response message without hitting the api,we are using stringcontent to pass input parameters as a serialized json which has 200k objects. our basic understanding is that its because content exceeds limited length. we are using .net core 3.1. how to increase the length of maximum allowed? we already tried placing [RequestSizeLimit(long.MaxValue)] on controller level. our web api call
[Route("~/api/Controller/somemethod")]
[HttpPost]
public string somemethod(List<DataMaster> Lists)
our api calling method
url = "https://localhost:44339/" + url;
HttpContent httpContent = new StringContent(inputParams, Encoding.UTF8, "application/json");
HttpRequestMessage request = new HttpRequestMessage
{
Method = new HttpMethod(method),
RequestUri = new Uri(url),
Content = httpContent
};
HttpClientHandler clientHandler = new HttpClientHandler();
clientHandler.ServerCertificateCustomValidationCallback = (sender, cert, chain, sslPolicyErrors) => { return true; };
HttpClient client = new HttpClient(clientHandler);
return client.SendAsync(request).Result;
You can set the MaxRequestBodySize (docs):
Gets or sets the maximum allowed size of any request body in bytes. When set to null, the maximum request length will not be restricted in ASP.NET Core. However, the IIS maxAllowedContentLength will still restrict content length requests (30,000,000 by default). This limit has no effect on upgraded connections which are always unlimited. This can be overridden per-request via IHttpMaxRequestBodySizeFeature.
services.Configure<IISServerOptions>(options =>
{
options.MaxRequestBodySize = 2147483647;
});

How to send limited json data (stored in SPIFF) to a local server from esp8266?

An example code:
#include <FS.h> //this needs to be first, or it all crashes and burns...
#include <ESP8266WiFi.h> //https://github.com/esp8266/Arduino
//needed for library
#include <DNSServer.h>
#include <ESP8266WebServer.h>
#include <WiFiManager.h> //https://github.com/tzapu/WiFiManager
#include <ArduinoJson.h> //https://github.com/bblanchon/ArduinoJson
//define your default values here, if there are different values in config.json, they are overwritten.
char mqtt_server[40];
char mqtt_port[6] = "8080";
char blynk_token[34] = "YOUR_BLYNK_TOKEN";
//flag for saving data
bool shouldSaveConfig = false;
//callback notifying us of the need to save config
void saveConfigCallback () {
Serial.println("Should save config");
shouldSaveConfig = true;
}
void setup() {
// put your setup code here, to run once:
Serial.begin(115200);
Serial.println();
//clean FS, for testing
//SPIFFS.format();
//read configuration from FS json
Serial.println("mounting FS...");
if (SPIFFS.begin()) {
Serial.println("mounted file system");
if (SPIFFS.exists("/config.json")) {
//file exists, reading and loading
Serial.println("reading config file");
File configFile = SPIFFS.open("/config.json", "r");
if (configFile) {
Serial.println("opened config file");
size_t size = configFile.size();
// Allocate a buffer to store contents of the file.
std::unique_ptr<char[]> buf(new char[size]);
configFile.readBytes(buf.get(), size);
DynamicJsonBuffer jsonBuffer;
JsonObject& json = jsonBuffer.parseObject(buf.get());
json.printTo(Serial);
if (json.success()) {
Serial.println("\nparsed json");
strcpy(mqtt_server, json["mqtt_server"]);
strcpy(mqtt_port, json["mqtt_port"]);
strcpy(blynk_token, json["blynk_token"]);
} else {
Serial.println("failed to load json config");
}
}
}
} else {
Serial.println("failed to mount FS");
}
//end read
// The extra parameters to be configured (can be either global or just in the setup)
// After connecting, parameter.getValue() will get you the configured value
// id/name placeholder/prompt default length
WiFiManagerParameter custom_mqtt_server("server", "mqtt server", mqtt_server, 40);
WiFiManagerParameter custom_mqtt_port("port", "mqtt port", mqtt_port, 5);
WiFiManagerParameter custom_blynk_token("blynk", "blynk token", blynk_token, 32);
//WiFiManager
//Local intialization. Once its business is done, there is no need to keep it around
WiFiManager wifiManager;
//set config save notify callback
wifiManager.setSaveConfigCallback(saveConfigCallback);
//set static ip
wifiManager.setSTAStaticIPConfig(IPAddress(10,0,1,99), IPAddress(10,0,1,1), IPAddress(255,255,255,0));
//add all your parameters here
wifiManager.addParameter(&custom_mqtt_server);
wifiManager.addParameter(&custom_mqtt_port);
wifiManager.addParameter(&custom_blynk_token);
//reset settings - for testing
//wifiManager.resetSettings();
//set minimu quality of signal so it ignores AP's under that quality
//defaults to 8%
//wifiManager.setMinimumSignalQuality();
//sets timeout until configuration portal gets turned off
//useful to make it all retry or go to sleep
//in seconds
//wifiManager.setTimeout(120);
//fetches ssid and pass and tries to connect
//if it does not connect it starts an access point with the specified name
//here "AutoConnectAP"
//and goes into a blocking loop awaiting configuration
if (!wifiManager.autoConnect("AutoConnectAP", "password")) {
Serial.println("failed to connect and hit timeout");
delay(3000);
//reset and try again, or maybe put it to deep sleep
ESP.reset();
delay(5000);
}
//if you get here you have connected to the WiFi
Serial.println("connected...yeey :)");
//read updated parameters
strcpy(mqtt_server, custom_mqtt_server.getValue());
strcpy(mqtt_port, custom_mqtt_port.getValue());
strcpy(blynk_token, custom_blynk_token.getValue());
//save the custom parameters to FS
if (shouldSaveConfig) {
Serial.println("saving config");
DynamicJsonBuffer jsonBuffer;
JsonObject& json = jsonBuffer.createObject();
json["mqtt_server"] = mqtt_server;
json["mqtt_port"] = mqtt_port;
json["blynk_token"] = blynk_token;
File configFile = SPIFFS.open("/config.json", "w");
if (!configFile) {
Serial.println("failed to open config file for writing");
}
json.printTo(Serial);
json.printTo(configFile);
configFile.close();
//end save
}
Serial.println("local ip");
Serial.println(WiFi.localIP());
}
void loop() {
// put your main code here, to run repeatedly:
}
I got this code from here (https://techtutorialsx.com/2017/01/08/esp8266-posting-json-data-to-a-flask-server-on-the-cloud/):
HTTPClient http; //Declare object of class HTTPClient
http.begin("http://anteph.pythonanywhere.com/postjson"); //Specify request destination
http.addHeader("Content-Type", "application/json"); //Specify content-type header
int httpCode = http.POST(JSONmessageBuffer); //Send the request
String payload = http.getString(); //Get the response payload
Serial.println(httpCode); //Print HTTP return code
Serial.println(payload); //Print request response payload
http.end(); //Close connection
But If I use:
int httpCode = http.POST(JSONmessageBuffer); //Send the request
This will send all the json data (i.e mqtt_server, mqtt_port, blynk_token). But I want to send only "blynk_token" and not the rest json data to server, So how can I achieve this?
Please suggest.
So to answer your question regarding "how to read and write limited Json data from SPIFF using DynamicJsonBuffer" you can take a look at your code right here where you read in the data.
if (configFile) {
Serial.println("opened config file");
size_t size = configFile.size();
// Allocate a buffer to store contents of the file.
std::unique_ptr<char[]> buf(new char[size]);
configFile.readBytes(buf.get(), size);
DynamicJsonBuffer jsonBuffer;
JsonObject& json = jsonBuffer.parseObject(buf.get());
json.printTo(Serial);
if (json.success()) {
Serial.println("\nparsed json");
strcpy(mqtt_server, json["mqtt_server"]); // copy mqtt_server value
strcpy(mqtt_port, json["mqtt_port"]); // copy mqtt_port value
strcpy(blynk_token, json["blynk_token"]); // copy blynk_token value
} else {
Serial.println("failed to load json config");
}
}
after this piece of code you have your 3 variables filled up with values.
These are mqtt_server, mqtt_port and blynk_token, which now contain the values read from the json which was saved in SPIFF.
If you then want to send the blynk_token to your local server you will have to put only that varialbe in your JSONmessageBuffer.
like so:
StaticJsonBuffer<300> JSONbuffer; //Declaring static JSON buffer
JsonObject& JSONencoder = JSONbuffer.createObject();
JSONencoder["blynk_token"] = blynk_token; // add blynk_token to the values sent to the server
char JSONmessageBuffer[300];
JSONencoder.prettyPrintTo(JSONmessageBuffer, sizeof(JSONmessageBuffer));
Serial.println(JSONmessageBuffer);
HTTPClient http; //Declare object of class HTTPClient
http.begin("http://anteph.pythonanywhere.com/postjson"); //Specify request destination
http.addHeader("Content-Type", "application/json"); //Specify content-type header
int httpCode = http.POST(JSONmessageBuffer); //Send the request
String payload = http.getString(); //Get the response payload
Serial.println(httpCode); //Print HTTP return code
Serial.println(payload); //Print request response payload
http.end(); //Close connection
this line JSONencoder["blynk_token"] = blynk_token; is your way to go. Here you say which values you want to sent to your server.
This tutorial might also help you :)