I'm currently working in C++, getting an HTTP response from a request that I write into a .txt file using ostream. This happens asynchronously and I don't want to change this.
Once the data is done being written, I want to read from the file
{"data":{"request":[{"type":"City","query":"London, United Kingdom"}],"weather":[{"date":"2013-04-21","astronomy".....
~somehow~ prettify the string using either an outside library like nlohmann/json or other(?) and then
a)print it to the console and
b) save it in a different file (pretty.json)
I am having trouble understanding which method to use from:
https://github.com/nlohmann/json
Any ideas how to approach this?
I was thinking getting the file line by line until I hit EOF into a sort of "buffer" and then running _json on that and saving the solution which can be displayed on the console...
My code so far
#include <cpprest/http_client.h>
#include <cpprest/filestream.h>
#include <iostream>
#include <sstream>
#include "json.hpp"
using namespace utility; // string conversion
using namespace web; // URI
using namespace web::http; // HTTP commands
using namespace web::http::client; // HTTP Client features
using namespace concurrency::streams; // Asynch streams, like Node
using json = nlohmann::json;
int main()
{
auto fileStream = std::make_shared<ostream>();
// Open stream to output file.
pplx::task<void> requestTask = fstream::open_ostream(U("results.txt"))
.then([=](ostream outFile)
{
*fileStream = outFile;
http_client client //gets the info
return client.request(methods::GET, stringBuilder.to_string());
})
.then([=](http_response response) // set up response handler
{
printf("Received response status code:%u\n", response.status_code());
return response.body().read_to_end(fileStream->streambuf());
})
.then([=](size_t) // close file stream
{
return fileStream->close();
})
.then([=]()
{
nlohmann::json j;
std::ifstream i;
i.open("results.txt"); // ?? <<< === this is where my question is
});
// Wait for all the outstanding I/O to complete, handle exceptions
try
{
requestTask.wait();
}
catch (const std::exception &e)
{
printf("Error exception:%s\n", e.what());
}
return 0;
}
SOLUTION:
.then([=]()
{
// read a JSON file
std::ifstream readFromFile("results.txt");
if (readFromFile.is_open()) {
nlohmann::json j;
readFromFile >> j;
// write prettified JSON to another file
std::ofstream writeToFile("pretty.json");
writeToFile << std::setw(4) << j << std::endl;
readFromFile.close();
writeToFile.close();
}
else {
std::cout << "unable to open file";
}
});
You have two choices to prettify with nlohmann.
Uses dump which produces a string
int indent = 4;
nlohmann::json data;
data.dump(indent);
Or use the stream output overload with field width set
std::ofstream o("pretty.json");
o << std::setw(4) << data << std::endl;
Related
Thank you for considering this problem.
I'm streaming a small json from a Web socket and can see the stringified json arrive to the client because it prints to the serial monitor, but then it deserializes to a 1 or 0 instead of my key:value pairs. I just want it to parse the json so that the rest of my program can use the values. I get no errors. Tried both Dynamic and Static json docs. Tried triple the memory requirement.
Arduino:
#include <WiFi.h>
#define ARDUINOJSON_ENABLE_ARDUINO_STREAM 1
#include <ArduinoJson.h>
#include <StreamUtils.h>
const char* ssid = "ssid";
const char* password = "pw";
const char* host = "10.0.0.250";
void setup()
{
Serial.begin(115200);
delay(10);
// We start by connecting to a WiFi network
int loopCount = 0;
StaticJsonDocument<384> doc;
DeserializationError error;
void loop()
{
//delay(5000);
++loopCount;
if (loopCount > 1) return;
Serial.print("connecting to ");
Serial.println(host);
// Use WiFiClient class to create TCP connections
WiFiClient client;
const int httpPort = 1337;
if (!client.connect(host, httpPort)) {
Serial.println("connection failed");
return;
}
// This will send the request to the server
client.print(String("GET ") + "HTTP/1.1\r\n" +
"Host: " + host + "\r\n" +
"Connection: close\r\n\r\n");
unsigned long timeout = millis();
while (client.available() == 0) {
if (millis() - timeout > 5000) {
Serial.println(">>> Client Timeout !");
client.stop();
return;
}
}
// Read all the lines of the reply from server and print them to Serial
while (client.available() > 0) {
ReadLoggingStream loggingClient(client, Serial);
error = deserializeJson(doc, loggingClient);
}
Serial.println("");
if (error) {
Serial.print(F("deserializeJson() failed: "));
Serial.println(error.f_str());
return;
}
//this doesn't work
int id = doc["id"]; // Should be 5 but I get 0 for every value
Serial.print("id: "); Serial.println(id);
}
/*Serial monitor:
14:21:25.905 ->
07:16:36.574 -> WiFi connected
07:16:36.574 -> IP address:
07:16:36.574 -> 10.0.0.113
07:16:36.574 -> connecting to 10.0.0.250
07:16:36.849 -> "{\"id\":5,\"nom\":\"whynot\",\"delayStart\":200,\"rampePWM\":11,\"pulseWelding\":200,\"speedBalayage\":0.4,\"speedWelding\":0.5,\"speedWire\":1.1,\"balayage\":0.8,\"pulseWire\":5,\"retractWire\":7}"
07:16:36.849 -> id: 0
*/
The tcp-socket is in my node express setup. The file projet.json is only the json seen above ^^ no white space.
var net = require('net');
var serverN = net.createServer(function(socket) {
fs.readFile("./data/projet.json", 'utf-8', (err, data) => {
if (err) {
throw err;
}
socket.write(JSON.stringify(data));
socket.pipe(socket);
});
});
serverN.listen(1337, '10.0.0.250');
I can only show you how i use it to get the right values. i use a DynamicJsonDocument in my solution:
DynamicJsonDocument root(2048);
DeserializationError err = deserializeJson(root, http.getString());
String TravelTimes = root["travelTime"];
Otherwise you can also try to output the values directly via the jsonobject
JsonObject object = doc.to<JsonObject>();
const char* id = object["id"];
The code parses the JSON string but then calls JsonDocument::to<T>() to obtain a JsonObject. This method clears the document - from https://arduinojson.org/v6/api/jsondocument/to/
Clears the JsonDocument and converts it to the specified type.
JsonDocument::as<T>() should be used instead:
JsonObject object = doc.as<JsonObject>();
From https://arduinojson.org/v6/api/jsondocument/as/
Casts JsonDocument to the specified type.
Unlike JsonDocument::to(), this function doesn’t change the content of the JsonDocument.
You can also use serializeJsonPretty() to display the JsonObject on the serial output. Instead of:
JsonObject object = doc.to<JsonObject>();
Serial.println(object);
this can be done with:
serializeJsonPretty(doc, Serial);
Thanks to bblanchon of ArduinoJson - The node socket was stringifying the json twice. I changed the socket to socket.write(data) instead of socket.write(JSON.stringify(data)) and it works.
See full explanation here
https://github.com/bblanchon/ArduinoJson/issues/1507
Thanks again!
I'm sending an HTTP Post request on my Android App to my Wemos D1 mini pro and want to parse the incoming data (which is a json). My current code just prints out the whole POST request and I need to trim it so I only get the needed data. There are several examples out there but nothing matched my needs or worked at all.
#include <ESP8266WiFi.h>
#include <WiFiClient.h>
#include <ArduinoJson.h>
const char* ssid = "myssid";
const char* password = "mypassword";
char c;
String readString = String(100);
WiFiServer wifiServer(80);
void setup() {
Serial.begin(9600);
delay(1000);
WiFi.begin(ssid, password);
WiFi.mode(WIFI_STA);
while (WiFi.status() != WL_CONNECTED) {
delay(1000);
Serial.println("Connecting..");
}
Serial.print("Connected to WiFi. IP:");
Serial.println(WiFi.localIP());
wifiServer.begin();
}
//for parsing the actual JSON later
//you can ignore this at this moment because I don't even get the needed string to parse it from JSON
void handleReceivedMessage(String message){
StaticJsonBuffer<500> JSONBuffer; //Memory pool
JsonObject& parsed = JSONBuffer.parseObject(message); //Parse message
if (!parsed.success()) { //Check for errors in parsing
Serial.println("Parsing failed");
return;
}
const char * name3 = parsed["name"]; //Get name from HTTP
Serial.println("name3");
}
void loop() {
WiFiClient client = wifiServer.available();
if (client) {
Serial.println("Client connected");
while (client.connected()) {
while (client.available()>0) {
//instream from mobile device
char c = client.read();
if (readString.length() < 100) {
//store characters to string
readString.concat(c);
//Serial.print(c);
}
Serial.print(c);
//if HTTP request has ended
if (c == '\n') {
//Serial.println(readString);
delay(50);
//handleReceivedMessage(readString);
readString = "";
client.stop();
}
}}}}
Well first of all you seem to be using ArduinoJson lib version 5, now I could share the code I worked with and never failed me with version 5. But i'm going to encourage you to update the library to version 6 and share with you my piece of code.
I use this normally when I need to get information out of API's
DynamicJsonDocument doc(1024);
char* payload1 = (char*)malloc(http.getSize() + 1);
http.getString().toCharArray(payload1, http.getSize() + 1);
Serial.println(payload1);
http.end();
auto error = deserializeJson(doc, payload1);
free(payload1);
if (error) {
Serial.print(F("deserializeJson() failed with code "));
Serial.println(error.c_str());
return;
}
serializeJsonPretty(doc, Serial);
now as you can see, I'm using a getString method from httpClient lib in order to fill my char array and than parse it into json object (pretty much the same thing you was attempting, only difference is the memory pointers and Memory allocations.
Hopefully this will work with you.
I am having problems saving the JSON response using json cpprestsdk so that I can pass it back to the calling function. The parse() function seems to do a shallow copy rather than a deep copy. Can someone please help me with the method that I should be using to make a deep copy of the results returned?
Thanks in advance.
functionA()
{
web::json::value myJsonObjSv;
http_client client(L"http://url.com:8080/getdata");
return client.request(methods::GET).then([](http_response response) -> pplx::task<json::value> {
if(response.status_code() == status_codes::OK) {
return response.extract_json();
}
return pplx::task_from_result(json::value()); }).then([&myJsonObjSv](pplx::task<json::value> previousTask) {
try {
// capture json response to json_resp
json::value const resp = previousTask.get();
// Now save the JSON object to return from this function
myJsonObjSv = json::value::parse(resp.serialize());
// This statement displays the contents of myJsonObjSv to be the same as resp
wcout << "myJsonObjSv[" myJsonObjSv.serialize() << "]" << endl;
}
catch( const http_exception& e) {
// print error
}
});
// This statement to display the contents of myJsonObjSv crashes since there was a shallow copy made rather than a deep copy
wcout << "myJsonObjSv[" myJsonObjSv.serialize() << "]" << endl;
}
I finally found a path file QFile would accept using QFile.exist() and a healthy dose of trial and error.
I want to know why the following works:
#include <QFile>
#include <QByteArray>
#include <QJsonObject>
#include <QJsonDocument>
QString path = QDir::currentPath(); // Get current dir
path.append("/noteLibrary.json");
QFile file(path); // Give QFile current dir + path to file
if (!file.exists()) { // Check to see if QFile found the file at given file_path
qDebug() << "NO FILE HERE";
}
qDebug() << path; // See what path was finally successful
file.open(QIODevice::ReadOnly); // Continue parsing document to confirm everything else is functioning normally.
QByteArray rawData = file.readAll();
// Parse document
QJsonDocument doc(QJsonDocument::fromJson(rawData));
// Get JSON object
QJsonObject json = doc.object();
// Access properties
qDebug() << json["die"].toString(); // Should output "280C4"
Successful output:
"/home/pi/noteLibrary.json"
"280C4"
But the following does NOT work:
#include <QFile>
#include <QByteArray>
#include <QJsonObject>
#include <QJsonDocument>
QFile file("/home/pi/noteLibrary.json"); // Give QFile current dir + path to file
if (!file.exists()) { // Check to see if QFile found the file at given file_path
qDebug() << "NO FILE HERE";
}
//qDebug() << path; // See what path was finally successful
file.open(QIODevice::ReadOnly); // Continue parsing document to confirm everything else is functioning normally.
QByteArray rawData = file.readAll();
// Parse document
QJsonDocument doc(QJsonDocument::fromJson(rawData));
// Get JSON object
QJsonObject json = doc.object();
// Access properties
qDebug() << json["die"].toString(); // Should output "280C4"
Error output:
NO FILE HERE
QIODevice::read (QFile, "/home/pi/Desktop/noteLibrary.json"): device not open
""
Why would QFile treat these differently? Is this a QString format issue? Or is the fact that I'm deploying this remotely to a Raspberry Pi 3 possibly to blame?
Regardless of what was wrong with my code above, with the below code giving QFile the absolute path does work equally to creating a QString with currentPath(). I must have had something else wrong, my mistake!
noteLibrary.json
{"note": [{
"profile": "C4",
"die": "280C4",
"pressure": 800,
"position": 10000
},
{
"profile": "CC4",
"die": "2280C4",
"pressure": 8800,
"position": 110000
}
],
"test": {
"profile": "CCC4",
"die": "22280C4",
"pressure": 88800,
"position": 1110000
}
}
main.cpp excerpt
QFile file("/home/pi/noteLibrary.json");
if (!file.exists()) qDebug() << "NO FILE FOUND";
file.open(QIODevice::ReadOnly);
QByteArray rawData = file.readAll();
QJsonDocument doc(QJsonDocument::fromJson(rawData)); // Parse document
QJsonObject jObj = doc.object(); // Get JSON object
qDebug() << jObj["test"];
Application Output
QJsonValue(object,QJsonObject({"die":"22280C4","position":1110000,"pressure":88800,"profile":"CCC4"}))
Seems odd that it displays property values in alphabetical order, not the order listed in the document.
Suppose I have a double Eigen matrix and I want to write it to a csv file. I find the way of writing into a file in raw format but I need commas between entries. Here is the code I foudn for simple writing.
void writeToCSVfile(string name, MatrixXd matrix)
{
ofstream file(name.c_str());
if (file.is_open())
{
file << matrix << '\n';
//file << "m" << '\n' << colm(matrix) << '\n';
}
}
Using format is a bit more concise:
// define the format you want, you only need one instance of this...
const static IOFormat CSVFormat(StreamPrecision, DontAlignCols, ", ", "\n");
...
void writeToCSVfile(string name, MatrixXd matrix)
{
ofstream file(name.c_str());
file << matrix.format(CSVFormat);
}
Here is what I came up;
void writeToCSVfile(string name, MatrixXd matrix)
{
ofstream file(name.c_str());
for(int i = 0; i < matrix.rows(); i++){
for(int j = 0; j < matrix.cols(); j++){
string str = lexical_cast<std::string>(matrix(i,j));
if(j+1 == matrix.cols()){
file<<str;
}else{
file<<str<<',';
}
}
file<<'\n';
}
}
This is the MWE to the solution given by Partha Lal.
// eigen2csv.cpp
#include <Eigen/Dense>
#include <iostream>
#include <fstream>
// define the format you want, you only need one instance of this...
// see https://eigen.tuxfamily.org/dox/structEigen_1_1IOFormat.html
const static Eigen::IOFormat CSVFormat(Eigen::StreamPrecision, Eigen::DontAlignCols, ", ", "\n");
// writing functions taking Eigen types as parameters,
// see https://eigen.tuxfamily.org/dox/TopicFunctionTakingEigenTypes.html
template <typename Derived>
void writeToCSVfile(std::string name, const Eigen::MatrixBase<Derived>& matrix)
{
std::ofstream file(name.c_str());
file << matrix.format(CSVFormat);
// file.close() is not necessary,
// desctructur closes file, see https://en.cppreference.com/w/cpp/io/basic_ofstream
}
int main()
{
Eigen::MatrixXd vals = Eigen::MatrixXd::Random(10, 3);
writeToCSVfile("test.csv", vals);
}
Compile with g++ eigen2csv.cpp -I<EigenIncludePath>.