How can a Eigen matrix be written to file in CSV format? - csv

Suppose I have a double Eigen matrix and I want to write it to a csv file. I find the way of writing into a file in raw format but I need commas between entries. Here is the code I foudn for simple writing.
void writeToCSVfile(string name, MatrixXd matrix)
{
ofstream file(name.c_str());
if (file.is_open())
{
file << matrix << '\n';
//file << "m" << '\n' << colm(matrix) << '\n';
}
}

Using format is a bit more concise:
// define the format you want, you only need one instance of this...
const static IOFormat CSVFormat(StreamPrecision, DontAlignCols, ", ", "\n");
...
void writeToCSVfile(string name, MatrixXd matrix)
{
ofstream file(name.c_str());
file << matrix.format(CSVFormat);
}

Here is what I came up;
void writeToCSVfile(string name, MatrixXd matrix)
{
ofstream file(name.c_str());
for(int i = 0; i < matrix.rows(); i++){
for(int j = 0; j < matrix.cols(); j++){
string str = lexical_cast<std::string>(matrix(i,j));
if(j+1 == matrix.cols()){
file<<str;
}else{
file<<str<<',';
}
}
file<<'\n';
}
}

This is the MWE to the solution given by Partha Lal.
// eigen2csv.cpp
#include <Eigen/Dense>
#include <iostream>
#include <fstream>
// define the format you want, you only need one instance of this...
// see https://eigen.tuxfamily.org/dox/structEigen_1_1IOFormat.html
const static Eigen::IOFormat CSVFormat(Eigen::StreamPrecision, Eigen::DontAlignCols, ", ", "\n");
// writing functions taking Eigen types as parameters,
// see https://eigen.tuxfamily.org/dox/TopicFunctionTakingEigenTypes.html
template <typename Derived>
void writeToCSVfile(std::string name, const Eigen::MatrixBase<Derived>& matrix)
{
std::ofstream file(name.c_str());
file << matrix.format(CSVFormat);
// file.close() is not necessary,
// desctructur closes file, see https://en.cppreference.com/w/cpp/io/basic_ofstream
}
int main()
{
Eigen::MatrixXd vals = Eigen::MatrixXd::Random(10, 3);
writeToCSVfile("test.csv", vals);
}
Compile with g++ eigen2csv.cpp -I<EigenIncludePath>.

Related

C++ How do I turn a string into a json object? [duplicate]

I'm currently working in C++, getting an HTTP response from a request that I write into a .txt file using ostream. This happens asynchronously and I don't want to change this.
Once the data is done being written, I want to read from the file
{"data":{"request":[{"type":"City","query":"London, United Kingdom"}],"weather":[{"date":"2013-04-21","astronomy".....
~somehow~ prettify the string using either an outside library like nlohmann/json or other(?) and then
a)print it to the console and
b) save it in a different file (pretty.json)
I am having trouble understanding which method to use from:
https://github.com/nlohmann/json
Any ideas how to approach this?
I was thinking getting the file line by line until I hit EOF into a sort of "buffer" and then running _json on that and saving the solution which can be displayed on the console...
My code so far
#include <cpprest/http_client.h>
#include <cpprest/filestream.h>
#include <iostream>
#include <sstream>
#include "json.hpp"
using namespace utility; // string conversion
using namespace web; // URI
using namespace web::http; // HTTP commands
using namespace web::http::client; // HTTP Client features
using namespace concurrency::streams; // Asynch streams, like Node
using json = nlohmann::json;
int main()
{
auto fileStream = std::make_shared<ostream>();
// Open stream to output file.
pplx::task<void> requestTask = fstream::open_ostream(U("results.txt"))
.then([=](ostream outFile)
{
*fileStream = outFile;
http_client client //gets the info
return client.request(methods::GET, stringBuilder.to_string());
})
.then([=](http_response response) // set up response handler
{
printf("Received response status code:%u\n", response.status_code());
return response.body().read_to_end(fileStream->streambuf());
})
.then([=](size_t) // close file stream
{
return fileStream->close();
})
.then([=]()
{
nlohmann::json j;
std::ifstream i;
i.open("results.txt"); // ?? <<< === this is where my question is
});
// Wait for all the outstanding I/O to complete, handle exceptions
try
{
requestTask.wait();
}
catch (const std::exception &e)
{
printf("Error exception:%s\n", e.what());
}
return 0;
}
SOLUTION:
.then([=]()
{
// read a JSON file
std::ifstream readFromFile("results.txt");
if (readFromFile.is_open()) {
nlohmann::json j;
readFromFile >> j;
// write prettified JSON to another file
std::ofstream writeToFile("pretty.json");
writeToFile << std::setw(4) << j << std::endl;
readFromFile.close();
writeToFile.close();
}
else {
std::cout << "unable to open file";
}
});
You have two choices to prettify with nlohmann.
Uses dump which produces a string
int indent = 4;
nlohmann::json data;
data.dump(indent);
Or use the stream output overload with field width set
std::ofstream o("pretty.json");
o << std::setw(4) << data << std::endl;

boost::property_tree::ptree accessing array's first complex element

My JSON is this:
{
"apps":[
{
"id":"x",
"val":"y",
}
]
}
I can get id's value "x" by looping, and exiting when it.first is id:
for (const ptree::value_type &app : root.get_child("apps"))
{
for (const ptree::value_type &it : app.second) {
if (it.first == "id") {
std::cout << it.second.get_value<std::string>().c_str() << std::endl;
}
}
}
What I want, however, is to get the id's value by something like this:
std::cout << root.get<std::string>("apps[0].id").c_str() << std::endl;
Of course this displays nothing to me, for I am probably using wrong syntax accessing 1st element of the apps array. It might be that this has to be done in a different way all together.
I have found only this ugly and dangerous method:
std::cout << root.get_child("apps").begin()->second.begin()->second.get_value<std::string>().c_str() << std::endl;
I cannot really use it this way as it won't throw an exception when the array is empty, it will core dump!
Below is the whole program to make it easier for any one who wants to help:
#include <boost/property_tree/ptree.hpp>
#include <boost/property_tree/json_parser.hpp>
using boost::property_tree::ptree;
int main()
{
std::stringstream ss("{\"apps\":[{\"id\":\"x\",\"val\":\"y\"}]}");
ptree root;
read_json(ss, root);
for (const ptree::value_type &app : root.get_child("apps"))
{
for (const ptree::value_type &it : app.second) {
if (it.first == "id") {
std::cout << it.second.get_value<std::string>().c_str() << std::endl;
}
}
}
std::cout << root.get_child("apps").begin()->second.begin()->second.get_value<std::string>().c_str() << std::endl;
return 0;
}
As the docs say, array elements are nodes with "" keys.
If you're after the first element, you're in luck:
root.get("apps..id", "")
The .. in the path selects the first empty key
Live On Coliru
#include <boost/property_tree/json_parser.hpp>
#include <iostream>
using boost::property_tree::ptree;
int main() {
std::stringstream ss(R"({"apps":[{"id":"x","val":"y"}]})");
ptree root;
read_json(ss, root);
std::cout << root.get("apps..id", "") << "\n";
}
BONUS
If you need to address elements other than the first, write a helper function. This would be a good start:
#include <string>
#include <stdexcept> // std::out_of_range
template <typename Tree>
Tree query(Tree& pt, typename Tree::path_type path) {
if (path.empty())
return pt;
auto const head = path.reduce();
auto subscript = head.find('[');
auto name = head.substr(0, subscript);
auto index = std::string::npos != subscript && head.back() == ']'
? std::stoul(head.substr(subscript+1))
: 0u;
auto matches = pt.equal_range(name);
if (matches.first==matches.second)
throw std::out_of_range("name:" + name);
for (; matches.first != matches.second && index; --index)
++matches.first;
if (index || matches.first==matches.second)
throw std::out_of_range("index:" + head);
return query(matches.first->second, path);
}
Here's some live tests using it:
Live On Coliru
#include <boost/property_tree/json_parser.hpp>
#include <iostream>
using boost::property_tree::ptree;
int main() {
std::stringstream ss(R"({
"apps": [
{
"id": "x",
"val": "y",
"id": "hidden duplicate"
},
{
"id": "a",
"val": "b"
}
]
})");
ptree root;
read_json(ss, root);
for (auto path : {
"apps..id", "apps.[0].id", // (equivalent)
//
"apps.[0].id[]", // invalid
"apps.[0].id[0]", // x
"apps.[0].id[1]", // hidden duplicate
"apps.[1].id", // a
"apps.[1].id[0]", // a
"apps.[1].id[1]", // out of range
"apps.[2].id", // out of range
"drinks" // huh, no drinks at the foo bar
}) try {
std::cout << "Path '" << path << "' -> ";
std::cout << query(root, path).get_value<std::string>() << "\n";
} catch(std::exception const& e) {
std::cout << "Error: " << e.what() << "\n";
}
}
Prints:
Path 'apps..id' -> x
Path 'apps.[0].id' -> x
Path 'apps.[0].id[]' -> Error: stoul
Path 'apps.[0].id[0]' -> x
Path 'apps.[0].id[1]' -> hidden duplicate
Path 'apps.[1].id' -> a
Path 'apps.[1].id[0]' -> a
Path 'apps.[1].id[1]' -> Error: index:id[1]
Path 'apps.[2].id' -> Error: index:[2]
Path 'drinks' -> Error: name:drinks

change string file?

I have an output of a string that is:
I need to print out only the parts that are between"tele": and then the ",". for example the first one i need to print out is hel. In the above string output. "text":"hel","source": is the first part. Every where there is a "tele": and the "," i need to be printed.
the code i have written so far prints out only the first one and is shown below
int comm_poss;
string econ;
tax= mes.find("tele");
econ=mes.substr(tax,15);
cout<<"msg test : "<<econ;
This prints out tele":"hel"," but not the other part that is between the "tele" and ","
how do i print only whats between those two and make the code do it multiple times?
thanks
EDIT: the other part in the output is "tele":"FIRS","code":" found near the end
Save yourself some trouble and use a JSON parser. For example using JsonCpp library and reading the JSON file from stdin (untested):
Json::Value root;
std::cin >> root;
for (int i = 0; i < root.size(); i++) {
std::string text = root[i]["text"].asString();
std::cout << text << std::endl;
}
Could something like this help.
string txt_test;
size_t poss_text;
poss_text= replyMsg.find("text:\":\""); // Search for text":"
replyMsg=replyMsg.substr(poss_text + 8); // Get rest of string
// Add 8 to get beyond text":"
poss_text= replyMsg.find("\","); // Search for ",
txt_test = replyMsg.substr(0, poss_text); // Get from start to ",
cout<< "msg test : " << txt_test;
You should however always check whether your string were found:
if (poss_text != std::string::npos)
{
// Found - so go on
}
else
{
// Not found - stop now
}
Now you can add the loop:
int main()
{
string replyMsg = "HEJtext:\":\"first\",\"hhhtext:\":\"second\",";
string txt_test;
size_t poss_text = 1;
while(poss_text != std::string::npos) // Loop while there is still something in replyMsg
{
poss_text= replyMsg.find("text:\":\""); // Search for text":"
if (poss_text != std::string::npos)
{
replyMsg=replyMsg.substr(poss_text + 8); // Get rest of string
// Add 8 to get beyond text":"
poss_text= replyMsg.find("\","); // Search for ",
if (poss_text != std::string::npos)
{
txt_test = replyMsg.substr(0, poss_text); // Get from start to ",
cout<< "msg test : " << txt_test << std::endl ;
replyMsg=replyMsg.substr(poss_text); // Prepare for next search
}
}
}
return 0;
}
This is a working solution.
I will not use any library since you probably want to know how to do it from scratch.
1. Using Strtok() with delim(':') and checking if the 6 characters to
the left is equal to "text", If TRUE, then pickup all characters to the right
until you hit ",".
2. When you're picking up characters to the right, to avoid reading past your domain, modify the content string by adding a '$' symbol to the end.
#include <iostream>
void get_text(char * token);
int main(){
char =content[]="586162004425048066\",\"text\":\"hel\",\"source\":...\"text\":\"yo\";$";
char * token;
token=strtok(content,":");
get_text(token);
while(token=strtok(NULL,":")){
get_text(token);
}
}
void get_text(char * token){
unsigned int len=strlen(token);
std::string temp;
if (len>=6){ // Reading the 6 characters to the left and storing it in temp
for (int i=-6; i<0; i++){
temp=temp+token[len+i];
}
}
if (temp=="\"text\""){
temp="";
unsigned i=len;
while(token[i]!=',' and token[i]!='$'){ // retrieve characters to the right until you hit a ',' or '$' (which indicates end of line as a boundary condition)
temp=temp+token[i];
i++;
}
std::cout<<temp;
}
}
OUTPUT
"hey" "yo"

Read CSV file into C++ with unknown size

I am trying to read in a csv file with an unknown number of rows. I tried using vector of vector approach but it wont compile.
vector<vector<double> > data;
ifstream file("data.csv");
int i = 0;
while (iss.good())
{
vector<int> row;
for (int j = 0; j < 2; j++) {
string val;
getline(iss, val, ',');
stringstream convertor(val);
cout << val << endl;
row.push_back(val);
}
data.push_back(row);
i++;
}
Thanks for the tip but even with the edit, the code doesnt compile. Welp.
Error message:
Q7.cc:18:10: error: ‘iss’ was not declared in this scope
Q7.cc:28:26: error: no matching function for call to ‘std::vector<int>::push_back(std::string&)’
the first line
vector<vector<double>> data;
has the fragment >> which is interpreted as a stream operator. It must look like
vector<vector<double> > data;
Note the space!

odeint streaming observer and related questions

I have a system of 4 coupled equations to solve and a parameter Gamma[i] to iterate over. Since I am quite new to C++, my code is a very rudimentary. If it looks sophisticated and elegant in certain parts, it is only because I have adapted code from the author of odeint. :)
This question is related to (http://stackoverflow.com/questions/12060111/using-odeint-function-definition/12066958#comment16253600_12066958) but not exactly the same. Please do not delete this. :(
Questions have been inserted between the lines of code.
#include <iostream>
#include <iterator>
#include <algorithm>
#include <boost/numeric/odeint.hpp>
#include <cmath>
#include <vector>
#include <fstream>
#include <iomanip>
using namespace std;
using namespace boost::numeric::odeint;
class NLI_class {
private:
double gamma;
public:
NLI_class (double r) : gamma(r) {}
void operator()( vector<double> &u , vector<double> &du , double z ) {
du[0] = u[0]*u[1]*cos(u[3]); //u1
du[1] = -u[0]*u[0]*cos(u[3]); //u2
du[2] = gamma * (2/(u[0]*u[0]) - 1/(u[1]*u[1])); //theta
du[3] = gamma * (1.0/(u[0]*u[0])); //phi1
du[4] = gamma * (1.0/(u[1]*u[1])); //phi2;
}
};
Question #1:
In my original program, I had something like this to pipe the output to a csv file:
inline void save(vector<double>& v, string filename)
{
ofstream output(filename);
for(int i=0;i<v.size();++i){
output << setprecision(64) << v[i] << endl;
}
}
How do I adapt streaming_observer to do what my save() does? Basically, I want to generate .csv files for each iteration i. At this point, I am doing it the ugly way, i.e compiling everything, opening a windows command prompt and then piping the exe output to a text file. This generates one big file with all iterations thrown in there.
This becomes very painful to analyze for a large number of iterations.
struct streaming_observer {
std::ostream &m_out;
streaming_observer( std::ostream &out ) : m_out( out ) {}
void operator()( const vector<double> &x , double t ) const
{
m_out << t;
for( size_t i=0 ; i < x.size() ; ++i )
m_out << "\t" << x[i];
m_out << "\n";
}
};
int main(){
vector<double> x( 5 );
vector<double> Gamma;
vector<double>delta;
const double pi=acos(-1.0);
short delta_n=5;
const double delta_step=(2*pi)/delta_n;
const double dz = 0.01;
const double zeta = 3.0;
const double theta_initial=0.0;
const double u20=tanh(zeta);
const double u10=sqrt(1.0-(u20*u20));
double d=0.0;
double G=0.0;
for(int i=0;i<=delta_n;i++){
//When i=0, the d=0.0 and G=0.0 are pushed into the vector.
delta.push_back(d);
Gamma.push_back(G);
// Compute delta and Gamma
d=d+delta_step;
G=-u10*u10*u20*sin(theta_initial+d);
}
save(delta,"delta.csv");
save(Gamma,"Gamma.csv");
Question#2:
The results I get here do not agree with what I get with what I get using a simple explicit Euler method. Hence, I would like to see the RK4 coefficients (preferably dump them to a file) or the intermediate steps. How can I get this information?
//Numeric Integration
for (unsigned i = 0; i < Gamma.size(); ++i) {
x[0] = u10;
x[1] = u20;
x[2] = 0.0;
x[3] = 0.0;
x[4] = 0.0;
NLI_class nli_obj(Gamma[i]);
integrate_const( runge_kutta4< vector<double > >(), nli_obj, x , 0.0 , 3.0 , dz,streaming_observer( std::cout ) );
}
}
Thank you for all those who helped!
Edit:
Is there some way to get a running error estimate? Note that u[0]*u[0]+u[1]*u[1]=1 at all times.
Question #1 :
I do not understand exactly what kind of output you need. But if you want to write the result after each iteration you can implement an output observer like this:
struct output_observer
{
string filename_;
size_t count_;
output_observer( const string &filename ) : filename_( filename ) , count_( 0 ) { }
void operator()( const state_type &x , time_type dt )
{
char fn[512] = "";
sprintf( fn , "%s_%04lu.csv" , filename_.c_str() , count_ );
ofstream fout( fn );
for( size_t i=0 ; i<x.size() ; ++i ) fout << x[i] << "\n";
++count_;
}
};
You can apply this observer simply by
integrate_const( runge_kutta4< vector<double > >() , nli_obj , x ,
0.0 , 3.0 , dz , output_observer( "filename" ) );
Is this the desired functionality?
Question #2 :
It is not possible to see the intermediate e steps of runge_kutta4. The coefficients are the standard ones for the classical Runge-Kutta method: http://en.wikipedia.org/wiki/Runge%E2%80%93Kutta_methods
Question #3 :
odeint has several error steppers, which estimate the error made during one step. You can use for example the Runge_Kutta Cash Karp algorithm;
runge_kutta_cash_karp54< state_type > rk;
state_type xerr;
rk.do_step( nli_obj , x , t , xerr );
which makes ONE step and estimates the error and writes the error result in xerr.