Since Dygraphs apparently does not have any functionality for adding separate series of data to a chart one at a time (but rather only loading all the data series of a chart at once from a CSV file, or an in-memory array of arrays) I'm looking to make some code to do this myself.
My reason? My problem/scenario is that I have a "base file" containing a series of data many million values large. I will then need to show many separate charts that display this large data series TOGETHER with a bunch of other respective smaller data series, and I'd very much rather not duplicate the large dataseries in a new CSV file on disk for each such chart, but rather first load the big "base data series" from the CSV "base file" directly from my Javascript, and then for each such chart integrate one such smaller data series with it before sending it off to rendering by means of a new Dygraph(...) call.
The CSV file loading functionality that already obviously exists somewhere inside the Dygraphs code is very nice, so I'd very much like to use it for this loading of the large "base data series" if possible, from a single separate CSV file.
So, in short, the question is:
How can I use the existing CSV file loading functionality of Dygraphs separately from inside my own code, in order to load arbitrary CSV files into the Dygraphs chart data array format in-memory, so that I can finally merge these multiple data series arrays using my own custom code?
What I'm hoping for is something like this:
loaded_data_series_1 = some_secret_internal_function_or_method_of_dygraphs('file1.csv');
loaded_data_series_2 = some_secret_internal_function_or_method_of_dygraphs('file2.csv');
merged_data_series = my_own_custom_dataseries_merging_code(loaded_data_series_1, loaded_data_series_2);
g = new Dygraph(document.getElementById('my_chart'), merged_data_series,{});
The key here would thus be to know what some_secret_internal_function_or_method_of_dygraphs() should be replaced with for this to work.
Could the Dygraph devs or anyone else possibly point me in the right direction here?
(I tried to look inside the Dygraphs code myself, but unfortunately got lost pretty quickly due to insufficient Javascript coding skills on my side)
Related
Blocked out the data for personal reasons.
Essentially I am passing images from a MongoDB to NodeRed and parsing the actual image file back to base64 to create a set of up to 100 images (records) and the "alarm" description. The table is done in HTML template with input coming from Mongo Query.
With 3 pieces of information that is "linked" together (Time, Alarm, Image) the information separately does no good.
What I'm trying to determine if there is a good way to download the full table (images included) in a logical way.
Methods tried:
Excel download - Formatted properly, rows/columns exist but images are not saved properly because they technically aren't being stored in the table, just referencing from the MongoDB Query.
Canvas->Table inside Canvas: this one has yielded little success. Idea: Make a canvas and have the table inside (yielding an image that could be saved, removing search-ability but easy to view). Either I don't fully understand how HTML canvas drawing works or Node-Red handles some things differently. I got a blank canvas when linking table to canvas by object ID of table.
SelectAll/Copy/Paste: This one works, pasting into a doc has everything formatted in table, but not automated/scripted.
Looking for a way to "triggered"/on button press to save the current table contents and save them. CtrlP/print to PDF and such work fine. Looking specifically to save the Table or an object similar in nature.
I cannot post/link the flow because of data/security reasons.
If there is a better option for managing data that needs to be in a format with 3 columns and 100 rows I am also open to changing the HTML. I am not an expect in best practices for frontend, this is only going to be used for creating a report.
My question is whether or not anybody knows of a better way to do what I'm already doing. I'm creating a report as a list, and trying to render it both in HTML and Excel.
I'm developing a shiny app that generates reports for Qualtrics surveys.
The results table is a list of HTML strings that I paste together and display in a shinydashboard. Here's a dput of the example results tables.
Here's how I'm creating the html results tables list -- the html_tabelize() function in my package. Here's a dput of the example input.
In the shiny server.R file the way I create the Excel file is with the following code:
output$downloadResults <- downloadHandler(
filename = 'tables.xls',
content = function(file) {
write(html_tabelize(main()[['blocks']]), file)
}
)
To summarize: I get the blocks, I run html_tabelize on them, and then I write the HTML output to a file called "tables.xls". When I open that file, because Excel can render HTML, it renders something like this:
My concern and problem with what I'm doing are two-fold:
If I were writing an Excel document instead of simply rendering HTML in Excel, then I could perhaps get a better formatted document. I'd like that.
When you download the results tables xls file and try to open it, you get a warning from Excel. I don't want the users of my app to see this warning, because it's distracting and could worry them about something that isn't really a concern.
I know that options exist for writing Excel files in R, but so far what I've seen indicates that their input must be either a data frame, or a list of data frames. The list I am rendering from has different types of components, like the question text, as well as data frames of results. Originally I was using pandoc, but pandoc, even when run from R, is a system binary, and it's difficult to list as a dependency (and if I can't list it as a dependency, it's tough to make sure it's installed for the users of my app). Additionally, I found out pandoc doesn't even convert to "real" Excel -- it also just saves HTML in a .xls file. Does anybody have any suggestions as to how I can improve this part of my app?
I am trying to export all the plots of my NetLogo model after simulation runs in a csv format with the primitive export-all-plots.
I haven't found yet the way to open this csv file with an external reader in order to get more clear plots. I tried with gnuplot but it looks like it's not able to open the csv format created with NetLogo:
"export-plots data (NetLogo 5.0.5)"
^
"C:\results\interface.csv", line 1: invalid command
How can I open csv plots with an external reader?
There are two complicating factors about NetLogo's plot export format. First, there's a three line header at the beginning (plus an empty line after) that just gives information about the model and when the data was generated. Next, there's data about the model settings, the plot state (pen colors and such). Finally, there's the data itself, which itself is somewhat complicated by the fact that you can have multiple pens per plot. So I'm not surprised gnuplot couldn't read it as is.
The table's are quite easy to use in GUI spreadsheet application, like Excel, LibreOffice's Calc, or Gnumeric. You can just select the data you want and generate the plots.
To do this at the command line, I'm afraid you might have to write a script to read it in. This should be pretty easy in something like Python or R. Just skip the metadata lines, and use a CSV parser to read in the rest.
You might also try using BehaviorSpace to generate the data, but make sure to use the table output. It let's you generate the data from many runs at once, and the format is a little more consistent. There are still 6 lines of metadata at the top, but you can just delete that. I believe this is more the standard practice in NetLogo.
I'm using JFreeChart to generate line graphs from a simple array of integers.
However, I'd like to use a csv file for the input of the graph. Are there any applications which do it automatically? Also, my csv file will new entries appended every 3 seconds. How can I generate dynamic graphs? I will have to use these graphs in a swing application. Thanks!
I've had good luck with org.h2.tools.Csv, part of H2 Database. You might also look at org.jfree.data.io.CSV, "A utility class for reading CategoryDataset data from a CSV file."
I need to pull data from csv file to SQL Server table. Which Control task should I use ? Is it Flat File ? What is the correct method to pull data ?
The problem is I have used Flat File Task for pulling csv file. But the csv file whihc I am having, contains headings as first row, then on the third row, I have the columns, and data starting from fifth row.
Another problem is, in this file column details comes again after 1000 data ie columns appears in two rows. Is it possible to pull data ? If so, HOW ?
While Valentino's suggestion should work, I suggest that first you work with the provider of the file to get them to provide the data in a better format. When we get stuff like this we almost always push it back and ask for properly formatted data. We get it too about 90% of the time. It will save you work if they will fix their own drek. In our case, the customers providing the data are paying for our programming services and when they understand how substantial an increase in the cost to them, they are usually nmore than willing to accomodate our needs.
I believe you'll first have to transform your file into a proper CSV file so that the SSIS Flat File Source component (Data Flow) can read it. If the source system cannot produce a real CSV file, we usually create custom .NET applications for the cleanup/conversion task.
An Execute Process task (Control Flow) that executes the custom app can then be called prior to the Data Flow.