Google apps script slow parsing - google-apps-script

Question

I was trying to parse vmstat using Google Apps script so that everyone in the company could use this to create a graph of the data. You can find my code here but this code is really slow. Is there something I can do to make this better or isn't Google Apps Script suitable for this? The problem the ammount of rows that needs to be processed. Any suggestions are welcome.
function doGet(){
var file = DriveApp.getFileById(id)
var docContent = file.getAs('application/octet-stream').getDataAsString();
var data = Charts.newDataTable()
.addColumn(Charts.ColumnType.STRING, 'TIME')
.addColumn(Charts.ColumnType.NUMBER, 'Memory');
var lines = docContent.split("\n");
Logger.log(lines.length);
var i = 1;
lines.forEach(function(line) {
if ((line.indexOf('mem') < 0) && (line.indexOf('free') < 0)) {
var values = line.match(/\S+/g);
data.addRow(['5',parseInt(values[3])]);
Logger.log(i)
}
if (i == 20){
return;
}
i++;
});
for( var i=0;i< lines.length;i++){
data.addRow(['5',10]);
}
data.build();
var chart = Charts.newAreaChart()
.setDataTable(data)
.setStacked()
.setRange(0, 400)
.setTitle('Memory')
.build();
return UiApp.createApplication().add(chart);
}

Answer 1

This isn't a problem of code optimization (although the code isn't perfect), as much as division of work.
The accepted approach to web application performance optimization involves separating three concerns; presentation, business logic and data accessref. With the exception of the generation of the vmstat output, you've got all of that in one place, making the user wait while you locate a file on Google Drive (using two exhaustive searches, btw) and then parse it into a Charts DataTable, and finally generate HTML (via UiApp).
You may find that the accessibility of a Google Apps Script presentation is useful to your organization. (I know in my workplace that our IT folks clamp down on in-house web servers, for example.) If so, consider what you have as prototype, and refactor it to give better perceived performance.
Presentation: Move from UiApp + Charts to HtmlService + Google Visualization. This moves the generation of the chart into the web client, instead of keeping it in the server. This will give a faster page load, to start.
Business Logic: This will be the rules that map your data into the Visualization. Like the Charts Service that is built over it, GViz uses DataTables with column definitions and rows of data.
One option here is to repeat the column definition & data load you already have, except on the client in JavaScript. Doing that will be significantly faster than via Google Apps Script.
A second option, which is even faster, especially with large datasets, is to load the data from an array.
google.visualization.arrayToDataTable(...)
Either way, you need to get your data to the JavaScript function that will build your chart.
Data Access: (I assume) you're currently running a shell script in Linux that calls vmstat and pipes the output to a file in your local Google Drive folder. (Alternatively, the script may be using the Drive API to push the file to Google Drive.) This file is plain text.
The change I'd make here would be to produce csv output from vmstat, and use Google Apps Script to import the csv into a spreadsheet. Then, you can use Sheet.getSheetValues() to read all the data in one shot, in a server side function to be called from the client JavaScript.
This would not be as fast as a local server solution, but it's probably the best way to do this using the Google Apps Script environment.
Edit: See more about this in my blog post, Converting from UiApp + Chart Service to Html Service + Google Visualization API.