Clock Trigger Builder Not calling function when scheduled - Google sheets app Script - google-apps-script

I am using the app script provided by Google to access their prediction API through sheets. I am trying to predict thousands of rows at once, however, after 6 minutes the maximum execution time is reached at the code stops.
I implemented a solution that I found using clock trigger builder. Once I run the function it goes for 5 mins, then it stops sets a trigger to recall the function within 2 mins.
The major problem is that the function is not called when scheduled. I see it in the current triggers list, but it never gets called again. Can you please explain why this is occurring.
My intention is to predict as many lines as possible in 5 min then stop set a trigger to call the predict function again within a few minutes start where it left off and continue until ever element has been predicted.
I also need to know how would I store then values in cache so that it would know all the information that it needs when the function is called again.
//This is the function that is used to predict a selection of data
function predict() {
try {
clearOutput();
var startTime= (new Date()).getTime();
var sheet = SpreadsheetApp.getActiveSheet();
var selection = sheet.getActiveSelection();
var instances = selection.getValues();
var project_number = getProjectNumber();
var model_name = getModelName();
var startRow = stRow;
var MAX_RUNNING_TIME = 300000;
var REASONABLE_TIME_TO_WAIT = 60000;
for (var i = startRow; i < instances.length; ++i) {
var currTime = (new Date()).getTime();
if(currTime - startTime >= MAX_RUNNING_TIME) {
var builder = ScriptApp.newTrigger('predict').timeBased().after(REASONABLE_TIME_TO_WAIT);
builder.create();
break;
} else {
var result = predictSingleRow(project_number, model_name, instances[i]);
selection.getCell(i + 1, 1).setValue(result);
}
}
} catch(e) {
Browser.msgBox('ERROR:' + e, Browser.Buttons.OK);
}
}

Few things as to why your code is not functioning as intended:
1) Since you mentioned,"I see it in the current triggers list, but it never gets called again" and looking at your code, I am unsure whether you intended to call the function again after it's execution has completed. If you do, this is because your for loop runs for a while until the length of the instances is obtained. Nothing in the script suggests that the function needs to be run again once it has finished iterating through instances. Refer to this link to see how to Manage Trigger Programmatically.
2) var builder = ScriptApp.newTrigger('predict').timeBased().after(REASONABLE_TIME_TO_WAIT);
This line of your code falls under an if condition which stops the execution for 1 minute (value is 60000). Hence, adding 1 minute to the time since execution started. Nowhere are you resetting the startTime counter to the time after the waiting time since once the value of currTime - startTime has exceeded MAX_RUNNING_TIME, the function will keep calling the if loop for all iterations of the for loop after that. Simply put, if startTime was 9:35 and currTime was 9:40, after waiting for 1 minute the currTime is 9:41 which is still more than the MAX_RUNNING_TIME(5 minutes) because value of startTime still remains 9:35. Resetting it to 9:41 at this point should resolve your problem.
3) Loosing the break in the if loop would probably help fix that as well.
EDIT:
Add a function as shown in the link I mentioned above:
function callTrigger(){
ScriptApp.newTrigger('predict')
.timeBased()
.everyMinutes(30)
.create();
}
Run the function callTrigger once from your editor and you should be good to go. Remember, for minutes you can only pass values 1,5,15 or 30.

Related

Speed up adding Items to Forms with Apps Script

I'm wondering if there is a more efficient way to add Items to Google Forms with App Script.
My script is taking a long time and I'm wondering if there's anyway to add items more efficiently using an array or something rather than one by one. Here's the code I'm running...
function addFormItems() {
var startTime = new Date(); var form = FormApp.create("Test") form.setIsQuiz(true)
for (var i = 1; i < 100; i++) {
form.addTextItem().setTitle("Question number " + i).setHelpText("Hint: answer should not be more than a couple of words")
}
var endTime = new Date();
//Get runTime in seconds var runTime = (endTime - startTime)/1000;
Logger.log("runtime is: " + runTime)
}
Currently it takes quite a long time a minute to a minute and a half (odd thing is every time I execute I get a very different runtime not sure why that happens) Any thoughts on how to speed this up is much appreciated.
I searched Documentation and couldn't find anything about adding multiple items with one call.

Script running multiple times onFormSubmit

I have the following code set to run based on a onFormSubmit trigger but it will sometimes run multiple times with the same submission. I want to verify if it already copied the row and if so to stop the script.
function toDo(){
var responses = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Form Responses 1");
var jobs = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Jobs");
var lastrow = responses.getLastRow();
var col = responses.getLastColumn();
var row = responses.getRange(lastrow, 1, 1, 19).getValues();
jobs.appendRow(row[0]);
//copyValuesOnly(copyFromRange, copyToRangeStart);
var si = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('Jobs');
var range = si.getRange("A2:R");
range.sort({column: 5,ascending: true}),({column: 1, ascending:true});
}
this is a known problem with GAS + Forms. The way that you solve it is by creating a script lock that rejects (causing them to return early) all other attempts within a period of time.
function toDo(){
SpreadsheetApp.flush();
var lock = LockService.getScriptLock();
try {
lock.waitLock(5000);
} catch (e) {
Logger.log('Could not obtain lock after 5seconds.');
return HtmlService.createHtmlOutput("<b> Server Busy please try after some time <p>")
// In case this a server side code called asynchronously you return a error code and display the appropriate message on the client side
return "Error: Server busy try again later... Sorry :("
}
var responses = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Form Responses 1");
var jobs = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Jobs");
var lastrow = responses.getLastRow();
var col = responses.getLastColumn();
var row = responses.getRange(lastrow, 1, 1, 19).getValues();
jobs.appendRow(row[0]);
//copyValuesOnly(copyFromRange, copyToRangeStart);
var si = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('Jobs');
var range = si.getRange("A2:R");
range.sort({column: 5,ascending: true}),({column: 1, ascending:true});
Utilities.sleep(5000);
lock.releaseLock)();
}
I've had scripts that do this up to 8 times, and usually do it every 2-3 seconds. With this solution you are making a lock at the beginning and then sleeping at the end to make sure that the process time is greater than the wait time. (Here I used 5 seconds, that should prevent the double entry).
I have noticed that if you just make another copy of the sheet with the script, this error goes away. Seems to reset whatever was the issue in the original copy. Also try dis-abling the response receipts on your google forms.
How to check:
Go to script editor and check under execution logs. If you see multiple instances of on form submit, then you probably have multiple triggers somehow and the trigger is running multiple times legitimately.
To fix:
Go to triggers tab and delete all unwanted triggers.
Check code if you are creating new trigger through code. And comment that out.
Possible Reason:
I would have expected the code such as below to overwrite existing trigger. I had 19 triggers created. This was because every time I generated the link, it called Initialize and I got a new trigger added. Thus I could see script running 19 times.
const initialize = () => { const form = FormApp.getActiveForm(); //ScriptApp.newTrigger('onFormSubmit').forForm(form).onFormSubmit().create(); };
I noticed the problem got solved if I renamed the function called by onFormSubmit and saved the script.
If I click "run" in the script editor I get 2 emails per form submit. If I click "run" again I get 3 emails per form submit. I reset to only 1 email per form submit if I change the function name again.
Somehow when I run the script it duplicates the triggers.

Measurement of execution time of built-in functions for Spreadsheet

Are there methods to measure the execution time when built-in functions completed for Spreadsheet? When I use several built-in functions (For example, IMPORTHTML and IMPORTXML), if I know the average execution-time, it is easy for me to use and design data sheet.
I measure it of custom functions using this script.
function myFunction() {
var start = new Date();
// do something
var end = new Date();
var executiontime = end - start;
}
Thank you so much for your time and advices.
Unfortunately, there are not measurement tools for retrieving the execution time of built-in functions. This has already been commented by #Rubén. So I thought of about the workarounds. How about the following workaround?
Flow :
Import a value to a cell. The value is anything good, because this is used as a trigger. Please do this by yourself.
Custom functions cannot use setValue(). So I used onEdit().
func1() imports a formula that you want to measure the execution time by the script launched by the trigger.
At func2(), after set the formula, the measurement is started. The confirmation when built-in function was completed is carried out using loop.
By measuring the cost per one call for getValue(), it was found that that was about 0.0003 s. So I thought that this can be used.
The result of measurement can be seen at Stackdriver as milliseconds.
Sample script :
function func1(range, formula){
range.setFormula(formula);
}
function func2(range){
var d = range.getValue();
while (r == d) {
var r = range.getValue();
}
}
function onEdit(){
var formula = '### Built-in function ###'; // Please set the built-in function you want to measure the execution time.
var label = "Execution time for built-in functions.";
var ss = SpreadsheetApp.getActiveSheet();
var cell = ss.getActiveCell();
var range = ss.getRange(cell.getRow(), cell.getColumn());
func1(range, formula);
console.time(label);
func2(range);
console.timeEnd(label);
}
Note :
When built-in functions with very long time is measured, an error may occur at getValue().
In my environment, the built-in function for 10 seconds worked fine.
Updated at November 11, 2020:
As the additional information, I would like to add one more sample script for measuring the execution time of when built-in functions completed for Spreadsheet and the result using the script.
This is a simple sample script for measuring the process cost of functions on a cell. At first, in order to confirm whether this script can be used for measuring the process cost of the function put in a cell, a custom function was used. Because when the custom function is used, the process time of the script can be known by using Utilities.sleep(time).
Sample script:
When you test this script, please copy and paste the following script to the container-bound script of Google Spreadsheet. When you run the function of main(), the process cost of =SAMPLE(5000) can be obtained.
// This is a sample custom formula. This is used for testing.
function SAMPLE(time) {
Utilities.sleep(time);
return "ok";
}
// This is a script for measuring the process cost.
function main() {
const obj = { formula: `=SAMPLE(5000)`, returnValue: "ok" }; // Set formula and response value.
const label = "Execution time";
const sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Sheet1");
const range = sheet.getRange("A1");
range.clear();
// start --- measure
console.time(label);
range.setFormula(obj.formula);
SpreadsheetApp.flush();
while (range.getDisplayValue() != obj.returnValue) {}
console.timeEnd(label);
// end --- measure
range.clear();
}
In this sample, when =SAMPLE(5000) is put to a cell, the value of ok is shown in the cell after 5 seconds. main() measures the process time for this.
This sample script checks the output value from the function. So please set returnValue. Please be careful this.
At the formula of Spreadsheet, when the formula is put to the cell, the measurement of process time is started. So in this sample, I included setFormula and flush to the measured process cost.
Experimental result:
As an experiment, it shows the change of process time with increasing the sleep time of the custom function as above image. This result indicates that the process time is linearly increased with the increase in the sleep time. It was found that the process time was at least, more than the sleep time, and the process time was large in about 0.5 s for each sleep time as the offset. It is considered that this offset includes the process costs of setFormula, flush, getDisplayValue and the while loop. But, from this image, it is considered that when the process costs of various formulas are measured using above script, those can be compared by the relative comparison. And from this result, it is considered that above script can be used for measuring the execution time of the function in a cell of Spreadsheet.
Reference:
Benchmark: Measuring Process Costs for Formulas in Cells on Google Spreadsheet using Google Apps Script
Google Sheets doesn't include a built-in tool to measure the recalculation time.
One alternative is to use the Chrome Developers Tools Timeline but bear in mind that functions like IMPORTHTML and IMPORTXML are not recalculated every time that the spreadsheet does (reference Set a spreadsheet’s location and calculation settings).
Related Q&A
SO
Google spreadsheet constantly recalculates functions
Web Applications
What's causing / how to get rid of long web calls in Google Spreadsheets
Where does the calculation take place?
One can make a Named Function (just to have it easily reusable) to measure formula execution using only formulas, no scripts needed.
BENCHMARK()
BENCHMARK() (returns execution time in seconds):
=N(
REDUCE(
0,
{0; 1; 0},
LAMBDA(
acc,
cur,
IF(
cur = 0,
LAMBDA(x, x)(NOW()) - acc,
acc + 0 * ROWS(func())
)
)
) * 24 * 60 * 60
)
Its single parameter is func - a function to run and time its execution. You just get a working formula, enclose it in LAMBDA() and pass to BENCHMARK(). Examples are at the bottom.
BENCHMARKN()
BENCHMARKN():
=N(
REDUCE(
0,
{0; SEQUENCE(number); 0},
LAMBDA(
acc,
cur,
IF(
cur = 0,
LAMBDA(x, x)(NOW()) - acc,
acc + 0 * ROWS(func())
)
)
) * 24 * 60 * 60 / number
)
There's another parameter number: func will be executed in a loop number times. Then total execution time will be divided by number to get an average execution time.
UPD: enclosing NOW() in LAMBDA(x, x)(NOW()) freezes the value and prevents recalculation caused by any change in the sheet. Got it from here.
Examples
Say, we want to sum all squares of natural numbers from 1 to 1000000. So we have our formula:
=SUM(ARRAYFORMULA(SEQUENCE(1000000)))
We time it like so:
=BENCHMARK(LAMBDA(SUM(ARRAYFORMULA(SEQUENCE(1000000)^2))))
And in a 10-step loop like so:
=BENCHMARKN(LAMBDA(SUM(ARRAYFORMULA(SEQUENCE(1000000)^2))), 10)
Here's my option to caclulate the difference between ArrayFormulas:
Create a template with sample data
Set the script: change array with formulas, template Spreadsheet Id, cell address to set formulas, cell address to iterate values.
This function will test 2 or more formulas:
Copy sample file to isolate the influence of previous iterations.
Set formula and calculate the time to calculate it the first time
Edit source data in a loop and calculate time needed for the formula to recalculate.
Sample output, plotted in Sheets:
↑ formulas for the tests are taken from this question.
The y-axis is the processing time, and bars are the iterations of the script.
the low bars are the the times to insert the formula,
and higher bars are the times to re-calculate the formula when the value changed.
The Code:
function test_formulas_speed() {
/** Options ↓ ********************************************************************************************* */
// https://docs.google.com/spreadsheets/d/1vQu7hVr7FwH8H5N8JOlOGfvjlJgKtpfoM2DPPjgUaLo/template/preview
var testSpreadsheetId = '1vQu7hVr7FwH8H5N8JOlOGfvjlJgKtpfoM2DPPjgUaLo';
var formulas = [
'=INDEX(if(A2:A="",,LAMBDA(srt, SORT(SCAN(0,MAP(SEQUENCE(ROWS(A2:A)),LAMBDA(v,if(v=1,0,if(INDEX(srt,v,1)<>INDEX(srt,v-1,1),1,0)))),LAMBDA(ini,v,IF(v=1,1,ini+1))),index(srt,,2),1) ) (SORT({A2:A,SEQUENCE(ROWS(A2:A))}))))',
'=LAMBDA(a,INDEX(if(a="",,COUNTIFS(a,a,row(a),"<="&row(a)))))(A2:A)'
];
// value from the first cell of arrayformula ↓
var returnValue = 1;
// range to insert the formula
var rA1 = 'B2';
var sheetName = 'Sheet1';
var iterations = 5;
var testChangeRaangeA1 = 'A5';
var sub_iterations = 5; // the number of times to change the value
/** Options ↑ ********************************************************************************************* */
var results = [];
var file = DriveApp.getFileById(testSpreadsheetId);
var results = []
var testOne_ = function(formula, indx) {
// prepare
var copy = file.makeCopy();
var id = copy.getId();
var ss = SpreadsheetApp.openById(id);
var s = ss.getSheetByName(sheetName);
var r = s.getRange(rA1);
var rCh = s.getRange(testChangeRaangeA1);
var addToResult = function(t) {
var result = new Date() - t;
if (results[indx]) {
results[indx].push(result);
} else {
results[indx] = [result];
}
}
// measure time
var t = new Date();
r.setFormula(formula);
SpreadsheetApp.flush();
// loop until expected value is returned
while (r.getDisplayValue() != returnValue) {}
addToResult(t);
for (var i = 0; i < sub_iterations; i++) {
t = new Date();
// clear the cells, because of the internal cache
rCh.clearContent();
rCh.setValue(i);
SpreadsheetApp.flush();
addToResult(t);
}
// clean up
copy.setTrashed(true);
return 0;
}
for (var i = 0; i < iterations; i++) {
formulas.forEach(testOne_);
}
console.log(results);
}
Ref:
This benchmark by #Tanaikech
To add another option you can do:
function myFunction() {
console.time('someFunction');
// do something
console.timeEnd('someFunction');
}
And then look at the Stackdriver logs for the Function Execution.
Example output would be:
Jul 3, 2020, 1:03:00 AM Debug someFunction: 80ms

Google Spreadsheets: Script to check for completion of ImportHTML

I am trying to scrape data of a website once day automatically. In Google Spreadsheets, i use the =ImportHTML() function to import data tables, and then I extract the relevant data with a =query(). These functions take between 10 and 30 seconds to complete calculation, every time I open the spreadsheet.
I use a scheduled Google Apps Script, to copy the data into a different sheet (where it is stored, so i can run statistics) every day.
My problem is that I am having trouble to make the script wait for the calculations to be finished, before the data is copied. The Result is that my script just copies the error Message "N/A".
I tried just adding a Utilities.sleep(60000);, but it didn't work.
Is it possible to create a loop, that checks for the calculation to finish? I tried this without success:
function checkForError() {
var spreadsheet = SpreadsheetApp.getActive();
var source = spreadsheet.getRange ("Today!A1");
if (source = "N/A") {
Utilities.sleep(6000);
checkForError();
} else {
moveValuesOnly();
}
}
Locks are for this. Look up lock services in the docs. Use a public lock.
Here's how I used Zig's suggestion (combined with my own check loop) to solve my similar problem:
// Get lock for public shared resourse
var lock = LockService.getPublicLock();
// Wait for up to 120 seconds for other processes to finish.
lock.waitLock(120000);
// Load my values below
// something like sheet.getRange("A1").setFormula('= etc...
// Now force script to wait until cell D55 set to false (0) before
// performing copy / pastes
var current = SpreadsheetApp.setActiveSheet(sheet.getSheets()[1]);
var ready = 1;
var count = 0;
while (true) {
// break out of function if D55 value has changed to zero or counter
// has hit 250
if (count >= 250) break;
// otherwise keep counting...
ready = current.getRange("D55").getValue();
if (ready == 0) {count = 400;}
Utilities.sleep(100);
++count;
}
// wait for spreadsheet to finish... sigh...
Utilities.sleep(200);
// Do my copy and pastes stuff here
// for example sheet.getRange("a1:b1").copyTo(sheet.getRange("a3"), {contentsOnly:true});
// Key cells are updated so release the lock so that other processes can continue.
lock.releaseLock();
// end script
return;
}
This has worked fantastic for me, stopped Google's sporadic service from ruining my work!
Thanks goes to Zig's suggestion!

Google Spreadsheet Triggers

I'm trying to create a trigger that activates on weekdays only and at a specific time, but I don't know what I'm doing wrong. Here is my code.
function createTriggers() {
var days = [ScriptApp.WeekDay.MONDAY, ScriptApp.WeekDay.TUESDAY,
ScriptApp.WeekDay.WEDNESDAY, ScriptApp.WeekDay.THURSDAY,
ScriptApp.WeekDay.FRIDAY];
var d = new Date();
var time = d.toLocaleTimeString();
if (time == '3:05:00 PM EDT') {
for (var i = 0; i < days.length; i++) {
ScriptApp.newTrigger(Lunch1)
.timeBased().onWeekDay(days[i])
.everyMinutes().create();
}
}
}
I'm guessing that you mean you want to make it so a function is triggered (runs) every weekday at a certain time of the day. Here's what I would do.
First, in the Script Editor:
Go to Resources - Current Project's Triggers
Click Add a new trigger
Select the function that you want to be triggered
Change the next box to "Time-driven"
Change the next box to "Hour timer"
Then change the last box to "every hour"
Then at the very beginning of your function, add this code:
var d = new Date();
if (d.getDay() == 6 || d.getDay() == 0) return;
// more info here: http://www.w3schools.com/jsref/jsref_getday.asp
That will stop the rest of the script from running if it's a Saturday or Sunday.
Then, say you want the script to only run at one specific time each weekday, you could set the time-driven trigger to "Every minute" and then add another if statement after the one above:
if (d.getHours() != 15 && d.getMinutes() != 5) return;
That will stop the rest of the script from running if it's not exactly 15:05 (3:05 PM).
This is definitely not the most efficient way to do this, but it works.
Also, this is worth mentioning: I'm not sure if there's anything that would prevent you from triggering the script every minute. Google does impose quotas for what your scripts can do, but I didn't see anything about how many times a script can be triggered in a day. Here's the chart: https://developers.google.com/apps-script/guides/services/quotas
You should do something like this:
function createTriggers(func, hour) {
var weekdays = [ScriptApp.WeekDay.MONDAY, ScriptApp.WeekDay.TUESDAY,
ScriptApp.WeekDay.WEDNESDAY, ScriptApp.WeekDay.THURSDAY,
ScriptApp.WeekDay.FRIDAY];
for (var dayIndex in weekdays) {
var day = weekdays[dayIndex];
ScriptApp.newTrigger(func).timeBased()
.onWeekDay(day).atHour(hour).create();
}
}
This code creates a trigger for weekdays in a specific time calling the func function.