Prevent volatile Google App Scripts function from recalculating unless values have changed - google-apps-script

I have a google app scripts function that updates every time I open the spreadsheet. I only want it to update if the data in the spreadsheet range changes.
I am using the function from the answer to this question: Maps.newDirectionFinder().SetArrive & SetDepart (Google Maps API) in GSheets App Script aren't impacting returned driving times w/ tested scripts
I tried to add a checkValue(e) function but it doesn't run at all. Ideally, I would want something built into the function rather than an external trigger function.

Google Sheets tends to recalculate custom functions every time the spreadsheet is opened. There is nothing you can do in the custom function itself to prevent that from happening.
The custom function you mention uses CacheService to cache results for up to six hours, which speeds performance and helps avoid exceeding rate limits. If it is the rate limits you are concerned with, using the cache should usually be enough to avoid overruns. Note that with this particular custom function, the cache will only be efficiently used when you provide a depart_time in the formula where you call the function.
If you have some other reason that makes it necessary to avoid recalculation, you cannot use a custom function but will have to implement the same functionality through a "regular" function, like this:
/**
* Gets the distance between a number of start addresses and end addresses.
*
* Uses GoogleMapsDistance() https://stackoverflow.com/a/73015812/13045193
*/
function runGoogleMapsDistance() {
const sheet = SpreadsheetApp.getActive().getSheetByName('Sheet1');
const startAddresses = sheet.getRange('A2:A').getValues();
const endAddresses = sheet.getRange('B2:B').getValues();
const result = GoogleMapsDistance(startAddresses, endAddresses);
sheet.getRange('C2')
.offset(0, 0, result.length, result[0].length)
.setValues(result);
}
You can run the function through a button, a custom menu item, a sidebar or a trigger.

Related

How can I make an Import function run without opening the sheet?

I have a Google Sheet that uses an IMPORTRANGE query to combine data from multiple other sheets. This combined import sheet is read by Google AppSheet. We have realized that the data AppSheet is reading is always outdated. It only reads the data as of the last time the sheet was manually opened.
I followed the steps in this post to try to fix this issue by creating this function: function refresh() {SpreadsheetApp.flush()}. I then set up a timed trigger to activate it once an hour. Logs show the function is running, but the data is still not updating until I manually open the sheet.
This is my first time using Apps Script. Any tips/ideas? Is there a different or better way to have the formulas update without opening the file?
Thank you for reading.
SpreadsheetApp.flush() only works for the script execution that calls it. If you need to refresh the data results from a formula it's uncertain how exactly the spreadsheet will respond as most of the formula calculations are done on the client side. You could verify this by yourself by using your web browser developer tools.
Anyway, spreadsheet formulas have several caveats so it will not be extrange that at some point you will have to rethink your solution. Assuming that you want to keep using AppSheet:
Use AppSheet for your front end and some no-code / low-code automation. Keep your app small, if you need many forms / views consider to distribute them among several apps.
Use Google Sheets only for data storage for your AppSheet app. Please bear in mind that it has 10 million cells limit for the whole spreadsheet, so you might want to delete the unused sheets and delete the unused columns and rows on each sheet.
You might use Google Apps Script to do the data import and transformation tasks. If you need that something be updated based on actions done on the AppSheet app, you might use an installable change trigger or use webhook from the AppSheet side to and a "simple" web application using Google Apps Script (you could use GET / POST http requests to trigger some Google Apps Script functions).
Also you might use other programming platforms for the data import / transformation tasks and keep using Google Sheets as your AppSheet database by using the Google Sheets API or other automation tools like Zappier, IFTTT, Integromat among many others.
solution #1
You can try this solution :
define a checkbox (for instance in A1 in tab Sheet1)
set this script
function myFunction() {
var chk = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('Sheet1').getRange('A1')
chk.setValue(false);
SpreadsheetApp.flush();
Utilities.sleep(500);
chk.setValue(true);
}
define a trigger on it
define the formula as follows
=if(A1,importrange("1n-rjSYb63Z2jySS3-M0BQ78vu8DTPOjG-SZM4i8IxXI","A:Z"),"")
when A1 is unchecked, the result will be empty, then check A1 to fill once again the result as expected
solution #2
by script, try for instance
function myFunction() {
var sh = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('Sheet9')
var data = SpreadsheetApp.openById('1n-rjSYb63Z2jySS3-M0BQ78vu8DTPOjG-SZM4i8IxXI').getSheets()[0].getDataRange().getValues()
sh.getRange(1,1,data.length,data[0].length).setValues(data)
}
put a daily triger as needed

Run formula calculation in Google Sheets without to open file

I have a scheduled addon, which writes data periodically into a Google Sheet, without to open file. I have a formula, which should make some calculations on written data and write calculated values into the sheet, so then Data Studio with this sheet as data source updates the visualization.
What is the way to periodically run a formula calculation and write calculated values without manual steps like opening file?
I'm pretty new to this topic: my search approaches to find something like convert formula to app script or schedule formula execution brought me not to any fruitful idea.
PS_ the formula, which should do the calculation is:
=IFERROR(((VLOOKUP(A2,'[1]SV'!$A:$B,2.0))/B2)+(IF(B2=1,"33,9",IF(B2=2,"16,28",IF(B2=3,"10,36",IF(B2=4,"7",IF(B2=5,"5,64")))))+IF(B2=6,"4,13",IF(B2=7,"3,27",IF(B2=8,"2,61",IF(B2=9,"2,18",IF(B2=10,"1,82")))))+IF(B2=11,"1,77",IF(B2=12,"1,81",IF(B2=13,"1,85",IF(B2=14,"1,9",IF(B2=15,"2,04")))))+IF(B2=16,"1,68",IF(B2=17,"1,61",IF(B2=18,"1,65",IF(B2=19,"1,62",IF(B2=20,"1,59","0")))))),0)
If i would know, how to convert the formula to app script, i would manage the rest - i'm familiar with running scripts with time based trigger. Or, maybe, there is a method to run formula on the same scheduled way, like scripts...?
*Disclaimer: I have not tried this and it might not work, if it doesn't, comment and I will try to provide another solution
OK here's what i would try:
In google scripts there is a trigger function, manly used for testing. what it does is it runs your script and sends you an email if there is a error, and hypothetically, you could use this to run you script periodically.
A couple of other things I may try if this doesn't work, leave a comment if you would like me to go in depth about these:
depending on how you want this to work, you could set up a web app to do this, and instead of opining google sheets open the web app with a timer built into the script, that would trigger as long as you had the web page opened.
It might also be possible to run the script automatically with a JavaScript function, requires more research.
If Google Data Studio is not showing you the correct value then you could use a time-driven trigger to save the formula results in another cell. Example:
function respondToTimeDrivenTrigger(e){
var spreadsheet = SpreadsheetApp.getActiveSpreadsheet();
/** The range holding the formula */
var range = spreadsheet.getRange('Sheet1!A1');
/** The value returned by the formula */
var value = range.getValue();
/** Write the value to the cell to the right or the source cell */
range.offset(0,1).setValue(value);
}
Resources
https://developers.google.com/apps-script/guides/sheets
https://developers.google.com/apps-script/guides/triggers/installable

In google sheets can I wrap a standard function in a custom function to control when it is run?

I originally asked this on WebApps, thinking that I was just missing a google trick. There I received the answer that it would require a custom function.
https://webapps.stackexchange.com/questions/129068/recalculate-google-sheet-on-demand
Here is my original question:
I have the following formula in a sheet:
=if(E1="HOLD",,query(Cust_Orders!B6:Z5000,"Select Y,G,I,H,K where H>0 "))
With E1 being a drop down with values HOLD and FETCH
The recalculation when I go to FETCH takes about 13 seconds.
But when it is in HOLD, the query doesn't exceute, and everything goes
blank. Not what I want.
I'm trying to avoid recalculating this query every time I make a
change in the Cust_Orders range, but keep the old values, as two
different pivot tables that are viewed by other people are dependent
on it.
Google spreadsheet recalc settings all are 'on change and foo'
Is there a way to do this?
Custom functions only update when one of their parameters changes. So if the function only depends on the HOLD/FETCH cell, but executes the formula is the query cell, I think I win.
My research:
This answer google sheets custom function built-in function is specific in telling the OP how to do what he wants in the script language. I suspect that rewriting the query in appscript would not be a net win.
This link Using Bound Google Scripts to Generate a Query Object
was proposed as a solution, but this in effect is rewriting my built-in function within the script. I want to use scripts as rarely as possible, and as generically as possible, as it makes long term maintenance and modification easier.
The query function above is an example. I am looking for a more general solution that allows me to use an arbitrary formula using the same script.
This discussion on google product forums: https://support.google.com/docs/forum/AAAABuH1jm01F-8MzzCxbY/?hl=en&gpf=%23!topic%2Fdocs%2F1F-8MzzCxbY says you can't call built-ins from scripts. But the question is almost 4 years old.
I have asked this question there too, but generally asking on Google Product Forums is a Hail Mary.
A viable solution:
An example of a script calling a built-in function.
A link to an add-on that allows recalculation of a range to be toggled on/off
A more general method than custom formulas to control recalc.
I do NOT want a script that emulates the desired built-in inside the script.
A more general method than custom formulas to control recalc.
What I'm doing on project for a client is to have the "expensive" formulas saved as variables on the script and have buttons to freeze/unfreeze the certain ranges (those that have a high impact on the recalculation time.
The "unfreeze" button adds the formulas to the spreadsheet
The "freeze" button put the formulas results over the range used by the formulas
There is a document property that stores the frozen/unfrozen spreadsheet state
A sidebar is used to show the buttons the spreadsheet status.
An example of a script calling a built-in function.
Scripts can get values, display values, formulas, and other stuff but they can not call built-in functions.
In the case of my client, they have one array formula (IMPORTRANGE, QUERY, FILTER, ARRAYFORMULA, etc) by sheet, all the formulas that are been included are on A1. The formulas are saved in an array of objects of the following form
var formulas = [
{
name:'Sheet1',
address:'A1',
formula:'=IMPORTRANGE(...)'
}
]
The address property, is included for future improvements.
The key code lines of the "freeze" function are
var dataRange = sheet.getDataRange();
dataRange.copyTo(dataRange, SpreadsheetApp.CopyPasteType.PASTE_VALUES, false);
Please note that the above lines copy-paste-as-values the whole data range.
The key code lines of the "unfreze" function are
formulas.forEach(function(item){
var sheet = spreadsheet.getSheetByName(item.name);
sheet.clear();
sheet.getRange(item.address).setFormula(item.formula);
});
Please note that the above lines clear the whole sheet.

Google Sheets Script Timeout Workaround

I'm building a script that uses a looping ImportHTML command to web scrape weather data based on zip code, and am currently running into an issue with the execution timing out every time the script is run.
The current way I have the script set up produces a correct result when run, but given that the script is pulling data from several hundred sources, it is taking a while and will not complete within the current time limit of Google scripts.
The sheet running the script utilizes 3 tabs:
ZIPS, which contains a list of zip code link values pulled from the
site that weather data is to be pulled from
Blank, which is simply an intermediary sheet used in the execution of the script
Result, where the final output is to be placed
In order to try and reduce the amount of read/write as much as possible, I changed the code from writing each result of the ImportHTML command as it was executed to appending to an array and writing this array at the end of the script. The code in it's current form is as follows:
function getTemps() {
var googleSheet = SpreadsheetApp.getActive();
// Read in Zip code link values
var sheet = googleSheet.getSheetByName('ZIPS');
var zipArray = sheet.getDataRange().getValues();
var arrayLength = zipArray.length;
//Set up sheet values
var blankSyntaxA = 'ImportHtml("https://www.wunderground.com/cgi-bin/findweather/getForecast?query=pz:';
var blankSyntaxB = '&zip=1", "table", 1)';
var tempResult = [];
// Writing Section
var sheet = googleSheet.getSheetByName('Blank');
for (var i = 0; i < arrayLength; i++)
{
var liveSyntax = blankSyntaxA+zipArray[i][0]+blankSyntaxB;
sheet.getRange('A1').setFormula(liveSyntax);
var importedData = sheet.getDataRange().getValues();
tempResult = tempResult.concat(importedData);
}
var sheet = googleSheet.getSheetByName('Result');
sheet.getRange(1,1,tempResult.length,8).setValues(tempResult);
}
I know the run time of the script could be reduced by eliminating the read/write contained within the For loop, but I'm not sure how to obtain the necessary HTML table without running the ImportHTML command within the 'Blank' sheet. Is there a way to run that command to fill the 'importedData' array without writing to a sheet?
Alternatively, I had considered utilizing a check on the runtime of the function and implementing a break as it neared the ~5 minute runtime limit, followed by a recursive call back to the original function, but I wasn't sure if this would actually mitigate the runtime issue, or even be possible given the nature of the recursive call.
Any advice on how this script could be modified to run within the script timeout parameter or modified to produce the complete desired outcome with all the necessary imported data would be appreciated. Thanks!
Is there a way to run that command to fill the 'importedData' array without writing to a sheet?
IMPORTHMTL is a Google Sheets built-in spreadsheet function. This kind of functions can't be ran / evaluated by Google Apps Script.
Related
How to evaluate a spreadsheet formula within a custom function?
Alternatively, I had considered utilizing a check on the runtime of the function and implementing a break as it neared the ~5 minute runtime limit, followed by a recursive call back to the original function, but I wasn't sure if this would actually mitigate the runtime issue, or even be possible given the nature of the recursive call.
Rather than a "mitigator" this is a workaround. There are several techniques like batch processing and parallel processing.
Reference
Exceeded maximum execution time in Google Apps Script
From answer to Threading in Google App Script
There is a great example from Bruce
Mcphearson. His example
Parallel Processing in Apps
Script uses
Map Reduce in exercise. He is utilizing triggers as well, but perhaps
may provide some different perspective.
Another alternative is to sign in to the Early Access Program to extend the execution time limit to 30 minutes.

How to pause App Scripts until spreadsheet finishes calculation

Because google spreadsheets does not support iterations, I wrote my own simple app script to adjust an input based upon the calculation of the spreadsheet output. However, after I change the input variable, the spreadsheet recalculates but app scripts does not seem to wait for that recalculation so I end up retrieving values such as "Thinking..." or "#NA". Is there a way to pause a script and wait for the calculation to complete before moving to the next line in the script?
Currently, I am just using a loop to watch the cell but I wanted to find out if there was a more elegant way to pause the execution until the sheet was done calculating.
I write a lot of Excel Macros and Excel VBA always waits for the calculation to complete before moving to the next line in the code. Apps Script does not seem to do this so I am hoping there is an easy way to do this.
A second question: Because this iteration can take some time, how does one interrupt and terminate a script from running? I can't seem to find a way to do this.
Here is a very simple way of preventing the next script from starting until the current script completes in google apps scripts. Just add a call for testWait() after each script you are processing successively. The SpreadsheetApp.flush() also seems to reset the timeout timer on the spreadsheet back to the default 5min so you have more time to process multiple scripts in one go.
//holds processing of next script till last one has completed
function testWait(){
var lock = LockService.getScriptLock(); lock.waitLock(300000);
SpreadsheetApp.flush(); lock.releaseLock();
}
Scripts timeout after about 6 minutes to prevent infinite loops and constantly running programs. I don't think there's a manual way to stop a script.
Edit: Oops, forgot to answer your first question: I thought onEdit ran after values were recalculated, but apparently I don't use enough formulas to see this. If it's not waiting, then the best way is to do something like this:
while(value === "Thinking..." || value === "#NA") {
Utilities.sleep(10000);
}
It pauses the script for a few seconds and then checks again.
I also write a bit in Excel VBA, where the native Calculation functions come in handy. I've run across the same problem in Google Apps/Docs several times, wanting to execute code whenever calculations are complete. I wonder why there's no native function in Google Apps/Docs to handle this. Anyhow, I wrote this code to solve the problem. Hope it helps.
function onEdit() {
Refresh();
};
function Refresh () {
var sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Sheet1");
var sheet2 = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Sheet2");
// set the range wherever you want to make sure loading is done
var range = sheet.getRange('A:A')
var values = range.getValues();
var string = values.toString();
var loading = "Loading";
do{
var randomWait = Math.floor(Math.random()*1+0); randomWait;
} while (string.search(loading) ==! 0);
range.copyTo(sheet2.getRange('A1'), {contentsOnly:true});
customMsgBox();
};
function customMsgBox() {
Browser.msgBox("Data refreshed.");
};
Here's an example in action:
https://docs.google.com/spreadsheet/ccc?key=0AkK50_KKCI_pdHJvQXdnTmpiOWM4Rk5PV2k5OUNudVE#gid=0
Make a copy if you want to play around with it.
I had the same problem. I resolved it by using 2 scripts as I needed to be sure the spreadsheet has the data I need.
The first script provides a seed value to populate the spreadsheet through a IMPORTXML function.
The second script processes this data.
I used time based triggers to run the scripts allowing for sufficient time for the first script to complete
You could put the recursive structure in the code.js file, rather than on the spreadsheet. Javascript can handle recursion really well. You could then have the script update the spreadsheet on each iteration (or each 100).