I have a scheduled addon, which writes data periodically into a Google Sheet, without to open file. I have a formula, which should make some calculations on written data and write calculated values into the sheet, so then Data Studio with this sheet as data source updates the visualization.
What is the way to periodically run a formula calculation and write calculated values without manual steps like opening file?
I'm pretty new to this topic: my search approaches to find something like convert formula to app script or schedule formula execution brought me not to any fruitful idea.
PS_ the formula, which should do the calculation is:
=IFERROR(((VLOOKUP(A2,'[1]SV'!$A:$B,2.0))/B2)+(IF(B2=1,"33,9",IF(B2=2,"16,28",IF(B2=3,"10,36",IF(B2=4,"7",IF(B2=5,"5,64")))))+IF(B2=6,"4,13",IF(B2=7,"3,27",IF(B2=8,"2,61",IF(B2=9,"2,18",IF(B2=10,"1,82")))))+IF(B2=11,"1,77",IF(B2=12,"1,81",IF(B2=13,"1,85",IF(B2=14,"1,9",IF(B2=15,"2,04")))))+IF(B2=16,"1,68",IF(B2=17,"1,61",IF(B2=18,"1,65",IF(B2=19,"1,62",IF(B2=20,"1,59","0")))))),0)
If i would know, how to convert the formula to app script, i would manage the rest - i'm familiar with running scripts with time based trigger. Or, maybe, there is a method to run formula on the same scheduled way, like scripts...?
*Disclaimer: I have not tried this and it might not work, if it doesn't, comment and I will try to provide another solution
OK here's what i would try:
In google scripts there is a trigger function, manly used for testing. what it does is it runs your script and sends you an email if there is a error, and hypothetically, you could use this to run you script periodically.
A couple of other things I may try if this doesn't work, leave a comment if you would like me to go in depth about these:
depending on how you want this to work, you could set up a web app to do this, and instead of opining google sheets open the web app with a timer built into the script, that would trigger as long as you had the web page opened.
It might also be possible to run the script automatically with a JavaScript function, requires more research.
If Google Data Studio is not showing you the correct value then you could use a time-driven trigger to save the formula results in another cell. Example:
function respondToTimeDrivenTrigger(e){
var spreadsheet = SpreadsheetApp.getActiveSpreadsheet();
/** The range holding the formula */
var range = spreadsheet.getRange('Sheet1!A1');
/** The value returned by the formula */
var value = range.getValue();
/** Write the value to the cell to the right or the source cell */
range.offset(0,1).setValue(value);
}
Resources
https://developers.google.com/apps-script/guides/sheets
https://developers.google.com/apps-script/guides/triggers/installable
I have a sheet that pulls alot of finance data from alot of web pages using HTML. Problem is that it slows down alot. Plus it starts giving errors due to high number of HTMLs and etc.. i looked for the following solutions:
Having an app script to make refreshes controlled..(not so effective)
Any way i could copy paste the old data automatically till new data comes in and updates old pasted values..(also not effective as i couldnt find an automated method)
Tried finding a way to play with google sheet auto-calculations to help....(also failed in that)
IS there a way to ristrict the sheet from auto refreshing so many times...?
In the documentation you can see that:
Functions that pull data from outside the spreadsheet recalculate at the following times:
ImportRange: 30 minutes
ImportHtml, ImportFeed, ImportData, ImportXml: 1 hour
GoogleFinance: may be delayed up to 20 minutes
If you want to recalculate the value for your importHTML functions with a lower frequency you should definitely use Apps Script to do the data fetch and then populate your Spreadsheet with the information.
You can use UrlFetchApp class on Apps Script to get the data and define your timing logic for the updates.
function myAppScriptFunction() {
var urls = ["your-website-url1", "...", "your-website-urlN"];
var response = UrlFetchApp.fetchAll(urls);
//... Parse the response and store the information you need in a table-like data structure
// Let's assume the variable parsedResponseData is created
var ss = SpreadsheetApp.getActiveSheet();
ss.getRange("your-range").setValues(parsedResponseData);
}
You should now wrap your function in your custom time logic to manage the updates.
You can achieve this with a time-driven trigger on Apps Script.
function triggerSetup() {
ScriptApp.newTrigger('myAppScriptFunction')
.timeBased()
.everyHours(6)
.create();
}
References:
Time-driven triggers
UrlFetchApp
Quota Limits
you may try this addon which freezes your spreadsheet on demand:
https://gsuite.google.com/marketplace/app/spreadsheet_freezer/526561533622
For Example, a common workflow that I use is :
var spreadsheet = SpreadsheetApp.openById(spreadsheetId);
var sheet = spreadsheet.getSheetByName(sheetName);
I call these 2 functions every time I want to read/write to my sheet. However, if there was a way to cache the result of
SpreadsheetApp.openById(spreadsheetId), which returns a Spreadsheet Object, then I would assume to have increased performance, as it forgoes function calls. So far, both PropertiesService and Cache only support strings, hence the need for a conversion. Using JSON.stringify on spreadsheet or sheet doesn't seem to work... any ideas?
I am using the Spreadsheet Services in Google app script to retrieve some data from the Internet and then mess a bit with it. The problem is when I set the ImportHtml value if the data set is larger than say a few rows I do not have access right away to the imported range and thus an error is thrown in the script:
example:
// create tmp sheet and import some data.
var sheet = this.createTmpSheet(); // custom method to create a temp sheet.
sheet.getRange('A1').setValue('=ImportHtml("someUrl","table",1)');
// at this point usually I can access the range
var range_to_copy = sheet.getDataRange();
// However if the data is more than 10-15 rows I get invalid dimention for range..
Any ideas how to wait for the 'readiness' of the import? None of the usuall triggers seemd like an appropriate choice. All I need is to have flow control in such a way as to be notified once the import completes, usually under 10 seconds.
As you noted, there's no trigger that will tell you when a spreadsheet recalculation has completed. (That's what is going on after you update a formula in a cell.)
You can induce your script to wait 10 seconds by using Utilities.sleep(10000);. Generally it's bad programming practice to rely on delays, but it's almost your only option here.
The other option would be to perform the html query yourself, parse the table of data into an array, and write it to the new sheet. If you're interested in that, have a look at html div nesting? using google fetchurl, to see how you could obtain your table of information.
Dont do it like that. Use urlFetch to get the data and write it yourself to the spreadsheet.
Because google spreadsheets does not support iterations, I wrote my own simple app script to adjust an input based upon the calculation of the spreadsheet output. However, after I change the input variable, the spreadsheet recalculates but app scripts does not seem to wait for that recalculation so I end up retrieving values such as "Thinking..." or "#NA". Is there a way to pause a script and wait for the calculation to complete before moving to the next line in the script?
Currently, I am just using a loop to watch the cell but I wanted to find out if there was a more elegant way to pause the execution until the sheet was done calculating.
I write a lot of Excel Macros and Excel VBA always waits for the calculation to complete before moving to the next line in the code. Apps Script does not seem to do this so I am hoping there is an easy way to do this.
A second question: Because this iteration can take some time, how does one interrupt and terminate a script from running? I can't seem to find a way to do this.
Here is a very simple way of preventing the next script from starting until the current script completes in google apps scripts. Just add a call for testWait() after each script you are processing successively. The SpreadsheetApp.flush() also seems to reset the timeout timer on the spreadsheet back to the default 5min so you have more time to process multiple scripts in one go.
//holds processing of next script till last one has completed
function testWait(){
var lock = LockService.getScriptLock(); lock.waitLock(300000);
SpreadsheetApp.flush(); lock.releaseLock();
}
Scripts timeout after about 6 minutes to prevent infinite loops and constantly running programs. I don't think there's a manual way to stop a script.
Edit: Oops, forgot to answer your first question: I thought onEdit ran after values were recalculated, but apparently I don't use enough formulas to see this. If it's not waiting, then the best way is to do something like this:
while(value === "Thinking..." || value === "#NA") {
Utilities.sleep(10000);
}
It pauses the script for a few seconds and then checks again.
I also write a bit in Excel VBA, where the native Calculation functions come in handy. I've run across the same problem in Google Apps/Docs several times, wanting to execute code whenever calculations are complete. I wonder why there's no native function in Google Apps/Docs to handle this. Anyhow, I wrote this code to solve the problem. Hope it helps.
function onEdit() {
Refresh();
};
function Refresh () {
var sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Sheet1");
var sheet2 = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Sheet2");
// set the range wherever you want to make sure loading is done
var range = sheet.getRange('A:A')
var values = range.getValues();
var string = values.toString();
var loading = "Loading";
do{
var randomWait = Math.floor(Math.random()*1+0); randomWait;
} while (string.search(loading) ==! 0);
range.copyTo(sheet2.getRange('A1'), {contentsOnly:true});
customMsgBox();
};
function customMsgBox() {
Browser.msgBox("Data refreshed.");
};
Here's an example in action:
https://docs.google.com/spreadsheet/ccc?key=0AkK50_KKCI_pdHJvQXdnTmpiOWM4Rk5PV2k5OUNudVE#gid=0
Make a copy if you want to play around with it.
I had the same problem. I resolved it by using 2 scripts as I needed to be sure the spreadsheet has the data I need.
The first script provides a seed value to populate the spreadsheet through a IMPORTXML function.
The second script processes this data.
I used time based triggers to run the scripts allowing for sufficient time for the first script to complete
You could put the recursive structure in the code.js file, rather than on the spreadsheet. Javascript can handle recursion really well. You could then have the script update the spreadsheet on each iteration (or each 100).