I am getting the following error in my Google sheet:
Service invoked too many times for one day: urlfetch
I know for a fact I am not making 100k calls, but I do have quite a few custom functions in my sheet. I tried to make a new sheet and copy/paste the script into that one, but I still get the same error. I then switched my account, made a new sheet, added the code, and I still got the error.
Is this just because I am on the same computer? Is Google smart enough to realize I am the same person trying to do it? I highly doubt that, so I am wondering why it would be throwing this error, even after switching accounts and making a new sheet.
In addition to that, is there any way to make sure I don't go over the limit in the future? This error sets me back at least a day with what I was working on. I do plan to write a script to just copy/paste the imported HTML as values into another sheet, but until I get that working, I need a temporary fix.
Sample code:
function tbaTeamsAtEvent(eventcode){
return ImportJSON("https://www.thebluealliance.com/api/v3/event/" + eventcode + "/teams?X-TBA-Auth-Key=" + auth_key);
}
function ImportJSONForTeamEvents(url, query, options){
var includeFunc = includeXPath_;
var transformFunc = defaultTransform_;
var jsondata = UrlFetchApp.fetch(url);
var object = JSON.parse(jsondata.getContentText());
var newObject = [];
for(var i = 0; i < object.length; i++){
var teamObject = {};
teamObject.playoff = object[i].alliances
newObject.push(teamObject);
}
return parseJSONObject_(object, query, "", includeFunc, transformFunc);
}
That is one "set" of code that is used for a specific function. I am pulling two different functions multiple times. I have about 600 of one function, and 4 of another. That would only be just over a thousand calls if all were run simultaneously.
I should note that I also have another sheet in my drive that automatically updates every hour with a UrlFetch. I do no believe this should affect this though, due to the very low pull rate.
I had a similar issue even though I was only calling two fetch calls in my functions and each function per data row. It exponentially grew, and with my data changing, every recalculate call also called those functioned, which VERY quickly hit the max.
My solution? I started using the Cache Service to temporarily store the results of the fetch calls, even if only for a few seconds, to allow for all the cells triggered by the same recalculation event to propagate using only the single call. This simple addition saved me thousands of fetch calls each time I accessed my sheets.
For reference:
https://developers.google.com/apps-script/reference/cache?hl=en
Related
I have a GoogleSheet with basically two sheets, which are very similar in terms of data collected.
I need to calculate same values for both sheets, but source data is in different columns.
Therefore I created three files in AppsScript:
Common.gs - with common function definitions
sheet1.gs
sheet2.gs - both sheet1 and sheet2 have only definitions of proper ranges in particular columns and one function to run script, which essentially calls functions defined in Common.gs, like so in sheet1.gs:
var sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("sheet1")
var createdColumn = sheet.getRange("E2:E").getValues()
var ackColumn = sheet.getRange("G2:G").getValues()
var resColumn = sheet.getRange("I2:I").getValues()
var timeToAckColumn = sheet.getRange(2,14,ackColumn.length,1)
var timeToResColumn = sheet.getRange(2,15,resColumn.length,1)
var yearAndWeekRange = sheet.getRange(2,16,createdColumn.length,2)
function calculateMetricsSheet1() {
calculateTimeDiff(createdColumn, ackColumn, timeToAckColumn)
calculateTimeDiff(ackColumn, resColumn, timeToResColumn)
calculateWeek(createdColumn, yearAndWeekRange)
}
example function implementation (they are basically very similar with minor differences):
function calculateWeek(createdColumn, yearAndWeekRange) {
var arrData = []
for(var i=0;i<createdColumn.length;i++) {
if(createdColumn[i][0].toString()=="") {
arrData.push(["",""])
continue
}
var createdDate = new Date(createdColumn[i][0])
var year = createdDate.getFullYear()
var week = Utilities.formatDate(createdDate, "GMT+1", "w")
arrData.push([year, week])
}
yearAndWeekRange.setValues(arrData)
}
the sheet2.gs is basically different column definitions, the functions called within calculateMetricsSheet2() are the same.
So what is the problem?
The script works perfectly fine for sheet2.gs, but for sheet1.gs it does collect proper data, calculates proper data, but the data does not appear in proper columns after Range.setValues() call.
No exceptions or errors appear in the console.
Documentation does not provide any kind of information what could be the problem.
I have really ran out of ideas what could be the cause of the issue.
Does anyone have any idea what is going on?
edit: It may be useful to put emphasis on the fact that each script runs function calling 3 other functions -> all of them end with Range.setValues({values}). And for one sheet all of them work, and for the other - none.
That's the reason I assume there is something wrong with the sheet itself, maybe some permissions/protection? But I couldn't find anything :(
edit2: I modified my code to iterate through the sheet 10 rows at a time, because I thought maybe when I get a whole column, something bad happens with data and breaks setValues() function.
Unfortunately - even if my code iterated 1 row at a time, it still did not work on sheet1, but worked on sheet2. So not a data problem.
The code you show always puts values in yearAndWeekRange which is always in the 'sheet1' sheet. To put the data into another sheet, you need to change the target range appropriately.
The dimensions of the range must match the dimensions of the array you put there. Use this pattern:
yearAndWeekRange.offset(0, 0, arrData.length, arrData[0].length).setValues(arrData);
I found out what is the problem.
Two scripts were pretty identical, even with naming of variables - ie ackColumn, resColumn etc.
Those were stored as a global variables, so even if I was running script1.gs, it used global variables from script2.gs, effectively writing proper data to wrong sheet.
separating global variables names fixed the issue.
Perhaps a rookie mistake, but I missed the fact, that if I have a variable defined outside any function, it becomes global and could be overwritten from other file
I've altered a script that gets data from MailChimp and then displays it in Google Sheets. However, the the data comes out in a random order, rather than ordered (e.g. by "campaign date") and also creates duplicates.
So I added two scripts that
Clear the previous data
Sort in date order
I want to run these scripts in a specific order so:
The cell ranges clear
The data imports from mailchimp
The data is rearranged in date order
Ideally I'd like to refresh this data every 15 minutes. What's the best way to do this? I can post my code but it's quite bloated and messy.
I have a similar problem which I have solved by creating a "parent" function that runs all the other functions in order. For robustness I have also included a simple logging function that logs this to a sheet so I can easily see that the functions have run. However, if you're running this every 15 mins that could quickly get out of hand, so you could either bin the logging, or use the built-in Logger (see reference here)
function runAllFunctions() {
logit("Clearing data");
clearCellRange();
logit("Data Cleared. Importing MailChimp Data");
importMailchimpData();
logit("MailChimp data imported. Reordering data.");
orderData();
logit("Data reordered. Update complete");
}
;
function logit(message) {
var logBook = SpreadsheetApp.openById("<insert ID here>")
var logSheet = logBook.getSheetByName("Log")
logSheet.appendRow([new Date(),message]);
};
I've been using Google Apps Script for about 2 months now to automate updating spreadsheets with information from a database. The scripts have been running this amount of time without any problems, until this week. Out of nowhere three of the scripts started failing. After diving into it I managed to fix two of the three spreadsheets, but one remains unresolved. One of the spreadsheets I fixed by manually clearing the sheet, despite that the script itself already does that. The other I had to create a new sheet for.
When running the script the log specifies that it is inserts the data as it should, but it clearly doesn't. It also displays an incorrect runtime (says between 15-20 seconds, but reality is 2-3 minutes). The exact error is
Execution failed: We're sorry, a server error occurred. Please wait a bit and try again.
The script itself triggers on a weekly basis. It duplicates the previous weeks sheet to maintain certain formatting and renames it, runs a query and inserts the resultset into the spreadsheet. Then it inserts a formula and that's it. The script still works. When using the script on a new spreadsheet it runs without a problem. It simply refuses to run on the existing spreadsheet, therefore I believe that that is where the problem lies.
I've tried manually clearing the old sheet (so before it gets duplicated), creating a new one, removing older sheets but all to no avail. The only solution I've encountered so far seems to be creating a new spreadsheet, but this isn't really an option. Some of these sheets are used by 40+ people on a daily basis. Having to replace and reshare a spreadsheet when this happens makes it a less reliable solution.
So my questions are why is this happening, and are there any solutions that will allow the use of the existing spreadsheet? Even if there is background data somewhere causing an error shouldn't that be removed when clearing a sheet.
Just in case I'll post the code snippit that writes to the spreadsheet, but I'm confident that this isn't the cause as it runs fine on a new spreadsheet and has run without a problem for nearly 2 months:
function buildWeek() {
var doc = SpreadsheetApp.getActiveSpreadsheet();
latestWeek();
var sheets = doc.getSheets();
var latestSheet = sheets[0];
Logger.log(latestSheet.getName());
var conn = Jdbc.getConnection("connection details");
var stmt = conn.createStatement();
var rs = stmt.executeQuery("query");
//Writing range start
var cell = latestSheet.getRange('A2');
var row = 0;
while (rs.next()) { // Inserts data from rows
var colcount = rs.getMetaData().getColumnCount();
for (var col = 0; col < colcount; col++) {
Logger.log(rs.getString(col + 1));
cell.offset(row, col).setValue(rs.getString(col + 1));
}
row++;
}
I wrote a custom google app script function in a script associated with my google doc spreadsheet. The function calls a third party service to get data. I can put the function in a cell:
=myfunction("something")
and it returns the correct value from the service. However, how can I keep this value updated so that it's showing the latest data from the service?
Update
For example:
=temperature("90120")
For getting the current temperature in a given zip code. Also my sheet may have dozens or hundreds of these so I'd prefer something that is performant and maintainable. It doesn't truly need to be continuous, polling once a minute or ideally more frequently could work. I'm wondering if there's some way from the script to set a timer to run to update a range of cells?
Not sure why you need dozens or hundreds.
1. Is the spreadsheet used by another process?
2. Is the spreadsheet visually reviewed by actual users?
If #1, you could replace the spreadsheet with a custom API via the content service to return JSON results for all temperatures.
If #2, you may hit limits or performance issues with so many functions firing so often. Why should fire the functions if no one is viewing the results. Alternatively, you could make it an on-demand with a custom menu option.
I have a similar problem.
This is how I am doing it atm, but its not the best solution. I am looking for a better one.
If any value at sheet Prices and column D changes.
Meaning if any cell value changes in the whole column it updates the custom function value.
//Search Price sheet with the given name. Return price. dummy param updates google ss once the "Prices" sheet values changed.
function searchPrice(price,dummy)
{
var SPREADSHEET_NAME = "Prices";
var SEARCH_COL_IDX = 2;
var RETURN_COL_IDX = 3;
var values = SpreadsheetApp.getActiveSpreadsheet().getSheetByName(SPREADSHEET_NAME).getDataRange().getValues();
for (var i = 0; i < values.length; i++)
{
var row = values[i];
if (row[SEARCH_COL_IDX] == price)
{
return row[RETURN_COL_IDX];
}
}
}
This is how you call it =searchPrice(B8,Prices!D:D)
Just give your custom function a dummy param. It doesn't do anything in the custom function.
I have a Google spreadsheet with an onEdit() trigger to create a second time based trigger.
Simply: when the status column is edited to 'Approved' a trigger is created to send a feedback email on a supplied project completion date.
var oneTimeOnly = ScriptApp.newTrigger("emailFeedback").timeBased().at(endDate).create();
I wish to pass a variable to the second trigger. I could create Project Property or add a column in the spreadsheet. However it would be simpler to pass the variable when creating the trigger.
When I insert any additional characters inside the newTrigger quotes this causes the entire contents of the function to be stored in the trigger (which subsequently fails).
var oneTimeOnly = ScriptApp.newTrigger("emailFeedback<strong>(regEmail)</strong>").timeBased().at(endDate).create();
.
Is there a way to store a variable inside the trigger?
Using ScriptDB and new Function(), I was able to create a method for creating dynamic trigger functions.
The gist of the solution is to store the code you want to trigger is the db with the parameters you want to pass:
"myFunction('Hello world')"
Then, when the script starts, as a global variable, you attach newly created functions from your ScriptDB. (I've done this dynamically in my link below.)
globalFunctions.callThisOne = new Function("e", "myFunction("Hello world"));
Finally, when you create your trigger, you created it using the globally accessible function as such:
ScriptApp.newTrigger("globalFunctions.callThisOne").timeBased().everyDay(1).create();
I have written up a short post about this and posted the source. Hopefully it's useful.
You can read more about it here: http://goo.gl/wbUqH6
Or see the code here: http://goo.gl/zjUiYe
Sorry, there is not a way to do this.
As I understand correctly, the question was how to pass data to a time triggered function in a google script project. Eoin described a situation, but you may facing to many.
A classic situation when your script proccesses a complex spreadsheet that may run for long minutes. As you may probably know each script has about 6 minutes runtime limit. This situation you should break your script smaller logical partitions and at the end of each one can create a new trigger for the next part. Okay, but the next part must know about some data of the current running script's variables. Because no way to pass these data via newTrigger() you can create a snapshot and put into the script property context in a serialized way.
An easy way to do by ScriptProperties.setProperty().
Use ScriptProperties.setProperty() to store serialized parameters that can be accessed by trigger method.
#user2166613 is right, but a bit short. Here is how to do it.
I show an example that uses an after() trigger. This is a really interesting use case, as it allows to decouple time consuming tasks from e.g. web app calls, so the calls return control immediately and the processing is done in the background.
In my example I adapt the column widths of a sheet in a function that this run delayed.
// call this function to set a time based trigger and transfer parameters
function setTimeTrigger_AdaptColumnWidths() {
var ssId = SpreadsheetApp.getActiveSpreadsheet().getId();
var wsId = SpreadsheetApp.getActiveSpreadsheet().getActiveSheet().getSheetId();
var scriptProperties = PropertiesService.getScriptProperties();
scriptProperties.setProperty('spreadsheetId', ssId);
scriptProperties.setProperty('worksheetId', wsId);
// Delay 10 secs, but real execution time may vary up to 15 min!
ScriptApp.newTrigger('adaptColumnWidths').timeBased().after(10000).create();
}
// this function is called by the trigger
function adaptColumnWidths() {
var scriptProperties = PropertiesService.getScriptProperties();
ssId = scriptProperties.getProperty('spreadsheetId');
wsId = scriptProperties.getProperty('worksheetId');
// getSheetById() is a custom function – see below – not yet in Spreadsheet Class!
sheet = getSheetById(SpreadsheetApp.openById(ssId), wsId);
// now do what we want to do in the timeBased trigger
for (var i = 1; i <= sheet.getLastColumn(); i++){
sheet.autoResizeColumn(i);
}
}
// -----
// custom function – hopefully this will become a method of Spreadsheet Class any time soon
function getSheetById(ss, wsId) {
var sheets = ss.getSheets();
for (var i=0; i<sheets.length; i++) {
if (sheets[i].getSheetId() == wsId) return sheets[i];
}
}
Please be aware that your are storing in a dataspace here that is common to all your function calls. So multiple calls in a short time can overwrite each others parameters.
In my use case this is not a problem as the spreadsheet and the worksheet will not change. But do NOT try to transfer data that can change from call to call.
The real moment of execution can vary up to 15 minutes (no matter what exact time you ask for), so there is plenty of room for multiple function calls to interfere with each other!!!