When I create a daily time-based trigger for the Google app script associated with my Google spreadsheet, I am prompted to select an execution time that is within an hour-long window, and it appears that a cron wrapper randomly assigns an exact execution time within that hour-long interval.
Because my application's specific use case has several data dependencies which may not be completed early in the hour, I was forced to divide my application into several stages, with separate triggers each delayed by an hour, to insure that the required data would be available.
For example, the trigger time that was initially assigned for my script was 6:03AM, but the data which usually arrived at 5:57AM, occasionally did not arrive until 6:10AM and the script had nothing to process for that day. As a blunt force solution, I deleted the 6-7AM trigger and re-created it to execute in the 7-8AM time slot to insure the required data was available. This required that the second stage of the script had to be moved to 8-9AM, resulting in script results which could be delayed by as much as 2-3 hours.
To improve this situation, I am contemplating integrating the two script processing stages and creating a more accurate script execution trigger time, say 6:30AM to be safe. Does anyone know if:
Is it possible, other than by observing daily processing, to discover the exact trigger execution time that has been assigned, and
If randomly assigned, can script triggers be created and deleted until an acceptably precise execution time is obtained?
Thanks in advance for any guidance provided.
If accuracy is paramount, you can forgo using apps script triggers altogether and leverage a 3rd party tool instead.
I'd recommend using cron-job.org. This service can create cron jobs that make POST requests to a url endpoint you specify, and you can schedule times accurate to a minute. To use it with Apps Script implement a doPost() to handle post requests and deploy your script as a Web APP. You then create a cron job using the service and pass it the web app's URL as an endpoint.
The cron job will fire at the scheduled time and you can perform any requisite operations inside the doPost() in response to the incoming POST request.
Thank you to random parts and Dimu Designs for the guidance. Based upon experimentation, here are the answers to my questions:
Is it possible, other than by observing daily processing, to discover the exact trigger execution time that has been assigned? Answer: No way except by observing the random trigger time assigned within the requested hour window.
If randomly assigned, can script triggers be created and deleted until an acceptably precise execution time is obtained? Answer: Yes. I adjusted my script's assigned execution time by observing a trigger's execution time (via email message timestamp), and deleting, recreating, and observing the randomly assigned trigger execution time until I got an acceptable minute within the requested hour window.
Related
I have a Google Calendar Trigger which fires whenever a change in ones users calendar is detected. The Trigger sends some data to a 3rd party system and does some logic. It is important that concurrent Triggers are not sending at the same time.
Therefore I am using the Lock Service which prevents exactly this.
var lock = LockService.getScriptLock();
try {
lock.waitLock(30000); // wait 30 seconds for others' use of the code section and lock to stop and then proceed
} catch (e) {
Logger.log('Could not obtain lock after 30 seconds.');
}
// this can take a few seconds
doSomeStuff();
lock.releaseLock();
// END - end lock here
return;
The problem is that sometimes the execution of a Trigger can take up to 10 seconds and it also can happen that sometimes multiple trigger are executed where every Trigger holds the lock.
This means that that the total execution time can easily be exceeded.
In my opinion this can not be solved with Google AppScript only. In order to handle this I guess the best way would be to have some kind of a queue where the trigger writes to and then Google Cloud Functions takes the data from this queue and runs the logic.
How to solve this?
The only queue like resource I could find in the Google World was Cloud Tasks but I am not sure if this is the best fit (it also needs a lot of setup work) when there are 10K Users where everyone has a calendarTrigger running.
Another Idea was that every UserTrigger writes the data to a database like Firestore and in the backend Google Cloud Functions read and delete the data to run the logic. So in this case the db would work also as a simplified queue.
In both cases the actually logic doSomeStuff() has to run in Google Cloud Functions. However since there are 10K users where every user can fire multiple Triggers I want to have full control over the amount of Google Cloud Functions run at the same time.
Summary
So in my head the solution is something like this:
trigger writes to
-> Queue (DB? Cloud Tasks?)
-> Backend Function watches Queue (Cloud Functions? Compute Engine?)
-> starts (max amount) Cloud functions to dequeue and run actual logic
Questions:
is such a "complex" structure involving those resources necessary?
if so, what are the best Google Cloud Resources to achieve this?
I'm working on an addon that needs to update spreadsheets in user specified intervals (from once an hour to once a year) - is there a smart, proven way to handle that with automatic trigger check/installation and multiple users between multiple spreadsheets?
From what I understand the triggers are installed individually per user - can anyone reveal the mystery behind how the time-driven triggers for add-ons work in the following cases?
The check whether triggers are installed doesn't 'see' if it's been been installed by other users and it can't manipulate it (eg. delete someone else's trigger when no longer needed). Any ideas on how to cope with that? Having 50 users with installed hourly triggers is not ideal.
Multiple spreadsheets with the addon installed: will the trigger run the called function in all of them simultaneously or do they require individual triggers per sheet? Dry testing seems to indicate that the triggers need to be set up individually per sheet, though would be great if someone could confirm their experience from an add-on that's already online.
In order to control a potential chaos I'm considering adding a 'last updated' property to scheduled update instances and have the triggered function compare the time difference between that and current time, but then it's still going back to having that function run every hour multiplied by the amount of users.
will the trigger run the called function in all of them simultaneously or do they require individual triggers per sheet?
Installable triggers are installed per [sheet | document | form] per [user] (well, to be precise, per script project). Given the fact that they are isolated for each user, they are also triggered respective to each user (as opposed to simple triggers).
Installable onOpen | onEdit | onChange
There is a case, though, where this might not be as easy - see installable onOpen triggers section in official guide. Although the general case still holds - even if the other account will trigger it, the trigger will be run against creator's account, which includes quotas and everything user-related, so be extra careful with that.
Any ideas on how to cope with that?
Although [installable] triggers are not shared across users, some resources are shared between them, so you have at least two possible ways to deal with concurrent triggers firing:
Preferred way: LockService. It is specifically designed to avoid race conditions such as your case: just wait (or reschedule if the action might take more than 5 minutes) until the lock can be aqcuired (note that you need to use a script or document lock).
Fallback way: if you intend to let users concurrently interact with the UI, the lock will fail (by design). You had a right idea of using PropertiesService (I assume that this is what you meant by "adding a property"), but beware that reads / writes are subject to quotas.
I recently answered a similar question - it has a working and tested sample of emulating LockService behaviour with PropertiesService (but again, be careful with quotas).
Notes
You have to be aware of the problem with time-based triggers and account for that (also the referred answer contians a simpler to implement solution for checking the timestamp).
In the documentation for developers (here) written about this error
Service using too much computer time for one day
that
This indicates that the script exceeded the total allowable execution time for one day. It most commonly occurs for scripts that run on a trigger, which have a lower daily limit than scripts executed manually.
And there is a daily limit for triggers total runtime (1hr for basic account).
But where can I get the limitation for manual executed script?
I'm pretty sure the limit is the same regardless of whether the script is manually executed or executed from a trigger; the limit is simply applied to your account, however you run the script.
As mentioned here: "Daily quotas are refreshed at the end of a 24-hour window; the exact time of this refresh, however, varies between users." I don't think Google publicly announces exactly what you get.
I run script on the sidebar that request calculation on the server side which stops after 300 seconds and the sidebar starts it again. This engine worked more than 12 hours without any fails. So, I suppose that there is no any limit for 'manually' executed scripts.
I'm not sure if I have encountered with a bug or is just me not fully understanding how triggers work, probably the second one :D.
I have a google apps script that collects information through a series of requests to our CRM's API and puts it on a Google Docs Spreadsheet, the script takes a while to run, however, it runs successfully (I run it manually every morning, takes about 3 minutes). But when I try to automate that task, I get an error message saying my script exceeded its execution time limit.
I tried setting up my triggers manually, creating them within the script, breaking down my script reducing the number of API requests (taking less then a minute to execute) and execute it several times so I get the whole information. Every single time I get the same error message.
Am I missing something here? Any help will be much appreciated!
Edit: The mail I received has this error message: "Exceeded maximum execution time".
Edit2: I was able to fix this (party). I came across the answer for this question Exceeded maximum execution time in Google Apps Script and worked like a charm.
There are some limits to Google Apps Scripts that you must keep in mind: see Quotas.
It is possible that you are exceeding your total runtime for Triggers. There is also the shorter individual execution time limits on Triggers. I think I remember reading somewhere that Triggers have a lower total execution time limit per execution, but I can't remember where I saw it, and I can't find a source, even in Google's documentation.
This SO question is also similar...
I wouldn't say it's a bug, however as my own painfully experience, I can tell it's not the same running the script manually than through a time-driven trigger.
Sometimes it's just slower, but others it fails for timeout.
I don't know your specifics (there is no code available), but I would recommend you to use a retry exponential backoff function wrapping any critical call and also check for a timeout warning (let's say at 5 minutes) within the main loop.
So I got the error message in the subject on the 1 (one) script I had running yesterday and I am assuming I will get a similar message today.
I have improved the script (which has a trigger to run once per minute) so it functions more along the lines of how it is supposed to however the error message got me thinking as to what sort of functions or bits of programs might be asking for more service time than others.
For example, I have had to use multiple sleep calls in my google apps script to allow the data import to run and again for the worksheet changes/copy paste calls to process. Are all those sleep calls counting against me in terms of service time used?
I would ask on the community's behalf that this be left as an open ended question not specific to the sleep function. What sorts of parts of a script are demanding service time and which are not (if any).
Every call to a service (Spreadsheet, Calendar or whatever) takes more time than regular JavaScript operations.
For example, if you have to modify 10 cells in a Spreadsheet,
calling range.setValue() 10 times takes far more time than having all the data in an array and then updating the spreadsheet in one go using range.setValues().
If you can paste pieces of your code, the community will be able to offer more advice on how to improve your script.
The limit is on CPU time used in time based triggers, and I believe those sleep calls are counted against your limit. I'd encourage you to find ways to avoid the sleep calls, or schedule your script to run less often.