I am calling a function on the server that adds a few hundred objects to the ScriptDB database from the client using google.script.run. However, I have found that the server function is called more than once so the database ends up with duplicates of these objects.
function serverFunction(bigarray) {
// This function is called multiple times
db.saveBatch(bigarray);
}
Yet I can verify that the code on the client that calls serverFunction is only run once.
function clientFunction() {
alert("This function is only called once.");
google.script.run.serverFunction(bigarray);
}
Could my server code be timing out and getting run again automatically by GAS?
If so, how long is the time out and is this functionality documented anywhere?
Is there any way I can avoid this?
Its currently 30 seconds. This is a known issue and will be fixed fairly soon. (Its not a regression per se since its been like this since day 1 but I need to fix it to match the scripts own five minute timeout).
Related
I have thousands of log files in a cloud storage bucket that I need to process and aggregate using an HTTP triggered cloud function and am looking for an approach to compute the task in the fastest possible way using parallelization.
At the moment, I have two cloud functions (nodejs 8):
The "main" function which a user is calling directly passing a list of log files that need to be processed; the function calls the "child" function for each provided log file that I also trigger with an HTTP request run parallel using async.each. The "child" function processes a single log file and returns the data to the "main" function which aggregates the results and, once all files are processed, sends the results back to the user.
If I call a child function directly, it takes about 1 second to complete a single file. I'd hope that if I call the main function to process 100 files in parallel the time will still be more or less 1 second. The first file in a batch is indeed returned after 1 second, but the time increases with every single file and the 100th file is returned after 7 seconds.
The most likely culprit is the fact that I'm running the child function using an HTTP request, but I haven't found a way to call them "internally". Is there another approach specific to Google Cloud Functions or maybe I can somehow optimise the parallelisation of HTTP requests?
The easiest approach is to simply share the code that does whatever the child function does, and invoke it directly from the main function. For some cases, it's simply easier and costs less due to fewer function invocations.
See also: Calling a Cloud Function from another Cloud Function
In Google apps script when using a client sided .HTML file you can call a server sided script using google.script.run.(Function name).
You can see the related documentation here: https://developers.google.com/apps-script/guides/html/reference/run
Now this script has been working with no problems over the first 6 months of its lifetime or so. I have not touched the program and I have not been notified or have located any newly deprecated code.
Over the course of the last couple months however, my users have been reporting that when they finish interacting with the HTML document, nothing happens when they close it and they have to repeat the entire process 3 or sometimes even 4 times before they will get it to go through.,
This means that when the user closes the client sided HTML window, the server sided function should be called to handle the remaining tasks but in some cases is not. This issue is completely random, and does not seem to be caused by anything specific.
I have taken some steps myself to attempt to solve the issue. I have wrapped the entirety of the code in try catch blocks, including the .HTML and .GS files. This means that if literally ANYTHING goes wrong in ANY script, I will be notified of it immediately. However, despite this being the case I am yet to receive any emails of it failing even though I watch it fail with my own eyes. I have added log commands before and after this function to see if it stops working all together or continues. In every case regardless of whether the function call is successful or not the log commands go through.
To me this can only mean that for some reason the function google.script.run is not working properly, and is failing to run the associated function, but is not returning an error message or stopping the script.
I am at an absolute loss since I have no error message, no reproducible steps, and no history of this being a problem before while suddenly starting to get worse and worse over time. I have checked Google's issue tracker to no results. If anyone else is using this function and is having problems I would love you to share your experiences here. If you have a solution please let me know as soon as possible. If I can't fix this issue I am going to have to use a new platform entirely.
Edit 10/2:
After looking further into this issue I have discovered a list of all executions on this project. I can see what functions were executed, when, and how long they took to execute. I can see that when the function that opens the HTML service is ran, the next function that should run does not always appear in the list. And when it doesn't, I can see that the user repeated their steps until it did run. This supports my theory that the function just isn't running when it should be after being called my script.run
Tl;dr: The affected computers are running so slowly that google.script.host.close would run before google.script.run.functionName() is able to be called and the information passed from the client to server, causing the function to never run but also not return an error. Adding Utilities.sleep(1000) fixes the issue.
I'm answering here in the situation that someone stumbles upon this thread in the future because they're having similar problems.
I was able to fix the issue by adding two lines of code between
google.script.run and google.script.host.close.
I added Google's Utilities.sleep(1000) to force the computer to wait one second between executing the function and closing the HTML window. I also added an HTML alert that shows that the function was called and didn't suffer from a runtime error.
I don't know exactly why this seems to have fixed the issue but I have a theory.
I have about 20 computers this spreadsheet runs on. Only about 6 of them were having the issue, and this wasn't brought to my attention until recently. As it turns out the 6 computers that were having the issue were the slowest computers of the bunch.
My theory is that the computers were so slow, and the internet bandwidth was fluctuating so much that the computer simply didn't have time to call google.script.run and pass off the information from the client sided HTML window that it simply got closed and cut off when google.script.host.close was run. This means that the function will not exist in the execution transcripts or history, nor will there be any sort of runtime error. All of those things were true in my situation. This also explains why I never had the issue on any of my own equipment in a testing environment since it didn't suffer from any slowdowns the other computers were having.
By adding both Utilities.sleep(1000) and the UI alert this forces the javascript to not continue to google.script.host.close until the user interacts with the UI alert (Which is just a confirmation window with an OK button) and afterwards waits a full second. This sacrifices a tiny bit of user friendly-ness for a more functional script. Since I have implemented this "fix" none of my users are reporting any issues and all of my execution history looks just fine.
Hopefully this helps any future passerbys.
In the comments you posted this function snippet:
Here is a basic copy of the script that utilizes google.script.run:
function onFailure(error) {
MailApp.sendEmail("sparkycbass#gmail.com", "Order book eror", "ERROR: " + error.message);
google.script.host.close();
}
function handleFormSubmit(formObject) {
google.script.run.withFailureHandler(onFailure).processForm(formObject)
google.script.host.close();
}
The problem here is that google.script.run is asynchronous - the call to your server-side function processForm is not guaranteed to be even initiated before the call to google.script.host.close() is made:
Client-side calls to server-side functions are asynchronous: after the browser requests that the server run the function doSomething(), the browser continues immediately to the next line of code without waiting for a response. This means that server function calls may not execute in the order you expect. If you make two function calls at the same time, there is no way to know which function will run first; the result may differ each time you load the page. In this situation, success handlers and failure handlers help control the flow of your code.
A proper pattern is to only call "destructive" commands - such as closing the host and therefore unloading all the relevant Apps Script instances - after the server has indicated the async operation completed. This is within the success handler of the google.script.run call:
.html
function onFailure(error) { // server function threw an unhandled exception
google.script.run.sendMeAnEmail("Order book error", "ERROR: " + error.message);
console.log(error);
document.getElementById("some element id").textContent = "There was an error processing that form. Perhaps try again?"
}
function onSuccess(serverFunctionOutput, userObj) {
// do stuff with `serverFunctionOutput` and `userObj`
// ...
google.script.host.close();
}
function handleFormSubmit(formObject) {
google.script.run
.withFailureHandler(onFailure)
.withSuccessHandler(onSuccess)
.processForm(formObject);
}
.gs
function processForm(formData) {
console.log({message: "Processing form data", input: formData});
// ...
}
function sendMeAnEmail(subject, message) {
console.log({message: "There was a boo-boo", email: {message: message, subject: subject}});
MailApp.sendEmail("some email", subject, message);
}
I deploy a google cloud function with lazy loading that loads data from google datastore. The last update time of my function is 7/25/18, 11:35 PM. It works well last week.
Normally, if the function is called less than about 30 minutes since last called. The function does not need to load data loaded from google datastore again. But I found that the lazy loading is not working since yesterday. Even the time between two function is less than 1 minute.
Does anyone meet the same problem? Thanks!
The Cloud Functions can fail due to several reasons such as uncaught exception and internal process crashes, therefore, it is required to check the logs files / HTTP responses error messages to verify the issue root cause and determine if the function is being restarted and generating Function execution timeouts that could explain why your function is not working.
I suggest you take a look on the Reporting Errors documentation that explains the process required to return a function error in order to validate the exact error message thrown by the service and return the error at the recommended way. Keep in mind that when the errors are returned correctly, then the function instance that returned the error is labelled as behaving normally, avoiding cold starts that leads higher latency issues, and making the function available to serve future requests if need be.
I had a problem with a script I wrote, the solution to the issue was lock service to avoid collisions with form submits. As I had no idea this would be an issue, I've had to go back and revisit old scripts.
I have script that has a few different functions, and it passes data from one function to another. Eventually it writes data to sheet, creates a PDF, can email it and stores the PDF to a folder in google drive.
Here's a brief example of what I mean
function firstFunction() {
//Do stuff, return something
return something;
secondFunction(something);
}
function secondFunction(something) {
// Do stuff, return test
thirdFunction(test);
}
function thirdFunction(test) {
// Do stuff, return that
return that;
fourthFunction(that);
}
function fourthFunction(that){
// finish doing stuff. Write data
}
I also have a separate script, that would invoke the first and iterate through a list of data, to bulk produce PDFs.
I'm worried that if 2 people invoke the script at the same time, I'll have issues again.
Given the example script I've given, Do I have to use LockService on each function? Or can I declare the lock in the first function and release it in the last.
I'm also curious on how it would sit with the 2nd script that invokes the first several times. Would adding the lock service in this one be sufficient, or would I also have to add it to the second too?
Thanks in advance.
EDIT BELOW
I just remembered I posted the real code on Code Review for advice, and boy did I get some!!
Code Review Post
I should think that you don't need Lock Service in this case at all.
In the Lock Service documentation it states:
[Lock Service] Prevents concurrent access to sections of code. This can be useful when you have multiple users or processes modifying a shared resource and want to prevent collisions. (documentation: https://developers.google.com/apps-script/reference/lock/lock-service#getScriptLock%28%29)
or [Class Lock] is particularly useful for callbacks and triggers, where a user action may cause changes to a shared resource and you want to ensure that aren't collisions. (documentation: https://developers.google.com/apps-script/reference/lock/lock#tryLock%28Integer%29)
Now, having read the script code that you link to in your edit, I saw no shared resources that the script is writing to. So I conclude no lock is required. (EDIT: On second reading, I see that the script is writing to a sheet once, the shared resource. So your lock can go within that function only.)
I will cross post this point to Google Apps Script Plus community https://plus.google.com/communities/102471985047225101769 since there are experts there who can confirm.
I want to use Google Apps Script to make custom functions for a spreadsheet. I've made an extremely simple function:
function foo(){
return "bar";
};
The problem is that I need this function in a couple hundred cells. When I paste the function =foo() into all of these cells, the function works in a few of the cells, but in most I get this error: "Service invoked too many times: spreadsheet. Try Utilities.sleep(1000) between calls."
[Screenshot here]
I guess I don't understand why this function, simple as it is, is considered an invocation of the Spreadsheet Services. I'm not even requesting any data (except for the function itself). Is that the problem? And if so, is there a workaround? Custom functions could make Google Spreadsheets infinitely more powerful, but this problem hamstrings the possibility of using a custom function in multiple cells. Suggestions?
(P.S. -- Using the Utilities.sleep() function as suggested by the error message doesn't help at all when all of the cells call their functions simultaneously; it only slows the rate at which individual cells repeatedly call the function.)
According to the Optimization section on the Apps Script Function Guide:
Each time a custom function is used in a spreadsheet, Google Sheets makes a separate call to the Apps Script server. If your spreadsheet contains dozens (or hundreds, or thousands!) of custom function calls, this process can be quite slow.
Consequently, if you plan to use a custom function multiple times on a large range of data, consider modifying the function so that it accepts a range as input in the form of a two-dimensional array, then returns a two-dimensional array that can overflow into the appropriate cells.
To do this, pass in an input that represents the size of the array you'd like to return. When you start executing your function check if the input parameter is an array with input.map. If it is, you can call the the function on each item and return that entire collection.
So in your case like this:
function foo(){
return "bar";
};
You can update the function like this:
function foo(input){
if (input.map) { // Test whether input is an array.
return input.map(foo); // Recurse over array if so.
} else {
// do actual function work here
return "bar";
}
};
And then call it like this:
By calling the function in the spreadsheet, you are invoking the Spreadsheet service by asking it to go round-trip to the server to run the results of your function. As a result, you have make a couple hundred requests in a very short period of time.
One work around might be to add your function a few cells at a time. Of course, when you subsequently open the sheet again, you will probably run into the same problem.
Depending on what your function is trying to accomplish, it might be worth using the built in spreadsheet functions. There is a lot of power there. Writing a function that acts on a range of values instead of a single cell might be another, better, option. It could be triggered through a custom menu item, or by using the script manager.
Keep in mind, batch actions are your best friend when it comes to working with spreadsheets.