I have a spreadsheet with a matrix set up to count how many times a student has had a lesson with a particular tutor.
The matrix works fine with this formula:
=ARRAYFORMULA(SUM(IF(TERM4!$B$6:$B$2398=B$1,IF(TERM4!$C$6:$C$2398=$A2,1,IF(TERM4!$D$6:$D$2398=$A2,1,FALSE()))),FALSE()))
however due to the number of students/tutors the matrix is 7000 cells, slowing the working sheet down considerably.
Is there a better way to do this. Can I run a google app script to count the matrix on a trigger (eg. once a week) to count the matrix, so the formulas are not slowing the sheet down.
I would also like the formula to return a blank rather than a 0 if the result is FALSE.
Thanks for your help!
Yes its possible to do it with gas. The only part that gets a little complex is that if your script takes over 5min it wont process all the rows. To avoid that, process it by chunks (say 100 at a time) and use scriptProperties to remember which spreadsheet and row you last processed. Each trigger will process as much as it can until all spreadsheets and rows are processed.
Related
I am running a setValues() function for a sheet with massive data count (20,000 * 50) so I finished the data in batch then used the setValues() function to update the sheet at once but the function is updating data in a manner that looks like (x * 50) sequentially, so I wanted to know the limitation of the maximum cells updated at once when using this function.
It seems that when the values are put to the sheet using setValues, at first, the cells are kept and the values are put to the cells. When I had tested the maximum number of cells for putting the values using setValues before, 2,625,000 cells could be put. But, when I tested over 2,635,000 cells, an error of Exceeded maximum execution time occurred. So at that time, I had resulted that the maximum cells were between 2,625,000 and 2,635,000. By the way, when 4,000,000 cells are tested, it was found that the 4,000,000 cells are kept in the destination sheet while the values couldn't be put. This is the result in my environment.
So, in the case of 20,000 * 50, I thought that 1,000,000 cells can be put with setValues. But I'm worried that when such data is put with setValues, the process cost was high. For example, after the script is finished and I checked the sheet, I was required to wait to completely put the values. I thought that this might be the reason for your issue.
In this case, when Sheets API instead of Spreadsheet service (SpreadsheetApp) is used for putting the values to the sheet, such an issue could be removed. Ref
From the above situation, in your situation, as a workaround, how about putting the values using Sheets API instead of setValues of Spreadsheet service?
Reference:
Benchmark: Reading and Writing Spreadsheet using Google Apps Script
There is no direct maximum for setValues(), but you might want to consider the following three points:
There is a (very high) maximum size of an array in javascript.
There is a maximum amount of columns, cells and new rows in a Google spreadsheet.
There is a maximum script runtime and your function might time out before setValues() finished executing.
Of those 3 limits the amount of values you can set with one setValues() call is most likely to be limited by the maximum script execution time.
tl;dr: My custom-formula cells all go Loading... and they start filling in results, but they get stuck. A reload immediately provides the completed results. I don't want to reload every time I change a value.
In my sheet almost all cells are interconnected, so changing one cell triggers a recalc. Each row is a year, each column is an identical formula across all rows. Each cell can refer to earlier columns in its row or anything on the prior row, plus a few absolute locations containing the "inputs". Changing an input triggers a recalc of everything.
The column with my custom function will go Loading..., then one cell at a time it returns the value, almost one second per cell. OK, slow, but fine. But often it just stops completely partway down. Sometimes it starts up again, but often it never does.
But the cells WERE all recalculated. My custom function was called and returned values promptly, execution time of 0.125 secs or less usually. If I reload in the browser, I immediately get the fully recalculated sheet. It looks like some link is being severed between Sheets in my browser and Google's servers, so I stop seeing updates.
This is the first time I've ever used Apps Script -- or JavaScript for that matter -- but I have been programming in other ways for decades. If it matters, the custom function is purely mathematical, calling no services except Math, not even Spreadsheet, getting everything it needs in its arguments, using a few functions in the same file. It's a detailed taxes calculator.
I'm using a recent Chromebook.
In order to avoid to have to reload your spreadsheet it's very likely that you will have to follow the guidelines for custom function optimization:
Summary: Instead of using one formula to calculate a single value use your formula to calculate multiple values and return them as an array.
The above will reduce the number of formulas and will improve your spreadsheet performance, but bear in mind that custom functions have a maximum execution time of 30 secs.
Also reducing the size of the data ranges in your spreadsheets and the number of blank rows and columns at the bottom and right of your data ranges will help in improve the performance of your spreadheet.
Related
How to use a custom function with an ArrayFormula
Room For Optimization?
I remember reading that Total Triggers Runtime is 1 hour for Consumer type users. I don't think I really understand what that means.
Let's say I programmatically create a trigger to run every 10 minutes, like so ...
ScriptApp.newTrigger("myFunction")
.timeBased()
.everyMinutes(10)
.create();
... and let it run around the clock.
myFunction does something very not time consuming like appending a couple rows to a spreadsheet.
My question is, when am I going to hit the said '1 hour' limit?
As you can see under Apps Script quota:
Those times refer to the total execution time of the function(s) that is / are being run on trigger.
When you go on https://script.google.com/u/0/home/executions, you can see all your executions, you can also see the executions for a particular trigger.
It should look like this:
So, if you sum the durations of all the executions of all the functions of type Trigger within the last 24 hours - it cannot exceed the Triggers total runtime.
I understand that it writes to the sheet when the log reaches 50,000 lines. But my script doesn't generate 50,000 lines of log, and I need to see the log once the script's finished. Is there a way to flush the log to the sheet once it finishes. Or reduce the threshold below 50k?
Resolved, and don't I feel stupid. I didn't notice Betterlog created another sheet/tab named Log in the sheet, and it's logging to that.
You can use Logger.getLog() to get the entire text at once.
But you will have to split it in smaller parts, because each cell of the sheet has a limit of 50k characters.
https://developers.google.com/apps-script/reference/base/logger#getLog%28%29
I have a script which adds a record to a ScriptDB datastore, then immediately queries all records and displays a count. 90% of the time, the displayed count is incremented as one would expect. However 10% of the time, the count is NOT incremented. Then on the next invocation the count is +2.
ScriptDb is still in 'experimental' mode, I guess you should raise an issue on the issue tracker if you have experienced this repeatedly.