Google Spreadsheet stopped storing data before 400000 cells are consumed - google-apps-script

I have a spreadsheet which collects users' feedback data. According to Google Documentation, size limit for a Google Spreadsheet is 400000 cells. I have written a GAS code which checks the total number of consumed cells in the spreadsheet. In case consumed cells' count is alarming, GAS creates a duplicate copy and clears data from the current spreadsheet.
Now, the spreadsheet suddenly stopped collecting data submissions since Jan 28, 2014. I checked the total cells consumed and found that it still needed approx 2500 cells to be 400000.
I looked through the Google documentation again and found:
"Spreadsheets also have overall storage limits. Some spreadsheets may reach these before hitting the 400,000 cell limit, particularly when individual cells have large amounts of text. In such cases, the spreadsheet will go into read-only mode to prevent data loss."
I tried looking for any method in Google Apps Script or Google Script where I can check a spreadsheet's data size with respect to its storage limit. I didn't find any such reference.
Can you please help on this? Is there any method in Google Apps Script to find out:
1. what is current data size of a Google spreadsheet?
2. what is the storage limit for a Google spreadsheet?
Thanks in advance.

No its not possible by apis.
Its not just byte size. Other things like total number of formulas also affect it. Converting to xls will only be an aproximation.

My test is to download the spreadsheet as an Excel file. I have one that's close to 5.5MB. Anything higher than that usually stops working. I haven't tried to automate that check, but perhaps it's possible to:
Use DriveApp to get the file as xlsx
Measure file's Blob's size

Related

Is maximum number of Apps Script projects created per day applicable to container-bounded projects?

I am thinking of building my own application using AppScript and Spreadsheet.
I wonder if the maximum limit of AppScripts per day(mentioned in the link below) is applied to container-bounded projects or not.
https://developers.google.com/apps-script/guides/services/quotas
I'd like to run a script which copies a spreadsheet with a container-bounded project everytime it is runned.
My script is going to be runned more than 100 times per day, so, in other words, it creates more than 100 spreadsheets and the corresponding container-bounded projects per day. This number theoretically exceeds the limit that the official reference says.
However, when I tried copying a spreadsheet manually more than 50 times, I found no error message or assertion. Also, the copied scripts are all working in each spreadsheet.
I still wonder if my use of AppScript is safe or not.
I would appreciate it if you kindly give me advice.

How to solve the limitation of IMPORTRANGE formula in google sheet?

I have 50 google sheet files for 50 students. They need to key in their answers in their own google sheet file when they are asked to do so. I have a main google sheet to consolidate their data by using IMPORTRANGE formula. This is my formula:
=QUERY({IMPORTRANGE(...);IMPORTRANGE(...);IMPORTRANGE(...);...},"Select * where Col1 is not null")
I will have 50 IMPORTRANGE in the formula. So as expected, the main google sheet is very lag when the 50 students start to key in their answers at the same time. Sometimes, the formula will show #Value when all the students started to answer the questions at their own google sheet file. I need to keep refreshing the main google sheet so that the data will come out, but it will disappear again in a short while then I need to refresh it again (although it will settle down once most of the students finished answering the questions).
I know that using IMPORTRANGE is really not an efficient way to consolidate their answers in main google sheet file but I don't have other better way.
I tried to write a script so that they can send in their data by clicking the button assigned with the script. However, all the students need to go through the authorization process when they run the script for the first time. They don't know how to proceed when they saw the authorization process (not very good in using computer).
May I know is there any ways or tricks that I can use to solve the IMPORTRANGE issue? Or there are some way to write the script where we are not required to go through the authorization process when we run the script for the first time?
Hope to get some advice and help on this as I couldn't find a better way from Google already. Any help will be greatly appreciated!
If I understood correctly what you are looking is that your spreadsheet show in real-time the data being entered simultaneously on 50 different spreadsheets. I'm afraid that Google Sheets is not the right tool for what you are trying to do the way that you are trying to do it. Basically you have two options : change it or use a different tool.
It's not a good idea to have and array of multiple IMPORTRANGE functions that are being edited simultaneously because while the official docs says that IMPORTRANGE functions are updated every 30 minutes when the source and the spreadsheet having formula are opened at the same time the import is done practically immediately and could happen multiple times during the recalculation making causing it to start over an over again.
Replacing the above array by script might help only if you are open to not have the destination spreadsheet updated on real time as scripts are slow.
Replacing the above array by a program that uses the Google Sheets API also might help only if you are open to not have the destination spreadsheet updated on real time as the spreadsheet refresh.
Regarding running a script without requiring authorization that is only possible when using simple triggers and / or removing all the scopes that require authorization to run. Please bear in mind that you might create installable triggers to run other using the authorization of the user who creates them.
Related
Combining multiple spreadsheets in one using IMPORTRANGE
Why do two users sometimes see different values from importrange?
Multiple IMPORTRANGE
Using that many IMPORTRANGE formulas is definitely a bad idea. What I'd suggest you to do:
keep a list of all your student spreadsheet in your main document
write a script that will browse through all of the spreadsheets from that list and copy/paste values into your main document
create a time based trigger that will run the script every X minutes (or hours), depending on how accurate you want the results to be
This is a simple solution, but efficient. Depending on the amount of data and number of students/spreadsheets you may consider other solutions (like writing a cloud function that will do the same as the script) but I think this will work for your use case

Google Sheets script general mechanics

I have put together a rather large script file that does a lot of math. It seems to me that if I place a script variable value on the spreadsheet (setValues()), the whole spreadsheet recalcs (i.e.; it takes awhile to refresh). Is that true?
What if I want to format a cell from script (e.g.; change a number cell to a percentage cell with 3 places)[FormattedSS.getRange(rangecoordinates).setNumberFormat("#.###%");]? When I set the format, does the whole spreadsheet recalculate?
I am new to Google Sheets. In Excel, I could set calculation off. Google sheets does not seem to have that option.
Google sheets has no manual calculation option. You would need to create a formula to check whether to run the rest of the formula. If the output is a single cell you can easily use a circular reference to retain the value.
When it comes to Google App Scripts every time you execute a script there will be an annoying delay, that is just how it works, the App Script should be avoided at all cost in a normal spreadsheet except for when actually necessary.
Google App Script Server -> Google Sheet (delay to send information back and forth)
Google Sheet always recalculates after any change

Importrange function - what is the latest maximum data size that can be used? Changed late March

Google sheets changed the size of data that can be imported using the 'importrange' function in late March. We want to write a formula that can search through all our sheets and identify any that will be affected by Google's latest change.
We know there has been a change, as Importranges we had previously setup, are now returning "... to large" type error.
It appears to a limit of data size, not cells - as we experienced the issue on importrange of only 2 columns, but one contained large amount of HTML code in each cell.
Does anyone know what the data size limit is?
If not, then does anyone have an idea for best way to write a script that can find the limit?
Or next step will be to write a script that can search through our 'network analysis' sheet (sheet/tool that shows all Google sheets that are connected by importrange), and identify those with the importrange issue.
FYI - Google appears to have changed quite a bit relating to Google sheets and apps script.
a) We lost the ability to save changes in a sheet (making it basically unusable), which we think is related to having >50 importranges referencing that sheet (google support advised to avoid >50 importranges going to one sheet).
b) Other change we noticed is that google apps script can definitely now run for longer than the previous limit of 5min. We previously were seeing some scripts, sometimes running >5min, all the way up to a max of 30min. But now we see it consistently on some scripts. We built a tool, that can automatically get a script to 'run again' if it did not complete last time. We have had to tweak that 'run again script', to ensure it does not try to run the script again within 30min of the first run, to ensure that first script is definitely still not running.

Delay in changing Google Spreadsheet content via Google apps Script?

I am totally new to Google-apps-script and this may be very poor question
I am making a basic setup using google forms and google apps script
From the responses of form I change my content in Google Spreadsheet accordingly
For example my query from form needed 10000 records to be selected and produced in whole another spreadsheet
I just wanted to know that is there some kind of delay introduced when I set and get values of any cell of spreadsheet on such a large scale? If so on what it depends and how as a programmer can I remove or optimize them?
Thanks is advance!
The Best Practices article by Google is the primary reference for this. The most important advice is to minimize the number of calls to spreadsheet methods by batching operations. So, select a range that contains all the records you need, get them all at once with getValues, process without further interaction with the spreadsheet, and output the result using setValues.
If you follow this advice, 10000 records is still a reasonable amount of data to process by a script.