Importing JSON Data into Google Sheets - json

I am trying to import mining data from Slush Pool via their API. I am using a Google Sheets script called ImportJSON which can be found here. This code does not include a function for API keys or access tokens so I added a wrapper based off information from this post. This wrapper was added at line 255 of the original ImportJSON file.
/**
*
* Wrapper
*
* #param {url} the URL to a http basic auth protected JSON feed
* #param {api_key} the api_key for authentication
* #param {query} always = ""
* #param {parseOptions} a comma-separated list of options that may alter processing of the data (optional)
*/
function ImportJSON_words(url, api_key, query, parseOptions) {
var header = {
headers: {
"X-SlushPool-Auth-Token": api_key
}
}
return ImportJSONAdvanced(url, header, query, parseOptions, includeXPath_, defaultTransform_)
}
The line "X-SlushPool-Auth-Token": api_key was included to satisfy Slush Pool's API authentication. They specifically say
An access profile token has to be included in the HTTP header field
named SlushPool-Auth-Token or X-SlushPool-Auth-Token to authenticate
your requests.
They also give this Python example using cURL:
curl https://slushpool.com/stats/json/btc/ -H "SlushPool-Auth-Token: <your access token>"
To import the JSON file I type the following formula in a Google Sheets cell.
=ImportJSON_words("https://slushpool.com/stats/json/btc/","xxxxxxxxxx","","noTruncate")
The URL is given by Slush Pool, the x's represent my API key, the query field is empty and "noTruncate" is one of the parse options given in the ImportJSON documentation which prevents the data from being shortened.
I am getting a #ERROR, namely a formula parse error and I am not seeing any imported data. I am not sure where I am going wrong with this API access token. Any advice would be appreciated.

Formula parse error usually occurs because there is a Google Sheets formula syntax error. This is unrelated to scripts for custom functions.
Double check that the spreadsheet region is correct and the decimal separator for such region. If the decimal separator is a . your formulas should have a , as function parameter separator otherwise they should use a ;
Check that your formula doesn't include:
curly quotes instead of strait quotes
any hidden character that might be inadvertently added when doing copy paste from a Wordpress / CMS generated web page.
a parenthesis like character instead of parenthesis (same reason as the above point)
Check that each parameter of your formula is properly set. One way to do that is to make a formula for each parameter i.e.
="https://slushpool.com/stats/json/btc/"
="xxxxxxxxxx"
etc.
Try using Google Sheets in incognito mode with all the extensions disabled.
While formula parse errors are unrelated to scripts, if all the previous have not worked...
While there are very few restrictions for custom functions names, try changing the function name (remove the underscore, be sure to not use a reserved name i.e. simple triggers function names, JavaScript reserved words, Google Sheets functions names)
It might be a good idea to start from scratch but instead of modifying the original IMPORTJSON .gs files, add a new .gs file and add to it your wrapper. This because sometimes one spreadsheet does "strange" things but others not. Also include #customfunction in the JSDoc comment of your custom function.

In order to create a custom function in the Google Sheets script you must follow a specific syntax. If the syntax is not followed when you try to use that function in the sheet Google's autocomplete function will not show it. The above wrapper had to be modified to include the return and customfunction parameters. Once these parameters were added the function was identified by Google in the sheet and the rest of the script ran correctly.
/**
*
* Wrapper
*
* #param {url} the URL to a http basic auth protected JSON feed
* #param {api_key} the api_key for authentication
* #param {query} always = ""
* #param {parseOptions} a comma-separated list of options that may alter processing of the data (optional)
*
* #return a two-dimensional array containing the data, with the first row containing headers
* #customfunction
**/
function ImportJSON_words(url, api_key, query, parseOptions) {
var header = {
headers: {
"X-SlushPool-Auth-Token": api_key
}
}
return ImportJSONAdvanced(url, header, query, parseOptions, includeXPath_, defaultTransform_)
}

Related

Automatically triggering a custom function after another custom function completes [duplicate]

I've written a custom Google Apps Script that will receive an id and fetch information from a web service (a price).
I use this script in a spreadsheet, and it works just fine. My problem is that these prices change, and my spreadsheet doesn't get updated.
How can I force it to re-run the script and update the cells (without manually going over each cell)?
Ok, it seems like my problem was that google behaves in a weird way - it doesn't re-run the script as long as the script parameters are similar, it uses cached results from the previous runs. Hence it doesn't re-connect to the API and doesn't re-fetch the price, it simply returns the previous script result that was cached.
See more info here(Add a star to these issues, if you're affected):
https://issuetracker.google.com/issues/36753882
https://issuetracker.google.com/issues/36763858
and Henrique G. Abreu's answer
My solution was to add another parameter to my script, which I don't even use. Now, when you call the function with a parameter that is different than previous calls, it will have to rerun the script because the result for these parameters will not be in the cache.
So whenever I call the function, for the extra parameter I pass "$A$1".
I also created a menu item called refresh, and when I run it, it puts the current date and time in A1, hence all the calls to the script with $A$1 as second parameter will have to recalculate. Here's some code from my script:
function onOpen() {
var sheet = SpreadsheetApp.getActiveSpreadsheet();
var entries = [{
name : "Refresh",
functionName : "refreshLastUpdate"
}];
sheet.addMenu("Refresh", entries);
};
function refreshLastUpdate() {
SpreadsheetApp.getActiveSpreadsheet().getRange('A1').setValue(new Date().toTimeString());
}
function getPrice(itemId, datetime) {
var headers =
{
"method" : "get",
"contentType" : "application/json",
headers : {'Cache-Control' : 'max-age=0'}
};
var jsonResponse = UrlFetchApp.fetch("http://someURL?item_id=" + itemId, headers);
var jsonObj = eval( '(' + jsonResponse + ')' );
return jsonObj.Price;
SpreadsheetApp.flush();
}
And when I want to put the price of item with ID 5 in a cell, I use the following formula:
=getPrice(5, $A$1)
When I want to refresh the prices, I simply click the "Refresh" -> "Refresh" menu item.
Remember that you need to reload the spreadsheet after you change the onOpen() script.
You're missing the fastidious caching bug feature. It works this way:
Google considers that all your custom functions depend only on their parameters values directly to return their result (you can optionally depend on other static data).
Given this prerequisite they can evaluate your functions only when a parameter changes. e.g.
Let's suppose we have the text "10" on cell B1, then on some other cell we type =myFunction(B1)
myFunction will be evaluated and its result retrieved. Then if you change cell B1 value to "35", custom will be re-evaluated as expected and the new result retrieved normally.
Now, if you change cell B1 again to the original "10", there's no re-evaluation, the original result is retrieved immediately from cache.
So, when you use the sheet name as a parameter to fetch it dynamically and return the result, you're breaking the caching rule.
Unfortunately, you can't have custom functions without this amazing feature. So you'll have to either change it to receive the values directly, instead of the sheet name, or do not use a custom function. For example, you could have a parameter on your script telling where the summaries should go and have an onEdit update them whenever a total changes.
What I did was similar to tbkn23. This method doesn't require any user action except making a change.
The function I want to re-evaluate has an extra unused parameter, $A$1. So the function call is
=myFunction(firstParam, $A$1)
But in the code the function signature is
function myFunction(firstParam)
Instead of having a Refresh function I've used the onEdit(e) function like this
function onEdit(e)
{
SpreadsheetApp.getActiveSheet().getRange('A1').setValue(Math.random());
}
This function is triggered whenever any cell in the spreadsheet is edited. So now you edit a cell, a random number is placed in A1, this refreshes the parameter list as tbkn23 suggested, causing the custom function to be re-evaluated.
There are settings where you can make NOW() update automatically:
If your custom function is inside a specific column, simply order your spreadsheet by that column.
The ordering action forces a refresh of the data, which invokes your custom function for all rows of that column at once.
Script Logic:
Custom Functions don't update unless it's arguments changes.
Create a onChange trigger to change all arguments of all custom functions in the spreadsheet using TextFinder
The idea to add a extra dummy parameter by #tbkn23 and use of the triggers by #Lexi Brush is implemented here with a random number as argument. This answer mainly differs due to usage of class TextFinder(a relatively new addition to Apps script), which is better because
No extra cell is required.
No menu is needed > No additional clicks needed. If you need a custom refresher, a checkbox is a better implementation
You can also change the formula itself instead of changing the parameters
The change/trigger can be configured to filter out only certain changes. For eg, the following sample script trigger filters out all changes except INSERT_GRID/REMOVE_GRID(Grid=Sheet). This is appropriate for the custom function that provides sheetnames. A edit anywhere isn't going to change the list of sheets/sheetnames, but inserting or removing sheet does.
Sample custom function(to refresh):
/**
* #customfunction
* #OnlyCurrentDoc
* #returns Current list of sheet names
*/
const sheetNames = () =>
SpreadsheetApp.getActive()
.getSheets()
.map((sheet) => sheet.getName());
Refresher function:
/**
* #description Automatically refreshes specified custom functions
* #author TheMaster https://stackoverflow.com/users/8404453
* #version 2.0.0
* #changelog
* Updated to support all custom functions and arguments
* Avoid eternal loops
*/
/**
* #listens to changes in a Google sheet
* #see https://developers.google.com/apps-script/guides/triggers/installable#managing_triggers_manually
*/
function onChange(e) {
/* Name of the custom function that is to be refreshed */
const customfunctionName = 'SHEETNAMES',
regexPattern = `=${customfunctionName}${String.raw`\(([^)]*?)?(?:,\s*?"RANDOM_ID_\d+")?\)`}`,
replacementRegex = `=${customfunctionName}${String.raw`($1,"RANDOM_ID_${
Math.floor(Math.random() * 500) + 1
}")`}`;
/* Avoid eternal loop
* Increase timeout if it still loops
*/
const cache = CacheService.getScriptCache(),
key = 'onChangeLastRun',
timeout = 5 * 1000 /*5s*/,
timediff = new Date() - new Date(JSON.parse(cache.get(key)));
if (timediff <= timeout /*5s*/) return;
cache.put(key, JSON.stringify(new Date()));
/* Following types of change are available:
* EDIT
* INSERT_ROW
* INSERT_COLUMN
* REMOVE_ROW
* REMOVE_COLUMN
* INSERT_GRID
* REMOVE_GRID
* FORMAT
* OTHER - This usually refers to changes made by the script itself or sheets api
*/
if (!/GRID|OTHER/.test(e.changeType)) return; //Listen only to grid/OTHER change
SpreadsheetApp.getActive()
.createTextFinder(regexPattern)
.matchFormulaText(true)
.matchCase(false)
.useRegularExpression(true)
.replaceAllWith(replacementRegex);
}
To Read:
Installable triggers
TextFinder
As noted earlier:
Custom Functions don't update unless it's arguments changes.
The possible solution is to create a checkbox in a single cell and use this cell as an argument for the custom function:
Create a checkbox: select free cell e.g. [A1], go to [Insert] > [Checkbox]
Make this cell an argument: =myFunction(A1)
Click checkbox to refresh the formula
Use a google finance function as a parameter. Like =GOOGLEFINANCE("CURRENCY:CADARS")
Those function force reload every x minutes
Since google app script is an extension of JS, functions should be able to handle more args than defined in function signature or fewer. So if you have some function like
function ADD(a, b) {
return CONSTANTS!$A$1 + a + b
}
then you'd call this func like
=ADD(A1, B1, $A$2)
where $A$2 is some checkbox (insert -> checkbox) that you can click to "refresh" after you needed to change the value from the sheet & cell CONSTANTS$A$1
I use a dummy variable in a function, this variable refers to a cell in the spreadsheet. Then I have a Myfunction() in script that writes a Math.Random number in that cell.
MyFunction is under a trigger service (Edit/Current Project Triggers) and you can choose different event-triggers, for example On-Open or time driven, there you can choose for example a time period, from 1 minute to a month.
Working off of Lexi's script as-is, it didn't seem to work anymore with the current Sheets, but if I add the dummy variable into my function as a parameter (no need to actually use it inside the function), it will indeed force google sheets to refresh the page again.
So, declaration like: function myFunction(firstParam,dummy) and then calling it would be as has been suggested. That worked for me.
Also, if it is a nuisance for a random variable to appear on all of your sheets that you edit, an easy remedy to limit to one sheet is as follows:
function onEdit(e)
{
e.source.getSheetByName('THESHEETNAME').getRange('J1').setValue(Math.random());
}
another solution to the caching problem.
have a dummy variable in your method.
pass
Filter(<the cell or cell range>,1=1)
as the value to that parameter.
e.g.
=getValueScript("B1","B4:Z10", filter(B4:Z10,1=1))
the output of filter is not used. however it indicates to the spreadsheet that this formula is sensitive to B4:Z10 range.
I had a similar issue creating a dashboard for work. Chamil's solution above (namely using Sheet's Filter function passed as the value to a dummy variable in your function) works just fine, despite the more recent comment from Arsen. In my case, I was using a function to monitor a range and could not use the filter on the same range since it created a circular reference. So I just had a cell (in my case E45 in the code below) in which I changed the number anytime I wanted my function to update:
=myFunction("E3:E43","D44",filter(E45,1=1))
As Chamil indicated, the filter is not used in the script:
function myFunction(range, colorRef, dummy) {
variable 'dummy' not used in code here
}
Today I solved this by
adding another parameter to my function:
function MY_FUNC(a, b, additional_param) { /* ... */ }
adding a value to this parameter as well, referencing a cell:
=MY_FUNC("a", "b", A1)
and putting a checkbox in that referenced cell (A1).
Now, it takes only one click (on the checkbox) to force recalculating my function.
What you could do is to set up another cell somewhere in the spreadsheet that will be updated every time a new sheet is added. Make sure it doesn't update for every change but only when you want to do the calculation (in your case when you add a sheet). You then pass the reference to this cell to your custom function. As mentioned the custom function can ignore this parameter.
Given that feature explained by Henrique Abreu, you may try the out-of-box spreadsheet function QUERY , that SQL liked query is what I use often in work
on raw data, and get data as summary to a different tab, result data is updated in real time following change in raw data.
My suggestion is based on the fact that your script has not advanced work such as URL fetch, just data work, as without actual data read, I cannot give a precise solution with QUERY.
Regarding the cache feature mentioned by Henrique Abreu (I don't have enough reputation to comment directly under his answer), I did testing and found that:
looks there is no cache working, testing function's script shown below:
function adder(base) {
Utilities.sleep(5000);
return base + 10;
}
applying that custom function adder() in sheet by calling a cell, and then changed that cell value forth and back, each time I see the loading message and total time more than 5 seconds.
It might be related to the update mentioned in this GAS issue:
This issue has now been fixed. Custom functions in New Sheets are now context aware and do not cache values as aggressively.
the issue mentioned in this topic remains, my testing suggests that, Google sheet recalculate custom function each time ONLY WHEN
value DIRECTLY called by function is changed.
function getCellValue(sheetName,row,col)
{
var ss= SpreadsheetApp.getActiveSpreadsheet();
var sh = ss.getSheetByName(sheetName);
return sh.getRange(row, col).getValue();
}
A change of any value in yellow cells will lead to recalculation of custom function; the real data source value change is ignored by function.
function containing cell's location is changed in sheet. ex. insert/remove a row/column above or left side.
I did not want to have a dummy parameter. YMMV on this.
1 A cell that is a 'List of Items', one is "Refresh"
2 Script with 'onEdit', if the cell is "Refresh":
a)Empty out the document cache
b)Fill doc cache with external data (a table in my case)
c)For all cells with my 'getStockoData(...' custom function
get the formula
set '=0'
set the fromula
d)Set the cell in (1) with a value of "Ready"
This does refresh the bits you want BUT IS NOT FAST.
I followed this video, from 1:44, and this worked for me.
You should use a specific update function, initialize a "current time" variable and pass this permanently updated variable to your custom function. Then go to Triggers and set up an "every-minute" time-driven trigger for the update function (or choose another time interval for updates).
The code:
function update() {
var dt = new Date();
var ts = dt.toLocaleTimeString();
var cellVal = '=CustomFunction("'+ ts + '")';
SpreadsheetApp.getActiveSheet().getRange('A1').setValue(cellVal);
}
Just add GOOGLEFINANCE("eurusd") as an additional argument to your custom function, like:
=myFunction(arg1, arg2, GOOGLEFINANCE("eurusd"))
As #Brionius said put an extra dinamic argument on the function. if you use now() you may have timeout problems make the update a little bit slower...
cell A1 = int(now()*1000)
cell A2 = function(args..., A1)
If you have written a custom function and used it in your spreadsheet as a formula, then each time you open the spreadsheet or any referencing cell is modified, the formula is recalculated.
If you want to just keep staring at the spreadsheet and want its values to change, then consider adding a timed trigger that will update the cells. Read more about triggers here

Autocomplete for custom functions added via a script in Google Sheets

I have added the very basic example script "Double" via the instructions given in the Google Apps Scripts guide. Simple stuff, but the function does not show up in my Google Sheet when I type in a cell "=doub..."
I get no errors in the script editor. I can run the default functions within Google Sheets, yet I can't get any new functions to appear.
The assumption is that you are trying to create a custom function and have it show up as an autocomplete item when you are typing formulas in cells.
Custom functions will appear in this list if their script includes a JsDoc #customfunction tag, as in the DOUBLE() example below.
/**
* Multiplies the input value by 2.
*
* #param {number} input The value to multiply.
* #return The input multiplied by 2.
* #customfunction
*/
function DOUBLE(input) {
return input * 2;
}
usually the commented portion of JavaScript is not too important but in this case it is. The comment must be properly structured to get the function to render as an autocomplete item when typing formulas in cells
https://developers.google.com/apps-script/guides/sheets/functions#autocomplete

Webapp to return specific text responses for a URL parameter

I am sorry cause I do not know any scripting language I am posting my concerns here. I am in need of script having the below functionality. I have created https://sites.google.com/site/iitmamritwater website.
I am in need a web service for a HTTP GET request like : https://sites.google.com/site/iitmamritwater?id=xxxxxxxxxx that would validate id as submitted in query param. Currently I have only 2 valid ID values each of 10 digits length. The web service response should state id validity with its balance - something like below:
If xxxxxxxxxx is one of the two possible correct values then response should be as below:
https://sites.google.com/site/iitmamritwater?id=xxxxxxxxxx
V;10.50;
If xxxxxxxxxy is not one of the two values then response should be as below:
https://sites.google.com/site/iitmamritwater?id=xxxxxxxxxy
N;00.00;
Its a very simple script but cause I do not know scripting I am stuck implementing the concept. I don't know even how to begin with - can some one please post the code for same - it would be highly helpful for me? It will save my project development time period.
See this answer for an example of serving multiple html pages using HtmlService. The basic idea is to write doGet() to accept a query parameter that it will use to select which html page to serve.
In your case, instead of serving HTML pages, you should use the ContentService to output the appropriate response, based on the evaluation of the received parameters.
function doGet(e) {
if (!e.parameter.id) {
// No id provided...
return ContentService.createTextOutput('Error - no id');
}
// else, get response for given id
var response = validateId( e.parameter.id );
// ...and serve it as text output
return ContentService.createTextOutput( response );
}
/**
* Get a response string for the given id
*
* #param {string} id 10 digit id
* #returns {string} Result of validation & lookup
*/
function validateId( id ) {
return "Under construction";
}

Refresh data retrieved by a custom function in Google Sheet

I've written a custom Google Apps Script that will receive an id and fetch information from a web service (a price).
I use this script in a spreadsheet, and it works just fine. My problem is that these prices change, and my spreadsheet doesn't get updated.
How can I force it to re-run the script and update the cells (without manually going over each cell)?
Ok, it seems like my problem was that google behaves in a weird way - it doesn't re-run the script as long as the script parameters are similar, it uses cached results from the previous runs. Hence it doesn't re-connect to the API and doesn't re-fetch the price, it simply returns the previous script result that was cached.
See more info here(Add a star to these issues, if you're affected):
https://issuetracker.google.com/issues/36753882
https://issuetracker.google.com/issues/36763858
and Henrique G. Abreu's answer
My solution was to add another parameter to my script, which I don't even use. Now, when you call the function with a parameter that is different than previous calls, it will have to rerun the script because the result for these parameters will not be in the cache.
So whenever I call the function, for the extra parameter I pass "$A$1".
I also created a menu item called refresh, and when I run it, it puts the current date and time in A1, hence all the calls to the script with $A$1 as second parameter will have to recalculate. Here's some code from my script:
function onOpen() {
var sheet = SpreadsheetApp.getActiveSpreadsheet();
var entries = [{
name : "Refresh",
functionName : "refreshLastUpdate"
}];
sheet.addMenu("Refresh", entries);
};
function refreshLastUpdate() {
SpreadsheetApp.getActiveSpreadsheet().getRange('A1').setValue(new Date().toTimeString());
}
function getPrice(itemId, datetime) {
var headers =
{
"method" : "get",
"contentType" : "application/json",
headers : {'Cache-Control' : 'max-age=0'}
};
var jsonResponse = UrlFetchApp.fetch("http://someURL?item_id=" + itemId, headers);
var jsonObj = eval( '(' + jsonResponse + ')' );
return jsonObj.Price;
SpreadsheetApp.flush();
}
And when I want to put the price of item with ID 5 in a cell, I use the following formula:
=getPrice(5, $A$1)
When I want to refresh the prices, I simply click the "Refresh" -> "Refresh" menu item.
Remember that you need to reload the spreadsheet after you change the onOpen() script.
You're missing the fastidious caching bug feature. It works this way:
Google considers that all your custom functions depend only on their parameters values directly to return their result (you can optionally depend on other static data).
Given this prerequisite they can evaluate your functions only when a parameter changes. e.g.
Let's suppose we have the text "10" on cell B1, then on some other cell we type =myFunction(B1)
myFunction will be evaluated and its result retrieved. Then if you change cell B1 value to "35", custom will be re-evaluated as expected and the new result retrieved normally.
Now, if you change cell B1 again to the original "10", there's no re-evaluation, the original result is retrieved immediately from cache.
So, when you use the sheet name as a parameter to fetch it dynamically and return the result, you're breaking the caching rule.
Unfortunately, you can't have custom functions without this amazing feature. So you'll have to either change it to receive the values directly, instead of the sheet name, or do not use a custom function. For example, you could have a parameter on your script telling where the summaries should go and have an onEdit update them whenever a total changes.
What I did was similar to tbkn23. This method doesn't require any user action except making a change.
The function I want to re-evaluate has an extra unused parameter, $A$1. So the function call is
=myFunction(firstParam, $A$1)
But in the code the function signature is
function myFunction(firstParam)
Instead of having a Refresh function I've used the onEdit(e) function like this
function onEdit(e)
{
SpreadsheetApp.getActiveSheet().getRange('A1').setValue(Math.random());
}
This function is triggered whenever any cell in the spreadsheet is edited. So now you edit a cell, a random number is placed in A1, this refreshes the parameter list as tbkn23 suggested, causing the custom function to be re-evaluated.
There are settings where you can make NOW() update automatically:
If your custom function is inside a specific column, simply order your spreadsheet by that column.
The ordering action forces a refresh of the data, which invokes your custom function for all rows of that column at once.
Script Logic:
Custom Functions don't update unless it's arguments changes.
Create a onChange trigger to change all arguments of all custom functions in the spreadsheet using TextFinder
The idea to add a extra dummy parameter by #tbkn23 and use of the triggers by #Lexi Brush is implemented here with a random number as argument. This answer mainly differs due to usage of class TextFinder(a relatively new addition to Apps script), which is better because
No extra cell is required.
No menu is needed > No additional clicks needed. If you need a custom refresher, a checkbox is a better implementation
You can also change the formula itself instead of changing the parameters
The change/trigger can be configured to filter out only certain changes. For eg, the following sample script trigger filters out all changes except INSERT_GRID/REMOVE_GRID(Grid=Sheet). This is appropriate for the custom function that provides sheetnames. A edit anywhere isn't going to change the list of sheets/sheetnames, but inserting or removing sheet does.
Sample custom function(to refresh):
/**
* #customfunction
* #OnlyCurrentDoc
* #returns Current list of sheet names
*/
const sheetNames = () =>
SpreadsheetApp.getActive()
.getSheets()
.map((sheet) => sheet.getName());
Refresher function:
/**
* #description Automatically refreshes specified custom functions
* #author TheMaster https://stackoverflow.com/users/8404453
* #version 2.0.0
* #changelog
* Updated to support all custom functions and arguments
* Avoid eternal loops
*/
/**
* #listens to changes in a Google sheet
* #see https://developers.google.com/apps-script/guides/triggers/installable#managing_triggers_manually
*/
function onChange(e) {
/* Name of the custom function that is to be refreshed */
const customfunctionName = 'SHEETNAMES',
regexPattern = `=${customfunctionName}${String.raw`\(([^)]*?)?(?:,\s*?"RANDOM_ID_\d+")?\)`}`,
replacementRegex = `=${customfunctionName}${String.raw`($1,"RANDOM_ID_${
Math.floor(Math.random() * 500) + 1
}")`}`;
/* Avoid eternal loop
* Increase timeout if it still loops
*/
const cache = CacheService.getScriptCache(),
key = 'onChangeLastRun',
timeout = 5 * 1000 /*5s*/,
timediff = new Date() - new Date(JSON.parse(cache.get(key)));
if (timediff <= timeout /*5s*/) return;
cache.put(key, JSON.stringify(new Date()));
/* Following types of change are available:
* EDIT
* INSERT_ROW
* INSERT_COLUMN
* REMOVE_ROW
* REMOVE_COLUMN
* INSERT_GRID
* REMOVE_GRID
* FORMAT
* OTHER - This usually refers to changes made by the script itself or sheets api
*/
if (!/GRID|OTHER/.test(e.changeType)) return; //Listen only to grid/OTHER change
SpreadsheetApp.getActive()
.createTextFinder(regexPattern)
.matchFormulaText(true)
.matchCase(false)
.useRegularExpression(true)
.replaceAllWith(replacementRegex);
}
To Read:
Installable triggers
TextFinder
As noted earlier:
Custom Functions don't update unless it's arguments changes.
The possible solution is to create a checkbox in a single cell and use this cell as an argument for the custom function:
Create a checkbox: select free cell e.g. [A1], go to [Insert] > [Checkbox]
Make this cell an argument: =myFunction(A1)
Click checkbox to refresh the formula
Use a google finance function as a parameter. Like =GOOGLEFINANCE("CURRENCY:CADARS")
Those function force reload every x minutes
Since google app script is an extension of JS, functions should be able to handle more args than defined in function signature or fewer. So if you have some function like
function ADD(a, b) {
return CONSTANTS!$A$1 + a + b
}
then you'd call this func like
=ADD(A1, B1, $A$2)
where $A$2 is some checkbox (insert -> checkbox) that you can click to "refresh" after you needed to change the value from the sheet & cell CONSTANTS$A$1
I use a dummy variable in a function, this variable refers to a cell in the spreadsheet. Then I have a Myfunction() in script that writes a Math.Random number in that cell.
MyFunction is under a trigger service (Edit/Current Project Triggers) and you can choose different event-triggers, for example On-Open or time driven, there you can choose for example a time period, from 1 minute to a month.
Working off of Lexi's script as-is, it didn't seem to work anymore with the current Sheets, but if I add the dummy variable into my function as a parameter (no need to actually use it inside the function), it will indeed force google sheets to refresh the page again.
So, declaration like: function myFunction(firstParam,dummy) and then calling it would be as has been suggested. That worked for me.
Also, if it is a nuisance for a random variable to appear on all of your sheets that you edit, an easy remedy to limit to one sheet is as follows:
function onEdit(e)
{
e.source.getSheetByName('THESHEETNAME').getRange('J1').setValue(Math.random());
}
another solution to the caching problem.
have a dummy variable in your method.
pass
Filter(<the cell or cell range>,1=1)
as the value to that parameter.
e.g.
=getValueScript("B1","B4:Z10", filter(B4:Z10,1=1))
the output of filter is not used. however it indicates to the spreadsheet that this formula is sensitive to B4:Z10 range.
I had a similar issue creating a dashboard for work. Chamil's solution above (namely using Sheet's Filter function passed as the value to a dummy variable in your function) works just fine, despite the more recent comment from Arsen. In my case, I was using a function to monitor a range and could not use the filter on the same range since it created a circular reference. So I just had a cell (in my case E45 in the code below) in which I changed the number anytime I wanted my function to update:
=myFunction("E3:E43","D44",filter(E45,1=1))
As Chamil indicated, the filter is not used in the script:
function myFunction(range, colorRef, dummy) {
variable 'dummy' not used in code here
}
Today I solved this by
adding another parameter to my function:
function MY_FUNC(a, b, additional_param) { /* ... */ }
adding a value to this parameter as well, referencing a cell:
=MY_FUNC("a", "b", A1)
and putting a checkbox in that referenced cell (A1).
Now, it takes only one click (on the checkbox) to force recalculating my function.
What you could do is to set up another cell somewhere in the spreadsheet that will be updated every time a new sheet is added. Make sure it doesn't update for every change but only when you want to do the calculation (in your case when you add a sheet). You then pass the reference to this cell to your custom function. As mentioned the custom function can ignore this parameter.
Given that feature explained by Henrique Abreu, you may try the out-of-box spreadsheet function QUERY , that SQL liked query is what I use often in work
on raw data, and get data as summary to a different tab, result data is updated in real time following change in raw data.
My suggestion is based on the fact that your script has not advanced work such as URL fetch, just data work, as without actual data read, I cannot give a precise solution with QUERY.
Regarding the cache feature mentioned by Henrique Abreu (I don't have enough reputation to comment directly under his answer), I did testing and found that:
looks there is no cache working, testing function's script shown below:
function adder(base) {
Utilities.sleep(5000);
return base + 10;
}
applying that custom function adder() in sheet by calling a cell, and then changed that cell value forth and back, each time I see the loading message and total time more than 5 seconds.
It might be related to the update mentioned in this GAS issue:
This issue has now been fixed. Custom functions in New Sheets are now context aware and do not cache values as aggressively.
the issue mentioned in this topic remains, my testing suggests that, Google sheet recalculate custom function each time ONLY WHEN
value DIRECTLY called by function is changed.
function getCellValue(sheetName,row,col)
{
var ss= SpreadsheetApp.getActiveSpreadsheet();
var sh = ss.getSheetByName(sheetName);
return sh.getRange(row, col).getValue();
}
A change of any value in yellow cells will lead to recalculation of custom function; the real data source value change is ignored by function.
function containing cell's location is changed in sheet. ex. insert/remove a row/column above or left side.
I did not want to have a dummy parameter. YMMV on this.
1 A cell that is a 'List of Items', one is "Refresh"
2 Script with 'onEdit', if the cell is "Refresh":
a)Empty out the document cache
b)Fill doc cache with external data (a table in my case)
c)For all cells with my 'getStockoData(...' custom function
get the formula
set '=0'
set the fromula
d)Set the cell in (1) with a value of "Ready"
This does refresh the bits you want BUT IS NOT FAST.
I followed this video, from 1:44, and this worked for me.
You should use a specific update function, initialize a "current time" variable and pass this permanently updated variable to your custom function. Then go to Triggers and set up an "every-minute" time-driven trigger for the update function (or choose another time interval for updates).
The code:
function update() {
var dt = new Date();
var ts = dt.toLocaleTimeString();
var cellVal = '=CustomFunction("'+ ts + '")';
SpreadsheetApp.getActiveSheet().getRange('A1').setValue(cellVal);
}
Just add GOOGLEFINANCE("eurusd") as an additional argument to your custom function, like:
=myFunction(arg1, arg2, GOOGLEFINANCE("eurusd"))
As #Brionius said put an extra dinamic argument on the function. if you use now() you may have timeout problems make the update a little bit slower...
cell A1 = int(now()*1000)
cell A2 = function(args..., A1)
If you have written a custom function and used it in your spreadsheet as a formula, then each time you open the spreadsheet or any referencing cell is modified, the formula is recalculated.
If you want to just keep staring at the spreadsheet and want its values to change, then consider adding a timed trigger that will update the cells. Read more about triggers here

Script to summarise data not updating [duplicate]

I've written a custom Google Apps Script that will receive an id and fetch information from a web service (a price).
I use this script in a spreadsheet, and it works just fine. My problem is that these prices change, and my spreadsheet doesn't get updated.
How can I force it to re-run the script and update the cells (without manually going over each cell)?
Ok, it seems like my problem was that google behaves in a weird way - it doesn't re-run the script as long as the script parameters are similar, it uses cached results from the previous runs. Hence it doesn't re-connect to the API and doesn't re-fetch the price, it simply returns the previous script result that was cached.
See more info here(Add a star to these issues, if you're affected):
https://issuetracker.google.com/issues/36753882
https://issuetracker.google.com/issues/36763858
and Henrique G. Abreu's answer
My solution was to add another parameter to my script, which I don't even use. Now, when you call the function with a parameter that is different than previous calls, it will have to rerun the script because the result for these parameters will not be in the cache.
So whenever I call the function, for the extra parameter I pass "$A$1".
I also created a menu item called refresh, and when I run it, it puts the current date and time in A1, hence all the calls to the script with $A$1 as second parameter will have to recalculate. Here's some code from my script:
function onOpen() {
var sheet = SpreadsheetApp.getActiveSpreadsheet();
var entries = [{
name : "Refresh",
functionName : "refreshLastUpdate"
}];
sheet.addMenu("Refresh", entries);
};
function refreshLastUpdate() {
SpreadsheetApp.getActiveSpreadsheet().getRange('A1').setValue(new Date().toTimeString());
}
function getPrice(itemId, datetime) {
var headers =
{
"method" : "get",
"contentType" : "application/json",
headers : {'Cache-Control' : 'max-age=0'}
};
var jsonResponse = UrlFetchApp.fetch("http://someURL?item_id=" + itemId, headers);
var jsonObj = eval( '(' + jsonResponse + ')' );
return jsonObj.Price;
SpreadsheetApp.flush();
}
And when I want to put the price of item with ID 5 in a cell, I use the following formula:
=getPrice(5, $A$1)
When I want to refresh the prices, I simply click the "Refresh" -> "Refresh" menu item.
Remember that you need to reload the spreadsheet after you change the onOpen() script.
You're missing the fastidious caching bug feature. It works this way:
Google considers that all your custom functions depend only on their parameters values directly to return their result (you can optionally depend on other static data).
Given this prerequisite they can evaluate your functions only when a parameter changes. e.g.
Let's suppose we have the text "10" on cell B1, then on some other cell we type =myFunction(B1)
myFunction will be evaluated and its result retrieved. Then if you change cell B1 value to "35", custom will be re-evaluated as expected and the new result retrieved normally.
Now, if you change cell B1 again to the original "10", there's no re-evaluation, the original result is retrieved immediately from cache.
So, when you use the sheet name as a parameter to fetch it dynamically and return the result, you're breaking the caching rule.
Unfortunately, you can't have custom functions without this amazing feature. So you'll have to either change it to receive the values directly, instead of the sheet name, or do not use a custom function. For example, you could have a parameter on your script telling where the summaries should go and have an onEdit update them whenever a total changes.
What I did was similar to tbkn23. This method doesn't require any user action except making a change.
The function I want to re-evaluate has an extra unused parameter, $A$1. So the function call is
=myFunction(firstParam, $A$1)
But in the code the function signature is
function myFunction(firstParam)
Instead of having a Refresh function I've used the onEdit(e) function like this
function onEdit(e)
{
SpreadsheetApp.getActiveSheet().getRange('A1').setValue(Math.random());
}
This function is triggered whenever any cell in the spreadsheet is edited. So now you edit a cell, a random number is placed in A1, this refreshes the parameter list as tbkn23 suggested, causing the custom function to be re-evaluated.
There are settings where you can make NOW() update automatically:
If your custom function is inside a specific column, simply order your spreadsheet by that column.
The ordering action forces a refresh of the data, which invokes your custom function for all rows of that column at once.
Script Logic:
Custom Functions don't update unless it's arguments changes.
Create a onChange trigger to change all arguments of all custom functions in the spreadsheet using TextFinder
The idea to add a extra dummy parameter by #tbkn23 and use of the triggers by #Lexi Brush is implemented here with a random number as argument. This answer mainly differs due to usage of class TextFinder(a relatively new addition to Apps script), which is better because
No extra cell is required.
No menu is needed > No additional clicks needed. If you need a custom refresher, a checkbox is a better implementation
You can also change the formula itself instead of changing the parameters
The change/trigger can be configured to filter out only certain changes. For eg, the following sample script trigger filters out all changes except INSERT_GRID/REMOVE_GRID(Grid=Sheet). This is appropriate for the custom function that provides sheetnames. A edit anywhere isn't going to change the list of sheets/sheetnames, but inserting or removing sheet does.
Sample custom function(to refresh):
/**
* #customfunction
* #OnlyCurrentDoc
* #returns Current list of sheet names
*/
const sheetNames = () =>
SpreadsheetApp.getActive()
.getSheets()
.map((sheet) => sheet.getName());
Refresher function:
/**
* #description Automatically refreshes specified custom functions
* #author TheMaster https://stackoverflow.com/users/8404453
* #version 2.0.0
* #changelog
* Updated to support all custom functions and arguments
* Avoid eternal loops
*/
/**
* #listens to changes in a Google sheet
* #see https://developers.google.com/apps-script/guides/triggers/installable#managing_triggers_manually
*/
function onChange(e) {
/* Name of the custom function that is to be refreshed */
const customfunctionName = 'SHEETNAMES',
regexPattern = `=${customfunctionName}${String.raw`\(([^)]*?)?(?:,\s*?"RANDOM_ID_\d+")?\)`}`,
replacementRegex = `=${customfunctionName}${String.raw`($1,"RANDOM_ID_${
Math.floor(Math.random() * 500) + 1
}")`}`;
/* Avoid eternal loop
* Increase timeout if it still loops
*/
const cache = CacheService.getScriptCache(),
key = 'onChangeLastRun',
timeout = 5 * 1000 /*5s*/,
timediff = new Date() - new Date(JSON.parse(cache.get(key)));
if (timediff <= timeout /*5s*/) return;
cache.put(key, JSON.stringify(new Date()));
/* Following types of change are available:
* EDIT
* INSERT_ROW
* INSERT_COLUMN
* REMOVE_ROW
* REMOVE_COLUMN
* INSERT_GRID
* REMOVE_GRID
* FORMAT
* OTHER - This usually refers to changes made by the script itself or sheets api
*/
if (!/GRID|OTHER/.test(e.changeType)) return; //Listen only to grid/OTHER change
SpreadsheetApp.getActive()
.createTextFinder(regexPattern)
.matchFormulaText(true)
.matchCase(false)
.useRegularExpression(true)
.replaceAllWith(replacementRegex);
}
To Read:
Installable triggers
TextFinder
As noted earlier:
Custom Functions don't update unless it's arguments changes.
The possible solution is to create a checkbox in a single cell and use this cell as an argument for the custom function:
Create a checkbox: select free cell e.g. [A1], go to [Insert] > [Checkbox]
Make this cell an argument: =myFunction(A1)
Click checkbox to refresh the formula
Use a google finance function as a parameter. Like =GOOGLEFINANCE("CURRENCY:CADARS")
Those function force reload every x minutes
Since google app script is an extension of JS, functions should be able to handle more args than defined in function signature or fewer. So if you have some function like
function ADD(a, b) {
return CONSTANTS!$A$1 + a + b
}
then you'd call this func like
=ADD(A1, B1, $A$2)
where $A$2 is some checkbox (insert -> checkbox) that you can click to "refresh" after you needed to change the value from the sheet & cell CONSTANTS$A$1
I use a dummy variable in a function, this variable refers to a cell in the spreadsheet. Then I have a Myfunction() in script that writes a Math.Random number in that cell.
MyFunction is under a trigger service (Edit/Current Project Triggers) and you can choose different event-triggers, for example On-Open or time driven, there you can choose for example a time period, from 1 minute to a month.
Working off of Lexi's script as-is, it didn't seem to work anymore with the current Sheets, but if I add the dummy variable into my function as a parameter (no need to actually use it inside the function), it will indeed force google sheets to refresh the page again.
So, declaration like: function myFunction(firstParam,dummy) and then calling it would be as has been suggested. That worked for me.
Also, if it is a nuisance for a random variable to appear on all of your sheets that you edit, an easy remedy to limit to one sheet is as follows:
function onEdit(e)
{
e.source.getSheetByName('THESHEETNAME').getRange('J1').setValue(Math.random());
}
another solution to the caching problem.
have a dummy variable in your method.
pass
Filter(<the cell or cell range>,1=1)
as the value to that parameter.
e.g.
=getValueScript("B1","B4:Z10", filter(B4:Z10,1=1))
the output of filter is not used. however it indicates to the spreadsheet that this formula is sensitive to B4:Z10 range.
I had a similar issue creating a dashboard for work. Chamil's solution above (namely using Sheet's Filter function passed as the value to a dummy variable in your function) works just fine, despite the more recent comment from Arsen. In my case, I was using a function to monitor a range and could not use the filter on the same range since it created a circular reference. So I just had a cell (in my case E45 in the code below) in which I changed the number anytime I wanted my function to update:
=myFunction("E3:E43","D44",filter(E45,1=1))
As Chamil indicated, the filter is not used in the script:
function myFunction(range, colorRef, dummy) {
variable 'dummy' not used in code here
}
Today I solved this by
adding another parameter to my function:
function MY_FUNC(a, b, additional_param) { /* ... */ }
adding a value to this parameter as well, referencing a cell:
=MY_FUNC("a", "b", A1)
and putting a checkbox in that referenced cell (A1).
Now, it takes only one click (on the checkbox) to force recalculating my function.
What you could do is to set up another cell somewhere in the spreadsheet that will be updated every time a new sheet is added. Make sure it doesn't update for every change but only when you want to do the calculation (in your case when you add a sheet). You then pass the reference to this cell to your custom function. As mentioned the custom function can ignore this parameter.
Given that feature explained by Henrique Abreu, you may try the out-of-box spreadsheet function QUERY , that SQL liked query is what I use often in work
on raw data, and get data as summary to a different tab, result data is updated in real time following change in raw data.
My suggestion is based on the fact that your script has not advanced work such as URL fetch, just data work, as without actual data read, I cannot give a precise solution with QUERY.
Regarding the cache feature mentioned by Henrique Abreu (I don't have enough reputation to comment directly under his answer), I did testing and found that:
looks there is no cache working, testing function's script shown below:
function adder(base) {
Utilities.sleep(5000);
return base + 10;
}
applying that custom function adder() in sheet by calling a cell, and then changed that cell value forth and back, each time I see the loading message and total time more than 5 seconds.
It might be related to the update mentioned in this GAS issue:
This issue has now been fixed. Custom functions in New Sheets are now context aware and do not cache values as aggressively.
the issue mentioned in this topic remains, my testing suggests that, Google sheet recalculate custom function each time ONLY WHEN
value DIRECTLY called by function is changed.
function getCellValue(sheetName,row,col)
{
var ss= SpreadsheetApp.getActiveSpreadsheet();
var sh = ss.getSheetByName(sheetName);
return sh.getRange(row, col).getValue();
}
A change of any value in yellow cells will lead to recalculation of custom function; the real data source value change is ignored by function.
function containing cell's location is changed in sheet. ex. insert/remove a row/column above or left side.
I did not want to have a dummy parameter. YMMV on this.
1 A cell that is a 'List of Items', one is "Refresh"
2 Script with 'onEdit', if the cell is "Refresh":
a)Empty out the document cache
b)Fill doc cache with external data (a table in my case)
c)For all cells with my 'getStockoData(...' custom function
get the formula
set '=0'
set the fromula
d)Set the cell in (1) with a value of "Ready"
This does refresh the bits you want BUT IS NOT FAST.
I followed this video, from 1:44, and this worked for me.
You should use a specific update function, initialize a "current time" variable and pass this permanently updated variable to your custom function. Then go to Triggers and set up an "every-minute" time-driven trigger for the update function (or choose another time interval for updates).
The code:
function update() {
var dt = new Date();
var ts = dt.toLocaleTimeString();
var cellVal = '=CustomFunction("'+ ts + '")';
SpreadsheetApp.getActiveSheet().getRange('A1').setValue(cellVal);
}
Just add GOOGLEFINANCE("eurusd") as an additional argument to your custom function, like:
=myFunction(arg1, arg2, GOOGLEFINANCE("eurusd"))
As #Brionius said put an extra dinamic argument on the function. if you use now() you may have timeout problems make the update a little bit slower...
cell A1 = int(now()*1000)
cell A2 = function(args..., A1)
If you have written a custom function and used it in your spreadsheet as a formula, then each time you open the spreadsheet or any referencing cell is modified, the formula is recalculated.
If you want to just keep staring at the spreadsheet and want its values to change, then consider adding a timed trigger that will update the cells. Read more about triggers here