I have this google script to import api calls into google sheet cells.
/**
* Imports JSON data to your spreadsheet
* #param url URL of your JSON data as string
* #param xpath simplified xpath as string
* #customfunction
*/
function IMPORTJSON(url,xpath){
try{
var res = UrlFetchApp.fetch(url);
var content = res.getContentText();
var json = JSON.parse(content);
var patharray = xpath.split(".");
//Logger.log(patharray);
for(var i=0;i<patharray.length;i++){
json = json[patharray[i]];
Utilities.sleep(1000);
}
//Logger.log(typeof(json));
if(typeof(json) === "undefined"){
return "Node Not Available";
} else if(typeof(json) === "object"){
var tempArr = [];
for(var obj in json){
tempArr.push([obj,json[obj]]);
}
return tempArr;
} else if(typeof(json) !== "object") {
return json;
}
}
catch(err){
return "Error getting data";
}
}
And I have various cells into google sheet that calls this function.
To update the cache and get updated values from the same api calls, i added a random number parameter at the end of each string that chance simultaneously for all the calls
=IF(C6>0;IMPORTJSON(concatenate("https://axieinfinity.com/graphql-server-v2/graphql?operationName=GetAxieBriefList&query=query%20GetAxieBriefList%20%7B%20axies(auctionType:All,owner:%22";H6;"%22,%20from:%200,%20sort:%20PriceAsc,%20size:%20100)%20%7B%20total%20%7D%20%7D");"data.axies.total";doNotDelete!$A$1);0)
My problem is that I want to put a delay into the script to avoid updating all the api calls in the same time.
How can I achieve this result? Is it possible just adding a utilities.sleep(500) into the script? Because I didn't have any success by doing that
It's very likely that using a custom function is not a good idea because they have several limitations, instead consider to use other means to update the cells like using Range.setValue, Range.setValues or the Advanced Sheet Service (more specifically batchUpdate). Google Apps Script functions having these methods could be called from a custom menu, a simple or installable trigger, from client-side code by using google.script.run, etc.
Some of the relevant limitations of custom functions are
30 seconds maximum execution time
all the formulas (including those having custom functions) are recalculated when the spreadsheet is opened.
Related
I try to use google sheets to write and read some data using post requests,
the writing part works, but it never returns any value back.
function doPost(e) { return handleResponse(e); }
function handleResponse(e) {
// Get public lock, one that locks for all invocations
// (https://gsuite-developers.googleblog.com/2011/10/concurrency-and-google-apps-script.html)
var lock = LockService.getPublicLock();
// Allow the write process up to 2 seconds
lock.waitLock(2000);
try {
// Generate a (not very good) UUID for this submission
var submissionID = e.parameter.id || 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function(c) {
var r = Math.random()*16|0, v = c == 'x' ? r : (r&0x3|0x8);
return v.toString(16);
});
// Open the spreadsheet document and select the right sheet page
var sheetName = e.parameter.sheet_name|| 'Sheet1';
var doc = SpreadsheetApp.openById(SCRIPT_PROP.getProperty("key"));
var sheet = doc.getSheetByName(sheetName);
//get information out of post request
var action = e.parameter.action || 'save';
var pName = e.parameter.name;
var rowNumber = findRow(pName,sheetName);
var headRow = e.parameter.header_row || 1;
var headers = sheet.getRange(headRow, 1, 1, sheet.getLastColumn()).getValues()[0];
// check for action is loading
if(action == 'load'){
//check if the name has data
if (rowNumber){
//loads all the give values out of the parameters
var answer = [];
Logger.log('hadders: ' + headers);
for (i in headers) {
if (e.parameter[headers[i].toLowerCase()] !== undefined) {
var val = sheet.getRange(rowNumber, 1, 1,sheet.getLastColumn()).getValues()[0][i];
answer.push(val);
}
}
Logger.log('answer: '+ answer);
// Return result in JSON
return ContentService
.createTextOutput({body:{parameter:{answer}}})
.setMimeType(ContentService.MimeType.JSON)
;
}else{
// return error name wasn't found in sheet.
return ContentService
.createTextOutput("can't find Name")
.setMimeType(ContentService.MimeType.TEXT)
;
}
}
The logger returns all the right values,
but logging the return value from this function ends up in an empty object.
I tried just making my own return object like:
return ContentService
.createTextOutput({body={parameter={answer=JSON.stringify(answer)}}})
.setMimeType(ContentService.MimeType.TEXT)
;
I know that I need to use &= instead of ,: but it still returned nothing.
In your script, how about modifying as follows?
From:
return ContentService
.createTextOutput({body:{parameter:{answer}}})
.setMimeType(ContentService.MimeType.JSON)
To:
return ContentService
.createTextOutput(JSON.stringify({body:{parameter:{answer}}}))
.setMimeType(ContentService.MimeType.JSON)
In the case of createTextOutput({body:{parameter:{answer}}}), the object cannot be directly put. So I thought that it is required to convert it to the string.
Note:
When you modified the Google Apps Script, please modify the deployment as a new version. By this, the modified script is reflected in Web Apps. Please be careful this.
You can see the detail of this in the report of "Redeploying Web Apps without Changing URL of Web Apps for new IDE".
I am trying to understand why a custom google sheet script can initially pull in JSON data when I have the sheet open, as defined in cell(A1).
20 minutes after closing the google sheet in the browser, I access the published html page of that sheet and the data is not displayed.
I wanted to see a proof of concept for a google-apps-script function that is run within google sheets and fetches a JSON page and displays the data as a google sheets table in html.
I went through this tutorial which gives an example function https://www.youtube.com/watch?v=EXKhVQU37WM&t=2s
* Imports JSON data to your spreadsheet Ex: IMPORTJSON("http://myapisite.com","city/population")
* #param url URL of your JSON data as string
* #param xpath simplified xpath as string
* #customfunction
*/
function IMPORTJSON(url,xpath){
try{
// /rates/EUR
var res = UrlFetchApp.fetch(url);
var content = res.getContentText();
var json = JSON.parse(content);
var patharray = xpath.split("/");
//Logger.log(patharray);
for(var i=0;i<patharray.length;i++){
json = json[patharray[i]];
}
//Logger.log(typeof(json));
if(typeof(json) === "undefined"){
return "Node Not Available";
} else if(typeof(json) === "object"){
var tempArr = [];
for(var obj in json){
tempArr.push([obj,json[obj]]);
}
return tempArr;
} else if(typeof(json) !== "object") {
return json;
}
}
catch(err){
return "Error getting data";
}
}
When I open the sheet in cell A1 the function IMPORTJSON is called. I get an error for around 5 seconds("error, unknown function") and then the data is fetched and correctly displayed.
When I go to the link of the published sheet page as html (file -> share -> publish to web) 20 minutes after closing the sheet, I get a #NAME? error in the cell with the function.
What the error looks like in chrome.
google doc: https://docs.google.com/spreadsheets/d/1iDuaKn5S6jnRJmGW6iENvQ6-cE5BK-R-ZEnIVBPJc5w/edit?usp=sharing
html published page
https://docs.google.com/spreadsheets/d/e/2PACX-1vRllWzf7g-lARy_PgRUNHc0Jz1AB9W8nN0tmfvLQzE4rpwq3j3C7DwiD154K6_UeilDUpkLSGO8UIJT/pubhtml?gid=0&single=true
Here is the function as defined in the Apps Script for the sheet
https://www.chicagocomputerclasses.com/google-sheets-import-json-importjson-function/
Function call(cellA1) (XXX is API key):
=IMPORTJSON("http://data.fixer.io/api/latest?access_key=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&symbols=USD,AUD,CAD,PLN,MXN&format=1","rates")
Why can't the data pulled by the function(when the sheet is open in the browser) be static on the sheet when the google sheet is closed, and the html version of the sheet is accessed?
I haven't called the function in the meantime by accessing the google sheet.
Unfortunately, this is a bug in the google sheet framework. Loading the html version of a sheet does not allow the sheet to call custom functions.
https://issuetracker.google.com/issues/218993622
I'm trying to use a parameter in a Google web app url to select which tab of a Google Sheet I want to extract data from. The request for data is made via a Chatfuel JSON GET request, which pulls data from the spreadsheet tab and returns it as formatted json code into the chatbot.
When I run the request without passing a parameter and manually enter the sheet tab name into the doGet function of the Google Sheet script as below it works fine.
URL - https://script.google.com/macros/s/xxx-xxx-xxx/exec
function doGet() {
var sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Retail");
var data = sheet.getDataRange().getValues();
elements = create_elements(data)
if (elements.length != 0) {
return buildImageGallery(elements);
} else {
return notFound()
}
}
I just can't quite figure out how to take a parameter from a url (e.g "type"), and then use that as part of the doGet function - this is what I'm roughly trying to do...
URL - https://script.google.com/macros/s/xxx-xxx-xxx/exec?type=Retail
function doGet() {
var type = e.parameter.type;
var sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName(type);
var data = sheet.getDataRange().getValues();
elements = create_elements(data)
if (elements.length != 0) {
return buildImageGallery(elements);
} else {
return notFound()
}
}
I'm sure it's soemthing simple, but I'm just getting started with this coding stuff so I'm not sure what I need to change. Thanks in advance for your help.
Does function doGet(e) work?
Is it a typo or you missed it? The e parameter inside the function.
I'm trying to import a column with all the closing BTC price into a Google sheet. (within a specific set of dates)
I already have this script that allows importJSON:
/**
* Imports JSON data to your spreadsheet Ex: IMPORTJSON("http://myapisite.com","city/population")
* #param url URL of your JSON data as string
* #param xpath simplified xpath as string
* #customfunction
*/
function IMPORTJSON(url,xpath){
try{
// /rates/EUR
var res = UrlFetchApp.fetch(url);
var content = res.getContentText();
var json = JSON.parse(content);
var patharray = xpath.split(".");
//Logger.log(patharray);
for(var i=0;i<patharray.length;i++){
json = json[patharray[i]];
}
//Logger.log(typeof(json));
if(typeof(json) === "undefined"){
return "Node Not Available";
} else if(typeof(json) === "object"){
var tempArr = [];
for(var obj in json){
tempArr.push([obj,json[obj]]);
}
return tempArr;
} else if(typeof(json) !== "object") {
return json;
}
}
catch(err){
return "Error getting data";
}
}
in which I am able to get for i.e. the last price with this JSON call:
=importJSON("https://api.coingecko.com/api/v3/coins/marketsvs_currency=eur&ids=bitcoin","0.current_price")
anyways I'm not able to make a correct JSON call or something similar to import a column with the btc closing price from a whatever date to current date.
I have installed a spreadsheet extension called cryptosheet and tried to use the SS() function but nothing works so far.
From your replying and your question, I could understand like below.
You want to retrieve the values of current_price using your script.
The URL is https://api.coingecko.com/api/v3/coins/bitcoin/history?date=30-12-2017&localization=false.
Modification points:
From your replying, it seems that the xpath is not used. By this, I think that the error occurs.
When I saw the data from the URL and your script, the formula might be as follows.
Modified formula:
=IMPORTJSON(A1, "market_data.current_price")
In this case, the URL of https://api.coingecko.com/api/v3/coins/bitcoin/history?date=30-12-2017&localization=false is put to the cell "A1".
Result:
I have created a sheet to keep my crypto holdings. I use this importJSON function I found on youtube : (I have changed the help text for myself)
/**
* Imports JSON data to your spreadsheet Ex: IMPORTJSON("https://api.coinmarketcap.com/v2/ticker/1/?convert=EUR","data/quotes/EUR/price")
* #param url URL of your JSON data as string
* #param xpath simplified xpath as string
* #customfunction
*/
function IMPORTJSON(url,xpath){
try{
// /rates/EUR
var res = UrlFetchApp.fetch(url);
var content = res.getContentText();
var json = JSON.parse(content);
var patharray = xpath.split("/");
//Logger.log(patharray);
for(var i=0;i<patharray.length;i++){
json = json[patharray[i]];
}
//Logger.log(typeof(json));
if(typeof(json) === "undefined"){
return "Node Not Available";
} else if(typeof(json) === "object"){
var tempArr = [];
for(var obj in json){
tempArr.push([obj,json[obj]]);
}
return tempArr;
} else if(typeof(json) !== "object") {
return json;
}
}
catch(err){
return "Error getting data";
}
}
I use this function to readout an API :
This is a piece of my script :
var btc_eur = IMPORTJSON("https://api.coinmarketcap.com/v2/ticker/1/?convert=EUR","data/quotes/EUR/price");
var btc_btc = IMPORTJSON("https://api.coinmarketcap.com/v2/ticker/1/?convert=BTC","data/quotes/BTC/price");
ss.getRange("B2").setValue([btc_eur]);
ss.getRange("H2").setValue([btc_btc]);
var bhc_eur = IMPORTJSON("https://api.coinmarketcap.com/v2/ticker/1831/?convert=EUR","data/quotes/EUR/price");
var bhc_btc = IMPORTJSON("https://api.coinmarketcap.com/v2/ticker/1831/?convert=BTC","data/quotes/BTC/price");
ss.getRange("B3").setValue([bhc_eur]);
ss.getRange("H3").setValue([bhc_btc]);
The last few days I get "Error getting data" errors. When I start manualy the script it works.
I than tried this code I found here :
ImportJson
function IMPORTJSON(url,xpath){
var res = UrlFetchApp.fetch(url);
var content = res.getContentText();
var json = JSON.parse(content);
var patharray = xpath.split("/");
var res = [];
for (var i in json[patharray[0]]) {
res.push(json[patharray[0]][i][patharray[1]]);
}
return res;
}
But this gives an error about : TypeError: Cannot read property "quotes" from null. What am I doing wrong ?
The big problem is your script call API at least 4 times. When few users do it too, the Google server call API too much times.
The API of Coinmarketcap has limited bandwidth. When any client reach this limit, the API return HTTP error 429. Google Scripts is on shared Google servers, that means lot of users looks as one client for Coinmarketcap API.
When API decline your request, your script fails – the error message corresponds to the assumed error (xpath cant find quotes component in empty varible).
This is ruthless behavior. Please, don't ruin API via mass calls.
You can load data from API at once and re-use it angain for each finding in data.
I have similar Spreadsheet automatically filled from Coinmarketcap API, you can copy it for your:
Coins spreadsheet
Google Script on GitHub.
This my script is strictly ask API only once for whole runtime and reusing one response for all queries.
Change of your script
Also you can make few changes in your Code for saving resources:
Change IMPORTJSON function from this:
function IMPORTJSON(url,xpath){
var res = UrlFetchApp.fetch(url);
var content = res.getContentText();
var json = JSON.parse(content);
...
to this:
function IMPORTJSON(json, xpath) {
...
and rutime section of code you can change like this:
var res = UrlFetchApp.fetch("https://api.coinmarketcap.com/v2/ticker/1/?convert=EUR");
var content = res.getContentText();
var json = JSON.parse(content);
var btc_eur = IMPORTJSON(json,"data/quotes/EUR/price");
var btc_btc = IMPORTJSON(json,"data/quotes/BTC/price");
ss.getRange("B2").setValue([btc_eur]);
ss.getRange("H2").setValue([btc_btc]);
...
Main benefit is: the UrlFetchApp.fetch is called only once.
Yes, I know, this code is not works 1:1 like your. That because that receive prices only for EUR and not for BTC. Naturally fetching comparation between BTC and BTC is unnecessary because it is always 1 and other values you can count matematically from EUR response – please don't abuse an api for such queries.
As Jakub said, the main issue is that all requests are counted as coming from the same Google server.
One solution which I consider easier is to put a proxy server in the middle, this can be done by either purchasing a server and setting it up (which is quite complex) or using a service like Proxycrawl which includes some free requests and after that, unless you run thousands of queries per month, it should cost you less than 1 USD per month.
To do that you just need to edit one line of the script:
var res = UrlFetchApp.fetch(url);
This line, becomes this:
var res = UrlFetchApp.fetch(`https://api.proxycrawl.com/?token=YOUR_TOKEN&url=${encodeURIComponent(url)}`);
Make sure to replace YOUR_TOKEN with your actual service token
Just this simple change will make the requests never fail as each request will be sent from a different IP instead of all coming from Google.