Delay in between two calls (POST and GET) in Google Apps script - json

My first request on StackOverflow! I really hope you can help me!
I want to create a process to automatize a data report from a system to a sheet; I thought I could use their API, apps script, and export the data on google sheets.
To do so, I need to run two calls on the API:
A POST call, which runs the report within the system (it requires a date range as body).
In return, I will get an ID that is associated with the data generated and it expires after some time.
A GET call, which is a URL that contains the ID generated in the first call and created with a concatenation.
The first call works fine; I get in return the ID successfully.
My problem is when I run the second call, I don’t get any data in return, and I don’t understand what’s the issue, I can see the URL is concatenated correctly because if I copy the URL from the log and I test it on another apps script or on Postman, it works perfectly fine!
Could someone help me in case I am doing something wrong?
Here’s the code:
function callEvents() {
var API_KEY = "xxx";
var data = { 'Start Date': '2021-05-03', 'End Date': '2021-06-03' }
var options = {
'method' : 'post',
'contentType': 'application/json',
'payload' : JSON.stringify(data) };
//This is the first call
var urlEncoded = encodeURI('https://website/api/dataviewresult/
etc/json/?api_key=' + API_KEY);
var url = UrlFetchApp.fetch(urlEncoded, options);
var result = JSON.parse(url.getContentText());
Logger.log(url.getContentText());
//here I retrieve the ID to use in the second call
var ipdataview = (result["contents"]["id"]);
Logger.log(ipdataview);
//here is the concatenation and the second call
var urlEncoded2 = encodeURI('https://website/api/dataviewresult/etc/json/'+ipdataview+'/?api_key=' + API_KEY);
Logger.log(urlEncoded2);
var response = UrlFetchApp.fetch(urlEncoded2);
Logger.log(response.getContentText());

I found out the solution, I was doing the second call (Get) too soon from the first one, so the data was not being picked up.
I used utilities.sleep() for about 2 seconds and it worked perfectly.
Thank you for everyone helping!
S

Related

how to use nextPageToken

I have a script that archives old classrooms, until the end of 2021 it was working fine.
In the lasts months I got an error (the script works ok, but terminate with error) and today I was investigating it, the script runs only once per month.
The error is due to a supposed change in .nextPageToken function.
var parametri = {"courseStates": "ARCHIVED"};
var page = Classroom.Courses.list(parametri);
var listaClassi = page.courses;
var xyz = page.nextPageToken;
if (page.nextPageToken !== '') {
parametri.pageToken = page.nextPageToken;
page = Classroom.Courses.list(parametri);
listaClassi = listaClassi.concat(page.courses);
};
var xyz has been added to better understand what was happening.
So, in this case the list does not have pagination, is only one page. var xyz returns "undefined", and the "if" statement results "true", this makes that variable listaClassi got appended the same content a second time. That generate the error and the abnormal end of the script.
I found an issue reported here https://issuetracker.google.com/issues/225941023?pli=1 that may be related with my problem.
Now I could change .nextPageToken with .getNextPageToken but I found no docs on the second function and many issues reporting that is not working, can anyone help me?
When using the nextPageToken value obtained to the response make sure to enter it as a separate parameter with a slightly different name. You will obtain nextPageToken in the response, the pageToken parameter needs to be entered in the request. It does look like you are doing it right, the way you add the parameter is a bit odd, yet it should be functional.
To discard problems with the Classroom API (that we can certainly take a look at) try with this simple code example in a new Google Apps Script project, remember you will need to add an Advanced service, information about advanced services can be found in this documentation article https://developers.google.com/apps-script/guides/services/advanced. Use listFiles as the main method in your Apps Script project.
function listFiles() {
var totalClasses = 0;
nextPageToken = "";
console.log("Found the following classes:")
do {
var response = loadPage(nextPageToken);
var classes = response.courses;
for (let x in classes){
console.log("Class ID: " + classes[x].id + " named: '" + classes[x].name + "'.");
}
totalClasses += classes.length;
} while (nextPageToken = response.nextPageToken)
console.log("There are " + totalClasses + " classes.")
}
function loadPage(token = ""){
return Classroom.Courses.list({
fields: 'nextPageToken,courses(id,name)',
pageSize: 10,
pageToken: token
});
}
When we first make the API call with Apps Script we don't specify a pageToken, since it is the first run we don't have one. All calls to the List method may return a nextPageToken value if the returned page contains an incomplete response.
while (nextPageToken = response.nextPageToken)
In my code at the line above once response.nextPageToken is empty (not in the response) inside the condition block JavaScript will return false, breaking the loop and allowing the code to finish execution.
To have your incident reviewed by a Google Workspace technician you can also submit a form to open a ticket with the Google Workspace API Support team at https://support.google.com/a/contact/wsdev.

Error "302 Moved Temporarily" in "Test as add-on" mode (Telegram)

The spreadsheet contains project 1, deployed as a webapp with permissions: Execute as: Me, Who has access: Anyone.
Webapp
function doPost(e) {
myLog('Received from Addon: ' + JSON.stringify(e));
// console.log('parameters from caller ' + JSON.stringify(e));
return ContentService.createTextOutput(JSON.stringify(e));
}
A webhook aTelegram-bot and this webapp is set.
I am using this spreadsheet for testing (as add-on) of another project 2.
Add-on
function sendPost() {
var sheetURL = SpreadsheetApp.getActiveSpreadsheet().getUrl();
// var webAppUrl = "https://script.google.com/macros/s/#####/exec"; // 7: Part_1 - WebApp: My
var webAppUrl = "https://script.google.com/macros/s/###/exec"; // 7: Part_1 - WebApp: Tester
// var auth = ScriptApp.getOAuthToken();
// var header = { 'Authorization': 'Bearer ' + auth };
var payload = { scriptName: 'updateData', sheetURL: 'sheetURL' };
var options = {
method: 'post',
// headers: header,
muteHttpExceptions: true,
payload: payload
};
var resp = UrlFetchApp.fetch(webAppUrl, options);
var respCode = resp.getResponseCode();
console.log('resp: ' + respCode);
myLog(respCode);
var respTxt = resp.getContentText();
myLog('Response from webApp: ' + respTxt);
console.log('resp: ' + respTxt);
}
Here is a short video of the process (EN-subtitles).
I run sendPost() and everything works fine. Project 2 sends data to the webapp, which returns it. Since this is a Container-bound script and not a standalone one, I cannot watch the logs in the GCC logger. Therefore, I look at them in the custom logger and the entries are added normally.
Also https://api.telegram.org/bot{API_token}/getWebhookInfo shows that there are no errors:
{"ok":true,"result": {"url":"https://script.google.com/macros/s/###/exec", "has_custom_certificate":false, "pending_update_count":0, "max_connections":40,"ip_address":"142.250.***.***"}}
Now I am sending a message from the chat with the bot. The doPost(e) function in the webapp accepts it and writes it to the spreadsheet.
However, everything is not limited to one message. Requests from the bot come and come, and the logger creates more and more new rows in the spreadsheet. This happens until I redeploy the webapp with the doPost () function commented out. I tried to figure out if this is a limited loop or not. My patience was only enough for 20 such iterations, because as a result, the messages start repeating at intervals of about 1 minute. Then I have to reinstall the webhook.
In any case, it interferes with testing the addon.
GetWebhookInfo is now showing that there is a "Wrong response from the webhook: 302 Moved Temporarily" error:
{"ok":true,"result": {"url":"https://script.google.com/macros/s/###/exec", "has_custom_certificate":false, "pending_update_count":1, "last_error_date":1635501472, "last_error_message":"Wrong response from the webhook: 302 Moved Temporarily", "max_connections":40,"ip_address":"142.250.1***.***"}}
Googling revealed several possible reasons. From url to the script has changed to MITM in your network.
I do not really believe in MITM and I suppose that this is due to the fact that the spreadsheet is open in testing mode as add-on and the URL of the webapp has changed in this mode. If so, then I'm not sure if this is the correct behavior of the testing system. In theory, such a situation should have been provided for and the webap url should remain unchanged. But maybe I'm wrong and the reason is different, so
QUESTION:
Has anyone come across such a situation and will suggest a workaround on how to test a script as an addon in such conditions?
http-status-code-302 refers to redirection. If ContentService is used, Google temporarily redirects the resource to a another domain to serve the content. This redirection is not performed when using HtmlService. So, if the issue is related to redirection, use HtmlService instead.

Script to open a long list of URLs one at a time every 5 seconds and then open the next URL

I have a google spreadsheet with a large number of API Urls.
They look like this => http://oasis.caiso.com/oasisapi/SingleZip?resultformat=6&queryname=PRC_LMP&version=1&startdatetime=20160101T08:00-0000&enddatetime=20160103T08:00-0000&market_run_id=DAM&grp_type=ALL
The database I am drawing from limits requests to one every 5 seconds.
When you follow the link it will download a zip file with cvs files.
I would like to write a script that will follow a URL, wait 6 seconds and then move on to the next URL on the list.
I would like it to stop when it gets to the last URL
I am imagining that I would need to use a "when" loop, but I cannot figure out how to install a wait period, or how to get it to open the URL.
HELP!!!!
I tried a batch URL follow, which failed because of the timing issue.
I began to write the When loop, but I am totally stuck.
I would like to run through the huge list of links fully once. To date I cannot make anything work.
function flink(){
var app = spreasheetapp
//access the current open sheet
var activesheet = app.getactivespreadsheet().getactivesheet()
var activecell= activesheet.getrange(11,11).openurl
//I am getting totally stuck here
I have tried using an iterator but I have no idea how to add the time delay and then I cannot seem to get the syntax for the iterator correct.
To access a url from AppsScript, you can use UrlFetchApp.fetch(url).
To force the script to wait for a certain amount of time, you can use Utilities.sleep(milliseconds).
References:
https://developers.google.com/apps-script/reference/url-fetch/url-fetch-app#fetch(String)
https://developers.google.com/apps-script/reference/utilities/utilities#sleepmilliseconds
I came up with this code that accesses each URL in the sheet. I added some comments so you can see what the script is doing step by step:
function accessURLs() {
// Copy urls
var ss = SpreadsheetApp.getActive();
var sheet = ss.getSheetByName("Links up to today");
// Copy the contents of the url to another column removing the formulas (your script will appreciate this)
sheet.getRange("C3:C").copyTo(sheet.getRange("D3:D"), {contentsOnly:true});
var column = 4; // URLs are in column 4
var row = 3; // URLs start at row 3
var url = sheet.getRange(row, column).getValue();
var responses = []
// Loop through all urls in the sheet until it finds a cell that is blank (no more urls left)
do {
var options = {
muteHttpExceptions: true
}
var response = UrlFetchApp.fetch(url, options); // Script makes a request to the url
responses.push(response); // The latest response is added to the responses array
Utilities.sleep(6000); // The script stops for 6 seconds
row++;
url = sheet.getRange(row, column).getValue();
} while (url != ""); // Cell is not blank
}
Take into account that if we are to access ~1400 URLs and the script stops for 6 seconds after each fetch, it will take more than 2 hours to access all URLs.
Also, take into account that this script is just getting the data coming from the URL requests (it's stored in the variable responses), but it is not doing anything else. Depending on what you want to do with this data, you might want to add some extra stuff there.
I hope this is of any help.

Google sheet custom function returns undefined Values sometimes

I have wrote a function to return latest weekly closing price of a stock in Google Sheet Apps Script.
But when using this in Google Sheets some cells are getting undefined data.
But the same cells are getting values correclty sometimes. Cant understand whats the problem here.
Also is there any option to debug my code when I execute the function from googlesheet cell?
function getWeeklyClosing(stockName){
var date =new Date()
var endDate = Utilities.formatDate(new Date(), "GMT+1", "yyyy/MM/dd")
var startDate = Utilities.formatDate(new Date(date.getTime()-10*(24*3600*1000)), "GMT+1", "yyyy/MM/dd")
var url ='https://www.quandl.com/api/v3/datasets/BSE/BOM'+stockName+'?start_date='+startDate+'&end_date='+endDate+'&collapse=weekly&api_key=3VCT1cPxzV5J4eGFwfvz';
var options =
{
'muteHttpExceptions': true,
"contentType" : "application/x-www-form-urlencoded",
"headers":{"Accept":"application/json"}
};
var response = JSON.parse(UrlFetchApp.fetch(url, options))
var weeklyEma=response.dataset.data[0][4];
return weeklyEma;
}
This answer corresponds to question rev 3 which had as title "How to use Promise in appscript?"
The title of the question is asking about an attempted solution rather than the actual problem ( an X-Y problem). The assumption that UrlFetchAp.fetch is asynchrous is wrong ( See Is google apps script synchronous?); the actual problem is getting undefined values on certain cells.
The solution will depend on the what you want to do when the the fetch response is
causing the undefined values. One alternative is to replace the undefined values by "" (an empty string) before sending the values to the spreadsheet that will cause having an empty cell on Google Sheets.
By the other hand, it could be that the API you are querying is not returning the JSON that you think, so first you have to understand it and then set the rules about how to send the result to the spreadsheet as not always it's possible to transform a JSON into a simple table structure.
UrlFetchApp.fetch(url) returns a HTTP response object. This response include a HTTP response code that could 200 for a successful fetch but could return other codes due to multiple reasons.
Some people in the past have suggest the use an algorithm called exponential back-off, like in this case Error message: "Cannot connect to Gmail". Be careful to not exceed the 30 seconds execution time limit.
Related
RESTful HTTP response codes

Google Sheet Script Returning #NAME?

I have set up a script to pull in data from a JSON API into a Google Sheet. I have set it to refresh by adding a third parameter which isn't used in the API call but is linked to a cell which another script adds the current time to. This ensures that the API is called regularly.
We are then using this Google Sheet to input data into Google Ads.
It all seems to function correctly, however, when the sheet has been closed for a while (e.g. overnight) and Google Ads tries to update from the sheet, it imports #NAME? instead of the correct API value.
I have set up another script which records the API values at regular intervals. This seems to record the values correctly, suggesting that the API calls are working whilst the sheet is closed.
// Make a POST request with a JSON payload.
// Datetime parameter isn't use in API call but is used to refresh data
function TheLottAPI(game,attribute,datetime) {
var data = {
'CompanyId': 'GoldenCasket',
'MaxDrawCount': 1,
'OptionalProductFilter': [game]};
Logger.log(data);
var options = {
'method' : 'post',
'contentType': 'application/json',
// Convert the JavaScript object to a JSON string.
'payload' : JSON.stringify(data)};
var response = UrlFetchApp.fetch('https://data.api.thelott.com/sales/vmax/web/data/lotto/opendraws', options);
Logger.log('output: '+ response);
// Convert JSON response into list
var json = JSON.parse(response)
var drawList=json ["Draws"];
// Extract attribute from list
for(var i=0;i<drawList.length;i++)
{var value=drawList[i][attribute];}
Logger.log(value)
return value;
SpreadsheetApp.flush();
};
// Set date & time to refresh API call
function RefreshTime() {
SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Attributes").getRange("K4").setValue(new Date().toTimeString());
}
The correct numeric values from the API should be shown, rather than the #NAME? error.
I have checked that the API call is functioning correctly by using another script to copy the current values. The API was updating at the appropriate times overnight.
function RecordDraws() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheetByName("Draw Amounts");
var source = sheet.getRange("A3:D3");
var values = source.getValues();
values[0];
sheet.appendRow(values[0]);
};
This is my guess
Google Sheets custom functions definitions are loaded when the spreadsheet is opened by using the Google Sheets UI, then formulas are calculated and as custom functions are already defined they are calculated correctly. If the spreadsheet isn't opened this way the custom functions definitions aren't loaded thus the spreadsheet doesn't know what to do with that function and returns #NAME?
If you are already running a script that updates some values, enhance that script to do the calculations that does your custom function.
Try converting this
SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Attributes").getRange("K4").setValue(new Date().toTimeString());
Into this:
SpreadsheetApp.openById("id").getSheetByName("Attributes").getRange("K4").setValue(new Date().toTimeString());
Because I don't think there is an "active sheet" when the Spreadsheet it's closed or the method is called from the API.