exceeded maximum execution time, google sheet, how to improve it? - function

Question

Good morning, everyone,
I come to see you because after having fought hard to make my google script work, on the execution page I see that my script works, however on my google sheet I have an error: "Exceeded maximum execution time". I've seen on the internet that for custom google script functions only leave 30 sec of execution, I'm not sure how to do that? add code to tweak this feature? I confess that I didn't understand the difference between a custom function and a google app script but I know that there is 6 minutes of execution time for scripts... here is an extract of my code:
// Standard functions to call the spreadsheet sheet and activesheet
function GetPipedriveDeals2() {
let ss = SpreadsheetApp.getActiveSpreadsheet();
let sheets = ss.getSheets();
let sheet = ss.getActiveSheet();
//the way the url is build next step is to iterate between the end because api only allows a fixed number of calls (100) this way i can slowly fill the sheet.
let url = "https://laptop.pipedrive.com/v1/products:(id)?start=";
let limit = "&limit=500";
//let filter = "&filter_id=64";
let pipeline = 1; // put a pipeline id specific to your PipeDrive setup
let start = 1;
//let end = start+50;
let token = "&api_token=XXXXXXXXXXXXXXXXXXXXXXXX"
let response = UrlFetchApp.fetch(url+start+limit+token); //
let dataAll = JSON.parse(response.getContentText());
let dataSet = dataAll;
//let prices = prices;
//create array where the data should be put
let rows = [], data;
for (let i = 0; i < dataSet.data.length; i++) {
data = dataSet.data[i];
rows.push([data.id,
GetPipedriveDeals4(data.id)
]);
}
Logger.log( 'function2' ,JSON.stringify(rows,null,8) ); // Log transformed data
return rows;
}
// Standard functions to call the spreadsheet sheet and activesheet
function GetPipedriveDeals4(idNew) {
let ss = SpreadsheetApp.getActiveSpreadsheet();
let sheets = ss.getSheets();
let sheet = ss.getActiveSheet();
//the way the url is build next step is to iterate between the end because api only allows a fixed number of calls (100) this way i can slowly fill the sheet.
let url = "https://laptop.pipedrive.com/v1/products/"+idNew+"/deals:(id,d93b458adf4bf84fefb6dbce477fe77cdf9de675)?start=";
let limit = "&limit=500";
//let filter = "&filter_id=64";
let pipeline = 1; // put a pipeline id specific to your PipeDrive setup
let start = 1;
//let end = start+50;
let token = "&api_token=XXXXXXXXXXXXXXXXXXXXXX"
let response = UrlFetchApp.fetch(url+start+limit+token); //
let dataAll = JSON.parse(response.getContentText());
let dataSet = dataAll;
//Logger.log(dataSet)
//let prices = prices;
//create array where the data should be put
let rows = [], data;
if(dataSet.data === null )return
else {
for (let i = 0; i < dataSet.data.length; i++) {
data = dataSet.data[i];
let idNew = data.id;
rows.push([data.id, data['d93b458adf4bf84fefb6dbce477fe77cdf9de675']]);
}
Logger.log( 'function4', JSON.stringify(rows,null,2) ); // Log transformed data
return rows;
}
}
Thank you all in advance.
EDIT : --------------------------FOR EACH LOOP----------------------------
function getPipedriveDeals(start = 0,apiRequestLimit = 39) {
console.log("start="+start);
//Make the initial request to get the ids you need for the details.
var idsListRequest = "https://laptop.pipedrive.com/v1/products:(id)?start=";
var limit = "&limit=" + apiRequestLimit;
var token = "&api_token=XXXXXXXXXXXXXXXXXXXXXXXXXXXXX";
var response = UrlFetchApp.fetch(idsListRequest + start + limit + token);
var data = JSON.parse(response.getContentText()).data;
//For every id in the response, construct a url (the detail url) and push to a list of requests
var requests = [];
console.log("data="+data);
data.forEach(function(product) {
var productDetailUrl = "https://laptop.pipedrive.com/v1/products/" + product.id + "/deals:(id,d93b458adf4bf84fefb6dbce477fe77cdf9de675)?start=";
requests.push(productDetailUrl + start + limit + token)
});
//With the list of detail request urls, make one call to UrlFetchApp.fetchAll(requests)
var responses = UrlFetchApp.fetchAll(requests);
return [responses,JSON.parse(responses[0].getContentText()).additional_data.pagination.more_items_in_collection];
}
function getAllDeals(){
var allResponses = [];
for(var i = 0; i<500; ){
var deals = getPipedriveDeals(start=i);
deals[0].forEach((response)=>{allResponses.push(response)});
if(deals[1]){
// If there are more items sleep for 1000 milliseconds
Utilities.sleep(1000);
i+=39;
}
else{
console.log("No more items in collection.");
break;
}
}
console.log("allResponses="+allResponses);
return allResponses;
}
About the weird behavior in the logs :
-------------------------------EDIT Number2----------------------------------
Until now when I executed =getPipedriveDeals()it return False (like I was saying in my comment) and also when I try to remove additional_data.pagination.more_items_in_collection (Because it's not very usefull for me) I don't know why but the script won't work anymore, and I have some difficulties to select the data I want to return.

Answer 1

You can use UrlFetchApp.fetchAll(requests) to save in quotas and likely a lot in script execution duration.
I have removed a few redundancies in your code to make this following example showing the use of fetchAll for your case.
Example:
function getPipedriveDeals(start = 0,apiRequestLimit = 50) {
console.log("start="+start);
//Make the initial request to get the ids you need for the details.
var idsListRequest = "https://laptop.pipedrive.com/v1/products:(id)?start=";
var limit = "&limit=" + apiRequestLimit;
var token = "&api_token=xxxxxxxxxxx";
var response = UrlFetchApp.fetch(idsListRequest + start + limit + token);
var data = JSON.parse(response.getContentText()).data;
//For every id in the response, construct a url (the detail url) and push to a list of requests
var requests = [];
console.log("data="+data);
data.forEach(function(product) {
var productDetailUrl = "https://laptop.pipedrive.com/v1/products/" + product.id + "/deals:(id,d93b458adf4bf84fefb6dbce477fe77cdf9de675)?start=";
requests.push(productDetailUrl + start + limit + token)
});
//With the list of detail request urls, make one call to UrlFetchApp.fetchAll(requests)
var responses = UrlFetchApp.fetchAll(requests);
return [responses,JSON.parse(responses[0].getContentText()).additional_data.pagination.more_items_in_collection];
}
Now suppose you have a plan that limits the quota to 50 requests per 2 seconds.
You could make the limit of requests in the fetchAll call to 50, and introduce a delay/sleep of (X) milliseconds to initiate a second request, starting with id=50 like:
function getAllDeals(){
var allResponses = [];
for(var i = 0; i<500; ){
var deals = getPipedriveDeals(start=i);
deals[0].forEach((response)=>{allResponses.push(response)});
if(deals[1]){
// If there are more items sleep for 1000 milliseconds
Utilities.sleep(1000);
i+=50;
}
else{
console.log("No more items in collection.");
break;
}
}
console.log("allResponses="+allResponses);
return allResponses;
}