Exceeded maximum execution time in Google App Script with Contacts API - google-apps-script

I am creating an script responsible of generating a report containing info about a set of groups and its contacts in google app script.
If I select a few groups, everything goes OK but If I select several (20 or more), then the scripts launches the "Exceeded maximum execution time" error.
How could I reduce the script time execution?
Thanks!
My (reduced) script
var allContactsInfo = []
/II. For each group
for(var i = 0 ; i < numberOfGroups ; i++)
{
var selected = eventInfo.parameter["group"+i]
var contactInfo = []
//III. If it has been selected by the user
if(selected)
{
//IV. Get the contact group and contacts
var contactGroup = ContactsApp.getContactGroupById(eventInfo.parameter["group_"+i+"_id"])
var contacts = contactGroup.getContacts()
//IV. Iterate over each group contact
for (var j=0; j < contacts.length; j++)
{
var contact = contacts[j];
contactInfo.push(contact.getGivenName())
contactInfo.push(contact.getFamilyName())
var groups = contact.getContactGroups()
var groupsAsArray = []
for (var k = 0 ; k < groups.length; k++)
{
groupsAsArray.push(groups[k].getName())
}
contactInfo.push(groupsAsArray.sort().join())
//V. Add the contact into the array
allContactsInfo.push(contactInfo)
}
}
...
...
//VI. Fill the spreadsheet with the array built within the loop
sheet.getRange(1, 1, allContactsInfo.length, headers.length).setValues(allContactsInfo);
UPDATE
After some tests, I have found the bottleneck is produced returning user's groups (via contact.getContactGroups()).
Returning name, surnames, emails, addresses and phones works OK but If I include user's groups too, then timeout exception appears...
UPDATE 2
Here it is my working solution, hope it helps :)
https://github.com/antacerod/google-app-script-6-minutes-limitation

Finally I have found another SO post.
It shows a nice workaround related to checkbox setValue method implementation and its second parameter (Sets whether the CheckBox should be checked and optionally fires an event if the value changes as a result of this call)
In my case, it works like a charm!
Access the workaround

Related

Google Apps Script for loop not processessing all items from getThreads()

I am attempting to create a function that reads the body of emails and extracts parts to place in a sheet.
I am currently using the below code to pull the emails.
var label = GmailApp.getUserLabelByName("VOIDS");
var threads = label.getThreads();
for (var i = 0; i <= threads.length; i++)
{
var message = threads[i].getMessages();
var body = message[0].getPlainBody();
//email processing//
threads[i].removeLable(label)
}
I've got the loop to do what I need it to do as far as processing the email and placing it where I need it, however it seems to be skipping emails. I've left out the code for the process as it's just a bunch of split() functions on the body variable to extract the appropriate information and paste it into a sheet.
The total number of emails skipped varies based on how many it has to process, but re-running the script results in the same emails being skipped each time.
All emails are having their label removed and all emails are identical save for a few value changes.
This is my first time working with GmailApp outside of sending emails. I'm sure that this is something super simple that I'm just missing, but despite all my Google searching I can't seem to find a solution.
Thank you!
Please bear in mind that getThreads might not return all the threads in your mailbox.
From the official reference docs
getThreads()
Gets the threads that are marked with this label.
This calls fail when the size of all threads is too large for the system to handle. Where the thread size is unknown, and potentially very large, please use getThreads(start, max) and specify ranges of the threads to retrieve in each call.
Resources
https://developers.google.com/apps-script/reference/gmail/gmail-label#getthreads
Related
Trying to understand getThreads in GAS
Make getThreads() app script call count over 500
A thread can contain more than one email
The line var body = message[0].getPlainBody(); implies that you are only proceeding the body of the first message of each thread.
To apply your request t all emails, you need to create a second loop, iterating through each email of each thread.
Sample:
var label = GmailApp.getUserLabelByName("VOIDS");
var threads = label.getThreads();
for (var i = 0; i <= threads.length; i++){
var messages = threads[i].getMessages();
for (var j = 0; j <= messages.length; j++){
var message = messages[j];
var body = message.getPlainBody();
//email processing//
}
threads[i].removeLable(label);
}

Is there a more efficient way to get arbitrary numbers of threads from Gmail with GmailApp.search()?

I am trying to get a large number of emails from a specific label in my Gmail and put them in a Google Drive document. How can I use GmailApp.search() to get the messages without missing any of them?
I tried simply using the non-range-defined GmailApp.search(). According to logs, it only returned 500 emails, while I have approximately 2000 in this label. The problem is that if there aren't 500 (or whatever number) more emails available, GmailApp.search() returns a blank array instead.
Say I have 600 emails, but don't know the exact number.
I can return the first 500 using GmailApp.search(searchQuery, 0, 500), but then the second search of 500 will return a blank array because only 100 emails remain. How can I get these remaining 100 in this scenario? Is stepping through the threads one at a time, as described in the code below, really the best solution if I want every single email?
for(var i = 0; i < ; i++) {
//get a single thread
var thread = GmailApp.search(searchQuery, i, 1);
//...
}
I want to output all of the emails, but what happens with GmailApp.search(searchQuery) with no specified range is that 500 emails are returned (at least according to Logger - I have it log a line after every email and it only gets up to 499, starting on 0).
More efficient than var thread = GmailApp.search(searchQuery, i, 1); is to estimate how many emails you maximally expect (e.g. 2000) and run search query 2000/500 times:
var threadArray=[];
for(var j = 0; j <3 ; j++){
var thread = GmailApp.search(searchQuery, j*500, 500);
thread.forEach(function(e){threadArray.push(e.getId());})
//now you have an array containing the Ids of all queried emails
}
This will only append to your array the found elements and no blank entries.
Try this:
function getAllThreads(qry) {
var qry=qry || 'label:qs-financial-institutions-wfb';
var threads=GmailApp.search(qry);
var html="";
for(var i=0;i<threads.length;i++) {
html+=Utilities.formatString('<br />%s -Thread Id:%s First Subject: %s Message Count: %s',i+1,threads[i].getId(),threads[i].getFirstMessageSubject(),threads[i].getMessageCount());
//You can put another loop in here to get all of the messages
}
var userInterface=HtmlService.createHtmlOutput(html).setWidth(1200);
SpreadsheetApp.getUi().showModelessDialog(userInterface, "Threads");
}

How to read all emails in gmail using google apps script

I'm trying to read ALL email in my gmail account - inbox, sent, draft, trash, emails with labels, archive, etc. I could live without the junk but I want everything else.
(all examples below use try {} catch {} to avoid errors with empty labels etc.)
I've tried
for (var i=StartLabel; i<=EndLabel; i++)
{
var label = labels[i].getName();
// get all messages, then join them into a single dimension array
var messages = GmailApp.getMessagesForThreads(GmailApp.search("label:" + label))
.reduce(function(a, b) {return a.concat(b);});
CountByLabels += messages.length;
}
That gives me everything in the labels (I think) but not the other stuff.
I tried other things, to get the inbox (to combine with the above) or all of the emails
var messages = GmailApp.getMessagesForThreads(GmailApp.getInboxThreads()).reduce(function(a, b) {return a.concat(b);});
CountInbox += messages.length;
but I only get about 549 results (GMail shows 5,478). If I add in the results from getPriorityInboxThreads I get 1,829 results.
I tried
// get all messages, then join them into a single dimension array
var messages = GmailApp.getMessagesForThreads(GmailApp.search("(is:unread OR is:read) in:anywhere")).reduce(function(a, b) {return a.concat(b);});
CountByLabels += messages.length;
I get 598 results.
I tried different search terms in the code directly above, eg:
is:unread = 528 results
is:read = 1,037 results
is:read OR is:unread = 599 results
None of them gave the right number, or even close, and incidentally if I try those search terms directly in gmail I get a totally different, and much higher, result for each - several thousand, or 'many'.
I don't think this is related to How to use Google App Scripts to retrieve Gmail emails in a customised way? as the numbers returned are not round numbers (eg 500).
I'm assuming that I can use getSpamThreads, getStarredThreads, getTrashThreads, getDraftMessages to get the relevant folders but until I understand why I'm only getting some emails from the inbox I don't trust those to give me everything.
Can anyone help?
Try this:
function allEmailsInLabels() {
var allLabels,i,j,L,L2,msgCount,theCount,threads,thisLabel;
msgCount = 0;
theCount = 0;
allLabels = GmailApp.getUserLabels();
L = allLabels.length;
for (i = 0; i < L; i++) {
Logger.log("label: " + allLabels[i].getName());
thisLabel = allLabels[i];
threads = thisLabel.getThreads();
//Logger.log('threads: ' + threads);
L2 = threads.length;
for (j = 0; j < L2; j++) {
msgCount = threads[j].getMessageCount();
//Logger.log('thread message count: ' + threads[j].getMessageCount());
// You could do something with threads[j] here like
// threads[j].moveToTrash();
theCount = theCount + msgCount;
};
};
//Logger.log('theCount: ' + theCount);
};
It first gets all the labels, then the threads, then the message count in each thread, and keeps a running count. You'll also need to get the messages in the inbox, that code doesn't include them. This is the sample code from the documentation that shows the basic concept:
// Log the subject lines of your Inbox
var threads = GmailApp.getInboxThreads();
for (var i = 0; i < threads.length; i++) {
Logger.log(threads[i].getFirstMessageSubject());
}
I had the same question. Reading a little bit more in the reference in the Google Developers Website, I discovered, reading about the function moveToInbox, a Google sample that used the Search to get all e-mails that weren't in the Inbox (https://developers.google.com/apps-script/reference/gmail/gmail-thread#movetoinbox). I decided to combine this with the getInboxThreads and with these two, my code was shorter and found every e-mail that I had received (less spam and junk).
function getEmails() {
var generalThreads, inboxThreads;
inboxThreads = GmailApp.getInboxThreads();
generalThreads = GmailApp.search('-in:inbox');
}
Every single email that was in the folder "All mail" in the Gmail was in these two variables after this.
I don't know if this can help anyone, but surely helped me.
I know this is coming a bit delayed, but having had the same problem and looking at some of the solutions offered here, I wanted to offer up my own solution, which also uses the search function:
function getEmails() {
var allEmailThreads = GmailApp.search('label:all')
}
This actually filters for every email, regardless of the mailbox, and seems to me to be the simplest solution to the question.
This is not an answer to your problem (but is probably one of the reasons your total results returned don't agree with what you are seeing in gmail inbox) but highlights one of the problems I encountered when calling getPriorityInboxThreads() is that it ignores any thread that is not flagged as "important" in the primary inbox.
//returns 10 threads and 1st message for each thread
function getThreads(){
var ret = '';
var threads = GmailApp.getPriorityInboxThreads(0, 10);
for (var i = 0 ; i < threads.length; i++) {
var id = threads[i].getId();
var message = GmailApp.getMessageById(id);
ret += "subject: " + message.getSubject() +'\n';
Logger.log("subject: " + message.getSubject());
/*Edited this out as it doesn't return anything
//check for labels on this thread
var labels = threads[i].getLabels();
for (var j = 0; j < labels.length; j++) {
Logger.log(labels[j].getName());
} */
}
return ret;
}
"Important" is classed as a system flag and getPriorityInboxThreads() ignores any thread that is not flagged important....
I would like to select all threads in "Primary" inbox irrespective of being labelled as "important".
To test, simply change any thread in inbox to important or not etc.
After I published a video on how to get Gmail messages into a Google spreadsheet, I received a feedback from some viewers that they could only get a number of messages but others fail to be processed. Therefore, I did some research and found that the process of getting emails may fail and make the system unable to handle the huge amount of emails. This is mentioned in the Gmail API here:
https://developers.google.com/apps-script/reference/gmail/gmail-label#getthreads
The documentation suggests to use getThreads(start, max) where start and max are the limiting parameters.
You may view the video and download the full code from YouTube and GitHub:
https://youtu.be/gdgCVqtcIw4

Google apps script: reliability issue.. setValue onformsubmit less than consistent

Hopefully this is a reasonably straight forward question.
Firstly, some context information:
I collect information from a form into a google spreadsheet to record entries to cycle races.
I use the onformsubmit trigger to run some code to do the following;
1) I check how many entries I have by rider Grade and compare these to some pre-set limits
2) Based on the pre-set limits, I work out whether the rider's entry is "provisionally entered" or "waitlisted" (I call this EntryType eg EntryType = 'waitlisted')
3) I populate some variables so as to send a confirmation email, quoting back entry details submitted by the user and advise them whether their entry is waitlisted or not.
4) I write the EntryType alongside the form submitted data in the spreadsheet so I have a record of what EntryType each rider was advised by email.
The code works fine apart from one little issue with step 4 above. Most of the time this works fine but if entries come in close together - eg a couple of seconds apart - step 4 may be left blank. The confirmation email will send, the form data will write to the spreadsheet, just that the "EntryType" will not be written to the spreadsheet alongside the form data.
I suspect that the data from the form for the next record coming in takes precedence and the write function fails without erroring? Just a guess.
Could someone offer some suggestions? I believe this code is very close, unfortunately not bullet proof as yet.
Regards, Colin
function onFormSubmit(e) {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var oGradeSubmitted = e.values[13];
var oGradeLevelSubmitted = e.values[14];
// Get the Type of Entry Limits to Apply (Handicap, Scratch or None)
var oRangeLT = ss.getRangeByName("oLimitType");
var oLimitType = oRangeLT.getValue();
//== start ===G R A D E L I M I T =================================================================================
if (oLimitType == 'Scratch') {
// Concatenate Generic Range Name & Grade Submitted to get Specific Ranges Names
var oLimitVar = "oLimit" + oGradeSubmitted; //These are now the LOWER waitlist limits
var oLimitUpper = "oLimitUpper" + oGradeSubmitted; //These are the UPPER waitlist limits
var oCountVar = "oCount" + oGradeSubmitted; //The count is based on grade submitted, not handicapping grade so as to not upset waitlist order
// Get Data from Specific Ranges
// 1) Grade Limit Data
//lower waitlist limits
var oRangeLV = ss.getRangeByName(oLimitVar);
var oLimitData = oRangeLV.getValues();
//upper waitlist limits
var oRangeLVU = ss.getRangeByName(oLimitUpper);
var oLimitDataUpper = oRangeLVU.getValues();
// 2) Grade Count Data
var oRangeCV = ss.getRangeByName(oCountVar);
var oCountData = oRangeCV.getValues();
// Write some Data into the same row as the current form submission data
var sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Form Responses");
var row = sheet.getLastRow();
if (+oCountData >= +oLimitData && +oCountData <= +oLimitDataUpper) {
sheet.getRange(row,18).setValue("Waitlisted");
var oEntryStatus = "Waitlisted";
} else {};
if (+oCountData > +oLimitDataUpper) {
sheet.getRange(row,18).setValue("Waitlisted but doubtful");
var oEntryStatus = "Waitlisted but doubtful";
} else {};
}
}
To get around the concurrency issue you could use the e.range parameter that is passed rather than getLastRow():
var row = e.range.getRow();
https://developers.google.com/apps-script/understanding_events (scroll to bottom)

How to alternate email forwarding recipients?

I'm trying to find out if it is possible to set something up to forward emails to alternating recipients.
When I receive emails, a GMail filter tags some of them, with "elephant", for example.
There are multiple people who can handle "elephant" emails, so I want to forward each new email to one of them, cycling through the list of "elephant handlers". (Joe, Amy, and Tammy, say.) Their email addresses are available in a spreadsheet.
A
1 joe#example.com
2 amy#example.com
3 tammy#example.com
Pseudo code:
Get first unread email
Forward email to Joe
Get next unread email
Forward email to Amy
Get next unread email
Forward email to Tammy
Go to start
How can I accomplish this in Google Apps Script, so that it handles all new emails as they arrive?
It's possible to do this in a number of ways. In the world of telephony, the behavior is called a Hunt Group. Here's one suggestion that borrows that concept to get you started.
Assumptions:
You're using Gmail.
The script will belong to the same account holder as the Gmail account.
Approach:
You will set up a filter in Gmail to identify incoming messages that meet your criteria - the "elephant" emails.
In this filter, you will apply a label - "elephant" - to new emails that will be used by the script to identify "work to do".
A spreadsheet will be used to contain a script that will scan for unread messages related to the "elephant" Label and forward them.
The script will be set to Trigger on a timer event at an interval that suits you.
The spreadsheet will contain a list of (properly formatted) destination email addresses; Joe, Amy, Tammy. The script will read these and use them in order.
Once an email is processed, it will be marked as "read". You could optionally un-label, relabel, archive, or trash them.
The Script
This script keeps track of which recipient will get the next forwarded message by using ScriptProperties. If you expect very large numbers of messages, you'll need to enhance it to support getting messages in batches.
Remember to change labelName appropriately.
The script does some error checking, but doesn't validate email addresses - it's possible that messages may end up failing the forwarding operation because of that. Caveat Emptor.
This is also available as a gist.
/**
* Retrieves a given user label by name and forwards unread messages
* associated with that that label to a member of the Hunt Group.
*/
function huntGroupForward() {
// get the label for given name
var labelName = "elephant"
var label = GmailApp.getUserLabelByName(labelName);
if (label == null) throw new Error("No messages for label "+labelName);
// get count of all threads in the given label
var threadCount = label.getUnreadCount();
if (threadCount == 0) return; // quick exit if nothing to do.
var threads = label.getThreads();
var messages = [];
for (var i in threads) {
if (threads[i].isUnread()) {
messages = messages.concat( threads[i].getMessages() );
}
}
for (var i = 0; i < messages.length; i++) {
if (messages[i].isUnread()) {
messages[i].forward(nextHuntGroupMember());
messages[i].markRead();
}
}
};
/*
* Global object to store working copy of the Hunt Group
*/
var huntGroup = { next : 0, members : [] };
/*
* Get the email address of the next Hunt Group Member
* to forward a message to.
*/
function nextHuntGroupMember() {
if (huntGroup.members.length == 0) {
// Load members
var sheet = SpreadsheetApp.getActiveSheet();
var data = sheet.getDataRange().getValues();
for (var i = 0; i < data.length; i++) {
huntGroup.members.push(data[i][0])
}
// Make sure we have members
if (huntGroup.members.length == 0) {
throw new Error("Found no email addresses");
}
}
// Retrieve next index. Properties are always stored as strings, so
// we need to parse the retrieved value to use it as a Number.
var next = parseInt(ScriptProperties.getProperty("nextHuntGroupMember"));
if (next != null) {
huntGroup.next = next;
}
else {
next = 0;
}
// get next member to be used
var nextMember = huntGroup.members[next];
// ... then move on to new next (increment modulo list length)
next = ++next % huntGroup.members.length;
// store the new next value
ScriptProperties.setProperty("nextHuntGroupMember", next);
return nextMember;
}
Trigger
Once you're happy with the script, set it up to run periodically. This is how you'd set it to run every hour: