I have tons of strings that I need to replace in a Google document. My performance to run my script is taking a huge hit and takes forever to run now. I am looking at how to optimize.
body.replaceText("oldvalue1",newValue1)
body.replaceText("oldvalue2",newValue2)
body.replaceText("oldvalue3",newValue3)
..
..
..
Is there a more optimal way to replace text in a Google Doc using google scripts?
As #Kos refers in the comments, the best approximation is using Google Docs API batchUpdates as Advanced Service inside Google Apps Script.
I leave you an example on how Docs API works as an Advanced Service. For the example I suppose you have an array of objects that contains the oldValue and newValue:
function batchUpdating(DocID = DocID) {
const replaceRules = [
{
toReplace: "oldValue1",
newValue: "newValue1"
},
...
]
const requestBuild = replaceRules.map(rule => {
var replaceAllTextRequest = Docs.newReplaceAllTextRequest()
replaceAllTextRequest.replaceText = rule.newValue
replaceAllTextRequest.containsText = Docs.newSubstringMatchCriteria()
replaceAllTextRequest.containsText.text = rule.toReplace
replaceAllTextRequest.containsText.matchCase = false
Logger.log(replaceAllTextRequest)
var request = Docs.newRequest()
request.replaceAllText = replaceAllTextRequest
return request
})
var batchUpdateRequest = Docs.newBatchUpdateDocumentRequest()
batchUpdateRequest.requests = requestBuild
var result = Docs.Documents.batchUpdate(batchUpdateRequest, DocID)
Logger.log(result)
}
Google Apps Script helps us handle the authorization flow, and gives us hints on how to construct our request. For example, Docs.newReplaceAllTextRequest(), helps us build the request for that service, giving us hints that it contains replaceText and containText. In any case, we could also supply the object directly to the request:
const requestBuild = replaceRules.map(rule => {
return {
replaceAllText:
{
replaceText: rule.newValue,
containsText: { text: rule.oldValue, matchCase: false }
}
}
})
To take in account
Each request is validated before being applied. If any request is not valid, then the entire request will fail and nothing will be applied.
If your script project uses a default GCP project created on or after April 8, 2019, the API is enabled automatically after you enable the advanced service and save the script project. If you have not done so already, you may also be asked to agree to the Google Cloud Platform and Google APIs Terms of Service as well.
Documentation:
ReplaceAllTextRequest
SubstringMatchCriteria
Array.forEach(obj => { body.replaceText(obj.old,obj.new)})
#kos way is probably a better way to go.
Related
I am in a similar situation to the OP of this post:
User access request when GAS run as the user
I need to run a web app as an 'active user', allow this user to access Drive, Docs, and Sheets resources, but not having the user direct access to them.
However my knowledge is much less on the subject.
As I understand it, I need to create a service account so that the script running as the 'active user' can access Drive, Sheet, and Docs resources that the active user does not have access to.
I am also looking at other resources as well as Google's documentation, but it's a bit overwhelming.
Can anyone explain the basics for this? Maybe a tutorial (or a link to such) that really inexperienced users can understand? I just need to get started on the right direction.
Thank you in advance!
Impersonation of users using App Script
It should be possible to generate a key and start the process of impersonation and call off the scopes and API.
function getJWT(sub) {
var header = { "alg": "RS256", "typ": "JWT" }
var encodedheader = Utilities.base64EncodeWebSafe(JSON.stringify(header))
var key = "-----BEGIN PRIVATE KEY----- fjsklfjl;sdjfasd -----END PRIVATE KEY-----\n"
var time = Math.floor(new Date().getTime() / 1000)
var claim = {
"iss": "yourserviceaccount#mail-p-any.iam.gserviceaccount.com",
"scope": "https://mail.google.com/",
"aud": "https://oauth2.googleapis.com/token",
"iat": time,
"exp": time + 3600,
"sub": sub[0]
}
var encodedclaim = Utilities.base64EncodeWebSafe(JSON.stringify(claim))
var input = encodedheader + "." + encodedclaim
var signed = Utilities.computeRsaSha256Signature(input, key)
var base64signed = Utilities.base64Encode(signed)
var jwt = encodedheader + "." + encodedclaim + "." + base64signed
return jwt
}
function getAccessToken(user) {
var payload = {
"grant_type": "urn:ietf:params:oauth:grant-type:jwt-bearer",
"assertion": getJWT(user)
}
var params = {
"method": "POST",
"contentType": "application/x-www-form-urlencoded",
"payload": payload,
"muteHttpExceptions": true
}
var response = UrlFetchApp.fetch("https://oauth2.googleapis.com/token", params)
var output = JSON.parse(response.getContentText())
console.log(output.access_token)
return output.access_token
}
You can also review the library and step by step process on how you can implement it in another way from here:
https://github.com/googleworkspace/apps-script-oauth2
My code sample was based on the sample script from:
https://developers.google.com/identity/protocols/oauth2/service-account#delegatingauthority
You can also review the other sample code from the references below.
This way you are able to impersonate the user and run or make calls on behalf of the user from your organization without having access to it. This might be where you can start your idea on how to start.
References
https://cloud.google.com/iam/docs/impersonating-service-accounts
https://github.com/googleworkspace/apps-script-oauth2/blob/main/samples/GoogleServiceAccount.gs
I got this to work, for the benefit of those who are the same level in this subject as I am, and in the similar situation. Anyone please expound or correct me if I'm wrong, thanks.
You cannot use the methods to access Drive, Docs, and Sheets in the
same code that runs as the 'active user'.
You have to access these Google services using the equivalent HTTP
API calls of the methods.
The HTTP API calls need a user that would interact with the resources
(because it's being called from publicly from the internet and not
from the script).
You create a service account for this. This acts as the user for the
calls.
I started with Ricardo Jose Velasquez Cruz's response, and found other resources, as I was calling the API from Apps Script.
https://medium.com/geekculture/how-to-use-service-accounts-and-oauth2-in-google-apps-script-99c4bc91dc31
Note that Apps Script requires an OAUTH2 library to connect, not sure why this was not built-in to GAS itself:
https://github.com/googleworkspace/apps-script-oauth2
How to create a service account and use it to access Google Drive (you use the same code to access Docs and Sheet as well, you just need to use the corresponding URL and parameters for the services):
https://www.labnol.org/code/20375-service-accounts-google-apps-script
it's basically the same code as another post I found here:
Google Service Accounts / API - I keep getting the Error: Access not granted or expired. (line 454, file "Service")
Hope this helps :)
I am trying to scrape very small information from a webpage using Cheerio and Google Apps Script.
I want to get the performance number from this webpage:
Following is the code snippet that I am using to get it:
function LinkResult(){
var url ='https://pagespeed.web.dev/report?url=http%3A%2F%2Fwww.juicecoldpressed.com%2F';
var result = UrlFetchApp.fetch(url);
var content = Cheerio.load(result.getContentText())
var item = content(".lh-gauge__percentage").text()
Logger.log(item)
}
As I run, this code does not show any output in the variable item. Surely there is something which I am missing, can you please guide me? Thank you.
Issue and workaround:
In this case, I'm worried that your goal might not be able to be directly achieved using the URL of https://pagespeed.web.dev/report?url=http%3A%2F%2Fwww.juicecoldpressed.com%2F and Cheerio. Because the HTML data retrieved from UrlFetchApp.fetch(url) is different from that on the browser. And, it seems that the value is calculated using a script.
Fortunately, in your situation, I thought that your values can be retrieved using PageSpeed Insights API. In this answer, I would like to propose achieving your goal using PageSpeed Insights API.
Usage:
1. Get Started with the PageSpeed Insights API.
Please check the official document for using PageSpeed Insights API. In this case, it is required to use your API key. And, please enable PageSpeed Insights API at the API console.
2. Sample script.
function myFunction() {
const apiKey = "###"; // Please set your API key.
const url = "http://www.juicecoldpressed.com/"; // Please set URL.
const apiEndpoint = `https://www.googleapis.com/pagespeedonline/v5/runPagespeed?key=${apiKey}&url=${encodeURIComponent(url)}&category=performance`;
const strategy = ["desktop", "mobile"];
const res = UrlFetchApp.fetchAll(strategy.map(e => ({ url: `${apiEndpoint}&strategy=${e}`, muteHttpExceptions: true })));
const values = res.reduce((o, r, i) => {
if (r.getResponseCode() == 200) {
const obj = JSON.parse(r.getContentText());
o[strategy[i]] = obj.lighthouseResult.categories.performance.score * 100;
} else {
o[strategy[i]] = null;
}
return o;
}, {});
console.log(values);
}
3. Testing.
When this script is run, you can see the returned value of { desktop: ##, mobile: ## } at the log. The values (the unit is %.) of desktop and mobile are the values for the desktop and the mobile, respectively.
Reference:
Get Started with the PageSpeed Insights API
as per the subject I would need to synchronize the contents of some Google drive folders with the same ones located in a shared remote server(web site) . Normally on the server itself, for internal things, I use a cron. But in this case I don't know what the correct solution is. I took a look online but, despite having an idea, I am not sure whether to build specific bees or if there is a more functional way. Thanks to those who have the patience to answer.
What about creating a Google Apps Script Web app deployment, and using it as a REST API?
These are the steps to follow:
Create a new Google Apps Script project.
Create a Web App that serves the content as a JSON. In my example I will list the folders of Google Drive, but you can use it as you wish. In this example I am implementing a simple API key system for preventing unwanted access.
Code.gs
const getDriveFolders = () => {
const listFolders = []
const folderIterator = DriveApp.getFolders()
while (folderIterator.hasNext()) {
listFolders.push(folderIterator.next().getName())
}
return listFolders
}
const errMsgRes = ContentService
.createTextOutput()
.append(JSON.stringify({ "msg": "INVALID API KEY" }))
.setMimeType(MimeType.JSON)
const succMsgRes = ContentService
.createTextOutput()
.setContent(JSON.stringify(getDriveFolders(), null, 2))
.setMimeType(MimeType.JSON)
const doGet = (e) => {
return e.parameter.api_key != "123123"
? errMsgRes
: succMsgRes
}
Click Deploy > Select type > Wep app. In the Who has acces field I put it Anyone. Grab the URL for later.
In your web site, you only need to fetch the data.
index.html
<div id="drive-content" ></div>
<script>
const GASURL = "<DEPLOYMENT_URL>";
const container = document.getElementById("drive-content");
fetch(GASURL + "?api_key=123123")
.then((d) => d.json())
.then((d) => {
console.log(d);
container.innerHTML += d.join("<br />");
});
</script>
This way, every time you enter your web site, the content will be updated.
Documentation
ContentService
DriveApp
doGet Request parameters
I'd like to completely undo any of Gmails built in category labels. This was my attempt.
function removeBuiltInLabels() {
var updatesLabel = GmailApp.getUserLabelByName("updates");
var socialLabel = GmailApp.getUserLabelByName("social");
var forumsLabel = GmailApp.getUserLabelByName("forums");
var promotionsLabel = GmailApp.getUserLabelByName("promotions");
var inboxThreads = GmailApp.search('in:inbox');
for (var i = 0; i < inboxThreads.length; i++) {
updatesLabel.removeFromThreads(inboxThreads[i]);
socialLabel.removeFromThreads(inboxThreads[i]);
forumsLabel.removeFromThreads(inboxThreads[i]);
promotionsLabel.removeFromThreads(inboxThreads[i]);
}
}
However, this throws....
TypeError: Cannot call method "removeFromThreads" of null.
It seems you can't access the built in labels in this way even though you can successfully search for label:updates in the Gmail search box and get the correct results.
The question...
How do you access the built in Gmail Category labels in Google Apps Script and remove them from an email/thread/threads?
Thanks.
'INBOX' and other system labels like 'CATEGORY_SOCIAL' can be removed using Advanced Gmail Service. In the Script Editor, go to Resources -> Advanced Google services and enable the Gmail service.
More details about naming conventions for system labels in Gmail can be found here Gmail API - Managing Labels
Retrieve the threads labeled with 'CATEGORY_SOCIAL' by calling the list() method of the threads collection:
var threads = Gmail.Users.Threads.list("me", {labels: ["CATEGORY_SOCIAL"]});
var threads = threads.threads;
var nextPageToken = threads.nextPageToken;
Note that you are going to need to store the 'nextPageToken' to iterate over the entire collection of threads. See this answer.
When you get all thread ids, you can call the 'modify()' method of the Threads collection on them:
threads.forEach(function(thread){
var resource = {
"addLabelIds": [],
"removeLabelIds":["CATEGORY_SOCIAL"]
};
Gmail.Users.Threads.modify(resource, "me", threadId);
});
If you have lots of threads in your inbox, you may still need to call the 'modify()' method several times and save state between calls.
Anton's answer is great. I marked it as accepted because it lead directly to the version I'm using.
This function lets you define any valid gmail search to isolate messages and enables batch removal labels.
function removeLabelsFromMessages(query, labelsToRemove) {
var foundThreads = Gmail.Users.Threads.list('me', {'q': query}).threads
if (foundThreads) {
foundThreads.forEach(function (thread) {
Gmail.Users.Threads.modify({removeLabelIds: labelsToRemove}, 'me', thread.id);
});
}
}
I call it via the one minute script trigger like this.
function ProcessInbox() {
removeLabelsFromMessages(
'label:updates OR label:social OR label:forums OR label:promotions',
['CATEGORY_UPDATES', 'CATEGORY_SOCIAL', 'CATEGORY_FORUMS', 'CATEGORY_PROMOTIONS']
)
<...other_stuff_to_process...>
}
My Goal: Changes in Google Drive => Push Notification to https://script.google.com/a/macros/my-domain/... => App is pushed to take action.
I don't want to setup an middle Webhook agent for receiving notification. Instead, let the Web App (by Google Script) to receive it and be pushed directly.
Since the relevant function is quite undocumented (just here: https://developers.google.com/drive/web/push) , below is the code I tried but failure.
1. Is above idea feasible??
2. My code doPost(R) seems cannot receive notification (R parameter) properly. Anyway, no response after I change the Google Drive. Any problem? (I have tried to log the input parameter R so as to see its real structure and decide if the parameter Obj for OAuth is the same as normal Drive App, but error occur before log)
function SetWatchByOnce(){
var Channel = {
'address': 'https://script.google.com/a/macros/my-domain/.../exec',
'type': 'web_hook',
'id': 'my-UUID'
};
var Result = Drive.Changes.watch(Channel);
...
}
function doPost(R) {
var SysEmail = "My Email";
MailApp.sendEmail(SysEmail, 'Testing ', 'Successfully to received Push Notification');
var Response = JSON.parse(R.parameters);
if (Response.kind == "drive#add") {
var FileId = Response.fileId;
MyFile = DriveApp.getFolderById(FileId);
...
}
}
function doGet(e) {
var HTMLToOutput;
var SysEmail = "My Email";
if (e.parameters.kind) {
//I think this part is not needed, since Push Notification by Drive is via Post, not Get. I should use onPost() to receive it. Right?
} else if (e.parameters.code) {
getAndStoreAccessToken(e.parameters.code);
HTMLToOutput = '<html><h1>App is successfully installed.</h1></html>';
} else { //we are starting from scratch or resetting
HTMLToOutput = "<html><h1>Install this App now...!</h1><a href='" + getURLForAuthorization() + "'>click here to start</a></html>";
}
return HtmlService.createHtmlOutput(HTMLToOutput);
}
....
Cloud Functions HTTP trigger(s) might also be an option ...
(which not yet existed at time of this question). this just requires setting the trigger URL as the notification URL, in the Google Drive settings - and adding some NodeJS code for the trigger; whatever it shall do. one can eg. send emails and/or FCM push notifications alike that. that trigger could also be triggered from App Script, with UrlFetchApp and there is the App Script API. one can have several triggers, which are performing different tasks (App Script is only one possibilty).
Cicada,
We have done similar functions to receive webhooks/API calls many times. Notes:
to get R, you need: var Response = R.parameters and then you can do Response.kind, Response.id, etc.
Logger will not work with doGet() and doPost(). I set it up a write to spreadsheet -- before any serious code. That way I know if it is getting triggered.