My point is to get some data from a html page (text and tables). However, when I am using the different functions in Google Sheets (importxml; importhtml); those ones are not working ("Could not fetch the URL"). I tried many alternative methods using google sheets by none are working due to website type (I guess).
In this way, as alternative option, I want to know if there is a specific code line(s) in Google Apps Script to copy values from open web page (opened in a Google Chrome tab) in order to paste those values in Google Sheets tab (opened in another tab of Google Chrome). I think this method will avoid to open the website & consequently the same error as import functions in Google Sheets.
As additional comments the UrlFetch is not permitted by my admin
As your admin doesn't allow the use of UrlFetch it's not possible because Google Apps Script is not able access the opened tabs of your web browser.
Reference
https://developers.google.com/apps-script/
Apps Script is just javascript so you could use a web data API like https://rapidapi.com/fusioncow11/api/web-data1
and you could implement it with something like:
const options = {
method: 'GET',
headers: {
data: 'https://www.google.com/',
style: 'html',
'X-RapidAPI-Key': 'keyhere',
'X-RapidAPI-Host': 'web-data1.p.rapidapi.com'
}
};
fetch('https://web-data1.p.rapidapi.com/', options)
.then(response => response.json())
.then(response => console.log(response))
.catch(err => console.error(err));
Related
I had written an html form in Apps Script, with JS and triggers and published it as a web app, then I have taken published script link for web app and used it in my embed HTML form in my website Google Site.
Till now, results are going well
When I enter Data to the input fields, data transfer successfully to my sheet. But the fields still filled with entered data, and the registrar can't recognize if the registration has been done successfully.
This is my code:
<form id="submit-to-google-sheet">....</form>
const scriptURL = 'https://script.google.com/macros/s/AKfycbwmPn259sS2-hLzY4Jg-KFLstdJKTpf89pqD4dDUn0tvcPBrvc9YK03G8BeeeH7IFnb/exec'
const form = document.forms['submit-to-google-sheet']
form.addEventListener('submit', e => {
e.preventDefault()
fetch(scriptURL, { method: 'POST', body: new FormData(form)})
.then(response => console.log('Success!', response))
.catch(error => console.error('Error!', error.message))
})
I have tried to add redirecting code inside the function which is :
windows.location("https:www.alhuqebi.com");
and it redirects me when i was testing (offline file) but when I copied the code and pasted it to the website, it didn't send data to the sheet or even redirect to the new page.
I have tried also another solution, so I removed the redirection script line and added the reset code as following:
document.getElementById('submit-to-google-sheet').reset();
and when I clicked the submit button, it reset the form without sending data to the sheet.
So according to the previous explanation, how can I :
make the form sending data to the sheet, with telling the customer that registration has been done successfully (whether redirect to thank you page nor resetting the form)??
I have a Google apps script project which I use as a web application to save some data gathered from a web page (JavaScript only) to My Google sheets (think of it as a simple database for just me).
It's no need to use auth for anyone else other than my account, because, I don't use their accounts/data for anything at all. I just need to use my account only, so, when I deploy it, I make it execute as me, and accessible to anyone:
.
When I click deploy after previous screen, it asks for my permission (the consent dialogue) to access my account's data, which I grant it, and after that everything is good, and the HTTP requests to this script works just fine.
.
The problem is:
This authentication expires after maybe 7 days (I'm not sure) and the script stops working, I discover that when the HTTP requests to it return error 403
To fix that and make it work again, I need to run the script again from the Google apps script editor which asks for the permissions (consent) again:
.
I can't use it like that and the web page stop working when the authentication gets revoked!
I'm not publishing the script (I don't want/don't need to). Do I?
My question is, how can I add the authentication in a way that makes it lasts and stops asking me for it over and over again?
The script on Google apps script works like this:
function doPost(request) {
return checkRequest(request);
}
function checkRequest(request) {
//check the request & save the sent data to a google sheet here;
//...
return sendResponse({
success: true,
data: {result: 'Saved the data!' }
});
}
function sendResponse(response) {
return ContentService
.createTextOutput(JSON.stringify(response))
.setMimeType(ContentService.MimeType.JSON);
}
And I call it from the web page using Ajax HTTP POST request, like this:
jQuery.ajax({
url: 'https://script.google.com/macros/s/{script-id}/exec',
method: 'POST',
dataType: "json",
data: {key: 'value'},
success: function (response) {
console.log(response);
},
error: function (response) {
console.error(response);
}
});
And this is the response the script returns after few days when the authentication expires:
This has been reported to Google
There is already a report on Google's Issue Tracker which detail the same kind of behaviour:
Random Deauthorizations for script since Editor Update with no changes to code
Google does seem to know about this issue. From the issue tracker link, a response was given:
[...] reviewing the documentation, it reads:
Authorizations by a test user will expire seven days from the time of consent.
Source
So I'm not sure we should expect anything different from these tests.
Also re-reading this thread, in your first comment you said that this was affecting projects that already published. Though I understand that you fixed the original projects that were having issues by un-linking the GCP projects.
A possible fix would be filling the scopes manually in the manifest for these types of issues. This is because Apps Script tries to infer the scopes needed at runtime, and sometimes this can result in Apps Script trying to gain more permissive scope than what is authorized.
See Setting Explicit Scopes
However, token expiry in around 7 days is to be expected for projects in "testing" state.
Which seems to be the case for the users in the report thread.
There are multiple reasons that this expiration may occur, as explained in the Using OAuth 2.0 to Access Google APIs page.
That being said, if it's causing problems you can file your own bug about it here in the Google Apps Script component.
References:
Random Deauthorizations for script since Editor Update with no changes to code
Using OAuth 2.0 to Access Google APIs | Google Identity
Setting up your OAuth consent screen - Google Cloud Platform Console Help
Authorization Scopes - Setting explicit scopes | Apps Script | Google Developers
I would like to create a Google Sheet in the Google Drive App Data folder (API docs at https://developers.google.com/drive/api/v3/appdata) using client side JS.
I can create non-Sheet files like this (after handling all the authentication stuff, as per the browser quickstart for the Sheets API):
gapi.client.drive.files.create({
resource: {
name: 'myfile.txt',
parents: ['appDataFolder'],
mimeType: 'text/plain'
},
fields: 'id'
}).then(res => {
console.log("Created OK: response was ", res);
}).catch(e => {
console.log("Creation failed: error was ", e)
})
However, if I use mimeType: 'application/vnd.google-apps.spreadsheet' then I get this error back:
reason: "notSupportedForAppDataFolderFiles",
message: "Method not supported for files within the Application Data folder."
This error message seems to be entirely undocumented. My suspicion is that this is just not allowed: you can't store Google Sheets in the App Data folder, probably because Sheets aren't really stored as files in Drive at all and they appear so through some sort of UI fakeout by the Drive team. But I have no confirmation of this.
The reason I'd like to do this is that an app which requests access to the App Data folder does not need to request access to all other files. If I can't put a Sheet in the App Data folder, then as far as I'm aware, in order to create a spreadsheet my app will need to request complete access to all the user's spreadsheets, which it completely does not need; it will use its own one and that's it. I don't want to ask for that access because users will (correctly) see that as massive overreach.
The documentation specifies:
The application data folder is automatically created when you attempt to create a file in it.
Use this folder to store any files that the user shouldn't directly interact with.
This folder is only accessible by your application and its contents are hidden from the user and from other Drive apps.
So while there is no direct information about Google sheets not being allowed in the application data folder, one can assume that a Google Sheets file does not meet the mentioned criteria.
As for scopes:
You can use the scope https://www.googleapis.com/auth/drive.file.
This is a narrow scope that gives the application only access to files that have been created by this application, see here
By using this scope, you avoid asking for access to all users files, and thus the users do not need to worry about providing this scopes to the application.
I've been trying to scrape a publicly-shared photo album I have on Google Photos: sharing the album provides a link along the format of photos.app.goo.gl/{SOME_ID}. The goal is to be able to retrieve the individual photo URLs (the URLs that don't expire, following the format lh3.googleusercontent.com and which can then be embedded onto any other website within an tag).
I would like to do this in Google Apps Script and have tried the following:
var response = UrlFetchApp.fetch("https://photos.app.goo.gl/{SOME_ID}");
Logger.log(response.getContentText());
However, the response doesn't display any of the images as if the page would instead need to be loaded some more, even though I'm testing this on an album with just 2 photos. If I inspect the page manually, I can clearly see the images + image links along the lh3.googleusercontent.com format. What should I change to my fetch request?
I've seen other implementations (outside of Apps Script) using Axios that managed to get the URLs I want, but haven't found a way of importing the Axios library into Apps Script.
Answer:
You can use the Google Photos API in Apps Script to get the individual photo URLs of a shared album.
More Information:
UrlFetchApp.fetch() returns an HTTPResponse object within Apps Script which contains Headers, HTML Content and other information such as the HTTP response code, as if the page was being fetched and loaded via a browser. There is also a set of URL Fetch limits as detailed on the Quotas for Google Services page which result in you getting a truncated response for sufficiently large pages.
The Photos API however has methods which are specifically designed for the purpose you are describing, and this data can be retrieved from the mediaItems REST resource. Bear in mind however, that this returns all photos and not ones that are in a specific album; further processing would need to be done from there.
Example Code:
After creating a new project in the Developers Console, you need to enable the Photos API from the APIs & Services > Library menu item, and then link it to your Apps Script Project by going to the script UI and following Resources > Cloud Platform project. Add the project number of the project just created in the Developer's Console and press Set Project.
Here is a small code snippet which will return photos from your Google Photos account, and log the individual photo URLs in the Logger:
function logPhotoUrls() {
var url = 'https://photoslibrary.googleapis.com/v1/mediaItems';
var options = {
headers: {
Authorization: 'Bearer ' + ScriptApp.getOAuthToken()
},
method: 'get',
muteHttpExceptions: false
};
var response = UrlFetchApp.fetch(url, options);
for (var i = 0; i < 10; i++) {
Logger.log(JSON.parse(response.getContentText()).mediaItems[i].productUrl);
}
}
You will also need to edit your appscript.json, which you can see by following View > Show manifest file to include the following:
{
"oauthScopes": ["https://www.googleapis.com/auth/photoslibrary.readonly",
"https://www.googleapis.com/auth/script.external_request"]
}
References:
Google Apps Script - Quotas for Google Services
Google Photos Library API
Google Photos API - Get started with REST
Google Photos API - Method mediaItems:list
Google Developers Console
Google Apps Script - Class UrlFetchApp
Google Apps Script - Manifest structure
I have a master spreadsheet which contains a script that makes a POST request to my server onEdit with the current spreadsheet ID.
function post2Server(){
ss = SpreadsheetApp.getActiveSpreadsheet();
payload = {};
payload['spreadsheet_id'] = ss.getId();
headers = {
'Content-Type': 'application/json',
'Accept' : 'application/json'
}
options = {
'method' : 'post',
'contentType': 'application/json',
'headers' : headers,
'payload' : JSON.stringify(payload)
}
res = UrlFetchApp.fetch(MY_SERVER_URL, options);
return;
}
This function works as intended on the master sheet. Now, when I use the python google API to create a copy of this master sheet, the script copies over, however, doesn't run. I get an error saying....
Server error occurred. Please try saving the project again.
Why isn't this running? Within the copy of the spreadsheet, I even create a new function which simply logs "hello" and receive the same error. It appears that after the Python SDK copied the master sheet, no functions run. Is this a permissions issue? How can I get the script to execute in any subsequent copy of the master sheet?
Usually my approach to this kind of requirements is to use the scripts.run method of the Google Apps Script REST API. In this way you are in full control of what scripts are executed and of the parameters.
The main conditions you have to met to use this method are the following:
Deploy the script project as an API executable
Provide a properly scoped OAuth token for the execution
Ensure that the script and the calling application share a common Cloud Platform project
If you look for more details there is a dedicated page in the documentation that explains how to run a specific method of your Apps Script and at the bottom of it you have a sample in many different language, including python.
Right now copying a sheet that contains a bound Google App Script using the Drive API authenticated through a Google Service Account will always render the script unusable due to a bug in the Drive API (filed here). Until that's fixed, there's no simple solution to this.
In our case, we were able to eliminate our use of the bound script by utilizing functions in Sheets, specifically the IMPORTDATA function as a means of sending web requests.