hoping for some support on our Google Drive API. We have a script that creates Google Drive folders when new clients are added. This script worked great for over 6 months, and suddenly broke. The error we get is:
Error refreshing the OAuth2 token, message: '{ "error" : "invalid_grant" }'
I'd like to stress that nothing in our code changed. The tokens simply stopped authenticating for all users.
We are going mad trying to figure out why this would "randomly" happen one day. Any help is greatly appreciated.
try {
require '/var/www/html/s2/googledrive/vendor/autoload.php';
define('APPLICATION_NAME', 'Drive API Quickstart');
define('CREDENTIALS_PATH', '~/.credentials/dirrec.json');
define('CLIENT_SECRET_PATH', '/var/www/html/client_secret.json');
define('SCOPES', 'https://www.googleapis.com/auth/drive');
$client = new Google_Client();
$client->setApplicationName(APPLICATION_NAME);
$client->setScopes(SCOPES);
$client->setAuthConfigFile(CLIENT_SECRET_PATH);
$client->setAccessType('offline');
// Load previously authorized credentials from a file.
require_once('include/upload_file.php');
$file = new UploadFile();
//get the file location
$file->temp_file_location = 'custom/modules/Accounts/dirrec.json';
$accessToken = $file->get_file_contents();
$client->setAccessToken($accessToken);
$service = new Google_Service_Drive($client);
// Refresh the token if it's expired.
if ($client->isAccessTokenExpired()) {
$client->refreshToken($client->getRefreshToken());
file_put_contents('custom/modules/Accounts/dirrec.json', $client->getAccessToken());
}
Related
I am in a similar situation to the OP of this post:
User access request when GAS run as the user
I need to run a web app as an 'active user', allow this user to access Drive, Docs, and Sheets resources, but not having the user direct access to them.
However my knowledge is much less on the subject.
As I understand it, I need to create a service account so that the script running as the 'active user' can access Drive, Sheet, and Docs resources that the active user does not have access to.
I am also looking at other resources as well as Google's documentation, but it's a bit overwhelming.
Can anyone explain the basics for this? Maybe a tutorial (or a link to such) that really inexperienced users can understand? I just need to get started on the right direction.
Thank you in advance!
Impersonation of users using App Script
It should be possible to generate a key and start the process of impersonation and call off the scopes and API.
function getJWT(sub) {
var header = { "alg": "RS256", "typ": "JWT" }
var encodedheader = Utilities.base64EncodeWebSafe(JSON.stringify(header))
var key = "-----BEGIN PRIVATE KEY----- fjsklfjl;sdjfasd -----END PRIVATE KEY-----\n"
var time = Math.floor(new Date().getTime() / 1000)
var claim = {
"iss": "yourserviceaccount#mail-p-any.iam.gserviceaccount.com",
"scope": "https://mail.google.com/",
"aud": "https://oauth2.googleapis.com/token",
"iat": time,
"exp": time + 3600,
"sub": sub[0]
}
var encodedclaim = Utilities.base64EncodeWebSafe(JSON.stringify(claim))
var input = encodedheader + "." + encodedclaim
var signed = Utilities.computeRsaSha256Signature(input, key)
var base64signed = Utilities.base64Encode(signed)
var jwt = encodedheader + "." + encodedclaim + "." + base64signed
return jwt
}
function getAccessToken(user) {
var payload = {
"grant_type": "urn:ietf:params:oauth:grant-type:jwt-bearer",
"assertion": getJWT(user)
}
var params = {
"method": "POST",
"contentType": "application/x-www-form-urlencoded",
"payload": payload,
"muteHttpExceptions": true
}
var response = UrlFetchApp.fetch("https://oauth2.googleapis.com/token", params)
var output = JSON.parse(response.getContentText())
console.log(output.access_token)
return output.access_token
}
You can also review the library and step by step process on how you can implement it in another way from here:
https://github.com/googleworkspace/apps-script-oauth2
My code sample was based on the sample script from:
https://developers.google.com/identity/protocols/oauth2/service-account#delegatingauthority
You can also review the other sample code from the references below.
This way you are able to impersonate the user and run or make calls on behalf of the user from your organization without having access to it. This might be where you can start your idea on how to start.
References
https://cloud.google.com/iam/docs/impersonating-service-accounts
https://github.com/googleworkspace/apps-script-oauth2/blob/main/samples/GoogleServiceAccount.gs
I got this to work, for the benefit of those who are the same level in this subject as I am, and in the similar situation. Anyone please expound or correct me if I'm wrong, thanks.
You cannot use the methods to access Drive, Docs, and Sheets in the
same code that runs as the 'active user'.
You have to access these Google services using the equivalent HTTP
API calls of the methods.
The HTTP API calls need a user that would interact with the resources
(because it's being called from publicly from the internet and not
from the script).
You create a service account for this. This acts as the user for the
calls.
I started with Ricardo Jose Velasquez Cruz's response, and found other resources, as I was calling the API from Apps Script.
https://medium.com/geekculture/how-to-use-service-accounts-and-oauth2-in-google-apps-script-99c4bc91dc31
Note that Apps Script requires an OAUTH2 library to connect, not sure why this was not built-in to GAS itself:
https://github.com/googleworkspace/apps-script-oauth2
How to create a service account and use it to access Google Drive (you use the same code to access Docs and Sheet as well, you just need to use the corresponding URL and parameters for the services):
https://www.labnol.org/code/20375-service-accounts-google-apps-script
it's basically the same code as another post I found here:
Google Service Accounts / API - I keep getting the Error: Access not granted or expired. (line 454, file "Service")
Hope this helps :)
I'm creating a spreadsheet file using google drive api, then pass ownership to another email, $automationEmail in this case. This is how I do it:
$permission = $this->insertPermission($service, $file->id, $automationEmail, 'user', 'owner');
And this is the insertPermission function
function insertPermission($service, $fileId, $value, $type, $role)
{
$newPermission = new Google_Service_Drive_Permission();
$newPermission->setEmailAddress($value);
$newPermission->setType($type);
$newPermission->setRole($role);
if ($role == 'owner') {
$permission = $service->permissions->create($fileId, $newPermission, array('fields' => 'id', 'transferOwnership' => 'true'));
} else {
$permission = $service->permissions->create($fileId, $newPermission);
}
if ($permission) {
return $permission;
}
return NULL;
}
Until now, this used to work perfectly fine. But recently it stopped working and gives me an error: Consent is required to transfer ownership of a file to another user.
For other permissions,like writer or reader, everything works fine. Did anything change in the API that I could not find? I searched for the solution, but seems like noone has had this issue before, or at least I could not find anything. Why did it stop working?
Issue:
Currently, when transfering file ownership between users who are not part of the same organization, the transfer requires the new owner to accept the invitation.
When the current owner (user #1) sets user #2 as the new owner, an invitation is sent to user #2, and user #1 remains the owner until user #2 accepts it:
You remain the file owner until the pending owner accepts your invitation.
Since this requires the new owner to accept the invitation, the transfer cannot be done programmatically via API, so this method fails for users who don't belong to the same organization.
From Issue Tracker https://issuetracker.google.com/issues/227973724#comment22 :
Following up here, this is the expected behavior as currently Drive does not support the changing of the ownership for items which are owned by gmail.com accounts.
Issue Tracker feature request:
API: Consent is required to transfer ownership of a file to another user
Reference:
Make someone else the owner of your file
Transferring ownership directly so someone did not work, so I used the following approach: In our google workspace we enabled Domain-wide delegation for the Google service account:
https://developers.google.com/admin-sdk/directory/v1/guides/delegation#delegate_domain-wide_authority_to_your_service_account
Then I created a dedicated user for Google drive files, logged in as this user to get the access token and saved it. After that I was able to create the new files on Google Drive as this dedicated user, so the files now belong to this email by default. This is how I did it:
$credentials = storage_path('credentials.json');
$token = [
'access_token' => config('app.flex_email_access_token')
];
$client = new Google_Client();
$client->setApplicationName('Sheets');
$client->setScopes([Google_Service_Sheets::DRIVE, Google_Service_Sheets::SPREADSHEETS]);
$client->setSubject(config('app.flex_email'));
$client->setAccessType('offline');
$client->setAuthConfig($credentials);
$client->setAccessToken($token);
$serviceDrive = new Google_Service_Drive($client);
$body = new Google_Service_Drive_DriveFile(array(
"name" => [$request->name],
'parents' => [$listing->pls_folder_id]
));
$spreadsheet = $serviceDrive->files->copy('yourspreadsheetid', $body);
In this example I copy the file, but the same approach works when creating them. Since the files now belong to the user I need by default, the ownership transfer issue has been avoided.
I've recently run into an issue authorising a new Google App Script project, specifically one using the Cloud SQL admin API.
The same code exists in previously authorised GAS projects and works fine, but if I take a copy of the GAS project and try to run a function for the first time I'm unable to complete the authorisation process. The screens I'm going through are listed below:
Authorisation Required. - clicked "Review Permissions"
Choose an account to authorise the Google project. - clicked my account
This app isn't verified! - clicked "Go to project
(unsafe)"
Google project wants access to this list of scopes.- clicked "Allow"
Authorisation is required to perform that action.
The warning screen (3) is a recent addition to the process. I don't rememeber encountering it when I've created and run new projects earlier this year. I'm wondering if Google has made any changes to their security implementation of OAuth2.0 recently.
Also, this issue only seems to affect REST calls to the Cloud SQL admin API. In the same project mentioned above I am able to run functions which write data to BigQuery tables in the same Google project which is also hosting the Cloud SQL instances. Clearly some scopes and code can be made to work.
The "https://www.googleapis.com/auth/sqlservice.admin" scope is included in the list of those I requested and approve. I even tried manually editing the URL to add more scopes being requested and it still doesn't get me passed the "Authorisation is required to perform that action" screen.
Does anyone have any idea's?
EDIT:
The code in question which is triggering the authentication.
// Function to get the ip address of a given CloudSQL instance
function _getInstanceIpAddress_(projectId, sqlInstance) {
var token = _getAuthenticationToken_();
// Create the header authorisation
var headers = {
"Authorization": "Bearer " + token
};
// Create the Cloud SQL instances get parameters
var parameters = {
"method": "get",
"headers": headers,
"instance": sqlInstance,
"project": projectId,
"muteHttpExceptions": true
};
// Create the url of the sql instances get API
var api = "https://www.googleapis.com/sql/v1beta4/projects/" + projectId + "/instances/" + sqlInstance + "?fields=ipAddresses";
try {
// Use the url fetch service to issue the https request and capture the response
var response = UrlFetchApp.fetch(api, parameters);
// Extract the ip address of the instance from the response
var content = JSON.parse(response.getContentText());
return content.ipAddresses[0].ipAddress;
} catch(err) {
_log_('ERROR', 'Getting ' + sqlInstance + ' instance ip address failed: ' + err);
return null;
}
}
function _getAuthenticationToken_() {
// Check we have access to the service
var service = getService();
if (!service.hasAccess()) {
var authorizationUrl = service.getAuthorizationUrl();
_log_('INFO', 'Open the following URL and re-run the script: ' + authorizationUrl);
return;
}
Logger.log('Passed Authentication');
//Get the Access Token
return service.getAccessToken();
function getService() {
// Create a new service with the given name. The name will be used when
// persisting the authorized token, so ensure it is unique within the
// scope of the property store.
return OAuth2.createService('companyName-dev-service')
// Set the endpoint URLs, which are the same for all Google services.
.setAuthorizationBaseUrl('https://accounts.google.com/o/oauth2/auth')
.setTokenUrl('https://accounts.google.com/o/oauth2/token')
// Set the client ID and secret, from the Google Developers Console.
.setClientId(CLIENT_ID)
.setClientSecret(CLIENT_SECRET)
// Set the name of the callback function in the script referenced
// above that should be invoked to complete the OAuth flow.
.setCallbackFunction('authCallback')
// Set the property store where authorized tokens should be persisted.
.setPropertyStore(PropertiesService.getUserProperties())
// Set the scopes to request (space-separated for Google services).
// this is admin access for the sqlservice and access to the cloud-platform:
.setScope(
'https://www.googleapis.com/auth/sqlservice.admin ' +
'https://www.googleapis.com/auth/cloud-platform')
//Removed because this Should be covered by cloud-platform
//'https://www.googleapis.com/auth/devstorage.read_write '
// Below are Google-specific OAuth2 parameters.
// Sets the login hint, which will prevent the account chooser screen
// from being shown to users logged in with multiple accounts.
.setParam('login_hint', Session.getActiveUser().getEmail())
// Requests offline access.
.setParam('access_type', 'offline')
// Forces the approval prompt every time. This is useful for testing,
// but not desirable in a production application.
.setParam('approval_prompt', 'force');
}
function authCallback(request) {
var cloudSQLService = getService();
var isAuthorized = cloudSQLService.handleCallback(request);
if (isAuthorized) {
_log_('INFO', 'Access Approved');
return HtmlService.createHtmlOutput('Success! You can close this tab.');
} else {
_log_('INFO', 'Access Denied');
return HtmlService.createHtmlOutput('Denied. You can close this tab');
}
}
}
If you think back about a year ago you may remember the Massive Phishing Attack Targets Gmail Users What you are seeing is googles response to that.
Web credentials that use specific scopes require that Google approve them before anyone but the developer who created the credentials in question can use it. It normally takes about a week to get approved or so Google says.
You didnt seen it before because this only recently hit App script OAuth client verification
Starting July 18, 2017, Google OAuth clients that request certain sensitive OAuth scopes will be subject to review by Google.
We had a similar issue with the Google Compute Engine API. Setting the scopes explicitly in the appsscript.json file as per this article solved it for us:
"oauthScopes": [
"https://www.googleapis.com/auth/spreadsheets.readonly",
"https://www.googleapis.com/auth/userinfo.email",
"https://www.googleapis.com/auth/script.container.ui",
"https://www.googleapis.com/auth/script.external_request",
"https://www.googleapis.com/auth/spreadsheets",
"https://www.googleapis.com/auth/compute",
"https://www.googleapis.com/auth/cloud-platform"
],
I have google spreadsheet template with custom add-on which I'm trying to copy
def copyTemplateSpreadsheet(Drive driveService) {
File templateCopy = new File()
templateCopy.setName("excel-template")
def copiedFile = driveService.files().copy(templateSpreadsheetId, templateCopy).execute()
setCorrectPermission(driveService, copiedFile.getId())
copiedFile
}
private void setCorrectPermission(Drive driveService, def fileId) {
Permission newPermission = new Permission();
newPermission.setType("anyone");
newPermission.setRole("writer");
driveService.permissions().create(fileId, newPermission).execute();
}
The problem is that copied spreadsheet has broken add-on (isn't displayed in add-ons menu). There is correct add-on code in script editor but when I try to run any function I get error message
"We're sorry, a server error occurred. Please wait a bit and try again"
Keep in mind that the very same code work well in my template spreadsheet. Even if I delete all the code and leave empty onOpen function the error still appears.
Copying add-ons works well when I do it using regular google drive website (drive.google.com) and also worked when I tried to use google's API Explorer (https://developers.google.com/drive/v3/reference/files/copy#try-it). The problem seems to only when using sdk (at least java one - I haven't tried any other)
Also keep in mind I'm using google service account created as described in this article https://developers.google.com/identity/protocols/OAuth2ServiceAccount#creatinganaccount
and creating Drive instance with following code
Drive getDriveService() throws GeneralSecurityException, IOException, URISyntaxException {
HttpTransport httpTransport = new NetHttpTransport();
JacksonFactory jsonFactory = new JacksonFactory();
GoogleCredential credential = new GoogleCredential.Builder()
.setTransport(httpTransport)
.setJsonFactory(jsonFactory)
.setServiceAccountId(G_SERVICE_EMAIL)
.setServiceAccountScopes(Arrays.asList(DriveScopes.DRIVE))
.setServiceAccountPrivateKeyFromP12File(PKC_12_FILE)
.build();
Drive service = new Drive.Builder(httpTransport, jsonFactory, null)
.setHttpRequestInitializer(credential)
.build();
return service;
}
not sure if it matters though, especially since everything else seems to work just fine
Any fix ideas? I'm open to any workarounds as long as they work.
Also I would be fine with creating new file and just adding add-on code but it seems like I can't do it with API
I've found an acceptable solution which I described there
How can I create spreadsheet with included gs script by API?
Hope it helps someone ;)
Following the comment in How can I create spreadsheet with included gs script by API?, this is the app Scripts solution for making an authenticated POST:
function sendToHR(url,data){
var forDriveScope = DriveApp.getStorageUsed(); //needed to get Drive Scope requested
var dataToSend = [getName(),getID()];
for(key in data){
dataToSend.push(data[key])
}
var paylod = {
"data" : dataToSend
};
paylod = JSON.stringify(paylod);
var param = {
"method":"POST",
"headers" : {"Accept":"application/json","Authorization": "Bearer " + ScriptApp.getOAuthToken()},
"payload": paylod
};
return UrlFetchApp.fetch(url,param).getContentText();
}
And I have an example in python, that might be a bit more useful to you, in order for the python script to execute as a certien user I downloaded a JSON file with the keys from the projects console -> create credential -> get Key and download the file
def get_service():
global http_auth
global delegated_credentials
scopes = ['https://www.googleapis.com/auth/userinfo.email']
keyfile = os.path.join(CURR_DIR, JSON_FILENAME)
credentials = ServiceAccountCredentials.from_json_keyfile_name(
keyfile, scopes=scopes)
delegated_credentials = credentials.create_delegated(ADMIN_EMAIL)
http_auth = delegated_credentials.authorize(Http())
return build('SERVICE', 'v1', http=http_auth,
discoveryServiceUrl='DISCOVERY API SERVICE')
ADMIN_EMAIL is the actual admin email address and CURR_DIR and JSON_FILENAME are related to the downloaded file in your case I'm guessing you dont need admin rights just download the JSON file from the console of your current project and use your email address. Mine works when using the discovery API but a regular POST should be a bit faster to make
I needed to create a script that uploads the resulting screen shots to google drive.
I was hoping I could just auth in as my google user, but that seems... harder? So I abandoned that tact. Next I moved onto service accounts. This works fine (now) for my service account, but when I attempt to specify a user ($auth->sub) I get "Unauthorized client or scope in request.".
function buildService($userEmail) {
$DRIVE_SCOPE = 'https://www.googleapis.com/auth/drive';
$SERVICE_ACCOUNT_EMAIL = 'notsupplied#developer.gserviceaccount.com';
$SERVICE_ACCOUNT_PKCS12_FILE_PATH = 'pathtofile.p12';
$key = file_get_contents($SERVICE_ACCOUNT_PKCS12_FILE_PATH);
$auth = new Google_Auth_AssertionCredentials(
$SERVICE_ACCOUNT_EMAIL,
array($DRIVE_SCOPE),
$key);
$auth->sub = 'myuser#gmail.com';
$client = new Google_Client();
$client->setAssertionCredentials($auth);
return new Google_Service_Drive($client);
}
I'd love to abandon the service account and just auth with my regular google user if thats just as easy. Or solve how (in the api settings maybe?) I can ensure myuser#gmail.com can be used.
Refresh_token is the key here. In a webbrowser use this link to approve your google user:
https://accounts.google.com/AccountChooser?service=lso&continue=https%3A%2F%2Faccounts.google.com%2Fo%2Foauth2%2Fauth%3Fresponse_type%3Dcode%26scope%3Dhttps%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive%26redirect_uri%3Dhttps%3A%2F%2Fwww.example.com%2Foauth2callback%26access_type%3Doffline%26client_id%3D<CLIENT_ID>%26hl%3Den%26from_login%3D1%26as%3D34eac985232ba748&btmpl=authsub&hl=en&approval_prompt=force&access_type=offline
which will return a URL like https://www.example.com/oauth2callback?code=
Then post code=&client_id=&client_secret=&redirect_uri=&grant_type=authorization_code to https://accounts.google.com/o/oauth2/token
This will return a "refresh_token" parameter. Save this. Very important. If you don't get one you have to go to https://security.google.com/settings/security/permissions to revoke permissions from your app.
After you get the refresh token you're good to go:
$client = new Google_Client();
$client->setClientId($client_id);
$client->setClientSecret($client_secret);
$client->setRedirectUri($redirect_uri);
$client->addScope("https://www.googleapis.com/auth/drive");
$client->setAccessType('offline');
$token = $client->refreshToken('<YOUR_REFRESH_TOKEN>');
$service = new Google_Service_Drive($client);