Access denied Drive App when trying to delete file - google-apps-script

I wrote a little script to convert all the .xls and .xlsx files in my google drive to google sheets. However, the results of the script are very unreliable, sometimes I get an error, in some random part of the code, saying: "We're sorry, a server error occurred. Please wait a bit and try again.". And other times I get an error saying "Access denied Drive App" on the line where I try to delete my converted .xls or .xlsx files. I have enabled the Advance Google Services for the Drive API.
function convertFolder(folder){
Logger.log('Folder name '+folder.getName());
var files = folder.searchFiles('(title contains ".xlsx" or title contains ".xls") and (not title contains ".~lock")');
while(files.hasNext()){
var xFile = files.next();
var name = xFile.getName().replace(/\.[^/.]+$/, "");
Logger.log('File name '+name);
if (name.indexOf(".~lock")!=-1)
continue; //for some reason the "and (not title contains ".~lock")" didn't do the trick - why!?!
var parents = xFile.getParents();
var parents_arr = new Array();
while (parents.hasNext()){
parents_arr.push({'kind':"drive#fileLink",'id':parents.next().getId()});
}
var ID = xFile.getId();
var xBlob = xFile.getBlob();
var newFile = { title : name+'.gsheet',key : ID, parents: parents_arr, mimeType: "application/vnd.google-apps.spreadsheet"};
file = Drive.Files.insert(newFile, xBlob, {convert: true});
xFile.setTrashed(true);
}
var folders = folder.getFolders();
while (folders.hasNext())
convertFolder(folders.next());
}
function convertAllExcel(){
var folders = DriveApp.getFoldersByName('Data');
while (folders.hasNext()){
var folder = folders.next();
convertFolder(folder);
}
SpreadsheetApp.getUi().alert("done");
}

I suspect you're hitting a rate limit. If you are doing more than 20 write operations (create, update, delete) in rapid succession, you need to throttle the requests to approximately one every 1.5 seconds.
You're probably thinking "wtf", but that's seriously the maximum throughput you can achieve with Drive.

I had a similar problem! While running the script logged in as the main user seems to not have the maximum throughput limit, sharing a document with other users and letting them run the script that does writing operations in other user's drive does. 'Utilities.sleep(1500);' did the trick for me.
delFolder = DriveApp.getFolderById("folder_id")
delFiles = delFolder.getFiles();
while (delFiles.hasNext()) {
var delFile = delFiles.next();
Utilities.sleep(1500);
delFile.setTrashed(true);

Related

Google Apps Script Timing out due to large number of files on G-Drive

Hi I have a lot of files on my G-drive and I wanted to delete them so I am trying an app script which I found online
`
// NEW FUNCTION TO GET LIST OF EMPTY FOLDERS
function getEmptyFolders() {
// Get sheet
var sheet = SpreadsheetApp.getActive().getSheetByName('Manage Empty Folders')
if (!sheet) {
SpreadsheetApp.getActive().insertSheet('Manage Empty Folders')
}
var sheet = SpreadsheetApp.getActive().getSheetByName('Manage Empty Folders')
var clearContent = sheet.clear()
var folders = DriveApp.getFolders()
var fileArray = [
["Folder Name", "Folder Id", "Folder URL"]
]
while (folders.hasNext()) {
var folder = folders.next()
// If the folder has no files or child folder in it, call them out.
if (folder.getFiles().hasNext() == false && folder.getFolders().hasNext() == false) {
var name = folder.getName()
var id = folder.getId()
var url = folder.getUrl()
// Push empty folder details into array
fileArray.push([name, id, url])
}
}
`
However whenever I run this it hangs and takes a lot of time and then the error message "Script limit exceeded" pops up which is obvious since there are a billion files out there in the drive. I can go iteratively like per 100 or per 1000 however the script may allow but the minute i use the line : var folders = DriveApp.getFolders() I guess this is taking time to fetch all folders and causing the scripts to fail. any idea how to select them manually like 100-200 files at a time please ?
i tried putting a for loop but so long as it takes to read the getfolders() i think there is no way to limit that?

Search files in shared Google Drive [duplicate]

I have some Google Apps script code that searchs for files and folders on TeamDrive.
The issue I am having is that if a file or folder is created by my colleague, when I run my script it can't find the file. If I create a file, and my colleague runs the script, the script can't find the file even though we both have access to view, edit and can see the files and folders in Drive. If one of us edits the file made by the other person, then it becomes visible from the search.
I ran into a similar problem with the Drive REST api when doing some android development. In Android when calling files().list(), It took my a while to find out that I had to set the following in order for my search to be successfull every single time.
.setSupportsTeamDrives(true)
.setIncludeTeamDriveItems(true)
.setCorpora("teamDrive")
.setTeamDriveId(myFolder.getTeamDriveId())
I assume I am running into the same issue with my apps script code.
//Create the N Google docs files
function CreateNFiles(){
var spreadsheet = SpreadsheetApp.getActive();
var Nmain = spreadsheet.getSheetByName("Nmain")
var spreadsheetId = spreadsheet.getId();
var pdfDir = "Form Data";
var TemplatesFolder = null;
//Check and see if there is a 'Form Data' folder
var NFolderId = null;
var RFolderId = DriveApp.getFileById(spreadsheetId).getParents().next().getId();
var files = DriveApp.searchFolders('parents="'+RFolderId+'" and trashed=false');
while (files.hasNext()) {
var myfile = files.next();
if(myfile.getName() == pdfDir){
NOFolderId = myfile.getId();
}
}
https://developers.google.com/apps-script/reference/drive/drive-app#searchFiles(String)
this says to refer to
https://developers.google.com/drive/api/v3/search-parameters#examples_for_teamdriveslist
so I could potentially use
corpora="teamDrive"
is there a way to setSupportsTeamDrives? and setIncludeTeamDriveItems? and setTeamDriveId? in google apps scripts
Finding Files and Folders in a Team Drive
Here's a couple of functions I've been working on for my own needs. They're still a work in progress but one can file folders within a team drive folder and another can find items within a team drive folder. The Logger.log is setup to display item number, title, id, and mimeType.
This one finds Items (either files or folders). You can tell them apart by their types.
function findItemsInTeamDriveFolder(teamDriveId,folderId){
var teamDriveId=teamDriveId || '0AFN5OZjg48ZvUk9PVA';
var folderId=folderId || '1LK76CVE71fLputdFAN-zuL-HdRFDWBGv';
var options={
"corpora":"teamDrive",
"includeTeamDriveItems":true,
"orderBy":"folder",
"q":Utilities.formatString('\'%s\' in parents',folderId),
"supportsTeamDrives":true,
"teamDriveId":teamDriveId
};
var files=Drive.Files.list(options);
var data=JSON.parse(files);
for(var i=0;i<data.items.length;i++){
Logger.log('\nItem: %s - Title: %s - Id: %s - Type:%s - Trashed: %s\n',i+1,data.items[i].title,data.items[i].id,data.items[i].mimeType,data.items[i].explicitlyTrashed?'true':'false');
}
}
This one just finds folders in a folder. It's not reentrant it's a one level deal but currently that's all I need.
function findFoldersInATeamDriveFolder(teamDriveId,folderId){
var teamDriveId=teamDriveId || '0AAc6_2qyI7C0Uk9PVA';
var folderId=folderId || '1HenWOXTSCg96iAvA0ZkgEA9EGKlch4fz';
var optionalArgs={
"corpora":"teamDrive",
"includeTeamDriveItems":true,
"orderBy":"folder",
"q":Utilities.formatString('\'%s\' in parents and mimeType = \'application/vnd.google-apps.folder\'',folderId),
"supportsTeamDrives":true,
"teamDriveId":teamDriveId
}
var list=Drive.Files.list(optionalArgs)
var data=JSON.parse(list);
for(var i=0;i<data.items.length;i++){
Logger.log('\nItem: %s - Title: %s - Id: %s - Type: %s - Trashed;%s\n',i+1,data.items[i].title,data.items[i].id,data.items[i].mimeType,data.items[i].explicitlyTrashed?'true':'false');
findItemsInTeamDriveFolder(teamDriveId,data.items[i].id)
}
}
I thought that they might be helpful.
Meta Data for a file:
Search Parameters:
Drive.Files.List Documentation:
I just used Coopers code to list files that were in a shared drive. I added the code to find the teamdriveID. Two things that cost me some time and might be helpful for others: the number of files is restricted to 100 per default. So I changed it to 200 here. Also, the options file includes trashed files (very confusing) so I filtered them out with an if statement - I am sure this can be done more elegantly but this worked :)
var resource = {
'value': 'emailstring',
'type': 'user',
'role': 'writer'
}
var teamDriveId
// If you have several Team Drives, loop through and give access
var TeamDrive = Drive.Teamdrives.list();
for(var i = 0; i < TeamDrive.items.length; i++) {
if(TeamDrive.items[i].name == "foldernamestring") {
// This ID may also be a single file inside a Team Drive
teamDriveId = TeamDrive.items[i].id;
Logger.log("found "+TeamDrive.items[i].name);
}
}
var options={
"corpora":"teamDrive",
"maxResults":200,
"includeTeamDriveItems":true,
"supportsTeamDrives":true,
"teamDriveId":teamDriveId
};
var files=Drive.Files.list(options);
var data=JSON.parse(files);
var nritems= data.items.length
Logger.log("nritems "+nritems);
for(var i=0;i<nritems;i++){
if (data.items[i].explicitlyTrashed == false){
Logger.log('\nItem: %s - Title: %s - Id: %s - Type:%s - Trashed: %s\n',i+1,data.items[i].title,data.items[i].id,data.items[i].mimeType,data.items[i].explicitlyTrashed?'true':'false');
}
}
I use simple Apps Script code to search folders in a Shared Drive:
function searchFolderByTitle(title,ShDrId){
var folders = DriveApp.searchFolders("title contains '"+title+"' and '"+ShDrId+"' in parents");
while (folders.hasNext()) {
var folder = folders.next();
Logger.log(folder.getName());
}
}
You should use the operator "in" not "=" with "parents" parameter.

Automatic Convert and Delete Filetype Server Error (Timing Out?)

I am trying to create a script that will convert xls documents in a folder to sheets versions, then delete the xls versions. The script works, but because I have plenty of xls files in this folder, I believe it's timing out because it is throwing up, "We're sorry, a server error occurred. Please wait a bit and try again. (line 6, file "Convert XLS")" error a little while after I start running it. Is there a way to execute this without the server timing out or something?
function convertXLS() {
var folder = DriveApp.getFolderById('File_ID');
var files = folder.getFilesByType(MimeType.MICROSOFT_EXCEL_LEGACY);
while (files.hasNext()) {
var file = files.next();
Drive.Files.copy({}, file.getId(), {convert: true});
}
// while (files.hasNext()) {
// Drive.Files.remove( file.getId());
//}
}
Update:
So I reconfigured the code to this:
function importXLS(){
var folder = DriveApp.getFolderById('File_ID');// you can also use a folder as starting point and get the files in that folder... use only DriveApp method here.
var files = folder.getFilesByType(MimeType.MICROSOFT_EXCEL_LEGACY);
while(files.hasNext()){
var xFile = files.next();
var name = xFile.getName();
if (name.indexOf('.xls')>-1){ // this check is not necessaey here because I get the files with a search but I left it in case you get the files differently...
var ID = xFile.getId();
var xBlob = xFile.getBlob();
var newFile = { title : name+'_converted',
key : ID,
'parents':[{"id":"File_ID"}]
}
file = Drive.Files.insert(newFile, xBlob, {
convert: true,
});
}
}
}
But now it's giving me this error, "API call to drive.files.insert failed with error: Internal Error (line 14, file "Convert XLS")" after about two minutes of it running. I don't know what to do :(
What you are facing is a time-out due to the execution limit for Apps Script.
To solve this, create a function that converts and deletes a single file, then run it multiple times.
To do that, you will use getFilesByType in conjunction with Advanced Drive Services with the Drive.files.insert function with options as {convert:true}.
Running it multiple times can be done with a time-based trigger and/or manually.
Since you will be getting files only of a specific MIME Type, when you run out of XLS files you should have no problems.
Hope this helps!

Delete old files from a Google Drive folder

I created this script to delete files published more than 3 hours ago. And even if the latest file is older than 3 hours, it will not be deleted, so the folder will never be empty.
I enabled google's advanced service called DRIVE API V2.
I activated a trigger to analyze the folder every 5 minutes, but often the files are not deleted, they remain in the folder. The script works sometimes yes and sometimes no.
I would like to know what is wrong or what I need to edit to make it work perfectly.
function getOldFileIDs() {
// Old date is 3 Hours
var oldDate = new Date().getTime() - 3600*1000*3;
var cutOffDate = new Date(oldDate).toISOString();
// Get folderID using the URL on google drive
var folder = DriveApp.getFolderById('XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX');
var files = folder.searchFiles('modifiedDate < "' + cutOffDate + '"');
var obj = [];
while (files.hasNext()) {
var file = files.next();
obj.push({id: file.getId(), date: file.getDateCreated()});
}
obj.sort(function(x, y) {return x.date < y.date ? 1 : -1});
obj.shift();
var fileIDs = obj.map(function(e) {return e.id});
return fileIDs;
};
function deleteFiles() {
var fileIDs = getOldFileIDs();
fileIDs.forEach(function(fileID) {
Drive.Files.remove(fileID);
});
};
I also want some help so that this script can analyze if there are files with the same name inside the folder, if it exists, it delete surplus ones so that there is always only one file with each name.
And new detail, a new error appeared that until earlier did not happen ... Saying that:
API call failed for drive.files.delete with error Insufficient permissions for this file (line 24)
Line 24:
Drive.Files.remove(fileID);
By using consumer (free) accounts, only the file owner is able to delete them. To prevent the error, before calling Drive.Files.remove(fileID) your code should check if you are the file owner.
Related
Is it possible to get the owner for a Google Drive file?
How to get the owner of file using Google Drive API
How to programmticly remove a file from "share wtih me" in Google drive
How to retrieve the owner or Drive ID of a document with Google APIs?
Try this:
obj.sort(function(a, b) {
var a=new Date(a.date).valueOf();
var b=new Date(b.date).valueOf();
return b-a;
});

Google Script - How to use unzip

I am downloading a .zip from a website. It contains one .txt file. I would like to access the data in the txt and write it to a spreadsheet. I'm open to either accessing it directly and not extracting the zip OR extracting the zip, saving the txt to a Google Drive Folder, and accessing it once it is saved.
When I use Utilities.unzip(), I can never get it to unzip the file and usually end up with an "Invalid argument" error. In the code below, the last section before else contains the unzip command. It successfully saves the file to the correct Google Folder but then I can't extract it.
function myFunction() {
// define where to gather data from
var url = '<insert url here>';
var filename = "ReportUploadTesting05.zip";
var response = UrlFetchApp.fetch(url, {
// muteHttpExceptions: true,
// validateHttpsCertificates: false,
followRedirects: true // Default is true anyway.
});
// get spreadsheet for follow up info
var Sp = SpreadsheetApp.getActiveSpreadsheet();
if (response.getResponseCode() === 200) {
// get folder details of spreadsheet for saving future files
var folderURL = getParentFolder(Sp);
var folderID = getIdFromUrl(folderURL);
var folder = DriveApp.getFolderById(folderID);
// save zip file
var blob = response.getBlob();
var file = folder.createFile(blob);
file.setName(filename);
file.setDescription("Downloaded from " + url);
var fileID = file.getId();
Logger.log(fileID);
Logger.log(blob)
// extract zip (not working)
file.setContent('application/zip')
var fileUnzippedBlob = Utilities.unzip(file); // invalid argument error occurs here
var filename = 'unzipped file'
var fileUnzipped = folder.createFile(fileUnzippedBlob)
fileUnzipped.setName(filename)
}
else {
Logger.log(response.getResponseCode());
}
}
I've followed the instructions on the Utilities page. I can get their exact example to work. I've tried creating a .zip on my computer, uploading it to Google Drive and attempted to open it unsuccessfully. Obviously there are some subtleties of using the unzip that I'm missing.
Could you help me understand this?
I was running into the same "Invalid arguments" error in my testing, so instead of using:
file.setContent('application/zip')
I used:
file.setContentTypeFromExtension();
And, that solved the problem for me. Also, as #tukusejssirs mentioned, a zip file can contain multiple files, so unzip() returns an array of blobs (as documented here). That means you either need to loop through the files, or if you know you only have one, explicitly reference it's position in the array, like this:
var fileUnzipped = folder.createFile(fileUnzippedBlob[0])
Here's my entire script, which covers both of these issues:
/**
* Fetches a zip file from a URL, unzips it, then uploads a new file to the user's Drive.
*/
function uploadFile() {
var url = '<url goes here>';
var zip = UrlFetchApp.fetch('url').getBlob();
zip.setContentTypeFromExtension();
var unzippedFile = Utilities.unzip(zip);
var filename = unzippedFile[0].getName();
var contentType = unzippedFile[0].getContentType();
var csv = unzippedFile[0];
var file = {
title: filename,
mimeType: contentType
};
file = Drive.Files.insert(file, csv);
Logger.log('ID: %s, File size (bytes): %s', file.id, file.fileSize);
var fileId = file.id;
// Move the file to a specific folder within Drive (Link: https://drive.google.com/drive/folders/<folderId>)
var folderId = '<folderId>';
var folder = DriveApp.getFolderById(folderId);
var driveFile = DriveApp.getFileById(fileId);
folder.addFile(driveFile);
}
I think the answer to your question may be found here. Is there a size limit to a blob for Utilities.unzip(blob) in Google Apps Script?
If the download is over 100 mb the full file cannot be downloaded. Due to that it will not be in the proper zip format. Throwing the cannot unzip file error.
I believe that the creation of the blob from a file (in this case the .zip file) requires the .next(); otherwise it did not work for me.
Also note that the .zip file might contain more than one file, therefore I included a for cycle.
Anyway, my working/tested solution/script is the following:
function unzip(folderName, fileZipName){
// Variables
// var folderName = "folder_name";
// var fileZipName = "file_name.zip";
var folderId = getFolderId(folderName);
var folder = DriveApp.getFolderById(folderId);
var fileZip = folder.getFilesByName(fileZipName);
var fileExtractedBlob, fileZipBlob, i;
// Decompression
fileZipBlob = fileZip.next().getBlob();
fileZipBlob.setContentType("application/zip");
fileExtractedBlob = Utilities.unzip(fileZipBlob);
for (i=0; i < fileExtractedBlob.length; i++){
folder.createFile(fileExtractedBlob[i]);
}
}