Delete multiple google drive files that are unorganized / orphaned - google-apps-script

I wanted to remove several thousand files in a Google Drive folder. I deleted the folder, which many in the community know (but I didn't) leaves the files present but orphaned. I can list the files via "is:unorganized owner:me" but can't select more than a few hundred at a time to (slowly) delete.
Is there a script that will search for and delete these, and only these, files?
Thanks

I believe your goal as follows.
You want to retrieve the files searched with is:unorganized owner:me and want to delete the files.
You want to reduce the process cost of this process.
Issue and workaround:
In the current stage, unfortunately, the files cannot be directly retrieved by searching is:unorganized owner:me with Drive API and Drive service. So as the current workaround, I would like to suggest the following flow.
Retrieve the file list of all files in the Google Drive. In this case, 'me' in owners and trashed = false is used as the search query. The method of "Files: list" of Drive API is used for this.
Retrieve the file list of files which have no parent folder using a script.
By above flow 1 and 2, the search of is:unorganized owner:me can be achieved using a script.
Move the files to the trash box using the retrieved file list using the method of "Files: update" of Drive API.
In this case, the files are not directly deleted. Because when the files are directly deleted, I thought that this situation is very dangerous. Of course, the files can be also directly deleted using the method of "Files: delete" of Drive API. Ref
From your question, I thought that there might be a lot of files you want to delete. So in this answer, I would like to suggest to use the batch requests.
When above flow is reflected to the script, it becomes as follows.
IMPORTANT: PLEASE BE CAREFUL!
When you use this sample script, please be careful this. I would like to recommend the following flow for using this script.
Retrieve the file list of is:unorganized owner:me and check whether all files of the file list are the files you want to delete.
You can retrieve the file list using const fileList = getFileList(token); in main() function.
When you could completely confirm that all files of the file list are the files you want to delete, please use const res = moveFilesToTrashBox(token, fileList);. By this, the files of file list are moved to the trash box.
Please confirm the trash box. When the files you don't want to delete are included, please restore them.
Sample script:
Before you use this script, please enable Drive API at Advanced Google services. And please run main() function.
// Move the files in the file list to the trash box.
function moveFilesToTrashBox(token, fileList) {
const limit = 100;
const split = Math.ceil(fileList.length / limit);
const res = [];
for (let i = 0; i < split; i++) {
const boundary = "xxxxxxxxxx";
const payload = fileList.splice(0, limit).reduce((s, {id}, i, a) => s += "Content-Type: application/http\r\n" +
"Content-ID: " + i + "\r\n\r\n" +
"PATCH https://www.googleapis.com/drive/v3/files/" + id + "\r\n" +
"Content-Type: application/json\r\n\r\n" +
JSON.stringify({trashed: true}) + "\r\n" +
"--" + boundary + (i == a.length - 1 ? "--" : "") + "\r\n" , "--" + boundary + "\r\n");
const params = {
method: "post",
contentType: "multipart/mixed; boundary=" + boundary,
payload: payload,
headers: {Authorization: "Bearer " + token},
muteHttpExceptions: true,
};
const r = UrlFetchApp.fetch("https://www.googleapis.com/batch/drive/v3", params);
res.push(r.getContentText());
}
return res;
}
// Retrieve the file list by searching with "is:unorganized owner:me".
function getFileList(token) {
const fields = decodeURIComponent("nextPageToken,files(name,id,mimeType,parents)");
const q = decodeURIComponent("'me' in owners and trashed = false");
let allFiles = [];
let pageToken = "";
do {
const res = UrlFetchApp.fetch(
`https://www.googleapis.com/drive/v3/files?pageSize=1000&fields=${fields}&q=${q}&pageToken=${pageToken}`,
{ headers: { authorization: `Bearer ${token}` } }
);
const obj = JSON.parse(res);
allFiles = allFiles.concat(obj.files);
pageToken = obj.nextPageToken;
} while (pageToken);
return allFiles.filter(({ parents }) => !parents);
}
// Please run this function.
function main() {
const token = ScriptApp.getOAuthToken();
// Retrieve the file list of all files in the Google Drive.
const fileList = getFileList(token);
console.log(fileList.length);
console.log(fileList);
// Move the files to the trash box using the retrieved file list.
// When you could completely confirm that all files of the file list are the files you want to delete, please use the below script.
// const res = moveFilesToTrashBox(token, fileList);
// console.log(res);
// DriveApp.createFile(); // This is used for automatically adding a scope of "https://www.googleapis.com/auth/drive".
}
References:
Files: list
Files: update
Batch request

Related

How do I query a bunch of CSV Files in Google Sheets? [duplicate]

I have a legacy database system (not web accessible) on a server which generates CSV or XLS reports to a Google Drive folder. Currently, I am manually opening those files in Drive web interface and converting them to Google Sheets.
I would rather this be automatic so that I can create jobs that append/transform and graph the data in other sheets.
Is it possible to output a native .gsheet file? Or is there a way to convert CSV or XLS to .gsheet programmatically after saving it to Google Drive either in Google Apps or via a Windows based script/utility?
You can programmatically import data from a csv file in your Drive into an existing Google Sheet using Google Apps Script, replacing/appending data as needed.
Below is some sample code. It assumes that: a) you have a designated folder in your Drive where the CSV file is saved/uploaded to; b) the CSV file is named "report.csv" and the data in it comma-delimited; and c) the CSV data is imported into a designated spreadsheet. See comments in code for further details.
function importData() {
var fSource = DriveApp.getFolderById(reports_folder_id); // reports_folder_id = id of folder where csv reports are saved
var fi = fSource.getFilesByName('report.csv'); // latest report file
var ss = SpreadsheetApp.openById(data_sheet_id); // data_sheet_id = id of spreadsheet that holds the data to be updated with new report data
if ( fi.hasNext() ) { // proceed if "report.csv" file exists in the reports folder
var file = fi.next();
var csv = file.getBlob().getDataAsString();
var csvData = CSVToArray(csv); // see below for CSVToArray function
var newsheet = ss.insertSheet('NEWDATA'); // create a 'NEWDATA' sheet to store imported data
// loop through csv data array and insert (append) as rows into 'NEWDATA' sheet
for ( var i=0, lenCsv=csvData.length; i<lenCsv; i++ ) {
newsheet.getRange(i+1, 1, 1, csvData[i].length).setValues(new Array(csvData[i]));
}
/*
** report data is now in 'NEWDATA' sheet in the spreadsheet - process it as needed,
** then delete 'NEWDATA' sheet using ss.deleteSheet(newsheet)
*/
// rename the report.csv file so it is not processed on next scheduled run
file.setName("report-"+(new Date().toString())+".csv");
}
};
// http://www.bennadel.com/blog/1504-Ask-Ben-Parsing-CSV-Strings-With-Javascript-Exec-Regular-Expression-Command.htm
// This will parse a delimited string into an array of
// arrays. The default delimiter is the comma, but this
// can be overriden in the second argument.
function CSVToArray( strData, strDelimiter ) {
// Check to see if the delimiter is defined. If not,
// then default to COMMA.
strDelimiter = (strDelimiter || ",");
// Create a regular expression to parse the CSV values.
var objPattern = new RegExp(
(
// Delimiters.
"(\\" + strDelimiter + "|\\r?\\n|\\r|^)" +
// Quoted fields.
"(?:\"([^\"]*(?:\"\"[^\"]*)*)\"|" +
// Standard fields.
"([^\"\\" + strDelimiter + "\\r\\n]*))"
),
"gi"
);
// Create an array to hold our data. Give the array
// a default empty first row.
var arrData = [[]];
// Create an array to hold our individual pattern
// matching groups.
var arrMatches = null;
// Keep looping over the regular expression matches
// until we can no longer find a match.
while (arrMatches = objPattern.exec( strData )){
// Get the delimiter that was found.
var strMatchedDelimiter = arrMatches[ 1 ];
// Check to see if the given delimiter has a length
// (is not the start of string) and if it matches
// field delimiter. If id does not, then we know
// that this delimiter is a row delimiter.
if (
strMatchedDelimiter.length &&
(strMatchedDelimiter != strDelimiter)
){
// Since we have reached a new row of data,
// add an empty row to our data array.
arrData.push( [] );
}
// Now that we have our delimiter out of the way,
// let's check to see which kind of value we
// captured (quoted or unquoted).
if (arrMatches[ 2 ]){
// We found a quoted value. When we capture
// this value, unescape any double quotes.
var strMatchedValue = arrMatches[ 2 ].replace(
new RegExp( "\"\"", "g" ),
"\""
);
} else {
// We found a non-quoted value.
var strMatchedValue = arrMatches[ 3 ];
}
// Now that we have our value string, let's add
// it to the data array.
arrData[ arrData.length - 1 ].push( strMatchedValue );
}
// Return the parsed data.
return( arrData );
};
You can then create time-driven trigger in your script project to run importData() function on a regular basis (e.g. every night at 1AM), so all you have to do is put new report.csv file into the designated Drive folder, and it will be automatically processed on next scheduled run.
If you absolutely MUST work with Excel files instead of CSV, then you can use this code below. For it to work you must enable Drive API in Advanced Google Services in your script and in Developers Console (see How to Enable Advanced Services for details).
/**
* Convert Excel file to Sheets
* #param {Blob} excelFile The Excel file blob data; Required
* #param {String} filename File name on uploading drive; Required
* #param {Array} arrParents Array of folder ids to put converted file in; Optional, will default to Drive root folder
* #return {Spreadsheet} Converted Google Spreadsheet instance
**/
function convertExcel2Sheets(excelFile, filename, arrParents) {
var parents = arrParents || []; // check if optional arrParents argument was provided, default to empty array if not
if ( !parents.isArray ) parents = []; // make sure parents is an array, reset to empty array if not
// Parameters for Drive API Simple Upload request (see https://developers.google.com/drive/web/manage-uploads#simple)
var uploadParams = {
method:'post',
contentType: 'application/vnd.ms-excel', // works for both .xls and .xlsx files
contentLength: excelFile.getBytes().length,
headers: {'Authorization': 'Bearer ' + ScriptApp.getOAuthToken()},
payload: excelFile.getBytes()
};
// Upload file to Drive root folder and convert to Sheets
var uploadResponse = UrlFetchApp.fetch('https://www.googleapis.com/upload/drive/v2/files/?uploadType=media&convert=true', uploadParams);
// Parse upload&convert response data (need this to be able to get id of converted sheet)
var fileDataResponse = JSON.parse(uploadResponse.getContentText());
// Create payload (body) data for updating converted file's name and parent folder(s)
var payloadData = {
title: filename,
parents: []
};
if ( parents.length ) { // Add provided parent folder(s) id(s) to payloadData, if any
for ( var i=0; i<parents.length; i++ ) {
try {
var folder = DriveApp.getFolderById(parents[i]); // check that this folder id exists in drive and user can write to it
payloadData.parents.push({id: parents[i]});
}
catch(e){} // fail silently if no such folder id exists in Drive
}
}
// Parameters for Drive API File Update request (see https://developers.google.com/drive/v2/reference/files/update)
var updateParams = {
method:'put',
headers: {'Authorization': 'Bearer ' + ScriptApp.getOAuthToken()},
contentType: 'application/json',
payload: JSON.stringify(payloadData)
};
// Update metadata (filename and parent folder(s)) of converted sheet
UrlFetchApp.fetch('https://www.googleapis.com/drive/v2/files/'+fileDataResponse.id, updateParams);
return SpreadsheetApp.openById(fileDataResponse.id);
}
/**
* Sample use of convertExcel2Sheets() for testing
**/
function testConvertExcel2Sheets() {
var xlsId = "0B9**************OFE"; // ID of Excel file to convert
var xlsFile = DriveApp.getFileById(xlsId); // File instance of Excel file
var xlsBlob = xlsFile.getBlob(); // Blob source of Excel file for conversion
var xlsFilename = xlsFile.getName(); // File name to give to converted file; defaults to same as source file
var destFolders = []; // array of IDs of Drive folders to put converted file in; empty array = root folder
var ss = convertExcel2Sheets(xlsBlob, xlsFilename, destFolders);
Logger.log(ss.getId());
}
The above code is also available as a gist here.
You can get Google Drive to automatically convert csv files to Google Sheets by appending
?convert=true
to the end of the api url you are calling.
EDIT:
Here is the documentation on available parameters:
https://developers.google.com/drive/v2/reference/files/insert
Also, while searching for the above link, I found this question has already been answered here:
Upload CSV to Google Drive Spreadsheet using Drive v2 API
(Mar 2017) The accepted answer is not the best solution. It relies on manual translation using Apps Script, and the code may not be resilient, requiring maintenance. If your legacy system autogenerates CSV files, it's best they go into another folder for temporary processing (importing [uploading to Google Drive & converting] to Google Sheets files).
My thought is to let the Drive API do all the heavy-lifting. The Google Drive API team released v3 at the end of 2015, and in that release, insert() changed names to create() so as to better reflect the file operation. There's also no more convert flag -- you just specify MIMEtypes... imagine that!
The documentation has also been improved: there's now a special guide devoted to uploads (simple, multipart, and resumable) that comes with sample code in Java, Python, PHP, C#/.NET, Ruby, JavaScript/Node.js, and iOS/Obj-C that imports CSV files into Google Sheets format as desired.
Below is one alternate Python solution for short files ("simple upload") where you don't need the apiclient.http.MediaFileUpload class. This snippet assumes your auth code works where your service endpoint is DRIVE with a minimum auth scope of https://www.googleapis.com/auth/drive.file.
# filenames & MIMEtypes
DST_FILENAME = 'inventory'
SRC_FILENAME = DST_FILENAME + '.csv'
SHT_MIMETYPE = 'application/vnd.google-apps.spreadsheet'
CSV_MIMETYPE = 'text/csv'
# Import CSV file to Google Drive as a Google Sheets file
METADATA = {'name': DST_FILENAME, 'mimeType': SHT_MIMETYPE}
rsp = DRIVE.files().create(body=METADATA, media_body=SRC_FILENAME).execute()
if rsp:
print('Imported %r to %r (as %s)' % (SRC_FILENAME, DST_FILENAME, rsp['mimeType']))
Better yet, rather than uploading to My Drive, you'd upload to one (or more) specific folder(s), meaning you'd add the parent folder ID(s) to METADATA. (Also see the code sample on this page.) Finally, there's no native .gsheet "file" -- that file just has a link to the online Sheet, so what's above is what you want to do.
If not using Python, you can use the snippet above as pseudocode to port to your system language. Regardless, there's much less code to maintain because there's no CSV parsing. The only thing remaining is to blow away the CSV file temp folder your legacy system wrote to.
In case anyone would be searching - I created utility for automated import of xlsx files into google spreadsheet: xls2sheets. One can do it automatically via setting up the cronjob for ./cmd/sheets-refresh, readme describes it all. Hope that would be of use.

Google Apps Script: Downloading files from Drive (same user)

I'm trying to write a Google Apps Script to download all files in a particular Drive folder (likely .csv files). I have found the getDownloadUrl() method but I haven't been able to figure out what to do with it. I'm currently trying the following code, where files is the list of the files in the folder:
while(files.hasNext()) {
var response = UrlFetchApp.fetch(files.next().getDownloadUrl());
Logger.log(response.getContentText());
}
When I try to run the code, however, I get a 401 error which I guess means I lack the proper authorization? But I was under the impression that I wouldn't need to go through all of the OAuth2 steps if everything was taking place within my one Google account. The Google guide to connecting to external APIs makes it look like I should be able to just fetch the url. I've already gotten access to my Drive files, because the download URL does exist when I run that method. What am I missing here? I'm really new to all of this so maybe it's something basic.
Thanks!
EDIT:
I managed to fix the 401 error by modifying the code as follows:
while(files.hasNext()) {
var response = UrlFetchApp.fetch(files.next().getDownloadUrl(),{headers: {Authorization: "Bearer " + ScriptApp.getOAuthToken()}});
Logger.log(response.getContentText());
}
But the issue remains that this only returns the contents to me, rather than downloading the file. How can I initiate a download from the results of this fetch call?
Besides listing all download links, I guess original poster also wants to download files to user's computer (according to earlier discussion).
To do this, encode blob with base 64 in server side (e.g. Google App Script) and download with data URI in client's browser. Below are code for this, with help of this answer.
Google App Script
...
function getBlobInBase64(fileId){
// omit authorization code if any
var file = DriveApp.getFileById(fileId);
var blob = file .getBlob();
return {
file_name: file.getName(),
mime: file.getMimeType(),
b64: Utilities.base64Encode(blob.getBytes());
}
...
Javascript that serve with index.html
...
function getFile(fileId){
google.script.run.withSuccessHandler((data) => {
var uri = 'data:' + data.mime + ';charset=ISO-8859-1;base64,' + encodeURIComponent(data.b64);
downloadURI(uri, data.file_name);
}).withFailureHandler((err) => {
console.log(err);
}).getBlobInBase64();
}
...
NOTE: I haven't run this code but the method should work as used in my other project.
This will log the file names & URLS for any files available for downloading (first 100 in root drive):
function myFunction() {
var files = DriveApp.getFiles();
var c = 0;
while (files.hasNext() && c<100) {
var file = files.next();
Logger.log("File Name: " + file.getName());
Logger.log(" Download URL: " + file.getDownloadUrl());
c++;
}
}
My answer might be a bit off but I think you have a better chance downloading files from Google Drive using the webContentLink as it is the method I commonly use. I obtain webContentLink by using Files.list and ask for webContentLink in the fields parameter. I run that link through the browser and it downloads the file.
If you are trying to download Google Drive files to local computer using Google Apps Script, Then please understand that Google Apps Script is a server side scripting language. It can't download and save files to your local drive.
Here is a webapp that may be helpful for you. It does not do exactly what you are looking for but you may be able to edit it and get a result. Hope it helps!
CODE:
function doGet(e) { // main function
var template = HtmlService.createTemplateFromFile('index.html'); // filename always!
return template.evaluate().setTitle('Search Drive').setSandboxMode(HtmlService.SandboxMode.IFRAME);
}
// Process the form
function processForm(searchTerm) {
var resultToReturn;
Logger.log('processForm was called! ' + searchTerm);
resultToReturn = SearchFiles(searchTerm); // Call to the search files function to search files on google drive
Logger.log('resultToReturn: ' + resultToReturn);
return resultToReturn; // return the results
}
function SearchFiles(searchTerm) {
var searchFor ="title contains '" + searchTerm + "'"; //single quotes are needed around searchterm
var owneris ="and 'YOUREmail#email.com' in Owners"; //email address to search for
var names = [];
Logger.log(searchFor + " " + owneris);
var files = DriveApp.searchFiles(searchFor + " " + owneris);
while (files.hasNext()) {
var file = files.next();
var fileId = file.getId();// To get FileId of the file
var lm = file.getLastUpdated();
var name = file.getName()+"|~|"+fileId+"|~|"+lm; // Im concatenating the filename with file id separated by |~|
names.push(name); // adding to the array
}
return names; // return results
}
INDEX.html
<html>
<head>
<base target="_top">
<script>
function displayMessage() {
var searchTerm;
searchTerm = document.getElementById('idSrchTerm').value;
console.log('searchTerm: ' + searchTerm );
// Below call means: call to processForm passing the searchTerm value (previously escaped) and after finish call the handleResults function
google.script.run.withSuccessHandler(handleResults).processForm(searchTerm.replace("'","\'"));
}
function handleResults(results){
console.log('Handle Results was called! ');
document.writeln('BACK<br/><br/>');
var length=results.length; // total elements of results
for(var i=0;i<length;i++)
{
var item=results[i];
item=item.split("|~|"); // split the line |~|, position 0 has the filename and 1 the file id
document.writeln("<b><a href='https://docs.google.com/document/d/"+item[1]+"' target='_blank'>"+item[0]+"</b></a> (Last modified: "+item[2]+")<br/><br/>"); // write result
}
document.writeln("End of results...");
}
</script>
</head>
<body><center><br/>
Search: <input type="text" id="idSrchTerm" name="search">
<input type="button" value="search files on Google Drive" name="submitButton" onclick="displayMessage()"/>
</center>
</body>
</html>

Converting from word document to native google doc

So I have some 2000 word documents I can generate with c#, however, I need them uploaded to google docs, in the native google docs format.
I looked at This guide to try to convert them, however, the code fails on the second to last line.
(My code below)
function myFunction() {
var folder = DriveApp.getFoldersByName("test").next();
var contents = folder.getFiles();
while (contents.hasNext()){
var file = contents.next();
var fileName = file.getName();
var officeFile = DriveApp.getFilesByName(fileName).next();
// Use the Advanced Drive API to upload the Excel file to Drive
// convert = true will convert the file to the corresponding Google Docs format
var uploadFile = JSON.parse(UrlFetchApp.fetch(
"https://www.googleapis.com/upload/drive/v2/files?uploadType=media&convert=true",
{
method: "POST",
contentType: officeFile.getMimeType(),
payload: officeFile.getBlob().getBytes(),
headers: {
"Authorization" : "Bearer " + ScriptApp.getOAuthToken()
},
muteHttpExceptions: true
}
).getContentText());
// Remove the file extension from the original file name
var fileName2 = officeFile.getName();
fileName2 = fileName2.substr(0, fileName2.lastIndexOf("."));
// Update the name of the Google Sheet created from the Excel sheet
DriveApp.getFileById(uploadFile.getID()).setName(fileName2); // FAILS HERE
//Logger.log(uploadFile.alternateLink);
}}
TypeError: Cannot find function getID in object [object Object]. (line 33, file "Code")
I understand what kind of error this is, however, I do not necessarily know how to solve it.
You don't need any UrlFetchApp manipulations. The Advanced Drive Service provides conversion as a part of copy method. Takes two lines of code:
var fileId = 'ID_of_Word_file';
Drive.Files.copy({}, fileId, {'convert': true});
You will need to enable this service in the scropt menu: see Resources > Advanced Services.
If iterating through a folder, get each file from file iterator with .next() and use getId() to get its id; the proceed as above.
The empty object in the first argument can be used to name the new file: e.g., it could be {'title': 'Name of new file'}

How to automatically import data from uploaded CSV or XLS file into Google Sheets

I have a legacy database system (not web accessible) on a server which generates CSV or XLS reports to a Google Drive folder. Currently, I am manually opening those files in Drive web interface and converting them to Google Sheets.
I would rather this be automatic so that I can create jobs that append/transform and graph the data in other sheets.
Is it possible to output a native .gsheet file? Or is there a way to convert CSV or XLS to .gsheet programmatically after saving it to Google Drive either in Google Apps or via a Windows based script/utility?
You can programmatically import data from a csv file in your Drive into an existing Google Sheet using Google Apps Script, replacing/appending data as needed.
Below is some sample code. It assumes that: a) you have a designated folder in your Drive where the CSV file is saved/uploaded to; b) the CSV file is named "report.csv" and the data in it comma-delimited; and c) the CSV data is imported into a designated spreadsheet. See comments in code for further details.
function importData() {
var fSource = DriveApp.getFolderById(reports_folder_id); // reports_folder_id = id of folder where csv reports are saved
var fi = fSource.getFilesByName('report.csv'); // latest report file
var ss = SpreadsheetApp.openById(data_sheet_id); // data_sheet_id = id of spreadsheet that holds the data to be updated with new report data
if ( fi.hasNext() ) { // proceed if "report.csv" file exists in the reports folder
var file = fi.next();
var csv = file.getBlob().getDataAsString();
var csvData = CSVToArray(csv); // see below for CSVToArray function
var newsheet = ss.insertSheet('NEWDATA'); // create a 'NEWDATA' sheet to store imported data
// loop through csv data array and insert (append) as rows into 'NEWDATA' sheet
for ( var i=0, lenCsv=csvData.length; i<lenCsv; i++ ) {
newsheet.getRange(i+1, 1, 1, csvData[i].length).setValues(new Array(csvData[i]));
}
/*
** report data is now in 'NEWDATA' sheet in the spreadsheet - process it as needed,
** then delete 'NEWDATA' sheet using ss.deleteSheet(newsheet)
*/
// rename the report.csv file so it is not processed on next scheduled run
file.setName("report-"+(new Date().toString())+".csv");
}
};
// http://www.bennadel.com/blog/1504-Ask-Ben-Parsing-CSV-Strings-With-Javascript-Exec-Regular-Expression-Command.htm
// This will parse a delimited string into an array of
// arrays. The default delimiter is the comma, but this
// can be overriden in the second argument.
function CSVToArray( strData, strDelimiter ) {
// Check to see if the delimiter is defined. If not,
// then default to COMMA.
strDelimiter = (strDelimiter || ",");
// Create a regular expression to parse the CSV values.
var objPattern = new RegExp(
(
// Delimiters.
"(\\" + strDelimiter + "|\\r?\\n|\\r|^)" +
// Quoted fields.
"(?:\"([^\"]*(?:\"\"[^\"]*)*)\"|" +
// Standard fields.
"([^\"\\" + strDelimiter + "\\r\\n]*))"
),
"gi"
);
// Create an array to hold our data. Give the array
// a default empty first row.
var arrData = [[]];
// Create an array to hold our individual pattern
// matching groups.
var arrMatches = null;
// Keep looping over the regular expression matches
// until we can no longer find a match.
while (arrMatches = objPattern.exec( strData )){
// Get the delimiter that was found.
var strMatchedDelimiter = arrMatches[ 1 ];
// Check to see if the given delimiter has a length
// (is not the start of string) and if it matches
// field delimiter. If id does not, then we know
// that this delimiter is a row delimiter.
if (
strMatchedDelimiter.length &&
(strMatchedDelimiter != strDelimiter)
){
// Since we have reached a new row of data,
// add an empty row to our data array.
arrData.push( [] );
}
// Now that we have our delimiter out of the way,
// let's check to see which kind of value we
// captured (quoted or unquoted).
if (arrMatches[ 2 ]){
// We found a quoted value. When we capture
// this value, unescape any double quotes.
var strMatchedValue = arrMatches[ 2 ].replace(
new RegExp( "\"\"", "g" ),
"\""
);
} else {
// We found a non-quoted value.
var strMatchedValue = arrMatches[ 3 ];
}
// Now that we have our value string, let's add
// it to the data array.
arrData[ arrData.length - 1 ].push( strMatchedValue );
}
// Return the parsed data.
return( arrData );
};
You can then create time-driven trigger in your script project to run importData() function on a regular basis (e.g. every night at 1AM), so all you have to do is put new report.csv file into the designated Drive folder, and it will be automatically processed on next scheduled run.
If you absolutely MUST work with Excel files instead of CSV, then you can use this code below. For it to work you must enable Drive API in Advanced Google Services in your script and in Developers Console (see How to Enable Advanced Services for details).
/**
* Convert Excel file to Sheets
* #param {Blob} excelFile The Excel file blob data; Required
* #param {String} filename File name on uploading drive; Required
* #param {Array} arrParents Array of folder ids to put converted file in; Optional, will default to Drive root folder
* #return {Spreadsheet} Converted Google Spreadsheet instance
**/
function convertExcel2Sheets(excelFile, filename, arrParents) {
var parents = arrParents || []; // check if optional arrParents argument was provided, default to empty array if not
if ( !parents.isArray ) parents = []; // make sure parents is an array, reset to empty array if not
// Parameters for Drive API Simple Upload request (see https://developers.google.com/drive/web/manage-uploads#simple)
var uploadParams = {
method:'post',
contentType: 'application/vnd.ms-excel', // works for both .xls and .xlsx files
contentLength: excelFile.getBytes().length,
headers: {'Authorization': 'Bearer ' + ScriptApp.getOAuthToken()},
payload: excelFile.getBytes()
};
// Upload file to Drive root folder and convert to Sheets
var uploadResponse = UrlFetchApp.fetch('https://www.googleapis.com/upload/drive/v2/files/?uploadType=media&convert=true', uploadParams);
// Parse upload&convert response data (need this to be able to get id of converted sheet)
var fileDataResponse = JSON.parse(uploadResponse.getContentText());
// Create payload (body) data for updating converted file's name and parent folder(s)
var payloadData = {
title: filename,
parents: []
};
if ( parents.length ) { // Add provided parent folder(s) id(s) to payloadData, if any
for ( var i=0; i<parents.length; i++ ) {
try {
var folder = DriveApp.getFolderById(parents[i]); // check that this folder id exists in drive and user can write to it
payloadData.parents.push({id: parents[i]});
}
catch(e){} // fail silently if no such folder id exists in Drive
}
}
// Parameters for Drive API File Update request (see https://developers.google.com/drive/v2/reference/files/update)
var updateParams = {
method:'put',
headers: {'Authorization': 'Bearer ' + ScriptApp.getOAuthToken()},
contentType: 'application/json',
payload: JSON.stringify(payloadData)
};
// Update metadata (filename and parent folder(s)) of converted sheet
UrlFetchApp.fetch('https://www.googleapis.com/drive/v2/files/'+fileDataResponse.id, updateParams);
return SpreadsheetApp.openById(fileDataResponse.id);
}
/**
* Sample use of convertExcel2Sheets() for testing
**/
function testConvertExcel2Sheets() {
var xlsId = "0B9**************OFE"; // ID of Excel file to convert
var xlsFile = DriveApp.getFileById(xlsId); // File instance of Excel file
var xlsBlob = xlsFile.getBlob(); // Blob source of Excel file for conversion
var xlsFilename = xlsFile.getName(); // File name to give to converted file; defaults to same as source file
var destFolders = []; // array of IDs of Drive folders to put converted file in; empty array = root folder
var ss = convertExcel2Sheets(xlsBlob, xlsFilename, destFolders);
Logger.log(ss.getId());
}
The above code is also available as a gist here.
You can get Google Drive to automatically convert csv files to Google Sheets by appending
?convert=true
to the end of the api url you are calling.
EDIT:
Here is the documentation on available parameters:
https://developers.google.com/drive/v2/reference/files/insert
Also, while searching for the above link, I found this question has already been answered here:
Upload CSV to Google Drive Spreadsheet using Drive v2 API
(Mar 2017) The accepted answer is not the best solution. It relies on manual translation using Apps Script, and the code may not be resilient, requiring maintenance. If your legacy system autogenerates CSV files, it's best they go into another folder for temporary processing (importing [uploading to Google Drive & converting] to Google Sheets files).
My thought is to let the Drive API do all the heavy-lifting. The Google Drive API team released v3 at the end of 2015, and in that release, insert() changed names to create() so as to better reflect the file operation. There's also no more convert flag -- you just specify MIMEtypes... imagine that!
The documentation has also been improved: there's now a special guide devoted to uploads (simple, multipart, and resumable) that comes with sample code in Java, Python, PHP, C#/.NET, Ruby, JavaScript/Node.js, and iOS/Obj-C that imports CSV files into Google Sheets format as desired.
Below is one alternate Python solution for short files ("simple upload") where you don't need the apiclient.http.MediaFileUpload class. This snippet assumes your auth code works where your service endpoint is DRIVE with a minimum auth scope of https://www.googleapis.com/auth/drive.file.
# filenames & MIMEtypes
DST_FILENAME = 'inventory'
SRC_FILENAME = DST_FILENAME + '.csv'
SHT_MIMETYPE = 'application/vnd.google-apps.spreadsheet'
CSV_MIMETYPE = 'text/csv'
# Import CSV file to Google Drive as a Google Sheets file
METADATA = {'name': DST_FILENAME, 'mimeType': SHT_MIMETYPE}
rsp = DRIVE.files().create(body=METADATA, media_body=SRC_FILENAME).execute()
if rsp:
print('Imported %r to %r (as %s)' % (SRC_FILENAME, DST_FILENAME, rsp['mimeType']))
Better yet, rather than uploading to My Drive, you'd upload to one (or more) specific folder(s), meaning you'd add the parent folder ID(s) to METADATA. (Also see the code sample on this page.) Finally, there's no native .gsheet "file" -- that file just has a link to the online Sheet, so what's above is what you want to do.
If not using Python, you can use the snippet above as pseudocode to port to your system language. Regardless, there's much less code to maintain because there's no CSV parsing. The only thing remaining is to blow away the CSV file temp folder your legacy system wrote to.
In case anyone would be searching - I created utility for automated import of xlsx files into google spreadsheet: xls2sheets. One can do it automatically via setting up the cronjob for ./cmd/sheets-refresh, readme describes it all. Hope that would be of use.

How To Download / Export Sheets In Spreadheet Via Google Apps Script

The task is to automate the manual process accomplished by the menu option "File | Download As | Plain Text"
I want to be able to control the saved file name, which cannot be done via the menu.
At the time this is invoked, the user would be sitting on the sheet in the spreadsheet. Ultimately, I'd make it a menu option, but for testing I'm just creating a function that I can run manually.
After reading several other threads for possible techniques, this is what I've come up with.
It builds a custom name for the file, makes the call, and the response code is 200.
Ideally, I'd like to avoid the open / save dialog. In other words, just save the file without additional user intervention. I'd want to save in a specific folder and I've tried it with a complete file spec, but the result is the same.
If I copy the URL displayed in the Logger and paste it into a browser, it initiates the open / save dialog, so that string works.
Here's the code as a function.
function testExportSheet() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var oSheet = SpreadsheetApp.getActiveSpreadsheet().getActiveSheet();
var sId = ss.getId();
var ssID=sId + "&gid=" + oSheet.getSheetId();
var url = "https://spreadsheets.google.com/feeds/download/spreadsheets/Export?key="
+ ssID + "&exportFormat=tsv";
Logger.log(url);
var fn = ss.getName() + "-" + oSheet.getSheetName() + ".csv";
var sHeaders = {"Content-Disposition" : "attachment; filename=\"" + fn + "\""};
var sOptions = {"contentType" : "text/html", "headers" : sHeaders};
Logger.log(sOptions);
x = UrlFetchApp.fetch(url, sOptions)
Logger.log(x.getResponseCode());
}
I have exported a spreadsheet as CSV directly into a local hard drive as follows:
Get the CSV content from current sheet using a variation of function convertRangeToCsvFile_() from the tutorial on this page https://developers.google.com/apps-script/articles/docslist_tutorial#section3
var csvFile = convertRangeToCsvFile_(...);
Then select a drive folder that is syncing to a local computer using Drive
var localFolder = DocsList.getFolderById("055G...GM");
And finally save the CSV file into the "local" folder
localFolder.createFile("sample.csv", csvFile);
That's it.
This app script returns a file for download instead of web page to display:
function doGet(){
var outputDocument = DocumentApp.create('My custom csv file name');
var content = getCsv();
var textContent = ContentService.createTextOutput(content);
textContent.setMimeType(ContentService.MimeType.CSV);
textContent.downloadAsFile("4NocniMaraton.csv");
return textContent;
}
In case you are looking to export all of the sheets in s spreadsheet to csv without having to manually do it one by one, here's another thread about it:
Using the google drive API to download a spreadsheet in csv format
The download can be done. But not the "Write to the hard drive" of the computer.
Write issue:
You mean write a file to the hard drive of the computer, using Google Apps Script? Sorry, but you will need more than GAS to do this. For security reasons, I doubt this is possible with only GAS, have never seen anything like this in GAS.
Google Drive API will let you do a download, just needs OAuth and the URL you gave.
This is the function that I found when looking up the same question. This function was linked to by #Mogsdad and the link no longer exists.
function convertRangeToCsvFile_(csvFileName) {
// Get the selected range in the spreadsheet
var ws = SpreadsheetApp.getActiveSpreadsheet().getActiveSelection();
try {
var data = ws.getValues();
var csvFile = undefined;
// Loop through the data in the range and build a string with the CSV data
if (data.length > 1) {
var csv = "";
for (var row = 0; row < data.length; row++) {
for (var col = 0; col < data[row].length; col++) {
if (data[row][col].toString().indexOf(",") != -1) {
data[row][col] = "\"" + data[row][col] + "\"";
}
}
// Join each row's columns
// Add a carriage return to end of each row, except for the last one
if (row < data.length-1) {
csv += data[row].join(",") + "\r\n";
}
else {
csv += data[row];
}
}
csvFile = csv;
}
return csvFile;
}
catch(err) {
Logger.log(err);
Browser.msgBox(err);
}
}
I found it here and here