Google Script - How to use unzip - google-apps-script

I am downloading a .zip from a website. It contains one .txt file. I would like to access the data in the txt and write it to a spreadsheet. I'm open to either accessing it directly and not extracting the zip OR extracting the zip, saving the txt to a Google Drive Folder, and accessing it once it is saved.
When I use Utilities.unzip(), I can never get it to unzip the file and usually end up with an "Invalid argument" error. In the code below, the last section before else contains the unzip command. It successfully saves the file to the correct Google Folder but then I can't extract it.
function myFunction() {
// define where to gather data from
var url = '<insert url here>';
var filename = "ReportUploadTesting05.zip";
var response = UrlFetchApp.fetch(url, {
// muteHttpExceptions: true,
// validateHttpsCertificates: false,
followRedirects: true // Default is true anyway.
});
// get spreadsheet for follow up info
var Sp = SpreadsheetApp.getActiveSpreadsheet();
if (response.getResponseCode() === 200) {
// get folder details of spreadsheet for saving future files
var folderURL = getParentFolder(Sp);
var folderID = getIdFromUrl(folderURL);
var folder = DriveApp.getFolderById(folderID);
// save zip file
var blob = response.getBlob();
var file = folder.createFile(blob);
file.setName(filename);
file.setDescription("Downloaded from " + url);
var fileID = file.getId();
Logger.log(fileID);
Logger.log(blob)
// extract zip (not working)
file.setContent('application/zip')
var fileUnzippedBlob = Utilities.unzip(file); // invalid argument error occurs here
var filename = 'unzipped file'
var fileUnzipped = folder.createFile(fileUnzippedBlob)
fileUnzipped.setName(filename)
}
else {
Logger.log(response.getResponseCode());
}
}
I've followed the instructions on the Utilities page. I can get their exact example to work. I've tried creating a .zip on my computer, uploading it to Google Drive and attempted to open it unsuccessfully. Obviously there are some subtleties of using the unzip that I'm missing.
Could you help me understand this?

I was running into the same "Invalid arguments" error in my testing, so instead of using:
file.setContent('application/zip')
I used:
file.setContentTypeFromExtension();
And, that solved the problem for me. Also, as #tukusejssirs mentioned, a zip file can contain multiple files, so unzip() returns an array of blobs (as documented here). That means you either need to loop through the files, or if you know you only have one, explicitly reference it's position in the array, like this:
var fileUnzipped = folder.createFile(fileUnzippedBlob[0])
Here's my entire script, which covers both of these issues:
/**
* Fetches a zip file from a URL, unzips it, then uploads a new file to the user's Drive.
*/
function uploadFile() {
var url = '<url goes here>';
var zip = UrlFetchApp.fetch('url').getBlob();
zip.setContentTypeFromExtension();
var unzippedFile = Utilities.unzip(zip);
var filename = unzippedFile[0].getName();
var contentType = unzippedFile[0].getContentType();
var csv = unzippedFile[0];
var file = {
title: filename,
mimeType: contentType
};
file = Drive.Files.insert(file, csv);
Logger.log('ID: %s, File size (bytes): %s', file.id, file.fileSize);
var fileId = file.id;
// Move the file to a specific folder within Drive (Link: https://drive.google.com/drive/folders/<folderId>)
var folderId = '<folderId>';
var folder = DriveApp.getFolderById(folderId);
var driveFile = DriveApp.getFileById(fileId);
folder.addFile(driveFile);
}

I think the answer to your question may be found here. Is there a size limit to a blob for Utilities.unzip(blob) in Google Apps Script?
If the download is over 100 mb the full file cannot be downloaded. Due to that it will not be in the proper zip format. Throwing the cannot unzip file error.

I believe that the creation of the blob from a file (in this case the .zip file) requires the .next(); otherwise it did not work for me.
Also note that the .zip file might contain more than one file, therefore I included a for cycle.
Anyway, my working/tested solution/script is the following:
function unzip(folderName, fileZipName){
// Variables
// var folderName = "folder_name";
// var fileZipName = "file_name.zip";
var folderId = getFolderId(folderName);
var folder = DriveApp.getFolderById(folderId);
var fileZip = folder.getFilesByName(fileZipName);
var fileExtractedBlob, fileZipBlob, i;
// Decompression
fileZipBlob = fileZip.next().getBlob();
fileZipBlob.setContentType("application/zip");
fileExtractedBlob = Utilities.unzip(fileZipBlob);
for (i=0; i < fileExtractedBlob.length; i++){
folder.createFile(fileExtractedBlob[i]);
}
}

Related

How to convert all types of files format that downloaded from url to pdf by using Apps Script

So, I have a case that I need to change all types of format (JPEG, PNG, doc, etc) from file that has been downloaded from url to pdf.
The process that I want:
First, there is a url that being generated by some application to the spreadsheet.
This url contain a file with not specified format (So, it can be JPEG, png, doc, pdf, etc). I need to download the file from the url and convert it to pdf format. After convert the file to pdf format, then the converted file will be saved in google drive and the file link from google drive will be inserted on spreadsheet.
Notes:
So, If this kind of thing is impossible to do, then you can make the other way like if the file format is .csv, .xlsx, .xls, .png, .jpeg, .pdf, or .doc, then it will convert the file to .pdf.
This is the illustration of the spreadsheet that I want to make:
Url File
Drive link
https://..../file1.jpeg
https://drive/.../file1.pdf
https://..../file2.csv
https://drive/..../file2.pdf
I have already made the code, but my code is still get the error because I only change the format (.doc) from the filename (filename.doc -> filename.pdf) and when I download the file it will not open because of that. If you guys have any suggestion to fix my code or have any different answer, It can be very helpful! Thank you
Here is the code that I've made:
function convert() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheetByName("");
var data = sheet.getDataRange().getValues();
var folder_id = '';
var array = [];
for (let i in data.length) {
if (data[i][2] != "" && data[i][16] == "") { //condition for if column 3 is having a url and the drive link is still empty, then get the url
array.push([data[i][2]]);
var get_name = array.toString().split("/")[7]; //to get the file name
var deleteformat= get_name .toString().split(".")[0]; //to delete the format
var response = UrlFetchApp.fetch(array, { muteHttpExceptions: true });
var rc = response.getResponseCode();
if (rc == 200) {
var fileBlob = response.getBlob().setName(deleteformat+ '.pdf');
var folder = DriveApp.getFolderById(folder_id );
if (folder != null) {
var file = folder.createFile(fileBlob);
var fileName = file.getName();
}
}
var file = DriveApp.getFolderById(folder_id ).getFiles();
while (file.hasNext()) {
var xfile = file.next();
if (xfile.getName() == fileName) {
var file_id = xfile.getId();
var url = DriveApp.getFileById(file_id ).getUrl();
}
}
sheet.getRange(i + 1, 17).setValue(url);
sheet.getRange(i + 1, 3).clearContent();
}
}
}
Use Blob.getAs(), like this:
var fileBlob = response.getBlob().getAs('application/pdf').setName(deleteformat + '.pdf');
You will also have to fix the UrlFetchApp.fetch() line so that it references a URL rather an array.

How can i REPLACE a file with another file (if their name is the same) using 'Trash' ing in Google App Script?

I am accessing a list of folders from a shared drive.
In here, I am converting a few excel files into spreadsheet. My issue is to replace the old converted files with the new file. This is because every time i run the script the new converted file(with same name) keeps on multiplying in the same folder together with the old one.
Here is the code:
function ConvertFiles() {
var sheet =
SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Sheet1");
var r= 2;
for(r= 2;r < sheet.getLastRow(); r++){
// Use getValue instead of getValues
var fileId = sheet.getRange(r,1).getValue();
var folderID = sheet.getRange(r,8).getValue();
var files = DriveApp.getFileById(fileId);
var name = files.getName().split('.')[0];
var blob = files.getBlob();
var newFile = {
// Remove '_converted' from name if existing to avoid duplication of the string before adding '_converted'
// This will allow to have newly converted file "replace" the old converted file properly
title: name.replace('_converted','') + '_converted',
parents: [{
id: folderID
}]
};
var destinationFolderId = DriveApp.getFolderById(folderID);
var existingFiles = destinationFolderId.getFilesByName(newFile.title);
// GOAL #1: To replace/update the old converted file into the latest one everytime the script runs (if it has the same filename)
// Find the file with same name of the file to be converted
while(existingFiles.hasNext()) {
// ID of the file with same converted name
var oldConvertedFileWithSameNameID = existingFiles.next().getId();
// Delete before writing
Drive.Files.remove(oldConvertedFileWithSameNameID);
//DriveApp.getFileById(oldConvertedFileWithSameNameID.getId()).setTrashed(true);
}
// Create new converted file then get ID
var newFileID = Drive.Files.insert(newFile, blob, { convert: true,supportsAllDrives: true }).id;
Logger.log(newFileID);
//var sheetFileID = newFileID.getId();
//var Url = "https://drive.google.com/open?id=" + sheetFileID;
var Url = "https://drive.google.com/open?id=" + newFileID;
// Add the ID of the converted file
sheet.getRange(r,9).setValue(newFileID);
sheet.getRange(r,10).setValue(Url);
}
}
My goal is
To replace the old converted file with the new one(if they have the same name) into the shared drive folder
To get to know how can i implement the setTrashed() inside the above code
I have tried using the Drive.Files.remove(oldConvertedFileWithSameNameID); but I am getting an error message GoogleJsonResponseException: API call to drive.files.delete failed with error: File not found:("fileid"). Then i saw an question on this [https://stackoverflow.com/questions/55150681/delete-files-via-drive-api-failed-with-error-insufficient-permissions-for-this]...so i guess that method is not suitable to implemented in shared folder.
So i how can i use setTrashed() method inside the above code?
I think you need to set the supportsAllDrives parameter:
Drive.Files.remove(oldConvertedFileWithSameNameID, {supportsAllDrives: true});
References:
Files:delete | Google Drive API | Google Developers - Parameters

Drive.Files.Update - Uploading a new revision to maintain doc ID error

I'm trying to upload a new revision to an existing file (original file) and maintain the ID for that file.
I have the Google Drive API (v2) enabled in the Advanced Google Services. I have the script as a standalone script (not attached to a google doc or spreadsheet based on something I read that said the Drive.Files.Update doesn't work for those scripts).
On this line of code:
Drive.Files.update({
title: currentFile.getName(), mimeType: currentFile.getMimeType()
}, originalFileID, blobOfNewContent);
I get the following error:
Execution failed: We're sorry, a server error occurred. Please wait a bit and try again. (line 13, file "Code") [8.215 seconds total runtime]
Any clues as to what else I can try? Basically, I need to maintain the ID of the file and just update the contents every so often from other file.
function overwriteFile(blobOfNewContent, originalFileID)
{
var currentFile;
currentFile = DriveApp.getFileById(originalFileID);
if (originalFileID != "")
{//If original File exsts
Drive.Files.update({
title: currentFile.getName(), mimeType: currentFile.getMimeType()
}, originalFileID, blobOfNewContent);
}
}
function getNewestSlidesInFolderAndUploadToStaticFileID()
{
// fileID of file that is "Original File"
var mainfileID = "some id";
var folders = DriveApp.getFoldersByName('Testing Slides');
var arryFileDates = [];
var objFilesByDate = {};
//find most recent file
while (folders.hasNext())
{
var folder = folders.next();
var files = folder.getFilesByType("application/vnd.google-apps.presentation");
while (files.hasNext())
{
var file = files.next();
// Make sure the fileID is not the static file
if (file.getId() != mainfileID)
{
var fileDate = file.getLastUpdated();
objFilesByDate[fileDate] = file.getId(); //Create an object of file names by file ID
arryFileDates.push(file.getLastUpdated());
} // end if id = mainfileID
} //end while (files.hasNext())
arryFileDates.sort(function(a, b) { return b - a});
var newestDate = arryFileDates[0];
Logger.log('Newest date is: ' + newestDate);
var newestFileID = objFilesByDate[newestDate];
Logger.log('newestFile: ' + newestFileID);
//return newestFile;
}//while folders.hasNext()
//Get the contents of the most recent file
var newestFile = DriveApp.getFileById(newestFileID);
var blob = newestFile.getBlob();
overwriteFile(blob, mainfileID);
//delete most recent file so that only the main file exists
newestFile.setTrashed(true);
}
I did find all this code here on stackoverflow and I just modified (hardcoded) some pieces for my needs. Thanks to the original coders for their assistance.

Skip processing .xls to Google Sheets script if the file already exists in Google Drive

I am currently using this code to automatically convert all uploaded .xls files in Google Drive to Google Sheets.
function importXLS(){
var files = DriveApp.searchFiles('title contains ".xls"');
while(files.hasNext()){
var xFile = files.next();
var name = xFile.getName();
if (name.indexOf('.xls')>-1){
var ID = xFile.getId();
var xBlob = xFile.getBlob();
var newFile = { title : name,
key : ID,
'parents':[{"id":"12FcKokB-ppW7rSBtAIG96uoBOJtTlNDT"}]
}
file = Drive.Files.insert(newFile, xBlob, {
convert: true
});
}
}
}
It works perfectly, but fails if there is already a file in the output folder with the same name. Even though I never technically get to see this error below (since it runs on a schedule and not fired manually like in the screenshot), I would prefer to simply skip the conversion process if the file already exists.
If possible, I would also like to avoid overwriting it each time, as I feel that would be a waste of processing time. How would I edit this code to say that if the file name already exists in that folder, skip the entire code completely?
Thanks!
Two things you can try:
Get the files names that are already in the destination folder and check if the file exists before you try copying.
Wrap the section of your code that does the copying in a try..catch statement.
Both of these should work independently, but using the try..catch statement will catch all errors, so it would be best to combine them. (You can review the error logs in the Developer Console.) Doing this you'll be able to skip files that have the same name as those already in your destination folder and any other error that might come up will not terminate your script from completing.
function importXLS(){
var files = DriveApp.searchFiles('title contains ".xls"');
var destinationFolderId = "12FcKokB-ppW7rSBtAIG96uoBOJtTlNDT";
var existingFileNames = getFilesInFolder(destinationFolderId);
while(files.hasNext()){
var xFile = files.next();
var name = xFile.getName();
try {
if (!existingFileNames[name] && (name.indexOf('.xls')>-1)) {
var ID = xFile.getId();
var xBlob = xFile.getBlob();
var newFile = { title : name,
key : ID,
'parents':[{"id": destinationFolderId}]
}
file = Drive.Files.insert(newFile, xBlob, {
convert: true
});
}
} catch (error) {
console.error("Error with file " + name + ": " + error);
}
}
}
/**
* Get an object of all file names in the specified folder.
* #param {string} folderId
* #returns {Object} files - {filename: true}
*/
function getFilesInFolder(folderId) {
var folder = DriveApp.getFolderById(folderId);
var filesIterator = folder.getFiles();
var files = {};
while (filesIterator.hasNext()) {
var file = filesIterator.next();
files[file.getName()] = true;
}
return files;
}

Creating a zip file inside Google Drive with Apps Script

I have a folder in Google Drive folder containing few files. I want to make a Google Apps Script that will zip all files in that folder and create the zip file inside same folder.
I found a video that has Utilities.zip() function, but there is no API reference for that. How do I use it? Thanks in advance.
Actually it's even easier than that. Files are already Blobs (anything that has getBlob() can be passed in to any function that expects Blobs). So the code looks like this:
var folder = DocsList.getFolder('path/to/folder');
folder.createFile(Utilities.zip(folder.getFiles(), 'newFiles.zip'));
Additionally, it won't work if you have multiple files with the same name in the Folder... Google Drive folders support that, but Zip files do not.
To make this work with multiple files that have the same name:
var folder = DocsList.getFolder('path/to/folder');
var names = {};
folder.createFile(Utilities.zip(folder.getFiles().map(function(f){
var n = f.getName();
while (names[n]) { n = '_' + n }
names[n] = true;
return f.getBlob().setName(n);
}), 'newFiles.zip'));
As DocsList has been deprecated, You can use the following code to zip an entire folder containing files and sub-folders and also keep its structure:
var folder = DriveApp.getFolderById('<YOUR FOLDER ID>');
var zipped = Utilities.zip(getBlobs(folder, ''), folder.getName()+'.zip');
folder.getParents().next().createFile(zipped);
function getBlobs(rootFolder, path) {
var blobs = [];
var files = rootFolder.getFiles();
while (files.hasNext()) {
var file = files.next().getBlob();
file.setName(path+file.getName());
blobs.push(file);
}
var folders = rootFolder.getFolders();
while (folders.hasNext()) {
var folder = folders.next();
var fPath = path+folder.getName()+'/';
blobs.push(Utilities.newBlob([]).setName(fPath)); //comment/uncomment this line to skip/include empty folders
blobs = blobs.concat(getBlobs(folder, fPath));
}
return blobs;
}
getBlobs function makes an array of all files in the folder and changes each file name to it's relative path to keep structure when became zipped.
To zip a folder containing multiple items with the same name use this getBlob function:
function getBlobs(rootFolder, path) {
var blobs = [];
var names = {};
var files = rootFolder.getFiles();
while (files.hasNext()) {
var file = files.next().getBlob();
var n = file.getName();
while(names[n]) { n = '_' + n }
names[n] = true;
blobs.push(file.setName(path+n));
}
names = {};
var folders = rootFolder.getFolders();
while (folders.hasNext()) {
var folder = folders.next();
var n = folder.getName();
while(names[n]) { n = '_' + n }
names[n] = true;
var fPath = path+n+'/';
blobs.push(Utilities.newBlob([]).setName(fPath)); //comment/uncomment this line to skip/include empty folders
blobs = blobs.concat(getBlobs(folder, fPath));
}
return blobs;
}
I was able to use the code that #Hafez posted but I needed to modify it because It was not working for me. I added the first three lines because I needed the folder ID which is a string value and is not the name of the folder.
var folderName = DriveApp.getFoldersByName("<folderName>");
var theFolder = folderName.next();
var folderID =theFolder.getId();
var folder = DriveApp.getFolderById(folderID);
var zipped = Utilities.zip(getBlobs(folder, ''), folder.getName()+'.zip');
folder.getParents().next().createFile(zipped);
function getBlobs(rootFolder, path) {
var blobs = [];
var files = rootFolder.getFiles();
while (files.hasNext()) {
var file = files.next().getBlob();
file.setName(path+file.getName());
blobs.push(file);
}
var folders = rootFolder.getFolders();
while (folders.hasNext()) {
var folder = folders.next();
var fPath = path+folder.getName()+'/';
blobs.push(Utilities.newBlob([]).setName(fPath)); //comment/uncomment this line to skip/include empty folders
blobs = blobs.concat(getBlobs(folder, fPath));
}
return blobs;
}
The only weird thing that I'm experiencing is that when I run the script it says TypeError: Cannot call method "getFiles" of undefined. (line 10, file "Code"). When I happened to look at the place where this script lives there was also 5 zip files that were complete. It works but I still get that error. Weird...but this code works for me. Thanks to everyone on this thread. Cheers!
There's no API reference indeed. You could open an issue request regarding this on Apps Script issue tracker. But deducing from what the code-completion shows, here is my understanding:
var folder = DocsList.getFolder('path/to/folder');
var files = folder.getFiles();
var blobs = [];
for( var i in files )
blobs.push(files[i].getBlob());
var zip = Utilities.zip(blobs, 'newFiles.zip');
folder.createFile(zip);
But I have not tested this code, so I don't know if it will work. Also, it may work only for files not converted to Google's format, or maybe only for those or a subset of it. Well, if you try it out and find something, please share here with us. One limit that you'll sure face is the filesize, it will probably not work if the zip file gets "too" big... yeah, you'll have to test this limit too.
If Hafez solution didn't worked out, and you get this error
TypeError: Cannot read property 'getFiles' of undefined
Try doing this
/**
* Creates a zipFile of the mentioned document ID and store it in Drive. You can search the zip by folderName.zip
*/
function createAndSendDocument() {
var files = DriveApp.getFolderById("DOCUMENT ID CAN BE FIND IN THE URL WHEN A DOCUMENT IS OPENED");
var folder = files;
var zipped = Utilities.zip(getBlobs(folder, ''), folder.getName() + '.zip');
folder.getParents().next().createFile(zipped);
}
function getBlobs(rootFolder, path) {
var blobs = [];
var files = rootFolder.getFiles();
while (files.hasNext()) {
var file = files.next().getBlob();
file.setName(path+file.getName());
blobs.push(file);
}
var folders = rootFolder.getFolders();
while (folders.hasNext()) {
var folder = folders.next();
var fPath = path+folder.getName() + '/';
blobs.push(Utilities.newBlob([]).setName(fPath)); //comment/uncomment this line to skip/include empty folders
blobs = blobs.concat(getBlobs(folder, fPath));
}
return blobs;
}