I'm trying to automatically generate manifest files with gulp, but can't find how to get filename, modify it and send it forward through pipe.
var fs = require('fs');
var content = 'test';
gulp.src('./wwwroot/**/*.file')
.pipe(fs.writeFileSync(??? , content))
.pipe(gulp.dest('./wwwroot/')); // should be same as original file
Where ??? on 4th line is, I'd like to have filename.file.manifest.
Code above is more of an idea, since gulp.dest and fs both write file.
Not with pipes, but nonetheless a nice solution using node glob
var content = 'test'
glob("./wwwroot/**/*.file", null, function(er, files) {
if(er)
return;
files.forEach(function(element) {
var manifestFile = element + '.manifest';
fs.writeFileSync(manifestFile, content);
console.log("Generated: " + element);
});
})
Related
I'm trying to create script for creating contracts. For this will be in 'Template File which is going to be text file on gDrive' example:
Client's VAT: $VAT
Client's Name: $Clients_name
How do I get this file content ( getBody(), which works only on Document Types, but you are not able to access Document instance, because all possibilities I've found lead me to File instance.. )
My target is to get exact copy of 'Template File' and go through him and change words starting with dollar, changing them by values from HTTP request. ( this part is already done. )
EDIT:
you might be interested about my try ( or it can help someone )
function createContract(obj)
{
const templateName = "RC - Contract Template";
const templateFile = getTemplate(templateName);
const templateText = templateFile.getAs("plain/text");
const file = DocumentApp.create(obj.filename);
const body = file.getBody();
const url = file.getUrl();
body.insertParagraph(0, templateText);
HtmlService.createHtmlOutput(url);
}
Note: getTemplate() is function going through all files and returning instance of File with name 'RC - Contract Template'
After some searching I've found out, that google has a big difference between "File", "Document", etc etc.
If someone gets in same situation, this is my way to get Content of file, when you know only name:
function getTemplateId(filename) {
const files = DriveApp.getFilesByType(MimeType.GOOGLE_DOCS);
while (files.hasNext()) {
var file = files.next();
if (file.getName() == filename) return file.getId();
}
return HtmlService.createHtmlOutput("Required template: " + filename + " does not exists!");
}
Note:Moved Solution posted by OP from question:
I am downloading a .zip from a website. It contains one .txt file. I would like to access the data in the txt and write it to a spreadsheet. I'm open to either accessing it directly and not extracting the zip OR extracting the zip, saving the txt to a Google Drive Folder, and accessing it once it is saved.
When I use Utilities.unzip(), I can never get it to unzip the file and usually end up with an "Invalid argument" error. In the code below, the last section before else contains the unzip command. It successfully saves the file to the correct Google Folder but then I can't extract it.
function myFunction() {
// define where to gather data from
var url = '<insert url here>';
var filename = "ReportUploadTesting05.zip";
var response = UrlFetchApp.fetch(url, {
// muteHttpExceptions: true,
// validateHttpsCertificates: false,
followRedirects: true // Default is true anyway.
});
// get spreadsheet for follow up info
var Sp = SpreadsheetApp.getActiveSpreadsheet();
if (response.getResponseCode() === 200) {
// get folder details of spreadsheet for saving future files
var folderURL = getParentFolder(Sp);
var folderID = getIdFromUrl(folderURL);
var folder = DriveApp.getFolderById(folderID);
// save zip file
var blob = response.getBlob();
var file = folder.createFile(blob);
file.setName(filename);
file.setDescription("Downloaded from " + url);
var fileID = file.getId();
Logger.log(fileID);
Logger.log(blob)
// extract zip (not working)
file.setContent('application/zip')
var fileUnzippedBlob = Utilities.unzip(file); // invalid argument error occurs here
var filename = 'unzipped file'
var fileUnzipped = folder.createFile(fileUnzippedBlob)
fileUnzipped.setName(filename)
}
else {
Logger.log(response.getResponseCode());
}
}
I've followed the instructions on the Utilities page. I can get their exact example to work. I've tried creating a .zip on my computer, uploading it to Google Drive and attempted to open it unsuccessfully. Obviously there are some subtleties of using the unzip that I'm missing.
Could you help me understand this?
I was running into the same "Invalid arguments" error in my testing, so instead of using:
file.setContent('application/zip')
I used:
file.setContentTypeFromExtension();
And, that solved the problem for me. Also, as #tukusejssirs mentioned, a zip file can contain multiple files, so unzip() returns an array of blobs (as documented here). That means you either need to loop through the files, or if you know you only have one, explicitly reference it's position in the array, like this:
var fileUnzipped = folder.createFile(fileUnzippedBlob[0])
Here's my entire script, which covers both of these issues:
/**
* Fetches a zip file from a URL, unzips it, then uploads a new file to the user's Drive.
*/
function uploadFile() {
var url = '<url goes here>';
var zip = UrlFetchApp.fetch('url').getBlob();
zip.setContentTypeFromExtension();
var unzippedFile = Utilities.unzip(zip);
var filename = unzippedFile[0].getName();
var contentType = unzippedFile[0].getContentType();
var csv = unzippedFile[0];
var file = {
title: filename,
mimeType: contentType
};
file = Drive.Files.insert(file, csv);
Logger.log('ID: %s, File size (bytes): %s', file.id, file.fileSize);
var fileId = file.id;
// Move the file to a specific folder within Drive (Link: https://drive.google.com/drive/folders/<folderId>)
var folderId = '<folderId>';
var folder = DriveApp.getFolderById(folderId);
var driveFile = DriveApp.getFileById(fileId);
folder.addFile(driveFile);
}
I think the answer to your question may be found here. Is there a size limit to a blob for Utilities.unzip(blob) in Google Apps Script?
If the download is over 100 mb the full file cannot be downloaded. Due to that it will not be in the proper zip format. Throwing the cannot unzip file error.
I believe that the creation of the blob from a file (in this case the .zip file) requires the .next(); otherwise it did not work for me.
Also note that the .zip file might contain more than one file, therefore I included a for cycle.
Anyway, my working/tested solution/script is the following:
function unzip(folderName, fileZipName){
// Variables
// var folderName = "folder_name";
// var fileZipName = "file_name.zip";
var folderId = getFolderId(folderName);
var folder = DriveApp.getFolderById(folderId);
var fileZip = folder.getFilesByName(fileZipName);
var fileExtractedBlob, fileZipBlob, i;
// Decompression
fileZipBlob = fileZip.next().getBlob();
fileZipBlob.setContentType("application/zip");
fileExtractedBlob = Utilities.unzip(fileZipBlob);
for (i=0; i < fileExtractedBlob.length; i++){
folder.createFile(fileExtractedBlob[i]);
}
}
Is it possible to remove files with same name from source? For example, let's say I have the following folder structure
a
---file1.txt
---file2.txt
---file3.txt
b
---file1.txt
When I select both folder in source I want in destination folder only file that aren't duplicates. In example above result would be
result
---file2.txt
---file3.txt
Optional, it would be great if I could duplicates somehow filter and write in separate folder.
By duplicates, I mean explicitly duplicates by name, file content is not important.
It took me awhile to get there but try this:
var gulp = require('gulp');
var fs = require('fs');
var path = require('path');
var flatten = require('gulp-flatten');
var filter = require('gulp-filter');
var folders = ['a', 'b', 'c']; // I just hard-coded your folders here
// this function is called by filter for each file in the above folders
// it should return false if the file is a duplicate, i.e., occurs
// in at least two folders
function isUnique(file) {
console.dir(file.history[0]); // just for fun
var baseName = file.history[0].split(path.sep);
baseName = baseName[baseName.length - 1];
// var fileParents = '././';
var fileParents = '.' + path.sep + '.' + path.sep;
var count = 0;
folders.forEach(function (folder) {
if (fs.existsSync(fileParents + folder + path.sep + baseName)) count++;
// could quit forEach when count >= 2 if there were a lot of folders/files
// but there is no way to break out of a forEach
});
if (count >= 2) { // the file is a duplicate
fs.unlinkSync(file.history[0]); // remove from 'Result' directory
return false;
}
else return true;
}
gulp.task('default', ['clump'], function () {
// create a filter to remove duplicates
const f = filter(function (file) { return isUnique(file); }, {restore: true, passthrough: false} );
const stream = gulp.src('./result/*.txt')
.pipe(f); // actually do the filtering here
f.restore.pipe(gulp.dest('duplicates')); // new stream with the removed duplicates
return stream;
});
// 'clump' runs first
// gathers all files into result directory
gulp.task('clump', function () {
return gulp.src('./**/*.txt')
.pipe(flatten()) // because the original folder structure in not wanted
.pipe(gulp.dest('result'));
});
Run it with 'gulp'. The default task will trigger the 'clump' task first.
Since your OP didn't require that any particular version of duplicated files be kept - like the newest or whatever - I haven't worried about that here. If in the 'Result' folder you want each version of a duplicated file, such as file1.txt (version from one folder) and file1.txt (from another folder) but obviously must be renamed to something that could be done in the 'clump' task.
Let me know if this works for you.
I wrote this code to iterate from the files of a folder:
function showList() {
var folder = DocsList.getFolderById('0B9HEC6UUJ_rsYWNPYko0MsrBRU0');
var files = folder.getFiles();
Logger.log("files = " + files);
arrayList = [];
for (var file in files) {
Logger.log("file = " + file);
var fileName = file.getName();
var fileId = file.getId();
var newArray = [fileName, "some info", fileId];
arrayList.push(newArray);
}
But in this line var fileName = file.getName();, I got this error: TypeError: Cannot find function getName in object 0.
The logs show this:
It seems there are files, but not the file that should get in the for loop. How to fix that?
Many problems in your js code:
1) thats not how you use 'in' in js. File will be an index so you need to do files[file]
2) even then its still wrong because iterating an array with 'in' will give you other things like the 'length' property.
Look up in the web how to iterate a js array.
How do I locate the path of the current folder? I just want to be able to get the path of the folder so that I can manipulate the files in the folder without typing the path into the scripts.
For a Spreadsheet I found this to work:
thisFileId = SpreadsheetApp.getActive().getId();
var thisFile = DriveApp.getFileById(thisFileId);
var parentFolder = thisFile.getParents()[0].getName();
Thanks to Corey's answer and to Thomas'one, here is a "full featured" version that shows the folder tree in the logger and every parents id as well... just for fun ;-)
function getScriptFolderTree() {
var thisScript = getThisScriptInDrive();
var names = []
var Ids = []
var folder = thisScript.getParents()[0];
while (folder.getName() != "Root"){
names.unshift(folder.getName());
Ids.unshift(folder.getId());
var parents = folder.getParents();
var folder = parents[0];
}
Logger.log('Root/'+names.join().replace(/,/g,'/'))
Ids.unshift(DriveApp.getRootFolder().getId())
Logger.log(Ids)
}
function getThisScriptInDrive() {
return DriveApp.getFileById("poiuytrezazertyujhgfdsdcvcxyydryfhchfh");
}
(ID's are truncated intentionally)
Note that this script is working nicely but it strangely stops working if the 'random string' is modified... I imagine that the search engine in drive doesn't like the change but I have no serious explanation (comments welcome) but after a few minutes it works again ;-)
Add a function like this to your script
function getThisScriptInDrive() {
return DriveApp.find("some unique string that wont be anywhere else")[0];
}
This will search Drive and find this script itself because it contains that string - right there in the function call! - no need to declare it anywhere else. As long as you use an obscure enough string - i'd recommend mashing a few hundred chars on your keyboard - it will be unique across drive and therefore just work.
Once you have a File for this script, you can call getParents() etc.
I tweaked Serge's code to write a generic function
function getThisFilePath() {
//let thisFile = DriveApp.getFileById(DocumentApp.getActiveDocument().getId()); // for a Doc
let thisFile = DriveApp.getFileById(SpreadsheetApp.getActiveSpreadsheet().getId()); // for a Sheet
let path = {name: '', folder: []};
let folder = thisFile.getParents().hasNext() ? thisFile.getParents().next() : undefined;
while (folder) {
path.folder.unshift({name: folder.getName(), id: folder.getId()});
folder = folder.getParents().hasNext() ? folder.getParents().next() : undefined;
}
path.folder.forEach(folder => path.name += '/' + folder.name);
Logger.log(path.name);
return path;
}
Logging output
11:02:57 AM Info /My Drive/Technology/Development/2022
You could do this:
function myFunction() {
var thisScript = getThisScriptInDrive();
var folder = thisScript.getParents()[0];
while (folder.getName() != "Root"){
var parents = folder.getParents();
for (var i in parents){
var folder = parents[i];
Logger.log(folder.getName());
}
}
}
function getThisScriptInDrive() {
return DocsList.find("`<jj!?=(<DW+.W/m7SBF:sgu/#B(&Cs3:{ajA~ys#KmN4&]ujhpZ~z[Tv?+dk}MpK,8pY=w&dny8N'74:.9H:~uCgY=7pRt4[Tn5")[0];
}
This only works good if the folder has only 1 parent because it only takes 1 path.
edit: Thanks to Corey G
I was able to get the folder containing the script that was running:
//
// PrintScriptFolder -- print the name of the folder in which the running script resides.
//
function PrintScriptFolder()
{
var scriptId = ScriptApp.getScriptId();
console.info('scriptId = ' + scriptId);
var file = DriveApp.getFileById(scriptId);
var folders = file.getParents();
if (folders.hasNext())
{
var folder = folders.next();
var name = folder.getName();
console.info('script folder name = ' + name);
}
}
The trick is using ScriptApp to get the ID of the running script. From there it's pretty straightforward: use DriveApp to get the file ID, getParents() to get a list of (one) parent folder, next() to get it, and getName() to get its name.
First answer, long time lurker (be kind ;-)
Looking for a simple getPath, I went for a more generic approach, with limitations:
//simple test stub
function test_getFilePath() {
const scriptId = ScriptApp.getScriptId();
const fullPath = getFilePath(scriptId);
console.log(fullPath);
}
/**
* getFilePath
* #fileId id of file to find path for
*
* NB: google allows for multiple parents, which is a PITA
* so I'm just taking the first one I find up the tree
*/
function getFilePath(fileId) {
const file = DriveApp.getFileById(fileId);
const rootFolder = DriveApp.getRootFolder()
const rootFolderId = rootFolder.getId();
const rootFolderName = rootFolder.getName();
let folder = file.getParents().next();
let folderNames = [];
while (folder.getId() !== rootFolderId){
folderNames.unshift(folder.getName());
folder = folder.getParents().next();
}
const fullPath = rootFolderName+'/'+folderNames.join().replace(/,/g,'/');
return fullPath;
}
console:
11:36:40 AM Info My Drive/GAS
Had I more time/inclination, I'd have writen a recursive function to capture and return an array of possible parent paths, but there's no use case for me and who the hell wants to keep a the same file in multiple folders anyway??
(I'm sure there are use cases, eg, equivalent to using symlinks, but it stil makes me feel dirty ;-)