How to zip up zip files using Gulp Zip - gulp

I've started using Gulp JS and must admit I'm finding it really useful.
One of the tasks I need to perform is zip up a collection of folders into individual zip files, one for each folder and then zip all this zipped files up into one single zip file. Using Gulp-Zip I've managed to get this far:
var modelFolders = [
'ELFH_Check',
'ELFH_DDP',
'ELFH_Free'
];
gulp.task('zipModels', function () {
for (var i = 0; i < modelFolders.length; i++) {
var model = modelFolders[i];
gulp.src('**/*', {cwd: path.join(process.cwd(), '/built_templates/' + model) })
.pipe(zip(model + '.zip'))
.pipe(gulp.dest('./built_templates'));
};
});
This works and outputs ELFH_Check.zip, ELFH_DDP.zip and ELFH_Free.zip. However, I then need to zip up these zip files into one zip file called "Templates.zip" and I've not managed to get this task to work:
// zip up model files
gulp.task('zipTemplate', ['zipModels'], function () {
gulp.src('*.zip', {cwd: path.join(process.cwd(), './built_templates/') })
.pipe(zip('Templates_.zip'))
.pipe(gulp.dest('./built_templates'));
});
Does anyone know if this is possible or what I'm doing wrong?

I saw the problem as well, and it seems to be related to the cwd option somehow. I'll investigate further.
After #OverZealous comment, I investigated further and found two issues:
As he said, you need to hint gulp to wait until the end of the dependency task (zipModels), by returning a stream from it. As you have multiple streams, you can use event-stream.merge to return a bundle stream.
The reason why the bundle zip wouldn't work, is because you cwd points to /built_templates/, and the second slash is causing some problem. To work properly, you need to remove the trailing slash, so it should be path.join(process.cwd(), '/built_templates').
IMPORTANT
Anyway, you should avoid temporary files. Gulp philosophy is to try using pipes to avoid IO. In that direction, what you want to do is to cut the intermediary dest steps, merge the streams, zip them, and finally, output them.
Something like that:
var es = require('event-stream');
var modelFolders = [
'ELFH_Check',
'ELFH_DDP',
'ELFH_Free'
];
gulp.task('zipModels', function () {
var zips = [],
modelZip;
for (var i = 0; i < modelFolders.length; i++) {
var model = modelFolders[i];
modelZip = gulp.src('**/*', {cwd: path.join(process.cwd(), '/built_templates/' + model) })
.pipe(zip(model + '.zip'));
// notice we removed the dest step and store the zip stream (still in memory)
zips.push(modelZip);
};
// we finally merge them (the zips), zip them again, and output.
return es.merge.apply(null, zips)
.pipe(zip('templates.zip'))
.pipe(gulp.dest('./'));
});
By the name of your folder (built_templates), it seems you have some other task that will generate the temporary built files. Preferably, you don't want these as well. You should pipe their streams directly to the ZIP stream, a finally, to the bundle-zip stream. By doing that, you would have a simple stream flow, with one disk read, and one disc write at the end, with no temporary files.
If you need them to be different tasks, consider having a function that will generate the stream up to the step before the gulp.dest pipe, and use this function on all subtasks.
Additionally, always try to hint your async tasks by returning a stream, a promise or receiving a callback function, and advise the end of the task.

Related

How to push a gulp pipeline onto another

I would like to process a series of folders through the build pipeline. For each folder I start another gulp pipeline that will create some source files. Finally I'd like to pipe them all together through the zip() plugin to produce a final artifact.
The following pseudocode shows my approach. However, I cannot push the result of the gulp.src to the current pipeline.
return gulp.src(someFolder)
.pipe(through(function(folder, enc, cb){
// how to add the result of the following line to my
// own pipeline????
this.push(gulp.src(path.join(folder, "whatever"))
.pipe(somePlugin()));
}))
.pipe(zip(....))
.pipe(gulp.dest(....));
Any hints how to combine those pipelines together?
This is exactly what gulp-foreach is intended for. Using that the equivalent of the code you posted would look like this:
var foreach = require('gulp-foreach');
return gulp.src(someFolder)
.pipe(foreach(function(stream, folder) {
return gulp.src(path.join(folder.path, "whatever"))
.pipe(somePlugin());
}))
.pipe(zip(....))
.pipe(gulp.dest(....));
Note that the above will discard all files from the original gulp.src(someFolder). They will not be included in the ZIP. If you want to keep those around as well you can use merge-stream:
var foreach = require('gulp-foreach');
var merge = require('merge-stream');
return gulp.src(someFolder)
.pipe(foreach(function(stream, folder) {
return merge(stream,
gulp.src(path.join(folder.path, "whatever"))
.pipe(somePlugin()));
}))
.pipe(zip(....))
.pipe(gulp.dest(....));

Gulp: how to pass parameters from watch to tasks

With gulp you often see patterns like this:
gulp.watch('src/*.jade',['templates']);
gulp.task('templates', function() {
return gulp.src('src/*.jade')
.pipe(jade({
pretty: true
}))
.pipe(gulp.dest('dist/'))
.pipe( livereload( server ));
});
Does this actually pass the watch'ed files into the templates task? How do these overwrite/extend/filter the src'ed tasks?
I had the same question some time ago and came to the following conclusion after digging for a bit.
gulp.watch is an eventEmitter that emits a change event, and so you can do this:
var watcher = gulp.watch('src/*.jade',['templates']);
watcher.on('change', function(f) {
console.log('Change Event:', f);
});
and you'll see this:
Change Event: { type: 'changed',
path: '/Users/developer/Sites/stackoverflow/src/touch.jade' }
This information could presumably be passed to the template task either via its task function, or the behavior of gulp.src.
The task function itself can only receive a callback (https://github.com/gulpjs/gulp/blob/master/docs/API.md#fn) and cannot receive any information about vinyl files (https://github.com/wearefractal/vinyl-fs) that are used by gulp.
The source starting a task (.watch in this case, or gulp command line) has no effect on the behavior of gulp.src('src-glob', [options]). 'src-glob' is a string (or array of strings) and options (https://github.com/isaacs/node-glob#options) has nothing about any file changes.
Hence, I don't see any way in which .watch could directly affect the behavior of a task it triggers.
If you want to process only the changed files, you can use gulp-changed (https://www.npmjs.com/package/gulp-changed) if you want to use gulp.watch, or you cold use gulp-watch.
Alternatively, you could do this as well:
var gulp = require('gulp');
var jade = require('gulp-jade');
var livereload = require('gulp-livereload');
gulp.watch('src/*.jade', function(event){
template(event.path);
});
gulp.task('templates', function() {
template('src/*.jade');
});
function template(files) {
return gulp.src(files)
.pipe(jade({
pretty: true
}))
.pipe(gulp.dest('dist/'))
}
One of the possible way to pass a parameter or a data from your watcher to a task. Is through using a global variable, or a variable that is in both blocks scops. Here is an example:
gulp.task('watch', function () {
//....
//json comments
watch('./app/tempGulp/json/**/*.json', function (evt) {
jsonCommentWatchEvt = evt; // we set the global variable first
gulp.start('jsonComment'); // then we start the task
})
})
//global variable
var jsonCommentWatchEvt = null
//json comments task
gulp.task('jsonComment', function () {
jsonComment_Task(jsonCommentWatchEvt)
})
And here the function doing the task work in case it interest any one, But know i didn't need to put the work in such another function i could just implemented it directly in the task. And for the file you have your global variable. Here it's jsonCommentWatchEvt. But know if you don't use a function as i did, a good practice is to assign the value of the global variable to a local one, that you will be using. And you do that at the all top entry of the task. So you will not be using the global variable itself. And that to avoid the problem that it can change by another watch handling triggering. When it stay in use by the current running task.
function jsonComment_Task(evt) {
console.log('handling : ' + evt.path);
gulp.src(evt.path, {
base: './app/tempGulp/json/'
}).
pipe(stripJsonComments({whitespace: false})).on('error', console.log).
on('data', function (file) { // here we want to manipulate the resulting stream
var str = file.contents.toString()
var stream = source(path.basename(file.path))
stream.end(str.replace(/\n\s*\n/g, '\n\n'))
stream.
pipe(gulp.dest('./app/json/')).on('error', console.log)
})
}
I had a directory of different json's files, where i will use comments on them. I'm watching them. When a file is modified the watch handling is triggered, and i need then to process only the file that was modified. To remove the comments, i used json-comment-strip plugin for that. Plus that i needed to do a more treatment. to remove the multiple successive line break. Whatever, at all first i needed to pass the path to the file that we can recover from the event parameter. I passed that to the task through a global variable, that does only that. Allow passing the data.
Note: Even though that doesn't have a relation with the question, in my example here, i needed to treat the stream getting out from the plugin processing. i used the on("data" event. it's asynchronous. so the task will mark the end before the work completely end (the task reach the end, but the launched asynchronous function will stay processing a little more). So the time you will get in the console at task end, isn't the time for the whole processing, but task block end. Just that you know. For me it doesn't matter.

how to modify config files using gulp

I use gulp to configure complex local setup and need to auto-edit files.
The scenario is:
determine if certain file contains certain lines after certain other line (found using regular expression)
if line is not found, insert the line.
optionally, delete some lines found in the file.
I need this to amend system configuration files and compile scenarios.
What would be the best way to do it in gulp?
Gulp is plain javascript. So what I would do if I were you is to create a plugin to pipe to the original config file.
Gulp streams emit Vinyl files. So all you really got to do is to create a "pipe factory" that transforms the objects.
It would look something like this (using EventStream):
var es = require('event-stream');
// you could receive params in here if you're using the same
// plugin in different occasions.
function fixConfigFile() {
return es.map(function(file, cb) {
var fileContent = file.contents.toString();
// determine if certain file contains certain lines...
// if line is not found, insert the line.
// optionally, delete some lines found in the file.
// update the vinyl file
file.contents = new Buffer(fileContent);
// send the updated file down the pipe
cb(null, file);
});
}
gulp.task('fix-config', function() {
return gulp.src('path/to/original/*.config')
.pipe(fixConfigFile())
.pipe(gulp.dest('path/to/fixed/configs');
});
Or you can use vinyl-map:
const map = require('vinyl-map')
const gulp = require('gulp')
const modify = map((contents, filename) => {
contents = contents.toString()
// modify contents somehow
return contents
})
gulp.task('modify', () =>
gulp.src(['./index.js'])
.pipe(modify)
.pipe(gulp.dest('./dist'))
})

How to pass a parameter to gulp-watch invoked task

I am trying to pass a parameter to a task that is being invoked by gulp-watch. I need it because I am trying to build a modular framework.
So if a file changes in module 1, the other modules don't need to be rebuild.
And I want just one function to create the concatted & uglified files per module.
This is what I got so far:
//here I need the 'module' parameter
gulp.task('script', function(module) { ... }
gulp.task('watch', function() {
gulp.watch('files/in/module1/*.js', ['script']); //here I want to pass module1
gulp.watch('files/in/module2/*.js', ['script']); //here I want to pass module2
});
A lot of the documentation/examples seems to be outdated (gulp.run(), gulp.start()).
I hope someone can help me out here.
I had the very same issue, searched for a while, and the "cleanest" way I came up with, uses the .on() event handler of gulp.watch(), and the .env property of gulp-util:
var gulp = require('gulp');
$.util = require('gulp-util');
var modules = {
module1: {}, // awesome module1
module2: {} // awesome module2
};
gulp.task('script', function(){
var moduleName = $.util.env.module;
// Exit if the value is missing...
var module = modules[moduleName];
if (!module) {
$.util.log($.util.colors.red('Error'), "Wrong module value!");
return;
}
$.util.log("Executing task on module '" + moduleName + "'");
// Do your task on "module" here.
});
gulp.task('watch', function () {
gulp.watch(['files/in/module1/*.js'], ['script']).on('change', function () {
$.util.env.module = 'module1';
});
gulp.watch(['files/in/module2/*.js'], ['script']).on('change', function () {
$.util.env.module = 'module2';
});
});
gulp-util also comes in handy if you need to pass (global) parameters from the shell:
[emiliano#dev ~]# gulp script --module=module1 --minify
Hope this helps someone else out there!
Regards.
In that i will answer directly the question "How to pass a parameter to gulp-watch invoked task"
My way of doing, and one of the possibility i see, is to use a global variable to pass the value between the two blocks. you set it just before launching the task in the watcher. And in the task, just at the start you pass it to a local variable.
See this answer for more details: https://stackoverflow.com/a/49733123/7668448
In what you want to achieve, you can too use just one watcher over the directory that hold all modules. If so is the structure. Then when a change happen, you can recover the changed file path. From that you can deduce what module does belong to. By getting the Module folder. That way you will not need to add a new watcher for each new module. Which can be nice when there is multiple contributors to the project for example when working on open source. And you do it one time, and don't have to care about adding anything. Just like with the delegation principle, with DOM event handling when there is multiple elements. Even if the chosen structure, doesn't have all the modules in one directory. You can stay pass multiple globs to the one watcher.
gulp.watch(['glob1/**/*.js', 'glob2/**/*.js',...], function(evt) {/*.....*/});
And following the structure you have, you can work your way to deduce what module is.
For the watcher here how i suggest you do it:
watch('./your/allModulesFolder/**/*.js', function (evt) {
rebuildModulWatchEvt = evt; //here you update the global var
gulp.start('rebuildModul'); // you start the task
})
The evt here hold multiple info: cwd, base, state, _contents ...etc And what interest us is path. So evt.path will give you the path of the changed file.
In your task either you do that:
gulp.task('rebuildModul', function() {
let evt = rebuildModulWatchEvt; // at all start you pass it to a local var
let filePath = evt.path; // how you get the changed file path
// your code go here for the rest, following your structure, get the path for the module folder
});
or you use a function :
gulp.task('rebuildModul', function() {
rebuildModulTaskRun(rebuildModulWatchEvt);
});
function rebuilModulTaskRun(evt) {
let filePath = evt.path;
// your code go here for the rest, following your structure, get the path for the module folder
}

How to start Gulp task with params?

I need to apply a build task for specific files. For finding them, I use the typical template. But I can't understood how to pass the arguments (file path) from gulp.src.
Desirable solution.
gulp.task('bundles', function() {
gulp.src('bundles/**/*.js').
pipe(gulp.start('build', file.path));
});
gulp.task('build', function (path) {
// use here
});
Question is a bit stale and I am not sure I totally understand what you're trying to achieve here, but I think what you're looking for is lazypipe
You might want to clarify your question if that's not what you're looking for
Example Usage:
var lazypipe = require('lazypipe'),
g = require('gulp-load-plugins')({lazy: true}),
jsTransformPipe = lazypipe()
.pipe(g.jshint) // <-- Notice the notation: g.jshint, not g.jshint()
.pipe(g.concat, 'bundle.js'), // <-- Notice how the param is passed to g.concat, as a second param to .pipe()
jsSourcePipe = lazypipe()
.pipe(gulp.src, './**/*.js');
gulp.task('bundle', function() {
jsSourcePipe()
.pipe(jsTransformPipe()) // <-- You execute the lazypipe by calling it as a function
.pipe(gulp.dest('../build/');
});
With lazypipe you basically create a pipe for future use; hope this help
(Can't comment because of rep, sorry)
I assume that your sample code isn't filled with everything, but why don't you merge those tasks and use your gulp.src() in your build task instead of calling another task.
Maybe it's useful for you but with what you're showing I can't find an explanation for why you do this instead of simply going with something like :
gulp.task('build', function (path) {
gulp.src('bundles/**/*.js)
//Your code for this task
});
Of course, it removes the bundles task, but it's not useful as is.
Don't hesitate to comment if I'm wrong and I'll try to help you as much as I can.
First off, gulp.task('build', function (path) won't ever work. The only valid argument for gulp tasks is a callback to signal asynchronous task completion. If you tried to do run the above, gulp would expect path to be a function and the task would never complete unless that function was called. In this example, the 'build' task should be a regular function called from the 'bundles' pipe, not a task.
The better question would be: How do I run a custom function inside a gulp pipe? Plugins like gulp-tap might get you close, but it's not difficult to create what is essentially an inline gulp plugin to call your function.
Gulp pipes receive a through2 object stream containing a vinyl file object, an encoding and a callback. Here's a basic skeleton for calling any arbitrary function against the files in a gulp pipe:
var gulp = require('gulp');
var through = require('through2');
gulp.task('stack', function() {
return gulp.src('./src/*.js')
.pipe(through.obj(function(file, enc, cb) {
// file.path is the full path to the file
myBuildFunction(file.path);
cb(null, file);
}))
.pipe(gulp.dest('./build/'));
})
This can be incredibly powerful. To modify the file's contents, just change the file.contents buffer. To rename or relocate the file, change file.path. Everything can be done in gulp's native pipes.