Gulp 4 glob to exclude specific files and folders not working - gulp

I recently installed gulp 4 [from 3.9.1] and I'm having an issue getting my globs to work as they used to.
I have a whole directory [less some other sub-directories and files] that I want to transfer.
Here's the relevant code:
var path_in_str = 'my_input_dir/';
var path_out_str = 'my_output_dir/';
return gulp.src([path_in_str+'**', path_in_str+'.*', '!'+path_in_str+'node_modules', '!'+path_in_str+'node_modules/**', '!'+path_in_str+'*.json'], {dot: true})
.pipe(gulp.dest(path_out_str));
Basically, I'm trying to prevent the node_modules folder from being transferred, and I also want to prevent all .json files in the home folder from being transferred as well.
What is happening is that the node_modules directory only [no content] is being transferred. Also, all the .json files are being transferred.
How can I tweak this to fix for Gulp 4's way of handling globs [as it appears to have changed slightly]?

I couldn't get this to work with native gulp, so I thought I'd try the node glob package, and this option seems to work best for the use-case.
First you would need to install it like so:
npm install glob --save-dev
Then reference it in your gulp gile:
var gp_glob = require('glob');
And finally, use it as the glob filter:
var input_list_arr = gp_glob.sync(path_in_str+'**', {ignore: [path_in_str+'node_modules/**', path_in_str+'*.json'], dot: true});
return gulp.src(input_list_arr, {base: path_in_str})
.pipe(gulp.dest(path_out_str));
Take note that we're using the ignore option instead of ! notation to filter the file/directory paths. Also note that the base option is specified when passing the filtered list into gulp.src.
This worked for me as intended. Hope it helps.

Related

Why Gulp 4 base option generate file in my current folder

There are so many guide for gulp 3, when following vscode guide(which use version 3), but I am use gulp 4 after run npm install gulp
I want to compile scss in projectroot/jsp/scss/*.scss, output to projectroot/jsp/out/css/...........
// Sass configuration
var gulp = require('gulp');
var sass = require('gulp-sass');
var { resolve } = require('path')
var cssOutput = resolve(__dirname, './jsp/out/css')
gulp.task('sass', gulp.series(function (done) {
return gulp.src('jsp/scss/*.scss', { base: './jsp/out/css' })
.pipe(sass())
.pipe(gulp.dest(function (f) {
done()
console.log(f.base)
return f.base;
}))
}));
gulp.task('default', gulp.series('sass', function () {
gulp.watch('jsp/scss/*.scss', gulp.series('sass'));
}))
when I run gulp -f gulpfile.js
it output index.css in my scss folder
How I solve this problem?
here is my folder
I know there are many guide about version 3 , but I am use version 4...
In your case you do not need the base option. This is sufficient:
return gulp.src('jsp/scss/*.scss')
.pipe(sass())
.pipe(gulp.dest("jsp/out/css"))
I will try to explain what I think is happening in your original code. See the explanation of the base option, from glob base from gulpjs documentation:
Glob base
A glob base - sometimes called glob parent - is the path segment
before any special characters in a glob string. As such, the glob base
of /src/js/**.js is /src/js/. All paths that match the glob are
guaranteed to share the glob base - that path segment can't be
variable.
Vinyl instances generated by src() are constructed with the glob base
set as their base property. When written to the file system with
dest(), the base will be removed from the output path to preserve
directory structures.
That last sentence is key. The base will be removed - so when you return the base in the dest function it has no effect. You set the base but it gets removed leaving the gulp.src path of 'jsp/scss/' as the actve directory structure.
That is my best guess as to what is happening when you return the same base that gets removed.
Now in my simplified code, the base is automatically 'jsp\scss' (everything before the glob *). Which gets removed so we don't need to worry about the scss folder anymore. So what folder structure do you want: 'jsp/out/css'
[Also, you chose a base which is not part of the file's directory structure - that seems strange to me (if not illegal). The purpose of base is to indicate later in src which folder structure to use AFTER the base, not including base.
So for example if you used a base of 'jsp' you would end up with its descendant folder structure 'scss/etc.' following whatever you put into gulp.dest which would act as the parent directories. ]

Using gulp for compiling of changed files only

I have lots of .jade, .styl and .coffee files resided in different subfolders.
I’d like to compile only changed files when they are changed.
I’m using gulp and I’ve come up to the following pattern:
var watch = require('gulp-watch'),
watch(['app/**/*.styl'], function (e) {
gulp.src(e.path)
.pipe(stylus({use: nib()}))
.pipe(gulp.dest('./app'))
However this pattern stores compiled file into the root of ./app folder, but not to the folder where the source file resides.
I’ve tried lots of stuff and all in vain.
The problem is that there is a lack of documentation and samples for gulp-watch and others.
Could anybody tell me how to store compiled file to the its source’s folder?
The problem is that you pass e.path (i.e. the full path of every changed file) as a glob pattern to gulp.src(). This means that your glob pattern does not actually contain a glob (like * or **), in which case the directory where the file is located is used as the default value for the base option to gulp.src(). When the files are then written with gulp.dest() that base option causes the entire directory structure to get stripped.
The solution is to use the streaming variant of gulp-watch instead of the callback variant ...
gulp.src('app/**/*.styl')
.pipe(watch('app/**/*.styl'))
.pipe(stylus({use: nib()}))
.pipe(gulp.dest('./app'));
... or provide an appropriate base option to the callback variant:
watch(['app/**/*.styl'], function (e) {
gulp.src(e.path, {base: 'app'})
.pipe(stylus({use: nib()}))
.pipe(gulp.dest('./app'));
});

how base option affects gulp.src & gulp.dest

I encountered an issue trying to copy a set of files and when calling .dest('some folder') the entire folder structure was lost.
I searched and found an answer suggesting that I should provide {base:'.'} as an option on my call to gulp.src(...) to resolve this issue.
The documentation for gulp.src options only says that its options are:
Options to pass to node-glob through glob-stream.
Looking into node-glob documentation for its options base is not listed there at all.
And the glob-stream options documentation only states that
"the Default is everything before a glob starts (see glob-parent)"
So no much help here either.
So, what effect does the base option passed to gulp.src have on the viny6l files in the created stream and how does it effect the gulp.dest command ?
(You're not looking at the official gulp documentation. http://github.com/arvindr21/gulp is just some guy's fork of the gulpjs github repo. The official repo is http://github.com/gulpjs/gulp/ where the base option is indeed documented.)
To answer your question:
If you don't specify the base option yourself, then everything before the first glob in your gulp.src() paths is automatically used as the base option and ommitted when writing to the destination folder.
Say you have the following files:
some/path/example/app/js/app.js
some/path/example/vendor/js/vendor.js
some/path/example/vendor/lib/js/lib.js
And this is your Gulpfile.js:
gulp.src('some/path/**/js/*.js')
.pipe(gulp.dest('output'));
In this case everything before the ** is automatically used as your base option. So the above is essentially equivalent to this:
gulp.src('some/path/**/js/*.js', {base:'some/path/'})
.pipe(gulp.dest('output'));
What this means is that some/path/ is stripped from the path of every file that matches the pattern in gulp.src(). The resulting structure in the output folder looks like this:
output/example/app/js/app.js
output/example/vendor/js/vendor.js
output/example/vendor/lib/js/lib.js
So a certain part of the directory structure of your source files is indeed lost. How much of your directory structure you lose depends on where the first glob in your gulp.src() pattern is.
If you want to avoid this you have to explicitly specify the base option:
gulp.src('some/path/**/js/*.js', {base:'.'})
.pipe(gulp.dest('output'));
Now some/path/ will not be stripped from your file paths, resulting in the following folder structure in output:
output/some/path/example/app/js/app.js
output/some/path/example/vendor/js/vendor.js
output/some/path/example/vendor/lib/js/lib.js
EDIT: If you pass an array of patterns to gulp.src() there's no way to specify a different base option for each of the array elements. This for example won't work:
gulp.src(
['source1/examples/**/*.html',
'source2/examples/**/*.html'],
{ base: ['source1/', // Doesn't work.
'source2/']} // Needs to be a string.
).pipe(gulp.dest('dist'));
Instead you have to follow the "Using multiple sources in one task" recipe. This lets you merge two streams each of which can receive its own base option:
var merge = require('merge-stream');
gulp.task('default', function() {
merge(gulp.src('source1/examples/**/*.html', {base: 'source1/'}),
gulp.src('source2/examples/**/*.html', {base: 'source2/'}))
.pipe(gulp.dest('dist'));
});

How to concat/uglify files from bower using gulp while ignoring .min files?

I am using bower and wiredep. I want to have a gulp task to combine and uglify all files into a single file... "allmyjs.min.js"
My problem is with libraries like "angular" where within the bower_components/angular directory, there are multiple js files:
angular.js
angular.min.js
index.js
All I want to pick up is the non-minified file so I can concat all my js and minify them, but how do I specify a rule in my list of files that can ignore the .min files?
Right now I have rules like:
"myjs/js/*.js"
"bower_components/angular/angular.js" <- this works, but I have to list out every single dependency which is a nightmare
//"bower_components/angular/*.js" <- this does not work because it picks up everything
I know when I do a wiredep, it looks like bower is smart enough to pick out the specific "JS" files that it injects into my HTML. Is there a way that I can do something in my gulpfile that is smart enough to read the dependencies in bower and include the correct/associated js files without me having to list out every single dependency: (e.g. "bower_comonents/angular/angular.js") for every single bower dependency?
In gulp.src a glob that begins with '!' excludes matching files from the glob results up to that point
var jsFiles = './js/**/*.js';
var minJsFiles = './js/**/*.min.js';
gulp.task('minify', function(){
return gulp.src([jsFiles, '!' + minJsFiles])
.pipe(doSomething()); // min files or some other stuff here
});

Overwrite file in gulp stream

I have following directory structure:
common
-services
--service1.js
--service2.js
-app
--gulpfile.js
--src
---services
----service1.js
----service3.js
I want to made gulp task that will take all files from common directory, take files from app directory and replace all files with same filenames in original stream. After that I will concat it and write to other directory.
I tried this:
var gulp = require('gulp'),
merge = require('gulp-merge'),
concat = require('gulp-concat');
gulp.task('templates', function () {
return merge(
gulp.src(['../common/**/*.js']),
gulp.src(['src/**/*.js'])
)
.pipe(concat('app.js'))
.pipe(gulp.dest('build/js'));
});
I expected to got content of common/services/service2.js, app/src/services/service1.js, app/src/services/service3.js in dest/app.js.
But instead I've got content of all files.
I tried to change cwd or base of gulp.src, but it has no effect.
I know that I can write this stream to tmp directory, and after that get files from it, but it seems not really like gulp-style solution. So how can I overwrite files with same file names in streams?
Ok, i can't find any existing solution for that, so I write my own gulp plugin: gulp-unique-files.