Just learning Gulp. Looks great, but I can't find any information on how to make a complete distribution with it.
Let's say I want to use Gulp to concatenate and minify my CSS and JS, and optimise my images.
In doing so I change the location of JS scripts in my build directory (eg. from bower_components/jquery/dist/jquery.js to js/jquery.js).
How do I automatically update my build HTML/PHP documents to reference the correct files? What is the standard way of doing this?
How do I copy over the rest of my project files?. These are files that need to be included as part of the distribution, such as HTML, PHP, various txt, JSON and all sorts of other files. Surely I don't have to copy and paste those from my development directory each time I do a clean build with Gulp?
Sorry for asking what are probably very n00bish questions. It's possible I should be using something else other than Gulp to manage these, but I'm not sure where to start.
Many thanks in advance.
Point #1
The way i used to achieve this:
var scripts = [];
function getScriptStream(dir) { // Find it as a gulp module or create it
var devT = new Stream.Transform({objectMode: true});
devT._transform = function(file, unused, done) {
scripts.push(path.relative(dir, file.path));
this.push(file);
done();
};
return devT;
}
// Bower
gulp.task('build_bower', function() {
var jsFilter = g.filter('**/*.js');
var ngFilter = g.filter(['!**/angular.js', '!**/angular-mocks.js']);
return g.bowerFiles({
paths: {
bowerDirectory: src.vendors
},
includeDev: !prod
})
.pipe(ngFilter)
.pipe(jsFilter)
.pipe(g.cond(prod, g.streamify(g.concat.bind(null, 'libs.js'))))
.pipe(getScriptStream(src.html))
.pipe(jsFilter.restore())
.pipe(ngFilter.restore())
.pipe(gulp.dest(build.vendors));
});
// JavaScript
gulp.task('build_js', function() {
return gulp.src(src.js + '/**/*.js', {buffer: buffer})
.pipe(g.streamify(g.jshint))
.pipe(g.streamify(g.jshint.reporter.bind(null, 'default')))
.pipe(g.cond(prod, g.streamify(g.concat.bind(null,'app.js'))))
.pipe(g.cond(
prod,
g.streamify.bind(null, g.uglify),
g.livereload.bind(null, server)
))
.pipe(gulp.dest(build.js))
.pipe(getScriptStream(build.html));
});
// HTML
gulp.task('build_html', ['build_bower', 'build_js', 'build_views',
'build_templates'], function() {
fs.writeFile('scripts.json', JSON.stringify(scripts));
return gulp.src(src.html + '/index.html' , {buffer: true})
.pipe(g.replace(/(^\s+)<!-- SCRIPTS -->\r?\n/m, function($, $1) {
return $ + scripts.map(function(script) {
return $1 + '<script type="text/javascript" src="'+script+'"></script>';
}).join('\n') + '\n';
}))
.pipe(gulp.dest(build.html));
});
It has the advantages of concatenating and minifying everything for production while include every files for testing purpose keeping error line numbers coherent.
Point 2
Copying files with gulp is just as simple as doing this:
gulp.src(path).pipe(gulp.dest(buildPath));
Bonus
I generally proceed to deployment by creating a "build" branch and just cloning her in the production server. I created buildbranch for that matter:
// Publish task
gulp.task('publish', function(cb) {
buildBranch({
branch: 'build',
ignore: ['.git', '.token', 'www', 'node_modules']
}, function(err) {
if(err) {
throw err;
}
cb();
});
});
To loosely answer my own question, several years later:
How do I automatically update my build HTML/PHP documents to reference the correct files? What is the standard way of doing this?
Always link to dist version, but ensure sourcemaps are created, so the source is easy to debug. Of course, the watch task is a must.
How do I copy over the rest of my project files?. These are files that need to be included as part of the distribution, such as HTML, PHP, various txt, JSON and all sorts of other files. Surely I don't have to copy and paste those from my development directory each time I do a clean build with Gulp?
This usually isn't a problem as there aren't offer too many files. Large files and configuration are often kept out if the repo, besides.
Related
I would like to deploy my web application to several environments. Using Continuous Integration I can run a task to generate a config.json for a particular environment. This file will contain, among others, the particular URLs to use for it.
{
"baseUrl": "http://www.myapp.es/",
"baseApiUrl": "http://api.myapp.es/",
"baseAuthUrl": "http://api.myapp.es/auth/"
}
The issue comes up when I try to set my different services through providers in the config phase. Of course, services are not available yet in the phase so I cannot use $http to load that json file and set my providers correctly.
Basically I would like to do something like:
function config($authProvider) {
$authProvider.baseUrl = config.baseAuthUrl;
}
Is there a way to load those values on runtime from a file? The only thing I can think about is having that mentioned task altering this file straight away. However I have several modules and therefore, that would have to do in all of them which doesn´t seem right.
You can create constants in the config of your main module:
Add $provide as a dependency in your config method
use the provider method to add all constants like this
$provide.provider('BASE_API_URL', {
$get: function () {
return 'https://myexample.net/api/';
}
});
You can use BASE_API_URL as a dependency in your services.
I hope this helps
Optionally you can set the url depending of your environment:
$provide.provider('BASE_API_URL', {
$get: function () {
if(window.location.hostname.toLowerCase() == 'myapp.myexample.net')
{
return 'https://myexample.net/api/' //pre-production
}else
{
return 'http://localhost:61132/'; //local
}
}
});
Regards!
Finally, the solution was generating an angular constants file using templating (gulp-template) through a gulp task. At the end, I am using a yaml file instead a json one (which is the one generated my CI engine with the proper values for the environment I want to deploy to).
Basically:
config.yml
baseUrl: 'http://www.myapp.es/'
baseApiUrl: 'http://api.myapp.es/'
auth:
url: 'auth/'
config.module.constants.template
(function () {
'use strict';
angular
.module('app.config')
.constant('env_variables', {
baseUrl: '<%=baseUrl%>',
baseApiUrl: '<%=baseApiUrl%>',
authUrl: '<%=auth.url%>'
});
}());
gulpfile.js
gulp.task('splicing', function(done) {
var yml = path.join(conf.paths.src, '../config/config.yml');
var json = yaml.safeLoad(fs.readFileSync(yml, 'utf8'));
var template = path.join(conf.paths.src, '../config/config.module.constants.template');
var targetFile = path.join(conf.paths.src, '/app/config');
return gulp.src(template)
.pipe($.template(json))
.pipe($.rename("config.module.constants.js"))
.pipe(gulp.dest(targetFile), done);
});
Then you just inject it in the config phase you need:
function config($authProvider, env_variables) {
$authProvider.baseUrl = env_variables.baseApiUrl + env_variables.authUrl;
}
One more benefit about using gulp for this need is that you can integrate the generation of these constants with your build, serve or watch tasks and literally, forget about doing any change from now on. Hope it helps!
I'm learning the gulp way of doing things after using grunt exclusively in the past. I'm struggling to understand how to pass multiple inputs to get multiple outputs w/gulp.
Let's say I have a large project that has specialized js on a per page basis:
The Grunt Way:
grunt.initConfig({
uglify: {
my_target: {
files: {
'dest/everypage.min.js': ['src/jquery.js', 'src/navigation.js'],
'dest/special-page.min.js': ['src/vendor/handlebars.js', 'src/something-else.js']
}
}
}
});
This may be a poor example as it violates the "do only one thing" principle since grunt-uglify is concatenating and uglifying. In any event I'm interested in learning how to accomplish the same thing using gulp.
Thanks to #AnilNatha I'm starting to think with more of a Gulp mindset.
For my case I have a load of files that need to be concatenated. I offloaded these to a config object that my concat task iterates over:
// Could be moved to another file and `required` in.
var files = {
'polyfills.js': ['js/vendor/picturefill.js', 'js/vendor/augment.js'],
'map.js': [
'js/vendor/leaflet.js',
'js/vendor/leaflet.markercluster.min.js',
'js/vendor/jquery.easyModal.js',
'js/vendor/jquery-autocomplete.min.js',
'js/vendor/underscore.1.8.3.js',
'js/map.js'
],
...
};
var output = './build/js';
// Using underscore.js pass the key/value pair to custom concat function
gulp.task('concat', function (done) {
_.each(files, concat);
// bs.reload(); if you're using browsersync
done(); // tell gulp this asynchronous process is complete
});
// Custom concat function
function concat(files, dest) {
return gulp.src(files)
.pipe($.concat(dest))
.pipe(gulp.dest(output));
}
This must be obvious but I can't find it. I want to preprocess my stylus/coffee files with a watcher in the dev environment and in production with a build task (isn't that common to all of us?) and also run a few more minification and uglification steps in production but I want to share the pipe steps common to both dev and production for DRY
The problem is that when I run the task which watches the files, the task which preprocesses does that to all the files since it has its own gulp.src statement which includes all stylus files.
How do I avoid compiling all files on watching while still keeping the compile task separate. Thanks
paths = {
jade: ['www/**/*.jade']
};
gulp.task('jade', function() {
return gulp.src(paths.jade).pipe(jade({
pretty: true
})).pipe(gulp.dest('www/')).pipe(browserSync.stream());
});
gulp.task('serve', ['jade', 'coffee'], function() {
browserSync.init({
server: './www'
});
watch(paths.jade, function() {
return gulp.start(['jade']);
});
return gulp.watch('www/**/*.coffee', ['coffee']);
});
One important thing in Gulp is not to duplicate pipelines. If you want to process your stylus files, it has to be the one and only stylus pipe. If you want to execute different steps in your pipe however, you have multiple choices. One that I would suggest would be a noop() function in conjunction with a selection function:
var through = require('through2'); // Gulp's stream engine
/** creates an empty pipeline step **/
function noop() {
return through.obj();
}
/** the isProd variable denotes if we are in
production mode. If so, we execute the task.
If not, we pass it through an empty step
**/
function prod(task) {
if(isProd) {
return task;
} else {
return noop();
}
}
gulp.task('stylus', function() {
return gulp.src(path.styles)
.pipe(stylus())
.pipe(prod(minifyCss())) // We just minify in production mode
.pipe(gulp.dest(path.whatever))
})
As for the incremental builds (building just the changed files with every iteration), the best way would be to get on the gulp-cached plugin:
var cached = require('gulp-cached');
gulp.task('stylus', function() {
return gulp.src(path.styles)
.pipe(cached('styles')) // we just pass through the files that have changed
.pipe(stylus())
.pipe(prod(minifyCss()))
.pipe(gulp.dest(path.whatever))
})
This plugin will check if the contents have changed with each iteration you have done.
I spend a whole chapter on Gulp for different environments in my book, and I found those to be the most suitable ones. For more information on incremental builds, you can also check on my article on that (includes Gulp4): http://fettblog.eu/gulp-4-incremental-builds/
I am having tried to read tutorials and browse other questions, but cannot find a good answer to how I can set up gulp so that I get javascript-files from (already installed) bower-components for development and for production.
First I tried a simple way. For development:
gulp.task('vendorScriptsDevelopment', function() {
return gulp.src(['bower_components/jquery/dist/jquery.js',
'bower_components/**/*.js',
'!bower_components/**/*.min.js'])
.pipe(filter('*.js'))
.pipe(concat('vendor-scripts.js'))
.pipe(gulp.dest('dev'))
});
And similar for production:
gulp.task('vendorScriptsProduction', function() {
return gulp.src(['bower_components/jquery/dist/jquery.min.js',
'bower_components/**/*.min.js'])
.pipe(filter('*.js'))
.pipe(concat('vendor-scripts.js'))
.pipe(gulp.dest('prod'))
});
I included jquery specifically first, since other plugins often depend on it.
But then I realize that some bower packages includes a lot of files, also various javascript-files that I do not want (I just want "the one" that typically also has a CDN-option (and are offered in two versions, normal js and minimized)).
One tutorial I have read uses the main-bower-files plugin for the development part, but then it goes on in the wrong direction and wants to make a minified version itself (as I understand, it is always best to use the packages included minified version, as that is optimized from the developers of the plugin).
How can I set up my two Gulp tasks so that they works as intended? Or am I forced to included all the files manually (like I included jquery manually in my examples)?
Ok, since nobody has replied, here is my attempt to answer my own question.
I created a function that generates an array of the filenames that I want. This uses main-bower-files and filters out the .js-files. If this is development, then that's it. If it is production, then I just change the file extension from .js to .min.js (and for safety checks if that file exists).
var concat = require('gulp-concat');
var mainBowerFiles = require('main-bower-files');
var fs = require('fs');
var vendorScripts = function (minified) {
var scripts = mainBowerFiles().filter(function (filename) {
return filename.match(/.+\.js$/)
});
if (minified) {
scripts = scripts.map(function (orgFilename) {
var minFilename = orgFilename.replace(/^(.+)\.js$/, '$1.min.js');
if (fs.existsSync(minFilename)) {
return minFilename
}
return orgFilename;
});
}
return scripts;
};
gulp.task('vendorScriptsDevelopment', function() {
return gulp.src(vendorScripts())
.pipe(concat('vendor-scripts.js'))
.pipe(gulp.dest('dev'))
});
gulp.task('vendorScriptsProduction', function() {
return gulp.src(vendorScripts(true))
.pipe(concat('vendor-scripts.js'))
.pipe(gulp.dest('dist'))
});
I should change my function to handle other assets too, like css.
If somebody has a better approach, I would be very glad for a suggestion!
Here is my solution for your problem enjoy:
Create a vendors.json file in your sources folder. Edit the file and make path to relevant files you want to include in the production folder. For example:
{
"js" : [
"lib/jquery/dist/jquery.js",
"lib/lodash/lodash.js",
"lib/angular/angular.js",
"lib/angular-sanitize/angular-sanitize.js",
"lib/angular-ui-router/release/angular-ui-router.js",
"lib/angular-ui-utils/ui-utils.js",
"lib/angular-bootstrap/ui-bootstrap-tpls.js",
"lib/chartjs/Chart.js",
"lib/pnotify/pnotify.core.js",
"lib/pnotify/pnotify.buttons.js",
"lib/angular-pnotify/src/angular-pnotify.js",
"lib/angular-prompt/dist/angular-prompt.js",
"lib/angular-mocks/angular-mocks.js"
],
"css" : [
"lib/bootstrap/dist/css/bootstrap.css",
"lib/bootstrap-rtl/dist/css/bootstrap-rtl.css",
"lib/bootstrap/dist/css/bootstrap-theme.css",
"lib/font-awesome/css/font-awesome.css",
"lib/pnotify/pnotify.core.css",
"lib/pnotify/pnotify.buttons.css"
],
"statics" : [
"lib/font-awesome/fonts/*"
]
}
Then in the gulpFile.js add this:
var sources = {
get 'vendor.js'(){
return getVendorSources().js;
},
get 'vendor.css'(){
return getVendorSources().css;
},
get 'vendor.statics'(){
return getVendorSources().statics;
}
};
function getVendorSources(){
return JSON.parse(fs.readFileSync('yourSourcesFolder/vendor.json', 'utf-8'));
}
gulp.task('vendor.css', function() {
return gulp.src(sources['vendor.css'])
.pipe(changed(paths.dist))
.pipe(gulp.dest(paths.dist));
});
and so on for the js and static files tasks.
I'm probably trying to make gulp do something that's not idiomatic, but here goes.
I want my build task to only run if the source files are newer than the output file.
In gulp, it seems standard practice to create a build task that always runs, and then set up a watch task to only run that build task when certain files change. That's okay, but it means that you always build on the first run.
So, is it possible to do what I want? Here's what I've got so far (newer is gulp-newer):
gulp.task('build_lib', function() {
return gulp.src(["app/**/*.ts"])
.pipe(newer("out/outputLib.js")) //are any of these files newer than the output?
** NEED SOMETHING HERE **
how do I say, "If I got _any_ files from the step before, replace all of them with a single hardcoded file "app/scripts/LibSource.ts" "?
.pipe(typescript({
declaration: true,
sourcemap: true,
emitError: false,
safe: true,
target: "ES5",
out: "outputLib.js"
}))
.pipe(gulp.dest('out/'))
});
I tried using gulpif, but it doesn't seem to work if there are no files going into it to begin with.
.pipe(gulpif(are_there_any_files_at_all,
gulp.src(["app/scripts/LibSource.ts"])))
However, my condition function isn't even called because there are no files on which to call it. gulpif calls the truthy stream in this case, so LibSource gets added to my stream, which isn't what I want.
Maybe doing all of this in a single stream really isn't the right call, since the only reason I'm passing those files through the "gulp-newer" filter is to see if any of them is newer. I'm then discarding them and replacing them with another file. My question still stands though.
You can write your own through/transform stream to handle the condition like so:
// Additional core libs needed below
var path = require('path');
var fs = require('fs');
// Additional npm libs
var newer = require('gulp-newer');
var through = require('through');
var File = require('vinyl');
gulp.task('build_lib', function() {
return gulp.src(["app/**/*.ts"])
.pipe(newer("out/outputLib.js"))
.pipe(through(function(file) {
// If any files get through newer, just return the one entry
var libsrcpath = path.resolve('app', 'scripts', 'LibSource.ts');
// Pass libsrc through the stream
this.queue(new File({
base: path.dirname(libsrcpath),
path: libsrcpath,
contents: new Buffer(fs.readFileSync(libsrcpath))
}));
// Then end this stream by passing null to queue
// this will ignore any other additional files
this.queue(null);
}))
.pipe(typescript({
declaration: true,
sourcemap: true,
emitError: true,
safe: true,
target: "ES5",
out: "outputLib.js"
}))
.pipe(gulp.dest('out/'));
});
I know like, this question was posted over 4 years ago, however; I am sure this problem crosses the path of everyone, and although I think I understand the question that is being asked, I feel that there is an easier way to perform this task, off which, I posted a similar question recently on stackoverflow at New to GULP - Is it necessary to copy all files from src directory to dist directory for a project?
It uses gulp-changed, and for me, it worked like a charm, so for others who may look at this post for similar reasons, have a look at my post and see if it is what you are looking for.
Kind Regards
You don't need to build first. You can on your 'first run' only run the watch task from which you run all the other ones.
example:
// Create your 'watch' task
gulp.task( 'watch', function() {
gulp.watch( 'scripts/*.js', [ 'lint', 'test', 'scripts' ] );
gulp.watch( 'styles/sass/*.scss', [ 'sass_dev' ] );
} );
// On your first run you will only call the watch task
gulp.task( 'default', [ 'watch' ] );
This will avoid running any task on startup. I hope this will help you out.
May I suggest gulp-newy in which you can manipulate the path and filename in your own function. Then, just use the function as the callback to the newy(). This gives you complete control of the files you would like to compare.
This will allow 1:1 or many to 1 compares.
newy(function(projectDir, srcFile, absSrcFile) {
// do whatever you want to here.
// construct your absolute path, change filename suffix, etc.
// then return /foo/bar/filename.suffix as the file to compare against
}