I'm working on site files outside of my local htdocs that I want to deploy to the local site. I will use gulp-watch to watch for changes, but I'm stumbling on just the rsync set up. File structure:
sitename/ --
--htdocs/sites/all/themes/themename
--source/themes/themename
And here's the gulpfile:
var gulp = require('gulp');
var rsync = require('gulp-rsync');
gulp.task('deploy', function() {
gulp.src('source/**')
.pipe(rsync({
root: 'source',
destination: '/htdocs/sites/all'
}));
});
And when I run gulp deploy
[11:36:53] Using gulpfile ~/Sites/sitename/gulpfile.js
[11:36:53] Starting 'deploy'...
[11:36:53] Finished 'deploy' after 5.23 ms
Nothing is written anywhere. Am I missing something obvious...
rsync is short for "remote sync", and thus needs a destination somehwere else than your local system. If you want to synchronize stuff from one directory to the other, just use the built-in gulp methods.
gulp.task('deploy', function() {
gulp.src('source/**')
.pipe(gulp.dest('/htdocs/sites/all');
});
This will copy everything you need to the other folder. If you are looking to also delete files, add this to your watcher:
var watcher = gulp.watch('./source/**/*', ['deploy']);
watcher.on('change', function(ev) {
if(ev.type === 'deleted') {
// path.relative gives us a string where we can easily switch
// directories
del(path.relative('./', ev.path).replace('./source','/htdocs/sites/all'));
}
});
Related
I'm using Gulp to compress a zip file and then upload it to AWS Lambda. The upload zip file is done manually. Only the process of compressing is handled by Gulp.
Here is my gulpfile.js
var gulp = require('gulp');
var zip = require('gulp-zip');
var del = require('del');
var install = require('gulp-install');
var runSequence = require('run-sequence');
var awsLambda = require("node-aws-lambda");
gulp.task('clean', function() {
return del(['./dist', './dist.zip']);
});
gulp.task('js', function() {
return gulp.src('index.js')
.pipe(gulp.dest('dist/'));
});
gulp.task('npm', function() {
return gulp.src('./package.json')
.pipe(gulp.dest('dist/'))
.pipe(install({production: true}));
});
gulp.task('zip', function() {
return gulp.src(['dist/**/*', '!dist/package.json'])
.pipe(zip('dist.zip'))
.pipe(gulp.dest('./'));
});
gulp.task('deploy', function(callback) {
return runSequence(
['clean'],
['js', 'npm'],
['zip'],
callback
);
});
After running the deploy task, a zip folder named dist.zip is created consists of a index.js file and a node_modules folder. The node_modules folder contains only a lodash library.
This is index.js
var _ = require('lodash');
console.log('Loading function');
exports.handler = (event, context, callback) => {
//console.log('Received event:', JSON.stringify(event, null, 2));
var b = _.chunk(['a', 'b', 'c', 'd', 'e'], 3);
console.log(b);
callback(null, event.key1); // Echo back the first key value
//callback('Something went wrong');
};
After using AWS lambda console to upload the dist.zip folder. There is an error showing that the lodash library cannot be found
{
"errorMessage": "Cannot find module 'lodash'",
"errorType": "Error",
"stackTrace": [
"Function.Module._load (module.js:276:25)",
"Module.require (module.js:353:17)",
"require (internal/module.js:12:17)",
"Object.<anonymous> (/var/task/index.js:1:71)",
"Module._compile (module.js:409:26)",
"Object.Module._extensions..js (module.js:416:10)",
"Module.load (module.js:343:32)",
"Function.Module._load (module.js:300:12)",
"Module.require (module.js:353:17)"
]
}
But in the zip folder, there is a node_modules directory that contains the lodash lib.
dist.zip
|---node_modules
|--- lodash
|---index.js
When i zip the node_modules directory and the file index.js manually, it works fine.
Does anyone have idea what wrongs ? Maybe when compressing using Gulp, there is a misconfigured for the lib path ?
I had same problem few days back.
Everyone pointed to gulp zip, however it was not problem with gulp zip.
Below worked fine:
gulp
.src(['sourceDir/**'], {nodir: true, dot: true} )
.pipe(zip('target.zip'))
.pipe(gulp.dest('build/'));
That is, note the below, in 2nd param of src, in the above:
{nodir: true, dot: true}
That is, we have to include dot files for the zip (ex: .config, .abc, etc.)
So, include above in .src of gulp, else all others like copy, zip, etc. will be improper.
The package gulp-zip is massively popular (4.3k downloads per day) and there does not seem to be any Gulp substitute. The problem is definitely with relative paths and how gulp-zip processes them. Even when using a base path option in the gulp.src function (example below), gulp-zip finds a way to mess it up.
gulp.task("default", ["build-pre-zip"], function () {
return gulp.src([
"dist/**/*"
], { base: "dist/" })
.pipe(debug())
.pipe(zip("dist.zip"))
.pipe(gulp.dest("./dist/"));
});
Since there is no good Gulp solution as of 1/4/2017 I suggest a work-around. I use Gulp to populate the dist folder first, exactly how I need it with the proper node_modules folder. Then it is time to zip the dist folder properly with relative file paths stored. To do that and also update Lambda, I use a batch file (Windows) of command line options to get the job done. Here is the upload.bat file I created to take the place of the gulp-zip task:
start /wait cmd /c "gulp default"
start /wait cmd /c "C:\Program Files\WinRAR\WinRAR.exe" a -r -ep1 dist\dist.zip dist\*.*
aws lambda update-function-code --zip-file fileb://dist/dist.zip --function-name your-fn-name-here
If you use WinRAR you will find their command line docs here, for WinZip go here. That .bat file assumes you are using the AWS Command Line Interface (CLI) which is a godsend; get it here.
If you are wishing this answer pointed you towards a 100% Gulp solution, to that I say, "You and me both!". Good luck.
I'm new to Gulp and I found a Gulpfile.js example I wanted to use to livereload my express app's server whenever a change takes place in either my app.js file or ./public directory. Here is the Gulpfile.js code:
var gulp = require('gulp'),
spawn = require('child_process').spawn,
node;
/**
* $ gulp server
* description: Launch the server. If there's a server already running, kill it.
*/
gulp.task('server', function() {
if (node) node.kill()
node = spawn('node', ['app.js'], {stdio: 'inherit'})
node.on('close', function (code) {
if (code === 8) {
gulp.log('Error detected, waiting for changes...');
}
});
})
/**
* $ gulp default
* description: Start the development environment
*/
gulp.task('default', function() {
gulp.run('server')
gulp.watch(['./app.js', './public/'], function() {
gulp.run('server')
})
})
// clean up if an error goes unhandled.
process.on('exit', function() {
if (node) node.kill()
})
In my terminal window I keep getting the following warning:
gulp.run() has been deprecated. Use task dependencies or gulp.watch task triggering instead.
Gulp is working and it is livereloading the web application like I want it to but I'd like to fix this issue to future proof my development process, as well as get rid of this annoying warning message.
Thanks for the help!
One option would be to simply replace all occurrences of gulp.run() with gulp.start():
gulp.task('default', function() {
gulp.start('server');
gulp.watch(['./app.js', './public/'], function() {
gulp.start('server');
});
});
However calling a task explicitly using gulp.start() is not the idiomatic way of doing things in gulp (although sometimes it's necessary).
The warning message you receive already hints at the idiomatic way of solving this:
Use task dependencies or gulp.watch task triggering
Task dependencies allow you to run a task before another task. That means you can get rid of the first gulp.run().
Task triggering in gulp.watch() allows you to run a task when a file changes. That means you can get rid of the second gulp.run().
Therefore your default task ends up looking like this:
gulp.task('default', ['server'], function() {
gulp.watch(['./app.js', './public/'], ['server']);
});
I am trying to run karma test via a gulp task. I use https://github.com/karma-runner/gulp-karma and for some reason gulp cannot locate my karma.conf.js. That file is located in the same folder as the gulpfile. The root of the project. No matter what path I put, I get the same error File ./karma.conf.js does not exist. I cannot figure out how to path it correctly. Here is the code for the gulp task.
gulp.task('tdd', function (done) {
new Server({
configFile: 'karma.conf.js'
}, done).start();
});
This is how I spool up karma using Gulp ( and both files are in the same root ).
var karma = require('karma');
gulp.task('karma', function (done) {
karma.server.start({
configFile: __dirname + '/karma.conf.js'
}, done);
});
UPDATE
If you are running NODE.js then
NODE Explnation for __dirname link
"The name of the directory that the currently executing script resides in."
If you are not running NODE.js, then perhaps all you needed was
configFile: '/karma.conf.js'
But if you are running NODE then use the first example.
If another destination directory is assigned to __dirname, then it doesn't work.
Try this:
var Server = require('karma').Server
gulp.task('test', function (done) {
new Server({
configFile: require('path').resolve('karma.conf.js'),
singleRun: true
}, done).start();
});
Short of it: started using Gulp recently (convert from Grunt), and am trying to use both Gulp's default watch task (not gulp-watch from npm) for SASS/JS/HTML and gulp-nodemon (from npm) to restart an Express server upon changes. When running just gulp watch, it works fine; and when running gulp server (for nodemon) that works fine. However, using both together (shown below in the configuration of the default task), the watch stuff isn't working. The task is running, and on the CLI gulp shows 'Starting' and 'Finished' for the watch tasks, but the files don't update.
Relevant task configurations:
Concat javascript:
gulp.task('js:app', function(){
return gulp.src([
pathSource('js/application/modules/**/*.js'),
pathSource('js/application/_main.js')
])
.pipe(concat('application.js'))
.pipe(gulp.dest('./build/assets/js')).on('error', utils.log);
});
Nodemon, restart on changes to express app:
gulp.task('express', function(){
return nodemon({script:'server.js', ext:'js', cwd: __dirname + '/express', legacyWatch: true})
.on('restart', function(){
//gulp.run('watch'); // doesn't work :(
});
});
Watch javascript changes, and run js:app for concat'ing.
gulp.task('watch', function(){
gulp.watch(pathSource('js/application/**/*.js'), ['js:app']);
});
Default task, to initialize gulp watch and nodemon simultaneously:
gulp.task('default', ['watch', 'express']);
If anyone has any ideas, thanks in advance!
gulp.run calls have been deprecated, so I'd try a different approach. Since you're already using gulp, may I suggest giving gulp-nodemon a try?
As per gulp-nodemon documentation, you can pass it an array of tasks to execute:
UPDATE: Here's the full gulpfile.js file, together with a working sample on github.
'use strict';
// Main dependencies and plugins
var gulp = require('gulp');
var jshint = require('gulp-jshint');
var concat = require('gulp-concat');
var uglify = require('gulp-uglify');
var rename = require('gulp-rename');
var nodemon = require('gulp-nodemon');
var assets = 'assets/js/**/*.js';
var publicDir = 'public/javascripts';
// Lint Task
gulp.task('lint', function () {
return gulp.src(assets)
.pipe(jshint())
.pipe(jshint.reporter('jshint-stylish'));
});
// Concatenate and minify all JS files
gulp.task('scripts', function () {
return gulp.src(assets)
.pipe(concat('global.js'))
.pipe(gulp.dest(publicDir))
.pipe(rename('global.min.js'))
.pipe(uglify())
.pipe(gulp.dest(publicDir));
});
// Watch Files For Changes
gulp.task('watch', function () {
gulp.watch(assets, ['lint', 'scripts']);
});
gulp.task('demon', function () {
nodemon({
script: 'server.js',
ext: 'js',
env: {
'NODE_ENV': 'development'
}
})
.on('start', ['watch'])
.on('change', ['watch'])
.on('restart', function () {
console.log('restarted!');
});
});
// Default Task
gulp.task('default', ['demon']);
This way, you spawn the watch task upon nodemon's start and ensure that the watch task is again triggered whenever nodemon restarts your app.
EDIT: seems you should be calling the on-change event from gulp-nodemon, which will handle compile tasks before the restart event triggers.
EDIT: It seems nodemon's on('change', callback) is removed from their API
FWIW, it seems that using the cwd parameter on gulp-nodemon's configuration actually sets the entire gulp cwd to that directory. This means future tasks will be executed in the wrong directory.
I had this problem when running gulp watch tasks on my frontend server at the same time as nodemon tasks on my backend server (in the same gulpfile), there was a race condition wherein if the nodemon command was executed first, the frontend stuff would actually build into (Home)/backend/frontend instead of (Home)/frontend, and everything would go pearshaped from there.
I found that using watch and script params on gulp-nodemon worked around this (although it still looks like nodemon is watching my entire project for changes rather than the built backend directory).
Just learning Gulp. Looks great, but I can't find any information on how to make a complete distribution with it.
Let's say I want to use Gulp to concatenate and minify my CSS and JS, and optimise my images.
In doing so I change the location of JS scripts in my build directory (eg. from bower_components/jquery/dist/jquery.js to js/jquery.js).
How do I automatically update my build HTML/PHP documents to reference the correct files? What is the standard way of doing this?
How do I copy over the rest of my project files?. These are files that need to be included as part of the distribution, such as HTML, PHP, various txt, JSON and all sorts of other files. Surely I don't have to copy and paste those from my development directory each time I do a clean build with Gulp?
Sorry for asking what are probably very n00bish questions. It's possible I should be using something else other than Gulp to manage these, but I'm not sure where to start.
Many thanks in advance.
Point #1
The way i used to achieve this:
var scripts = [];
function getScriptStream(dir) { // Find it as a gulp module or create it
var devT = new Stream.Transform({objectMode: true});
devT._transform = function(file, unused, done) {
scripts.push(path.relative(dir, file.path));
this.push(file);
done();
};
return devT;
}
// Bower
gulp.task('build_bower', function() {
var jsFilter = g.filter('**/*.js');
var ngFilter = g.filter(['!**/angular.js', '!**/angular-mocks.js']);
return g.bowerFiles({
paths: {
bowerDirectory: src.vendors
},
includeDev: !prod
})
.pipe(ngFilter)
.pipe(jsFilter)
.pipe(g.cond(prod, g.streamify(g.concat.bind(null, 'libs.js'))))
.pipe(getScriptStream(src.html))
.pipe(jsFilter.restore())
.pipe(ngFilter.restore())
.pipe(gulp.dest(build.vendors));
});
// JavaScript
gulp.task('build_js', function() {
return gulp.src(src.js + '/**/*.js', {buffer: buffer})
.pipe(g.streamify(g.jshint))
.pipe(g.streamify(g.jshint.reporter.bind(null, 'default')))
.pipe(g.cond(prod, g.streamify(g.concat.bind(null,'app.js'))))
.pipe(g.cond(
prod,
g.streamify.bind(null, g.uglify),
g.livereload.bind(null, server)
))
.pipe(gulp.dest(build.js))
.pipe(getScriptStream(build.html));
});
// HTML
gulp.task('build_html', ['build_bower', 'build_js', 'build_views',
'build_templates'], function() {
fs.writeFile('scripts.json', JSON.stringify(scripts));
return gulp.src(src.html + '/index.html' , {buffer: true})
.pipe(g.replace(/(^\s+)<!-- SCRIPTS -->\r?\n/m, function($, $1) {
return $ + scripts.map(function(script) {
return $1 + '<script type="text/javascript" src="'+script+'"></script>';
}).join('\n') + '\n';
}))
.pipe(gulp.dest(build.html));
});
It has the advantages of concatenating and minifying everything for production while include every files for testing purpose keeping error line numbers coherent.
Point 2
Copying files with gulp is just as simple as doing this:
gulp.src(path).pipe(gulp.dest(buildPath));
Bonus
I generally proceed to deployment by creating a "build" branch and just cloning her in the production server. I created buildbranch for that matter:
// Publish task
gulp.task('publish', function(cb) {
buildBranch({
branch: 'build',
ignore: ['.git', '.token', 'www', 'node_modules']
}, function(err) {
if(err) {
throw err;
}
cb();
});
});
To loosely answer my own question, several years later:
How do I automatically update my build HTML/PHP documents to reference the correct files? What is the standard way of doing this?
Always link to dist version, but ensure sourcemaps are created, so the source is easy to debug. Of course, the watch task is a must.
How do I copy over the rest of my project files?. These are files that need to be included as part of the distribution, such as HTML, PHP, various txt, JSON and all sorts of other files. Surely I don't have to copy and paste those from my development directory each time I do a clean build with Gulp?
This usually isn't a problem as there aren't offer too many files. Large files and configuration are often kept out if the repo, besides.