Gulp task to replace all Root-Relative URLs with Document-Relative URLs - html

There are 4 distinct URL types;
Absolute http://www.example.com/images/icons.png
Document-Relative ../images/icons.png
Root-Relative /images/icons.png
Protocol-Relative //www.example.com/images/icons.png
I have a large static file site (html, css, js) which is built with Jigsaw. This framework takes PHP templates and compiles them into static HTML. I am also using Gulp tasks to compile assets (sass, js..etc).
Using Jigsaw's build process I can either have the site built with full Absolute paths/urls (http://example.com/path-to/page) or Root-Relative (/path-to/page).
This is great but now the client wants the site to use Document-Relative as they are now hosting the site in a subdirectory with a URL pointing to that subdirectory.
E.g. http://example.com would point to http://xx.server.com/hosted-files/directory
My issue is that Jigsaw doesn't allow for Document-Relative URLs. Is there a gulp/node script I can use to convert all references (image sources, links, css paths..etc)? Or is there another solution (e.g. using .htacccess)?
TLDR;
I need to replace any Absolute or Root-Relative references in multiple HTML files and directories with Document-Relative paths and URLs. Or is there another solution (e.g. using .htacccess)?

I have managed to solve my own issue with what I feel is a "hacky" fix.
I've basically created a custom gulp plugin replaces URLs/paths..etc with Document-Relative paths.
gulpfile.js - relative-urls task runs after all other tasks have completed.
const relative = require('./tasks/document-relative');
gulp.task('relative-urls', function() {
return gulp.src('build/**/*.html')
.pipe( relative({
directory: 'build',
url: 'http://localhost:8000',
}) )
.pipe( gulp.dest('build') );
});
./tasks/document-relative.js - plugin
'use strict';
const fs = require('fs');
const PluginError = require('plugin-error');
const through = require('through2');
const PLUGIN_NAME = 'document-relative';
let count = 0;
module.exports = function(options) {
// Remove slashes from beginning and end of string
const strip_slashes = (string) => {
return string ? string.replace(/^\/|\/$/g, '') : null;
}
// Users options object
options = options || {};
// Cleanup options
const base_dir = strip_slashes(options.directory);
const url = strip_slashes(options.url) + '/';
return through({
objectMode: true,
writable: true,
readable: true
},
function(file, enc, callback) {
count++;
// Check for null file
if (file.isNull()) {
return callback(null, file);
}
if (file.isStream()) {
this.emit('error', new PluginError(PLUGIN_NAME, 'Stream not supported!'));
return callback(null, file);
}
if (file.isBuffer()) {
// Get contents of this file
let html = file.contents.toString(enc);
// This files full path (/home/usr/project/build/page/example/index.html)
const path = file.path;
// Path task was run from (/home/usr/project/)
const cwd = file.cwd+( base_dir ? '/'+base_dir : '' );
// Project specific path (/page/example/index.html)
const relative = path.replace(cwd, '');
// Get array of directories ['page', 'example', 'index.html']
let paths = strip_slashes(relative).split('/');
// Remove last item ['page', 'example']
paths.pop();
// Add ../ for nth number of paths in array
let rel_path = paths.length === 0 ? '' : ('../'.repeat(paths.length));
// Replace dom attributes (e.g. href="/page/example")
html = html.replace( /(?:(?!="\/\/)="\/)/g, '="'+rel_path );
// Replace inline background (e.g. background: url('/image/something.jpg'))
html = html.replace( /url\(\'\//g, 'url(\''+rel_path );
html = html.replace( /url\('\//g, 'url(''+rel_path );
// If user defined URL, match and remove
if (url && url.length) {
html = html.replace( new RegExp(url, 'g'), rel_path );
}
// Overwrite file
fs.writeFileSync(file.path, html, {
encoding: enc,
flag:'w'
});
return callback();
}
});
};
This basically opens all .html files in my build folder, calculates how many paths deep each file is (/folder1/folder2/index.html) and replaces any instances of url (http://localhost:8000) with ../ repeated for the number of paths calculated.

Node has path.relative.
Read Levi Coles' own answer to understand where url and directory come from.
const path = require("path");
// create a regular expression from your url property.
const domain_expression = new RegExp(url);
// Once you have an offending 'href' you can do this.
// - Here 'href' is Absolute, but this same code would work
// with Root-Relative paths too.
const href = "http://localhost:8000/images/icons.png";
const file_index = href.lastIndexOf("/") + 1;
const file_component = href.substring(file_index);
const root_relative = href.replace(domain_expression, "");
const relative_href = path.relative(`${directory}/${root_relative}`, directory);
const _href = relative_href + file_component;
// _href = ../../icons.png

Related

overwrite all files from one folder to another folder with gulp

I already have my method that compresses the content of the files with gulp-minify-css in a temporary folder which I hope to later delete with gulp if possible but I don't see how to replace the files first, I have:
var revReplace = require('gulp-rev-replace'); // -> this folder is not recognized
gulp.task('revrep', () => {
return gulp.src('app/Styles/CssArqMincss')
.pipe(gulp.dest('app/Content/CSSArq/'));
})
and how can I later delete the folder /Styles/CssArqMincss?
Thanks
To delete files from folder location, you can use this task and adapt it.
const { src, task } = require( 'gulp' );
const clean = require("gulp-clean");
const logger = require('node-color-log');
function cleanFolderTask() {
const yourPathRoot = "pathToYourFolder"; //I would recommend using config file to store paths, such as : config.path.artifact.root;
logger.color('yellow').log(`Cleaning ${yourPathRoot}`);
return src(yourPathRoot ,{allowEmpty: true},{read: false}).pipe(clean({force:true}));
};
const cleanFolderTask= task('clean:folder', cleanFolderTask);
exports.cleanFolderTask= cleanFolderTask;
To copy from one folder to another one, I would also use the previous task to first clean the destination and then :
const { src, dest, task } = require( 'gulp');
const logger = require('node-color-log');
function copyFilesTask(callback) {
let filesLoc = "path/to/your/files";
let filesDest = "path/to/destination"; // I would recommend using config file to store paths, such as : config.path.artifact.files and config.path.solution.files;
logger.color('yellow').log(`Copy ${filesDest }`);
src(`${filesLoc}/**/*`)
.pipe(dest(filesDest),
callback());
};
const copyFiles = task('copy:files', copyFilesTask);
exports.copyFiles = copyFiles;

How to move globbed gulp.src files into a nested gulp.dest folder

QUESTION PART 1: OUTPUTTING TO A NESTED DYNAMIC FOLDER
I use Gulp.js for graphic email development. My employer is switching to a different marketing platform which requires our email templates to be in a different folder structure. I'm having trouble outputting to nested folders when gulp.src uses globbing. I'd appreciate your help!
Here is a simplified example the gulp.src folder:
build/template1/template1.html
build/template2/template2.html
build/template3/template4.html
build/template4/template4.html
Here is a simplified example the gulp.src folder:
build/template1/theme/html/template1.html
build/template2/theme/html/template2.html
build/template3/theme/html/template4.html
build/template4/theme/html/template4.html
I want to do something like a wildcard for the dynamic template folders ...
gulp.task('moveFile', function(){
return gulp.src('./build/*/*.html')
.pipe(gulp.dest('./build/*/theme/html'));
});
... But globbing only works in the gulp.src. How can I output to a dynamic folder when using a globbed gulp.src? the closest I can get is putting the /theme folder at the same level as the template folders, not inside each as desired.
Thank you for your help!
QUESTION PART 2: OUTPUTTING A *RENAMED FILE* TO A NESTED DYNAMIC FOLDER
Mark's answered my question (Thanks #Mark!), but I over-simplified my use case so I'm adding a Part 2.
In addition to nesting the file, I need to rename it. (I had this part working originally, but can't get the 2 parts to work together.) Referring to the gulp-rename documentation, I made 3 different attempts. It's so close but I'd appreciate a little more help. :-)
// ATTEMPT 1: Using gulp-rename mutating function method
gulp.task('createTwig', function(){
return gulp.src('./build/*/*.html')
.pipe(rename(
function (path) {
path.basename = "email";
path.extname = ".html.twig";
},
function (file) {
console.log(file.dirname);
file.dirname = nodePath.join(file.dirname, 'theme/html');
}
))
.pipe(gulp.dest('./build/'));
});
// ATTEMPT 2: Using gulp-rename fixed object method
gulp.task('createTwig', function(){
return gulp.src('./build/*/*.html', { base: process.cwd() })
.pipe(rename(
{
basename: "email",
extname: ".html.twig"
},
function (file) {
console.log(file.dirname);
file.dirname = nodePath.join(file.dirname, 'theme/html');
}
))
.pipe(gulp.dest('./build/'));
});
// ATTEMPT 3: Using gulp-rename mutating function method
gulp.task('createTwig', function(){
return gulp.src('./build/*/*.html')
.pipe(rename(
function (path, file) {
path.basename = "email";
path.extname = ".html.twig";
console.log(file.dirname);
file.dirname = nodePath.join(file.dirname, 'theme/html');
}
))
.pipe(gulp.dest('./build/'));
});
This works:
const rename = require("gulp-rename");
const path = require("path");
gulp.task('moveFile', function(){
return gulp.src(['build/**/*.html'])
.pipe(rename(function (file) {
console.log(file.dirname);
file.dirname = path.join(file.dirname, 'theme/html');
}))
.pipe(gulp.dest('build')) // build/template1/theme/html
});
I tried a few ways, including trying the base option and gulp-flatten and using a function in gulp.dest but this was the easiest.
Question Part #2:
gulp.task('createTwig', function(){
return gulp.src(['build/**/*.html'])
.pipe(rename(function (file) {
file.basename = "email";
file.extname = ".html.twig";
file.dirname = path.join(file.dirname, 'theme/html');
}))
.pipe(gulp.dest('build')) // build/template1/theme/html
});
path.basename/extname are just "getters", you cannot set those values.

how to create folder and upload images in ipfs

https://proto.school/#/mutable-file-system
I have gone through this link but don’t know how to do same thing in node.js
I have added 'hello' word into the IPFS network and its working fine and also i have used image to upload into ipfs but i want to know how can i create folder in ipfs network and upload images into that folder
So my problem is that how to create the folder and upload picture into that folder.
Here is my code.
const addFile = async () => {
//const Added = await ipfs.add('hello');
const fsReadImgData = fs.readFileSync('image1.jpg');
var ipfsSave = await ipfs.add({
path:image1.jpg,
content: fsReadImgData
});
return fsReadImgData;
}
const fileHash = await addFile();
First read file into a buffer (or replace with however you're getting the image data:
const imgdata = fs.readFileSync('/yourfile.jpg');
Regular IPFS files method (immutable, you do not expect to update these):
let added = await ipfs.add({
path: 'images/yourfile.jpg',
content: imgdata
}, { wrapWithDirectory: true })
Mutable filesystem method (you expect to update and change the files):
await ipfs.files.mkdir('/images')
await ipfs.files.write(
'/images/yourfile.jpg',
imgdata,
{create: true})

Run a gulp plugin that requires the current file path and name

I am using purgeCSS to removed unused CSS. My challenge is that I need to do this dynamically. Depending on the current .css file that is being processed, I need to get its path and file name so I can dynamically insert the content HTML path for Purge to run.
Here is how my code looks like:
const gulp = require("gulp"),
appRoot = require("app-root-path"),
sass = require("gulp-sass"),
purgecss = require("gulp-purgecss"),
tap = require("gulp-tap"),
path = require("path"),
utilities = require(appRoot + "/Tools/Utilities-Functions/utilities-functions.js");
gulp.task("sass", () => {
let htmlContentPath = "";
return (
gulp
.src("./Pages/**/*.scss")
// Compile .scss into .css
.pipe(sass())
// Get path for HTML file (dynamic)
.pipe(
tap(function (file, t) {
let fileName = path.basename(file.path);
// This is a simple function that returns the file name without extension (homepage.css >> homepage)
fileName = utilities.getFileNameWithoutExtension(fileName);
htmlContentPath = "/fullPath/Pages/" + fileName + "/compiled/html/" + fileName + ".html";
})
)
// Remove unused CSS
.pipe(
purgecss({
content: [htmlContentPath]
})
)
// Set the destination folder (main css)
.pipe(gulp.dest("./dist/css"))
);
})
For some reason happens that "htmlContentPath" for the Purge is empty. Even though I would expect "tap" plugin to always set a value to it. As a result this provokes an error on the purgecss:
As stated above, this error is due to having "htmlContentPath" empty.
Another attempt I tried was to do the Purge inside the Tap plugin, like this:
const gulp = require("gulp"),
appRoot = require("app-root-path"),
sass = require("gulp-sass"),
purgecss = require("gulp-purgecss"),
tap = require("gulp-tap"),
path = require("path"),
utilities = require(appRoot + "/Tools/Utilities-Functions/utilities-functions.js");
gulp.task("sass", () => {
return (
gulp
.src("./Pages/**/*.scss")
// Compile .scss into .css
.pipe(sass())
// Get path for HTML file (dynamic)
.pipe(
tap(function (file, t) {
let fileName = path.basename(file.path);
// This is a simple function that returns the file name without extension (homepage.css >> homepage)
fileName = utilities.getFileNameWithoutExtension(fileName);
let htmlContentPath = "/fullPath/Pages/" + fileName + "/compiled/html/" + fileName + ".html";
// Remove unused CSS
purgecss({
content: [htmlContentPath]
})
})
)
// Set the destination folder (main css)
.pipe(gulp.dest("./dist/css"))
);
})
This time it doesn't give an error, but the Purge is totally ignored...
Any solution on how I could solve this?
After attempting dozens of approaches here is the one that worked for me and thought would be worth sharing with others that might be going through a similar challenge:
const gulp = require("gulp"),
appRoot = require("app-root-path"),
sass = require("gulp-sass"),
path = require("path"),
utilities = require(appRoot + "/Tools/Utilities-Functions/utilities-functions.js"),
fs = require("fs"),
through = require("through2"),
uncss = require("uncss");
gulp.task("sass", () => {
return (
gulp
.src("./Pages/**/*.scss")
// Compile .scss into .css
.pipe(sass())
// Remove unused CSS
.pipe(
through.obj(function(file, encoding, callback) {
try {
const cssFileContent = file.contents.toString(); // Get the css file contents
let transformedFile = file.clone(), // Clone new file for manipulation
fileName = path.basename(file.path),
htmlFilePath;
// This is a simple function that returns the file name without extension (homepage.css >> homepage)
fileName = utilities.getFileNameWithoutExtension(fileName);
// File path for the .html file
htmlFilePath = "/fullPath/Pages/" + fileName + "/compiled/html/" + fileName + ".html";
// Check if there is any css to be checked and if .html file exists
if (cssFileContent.length && fs.existsSync(htmlFilePath)) {
// Call uncss to remove unused css
uncss([htmlFilePath], { raw: cssFileContent }, function(error, output) {
if (error) {
callback(null, transformedFile);
}
// Set new contents with the "used" css only (uncss' output)
transformedFile.contents = Buffer.from(output);
callback(null, transformedFile);
});
} else {
callback(null, transformedFile);
}
} catch (e) {
console.log("Gulp error - uncss: " + e.message);
callback(null, transformedFile);
}
})
)
// Set the destination folder (main css)
.pipe(gulp.dest("./dist/css"))
);
});
Basically I built a custom gulp stream using through. This allows you to read information about the current file processed, do whatever logic you want, and then invoke callback with the new transformed file.
In more details what I have done:
Read file information (file name and its location)
Get the location for the HTML I want to check my CSS against
Run uncss (instead of purgecss I was using initially) because on this tool I can send raw CSS which is handy in my case
From the output of uncss, I affect the contents of the CSS file with this output
Invoke callback with this new transformed file

Gulp - Delete empty folders recursively

I want to delete all folders and subfolders inside of a given directory if they only contain folders and no files. Is there an easy way to do that?
What I found until now:
https://www.npmjs.com/package/gulp-recursive-folder
https://www.npmjs.com/package/gulp-count
https://www.npmjs.com/package/gulp-path
You can use delete-empty:
gulp.task('delete-empty-directories', function() {
deleteEmpty.sync('foo/');
});
This recursively deletes all empty folders below foo/.
Here's a rough start, so you'll just need to loop for recursion I guess.
var modules = {
gulp : require('gulp'),
fs : require('fs'),
path : require('path'),
del : require('del'),
map : require('map-stream')
};
modules.gulp.task('folder-delete', function() {
// get folder list inside of the dir passed in
function getFolders(dir) {
return modules.fs.readdirSync(dir)
.filter(function(file) {
return modules.fs.statSync(modules.path.join(dir, file)).isDirectory();
});
}
var dir = '../src/', // (update with your path to the root folder)
folders = getFolders(dir),
hasFile = 0;
var folderMap = folders.map(function(folder) {
hasFile = 0; // reset for each folder
return modules.gulp.src(dir + folder + '/**/*')
.pipe(modules.map(function(file, cb) {
hasFile = 1;
cb(null, file);
}))
.on('end', function() {
console.log(hasFile, ' - ', folder);
if (!hasFile) {
modules.del([dir + folder], { force: true }).then(function() {
console.log('Deleted ' + dir + folder);
});
}
})
});
return folderMap;
});
Basically, this is setting the directory at ../src/, getting the folders at the root of that dir, then runs the src under those directories. It then uses map to see if there was a file added to the stream, then updates a variable if so. After the task finishes and if the variable has not been updated, then it will delete the folder.
As stated above, you could probably just loop through the directories for the recursion (or you could use one of the plugins you've mentioned).
The accepted solution, which uses the delete-empty package, doesn't work for me (and for others too). Plus, the creator seems to have stopped maintaining the package (last update 3 years ago).
What works just fine, and probably is much more future save, is a combination of sync-exec and the OS find command.
const syncExec = require("sync-exec")
syncExec("find sample/directory/ -type d -empty -delete")