How do I convert an application cache manifest to a service worker - html

The application cache is now deprecated and browsers like Chrome are removing support.
We have an application that can work 100% offline while storing data in the indexeddb and syncing later when the user is back online. We need to transition this site from using application cache to service worker. We will be using Workbox for our service worker.
There are three main sections of our cache manifest that we must covert.
CACHE Section
This is a list of assets to precache. This is probably the most straight forward to transition as we are using workbox to precache these files.
NETWORK Section
We are using * here (probably most common) so that's probably not going to be an issue.
FALLBACK Section
We have quite a few entries in the fallback section. Basically they are redirect to the login page and are there in case someone refreshes the page offline.
Example:
FALLBACK:
/search /login
/customer-edit /login
/foo-bar-baz /login
...
My question:
Is there either 1) a general guide to converting application cache/cache manifest to service workers or 2) some specific guidance for converting the FALLBACK section to the equivalent functionality in a service worker.
Google and Duck Duck Go have not been extremely helpful.
There are existing projects to upgrade the app cache to service workers but most appear very beta, example from Google Chrome Labs: github.com/GoogleChromeLabs/sw-appcache-behavior

This is the solution I came up with using Google's workbox.
Sidenote: Workbox appears to have a has a solution for most common service worker use cases and has a very flexible distribution model that makes it quite easy to work with, either in a vanilla js environment or with your framework of choice.
We ended up converting our server side AppCache (cach manifest code) to generate a service worker. (How to precaching an url list coming from a json file?)
Depending on your server side language you're code will vary, but this is the end product that worked for us:
service-worker.js (generated server side)
const productVersion = "3.01";
importScripts('/assets/workbox/workbox-sw.js');
workbox.setConfig({
modulePathPrefix: '/assets/workbox/'
});
const { precacheAndRoute, createHandlerBoundToURL } = workbox.precaching;
const { NavigationRoute, registerRoute, setCatchHandler } = workbox.routing;
precacheAndRoute([
// cache index html
{url: '/', revision: '3.01' },
// web workers
{url: '/assets/some-worker.js?ver=3.01', revision: '' },
{url: '/assets/other-worker.js?ver=3.01', revision: '' },
// other js files
{url: '/assets/shared-function.js', revision: '3.01' },
// ... removed for brevity
// css
{url: '/assets/site.css', revision: '3.01' },
{url: '/assets/fonts/fonts.css', revision: '3.01' },
// svg's
{url: '/assets/images/icon.svg', revision: '3.01' },
{url: '/assets/images/icon-2.svg', revision: '3.01' },
// png's
{url: '/assets/images/img-1.png', revision: '3.01' },
{url: '/assets/images/favicon/apple-touch-icon-114x114.png', revision: '3.01' },
// ...
// ...
// fonts
{url: '/assets/fonts/lato-bla-webfont.eot', revision: '3.01' },
{url: '/assets/fonts/lato-bla-webfont.ttf', revision: '3.01' },
// sounds
{url: '/assets/sounds/keypress.ogg', revision: '3.01' },
{url: '/assets/sounds/sale.ogg', revision: '3.01' },
]);
// Routing for SPA
// This assumes DEFAULT_URL has been precached.
const DEFAULT_URL = '/';
const handler = createHandlerBoundToURL(DEFAULT_URL);
const navigationRoute = new NavigationRoute(handler, {
denylist: [
new RegExp('/ping'),
new RegExp('/upgrade'),
new RegExp('/cache.manifest'),
],
});
registerRoute(navigationRoute);
// This allows the main window to signal the service worker that
// it should go ahead and install if it's waiting.
addEventListener('message', (event) => {
if (event.data && event.data.type === 'SKIP_WAITING') {
skipWaiting();
}
});
There's a few other things to note. We had to figure out how to smoothly upgrade from App?Cache to Service Workers. Turns out the generating an empty cache manifest for us did the trick.
We already had an upgrade process in place (prompt user to upgrade or force automatic upgrade with a count down timer) so we had to do some work to get that to work with service workers. Notice the end of the service worker file has addEventListener code. We actually call that from an upgrade page to get a smooth upgrade process. It goes something like this:
A) Upgrade script detects new version is available (many ways to do this, api call polling, etc)
B) If user accepts or timer expires redirect user to an upgrade page. This step is vital b/c you can't update a service worker with an app still running. So navigate to the upgrade page, wait for the service worker to get installed then tell it to skip waiting and the redirect to main (login) screen.
C) User is happily running new version of app.
Upgrade page code:
(this is a good page to show an "updating" UI of some type)
<script type="module">
import { Workbox } from '/assets/workbox/workbox-window.prod.mjs';
if ('serviceWorker' in navigator) {
const wb = new Workbox('/serviceworker');
// This code exists b/c a service worker can't update with just a refresh/reload in the
// browser. This is b/c on a reload, the old and new page exist simultaneously and the old MUST
// unload before the new service worker can automatically assume control. Also if multiple pages
// are open, this blocks the service worker from taking control (multiple pages should not an issue with this app).
// This code activates a waiting service worker and _then_ redirects back to the app.
// Add an event listener to detect when the registered
// service worker has installed but is waiting to activate.
wb.addEventListener('waiting', (event) => {
// Set up a listener that will reload the page as soon as the previously waiting
// service worker has taken control.
wb.addEventListener('controlling', (event) => {
window.location.replace('/login');
});
// Send a message telling the service worker to skip waiting.
// This will trigger the `controlling` event handler above.
wb.messageSW({type: 'SKIP_WAITING' });
});
wb.register();
}
// set a timeout in case the service worker has already installed.
setTimeout(function () {
window.location.replace('/login');
}, 30000);
</script>
Main page (index.html, etc)
(Handles if the user is coming to the app with a server worker ready to activate, so it needs a refresh to get the right assets/code loaded)
<script type="module">
import { Workbox } from '/assets/workbox/workbox-window.prod.mjs';
if ('serviceWorker' in navigator) {
const wb = new Workbox('/serviceworker');
wb.addEventListener('activated', (event) => {
// `event.isUpdate` will be true if another version of the service
// worker was controlling the page when this version was registered.
if (!event.isUpdate) {
// service worker was updated and activated for the first time.
// If your service worker is configured to precache assets, those
// assets should all be available now.
// this will only happen if the browser was closed when a new version was made available
// and it will only happen once per service worker install.
// Reload to so all libs are correct version.
window.location.reload(true);
}
});
wb.register();
}
</script>

Related

Chrome manifest V3 extensions and externally_connectable documentation

After preparing the migration of my chrome manifest V2 extension to manifest V3 and reading about the problems with persistent service workers I prepared myself for a battle with the unknown. My V2 background script uses a whole bunch of globally declared variables and I expected I need to refactor that.
But to my great surprise my extension background script seems to work out of the box without any trouble in manifest V3. My extension uses externally_connectable. The typical use case for my extension is that the user can navigate to my website 'bla.com' and from there it can send jobs to the extension background script.
My manifest says:
"externally_connectable": {
"matches": [
"*://localhost/*",
"https://*.bla.com/*"
]
}
My background script listens to external messages and connects:
chrome.runtime.onMessageExternal.addListener( (message, sender, sendResponse) => {
log('received external message', message);
});
chrome.runtime.onConnectExternal.addListener(function(port) {
messageExternalPort = port;
if (messageExternalPort && typeof messageExternalPort.onDisconnect === 'function') {
messageExternalPort.onDisconnect(function () {
messageExternalPort = null;
})
}
});
From bla.com I send messages to the extension as follows
chrome.runtime.sendMessage(EXTENSION_ID, { type: "collect" });
From bla.com I receive messages from the extension as follows
const setUpExtensionListener = () => {
// Connect to chrome extension
this.port = chrome.runtime.connect(EXTENSION_ID, { name: 'query' });
// Add listener
this.port.onMessage.addListener(handleExtensionMessage);
}
I tested all scenarios including the anticipation of the famous service worker unload after 5 minutes or 30 seconds inactivity, but it all seems to work. Good for me, but something is itchy. I cannot find any documentation that explains precisely under which circumstances the service worker is unloaded. I do not understand why things seem to work out of the box in my situation and why so many others experience problems. Can anybody explain or refer to proper documentation. Thanks in advance.

next-pwa runtimecaching does not produce strategies

I start using next-pwa and the basic setup worked like a charm. Now it want to play with the runtime caching option, which does not work for me:
My next.config.js includes the standard cache entries plus a custom one that should use the strategy StaleWhileRevalidate for each request going to /api/todoitem:
const withPWA = require("next-pwa");
const defaultRuntimeCaching = require("next-pwa/cache");
module.exports = withPWA({
reactStrictMode: true,
pwa: {
dest: "public",
disable: false, // disable PWA
register: true,
skipWaiting: true,
runtimeCaching: [
{
urlPattern: /\/api\/todoitem/,
method: "GET",
handler: "StaleWhileRevalidate",
options: {
cacheName: "todoApp-api",
expiration: {
maxEntries: 64,
maxAgeSeconds: 24 * 60 * 60, // 24 hours
},
},
},
...defaultRuntimeCaching,
],
},
});
Restart npm run dev fire up the browser -> fetch GET /api/todoitem -> and console.log tells me
workbox Network request for '/api/todoitem' returned a response with status '200'.
workbox Using NetworkOnly to respond to '/api/todoitem'
I tried a number of combinations of regexes, including defaultRuntimeCaching before or after my runtimeCache entry to no avail.
Any hints to get custom runtimeCache rules working would be greatly appreciated.
next.js 12.0.7
next-pwa 5.4.4
node.js v14.18.2
After some research I found:
In development mode, next-pwa creates a service worker that disables caching. It even tells me so on the console ;-):
[PWA] Build in development mode, cache and precache
are mostly disabled. This means offline support is
disabled, but you can continue developing other
functions in service worker.
When building the app via next build it creates a service worker that uses my custom rules and when starting the app next start the rules seem to work.
A bit hard to debug, so I tried to set mode: "production" on my developer machine but then for some reason the service worker gets rebuilt at every other request which brings the app to a grinding halt.

net::ERR_CONNECTION_RESET with service worker in Chrome

I have a very simple service worker to add offline support. The fetch handler looks like
self.addEventListener("fetch", function (event) {
var url = event.request.url;
event.respondWith(fetch(event.request).then(function (response) {
//var cacheResponse: Response = response.clone();
//caches.open(CURRENT_CACHES.offline).then((cache: Cache) => {
// cache.put(url, cacheResponse).catch(() => {
// // ignore error
// });
//});
return response;
}).catch(function () {
// check the cache
return getCachedContent(event.request);
}));
});
Intermittently we are seeing a net::ERR_CONNECTION_RESET error for a particular script we load into the page when online. The error is not coming from the server as the service worker is picking up the file from the browser cache. Chrome's network tab shows the service worker has successfully fetched the file from the disk cache but the request from the browser to the service worker shows as (failed)
Does anyone know the underlying issue causing this? Is there a problem with my service worker implementation?
This is likely due to a bug in Chrome (and potentially other browsers as well) that could result in a garbage collection event removing a reference to the response stream while it's still being read.
Its fix in Chrome is being tracked at https://bugs.chromium.org/p/chromium/issues/detail?id=934386.

How do I uninstall a Service Worker?

After deleting /serviceworker.js from my root directory, Chrome still runs the service worker that I removed from my webroot. How do I uninstall the service worker from my website and Chrome so I can log back into my website?
I've tracked the issue down to Service Work's cache mechanism and I just want to remove for now until I have time to debug it. The login script that I'm using redirects to Google's servers for them to login to their Google account. But all I get from the login.php page is an ERR_FAILED message.
Removing Service Workers Programmatically:
You can remove service workers programmatically like this:
navigator.serviceWorker.getRegistrations().then(function(registrations) {
for(let registration of registrations) {
registration.unregister();
}
});
Docs: getRegistrations, unregister
Removing Service Workers Through The User Interface
You can also remove service workers under the Application tab in Chrome Devtools.
You can also go to the URL: chrome://serviceworker-internals/ and unregister a serviceworker from there.
You can do this through Chrome Developer Tool as well as Programatically.
Find all running instance or service worker by typing
chrome://serviceworker-internals/
in a new tab and then select the serviceworker you want to unregister.
Open Developer Tools (F12) and Select Application. Then Either
Select Clear Storage -> Unregister service worker
or
Select Service Workers -> Choose Update on Reload
Programatically
if(window.navigator && navigator.serviceWorker) {
navigator.serviceWorker.getRegistrations()
.then(function(registrations) {
for(let registration of registrations) {
registration.unregister();
}
});
}
In Google Chrome, you can go to Developer tools (F12) -> Application -> Service worker and unregister the service workers from the list for that specific domain.
This method is effective in development mode of a site and mostly they run on localhost which is you may need for other project's development.
FYI, in case you are using MacOS Safari browser, there is one way to forcibly unregister a service worker (steps and images for Safari 12.1):
Safari > Preferences... > Privacy > Manage Website Data…
Enter domain name (ex. 'localhost'), click "Remove"
Note: In addition to service workers, this also will erase all caches, cookies, and databases for this domain.
In addition to the already correct answers given, if you want also to delete the SW cache you can invoke the following method:
if ('caches' in window) {
caches.keys()
.then(function(keyList) {
return Promise.all(keyList.map(function(key) {
return caches.delete(key);
}));
})
}
More in this article (Paragraph: "Unregister a service worker")
Another possibility, via Browser, is by accessing the "Cache Storage" section and click on the "Clear Site Data" button:
You should detecte two API in your devices: getRegistrations and getRegistration. The service-worker not has a unique set of APIs in all platforms. For example, some browsers only have a navigator.serviceWorker.getRegistration, no navigator.serviceWorker.getRegistrations. So you should consider with both.
safely uninstall Service Worker
if ('serviceWorker' in navigator) {
navigator.serviceWorker.getRegistrations().then(function (registrations) {
for (const registration of registrations) {
// unregister service worker
console.log('serviceWorker unregistered');
registration.unregister();
}
});
}
to detect service worker:
navigator.serviceWorker.controller
Code to for deletion of service worker:
navigator.serviceWorker.getRegistrations()
.then(registrations => {
registrations.forEach(registration => {
registration.unregister();
})
});
navigator.serviceWorker.getRegistrations().then(function(registrations) {
for(let registration of registrations) {
registration.unregister()
} })
if(window.navigator && navigator.serviceWorker) {
navigator.serviceWorker.getRegistrations()
.then(function(registrations) {
for(let registration of registrations) {
registration.unregister();
}
});
}
if ('caches' in window) {
caches.keys()
.then(function(keyList) {
return Promise.all(keyList.map(function(key) {
return caches.delete(key);
}));
})
}
if ('serviceWorker' in navigator) {
navigator.serviceWorker.getRegistrations().then(function (registrations) {
for (const registration of registrations) {
// unregister service worker
console.log('serviceWorker unregistered');
registration.unregister();
setTimeout(function(){
console.log('trying redirect do');
window.location.replace(window.location.href); // because without redirecting, first time on page load: still service worker will be available
}, 3000);
}
});
}
IF your service worker don't let you update your files. You will need to replace serviceworker file (sw.js / ServiceWorker.js) with the next code:
self.addEventListener('install', function(e) {
self.skipWaiting();
});
self.addEventListener('activate', function(e) {
self.registration.unregister()
.then(function() {
return self.clients.matchAll();
})
.then(function(clients) {
clients.forEach(client => client.navigate(client.url))
});
});
Source here
as for me , i just use a new nonexistent scope service worker to replace old one,
ServiceWorker: {
events: true,
// what range of URLs a service worker can control. Use a nonexistent path to disable ServiceWorker
scope: '/disable-service-worker/',
},
as for the app.js, i add below code to unregister old sw:
if ('serviceWorker' in navigator) {
navigator.serviceWorker.getRegistrations().then(registrations => {
for (const registration of registrations) {
// keep only serviceWorker which scope is /disable-service-worker/, The purpose is to make serviceWorker useless
if (registration.scope.includes('/disable-service-worker/') === false) {
registration.unregister()
}
}
});
// clear cache of service worker
caches.keys().then(keyList => {
return Promise.all(
keyList.map(key => {
return caches.delete(key);
}),
);
});
}
It can also be done in Chrome through application tab:
This code is compatible with Internet Explorer:
if (navigator.serviceWorker) {
navigator.serviceWorker.getRegistrations().then(
function(registrations) {
for (let idx in registrations) {
registrations[idx].unregister()
}
})
}
IE doesn't support 'for...of' and 'for...of' construction may lead to "'SCRIPT1004: Expected ';'" error
The other answers all add code to the live website to remove the service worker. However I didn't want to leave that live code running forever so I developed a solution that works from within the service worker itself. The steps are below, I posted more detail and explanation on my blog.
Delete the code that registers the service worker.
Replace the service worker script with the following file. The new code must be available from the same URL the previous service worker was at. If you had multiple service worker URLs in the past you should duplicate the code at all of them.
console.log("Cleanup Service Worker Starting");
caches.keys()
.then(keys =>
Promise.all(
keys.map(async key => console.log("caches.delete", key, await caches.delete(key)))))
.then(async () => {
console.log("registration.unregister", await registration.unregister());
})
.then(() => console.log("DONE"))
.catch(console.error);
This code is fairly straight forward. First it deletes all the caches, then it unregisters itself.
Users' browsers will automatically check for an updated service worker the next time they load your website, or the next event 24h after the last service worker check. This means that existing users will run this cleanup on their next visit.
If You want to unregister all of the registered service workers in Browser,
you can do it by opening ex.
Chrome: chrome://serviceworker-internals/
Brave brave://serviceworker-internals/
open DevTools > Console and paste this:
$$('.unregister').forEach(b => b.click())
Open this page: chrome://serviceworker-internals and click to unregister button.
If you want to unregister all service worker open the developer tools and run this code on above page.
document.querySelectorAll("button.unregister").forEach(item=>{ item.click()})
Open in Chrome
chrome://serviceworker-internals/?devtools
then F12 in Console
$$('.unregister').forEach(b => b.click())
Typical JavaScript loop thats compatible with everything:
navigator.serviceWorker.getRegistrations().then(function(registrations) {
var registrationslength = registrations.length;
for (var i = 0; i < registrationslength; i++) {
registrations[i].unregister();
}
})

"require not defined" when testing with proxyquireify stubs

Hello i have a project that uses gulp for the build framework, and used karma with jasmine for the testing.
I am trying to integrate proxyquireify to mock the requires, i just added proxyquireify as browserify plugin in karma config, as i am using karma-browserify.
But this results in an error when running the tests, in the first line, saying 'require is undefined'.
What am i doing wrong?
here is my karma config
// Karma configuration
// Generated on Wed Nov 26 2014 17:57:28 GMT+0530 (IST)
module.exports = function(config) {
config.set({
// base path that will be used to resolve all patterns (eg. files, exclude)
basePath: '',
// frameworks to use
// available frameworks: https://npmjs.org/browse/keyword/karma-adapter
frameworks: ['browserify', 'jasmine'],
// list of files / patterns to load in the browser
files: [
'./components/renderer/**/*.spec.js',
'./components/tracker/**/*.spec.js'
],
// list of files to exclude
exclude: [
],
// preprocess matching files before serving them to the browser
// available preprocessors: https://npmjs.org/browse/keyword/karma-preprocessor
preprocessors: {
'./components/**/*.spec.js' : [ 'browserify' ]
},
browserify: {
debug:true,
plugin: ['proxyquireify/plugin']
},
// test results reporter to use
// possible values: 'dots', 'progress'
// available reporters: https://npmjs.org/browse/keyword/karma-reporter
reporters: ['spec'],
// web server port
port: 9876,
// enable / disable colors in the output (reporters and logs)
colors: true,
// level of logging
// possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
logLevel: config.LOG_INFO,
// enable / disable watching file and executing tests whenever any file changes
autoWatch: false,
// start these browsers
// available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
browsers: ['Chrome'],
// Continuous Integration mode
// if true, Karma captures browsers, runs the tests and exits
singleRun: false
});
};
proxyquireify works internally by substituting the require function provided by browserify.
In this case it seems the new substituted require function was not exposed to global scope.
I went through the code and found out proxyquireify creates the new require function in node_modules/proxyquireify/lib/prelude.js named as newRequire.
the issue i was having was that the newRequire function was not exposed in the global scope as the require function, so i changed node_modules/proxyquireify/lib/prelude.js so that
// Override the current require with this new one
return newRequire;
becomes
// Override the current require with this new one
require = newRequire;
and the newRequire was properly exposed to global scope and everything worked fine. Since this change is reset every time i do a npm install, i created a gulp task in my case which does this change every time before tests are run, i will add the gulp task for reference
// Task to modify proxyquireify so that it works properly, there is a bug in the npm library
gulp.task('_patch:proxyquireify', function() {
gulp.src(['./node_modules/proxyquireify/lib/prelude.js'])
.pipe(replace(/return newRequire;/g, 'require = newRequire;'))
.pipe(gulp.dest('./node_modules/proxyquireify/lib'));
});
I run this task before executing the test tasks, like this
// Task to run tests
gulp.task('run:test', ['_patch:proxyquireify'], function() {
//logic to run tests
};
I hope this helps, thanks