I don't want to use expiration, because I don't know how often my resources (css, js, etc) will change. A case might be where I push something to production, but soon find out I have some bugs in the resources. So I fixed the bugs, then push the updated resources into production, but they're in production before the expiration date. How do I force the browser to use the updated resources instead of cache?
I don't want to use fingerprinting (dynamic resource filename based of version type), because those resources are located in an svn directory. Meaning, I would have to manually modify the filename using svn every time I change the code version (maybe there's somewhat to automate this?).
Utilizing http-equiv="last-modified" would be the preferred option, but that doesn't seem to work properly.
Also maybe I just don't know how all this works. Please enlighten me. Many thanks in advance.
-- Tri
One option would be to have all references to those resources to append a version number to the query string or elsewhere in the URL, for example "styles/site.css?version=1.0". That way, since the URL will change during your next release, your browser will see it as a different URL and will not use the file from cache.
If you ever change to using a CDN, deploying all resources to version numbered folders can achieve the same goal.
Related
I want to be able to get the checksum of a chrome extension and match that against the source code in a reliable way? the optimal way would be if a checksum is exposed through chrome apis without having to read the files.
I have searched hard for this and it seems that there is no reliable way, but i wanted to see if others have different ideas. Here are things that i considered:
I have got an extension with both the source code, and the extension files in chrome, i have found that after comparing both directories that manifest.json got its styling changed: indention and order of properties, and one icon was modified - probably compressed. means even after reading files content this is not reliable.
The chrome folder has a metadata folder which has hashes used to ensure the files are not modified on disks, however using these hashes is also not reliable, since these hashes are probably after the modifications mentioned above, and also there is no mention in the documentation about how these hashes are calculated, and so they can change without a prior notice.
Context: I have a production application (here if you want to look) that is currently using static asset revisioning using the gulp-rev-all package which is like gulp-rev except that it also handles dependencies when generating content hashes. It generates a new set of files that have static names (eg goals.js becomes goals.6a5aa614.js) and that reference each other using those static names. Then I serve those files up with the Fastly CDN on production, so my NodeJS server isn't being actively used for static assets. This has been working great.
Now I'm in the process of making the site work offline with service-workers. Since I'd done a lot of work on the syncing logic last year, the dynamic part of the site has been pretty easy to overhaul. But I'm a bit at a loss for what to do with my static assets.
I figured I'd use workbox, and that seems to work okay. But the workbox precache uses queries for busting the cache rather than changing filenames, and it seems dumb to do both. But if I stop using the versioned names, then how do I bust the cache on browsers that don't support service worker?
(I have another related question, which is does it make sense to keep using Fastly given that Fastly's responses will be opaque to the SW and therefore not necessarily a good option for precaching? Although without Fastly the app would become a lot slower for anyone who isn't using service workers, which sounds antithetical to the PWA approach. Should I add an nginx cache or something? (I have little idea what this is but I've heard it mentioned a few times))
It seems to me like there's got to be an elegant solution for this, but my understanding of gulp is sufficiently limited that it's hard for me to know what is possible, and my understanding of ServiceWorkers & caching is sufficiently limited that it's hard for me to know exactly what I want.
I'm therefore having trouble getting any traction on this question:
How can I adapt my gulp static asset revisioning to work with ServiceWorkers?
One thing that would be helpful is just a link to examples of how other production applications handle this.
Service worker works best on top of good regular caching strategy. You should continue to revision your static file names then cache them in the service worker. Avoid libraries that alter the URL via a query string, you don't need that feature as you're already revisioning the URL.
If your assets are served from another origin (I guess this is what you mean when you're talking about Fastly), then allow them to be requested via CORS (via Access-Control-Allow-Origin: *), that means they won't be opaque.
You should keep the file-revisioned assets. For a complete example using gulp and precaching take a look here.
You basically want to use a cache-first, then network pattern. You can match for requests to /goals.*.js/ => and then, depending on your application, you could decide to use the cached goals.js even if the [hash] doesn't match, and then download the new goals.[hash].js in the background.
Or, if the hash doesn't match, you may want to go with a network first, fallback to fuzy-matching cache of goals.js.
As for Nginx. It is often suggested to use a reverse proxy for the static asset serving. Node.js is not great for this task. Here is a good working example. If you go with this setup your flow for the static assets would look like this:
CDN => <= Nginx => Node.js Origin.
If you use AWS. Then a typical setup with Cloudfront CDN would involve setting your Nginx reverse proxy node.js EC2 box as the origin. Then you'd setup a behavior for "/" route and your "/assets" route.
The "/" behavior would likely have a short TTL, while the "/assets/" behavior (route in Cloudfront) would have your long-term (max-age=3153600) caching strategy.
In this scenario almost all static assets will be served from the CDN (Cloudfront). It will only have to go back to your origin when you deploy new code with a new set of file-revisioned assets.
You then use the service worker to make all repeat visits extremely quick, potentially even using an outdated asset (matching name, different hash) on initial repeat visit, by going cache first, then network. Thus all repeat users with the service worker will have a fast as possible initial page load.
Those without it will still get all the benefits of file revisioned, long-term browser cached assets with CDN edge serving.
In the company I am I was asked to write an autoupdate function a la chrome. I.e. It should check periodically whether a new version is available, download the new version and apply it silently the next time the application starts.
I already have something up and running but it is more like a dirty hack than something I feel happy about it. So, I would like to know how to design and implement such a solution. My horrible hack works as this:
Have a mechanism to check whether a new version exists (a database query or a web service)
Download a full zip with the whole new version.
Check file signature. If everything went alright, set a registry value: must update to true.
When the application restarts, if the must update value is true, launch an update program and exist.
The update deletes the contents of the application folder, unzips the update and replaces the old contents, launches the application and exits.
Now, I would like to change it, so it works cleaner. I am planning to send the update as a bsdiff file. It gets downloaded. But the question is, what happens next?
When do apply the update?
Who is in charge of applying the patch? is it the program itself or is it a third program, as I did, which is in charge of applying the patch and relaunch the application?
If your going down the C++ route you can go to chromium and download the Chrome source code and dig around to see how the update is done, this might give you a better idea on how to approach it. Here's an article that might help.
If your familiar with .NET the recently release nuget also has an auto update feature that might be useful to look at, you can get the source code from here. David Ebbo has a blog about how its done here.
I'm not up to date on Delphi but you might be able to use either of the above options.
The workflow you proposed is more or less like it should work, but there's no need to re-invent the wheel - there are plenty libraries out there that will do this for you. Using a 3rd party library has the benefit of keeping your code cleaner while making sure the dirty process of auto-update is contained and working flawlessly.
Trust me, I know. I'm the author of NAppUpdate, an app update framework for .NET (which you might want to try out or learn from).
So, after giving it a lot of though, this is what I came with (for active directory I will refer to the directory where the main program lies, active program is the main program and update program is the one that replaces the active program and its resource files):
The active program checks if there is a new version every certain amount of time. If so, download it
Prepare new version in a separate folder (this can be done by copying the contents of the directory with the program to a subdirectory and applying a binary patch, or simply unziping the new version).
Set a flag that indicates that a new version is ready.
When a program is exiting (and one has to control for different interrupts here):
The active program checks the new version ready flag. Launch the update program and exit.
The update program checks if it can write in the active directory. If so, replaces the contents with the prepared version.
The update program has to recheck links and update them accordingly.
So guys, if you have a better workflow, please tell me.
You could literally use the Google Chrome update workflow by using the Google Chrome updater:
http://code.google.com/p/omaha/
They open sourced it Feb 2009.
I'm working on a project with two other developers that's built on FireBreath. So far, I've been able to get things working perfectly on my machine, but we need to coordinate our development via Mercurial. So I pushed my files to the repository and thought all was well.
Unfortunately, that doesn't work.
The various .vcproj files that make up the solution all contain hard-coded references to my local file system. This works fine for me, because I'm not moving the project around. But when you try to build the solution on another machine with a different file structure (different drive letter, different folder location, etc.) everything breaks.
I used FireBreath's standard project generation script (Python) and then the Visual Studio CMake script (prep2008.cmd) to generate the solution files. What can I do to tweak things so that other developers can use the same code base?
If your developers are not using the same build/make/project files, this could quickly become a maintenance nightmare. So you should definitively all use the same .vcproj files. (An exception to this would be if the project files were generated from some other files. In that case treat those other files in the way described above.)
there's two ways to deal with the problem of differing setups on different machines. One is to make all paths relative to the project's path. The other is to use environment variables to refer to files/tools/libraries/whatever. IME it's best to use relative paths for everything that can be checked out with the project, and use environment variables for the rest. Add a script that checks for the existence of all necessary environment variable, pointing out the meaning of any missing ones, and run this as a build prerequisite, so whoever tries to get a new build machine up and running gets hints at what to do.
To make sure that everyone caught the updated comments from sbi's answer, let me give you the "definitive" answer from the FireBreath devs.
Your build directory is disposable; you should never share .vcproj files. Instead, you should regenerate your build/ directory any time you change the project and on each new computer, just like any project that uses CMake.
For more information, see http://colonelpanic.net/2010/11/firebreath-tips-working-with-source-control/
For reference, I am the primary author of FireBreath and I wrote the article.
I'm not familiar with FireBreath, but you need to make the references relative, and then recreate that relative structure on every machine. That is, if your project sits in "c:\myprojects\thisproject" and has an additional include directory "c:\mydir\mylib\include", then the latter path needs to be replaced with "....\mydir\mylib\include".
EDIT: I rewrote my anyswer to make it clearer. When I got you correctly, your problem is that FireBreath generates those .vcproj files with absolute paths in it, and you want to use this .vcproj files on a different developer machine.
I see 3 options:
Live with it. That means, make sure, every team member has the same file structure / view to the file system, tools installed in the same place.
Ask the authors of FireBreath to change their .vcproj generator to allow relative paths, use of environment variables etc.
If 1 or 2 does not work, write a program or script for changing the absolute path to relatives in those .vcproj files. Run this script whenever you have to regenerate your FireBreath project.
What you should not do due to the FireBreath FAQ: don't change the .vcproj manually, those changes will be lost next time the project is regenerated.
EDIT: seems that "option 4." turned out to be the best solution: generating those .vcproj files for each developer individually. Hope my suggestions were helpful, either.
I want to give credit to all open source libraries we use in our (commercial) application. I thought of showing a HTML page in our about dialog. Our build process uses ant and the third party libs are committed in svn.
What do you think is the best way of generating the HTML-Page?
Hard code the HTML-Page?
Switch dependency-management to apache-ivy and write some ant task to generate the html
Use maven-ant-tasks and write some ant task to generate the HTML
Use maven only to handle the dependencies and the HTML once, download them and commit them. The rest is done by the unchanged ant-scripts
Switch to maven2 (Hey boss, I want to switch to maven, in 1 month the build maybe work again...)
...
What elements should the about-dialog show?
Library name
Version
License
Author
Homepage
Changes made with link to source archive
...
Is there some best-practise-advice? Some good examples (applications having a nice about-dialog showing the dependencies)?
There are two different things you need to consider.
First, you may need to identify the licenses of the third-party code. This is often down with a THIRDPARTYLICENSE file. Sun Microsystems does this a lot. Look in the install directory for OpenOffice.org, for example. There are examples of .txt and .html versions of such files around.
Secondly, you may want to identify your dependencies in the About box in a brief way (and also refer to the file of license information). I would make sure the versions appear in the About box. One thing people want to quickly check for is an indication of whether the copy of your code they have needs to be replaced or updated because one of your library dependencies has a recently-disclosed bug or security vulnerability.
So I guess the other thing you want to include in the about box is a way for people to find your support site and any notices of importance to users of the particular version (whether or not you have a provision in your app for checking on-line for updates).
Ant task seems to be the best way. We do a similar thing in one of our projects. All the open source libraries are present in a specified folder. An Ant task reads the manifest of these libraries, versions and so on and generates an HTML, copies into another specified folder from where it is picked up by the web container.
Generating the page with each build would be wasteful if the libraries are not going to change often. Library versions may change, but the actual libraries don't. Easier to just create a HTML page would be the easiest way out, but that's one more maintenance head ache. Generate it once and include it with the package. The script can always be run again in case some changes are being made to the libraries (updating versions, adding new libraries).