I have a lot of images in a folder that are used in the application. When using the cache manifest it would be easier maintenance wise if I could specify a wild card to load all the images or files in a certain directory to be cached.
E.g.
CACHE MANIFEST
# 2011-11-3-v0.1.8
#--------------------------------
# Pages
#--------------------------------
../index.html
../edit.html
#--------------------------------
# JavaScript
#--------------------------------
../js/jquery.js
../js/main.js
#--------------------------------
# Images
#--------------------------------
../img/*.png
Can this be done? Have tried it in a few browsers with ../img/* as well but it doesn't seem to work.
It would be easier, but how's it going to work? The manifest file is something which is parsed and acted upon in the browser, which has no special knowledge of files on your server other than what you've told it. If the browser sees this:
../img/*.png
What is the first image the browser should request from the server? Let's start with these:
../img/1.png
../img/2.png
../img/3.png
../img/4.png
...
../img/2147483647.png
That's all the images that might exist with a numeric name, stopping semi-arbitrarily at 231-1. How many of those 2 billion files exist in your img directory? Do you really want a browser making all those requests only to get 2 billion 404s? For completeness the browser would probably also want to request all the zero-filled equivalents:
../img/01.png
../img/02.png
../img/03.png
../img/04.png
...
../img/001.png
../img/002.png
../img/003.png
../img/004.png
...
../img/0001.png
../img/0002.png
../img/0003.png
../img/0004.png
...
Now the browser's made more than 4 billion HTTP requests for files which mostly aren't there, and it's not yet even got on to letters or punctuation in constructing the possible filenames which might exist on the server. This is not a feasible way for the manifest file to work. The server is where the files in the img directory are known, so it's on the server that the list of files has to be constructed.
I don't think it works that way. You'll have to specify all of the images one by one, or have a simple PHP script to loop through the directory and output the file (with the correct text/cache-manifest header of course).
It would be a big security issue if browsers could request folder listings - that's why Tomcat turns that capability off by default now.
But, the browser could locate all matches to the wildcards referenced by the pages it caches. This approach would still be problematic (like, what about images not initially used but set dynamically by JavaScript, etc., and it would require that all cached items not only be downloaded but parsed as well).
If you are trying automate this process, instead of manually doing it. Use a script, or as I do I use manifestR. It will output your manifest/appcache file and all you have to do is copy and paste. I've used it successfully and usually only have to make a few changes.
Also, I recommend using the network header with the wild card:
NETWORK:
*
This allows all assets from other linked domains via JSON, for instance, to download into the cache. I believe that this is the only header where you can specify a wildcard. Like the others have said here, it's for security reasons.
The cache manifest is now deprecated and you should use HTML headers to control caching.
For example:
<meta http-equiv="Cache-control" content="public">
Public - may be cached in public shared caches.
Private - may only be cached in private cache.
No-Cache - may not be cached.
No-Store - may be cached but not archived.
Related
Well, using HTML5 file handlining api we can read files with the collaboration of inpty type file. What about ready files with pat like
/images/myimage.png
etc??
Any kind of help is appreciated
Yes, if it is chrome! Play with the filesytem you will be able to do that.
The simple answer is; no. When your HTML/CSS/images/JavaScript is downloaded to the client's end you are breaking loose of the server.
Simplistic Flowchart
User requests URL in Browser (for example; www.mydomain.com/index.html)
Server reads and fetches the required file (www.mydomain.com/index.html)
index.html and it's linked resources will be downloaded to the user's browser
The user's Browser will render the HTML page
The user's Browser will only fetch the files that came with the request (images/someimages.png and stuff like scripts/jquery.js)
Explanation
The problem you are facing here is that when HTML is being rendered locally it has no link with the server anymore, thus requesting what /images/ contains file-wise is not logically comparable as it resides on the server.
Work-around
What you can do, but this will neglect the reason of the question, is to make a server-side script in JSP/PHP/ASP/etc. This script will then traverse through the directory you want. In PHP you can do this by using opendir() (http://php.net/opendir).
With a XHR/AJAX call you could request the PHP page to return the directory listing. Easiest way to do this is by using jQuery's $.post() function in combination with JSON.
Caution!
You need to keep in mind that if you use the work-around you will store a link to be visible for everyone to see what's in your online directory you request (for example http://www.mydomain.com/my_image_dirlist.php would then return a stringified list of everything (or less based on certain rules in the server-side script) inside http://www.mydomain.com/images/.
Notes
http://www.html5rocks.com/en/tutorials/file/filesystem/ (seems to work only in Chrome, but would still not be exactly what you want)
If you don't need all files from a folder, but only those files that have been downloaded to your browser's cache in the URL request; you could try to search online for accessing browser cache (downloaded files) of the currently loaded page. Or make something like a DOM-walker and CSS reader (regex?) to see where all file-relations are.
Gurus of SO
I am trying to play with CACHE MANIFEST/HTML5. My app is JS heavy and built on jquery/jquerymobile.
This is an excerpt of what my Manifest looks like
CACHE MANIFEST
FALLBACK:
/
NETWORK:
*
CACHE:
/css/style.css
/js/jquery.js
But somehow, the app doesn't load the files the first time itself and the entire app breaks down.
Is my format wrong?
Should I never load JS into the Cache?
How should I treat this differently to always check the network first if anything isn't available and only load stuff available from the Cache?
Thank you.
I tried a simple page with your cache manifest and it worked fine for me, so I'm not really sure what the problem is. But,
Yes, there is something wrong with the format. The entries in the FALLBACK section need to have two parts: a pattern, and a URL. This says "if any page matching the pattern is not available offline, display the URL instead (which will be cached)." The main example of this (as shown here) is "/ /offline.html", which means "for all pages, if we are offline and they are not cached, display /offline.html instead." However, I don't think this is the source of your problem since I tested it with your exact manifest and it still worked.
There is nothing special about JS files. It should be fine to load them into the cache.
I don't understand the third question. There are possibly two goals here: a) how do you check to see if there is a newer version of the file available online first, before going back to the cache, and b) how do you check the network to see if there is a file that is not cached, and if we are offline, fall back to an error page. The answer to (a) is that once you have turned on the cache manifest, things work very differently. It will never check for new versions of the files unless there is a new version of the manifest also. So you must always update the manifest whenever you change any files. The answer to (b) is the FALLBACK section.
See Dive Into HTML5's excellent chapter on this, particularly the section "The fine art of debugging, a.k.a. “Kill me! Kill me now!”" which explains how the manifest updates.
Also I don't think we've gotten to the meat of your question, because it's unclear what you mean by "the app doesn't load the files the first time itself". Which files don't load? Do they load properly after a refresh? Etc.
The only way I got this to work to refresh a cache was to rename the manifest file with a commit number or timestamp, and change the cache declaration to
<html manifest='mymanifest382330.manifest'>
I made this part of my build.
Every URL can be linked to a single cache manifest. But I want several cache manifests linked to a same URL. Here is the reason:
Some files I want to be cached are rarely updated and large.
So everytime the cache gets updated these large files get re-downloaded even though they may not have been changed.
So I want to split up the cache. One cache for theses rarely updated large files and another cache for the often updated light files.
Do you guys have any idea how to split up an HTML5 cache?
The most efficient way is:
a) Use far-future expiration date (max-age) on all resources mentioned in manifest's CACHE section and add timestamp suffix to each file in the CACHE section, e.g.:
CACHE:
menu_1355817388000.js
toolbar_1355817389100.js
b) When any of the above files change on the server, regen/update manifest to change the timestamp. Only the file with the modified timestamp will get downloaded next time. Mission accomplished.
Note: Reload the page twice in the browser, as on the first refresh browser downloads just the manifest and uses old cached resources to paint the page. This is done to speed up displaying the page (there are tricks to handle this issue of double refresh, but they are outside the scope of your question)
See more info in this long but best article I ever seen on appcache.
Use an iframe
Your page's cache manifest would include the light files and the cache manifest of an iframe loaded by this page would include the large files
On chrome the iframe's application cache will also be used for the page. I didn't tested this method on other browsers yet.
see a live example at http://www.timer-tab.com and if you are using chrome see its split up cache at chrome://appcache-internals/
When the manifest file is changed and the files of the application cache are downloaded again, the normal HTTP caching rules still apply. This means that if you set the correct HTTP caching headers for these large files, you'll get a 304 so these files are not downloaded again. So it's not necessary to split the application cache.
Maybe an answer but I'd more like to shed some light on my findings as a I troubleshoot my own webapp.
I've discovered that I can use 2 iframes (manifest_framework) and (manifest_media) to load the manifests, but i'm still not exactly clear how they are targetted, but I had limited success.
manifest_framework:
CACHE MANIFEST
CACHE:
appdata.ini
dialog.png
jquery.min.js
login.htm
login.js
manifest.appcache.js
NETWORK:
*
FALLBACK:
manifest_media:
CACHE MANIFEST
CACHE:
manifest_fwk.php
od/audio_track_1_1.m4a
od/audio_track_1_2.m4a
od/audio_track_1_3.m4a
od/audio_track_1_4.m4a
od/video_1.mp4
od/video_2.mp4
od/video_3.mp4
NETWORK:
*
FALLBACK:
./ webapp.php
./index.php is the page the 'landing page' which itself isn't cached but falls back to webapp.php when offline.
What I don't understand is how these link to the webapp.php page.
I am finding I can only get access to one or the other manifests cache.
The above works in mobile safari, the media would be cached, and image but not necessarily the JS or images in the framework manifest.
Anyone have more examples where multiple manifests are referenced from the one URL/page?
The W3C working group has abandoned the file system api, so it SHOULD NOT BE USED anymore.
We'll likely see it fall off the next version of Chrome.
http://www.w3.org/TR/file-system-api/
CACHE MANIFEST
# This is a comment.
# Cache manifest version 0.0.1
# If you change the version number in this comment,
# the cache manifest is no longer byte-for-byte
# identical.
demoimages/mypic.jpg
demoimages/yourpic.jpg
demoimages/ourpic.jpg
sr/scroll.js
NETWORK:
# All URLs that start with the following lines
# are whitelisted.
# whitelisted items are needed to help the site function, you could put regularly
# changing items here
http://example.com/examplepath/
http://www.example.org/otherexamplepath/
CACHE:
# Additional items to cache.
demoimages/allpics.jpg
FALLBACK:
demoimages/currentImg.jpg images/stockImage.jpg`
If the Iframe trick does not work, use the HTML5 FileSystem API
See http://updates.html5rocks.com/2012/04/Taking-an-Entire-Page-Offline-using-the-HTML5-FileSystem-API
I was wondering if it's possible to include hashes of external files within a HTML file. This should basically serve 2 purposes:
Including unencrypted content into encrypted pages. (The hashes would ensure the integrity of the data)
Allow more caching for resources that are used on multiple pages
Let's focus on the second case and clarify it with a made-up example:
<script type="text/javascript" src="jQuery-1.5.1.min.js" hash-md5="b04a3bccd23ddeb7982143707a63ccf9">
Browsers could now download and cache the file initially. For every following page that uses the same hash, it would be clear that the cached version could be used. This technique should work independent of file origin, file type, transmission protocol and without even hitting the server once to know that a file is already cached locally.
My question is: Is such a mechanism available in HTML?
The following example is just to clarify the idea further and does not add new information.
An example of a library included in 2 unrelated pages would lead to the following steps.
User navigates to page A for the first time
Browser loads page A and looks for external files (images, scripts, …)
Browser finds page A includes a script with hash b04a3bccd23ddeb7982143707a63ccf9
Browser checks its cache and finds no file with that hash
Browser downloads the file from the given URL (gives a file on page A's domain)
Browser calculates hash and compares it with the hash as stated on page A
Browser adds file to its cache using the hash. If calculated hash would not have matched given hash, the file would have been rejected with an error message
Browser executes file.
At some point later in time:
User navigates to page B for the first time
Browser loads page B and looks for external files (images, scripts, …)
Browser finds page B includes a script with hash b04a3bccd23ddeb7982143707a63ccf9
Browser checks its cache and finds a file with that hash
Browser loads file from cache. The browser did not care about the URL given on page B pointing to the file. Also, it did not matter how the file's content found its way into the cache – protocol, encryption of connection and source are ignored. No connection to any server was made to load the file for page B
Browser executes file.
It's basically a kernel of a good idea, but I don't think there's anything in HTML to support it. Might be able to kludge something together with JavaScript, I suppose.
It is not necessary and not a new idea at all.
You can do this, using your example, omitting attr. "type" (for brevity):
<script src="jQuery-1.5.1.min.js?b04a3bccd23ddeb7982143707a63ccf9">
This has been practised using the file's timestamp instad of MD5 for a long time on quite a few sites, Rails supports it too, see here (look for "timestamp"), or here for an example with PHP.
Also see How to set up caching for css/js static properly
I have an application which used to use the HTML5 offline cache. Now I've decided to not use it anymore and removed the manifest attribute from the index.html file. However, browsers still regard this site as cached and refuse to update the index.html file.
Even updating the manifest doesn't help. How can I remove the site from the user's offline caches? Am I stuck with a cached web site forever?
You need to make sure the manifest file isn't being cached, which by default it will be.
Adding
ExpiresActive On
ExpiresDefault "access"
To your .htaccess will stop everything being cached, though you really just want the manifest file to be cached in this way like this: (remember to update filename)
<Files cache.manifest>
ExpiresActive On
ExpiresDefault "access"
</Files>
You really need to do that first, but this should alleviate the problem.
I'd recommend reading through Mark Pilgrim's page on this as well.
Try changing contents of your manifest to simply CACHE MANIFEST with no files listed. The clients should retrieve the new manifest next time they hit the site and their cache should be removed.
Note however that they won't be using this new, empty manifest until they refresh the page.
I've found that in some cases on some browsers they don't necessarily grab the new manifest right away. This behavior seems inconsistent though. When this happens I tend to clear their caches / offline storage manually in order to force them to update (though I understand you can't necessarily get users to do this).