I have a caching issue. Chrome does not always load newer versions of site assets, most often Javascript files loaded by Require.js. Right now I've been having this problem for over 24 hours with a particular file.
If I load the page with devtools (network tab) open, the offending files typically show a HTTP 200 response, but in the "Size" column it shows "(from cache)". In the Headers details it shows "Provisional headers are shown". Wireshark shows that the file is indeed not requested from the server.
Chrome shows the Last-Modified date of the file as Sat, 06 Dec 2014 01:27:55 GMT, but my below raw request to the server clearly indicates the file has changed much more recently.
If I do a raw request myself I don't see anything in the headers returned by the server that should cause this problem:
GET /js/path/to/file.js HTTP/1.1
Host: static.mydomain.com
User-Agent: Matt
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Vary: Accept-Encoding
Content-Type: application/javascript
Accept-Ranges: bytes
ETag: "4203477418"
Last-Modified: Fri, 16 Jan 2015 18:28:30 GMT
Content-Length: 5704
Date: Fri, 16 Jan 2015 21:05:06 GMT
Server: lighttpd/1.4.33
.... data here ...
The issue has been reported by chrome users on multiple OSes with different versions of chrome, but I do not typically receive reports of caching issues on other browsers. (Right now I'm on "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.71 Safari/537.36")
Edit:
The problem seems to be most offensive with files loaded by Require.js, though I have encountered it with javascript directly referenced in the page as well.
What am I missing here? Why won't chrome check for a new version of the file?
As it turns out, browsers have poorly documented behavior regarding caching when the Cache-Control header is not specified in the server response (well, at least when no caching behavior is specified). In general, it seems that in this case a browser determines how long to cache the item for based on the file's last modified date (if declared in the response), the current date, and ????
See: https://webmasters.stackexchange.com/questions/53942/why-is-this-response-being-cached
Sadly, Google's official page on HTTP caching does not mention what happens if the header is not set: https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/http-caching
Edit:
I ran across some more specific information about the heuristics used, here: What heuristics do browsers use to cache resources not explicitly set to be cachable?
Related
Background
I have created a PWA test project to find out how to get audio caching working with Workbox,
including scrub/seek using the range requests plugin.
I want the app to precache all audio and for this audio to be playable offline, including scrub/seek.
Pre-caching the audio can be done in one of two ways:
Using Workbox injectManifest.
By manually adding audio files to a cache using cache.add(URL)
But audio files cached with the first method (injectManifest) will not scrub/seek because the Workbox pre-cache does not
support range request headers. So you need to put a range request enabled router in front of the
pre-cache for audio files if you want to be able to scrub through/seek within a cached audio file.
Problem
Pre-cached audio with a range requests enabled router will play and scrub/seek fine in Chrome and Firefox when app is served
from localhost but fails to play in Chrome when served from Firebase.
I see the same error for all audio files that are pre-cached with a range requests router in front of them:
Router is responding to: /media/audio/auto-pre-cached.mp3
Using CacheOnly to respond to '/media/audio/auto-pre-cached.mp3'
No response found in the 'act-auto-pre-cache-wbv4.3.1-actv0.0.1' cache.
The FetchEvent for "https://daffinm-test.firebaseapp.com/media/audio/auto-pre-cached.mp3" resulted in a network error response: the promise was rejected.
CacheOnly.mjs:115 Uncaught (in promise) no-response: The strategy could not generate a response for 'https://daffinm-test.firebaseapp.com/media/audio/auto-pre-cached.mp3'.
at CacheOnly.makeRequest (https://storage.googleapis.com/workbox-cdn/releases/4.3.1/workbox-strategies.dev.js:343:15)
Chrome versions tried:
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36
The files are present in the Workbox caches. The only difference I can see between locahost and Firebase is in the cached response headers:
Localhost
cache-control: max-age=3600
content-length: 3770956
content-type: audio/mpeg; charset=utf-8
Date: Mon, 07 Oct 2019 09:37:03 GMT
etag: "12456134-3770956-"2019-09-29T20:05:00.314Z""
last-modified: Sun, 29 Sep 2019 20:05:00 GMT
server: ecstatic-2.2.2
Firebase
accept-ranges: bytes
cache-control: max-age=3600
content-encoding: gzip
content-length: 3686565
content-type: audio/mpeg
date: Mon, 07 Oct 2019 11:47:43 GMT
etag: 267d9ec42517198c01e2cad893f1b14662a2d91904bc517aeda244c30358457c
last-modified: Mon, 07 Oct 2019 03:48:25 PDT
status: 200
strict-transport-security: max-age=31556926; includeSubDomains; preload
vary: x-fh-requested-host, accept-encoding
x-cache: MISS
x-cache-hits: 0
x-served-by: cache-lhr7363-LHR
x-timer: S1570448862.315027,VS0,VE1472
Firefox works fine in both cases.
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0
Code
You can find the code for the test app here including a full description of the test setup, expectations and results:
https://github.com/daffinm/audio-cache-test
And the app is currently deployed on Firebase here if you want to take a look:
https://daffinm-test.firebaseapp.com/
Question
Does anyone have any idea what's going on here and why the pre-cached audio with range request routers are failing to play in Chrome? Is this a Chrome bug and/or a Workbox mis-configuration and/or a Firebase configuration issue - or something completely different? (I have contacted Firebase support and they are being very helpful but are currently unable to enlighten me).
The presence of the Vary header in the Firebase responses sounds like the culprit. By default, the Cache Storage API will use the Vary header when determining whether or not there's a cache match. You can override this default behavior by passing in {ignoreVary: true} when querying the Cache Storage API. Workbox supports this as an option you can provide when creating your strategy, via the matchOptions parameter.
It looks like you're already passing in ignoreSearch: true, so you can just add ignoreVary: true alongside that.
I have an odd issue. I an using angularjs (Ionic) to load an external json file via a httpPromise.
All has been working correctly until yesterday when the remote files were moved to another host.
Now the problem is on an iphone4 running ios7 it tries to load the file but can't and crashes out with memory usage issues. inspecting it using xcode it quickly climbs to over 300mb and then crashes. does the same thing on two devices. runs fine on other phone emulators etc.
Now if I host the file on another server it works as expected.
the different response headers are:
the one that fails:
Accept-Ranges bytes
Connection close
Content-Length 721255
Content-Type application/json
Date Thu, 11 Dec 2014 06:04:15 GMT
Last-Modified Thu, 11 Dec 2014 05:12:57 GMT
Server LiteSpeed
Vary User-Agent
Working host:
Accept-Ranges bytes
Connection keep-alive
Content-Encoding gzip
Content-Type application/json
Date Thu, 11 Dec 2014 06:05:01 GMT
Last-Modified Thu, 11 Dec 2014 03:29:48 GMT
Server nginx/1.6.2
Transfer-Encoding chunked
Vary Accept-Encoding,User-Agent
Code used to get json file.
var deferred = $q.defer(),
httpPromise = $http.get('http://someurl.com.au/deals.json');
httpPromise.then(function success(response) {
so after all of that my question is why would the json file not load or return an error of some sort?
But cause such a memory spike?
The only main difference I see between servers is the connection configuration.
working uses Connection keep-alive and the one that fails is closed.
thoughts?
Additionally, I've just tried it on 3g and it works fine but via wireless it doesn't work?
i think you need to focus on the Content-Encoding gzip which is probably what is saving you from memory issues
I'm using nginx and Dojo to build an embedded UI driven by a set of JSON files. Our primary target browser is Chrome, but it should work with all modern browsers.
Changing the JSON files can change the UI drastically, and I use this to give different presentations to different users. See my previous question for the details (Configure nginx to return different files to different authenticated users with the same URI), but basically my nginx configuration is such that the same URI with different users can yield different content.
This all works very well, except when someone switches to a different user. Some browsers will grab those JSON files from their own internal cache without even checking with the server, which leaves the UI display the previous user's presentation. Reloading the page fixes it, but boy! would I rather the right thing happened automatically.
The obvious solution is to use the various cache headers, but they don't appear to help. I'm using the following nginx directives:
expires epoch;
etag off;
if_modified_since off;
add_header Last-Modified "";
... which yields the following response headers:
HTTP/1.1 200 OK
Server: nginx/1.4.1
Date: Wed, 24 Sep 2014 16:58:32 GMT
Content-Type: application/octet-stream
Content-Length: 1116
Connection: keep-alive
Expires: Thu, 01 Jan 1970 00:00:01 GMT
Cache-Control: no-cache
Accept-Ranges: bytes
This looks pretty conclusive to me, but the problem still occurs with Chrome 36 for OS X and Opera 24 for OS X (although Firefox 29 and 32 do the right thing). Chrome is content to grab files from its cache without even referring to the server.
Here's a detailed example, with headers pulled from Chrome's Network debug panel. The first time Chrome fetches /app/resources/states.json, Chrome reports
Remote Address:75.144.159.89:8765
Request URL:http://XXXXXXXXXXXXXXX/app/resources/screens.json
Request Method:GET
Status Code:200 OK
with request headers:
Accept:*/*
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8
Authorization:Basic dm9sdGFpcndlYjp2b2x0YWly
Cache-Control:max-age=0
Connection:keep-alive
Content-Type:application/x-www-form-urlencoded
DNT:1
Host:suitable.dyndns.org:8765
Referer:http://XXXXXXXXXXXXXXXXXXXXXX/
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36
X-Requested-With:XMLHttpRequest
and response headers:
Accept-Ranges:bytes
Cache-Control:no-cache
Connection:keep-alive
Content-Length:2369
Content-Type:application/octet-stream
Date:Wed, 24 Sep 2014 17:19:46 GMT
Expires:Thu, 01 Jan 1970 00:00:01 GMT
Server:nginx/1.4.1
Again, all fine and good. But, when I change the user (by restarting Chrome and then reloading the parent page), I get the following Chrome report:
Remote Address:75.144.159.89:8765
Request URL:http://suitable.dyndns.org:8765/app/resources/states.json
Request Method:GET
Status Code:200 OK (from cache)
with no apparent contact to the server.
This doesn't seem to happen with all files. A few .js files are cached, most are not; none of the .css files seem to be cached; all the .html files are cached, and all of the .json files are cached.
How can I tell the browser (I'm looking at you, Chrome!) that these files are good at the moment it requests them, but will never again be good? Is this a Chrome bug? (If so, it's strange that Opera also shows the problem.)
I believe I've found the problem. Apparently "Cache-Control: no-cache" is insufficient to tell the browser to, um, not cache the data. I added "no-store":
Cache-Control:no-store, no-cache
and that did the trick. No more caching by Chrome or Opera.
I had the same problem, with json being cached...
If you control the client application-code, a possible workaround is to just add a random-value query-parameter at the end of the URL.
So instead of calling:
http://XXXXXXXXXXXXXXX/app/resources/screens.json
you call, for example:
http://XXXXXXXXXXXXXXX/app/resources/screens.json?rand=rrrrrrrrrr
where rrrrrrrrrr is some random-value that is different in each call.
Then, the browser will not be able to reuse any cached values.
I am reasonably new to browser caching. I am attempting to get Chrome to permanently cache any static file with a query parameter (for cache busting purposes). I have set Cache-Control and Expires headers way into the future, which should be adequate to say "cache this forever". The resulting response headers:
HTTP/1.1 200 OK
Cache-Control: public, max-age=315360000
Connection: keep-alive
Content-Encoding: gzip
Content-Type: application/x-javascript
Date: Wed, 16 Jul 2014 09:29:54 GMT
Last-Modified: Wed, 16 Jul 2014 03:44:14 GMT
Server: nginx/1.6.0
Transfer-Encoding: chunked
Vary: Accept-Encoding
Firefox and Safari seem to respect this for all cachebusted (?v= query parameter) files. Chrome mostly follows the directives, except for Javascript. Most of the time it does a request with an If-Modified-Since header rather than loading from cache. Sometimes one of them will load from cache and the other will yield a request resulting in a 304. Usually when loading the page from a new tab it will load from cache, but not if you hit enter in the address bar.
I've observed other websites using what I think are the exact same headers, and the files are always loaded from cache. Some of them load from cache even if you do a refresh.
I understand cache behaviour is somewhat unpredictable, but I want to make sure I'm not overseeing something that's making Chrome do that?
I had the same issue with chrome and after some hours of trial and error I figuered out, that chrome seems to have a problem with the Vary Header
I've got this snippet in my Apache / .htaccess config and as soon as I comment the line "Header append Vary Accept-Encoding" Chrome starts caching .js and .css files
<FilesMatch "(\.js\.gz|\.css\.gz)$">
# Serve correct encoding type.
Header set Content-Encoding gzip
# Force proxies to cache gzipped & non-gzipped css/js files separately.
#Header append Vary Accept-Encoding
</FilesMatch>
It still does not work while running the request via our nignx server, because it is adding the Vary: Accept-Encoding header too, when delivering gzip compressed.
So far I can guess this is a problem that only happens with chrome and as a workaround I would change the configuration to append the header only if chrome (haven't checked for safari) is not the client until there is a better fix:
<FilesMatch "(\.js\.gz|\.css\.gz)$">
# Serve correct encoding type.
Header set Content-Encoding gzip
# Force proxies to cache gzipped & non-gzipped css/js files separately.
BrowserMatch "Chrome" ChromeFound
Header append Vary Accept-Encoding env=!ChromeFound
</FilesMatch>
I have an offline-enabled website that uses a cache manifest. I'm finding with Chrome that it is serving an older version of my stylesheet, even if I do a "Empty Cache and Hard Reload"
If I append ?foo=bar to the URL of the page or the CSS, the new version of the CSS is delivered.
My manifest is dynamically generated at /Manifest/Index (e.g. )
If I open the page in Chrome and check out Fiddler, I see a single request is made to the web server, as expected:
# Result Protocol Host URL Body Caching Content-Type Process Comments Custom
6 200 HTTP 10.6.4.67 /Manifest/Index 2,476 no-cache Expires: -1 text/cache-manifest; charset=utf-8 chrome:5484
Here is the header detail for /Manifest/Index
GET /Manifest/Index HTTP/1.1
Host: 10.6.4.67
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko)
Chrome/23.0.1271.97 Safari/537.11
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
HTTP/1.1 200 OK
Date: Thu, 10 Jan 2013 17:59:42 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 4.0.30319
X-AspNetMvc-Version: 4.0
Cache-Control: no-cache
Pragma: no-cache
Expires: -1
Content-Type: text/cache-manifest; charset=utf-8
Content-Length: 2476
Can anyone tell me why on earth a CSS file reference in this cache-manifest isn't updating unless I append a cache-busting querystring variable to the CSS? Especially even if I empty Chrome's cache??!
More info:
If I update the cache-manifest, I can open up Chrome's console and see the App Cache events fire:
Document was loaded from Application Cache with manifest /Manifest/Index
Application Cache Checking event
Application Cache Downloading event
Application Cache Progress event (0 of 61) http://x.x.x.x/Content/themes/base/jquery.ui.progressbar.css
Application Cache Progress event (1 of 61) http://x.x.x.x/Content/themes/base/jquery.ui.accordion.css
Snip
Application Cache Progress event (54 of 61) http://x.x.x.x/Content/Site.css
I do notice that some of the items in this list, like Site.css, are underlined. Why is that?
Thanks,
Chris
Clear your appcache in Chrome using: chrome://appcache-internals/ and remove it there.
Also you need to rebuild your manifest file each time you change the files contained in it for the new copied to be downloaded.
This is accomplished by using a random number in your manifest and generating it when files are edited.
For example in node.js
function generateCacheManifest(...) {
manifest = 'CACHE MANIFEST';
manifest += '#version ' + Math.random();
...
}
Yes the random number can be in a comment. The point is that Chrome will check the cache manifest and when it sees that nothing has changed it will not fetch the updated files.
Change a file, change your manifest, it's that simple.