Caching with Rails app and Chrome Browser - google-chrome

I expect the app should respond with code 304 instead of 200.
But even IF-NONE-MATCH equals ETAG it doesn't happen.
I use 'Cache-Control: no-cache' to not store a response in cache to be validated each time. Otherwise Chrome uses it's disc cache, which is unacceptable.
Request:
GET /api/v4/record/11728 HTTP/1.1
Host: host.domain.com
Connection: keep-alive
Authorization: Basic YWRtaW467Uc2Zs0eTIwMTM=
Origin: https://host.domain.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36
Accept: application/json, text/plain, */*
DNT: 1
Referer: https://host-ui.domain.com/some_page
Accept-Encoding: gzip, deflate, sdch, br
If-None-Match: W/"39dcd8467e47701a69c617333f7b6dac"
If-Modified-Since: Thu, 13 Apr 2017 16:09:25 GMT
Name
Response:
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Methods: GET,POST,PUT,DELETE,PATCH,HEAD,OPTIONS
Access-Control-Allow-Origin: https://host-ui.domain.com
Cache-Control: no-cache
Content-Encoding: gzip
Content-Type: application/json; charset=utf-8
Date: Thu, 13 Apr 2017 16:20:31 GMT
ETag: W/"39dcd8467e47701a69c617333f7b6dac"
Last-Modified: Thu, 13 Apr 2017 16:09:25 GMT
Server: nginx/1.8.1 + Phusion Passenger 4.0.60
Status: 200 OK
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Powered-By: Phusion Passenger 4.0.60
X-Request-Id: ab87433e-62bd-437f-ad7c-0e1d3f95257b
X-Runtime: 0.209121
X-XSS-Protection: 1; mode=block
transfer-encoding: chunked
Connection: keep-alive
In the application common action looks like:
def action
record = Model.find(params['id'])
if stale?(record)
hard_work_result = to_do_somethig
render json: {
success: 0,
result: hard_work_result
}
end
end

There seems to be a history behind this but short story: gzip is messing with Weak ETags
Weak ETag: ETag: W/"8763458...
Strong ETag: ETag: "8763458...
You can test that this is the problem if you use cURL and disable gzip from Accept-Encoding (or use modheaders in chrome: Disable gzip compression in chrome )
References:
Getting no 304 response in Chrome/Safari but via curl
https://masa331.github.io/2016/01/06/roda-etag-caching-gotcha.html
Weak ETAGs in Rails?
Solution?
When running from nginx itself, the problem did not occur. The first link above suggests that if you are having the problem from within nginx then adding an etag on; after the gzip on; fixes the problem. Specifically though weak etags were not returned. Running from within nginx would seem to be the only option if you want gzip enabled.
my versions
Server: nginx/1.10.2 + Phusion Passenger 5.1.2`)
ii ruby-rails 2:4.2.6-1
ii ruby2.3 2.3.1-2~16.04
Ive managed to work around the problem when running in passenger standalone by editing the nginx.conf.erb and disabling gzip. To do this, you first need to get the template of the conf file:
passenger start --debug-nginx-config
this will put a file called nginx.conf.erb in your current directory. Then you can edit this file to say:
gzip off;
and then start passenger again with this file
passenger start --nginx-config-template nginx.conf.erb
see here for details of downloading and using the nginx.conf.erb
In either case you should be able to see that weak etags are not returned

Related

HTTP 500 internal server error when getting Comments from Box API v2

When I try to make a request to get the comments of a specific file which contains a large number of comments (around 70000), I get a 500 Internal Server Error.
Here is the request that I am sending and the response that is coming back (I have replaced the values of the fileId and token in the request):
GET /2.0/files/[my_file_id]/comments?fields=id,message,tagged_message,created_at,item,created_by HTTP/1.1
Host: api.box.com
Authorization: Bearer [my_token]
Accept: application/json
Accept-Encoding: gzip, deflate
User-Agent: CData Data Provider Engine - www.cdata.com - Accepts: gzip
Connection: close
Response:
HTTP/1.1 500 Internal Server Error
Server: ATS
Date: Mon, 06 Feb 2017 15:19:30 GMT
Content-Type: text/html; charset=UTF-8
Content-Encoding: gzip
Vary: Accept-Encoding
Content-Length: 20
Age: 82
Connection: close
Is there any reason why I am getting this kind of error?
This sounds like it might be a bug with the API. Try posting on the Box Developer forums or submitting a bug.

Chrome not caching preflight

I'm implementing a REST API that should support cross domain requests. Using CORS I want to achieve this. Almost all of my requests are 'not-simple', meaning for all non-GET requests a preflight request must be send by the browser.
To limit the amount of preflight/OPTIONS requests I try to let the browser cache the OPTIONS requests. This seems to work in Firefox and Safari, but not in Chrome. I know Chrome will only cache the preflight requests for only 10 minutes, but in my case it seems no caching takes place at all.
These are the HTTP requests and responses sent/received by Chrome:
Request:
OPTIONS /api/v1/sessions HTTP/1.1
Host: xxxxxxx
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Access-Control-Request-Method: POST
Origin: http://localhost:8000
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.107 Safari/537.36
Access-Control-Request-Headers: content-type
Accept: */*
Referer: http://localhost:8000/
Accept-Encoding: gzip, deflate, sdch
Accept-Language: nl-NL,nl;q=0.8,en-US;q=0.6,en;q=0.4
Response:
HTTP/1.1 200 OK
Date: Sun, 26 Jul 2015 09:33:27 GMT
Server: Apache/2.4.7 (Ubuntu)
X-Powered-By: PHP/5.5.9-1ubuntu4.9
Cache-Control: private, max-age=1440, pre-check=1440
Access-Control-Allow-Origin: http://localhost:8000
Access-Control-Allow-Methods: GET,POST,PATCH,DELETE
Access-Control-Max-Age: 86400
Access-Control-Allow-Headers: content-type
Content-Length: 0
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=utf-8
You have Pragma: no-cache & Cache-Control: no-cache headers set in the request.
Try removing them.
Api requests by default do not set these headers, and I doubt chrome does
either.
You should check your code and find out where they are
set from.
Now, given that its working fine on other browsers, you'd better check if you have set no-cache option on Dev Tools.
Preflight caching is a known bug in 98 version.
Follow below ticket for more details
https://bugs.chromium.org/p/chromium/issues/detail?id=1298477

Browser is not caching images in HTTPS (HTTP works fine). Even with Cache-Control: public

I'm trying to follow Google's caching recommendation, but neither IE nor Chrome are caching my images when HTTPS is used. The second request is not even a conditional GET. If I simply switch to HTTP, it works fine.
Here's request information, according to Chrome's request logger:
Remote Address: ::1:443
Request URL: https://localhost/getmyimage.php?id=123
Request Method: GET
Status Code: 200 OK
Request Headers
Accept: image/webp,*/*;q=0.8
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en;q=0.8
Connection: keep-alive
Cookie: PHPSESSID=gbk4vk7ejlr20nqgajcqgskul7
Host: localhost
Referer: https://localhost/
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36
Query String Parameters
id: 123
Response Headers
Cache-Control: public
Connection: Keep-Alive
Content-Length: 3224
Content-Type: image/png
Date: Tue, 27 May 2014 06:53:03 GMT
Expires: Mon, 25 Aug 2014 06:53:03 GMT
Keep-Alive: timeout=5, max=99
Last-Modified: Mon, 24 Feb 2014 02:17:21 GMT
Server: Apache/2.4.7 (Win32) OpenSSL/1.0.1e PHP/5.5.9
X-Powered-By: PHP/5.5.9
i think this is happening because of the url format, you can use apache's mod_rewrite to make a url format for images given by this script to look like localhost/image/123.png
EDIT
after reading your comment, i can say that it's not about your server's config, you can't do anything about it according to this and this because of HTTPS implementations since you have Cache-control: public set already.

Chrome does not make additional request for seeking video file

I try to achieve pseudo streaming, I have html like so:
<video src="GetVideo.ashx?id=mp4" controls></video>
after loading page Chrome 28.0.1500.72 m sends request (even before clicking play):
GET /GetVideo.ashx?id=mp4 HTTP/1.1
Host: localhost
Connection: keep-alive
Accept-Encoding: identity;q=1, *;q=0
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.72 Safari/537.36
Accept: */*
Referer: http://localhost/JWPlayerTestMp4Proper.aspx
Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.6,en;q=0.4
Cookie: jwplayer.volume=12
Range: bytes=0-
And server responds with
HTTP/1.1 206 Partial Content
Cache-Control: private
Content-Length: 5186931
Content-Type: video/mp4
Content-Range: bytes 0-5186930/5186931
Accept-Ranges: bytes
Server: Microsoft-IIS/8.0
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Mon, 22 Jul 2013 08:13:28 GMT
File starts to play after clicking play, but the problem is if i try to seek to yet not downloaded part, it does not send additional request for that part, it simply waits until file is downloaded upon specified position.
When I do the same in Firefox 22.0:
first request (after page loading):
GET http://localhost/GetVideo.ashx?id=mp4 HTTP/1.1
Host: localhost
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0
Accept: video/webm,video/ogg,video/*;q=0.9,application/ogg;q=0.7,audio/*;q=0.6,*/*;q=0.5
Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3
Range: bytes=0-
Referer: http://localhost/JWPlayerTestMp4Proper.aspx
Connection: keep-alive
first response (it is the same as for Chrome):
HTTP/1.1 206 Partial Content
Cache-Control: private
Content-Length: 5186931
Content-Type: video/mp4
Content-Range: bytes 0-5186930/5186931
Accept-Ranges: bytes
Server: Microsoft-IIS/8.0
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Mon, 22 Jul 2013 08:28:19 GMT
second request (after seeking to yet not downloaded part):
GET http://localhost/GetVideo.ashx?id=mp4 HTTP/1.1
Host: localhost
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0
Accept: video/webm,video/ogg,video/*;q=0.9,application/ogg;q=0.7,audio/*;q=0.6,*/*;q=0.5
Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3
Range: bytes=2490368-
Referer: http://localhost/JWPlayerTestMp4Proper.aspx
Connection: keep-alive
second response:
HTTP/1.1 206 Partial Content
Cache-Control: private
Content-Length: 2696563
Content-Type: video/mp4
Content-Range: bytes 2490368-5186930/5186931
Accept-Ranges: bytes
Server: Microsoft-IIS/8.0
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Mon, 22 Jul 2013 08:35:34 GMT
IE 10 is working in the same way as Firefox.
What response header Chrome expects to be able to behave in the same way - to make addtional requests after seeking to not-downloaded part?
It turned out response header was correct.
The problem was I was using video file with small length and chrome appears to have some kind of optimization not sending additional request if difference in time is too small (less then 30 seconds or so).

Is there a way to make a GET request without getting the body of the content?

I'm trying to do HEAD requests to follow 302 links, however this link: http://news.google.com/news/url?sa=t&fd=R&usg=AFQjCNGrJk-F7Dmshmtze2yhifxRsv8sRg&url=http://www.mtv.com/news/articles/1647243/20100907/story.jhtml
is troublesome because a HEAD request returns a 200 OK and a GET request returns the expected 302 Status code.
So I'll need to do a GET request but I'd rather not have to pay for the extra bandwidth times that will come from getting the entire HTML document. Anyone know a hack to do a GET without getting the body returned?
UPDATE: took David's advice to do a Range header but they seem to still be ignoring it
GET /news/url?sa=t&fd=R&usg=AFQjCNGrJk-F7Dmshmtze2yhifxRsv8sRg&url=http://www.mtv.com/news/articles/1647243/20100907/story.jhtml HTTP/1.1
Range: bytes=0-10
x-ms-range: 0-600
Host: news.google.com
Connection: Keep-Alive
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
HTTP/1.1 302 Moved Temporarily
Content-Type: text/html; charset=UTF-8
Location: http://www.mtv.com/news/articles/1647243/20100907/story.jhtml
Content-Length: 258
Date: Wed, 08 Sep 2010 20:28:16 GMT
Expires: Wed, 08 Sep 2010 20:28:16 GMT
Cache-Control: private, max-age=0
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Server: GSE
Set-Cookie: PREF=ID=ef5f1bc768645c5e:TM=1283977696:LM=1283977696:S=5n26IrEDpcQTJIb1; expires=Fri, 07-Sep-2012 20:28:16 GMT; path=/; domain=.google.com
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>302 Moved</TITLE></HEAD><BODY>
<H1>302 Moved</H1>
The document has moved
here.
</BODY></HTML>
File a bug with the web server's owner.
Try using the Range header in your request.
If that doesn't work, can you just hang up the connection after you get the headers you want?
In the specific example you cite, you could just pull it out of the original URL's "url" parameter. But for a more generic approach, I'd stick to David M.'s suggestions