How does the Chrome browser decide when to send OPTIONS? - google-chrome

I have an AngularJS WebAPI application.
As far as I can understand the OPTIONS request is constructed automatically by the browser.
POST http://localhost:3048/Token HTTP/1.1
Host: localhost:3048
Connection: keep-alive
Content-Length: 78
Accept: application/json, text/plain, */*
Origin: http://localhost:2757
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36
Content-Type: application/x-www-form-urlencoded
Referer: http://localhost:2757/Auth/login
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.8
grant_type=password&username=xxx%40live.com&password=xxx
Response:
HTTP/1.1 200 OK
Cache-Control: no-cache
Pragma: no-cache
Content-Length: 971
Content-Type: application/json;charset=UTF-8
Expires: -1
Server: Microsoft-IIS/8.0
Access-Control-Allow-Origin: *
Set-Cookie: .AspNet.Cookies=CpvxrR1gPFNs0vP8GAmcUt0EiKuEzLS1stLl-70O93wsipJkLUZuNdwC8tZc5M0o1ifoCjvnRXKjEBk3nLRbFlbldJLydW2BWonr5JmBjRjXZyKtcc29ggAVhZlc2E-3gGDlyoZLAa5Et8zrAokl8vsSoXmHnsjrxZw0VecB_Ry98Ln84UuKdeHlwSBnfaKKJfsN-u3Rsm6MoEfBO5aAFEekhVBWytrYDx5ks-iVok3TjJgaPc5ex53kp7qrtH3izbjT7HtnrsYYtcfPtmsxbCXBkX4ssCBthIl-NsN2wObyoEqHMpFEf1E9sB86PJhTCySEJoeUJ5u3juTnPlQnHsk1UTcO0tDb39g-_BD-I4FWS5GMwxLNtmut3Ynjir0GndwqsvpEsLls1Y4Pq7UuVCTn7DMO4seb64Sy8oEYkKZYk9tU4tsJuGD2CAIhdSc-lAmTAA78J5NOx23klkiuSe_SSiiZo5uRpas_1CFHjhi1c8ItEMpgeTsvgTkxafq5EOIWKPRxEHbCE8Dv106k5GlKK5BaH6z7ESg5BHPBvY8; path=/; HttpOnly
X-SourceFiles: =?UTF-8?B?QzpcR1xhYmlsaXRlc3Qtc2VydmVyXFdlYlJvbGVcVG9rZW4=?=
X-Powered-By: ASP.NET
Date: Tue, 13 Jan 2015 04:54:55 GMT
{"access_token":"TkJ2trqT ....
Now logged in
I log out which is nothing more than removing the token and log in again. Something happens that is different. Before it did not send the OPTIONS but now it does. Is there something resulting from a previous request/response that would influence the browser to act different the second time I log in?
OPTIONS http://localhost:3048/Token HTTP/1.1
Host: localhost:3048
Connection: keep-alive
Access-Control-Request-Method: POST
Origin: http://localhost:2757
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36
Access-Control-Request-Headers: accept, authorization, content-type
Accept: */*
Referer: http://localhost:2757/Auth/login
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8
Response:
HTTP/1.1 400 Bad Request
Cache-Control: no-cache
Pragma: no-cache
Content-Length: 34
Content-Type: application/json;charset=UTF-8
Expires: -1
Server: Microsoft-IIS/8.0
X-SourceFiles: =?UTF-8?B?QzpcR1xhYmlsaXRlc3Qtc2VydmVyXFdlYlJvbGVcVG9rZW4=?=
X-Powered-By: ASP.NET
Date: Tue, 13 Jan 2015 04:56:32 GMT
{"error":"unsupported_grant_type"}
If I do a browser reset and reload of the page then it goes back to like before where it does not send OPTIONS the first time and I am able to log in.
Probably I need to change something on the server so it handles options.
BUT why does my browser (Chrome) not send OPTIONS the first time?

Whether the Chrome (or any other browser) sends an OPTIONS request is exactly specified by the CORS specfication:
When the cross-origin request algorithm is invoked, these steps must be followed:
...
2. If the following conditions are true, follow the simple cross-origin request algorithm:
The request method is a simple method and the force preflight flag is unset.
Each of the author request headers is a simple header or author request headers is empty.
3. Otherwise, follow the cross-origin request with preflight algorithm.
Note: Cross-origin requests using a method that is simple with author request headers that are not simple will have a preflight request to ensure that the resource can handle those headers. (Similarly to requests using a method that is not a simple method.)
Your OPTIONS request contains the following request header:
Access-Control-Request-Headers: accept, authorization, content-type
This means that your Angular app has inserted the non-simple Authorization request header, probably as a part of an authentication scheme. Non-simple "author request headers" trigger the OPTIONS request, as you can see in the above quote.
To allow the request to succeed, your server should handle OPTIONS request and respond with:
Access-Control-Allow-Origin: https://example.com
Access-Control-Allow-Headers: authorization
To learn more about CORS, see https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS.

When you first login you most likely set the Authorization HTTP header somewhere in your login procedure. On the other side, you forgot to remove this header when the user logs out.
When you try to login again, the Authorization HTTP header is still present. This triggers the browser to perform a preflight request (see explanation of Rob W: https://stackoverflow.com/a/27924344/548020. Considering that you try to login with a grant type password it does not make sense to send an Authorization header, as this implies that you are already authorized (= logged in). Your are basically asking your backend to log you in and at the same time telling your backend that you are already authorized (= logged in).
This can be fixed by simple removing the Authorization HTTP header when the user logs out.

You can also clean your Headers when you login, before sending your POST request:
delete $http.defaults.headers.common['Authorization'];

Related

Website refuses headless chrome connections

Im trying to implement simple scraper, however I encoutered some problem. Somehow website is refusing connections from headless chrome. This is first and the only request, there is no any javascript execution. Requests from normal chrome works well so it's definitly not a banned ip. What can be wrong here? How are they posibly detecting it?
I'm running normal headless chrome and then I replace user agent, that's all.
.\chrome.exe --headless --remote-debugging-port=9222
General:
Request URL: https://www.adidas.de/
Request Method: GET
Status Code: 403
Remote Address: 23.210.248.137:443
Referrer Policy: no-referrer-when-downgrade
Response Headers:
cache-control: max-age=0, no-cache, no-store
content-length: 1952
content-type: text/html
date: Thu, 26 Dec 2019 16:16:49 GMT
expires: Thu, 26 Dec 2019 16:16:49 GMT
pragma: no-cache
status: 403
Request Headers:
:authority: www.adidas.de
:method: GET
:path: /
:scheme: https
accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
accept-encoding: gzip, deflate, br
cache-control: max-age=0
sec-fetch-mode: navigate
sec-fetch-site: none
sec-fetch-user: ?1
upgrade-insecure-requests: 1
user-agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36
#RobertHarvey: Yes, you are correct: sending Accept-Language is a must for some websites. You can either do it with puppeteer via their API, or with chrome-remote-interface by intercepting requests and adding header directly.

Chrome not caching preflight

I'm implementing a REST API that should support cross domain requests. Using CORS I want to achieve this. Almost all of my requests are 'not-simple', meaning for all non-GET requests a preflight request must be send by the browser.
To limit the amount of preflight/OPTIONS requests I try to let the browser cache the OPTIONS requests. This seems to work in Firefox and Safari, but not in Chrome. I know Chrome will only cache the preflight requests for only 10 minutes, but in my case it seems no caching takes place at all.
These are the HTTP requests and responses sent/received by Chrome:
Request:
OPTIONS /api/v1/sessions HTTP/1.1
Host: xxxxxxx
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Access-Control-Request-Method: POST
Origin: http://localhost:8000
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.107 Safari/537.36
Access-Control-Request-Headers: content-type
Accept: */*
Referer: http://localhost:8000/
Accept-Encoding: gzip, deflate, sdch
Accept-Language: nl-NL,nl;q=0.8,en-US;q=0.6,en;q=0.4
Response:
HTTP/1.1 200 OK
Date: Sun, 26 Jul 2015 09:33:27 GMT
Server: Apache/2.4.7 (Ubuntu)
X-Powered-By: PHP/5.5.9-1ubuntu4.9
Cache-Control: private, max-age=1440, pre-check=1440
Access-Control-Allow-Origin: http://localhost:8000
Access-Control-Allow-Methods: GET,POST,PATCH,DELETE
Access-Control-Max-Age: 86400
Access-Control-Allow-Headers: content-type
Content-Length: 0
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=utf-8
You have Pragma: no-cache & Cache-Control: no-cache headers set in the request.
Try removing them.
Api requests by default do not set these headers, and I doubt chrome does
either.
You should check your code and find out where they are
set from.
Now, given that its working fine on other browsers, you'd better check if you have set no-cache option on Dev Tools.
Preflight caching is a known bug in 98 version.
Follow below ticket for more details
https://bugs.chromium.org/p/chromium/issues/detail?id=1298477

Why does Browser still sends request for cache-control public with max-age?

I have Amazon S3 objects, and for each object, I have set
Cache-Control: public, max-age=3600000
That is roughly 41 days.
And I have Amazon CloudFront Distribution set with Minimum TTL also with 3600000.
This is the first request after clearing cache.
GET /1.0.8/web-atoms.js HTTP/1.1
Host: d3bhjcyci8s9i2.cloudfront.net
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.57 Safari/537.36
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
And Response is
HTTP/1.1 200 OK
Content-Type: application/x-javascript
Content-Length: 226802
Connection: keep-alive
Date: Wed, 28 Aug 2013 10:37:38 GMT
Cache-Control: public, max-age=3600000
Last-Modified: Wed, 28 Aug 2013 10:36:42 GMT
ETag: "124752e0d85461a16e76fbdef2e84fb9"
Accept-Ranges: bytes
Server: AmazonS3
Age: 342557
Via: 1.0 6eb330235ca3971f6142a5f789cbc988.cloudfront.net (CloudFront)
X-Cache: Hit from cloudfront
X-Amz-Cf-Id: 92Q2uDA4KizhPk4TludKpwP6Q6uEaKRV0ls9P_TIr11c8GQpTuSfhw==
Even while Amazon clearly sends Cache-Control, Chrome still makes second request instead of reading it from Cache.
GET /1.0.8/web-atoms.js HTTP/1.1
Host: d3bhjcyci8s9i2.cloudfront.net
Connection: keep-alive
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.57 Safari/537.36
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
If-None-Match: "124752e0d85461a16e76fbdef2e84fb9"
If-Modified-Since: Wed, 28 Aug 2013 10:36:42 GMT
Question:
Why does chrome makes second request?
Expires
This behavior changes when I put an explicit Expires attribute in headers. Browser will not send subsequent request for Expires header, but for cache-control public, it does send it. My all S3 objects will never change, they are immutable, when we change file, we put them as new object with new URL.
In Page Script Reference
Chrome makes subsequent requests only sometimes, I did this test by actually typing URL in browser. When script is referenced by HTML page, for few subsequent requests chrome loads cached scripts, but once again after sometime, once in a while it does send request to server. There is no Disk Size issue here, Chrome has sufficient cache space.
Problem is we get charged for every request, and I want S3 objects to be cached forever, and should be loaded from Cache and should never connect to server back.
When you press F5 in Chrome, it will always send requests to the server. These will be made with the Cache-Control:max-age=0 header. The server will usually respond with a 304 (Not Changed) status code.
When you press Ctrl+F5 or Shift+F5, the same requests are performed, but with the Cache-Control:no-cache header, thus forcing the server to send an uncached version, usually with a 200 (OK) status code.
If you want to make sure that you're utilizing the local browser cache, simply press Enter in the address bar.
If the HTTP Response contains the etag entry, the conditional request will always be made. ETag is a cache validator tag. The client will always send the etag to the server to see if the element has been modified.
If Chrome Developer Tools are open (F12), Chrome usually disables caching.
It is controllable in the Developer Tools settings - the Gear icon to the right of the dev-tools top bar.
If you are hitting the refresh button for loading the particular page or resource, the if-modified-since header request is sent everytime, if you instead request the page/resource as a separate request in a new tab or via a link in a script or html page, it will load the page/resource from the browser cache itself.
This is what has happened in my case, may be this is the general universal case. I am not completely sure, but this is what I gathered via my digging.
Chrome adds Cache-control: max-age=0 header when you use self-signed certificate. Switching from HTTPS to HTTP will remove this header.
Firefox doesn't add this header.

How do i protect $.ajax datatype:json Post parameters

I am using the parameters for updating via Web service method.
Please see the post below.
Post: http://myweb.com:8241/web/Dashboard.aspx/BindDatatable
Host: localhost:8241
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0
Accept: application/json, text/javascript, */*; q=0.01
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Content-Type: application/json; charset=utf-8
X-Requested-With: XMLHttpRequest
Referer: http://myweb.com:8241/web/Dashboard.aspx
Content-Length: 49
Cookie: .ASPXANONYMOUS=ltv9ZiqjzgEkAAAANGZlZTY2ODItYmFkMy00OWI5LWIxMDktNGU5NTg4M2IyOTVj2fU3SjopgaEx5DOYla827v7hFQNzpmfoFvRqDv1859g1; ASP.NET_SessionId=1bro3mtv1gbcqswhlced251h
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Send Post Content?
{stat: "201541",search:"",points:"100",delete:""}
Is it possible to encrypt this post content? Or revise my code not to use json? I am just confuse about this, and i think i am about to build vulnerability here. Please help.
Encrypt? Sure, use HTTPS.
Protect? Against what? If you are concerned about someone posting whatever they want to your web service, or seeing how it works, there is nothing you can do to prevent that. All you can do is not trust data from the user.

Google Chrome Cancels 4xx Client Error Response

Using Google Chrome I'm opening Flex4.5 Client that makes GET Http Request to Rails back end that renders back json response.
If the response is 4xx Client Error then Chrome Developer Tools network tab shows the request as canceled, and I can't access the response error message through the Fault content in Flex.
This happens only on chrome. It works fine for FF and IE or if I execute the query in the chrome browser.
Below are the request and response headers copied from Chrome Developer Tools
Thank you for your help
GET (canceled) application/json Other
Request Header
GET url HTTP/1.1
Host: localhost:3000
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.83 Safari/535.11
Accept: */*
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Response Header
HTTP/1.1 404 Not Found
Content-Type: application/json; charset=utf-8
X-Ua-Compatible: IE=Edge
Cache-Control: no-cache
X-Runtime: 0.500000
Content-Length: 30
Server: WEBrick/1.3.1 (Ruby/1.9.2/2011-07-09)
Date: Wed, 28 Mar 2012 21:53:40 GMT
Connection: Keep-Alive