I am trying to implement a websockets server in java but I am having trouble understanding the version of websocket protocol does google chrome implements. In this blog post the Chromium team mentions that they have implemented version 76 of the websockets protocol which should have the following handshake request (copied from official version 76 spec (section 1.2)):
GET /demo HTTP/1.1
Host: example.com
Connection: Upgrade
Sec-WebSocket-Key2: 12998 5 Y3 1 .P00
Sec-WebSocket-Protocol: sample
Upgrade: WebSocket
Sec-WebSocket-Key1: 4 #1 46546xW%0l 1 5
Origin: http://example.com
^n:ds[4U
where as the handshake request I am getting from google chrome(8.0.552.237) is :
GET / HTTP/1.1
Upgrade: WebSocket
Connection: Upgrade
Host: localhost:8082
Origin: http://localhost:8081
Sec-WebSocket-Key1: g3 I 9w F : Kt 3?u784C-0 99 5z
Sec-WebSocket-Key2: 4 S5l&Y4m]11+4 032s Ecfl
ïÿÁëƒ^
Can somebody please explain me why is there a discrepancy between googles version and the official spec. And what would be the right format of the response for google's version.
PS: does anybody has any estimate by when the websockets protocol would be finalized ?
Chrome currently implements draft-hixie-thewebsocketprotocol-76 which is also known as draft-ietf-hybi-thewebsocketprotocol-03. The protocol and specs are now being published by the IETF HyBi working group thus the 03 numbering is actually more correct and the next versions of the spec will continue that numbering (but most people still call it v76).
The handshake you are getting from Chrome-8.0.552 does match the spec. The differences are because the ordering of the headers can vary and the keys will have different values for every new connection and the host, origin, path, and protocol values depend on how the Javascript invokes the connection and also depend on the URL of the invoking page.
The response will look something like this (although not exactly because the response is generated based on the client handshake):
HTTP/1.1 101 WebSocket Protocol Handshake
Upgrade: WebSocket
Connection: Upgrade
Sec-WebSocket-Origin: http://example.com
Sec-WebSocket-Location: ws://example.com/demo
Sec-WebSocket-Protocol: sample
8jKS'y:G*Co,Wxa-
I've been following the HyBi working group and there is no estimate for when the protocol will be finalized. However, a new draft (04) has already been published trying to address security concerns of the browser vendors. A (05) version will likely be published in the next few weeks. The browser vendors probably won't implement 04 (because there are still significant areas of low consensus). If most of the browser vendors adopt 05 and there are no additional security concerns I suspect that the final version of the protocol will not change much from there.
It's important to note however, that the browser API for WebSockets is likely not going to be affected by changes in the protocol.
Related
For my cryptocurrency market data website
I have enabled early hints on Cloudlare
I am generating early hints on SvelteKit/Vite web server hooks
I have verified the presence of Link headers in the HTTP reply
Link: </fonts3.css>; rel="preconnect", </_app/assets/start-6f5e0715.css>; rel="preconnect", </_app/assets/pages/__layout.svelte-f31b19cc.css>; rel="preconnect", </_app/assets/pages/index.svelte-84a34be8.css>; rel="preconnect"
I have enabled Chrome command line flag to respect early hints
Now, how do I verify that
Cloudflare is correctly generating HTTP/1.1 103 Early Hints. Can I display this information with curl or wget?
Chrome is respecting the early hint - where this is shown in the Performance tab of webr developer tools, or somewhere else?
I have no luck to verify Chrome's response from Chrome's devtool. When I monitor the network traffic, Chrome devtool does not show 103 responses at all. However, I am able to verify the server 103 responses using curl. Here is a curl output sample:
$ curl -k -v —http2 https://evanella.org
...
Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
Using Stream ID: 1 (easy handle 0x7fa7a2009200)
GET / HTTP/2
Host: evanella.org
User-Agent: curl/7.64.1
Accept: /
...
Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 103
< link: https://evanella.org/; rel=preload;
< link: https://evanella.org/wp-content/themes/twentytwentytwo/assets/fonts/SourceSerif4Variable-Roman.ttf.woff2; rel=preload; as=font
< link: https://evanella.org/wp-content/themes/twentytwentytwo/assets/images/flight-path-on-transparent-d.png; rel=preload; as=image
< link: https://evanella.org/wp-content/themes/twentytwentytwo/style.css?ver=1.1; rel=preload; as=style
< link: https://evanella.org/wp-includes/blocks/navigation/view.min.js?ver=3776ea67846b3bb10fe8f7cdd486b0ba; rel=preload; as=script
...
< HTTP/2 200
< server: h2o/2.3.0-DEV#998a7fb7b
< content-type: text/html; charset=UTF-8
< link: https://evanella.org/wp-json/; rel="https://api.w.org/"
< vary: Accept-Encoding
BTW, I configured my own webserver to generate the 103 early hints responses.
This site is an online tool for testing whether Early Hints are being provided:
https://code103.hotmann.de
In chrome you'd enter this in console:
performance.getEntriesByName('https://yourwebsite.com/image.jpg')[0].initiatorType
which will output link or early-hints respectively (maybe some more values, etc.), which, in case of early-hints indicates that certain asset download was initiated by 103 header.
In my case i'm stuck on receiving link where early-hints are expected. Will update this response if I learn something more during the debug process.
After updating to Chrome 40.0.2214.111, variably when I visit certain Google related sites (like http://youtube.com and get presented with an ad before the video), the browser downloads a file named f.txt.
I do not have any adblock plugins installed.
f.txt contains a few lines of JavaScript...starting with:
if (!window.mraid) {document.write('\x3cdiv class="GoogleActiveViewClass" ' +'id="DfaVisibilityIdentifier_3851468350"\x3e');}document.write('\x3ca target\x3d\x22_blank\x22 href\x3d\x22https://adclick.g.doubleclick.net/pcs/click?xai\x3dAKAOjsvDhmmoi2r124JkMyiBGALWfUlTX-zFA1gEdFeZDgdS3JKiEDPl3iIYGtj9Tv2yTJtASqD6S-yqbuNQH5u6fXm4rThyCZ0plv9SXM-UPKJgH4KSS08c97Eim4i45ewgN9OoG3E_
In looking up the issue on Google, others have experienced the same, but I have not found any resolution or understanding of why this is happening. I assume it is a content-disposition related bug with some of the JS files loaded on the page, and will clear up in a future patch.
Wondering if anybody else had experienced / insight.
This issue appears to be causing ongoing consternation, so I will attempt to give a clearer answer than the previously posted answers, which only contain partial hints as to what's happening.
Some time around the summer of 2014, IT Security Engineer Michele Spagnuolo (apparently employed at Google Zurich) developed a proof-of-concept exploit and supporting tool called Rosetta Flash that demonstrated a way for hackers to run malicious Flash SWF files from a remote domain in a manner which tricks browsers into thinking it came from the same domain the user was currently browsing. This allows bypassing of the "same-origin policy" and can permit hackers a variety of exploits. You can read the details here: https://miki.it/blog/2014/7/8/abusing-jsonp-with-rosetta-flash/
Known affected browsers: Chrome, IE
Possibly unaffected browsers: Firefox
Adobe has released at least 5 different fixes over the past year while trying to comprehensively fix this vulnerability, but various major websites also introduced their own fixes earlier on in order to prevent mass vulnerability to their userbases. Among the sites to do so: Google, Youtube, Facebook, Github, and others. One component of the ad-hoc mitigation implemented by these website owners was to force the HTTP Header Content-Disposition: attachment; filename=f.txt on the returns from JSONP endpoints. This has the annoyance of causing the browser to automatically download a file called f.txt that you didn't request—but it is far better than your browser automatically running a possibly malicious Flash file.
In conclusion, the websites you were visiting when this file spontaneously downloaded are not bad or malicious, but some domain serving content on their pages (usually ads) had content with this exploit inside it. Note that this issue will be random and intermittent in nature because even visiting the same pages consecutively will often produce different ad content. For example, the advertisement domain ad.doubleclick.net probably serves out hundreds of thousands of different ads and only a small percentage likely contain malicious content. This is why various users online are confused thinking they fixed the issue or somehow affected it by uninstalling this program or running that scan, when in fact it is all unrelated. The f.txt download just means you were protected from a recent potential attack with this exploit and you should have no reason to believe you were compromised in any way.
The only way I'm aware that you could stop this f.txt file from being downloaded again in the future would be to block the most common domains that appear to be serving this exploit. I've put a short list below of some of the ones implicated in various posts. If you wanted to block these domains from touching your computer, you could add them to your firewall or alternatively you could use the HOSTS file technique described in the second section of this link: http://www.chromefans.org/chrome-tutorial/how-to-block-a-website-in-google-chrome.htm
Short list of domains you could block (by no means a comprehensive list). Most of these are highly associated with adware and malware:
ad.doubleclick.net
adclick.g.doubleclick.net
secure-us.imrworldwide.com
d.turn.com
ad.turn.com
secure.insightexpressai.com
core.insightexpressai.com
I experienced the same issue, same version of Chrome though it's unrelated to the issue. With the developer console I captured an instance of the request that spawned this, and it is an API call served by ad.doubleclick.net. Specifically, this resource returns a response with Content-Disposition: attachment; filename="f.txt".
The URL I happened to capture was https://ad.doubleclick.net/adj/N7412.226578.VEVO/B8463950.115078190;sz=300x60...
Per curl:
$ curl -I 'https://ad.doubleclick.net/adj/N7412.226578.VEVO/B8463950.115078190;sz=300x60;click=https://2975c.v.fwmrm.net/ad/l/1?s=b035&n=10613%3B40185%3B375600%3B383270&t=1424475157058697012&f=&r=40185&adid=9201685&reid=3674011&arid=0&auid=&cn=defaultClick&et=c&_cc=&tpos=&sr=0&cr=;ord=435266097?'
HTTP/1.1 200 OK
P3P: policyref="https://googleads.g.doubleclick.net/pagead/gcn_p3p_.xml", CP="CURa ADMa DEVa TAIo PSAo PSDo OUR IND UNI PUR INT DEM STA PRE COM NAV OTC NOI DSP COR"
Date: Fri, 20 Feb 2015 23:35:38 GMT
Pragma: no-cache
Expires: Fri, 01 Jan 1990 00:00:00 GMT
Cache-Control: no-cache, must-revalidate
Content-Type: text/javascript; charset=ISO-8859-1
X-Content-Type-Options: nosniff
Content-Disposition: attachment; filename="f.txt"
Server: cafe
X-XSS-Protection: 1; mode=block
Set-Cookie: test_cookie=CheckForPermission; expires=Fri, 20-Feb-2015 23:50:38 GMT; path=/; domain=.doubleclick.net
Alternate-Protocol: 443:quic,p=0.08
Transfer-Encoding: chunked
Accept-Ranges: none
Vary: Accept-Encoding
FYI, after reading this thread, I took a look at my installed programs and found that somehow, shortly after upgrading to Windows 10 (possibly/probably? unrelated), an ASK search app was installed as well as a Chrome extension (Windows was kind enough to remind to check that). Since removing, I have not have the f.txt issue.
This can occur on android too not just computers. Was browsing using Kiwi when the site I was on began to endlessly redirect so I cut net access to close it out and noticed my phone had DL'd something f.txt in my downloaded files.
Deleted it and didn't open.
Seems related to https://groups.google.com/forum/#!msg/google-caja-discuss/ite6K5c8mqs/Ayqw72XJ9G8J.
The so-called "Rosetta Flash" vulnerability is that allowing arbitrary
yet identifier-like text at the beginning of a JSONP response is
sufficient for it to be interpreted as a Flash file executing in that
origin. See for more information:
http://miki.it/blog/2014/7/8/abusing-jsonp-with-rosetta-flash/
JSONP responses from the proxy servlet now:
* are prefixed with "/**/", which still allows them to execute as JSONP
but removes requester control over the first bytes of the response.
* have the response header Content-Disposition: attachment.
I have this problem with compression, and I am not sure if it is a bug. My WebSocket server does not support context takeover, and I am having problems sending messages, but not receiving.
The browser issues a request like this:
GET /socket HTTP/1.1
Host: thirdparty.com
Origin: http://example.com
Connection: Upgrade
Upgrade: websocket
Sec-WebSocket-Version: 13
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits, x-webkit-deflate-frame
If the server does not specify any option about context takeover:
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Access-Control-Allow-Origin: http://example.com
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
Sec-WebSocket-Extensions: permessage-deflate
I can read and write the first message, but cannot do subsequent reads or writes, because Chrome expects the server is keeping the context.
So my server provides this answer:
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Access-Control-Allow-Origin: http://example.com
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
Sec-WebSocket-Extensions: permessage-deflate; client_no_context_takeover; server_no_context_takeover
And now I can receive messages without problems, but again, I can only send the first message, the second message fails, and I see an error in Chrome saying that it failed at inflating the frame. I tried to send two identical strings, and I can see how the server is sending twice the same data, but the client fails to decompress it the second time.
So it seems that Chrome accepts the client_no_context_takeover parameter that specify that the client won't use the same compression context for all messages when compressing, but ignores server_no_context_takeover indicates the server won't use the same context.
Is this a bug in Chrome? I am not clear about if I can send options back that have not been offered/requested by the client.
Is there any other option I can use to disable the client context takeover?
UPDATE:
In WebSocketPerMessageDeflate.cpp in the Chromium source code, I can see:
if (clientNoContextTakeover != parameters.end()) {
if (!clientNoContextTakeover->value.isNull()) {
m_failureReason = "Received invalid client_no_context_takeover parameter";
return false;
}
mode = WebSocketDeflater::DoNotTakeOverContext;
++numProcessedParameters;
}
But also:
if (serverNoContextTakeover != parameters.end()) {
if (!serverNoContextTakeover->value.isNull()) {
m_failureReason = "Received invalid server_no_context_takeover parameter";
return false;
}
++numProcessedParameters;
}
In the first snippet, it is setting the "mode" variable, but in the second one is not doing nothing, so it seems it is basically ignoring the parameter.
Cheers.
A server must send a server_no_context_takeover parameter in a response only if the client requested the "no context takeover". In essence, the server acknowledges the client's request.
If a server decides to do "no context takeover" for sending on it's own (without the client having requested it), that's fine. In this case, no parameter is sent by the server.
A deflate sender can always on it's own drop compression context and/or reduce compression window size. There is no need to tell the receiver. The deflate wire format has enough information for the receiver to cope with that.
Here is how configuration and handshake looks with Crossbar.io.
I finally found the problem.
https://datatracker.ietf.org/doc/html/draft-ietf-hybi-permessage-compression-17#section-8.2.3
8.2.3.4. Using a DEFLATE Block with BFINAL Set to 1
Going through the examples in the draft I found my server was sending slightly different payloads. Turned out that the problem was the BFINAL, I need to set it to 0 by adding a 0 byte at the end.
Now it works.
I came across this code change in Chromium. It says Chromium now supports both handshake versions, which the code seems to confirm. I mean the second version at Wikipedia (draft-ietf-hybi-thewebsocketprotocol-06).
However, when I connect to my server, the only thing I obtain is the old version, i.e. including these headers:
Sec-WebSocket-Key1: 4 #1 46546xW%0l 1 5
Sec-WebSocket-Key2: 12998 5 Y3 1 .P00
but not the new version which would be a request containing:
Sec-WebSocket-Key: x3JJHMbDL1EzLkh9GBhXDw==
What am I missing here? I downloaded the latest nightly build and it has been included more than two weeks ago, so that cannot be the cause I guess.
How can I make a WebSocket send the new handshake version?
The code link you posted is for the server-side of the handshake (there is a few places this will likely be used in Chrome such as remote debugging and as a proxy for extensions).
If you really want use the new HyBi-07 protocol version you can try using this branch of web-socket-js that I made. Once Chrome switch to the new protocol, web-socket-js will switch by default also. In order to make web-socket-js work in a browser that already has WebSockets support you will need make some minor tweaks to it to use a different object name instead of WebSocket.
I expect Chrome/WebKit will add the new protocol before long. Note that the API changes to add binary support have only recent been decided so Chrome the new protocol may be added before the API fully supports the new functionality enabled by the protocol.
The only browser I know of that implements the 07 protocol is this build of FF4:
http://www.ducksong.com/misc/websockets-builds/ws-07/
Has anyone read Hickson's May 2010 draft-hixie-thewebsocketprotocol-76 WebSocket protocol?
Here is the the source of an .htm file:
<html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<script type="text/javascript">
var socket = new WebSocket('ws://localhost:8181/websession');
socket.onopen = function() {
alert('handshake successfully established. May send data now...');
};
socket.onclose = function() {
alert('connection closed');
};
</script>
</head>
<body>
</body>
</html>
If I have a TCP port listening on 8181, this is the request I get when I load the .htm file above in Chrome:
GET /websession HTTP/1.1
Upgrade: WebSocket
Connection: Upgrade
Host: localhost:8181
Origin: null
[\n]
(Where [\n] is CRLF character.)
What should I return to this handshake opener? draft-hixie-thewebsocketprotocol-76 shows:
HTTP/1.1 101 WebSocket Protocol Handshake
Upgrade: WebSocket
Connection: Upgrade
Sec-WebSocket-Origin: http://example.com
Sec-WebSocket-Location: ws://example.com/demo
Sec-WebSocket-Protocol: sample
8jKS'y:G*Co,Wxa-
This response causes socket.onclose to fire though.
Draft 76 renamed the WebSocket- response headers to Sec-WebSocket-, and added some unnecessarily ugly Key header and request body crypto stuff to which the 8jKS'y:G*Co,Wxa- is a response. But that's only the correct response for the example included in the draft; it's no good returning that specific string for any other request. See this post for an explanation of how to implement the new protocol.
In any case, unless you are using the latest development builds, Chrome/Chromium will still be using the old draft 75 protocol (as the request you posted demonstrates), and won't talk to the a server that implements the new protocol. See the Chromium blog for more info. If you need to support old/current Chrome versions you effectively have to implement two WebSocket protocols.
This is always the risk in developing stuff against a protocol that is not yet standardised. You can expect annoying interinoperability until WebSocket is finalised; you may prefer to hold off until then.
(Trying to actually read the spec and work out what exactly has changed amongst the reams of unreadable parsing algorithms, is an exercise in frustration. I have no idea why it is written like this instead of the usual BNF-style specifications RFCs like. It's as if Hixie wrote a parser in C and then wrote an automated tool to turn the code into English. C would have been more readable TBH.)
You might find the wsproxy included in noVNC to be useful as a reference. It transparently supports both WebSockets v75 and v76 clients.
wsproxy is a generic WebSockets to TCP socket proxy. There is both a C and python version of wsproxy included with noVNC.
http://github.com/kanaka/noVNC/tree/master/utils/
Also, just to keep things interesting, the latest (no version yet) draft proposal changes things up again (in particular it changes how things are framed): http://www.whatwg.org/specs/web-socket-protocol/