RFC 5861 proposes the Cache-Control extensions stale-while-revalidate:
When present in an HTTP response, the stale-while-revalidate Cache-
Control extension indicates that caches MAY serve the response in
which it appears after it becomes stale, up to the indicated number
of seconds.
and stale-if-error:
The stale-if-error Cache-Control extension indicates that when an
error is encountered, a cached stale response MAY be used to satisfy
the request, regardless of other freshness information.
to allow stale content to be used for longer when a response cannot be revalidated.
These are supported by CDNs, including Fastly and KeyCDN.
stale-while-revalidate is also mentioned in the Fetch standard (stale-if-error is not).
What is the state of support in browsers?
As of May 2019, according to the compatibility tables of Mozilla and Chrome, stale-while-revalidate is:
supported in the beta releases of Firefox 68 (tracking)
due to be supported in Chrome 75 (tracking)
These will be the main releases of those browsers on 2019-07-09 and 2019-06-09 respectively.
stale-if-error doesn't appear to have current, or planned, support.
(See the RFC author's blog Chrome and Stale-While-Revalidate for more detail on why to implement this in browsers.)
Related
The new chrome versions 72+ does not send the requestHeaders .
there was a solution:
DevTools Protocol network inspection is located quite high in the network stack. This architecture doesn't let us collect all the headers that are added to the requests. So the ones we report in Network.requestWillBeSent and Network.requestIntercepted are not complete; this will stay like this for the foreseeable future.
There are a few ways to get real request headers:
• the crude one is to use proxy
• the more elegant one is to rely on Network.responseReceived DevTools protocol event. The actual headers are reported there as requestHeaders field in the Network.Response.
This worked fine with the old chromes but not with the last versions. here is a small summery I made for the versions a coulded test
a solution for chrome v67 was to add this flags to disable Site Isolation :
chrome --disable-site-isolation-trials --disable-features=IsolateOrigins,site-per-process --disable-web-security
Now all of this does not work with the last chrome v73
maybe it is caused by this:
Issue 932674: v72 broke devtools request interception inside cross-domain iframes
you can use Fetch protocol domain that is available since m74
the solution gaven does not work neither, the Fetch.requestPaused does not contain the request headers...
I found some info that maybe causes that:
DevTools: do not expose raw headers for cross-origin requests
DevTools: do not report raw headers and cookies for protected subresources. In case subresource request's site needs to have its document protected, don't send raw headers and cookies into the frame's renderer.
or it is caused when it is an HTTP/2 server?
Does the HTTP/2 header frame factor into a response’s encodedDataLength? (Remote Debugging Protocol)
...headersText is undefined for HTTP/2 requests
link
1- How can I get the Request Headers using the Chrome Devtool Protocol with chrome v73+?
2- Can a webextension solve that?
3- Is there another way which will be stable and last longuer? like tshark+sslkeylogfile which I'm attempting to avoid. thank you
I've been investigating the implementation of CTAP2 protocol with Google Chrome (protocol between Fido Authenticator and Fido Client (e.g. Chrome)), which should be supported in Chrome since version 69.
So I dumped the chrome request to the authenticator in two cases:
1. U2F (CTAP1) with yubico demo website.
Here everything goes as expected and chrome follows the CTAP1 protocol.
2. FIDO2/Webauthn (CTAP2) with webauthn.org
And the browser has a strange behaviour:
It starts by following the CTAP2 protocol by sending a AuthenticatorGetInfoRequest
The Authenticator replies with AuthenticatorGetInfoResponse
But then Chrome switches to the CTAP1 protocol by sending a Registration Request Message (U2F_REGISTER)
Is it a problem with Chrome implementation: CTAP2 is not enabled entirely yet (it seems to be implemented in Chromium sources)?
Or is the problem coming from the authenticator: sending a response that makes Chrome switch to CTAP1?
Or something else entirely?
In AuthenticatorGetInfoRequest, the authenticator must respond with a string list of the supported versions among "U2F_V2" and "FIDO_2_0". Note also in case of NFC or CCID, a "FIDO_2_0" (CTAP2) only authenticator shall anwser directly "FIDO_2_0" on the SELECT command. I don't know about HID and BLE.
So maybe your authenticator is listing "U2F_V2" in the protocol versions list in AuthenticatorGetInfoRequest.
I also imagine the Chrome code base about this was moving fast and can be partial or not strict. To my tests in early 2020, the Chrome CTAP capabilities seem now stable and compliant.
I'm trying to download a file using selenium by simulating click on a download button but Chrome reports ERR_BLOCKED_BY_XSS_AUDITOR. If I use the "--disable-xss-auditor" argument to bypass, the page would be reloaded and nothing get downloaded. What seems strange to me is that when I actually download the file with my mouse in a Chrome session that's even controlled by selenium, the file downloads well.
Please help me understand what xss auditor does? Why can't I download the file with selenium?
BTW, I'm using python if it matters.
Thanks
X-XSS-Protection
The HTTP X-XSS-Protection response header is a feature of Internet Explorer, Chrome and Safari that stops pages from loading when they detect reflected cross-site scripting (XSS) attacks. Although these protections are largely unnecessary in modern browsers when sites implement a strong Content-Security-Policy that disables the use of inline JavaScript ('unsafe-inline'), they can still provide protections for users of older web browsers that don't yet support CSP.
Header type Response header
----------- ---------------
Forbidden header name no
Syntax
X-XSS-Protection: 0: Disables XSS filtering.
X-XSS-Protection: 1: Enables XSS filtering (usually default in browsers). If a cross-site scripting attack is detected, the browser will sanitize the page (remove the unsafe parts).
X-XSS-Protection: 1: mode=block Enables XSS filtering. Rather than sanitizing the page, the browser will prevent rendering of the page if an attack is detected.
X-XSS-Protection: 1: report= (Chromium only) Enables XSS filtering. If a cross-site scripting attack is detected, the browser will sanitize the page and report the violation. This uses the functionality of the CSP report-uri directive to send a report.
Background
As per Intent to Ship: Changes to the XSS Auditor Chromium team made two changes:
Change the default behavior to X-XSS-Protection: 1; mode=block, which blocks the page load by navigating to a unique origin when XSS is detected, rather than filtering out specific scripts.
Deprecate the filter mode, with the intent to remove it completely at some future date.
Implementation Status
XSS Auditor blocks by default: Chrome's XSS Auditor should block pages by default, rather than filtering out suspected reflected XSS. Moreover, we should remove the filtering option, as breaking specific pieces of page's script has been an XSS vector itself in the past.
As per XSS Auditor: Block by default, remove filtering this issue was discussed and a fix was attempted. Some more discussion happened in False positives with ERR_BLOCKED_BY_XSS_AUDITOR and finally in ERR_BLOCKED_BY_XSS_AUDITOR on bona fide site when posting to a forum Chromium team decided Status: WontFix
Solution
You need to induce WebDriverWait for the desired element to be clickable. Here are some examples of the WebDriverWait implementation:
Java:
new WebDriverWait(driver, 20).until(ExpectedConditions.elementToBeClickable(By.linkText("text_within_the _link"))).click();
Python:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.LINK_TEXT, "text_within_the _link"))).click()
C#:
new WebDriverWait(driver, TimeSpan.FromSeconds(10)).Until(ExpectedConditions.ElementToBeClickable(By.LinkText("text_within_the _link"))).Click();
Reference
Event 1046 - Cross-Site Scripting Filter
The misunderstood X-XSS-Protection
I slowed down the clicks (2 clicks needed to download, added a sleep between them) and it works! Have no idea what happened...
XSS Auditor is a built-in function of Chrome and Safari which is designed to mitigate Cross-site Scripting (XSS) attacks. It aims to identify if query parameters contain malicious JavaScript and block the response if it believes the payloads were injected into the server response.
XSS is a vulnerability that occurs when the data get (mis)interpreted as code and executed on a victim's browser. The idea is to use a headless browser like Selenium WebDriver, and inject XSS payloads along with functional and user interaction tests
Python don't have anything to do with that, I think that might be the chrome version or something
i have shared the link which will help you understand better.
Chrome: ERR_BLOCKED_BY_XSS_AUDITOR details
After updating to Chrome 40.0.2214.111, variably when I visit certain Google related sites (like http://youtube.com and get presented with an ad before the video), the browser downloads a file named f.txt.
I do not have any adblock plugins installed.
f.txt contains a few lines of JavaScript...starting with:
if (!window.mraid) {document.write('\x3cdiv class="GoogleActiveViewClass" ' +'id="DfaVisibilityIdentifier_3851468350"\x3e');}document.write('\x3ca target\x3d\x22_blank\x22 href\x3d\x22https://adclick.g.doubleclick.net/pcs/click?xai\x3dAKAOjsvDhmmoi2r124JkMyiBGALWfUlTX-zFA1gEdFeZDgdS3JKiEDPl3iIYGtj9Tv2yTJtASqD6S-yqbuNQH5u6fXm4rThyCZ0plv9SXM-UPKJgH4KSS08c97Eim4i45ewgN9OoG3E_
In looking up the issue on Google, others have experienced the same, but I have not found any resolution or understanding of why this is happening. I assume it is a content-disposition related bug with some of the JS files loaded on the page, and will clear up in a future patch.
Wondering if anybody else had experienced / insight.
This issue appears to be causing ongoing consternation, so I will attempt to give a clearer answer than the previously posted answers, which only contain partial hints as to what's happening.
Some time around the summer of 2014, IT Security Engineer Michele Spagnuolo (apparently employed at Google Zurich) developed a proof-of-concept exploit and supporting tool called Rosetta Flash that demonstrated a way for hackers to run malicious Flash SWF files from a remote domain in a manner which tricks browsers into thinking it came from the same domain the user was currently browsing. This allows bypassing of the "same-origin policy" and can permit hackers a variety of exploits. You can read the details here: https://miki.it/blog/2014/7/8/abusing-jsonp-with-rosetta-flash/
Known affected browsers: Chrome, IE
Possibly unaffected browsers: Firefox
Adobe has released at least 5 different fixes over the past year while trying to comprehensively fix this vulnerability, but various major websites also introduced their own fixes earlier on in order to prevent mass vulnerability to their userbases. Among the sites to do so: Google, Youtube, Facebook, Github, and others. One component of the ad-hoc mitigation implemented by these website owners was to force the HTTP Header Content-Disposition: attachment; filename=f.txt on the returns from JSONP endpoints. This has the annoyance of causing the browser to automatically download a file called f.txt that you didn't request—but it is far better than your browser automatically running a possibly malicious Flash file.
In conclusion, the websites you were visiting when this file spontaneously downloaded are not bad or malicious, but some domain serving content on their pages (usually ads) had content with this exploit inside it. Note that this issue will be random and intermittent in nature because even visiting the same pages consecutively will often produce different ad content. For example, the advertisement domain ad.doubleclick.net probably serves out hundreds of thousands of different ads and only a small percentage likely contain malicious content. This is why various users online are confused thinking they fixed the issue or somehow affected it by uninstalling this program or running that scan, when in fact it is all unrelated. The f.txt download just means you were protected from a recent potential attack with this exploit and you should have no reason to believe you were compromised in any way.
The only way I'm aware that you could stop this f.txt file from being downloaded again in the future would be to block the most common domains that appear to be serving this exploit. I've put a short list below of some of the ones implicated in various posts. If you wanted to block these domains from touching your computer, you could add them to your firewall or alternatively you could use the HOSTS file technique described in the second section of this link: http://www.chromefans.org/chrome-tutorial/how-to-block-a-website-in-google-chrome.htm
Short list of domains you could block (by no means a comprehensive list). Most of these are highly associated with adware and malware:
ad.doubleclick.net
adclick.g.doubleclick.net
secure-us.imrworldwide.com
d.turn.com
ad.turn.com
secure.insightexpressai.com
core.insightexpressai.com
I experienced the same issue, same version of Chrome though it's unrelated to the issue. With the developer console I captured an instance of the request that spawned this, and it is an API call served by ad.doubleclick.net. Specifically, this resource returns a response with Content-Disposition: attachment; filename="f.txt".
The URL I happened to capture was https://ad.doubleclick.net/adj/N7412.226578.VEVO/B8463950.115078190;sz=300x60...
Per curl:
$ curl -I 'https://ad.doubleclick.net/adj/N7412.226578.VEVO/B8463950.115078190;sz=300x60;click=https://2975c.v.fwmrm.net/ad/l/1?s=b035&n=10613%3B40185%3B375600%3B383270&t=1424475157058697012&f=&r=40185&adid=9201685&reid=3674011&arid=0&auid=&cn=defaultClick&et=c&_cc=&tpos=&sr=0&cr=;ord=435266097?'
HTTP/1.1 200 OK
P3P: policyref="https://googleads.g.doubleclick.net/pagead/gcn_p3p_.xml", CP="CURa ADMa DEVa TAIo PSAo PSDo OUR IND UNI PUR INT DEM STA PRE COM NAV OTC NOI DSP COR"
Date: Fri, 20 Feb 2015 23:35:38 GMT
Pragma: no-cache
Expires: Fri, 01 Jan 1990 00:00:00 GMT
Cache-Control: no-cache, must-revalidate
Content-Type: text/javascript; charset=ISO-8859-1
X-Content-Type-Options: nosniff
Content-Disposition: attachment; filename="f.txt"
Server: cafe
X-XSS-Protection: 1; mode=block
Set-Cookie: test_cookie=CheckForPermission; expires=Fri, 20-Feb-2015 23:50:38 GMT; path=/; domain=.doubleclick.net
Alternate-Protocol: 443:quic,p=0.08
Transfer-Encoding: chunked
Accept-Ranges: none
Vary: Accept-Encoding
FYI, after reading this thread, I took a look at my installed programs and found that somehow, shortly after upgrading to Windows 10 (possibly/probably? unrelated), an ASK search app was installed as well as a Chrome extension (Windows was kind enough to remind to check that). Since removing, I have not have the f.txt issue.
This can occur on android too not just computers. Was browsing using Kiwi when the site I was on began to endlessly redirect so I cut net access to close it out and noticed my phone had DL'd something f.txt in my downloaded files.
Deleted it and didn't open.
Seems related to https://groups.google.com/forum/#!msg/google-caja-discuss/ite6K5c8mqs/Ayqw72XJ9G8J.
The so-called "Rosetta Flash" vulnerability is that allowing arbitrary
yet identifier-like text at the beginning of a JSONP response is
sufficient for it to be interpreted as a Flash file executing in that
origin. See for more information:
http://miki.it/blog/2014/7/8/abusing-jsonp-with-rosetta-flash/
JSONP responses from the proxy servlet now:
* are prefixed with "/**/", which still allows them to execute as JSONP
but removes requester control over the first bytes of the response.
* have the response header Content-Disposition: attachment.
Is it possible for a Chrome extension to listen for streaming audio from any of the browser's tabs? I would like to capture the streaming audio data and then analyse it.
Thanks
You could try 3 ways, neither one does provide 100% guarantee to meet your needs.
Before going into more detailed descriptions, I must note that Chrome extensions do not provide convenient tools for working on per connection level - sufficiently low level, required for stream capturing. This is by design. This is why the 1-st way is:
To look at other browsers, for example Firefox, which provides low-level APIs for connections. They are already known to be used by similar extensions. You may have a look at MediaStealer. If you do not have a specific requirement to build your system on Chrome, you should possibly move to Firefox.
You can develop a Chrome extension, which intercepts HTTP-requests by means of webRequest API, analyses their headers and extracts media urls (such as containing audio/mpeg MIME-type, for example, in HTTP-headers). Just for a quick example of code you make look at the following SO question - How to change response header in Chrome. Having the url you may force appropriate media download as a file. It will land in default downloads folder and may have unfriendly name. (I made such an extension, but I do not have requirements for further processing). If you need to further process such files, it can be a challenge to monitor them in the folder, and run additional analysis in a separate program.
You may have a look at NPAPI plugins in general, and their streaming APIs in particular. I can imagine that you create a plugin registered for, again, audio/mpeg MIME-type, and receives the data via NPP_NewStream, NPP_WriteReady and NPP_Write methods. The plugin can be wrapped into a Chrome extension. Though I made NPAPI plugins, I never used this API, and I'm not sure it will work as expected. Nethertheless, I'm mentioning this possibility here for completenees. This method requires some coding other than web-coding, meaning C/C++. NB. NPAPI plugins are deprecated and not supported in Chrome since September 2015.
Taking into account that you have some external (to the extension) "fingerprinting service" in mind, which sounds like an intelligent data processing, you may be interested in building all the system out of a browser. For example, you could, possibly, involve a HTTP-proxy, saving media from passing traffic.
If you're writing a Chrome extension, you can use the Chrome tabCapture API to record audio.
chrome.tabCapture.capture({audio: true}, function(stream) {
var recorder = new MediaRecorder(stream);
[...]
});
The rest is left as an exercise to the reader; MDN has more documentation on how to use MediaRecorder.
When this question was asked in 2013, neither chrome.tabCapture nor MediaRecorder existed.
Mac OSX solution using soundflower: http://rogueamoeba.com/freebies/soundflower/
After installing soundflower it should appear as a separate audio device in the sound preferences (apple > system preferences > sound). Divert the computer's audio to the 2ch option (stereo, 16ch is surround), then inside a DAW, such as 'audacity', set the audio input as soundflower. Now the sound should be channeled to your DAW ready for recording.
Note: having diverted the audio from the internal speakers to soundflower you will only be able to hear the audio if the 'soundflowerbed' app is actually open. You know it's open if there's a 8 legged blob in the top right task bar. Clicking this icon gives you the sound flower options.
My privoxy has the following log:
2013-08-28 18:25:27.953 00002f44 Request: api.audioaddict.com/v1/di/listener_sessions.jsonp?_method=POST&callback=_AudioAddict_WP_ListenerSession_create&listener_session%5Bid%5D=null&listener_session%5Bis_premium%5D=false&listener_session%5Bmember_id%5D=null&listener_session%5Bdevice_id%5D=6&listener_session%5Bchannel_id%5D=178&listener_session%5Bstream_set_key%5D=webplayer&_=1377699927926
2013-08-28 18:25:27.969 0000268c Request: api.audioaddict.com/v1/ping.jsonp?callback=_AudioAddict_WP_Ping__ping&_=1377699927928
2013-08-28 18:25:27.985 00002d48 Request: api.audioaddict.com/v1/di/track_history/channel/178.jsonp?callback=_AudioAddict_TrackHistory_Channel&_=1377699927942
2013-08-28 18:25:54.080 00003360 Request: pub7.di.fm/di_progressivepsy_aac?type=.flv
So I got the stream url and record it:
D:\Profiles\user\temp>wget pub7.di.fm/di_progressivepsy_aac?type=.flv
--18:26:32-- http://pub7.di.fm/di_progressivepsy_aac?type=.flv
=> `di_progressivepsy_aac#type=.flv'
Resolving pub7.di.fm... done.
Connecting to pub7.di.fm[67.221.255.50]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [video/x-flv]
[ <=> ] 1,234,151 8.96K/s
I got the file that can be reproduced in any multimedia pleer.