I want to monitor all the http request send from my chrome browser, and collect all the http request to another server.
I have read This question and These examples, but I don't know how to get the content of the request.
For example, when I browse google.com in my chrome browser with this chrome extension, this extension will send the following the another server which will collect it:
GET http://www.google.com/ HTTP/1.1
Accept: text/html, application/xhtml+xml, */*
Accept-Language: zh-CN
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.2)
Accept-Encoding: gzip, deflate
Proxy-Connection: Keep-Alive
Host: www.google.com
Pragma: no-cache
Cookie: blabla...
I can not find how to get the content of all the requests from here, is that possible?
Sorry, it's not possible at the moment. All you can get is headers.
Related
On Thursday (2017-04-26), I began seeing the following error when I logged into my application using my Authenticator JSF page.
[#|2017-04-30T15:18:51.649-0500|WARNING|glassfish
4.1|javax.enterprise.web|_ThreadID=30;_ThreadName=http-listener-1(2);_TimeMillis=1493583531649;_LevelValue=
StandardWrapperValve[Faces Servlet]: Servlet.service() for servlet
Faces Servlet threw exception
javax.faces.application.ViewExpiredException:
viewId:/security/Authenticator.xhtml - View
/security/Authenticator.xhtml could not be restored. at
com.sun.faces.lifecycle.RestoreViewPhase.execute(RestoreViewPhase.java:212)
at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
at
com.sun.faces.lifecycle.RestoreViewPhase.doPhase(RestoreViewPhase.java:123)
at
com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:198)
My Authenicator.xhtml page is backed by a Authenticator.java class with the following header.
#Named
#ViewScoped
public class Authenticator implements Serializable {
During my research, I discovered the following:
I am able to log into my application using Chrome 58.0.3029.81 one time after restarting the computer running the GlassFish 4.1.2 server. If I log off, I will get the above error on every future log in attempt. (This is a weird one.)
I can log in using Internet Explorer
I can log in using Chrome versions older the 58.0.3029.81.
I can log in using Chrome 57.0.2987.132 on my Android telephone
I can log in using Chrome 58.0.3029.81 if I change the javax.faces.STATE_SAVING_METHOD variable in my web.xml file from server to client.
Why would Chrome 58.0.3029.81 kill the Authenticator view resulting in the ViewExpiredException?
As requested, I analyzed the network traffic and determined that Chrome 58.0.3029.81 sends two additional Get requests during the Authenticator.xhtml display process than Chrome 57.0.2987.133 sends.
Chrome 57:
GET /webapp/security/Authenticator.xhtml HTTP/1.1
GET /webapp/security/RES_NOT_FOUND HTTP/1.1
GET /webapp/security/RES_NOT_FOUND HTTP/1.1
POST /webapp/security/Authenticator.xhtml HTTP/1.1
Chrome 58:
GET /webapp/security/Authenticator.xhtml HTTP/1.1
GET /webapp/security/RES_NOT_FOUND HTTP/1.1
GET /webapp/security/RES_NOT_FOUND HTTP/1.1
GET /webapp/security/RES_NOT_FOUND HTTP/1.1
GET /webapp/security/RES_NOT_FOUND HTTP/1.1
POST /webapp/security/Authenticator.xhtml HTTP/1.1
Since I don't know why Chrome sends the RES_NOT_FOUND gets in the first place I don't know if sending two extra is a bad thing but it seems to be related to GlassFish 4.1.2 not being able to reconnect to the Authenticator view.
Could this be an issue with my Authenticator.xhtml page or is it a Chrome 58/GlassFish 4.1.2 issue?
The following is a comparison of the Post information:
Chrome 57 Post
POST /webapp/security/Authenticator.xhtml HTTP/1.1
Host: localhost:8080
Connection: keep-alive
Content-Length: 205
Cache-Control: max-age=0
Origin: http://localhost:8081
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36
Content-Type: application/x-www-form-urlencoded
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Referer: http://localhost:8081/webapp/security/Authenticator.xhtml
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.8
Cookie: JSESSIONID=4067aa3d0df7f2bc26b8200a8c4a;
modena_expandeditems=j_idt32%3Awelcome-menu
authentication-form=authentication-form&authentication-form%3AuserName=XXX&authentication-form%3Apassword=XXX&authentication-form%3Aj_idt93=&javax.faces.ViewState=-4577625721740212982%3A4298605796688550126
Chrome 58 Post
POST /webapp/security/Authenticator.xhtml HTTP/1.1
Host: localhost:8080
Connection: keep-alive
Content-Length: 204
Cache-Control: max-age=0
Origin: http://172.24.1.125:8081
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.81 Safari/537.36
Content-Type: application/x-www-form-urlencoded
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Referer: http://172.24.1.125:8081/webapp/security/Authenticator.xhtml
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.8
Cookie: JSESSIONID=4089ef02f0bca32d331de1f5404f
authentication-form=authentication-form&authentication-form%3AuserName=XXX&authentication-form%3Apassword=XXX&authentication-form%3Aj_idt93=&javax.faces.ViewState=3383766421781608154%3A6418504070036764787
The only difference that I see is that Chrome 57 appended "; modena_expandeditems=j_idt32%3Awelcome-menu" after the JSESSIONID.
This turned out to be an issue with version 2.1.1 of the PrimeFaces premium theme called Modena and PrimeFaces 6. During HTTP analysis, I noticed that Chrome 57 sent 2 RES_NOT_FOUND requests and Chrome 58 sent 4 RES_NOT_FOUND requests. This was a known issue with Modena 2.1.1 as documented in the following PrimeFaces Modena Forum issue:
PrimeFaces Modena Forum Issue
During each RES_NOT_FOUND request, the JSESSIONID would change and something about the additional 2 changes in Chrome 58 would break the link between JSESSION and ViewState.
Upgrading Modena to version 2.1.3 eliminated all the RES_NOT_FOUND requests and resolved the ViewExpired issue.
It's not a programming question, but it will be interesting for some developers.
Today I came across a question where user were asking about parsing Json data in C#. Nothing new, but he gave a link with Json file:
http://sapi.confirmtkt.com/api/platform/hotel/gethotels?city=Bangalore&checkinDate=08-01-2016&checkoutDate=09-01-2016&adults=2&rooms=1&children=0&childrenages=
The page is not available anymore!.
If you browse the above link in Google Chrome you can notice that the file will be shown as XML. Then I checked the url in Edge and it showed me a Json file.
Google Chrome
Edge
I'm a bit confused. Why Google Chrome shows Json file as XML?
That's because Chrome is sending different data in the Accept HTTP header:
Chrome's request:
GET http://sapi.confirmtkt.com/api/platform/hotel/gethotels?city=Bangalore&checkinDate=08-01-2016&checkoutDate=09-01-2016&adults=2&rooms=1&children=0&childrenages= HTTP/1.1
Host: sapi.confirmtkt.com
Connection: keep-alive
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.48 Safari/537.36
Accept-Encoding: gzip, deflate, sdch
Accept-Language: es,en;q=0.8
Edge's request:
GET http://sapi.confirmtkt.com/api/platform/hotel/gethotels?city=Bangalore&checkinDate=08-01-2016&checkoutDate=09-01-2016&adults=2&rooms=1&children=0&childrenages= HTTP/1.1
Accept: text/html, application/xhtml+xml, image/jxr, */*
Accept-Language: es-CL,es;q=0.8,en-US;q=0.5,en;q=0.3
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2486.0 Safari/537.36 Edge/13.10586
Accept-Encoding: gzip, deflate
Host: sapi.confirmtkt.com
Connection: Keep-Alive
You can see how Chrome is accepting application/xml;q=0.9 in the Accept header. So, I fired up Fiddler and intercepted the GET request, deleted that part of the accept header and now the server replies back with json rather than XML.
TL;DR: The server was giving back two different responses for the same URL because of the Accept header.
I have a compressed resource that when viewed in IE, loads from the cache as expected. However when my application loads the same URL, Wininet ignores the cache and always downloads the content from the server. With dynamic content compression disabled on IIS, my application behaves the same as IE (the HTTP GET includes an If-None-Match header.)
What can I do to get my app behave the same as IE?
_httpClient = new HttpClient(new WebRequestHandler
{
CachePolicy = new HttpRequestCachePolicy(HttpRequestCacheLevel.Default),
AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip
});
using (Task<HttpResponseMessage> tget = _httpClient.GetAsync(uri, HttpCompletionOption.ResponseHeadersRead, _cancel))
{
tget.Wait();
response = tget.Result;
}
The HTTP headers for my application's GET:
GET https://beautykiosktest.coinstar.com/conductor/configuration/files/Promos.xml?kioskid=ENG20130027 HTTP/1.1
Accept: */*
Accept-Language: en-US
User-Agent: ConfigurationService/2.3.0.0
Host: beautykiosktest.coinstar.com
Accept-Encoding: gzip, deflate
HTTP/1.1 200 OK
Cache-Control: max-age=30
Transfer-Encoding: chunked
Content-Type: application/xml; charset=utf-8
Content-Encoding: gzip
ETag: "cHmrX5Fp2Z+ETV/qIXiS2A=="
Vary: Accept-Encoding
Server: Microsoft-IIS/8.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Tue, 17 Mar 2015 00:39:18 GMT
IE's GET with a 304 response is what I expect
GET https://beautykiosktest.coinstar.com/conductor/configuration/files/Promos.xml?kioskid=ENG20130027 HTTP/1.1
Accept: image/jpeg, image/gif, image/pjpeg, application/x-ms-application, application/xaml+xml, application/x-ms-xbap, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*
Accept-Language: en-US
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/7.0; SLCC2; .NET CLR 2.0.50727; InfoPath.3; .NET4.0C; .NET4.0E; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET CLR 1.1.4322)
Accept-Encoding: gzip, deflate
Host: beautykiosktest.coinstar.com
If-None-Match: "cHmrX5Fp2Z+ETV/qIXiS2A=="
DNT: 1
Connection: Keep-Alive
HTTP/1.1 304 Not Modified
Cache-Control: max-age=30
ETag: "cHmrX5Fp2Z+ETV/qIXiS2A=="
Server: Microsoft-IIS/8.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Tue, 17 Mar 2015 00:42:13 GMT
I ended up filing a ticket with Microsoft to resolve this and found out that there is a bug in .NET 4 which causes this. The problem is that a cache entry populated from a response with a Vary header doesn't get matched properly on the next GET because even though the AutomaticDecompression property is set, .NET hasn't added the Accept-Encoding header into the request before doing a WinINET cache test.
The workaround/fix is to add an "Accept-Encoding" default header to the HttpClient instance before using it.
_httpClient.DefaultRequestHeaders.Add("Accept-Encoding", "gzip, deflate");
In my application I needed to make periodic cross-domain HTTP POST requests to receive up-to-date data from a server (periodic polling). The application was not working in IE8 while it was in Chrome. So I decided to debug this with Wireshark:
I executed 2 equivalent codes in IE8 and Chrome. I monitored my network with Wireshark.
The wireshark filter is:
http.request.full_uri == "http://www.andlabs.net/html5/uCOR.php"
I noticed that IE8 only sends the request once and returns the same cached response for the following invocations. Chrome, on the other hand, sends a new request each time.
The code I used for IE8:
var cor = new XDomainRequest();
cor.onload = function() { alert(cor.responseText);}
cor.open('POST', 'http://www.andlabs.net/html5/uCOR.php');
cor.send();
The code I used for Chrome:
var cor = new XMLHttpRequest();
cor.onload = function() { alert(cor.responseText);}
cor.open('POST', 'http://www.andlabs.net/html5/uCOR.php');
cor.send();
In order to prevent cached response in IE8 I tried the following code and it worked:
var cor = new XDomainRequest();
cor.onload = function() { alert(cor.responseText);}
cor.open('POST', 'http://www.andlabs.net/html5/uCOR.php');
cor.send(''+new Date());
Why does IE8 behave this way and is there any way to solve this in a different way than what I did? Notice that I cannot use the same trick for GET requests.
By the way the request and the response for IE are as follows:
request:
POST /html5/uCOR.php HTTP/1.1
Accept: */*
Origin: http://jsbin.com
Accept-Language: en-US
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C)
Host: www.andlabs.net
Content-Length: 0
Connection: Keep-Alive
Cache-Control: no-cache
response:
HTTP/1.1 200 OK
Content-Type: text/html
Server: Microsoft-IIS/7.0
Access-Control-Allow-Origin: *
X-Powered-By: ASP.NET
Date: Tue, 17 Jan 2012 21:41:39 GMT
Content-Length: 180
This is a page from www.andlabs.net which is accessible from any website through Cross Origin Requests<br>This page contains the following header:<br>Access-Control-Allow-Origin: *
It looks like there aren't any caching headers on the response, so browsers may behave differently. Can you add the following header to the response: Cache-Control: no-cache
I developed my own custom web server for a robot I built (video), without really knowing the intricacies of HTTP. One issue I am running into is I send a request from a browser, say, http://192.168.2.10/r?cmd=doStuff and the robot responds. Then I send the same request again from the browser (by clicking on a bookmark I saved), and the request never goes out, it just shows the response from the previous request.
Here is what comes from the browser (IE8 in particular):
GET /r?cmd=s HTTP/1.1
Accept: image/jpeg, application/x-ms-application, image/gif, application/xaml+xml, image/pjpeg, application/x-ms-xbap, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*
Accept-Language: en-US
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; Tablet PC 2.0; InfoPath.2; .NET4.0C; .NET4.0E; Zune 4.7)
Accept-Encoding: gzip, deflate
Host: 192.168.2.10
Connection: Keep-Alive
The robot responds with the following:
HTTP/1.1 200 OK
Content-Type: text/html; charset=UTF-8
Server: The Little Robot That Could 1.0
Date: Thu, 01 Jan 2009 00:05:00 GMT
Content-Length: 4
X-Powered-By: Little Robot HTTP Server
Do I need to throw some other HTTP header into the mix to prevent the browser from caching the response?
Yes, you need cache-control header.
Cache-Control: no-cache;max-age=0
I would try throwing in a Pragma: no-cache. Should work on any modern browser.
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
Well.
The real problem here is using GET for "do something".