I want to access live XHR request data in Chrome programatically. (I can't just connect to the website using a script, it must go through Chrome)
So far, I've found the easiest way to do this is to launch Chrome with remote-debugging turned on. And then use Chrome DevTools to get the data:
import PyChromeDevTools
chrome = PyChromeDevTools.ChromeInterface()
chrome.Network.enable()
chrome.Page.enable()
chrome.Page.navigate(url='http://example.com/')
The Chrome DevTools protocol guide is here:
https://chromedevtools.github.io/devtools-protocol/
But I don't know how to actually access the live XHR requests.
Here's the data shown in the Chrome DevTools UI:
The connect requests contain everything I need in the response body. I simply need access to them live (as they continue to stream in).
I just want to access this data using DevTools-protocol so I can further process it.
Any ideas how?
Related
From the dev tools, is it possible to tell Chrome to return, for example, a 404 when a specific url is requested (leaving other requests to process normally)?
You can block specific request URLs or patterns of URLs from the Network panel.
Manage your blocked URLs from the Request Blocking tab.
Official feature announcement: https://developers.google.com/web/updates/2017/04/devtools-release-notes#block-requests
Blocking the request with chrome devtools is the easiest solution. But if you want to mock a specific error code, let say 401 or 404 you can use a chrome extension to do it.
There are plenty of them in the store, I just tried one randomly and it allowed me to mock a specific error code https://chrome.google.com/webstore/detail/mokku-mock-api-calls-seam/llflfcikklhgamfmnjkgpdadpmdplmji?hl=en
I am having a small issue whereby Chrome (Version 60.0.3112.113, Mac OS) is returning a failed status response from a CORS preflight OPTIONS request.
The endpoint it is querying is a nodejs server which previously did not respond correctly to the preflight request. I have since fixed this.
The preflight request works in all other browsers, and works in Chrome on all other computers. I have tested using Browserling, and everything works as expected.
As such I am assuming (with 99% confidence) that this is some sort of caching issue with Chrome on my development computer. I have however been unable to resolve this, and have at this point tried deleting any/all cache options that I can find in the various Chrome options menus.
Can anyone share any insight?
I could not find a way of clearing whatever internal cache Chrome is using in this regard.
My resolution was simply to append a query string (based on the build time) to the request such that Chrome does not use this internal cache.
This is a good way of versioning resources (JS, CSS, API endpoints etc) anyway.
I would like to view HTTP POST data that was sent in Chrome.
The data is in memory now, and I have the ability to resubmit the form.
I know that if I resubmit the server will throw an error. Is there anyway I can view the data that is in Chrome's memory?
Go to Chrome Developer Tools (Chrome Menu -> More Tools -> Developer Tools)
Choose "Network" tab
Refresh the page you're on
You'll get list of http queries that happened, while the network console was on. Select one of them in the left
Choose "Headers" tab
Voila!
You can filter for HTTP POST requests with the Chrome DevTools. Just do the following:
Open Chrome DevTools (Cmd+Opt+I on Mac, Ctrl+Shift+I or F12 on
Windows) and click on the "Network" tab
Click on the "Filter" icon
Enter your filter method: method:POST
Select the request you want to debug
View the details of the request you want to debug
Screenshot
Tested with Chrome Version 53.
You can use Canary version of Chrome to see request payload of POST requests.
Another option that may be useful is a dedicated HTTP debugging tool. There's a few available, I'd suggest HTTP Toolkit: an open-source project I've been working on (yeah, I might be biased) to solve this same problem for myself.
The main difference is usability & power. The Chrome dev tools are good for simple things, and I'd recommend starting there, but if you're struggling to understand the information there, and you need either more explanation or more power then proper focused tools can be useful!
For this case, it'll show you the full POST body you're looking for, with a friendly editor and highlighting (all powered by VS Code) so you can dig around. It'll give you the request & response headers of course, but with extra info like docs from MDN (the Mozilla Developer Network) for every standard header and status code you can see.
A picture is worth a thousand StackOverflow answers:
On Chrome 96 DevTools Network Tab;
After clicking the request, there is a "Payload" tab that shows the Form Data:
You can view the data as url encoded / decoded:
You can view the data as source / parsed:
Even if the method is GET you can see the Payload as Query String Parameters:
It has a tricky situation: If you submit a post form, then Chrome will open a new tab to send the request. It's right until now, but if it triggers an event to download file(s), this tab will close immediately so that you cannot capture this request in the Dev Tool.
Solution:
Before submitting the post form, you need to cut off your network, which makes the request cannot send successfully so that the tab will not be closed. And then you can capture the request message in the Chrome Devtool(Refreshing the new tab if necessary)
The Network tab is all you need...
The other people made very nice answers, but I would like to complete their work with an extra development tool. It is called Live HTTP Headers and you can install it into your Firefox, and in Chrome we have the same plug in like this.
Working with it is queit easy.
Using your Firefox, navigate to the website which you want to get your post request to it.
In your Firefox menu Tools->Live Http Headers
A new window pop ups for you, and all the http method details would be saved in this window for you. You don't need to do anything in this step.
In the website, do an activity(log in, submit a form, etc.)
Look at your plug in window. It is all recorded.
Just remember you need to check the Capture.
At the moment I am developing a Google Chrome Extension in which I have to fetch data from a server. I am doing this with an XMLHttpRequest and all is going well, except for the fact when I launch Google Chrome and immediately click my extension, it won't open until the XMLHttpRequest is completed. The request is sent to a rather slow server, that's where the problem is. But is there a way in which you might run the XMLHttpRequest in another thread or does Google Chrome give you another way to fetch data from a server that runs in the background so I can open my extension while it's still getting it's data from the server?
Maybe good to know, the XMLHttpRequest runs in the background page, not the popup page.
Looks like the Asynchronous parameter has to be set to true.
Is there a way, or an extension, that lets me watch the "traffic" going through a WebSocket? For debugging purposes I'd like to see the client and server requests/responses.
Chrome developer tools now have the ability to list WebSocket frames and also inspect the data if the frames are not binary.
Process:
Launch Chrome Developer tools
Load your page and initiate the WebSocket connections
Click the Network Tab.
Select the WebSocket connection from the list on the left (it will have status of "101 Switching Protocols".
Click the Messages sub-tab. Binary frames will show up with a length and time-stamp and indicate whether they are masked. Text frames will show also include the payload content.
If your WebSocket connection uses binary frames then you will probably still want to use Wireshark to debug the connection. Wireshark 1.8.0 added dissector and filtering support for WebSockets. An alternative may be found in this other answer.
They seem to continuously change stuff in Chrome, but here's what works right now :-)
First you must click on the red record button or you'll get nothing.
I never noticed the WS before but it filters out the web socket
connections.
Select it and then you can see the Frames (now called Messages) which will show you error messages etc.
Chrome Canary and Chromium now have WebSocket message frame inspection feature. Here are the steps to test it quickly:
Navigate to the WebSocket Echo demo [dead as of 2022], hosted on the websocket.org site https://echo.websocket.events/.ws (run by the company Lob) or you can locally run the Echo Server project.
Turn on the Chrome Developer Tools.
Click Network, and to filter the traffic shown by the Dev Tools, click WebSockets.
In the Echo demo, click Connect. On the Headers tab in Google Dev Tool you can inspect the WebSocket handshake.
Click the Send button in the Echo demo.
THIS STEP IS IMPORTANT: To see the WebSocket frames in the Chrome Developer Tools, under Name/Path, click the echo.websocket.org entry, representing your WebSocket connection. This refreshes the main panel on the right and makes the WebSocket Frames tab show up with the actual WebSocket message content.
Note: Every time you send or receive new messages, you have to refresh the main panel by clicking on the echo.websocket.org entry on the left.
I also posted the steps with screen shots and video.
My recently published book, The Definitive Guide to HTML5 WebSocket, also has a dedicated appendix covering the various inspection tools, including Chrome Dev Tools, Chrome net-internals, and Wire Shark.
If you don't have a page which is accessing the websocket, you can open up the Chrome console and type your JavaScript in:
var webSocket = new WebSocket('ws://address:port');
webSocket.onmessage = function(data) { console.log(data); }
This will open up the web socket so you can see it in the network tab and in the console.
The other answers cover the most common scenario: watch the content of the frames (Developer Tools -> Network tab -> Right click on the websocket connection -> frames).
If you want to know some more informations, like which sockets are currently open/idle or be able to close them you'll find this url useful
chrome://net-internals/#sockets
You have 3 options: Chrome (via Developer Tools -> Network tab), Wireshark, and Fiddler (via Log tab), however they all very basic. If you have very high volume of traffic or each frame is very large, it becomes very difficult to use them for debugging.
You can however use Fiddler with FiddlerScript to inspect WebSocket traffic in the same way you inpect HTTP traffic. Few advantages of this solution are that you can leverage many other functionalities in Fiddler, such as multiple inspectors (HexView, JSON, SyntaxView), compare packets, and find packets, etc.
Please refer to my recently written article on CodeProject, which show you how to Debug/Inspect WebSocket traffic with Fiddler (with FiddlerScript). http://www.codeproject.com/Articles/718660/Debug-Inspect-WebSocket-traffic-with-Fiddler
If you want a better experience, I'd suggest using Postman to debug WebSocket requests. It has been released as a new feature.
https://stackoverflow.com/a/43754722/15988851
I'm just posting this since Chrome changes alot, and none of the answers were quite up to date.
Open dev tools
REFRESH YOUR PAGE (so that the WS connection is captured by the network tab)
Click your request
Click the "Frames" sub-tab
You should see somthing like this:
Chrome developer tools allows to see handshake request which stays pending during the opened connection, but you can't see traffic as far as I know. However you can sniff it for example.
Short answer for Chrome Version 29 and up:
Open debugger, go to the tab "Network"
Load page with websocket
Click on the websocket request with upgrade response from server
Select the tab "Frames" to see websocket frames
Click on the websocket request again to refresh frames
I have used Chrome extension called Simple WebSocket Client v0.1.3 that is published by user hakobera. It is very simple in its usage where it allows opening websockets on a given URL, send messages and close the socket connection. It is very minimalistic.
As for a tool I started using, I suggest firecamp Its like Postman, but it also supports websockets and socket.io.