Basically what I need is a way to automatize the result of the following operations:
open a new tab;
open the Network tab in the developer tools;
load an URL;
select "Save All as HAR".
Often, proposed solutions involves the use of PhantomJS, browsermob-proxy, or pcap2har; those won't fit my case since I need to work with SPDY traffic.
I tried to dive into the Google Chrome Extensions API and indeed I managed to automatize some tasks, but still no luck for what concerns the HAR files generation. Now this method is particularly promising but I still can't figure out how would I use it.
In other words, I need something like this experiment from the Google guys. Note the following:
We used Chrome's remote debugging interface with a custom client that starts up the browser on the phone, clears its cache and other state, initiates a web page load, and receives the Chrome developer tools messages to determine the page load times and other performance metrics.
Any ideas?
Solution
For the curious, I ended up with a Node.js module that automates such kind of tests: chrome-har-capturer. This also gave me the opportunity to dig deeper into the Remote Debugging Protocol and to write a lower-level Node.js interface for general-purpose Chrome automation: chrome-remote-interface.
The short answer is, there is no way to get at the data you are after directly. The getHAR method is only applicable to extensions meant to extend DevTools itself. The good news is, you can construct the HAR file yourself without too much trouble - this is exactly what phantom.js does.
Start Chrome with remote debugging
Connect to Chrome on the debugging port with a websocket connection
Enable "Network" debugging, you can also clear cache, etc - see Network API.
Tell the browser to navigate to the page you want to capture, and Chrome will stream all the request meta-data back to you.
Massage the network data into HAR format, ala phantom.js
...
Profit.
For a head start, I have a post that with sample Ruby code that should you get started with steps 1-4: http://www.igvita.com/2012/04/09/driving-google-chrome-via-websocket-api/
By now there's a browser plugin to do that: https://github.com/devtools-html/har-export-trigger
It uses the WebExtensions DevTools API and I got it to work with both Firefox and Chrome.
See my code for Chrome here: https://github.com/theri/web-measurement-tools/blob/master/load/load_url_using_chrome.py#L175
Automatically installing the plugin in Chrome is a bit more complicated than in Firefox, but feasible - I extracted the plugin archive locally and then link to it in chrome_prefs.json (see same repository).
Not sure if it helps, HAR Recorder uses chrome debug protocol to record HAR and generate a har file (without opening devtools). If you want a variation, you can fork and make changes on it.
Related
I am building a chrome extension that is supposed to aid in the debugging of software that runs on top of a website. This software can have a debug mode enabled that will cause a lot of output to the console using console.log.
I want to use my chrome extension to parse the console messages and show the important events in the UI for quicker debugging. However, I am not seeing a way to simply do this with the API. Is there something I am missing? Should I override the console.log function? How would I go about doing that?
There are two methods.
Override console.log, console.warn, and so on, in page context (this is important!). There are lots of examples (here's a random one). In your case it'll be even simpler as you'll just call the original method and transfer the arguments via CustomEvent to your content script (example), which will accumulate them.
Use chrome.debugger API with Console.messageAdded or Runtime.consoleAPICalled events. This will show a message bar in the entire browser about debugger being active unless you hide it globally by running chrome with --silent-debugger-extension-api command line, but that's somewhat dangerous if you accidentally install a malicious extension that uses chrome.debugger API.
I need to log the information shown in network tab to a file. I can do it by firing network tab first and then exporting it to a file. But is it possible to run the network tab and exporting it to a file in the background automatically whenever the Chrome is opened?
Is it possible to do?
It depends on your requirement. If your extension involves DevTools, and therefore it is open, you can use the chrome.devtools.network.getHAR() method to get the network traffic. You don't need to navigate to the Network tab.
However, if you want to access the network data without DevTools being opened, this API will not work, as it's only exposed to the DevTools instance. There are a couple of possible options.
Option 1
You could use the chrome.webRequest API to intercept each request/response and append whatever data you want/can to an object. You could then use the chrome.downloads API to download the data. In your case, you could use a data URI.
var url = 'data:application/json;base64,' + btoa(data);
chrome.downloads.download({url: url, filename: 'notQuiteAHAR'json'});
I haven't tested this in practice, and I'm not too sure if you can determine when all requests are done before calling the download.
Option 2
Use the more low level chrome.debugger API, as per the comment by #wOxxOm. The debugging protocol only allows one instance of the debugger at a time, so this will only ever work if you don't have DevTools running. The API exposes a lot more than the chrome.webRequest API, but requires a bit of work to get all the data you need.
There's a repository called chrome-har-capturer, which uses the debugging protocol. Of particular interest is har.js, which uses the events found in the debugger API to manually construct the HAR. The purpose of the library is for remote debugging purposes, but I believe you can use the debugger API in an extension, and so you could probably use aspects of this repository.
As suggested by Gideon Pyzer, HAR Recorder uses chrome debug protocol to record HAR and generate a har file (without opening devtools). If you want a variation, you can fork and make changes on it.
Is there any way to add a time delay to a script/asset loading in the Google Chrome Browser via the Chrome Developer Tools? Or to block a script loading entirely?
The reason I want to do this is to see how a site performs when a script/asset suffers from delayed loading or failed loading.
In Chrome Developer Tools when you are in the Network you can add custom throttling. You can specify download and upload speed as well as request latency. But this will apply to all resources and not only to a specific one.
--- edit ---
For delay individual URLs on any page, you can use a chrome extension (since it can intercept browser requests). I use https://chrome.google.com/webstore/detail/url-throttler/kpkeghonflnkockcnaegmphgdldfnden
If the asset is a third party or hosted on a different domain, there is a Chrome plugin that's designed to test what you are calling loading delays, that are also called SPOF (Single Point Of Failure). It might not be very intuitive at first use, but it's very helpful :
The plugin is called SPOF-O-Matic and can be found here: https://chrome.google.com/webstore/detail/spof-o-matic/plikhggfbplemddobondkeogomgoodeg
Following solutions have nothing with Chrome Devtools but they work.
If you don't mind redirects then you can try Slowwly or deelay.me.
Other alternative for non-windows OS is Comcast.
You can do it by combining multiple resources.
I use http://www.deelay.me/ to generate the delayed url.
I then combine it with requestly extension (https://requestly.io/) to create a host replace rule to target a specific resource
This devtools extension can fit your requirement, Its a Chrome/Firefox devtools extension that can simulate http request delay for any configurable URL.
Chrome devtools extension webstore
You can use Requestly Browser Extension or Desktop App to modify your network requests. Delaying/Throttling scripts is one of the use-cases amongst many already supported in Requestly.
Here's how you can do it. Once you install Requestly, Use the Delay Request feature.
Create New Rule & Select Delay Rule Type
Define URL (or URL Pattern) and the delay value
This article explains 3 different approaches to add delay/throttle APIs (or network requests). However, in your case Requestly extension based approach should work best.
PS - I built Requestly.
I am trying to build an extension that would notify a user when new version of Chrome is available.
I tried to inspect network traffic when Chrome is checking for an update and it is sending a request to http://74.125.95.113/service/update2?w=3:{long_encoded_string} page that returns XML with information I need:
<?xml version="1.0" encoding="UTF-8"?>
<gupdate xmlns="http://www.google.com/update2/response" protocol="2.0" server="prod">
<daystart elapsed_seconds="31272"/>
<app appid="{8A69D345-D564-463C-AFF1-A69D9E530F96}" status="ok">
<updatecheck status="noupdate"/>
<ping status="ok"/>
</app>
</gupdate>
Besides sending {long_encoded_string} as URL parameter it is also sending some encoded cookie.
Maybe someone familiar with Chrome build process can shed some light on those encoded strings and how to build them? Maybe there is another easier way (I have a feeling that string encoding is a dead end for me)?
Google Chrome uses omaha to do updates to clients. The protocol is described here: http://omaha.googlecode.com/svn/wiki/cup.html. One thing you have to notice is that Google Chrome automatically downloads the update to your computer and then notifies (via icon on the tool menu). Unless you force check by opening the about dialog (in Windows).
As you have noticed, the Chrome GUID is {8A69D345-D564-463c-AFF1-A69D9E530F96}
The best way to see how Google Chrome is updating is to check the source code which is public. Google just ifdefs their version of Chrome in Chromium.
Google Chrome Code
The base class where all the updates happen is in UpgradeDetector, it basically checks for an upgrade every 1 hour for the dev channel and once a day for all the other channels and builds (stable / beta). The Chromium way to do a scheduled events is through Tasks, in this case it is caleld a DetectUpgradeTask which checks for a specific BrowserDistribution::GetSpecificDistribution. There are many browser distributions, for Windows it is called GoogleChromeDistribution which is in charge to figure out what the version that needs to be update..
So why am I saying all this, Google Chrome is just querying registry settings and local files to figure out if a new update exists. The the UpgradeDetector just compares the distributions if they are the same. The implies that Omaha does the whole update mechanism. And the best part to figure out what omaha does is to look at their omaha update protocol.
The Omaha Protocol
From quickly glancing at the protocol, the approach your taking is the correct one, but you have to figure out the public key. In this case, the w, which differs for every request. You can read more about this in the "Protocol observations" in the omaha update protocol. It does this to do a securely check for download updates and they do this to protect the communication. They want the connection that checks for updates to be authentic and fresh so an attacker cannot replace or modify the message nor trick the client to upgrading a vulnerable version.
So what now
It isn't just a simple request to the server to do an update check. The Omaha client protocol provides an alternative to SSL for update checks and does client-server requests to see if its a valid connection. They are doing all this to protect the communication as explained before.
Unfortunately I don't think there is a "Chrome Extension" HTML'sh way to do this unless you implement that handshake yourself using NPAPI. Don't take my word for granted, I might be totally wrong :) Unless you can do the handshake all through XHR requests.
Since you want to check if Chrome has been updated and not installed, you have to verify that a new distribution has been downloaded as explained above in the code GoogleChromeDistribution which definitely requires NPAPI to read the registry.
I am on a mission to expand my knowledge and create an extension for chrome similar to how firephp works. I want to integrate with my existing logging and debugging api within my framework and I want to be able to send these messages to the console. Nothing really robust to start with just a way to send debug messages to the chrome console from php.
The plan is to send the messages via the headers and have the extension read those headers and interpret them. I've been trying to find information on accessing the response headers and can't find any examples. Some of the research has led me to possibly having to develop an NPAPI plugin to be able to accomplish this.
Before traveling down a dead end path I wanted to get the communities opinion here on which path I should be taking to find a solution.
Chrome cannot currently do this, but Google is working on it. A preliminary and incomplete implementation is in the development version of Chrome, or in Chrome Canary.
http://code.google.com/chrome/extensions/experimental.webRequest.html
onHeadersReceived
http://dev.chromium.org/developers/design-documents/extensions/notifications-of-web-request-and-navigation
You can track progress here.
http://crbug.com/50943
The web request api is now in stable and can be used to access header events.
https://developer.chrome.com/extensions/webRequest.html
Here is an extension that does what you are trying to do. It uses cookies to communicate, from what I can tell.