Multi-client remote debugging using ChromeDriver and Chrome DevTools protocol - google-chrome

Question

So with Chrome 63 there is now support for Multi-client remote debugging (https://developers.google.com/web/updates/2017/10/devtools-release-notes)
What I want to achieve is use the Chrome DevTools Protocol HeapProfiler with some selenium tests. I'm running version 64 Chrome dev channel and ChromeDriver 2.33.
ChromeOptions options = newChromeOptions();
options.addArguments("--remote-debugging-port=9222");
WebDriver driver = new ChromeDriver(options);
... selenium stuff
A new chrome window will open and hang until it times out. I can confirm that the chrome window opened is chrome 64 by going to help > about google chrome to check the version.
I get this error which appears to be the the webdriver losing connection.
Exception in thread "main" org.openqa.selenium.WebDriverException: chrome not
reachable
The DevTools Protocol is working because I am able to open http://localhost:9222 in another chrome window and see debugging interface.
Has anyone been able to get these two things to work together?
Thanks :)

Answer 1

Here the catch was that if you pass the "remote-debugging-port" switch then chromedriver has a bug where it still internally assigns a randon port and keep trying to connect to it rather than connecting to 9222 port.
options.addArguments("--remote-debugging-port=9222");
We can solve this by skipping this command switch and let chrome decides this random port and extract this port number from chromedriver logs.
I made it work and here I have blogged it in detail.
https://medium.com/#sahajamit/selenium-chrome-dev-tools-makes-a-perfect-browser-automation-recipe-c35c7f6a2360

Answer 2

Selenium 4 release will have a user friendly API for Chrome DevTools protocol. I just finished implementing Network and Performance domains for the Selenium Java client. https://github.com/SeleniumHQ/selenium/pull/7212
In addition, there is a generic API for all domains in Java client that was merged a while ago. All those new features will be released probably in the next Alpha release.
This is a nice article on how to use Log:
https://codoid.com/selenium-4-chrome-devtools-log-entry-listeners/

Answer 3

Here is what i do to get the information needed fro remotedebugging nd additionally to prevent defining the port. I get it through the SeleniumLog-API
DesiredCapabilities capabilities = DesiredCapabilities.chrome();
ChromeOptions options = new ChromeOptions();
options.setBinary(chromeBin);
capabilities.setCapability(ChromeOptions.CAPABILITY, options);
LoggingPreferences logPref = new LoggingPreferences();
logPref.enable(LogType.DRIVER, Level.ALL);
driverInstance = new ChromeDriver(capabilities);
LogEntries x = driverInstance.manage().logs().get(LogType.DRIVER);
for(LogEntry e:x.getAll()){
if(e.getMessage().contains("DevTools request:")){
String url = e.getMessage().replaceFirst("DevTools request:", "").trim();
}
if(e.getMessage().contains("DevTools response:")){
String json = e.getMessage().replaceFirst("DevTools response:", "");
try {
if("page".equals(JSONUtil.get(json,"type" ))){
webSocketDebuggerUrl = JSONUtil.get(json,"webSocketDebuggerUrl" );
}
} catch (Exception e1) {
e1.printStackTrace();
}
}
System.out.println(e.getMessage());
}
The JSONUtil i use is my own tool, so don't wonder, just replace with whatever code to extract from the jsontext.

Answer 4

Here's a fairly robust implementation in java using the same target tab with selenium 3.13 & cdp4j 3.0.2-SNAPSHOT. Easily translates to any language.
package com.company;
import com.google.gson.JsonElement;
import com.google.gson.JsonParser;
import io.webfolder.cdp.session.SessionFactory;
import org.openqa.selenium.HasCapabilities;
import org.openqa.selenium.Keys;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeDriverService;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.URL;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.Map;
public class Main {
public static void main(String[] args) {
System.setProperty(ChromeDriverService.CHROME_DRIVER_EXE_PROPERTY, "C:\\path\\to\\chromedriver.exe");
var driver = new ChromeDriver();
try {
var cdp = findCdpEndpoint(driver);
System.out.println(cdp.toString());
try (var factory = new SessionFactory(cdp.getPort())) {
driver.navigate().to("https://google.com");
String seTargetId = getSeTargetId(cdp, driver.getTitle());
try (var session = factory.connect(seTargetId)) {
session.waitDocumentReady();
session.sendKeys("Astronauts");
driver.getKeyboard().sendKeys(Keys.RETURN);
session.wait(2000);
driver.navigate().to("http://www.google.com");
session.waitDocumentReady();
}
}
} catch (Exception ex) {
System.out.println(ex.toString());
}
driver.quit();
}
private static String getSeTargetId(URL cdp, String title) throws IOException {
for (JsonElement element : new JsonParser().parse(new InputStreamReader(cdp.openStream(), "UTF-8")).getAsJsonArray()) {
var object = element.getAsJsonObject();
if (title == null || title.isEmpty()
? object.get("type").getAsString().equalsIgnoreCase("page")
: object.get("title").getAsString().equalsIgnoreCase(title)) {
return object.get("id").getAsString();
}
}
throw new IllegalStateException("Selenium target not found.");
}
private static URL findCdpEndpoint(WebDriver driver) throws IOException {
var capChrome = (Map<?,?>) ((HasCapabilities)driver).getCapabilities().getCapability("chrome");
var userDataDir = (String) capChrome.get("userDataDir");
var port = Integer.parseInt(Files.readAllLines(Paths.get(userDataDir, "DevToolsActivePort")).get(0));
return new URL("http", "localhost", port, "/json");
}
}