How to download a file with Selenium and ChromeDriver - google-chrome

I have a website to test with selenium and ChromeDriver (on windows), where I like to test the functionality to export data and import it again.
The export creates a xml file that is downloaded on ones computer. When running this with webdriver, Chrome asks me whether to keep the file or discard it, as it might be a potential threat.
How can I switch off this behavior inside my test ? Is there a chrome setting I can use, so that a file is no matter what downloaded ?
Thanks

Try this. Executed on windows
(How to control the download of files with Selenium Python bindings in Chrome)
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_experimental_option("prefs", {
"download.default_directory": r"C:\Users\xxx\downloads\Test",
"download.prompt_for_download": False,
"download.directory_upgrade": True,
"safebrowsing.enabled": True
})

The below Program will help you to download the files in Chrome with
Desired-Capabilities. It is a rich class having lot of utilities, you can go through it in your free time.
public class DownloadChromeFile {
public static void main(String[] args) {
System.setProperty("webdriver.chrome.driver","./chromedriver.exe");
String downloadFilepath = "c:\\download";
HashMap<String, Object> chromePrefs = new HashMap<String, Object>();
chromePrefs.put("profile.default_content_settings.popups", 0);
chromePrefs.put("download.default_directory", downloadFilepath);
ChromeOptions options = new ChromeOptions();
HashMap<String, Object> chromeOptionsMap = new HashMap<String, Object>();
options.setExperimentalOption("prefs", chromePrefs);
options.addArguments("--test-type");
options.addArguments("--disable-extensions"); //to disable browser extension popup
DesiredCapabilities cap = DesiredCapabilities.chrome();
cap.setCapability(ChromeOptions.CAPABILITY, chromeOptionsMap);
cap.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true); // Bydefault it will accepts all popups.
cap.setCapability(ChromeOptions.CAPABILITY, options);
driver = new ChromeDriver(cap);
driver.get("Your Application Url");
driver.findElement(By.xpath("Export Button xpath")).click();
}
}

Related

Tests slow because High cpu using Headless chrome with version 87

When I run tests without using headless chrome, the tests take on average 40-50 seconds. When I run the tests using headless chrome it takes a lot longer (190 seconds on average). I'm using chromedrivermanager which runs on 87.0.4280.20.
The browser class without using headless:
private String baseUrl = ConfigHandler.getPropertyValue("url");
private WebDriver driver;
public Browser() {
WebDriverManager.chromedriver().setup();
Map<String, Object> prefs = new HashMap<>();
ChromeOptions chromeOptions = new ChromeOptions();
String FilesPath = System.getProperty("user.dir") + File.separator + SeleniumUtilities.getDownloadsPath();
prefs.put("download.default_directory", FilesPath);
chromeOptions.setExperimentalOption("prefs", prefs);
this.driver = new ChromeDriver(chromeOptions);
this.driver.manage().window().maximize();
}
Browser class using headless chrome:
private String baseUrl = ConfigHandler.getPropertyValue("url");
private WebDriver driver;
public Browser() {
WebDriverManager.chromedriver().setup();
Map<String, Object> prefs = new HashMap<>();
ChromeOptions chromeOptions = new ChromeOptions();
System.out.println("working on server");
chromeOptions.addArguments("--window-size=1400,900");
chromeOptions.addArguments("--headless");
chromeOptions.addArguments("--no-proxy-server");
chromeOptions.addArguments("--proxy-server='direct://'");
chromeOptions.addArguments("--proxy-bypass-list=*");
String FilesPath = System.getProperty("user.dir") + File.separator + SeleniumUtilities.getDownloadsPath();
prefs.put("download.default_directory", FilesPath);
chromeOptions.setExperimentalOption("prefs", prefs);
this.driver = new ChromeDriver(chromeOptions);
}
Any Ideas?
EDIT:
I noticed in headless chrome the cpu get really high, unlike without using headless. Why the cpu so high when running in headless?

Multi-client remote debugging using ChromeDriver and Chrome DevTools protocol

So with Chrome 63 there is now support for Multi-client remote debugging (https://developers.google.com/web/updates/2017/10/devtools-release-notes)
What I want to achieve is use the Chrome DevTools Protocol HeapProfiler with some selenium tests. I'm running version 64 Chrome dev channel and ChromeDriver 2.33.
ChromeOptions options = newChromeOptions();
options.addArguments("--remote-debugging-port=9222");
WebDriver driver = new ChromeDriver(options);
... selenium stuff
A new chrome window will open and hang until it times out. I can confirm that the chrome window opened is chrome 64 by going to help > about google chrome to check the version.
I get this error which appears to be the the webdriver losing connection.
Exception in thread "main" org.openqa.selenium.WebDriverException: chrome not
reachable
The DevTools Protocol is working because I am able to open http://localhost:9222 in another chrome window and see debugging interface.
Has anyone been able to get these two things to work together?
Thanks :)
Here the catch was that if you pass the "remote-debugging-port" switch then chromedriver has a bug where it still internally assigns a randon port and keep trying to connect to it rather than connecting to 9222 port.
options.addArguments("--remote-debugging-port=9222");
We can solve this by skipping this command switch and let chrome decides this random port and extract this port number from chromedriver logs.
I made it work and here I have blogged it in detail.
https://medium.com/#sahajamit/selenium-chrome-dev-tools-makes-a-perfect-browser-automation-recipe-c35c7f6a2360
Selenium 4 release will have a user friendly API for Chrome DevTools protocol. I just finished implementing Network and Performance domains for the Selenium Java client. https://github.com/SeleniumHQ/selenium/pull/7212
In addition, there is a generic API for all domains in Java client that was merged a while ago. All those new features will be released probably in the next Alpha release.
This is a nice article on how to use Log:
https://codoid.com/selenium-4-chrome-devtools-log-entry-listeners/
Here is what i do to get the information needed fro remotedebugging nd additionally to prevent defining the port. I get it through the SeleniumLog-API
DesiredCapabilities capabilities = DesiredCapabilities.chrome();
ChromeOptions options = new ChromeOptions();
options.setBinary(chromeBin);
capabilities.setCapability(ChromeOptions.CAPABILITY, options);
LoggingPreferences logPref = new LoggingPreferences();
logPref.enable(LogType.DRIVER, Level.ALL);
driverInstance = new ChromeDriver(capabilities);
LogEntries x = driverInstance.manage().logs().get(LogType.DRIVER);
for(LogEntry e:x.getAll()){
if(e.getMessage().contains("DevTools request:")){
String url = e.getMessage().replaceFirst("DevTools request:", "").trim();
}
if(e.getMessage().contains("DevTools response:")){
String json = e.getMessage().replaceFirst("DevTools response:", "");
try {
if("page".equals(JSONUtil.get(json,"type" ))){
webSocketDebuggerUrl = JSONUtil.get(json,"webSocketDebuggerUrl" );
}
} catch (Exception e1) {
e1.printStackTrace();
}
}
System.out.println(e.getMessage());
}
The JSONUtil i use is my own tool, so don't wonder, just replace with whatever code to extract from the jsontext.
Here's a fairly robust implementation in java using the same target tab with selenium 3.13 & cdp4j 3.0.2-SNAPSHOT. Easily translates to any language.
package com.company;
import com.google.gson.JsonElement;
import com.google.gson.JsonParser;
import io.webfolder.cdp.session.SessionFactory;
import org.openqa.selenium.HasCapabilities;
import org.openqa.selenium.Keys;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeDriverService;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.URL;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.Map;
public class Main {
public static void main(String[] args) {
System.setProperty(ChromeDriverService.CHROME_DRIVER_EXE_PROPERTY, "C:\\path\\to\\chromedriver.exe");
var driver = new ChromeDriver();
try {
var cdp = findCdpEndpoint(driver);
System.out.println(cdp.toString());
try (var factory = new SessionFactory(cdp.getPort())) {
driver.navigate().to("https://google.com");
String seTargetId = getSeTargetId(cdp, driver.getTitle());
try (var session = factory.connect(seTargetId)) {
session.waitDocumentReady();
session.sendKeys("Astronauts");
driver.getKeyboard().sendKeys(Keys.RETURN);
session.wait(2000);
driver.navigate().to("http://www.google.com");
session.waitDocumentReady();
}
}
} catch (Exception ex) {
System.out.println(ex.toString());
}
driver.quit();
}
private static String getSeTargetId(URL cdp, String title) throws IOException {
for (JsonElement element : new JsonParser().parse(new InputStreamReader(cdp.openStream(), "UTF-8")).getAsJsonArray()) {
var object = element.getAsJsonObject();
if (title == null || title.isEmpty()
? object.get("type").getAsString().equalsIgnoreCase("page")
: object.get("title").getAsString().equalsIgnoreCase(title)) {
return object.get("id").getAsString();
}
}
throw new IllegalStateException("Selenium target not found.");
}
private static URL findCdpEndpoint(WebDriver driver) throws IOException {
var capChrome = (Map<?,?>) ((HasCapabilities)driver).getCapabilities().getCapability("chrome");
var userDataDir = (String) capChrome.get("userDataDir");
var port = Integer.parseInt(Files.readAllLines(Paths.get(userDataDir, "DevToolsActivePort")).get(0));
return new URL("http", "localhost", port, "/json");
}
}

Setting device name in serenity.properties file for chrome driver

How do I set the mobile emulation for Nexus 5 view in Serenity managed chrome driver?
I tried going through this link:
https://johnfergusonsmart.com/configuring-chromedriver-easily-with-serenity-bdd/
Which explain setting preferences for chrome.
Chrome preferences
You can also provide more advanced options using the setExperimentalOption() method:
Map<String, Object> chromePrefs = new HashMap<String, Object>();
chromePrefs.put("download.default_directory", downLoadDirectory);
chromePrefs.put("profile.default_content_settings.popups", 0);
chromePrefs.put("pdfjs.disabled", true);
ChromeOptions options = new ChromeOptions();
options.setExperimentalOption("prefs", chromePrefs);
In Serenity, you would pass these using properties prefixed with the chrome_preferences prefix, e.g.
chrome_preferences.download.default_directory = /my/download/directory
chrome_preferences.profile_default_content_settings.popups = 0
chrome_preferences.pdfjs.disabled=true
From this, I tried setting the mobileEmulation as
chrome.capabilities.mobile_emulation.device_name= Google Nexus 5
chrome.options.mobileEmulation.deviceName= Google Nexus 5
and a few other logical variants, but none of them succeeded.
The best way I found to help me in this issue it to create a custom WebDriver.
I had to create a class which extends DriverSource. And then link it to the Serenity Properties file. This will give me the driver I need.
http://www.thucydides.info/docs/serenity/#_custom_webdriver_implementations
You can add your own custom WebDriver provider by implementing the
DriverSource interface. First, you need to set up the following system
properties (e.g. in your serenity.properties file):
webdriver.driver = provided
webdriver.provided.type = mydriver
webdriver.provided.mydriver = com.acme.MyPhantomJSDriver
thucydides.driver.capabilities = mydriver
Your custom driver must implement the DriverSource interface, as shown
here:
public class MyPhantomJSDriver implements DriverSource {
#Override
public WebDriver newDriver() {
try {
DesiredCapabilities capabilities = DesiredCapabilities.phantomjs();
// Add
return new PhantomJSDriver(ResolvingPhantomJSDriverService.createDefaultService(),
capabilities);
}
catch (IOException e) {
throw new Error(e);
}
}
#Override
public boolean takesScreenshots() {
return true;
}
}
This driver will now take screenshots normally.

Export HAR using chromedriver

Is it possible to export HAR using chromedriver similar to what I can do with netexpert+firebug with Firefox?
Yes, using BrowsermobProxy you can generate HAR file using chromedriver.
Here is a script in python to programatically generate HAR file using Selenium, BrowserMob Proxy and chromedriver. Python Packages for selenium and browsermob-proxy are needed to run this script.
from browsermobproxy import Server
from selenium import webdriver
import os
import json
import urlparse
server = Server("path/to/browsermob-proxy")
server.start()
proxy = server.create_proxy()
chromedriver = "path/to/chromedriver"
os.environ["webdriver.chrome.driver"] = chromedriver
url = urlparse.urlparse (proxy.proxy).path
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--proxy-server={0}".format(url))
driver = webdriver.Chrome(chromedriver,chrome_options =chrome_options)
proxy.new_har("http://stackoverflow.com", options={'captureHeaders': True})
driver.get("http://stackoverflow.com")
result = json.dumps(proxy.har, ensure_ascii=False)
print result
proxy.stop()
driver.quit()
You can enable performance log via chromedriver and analyze the network traffic to build HAR on your own.
Please checkout the code at
https://gist.github.com/Ankit3794/01b63199bd7ed4f2539a088463e54615#gistcomment-3126071
Steps:
Initiate ChromeDriver instance with enabling Logging Preference
DesiredCapabilities capabilities = DesiredCapabilities.chrome();
ChromeOptions chromeOptions = new ChromeOptions();
chromeOptions.addArguments("ignore-certificate-errors");
chromeOptions.addArguments("disable-infobars");
chromeOptions.addArguments("start-maximized");
// More Performance Traces like devtools.timeline, enableNetwork and enablePage
Map<String, Object> perfLogPrefs = new HashMap<>();
perfLogPrefs.put("traceCategories", "browser,devtools.timeline,devtools");
perfLogPrefs.put("enableNetwork", true);
perfLogPrefs.put("enablePage", true);
chromeOptions.setExperimentalOption("perfLoggingPrefs", perfLogPrefs);
// For Enabling performance Logs for WebPageTest
LoggingPreferences logPrefs = new LoggingPreferences();
logPrefs.enable(LogType.PERFORMANCE, Level.ALL);
capabilities.setCapability("goog:loggingPrefs", logPrefs);
capabilities.merge(chromeOptions);
Get "message" JSONObject from Performance Logs
private static JSONArray getPerfEntryLogs(WebDriver driver) {
LogEntries logEntries = driver.manage().logs().get(LogType.PERFORMANCE);
JSONArray perfJsonArray = new JSONArray();
logEntries.forEach(entry -> {
JSONObject messageJSON = new JSONObject(entry.getMessage()).getJSONObject("message");
perfJsonArray.put(messageJSON);
});
return perfJsonArray;
}
Get HAR by passing PerfLogs
public static void getHAR(WebDriver driver, String fileName) throws IOException {
String destinationFile = "/HARs/" + fileName + ".har";
((JavascriptExecutor) driver).executeScript(
"!function(e,o){e.src=\"https://cdn.jsdelivr.net/gh/Ankit3794/chrome_har_js#master/chromePerfLogsHAR.js\",e.onload=function(){jQuery.noConflict(),console.log(\"jQuery injected\")},document.head.appendChild(e)}(document.createElement(\"script\"));");
File file = new File(destinationFile);
file.getParentFile().mkdirs();
FileWriter harFile = new FileWriter(file);
harFile.write((String) ((JavascriptExecutor) driver).executeScript(
"return module.getHarFromMessages(arguments[0])", getPerfEntryLogs(driver).toString()));
harFile.close();
}

webengine crash on Javafx 2.2.25

I am finding a issue when loading a html file in javafx using webEngine.
But in my case if i use java update 9 everything works fine but when i use any update after that like java update 25 with javafx version 2.2.25 ... the app crashes with EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x000000006e103b24, pid=7644, tid=8180.
here is what i am trying to do:
WebView webView = new WebView();
webEngine = webView.getEngine();
JSObject window = (JSObject) webEngine.executeScript("window"); // Crashes at this point but everything works fine with older version of java fx.
window.setMember("javafx", baseui);
scene = new Scene(webView, 1000, 800);
primaryStage = stage;
stage.setScene(scene);
stage.show();
stage.setIconified(true);
webEngine.getLoadWorker().stateProperty().addListener(
new ChangeListener<State>(){
#Override
public void changed(ObservableValue<? extends State> ov, State oldState, State newState) {
if(newState == State.SUCCEEDED){
JSObject window = (JSObject)webEngine.executeScript("window");
window.setMember("javafx", baseui);
}
}
});
URL urlHello = getClass().getResource("*.html"); // My HTML file
webEngine.load(urlHello.toExternalForm());
I tried solutions in this post like Platform.RunLater but of no help.
Is this bug fixed?? Any help is appreciated.
Thanks,
nitin