I want to use Chrome browser in headless mode to produce images (PNG, JPEG) out of SVG graphic. Code works in normal interactive mode, but I have problems to use it in headless mode.
My main problem is that headless Chrome exits before drawing of HTML page is completed. As I understand, if I start Chrome with following arguments:
chromium --headless http://myserver.org
It exits together with document.onload event. But at this moment not all data fetched from the server (I using XMLHttpRequest) and therefore drawing is not complete.
I found workaround if I start chrome with debugging port enabled like:
chromium --headless --remote-debugging-port=7777 http://myserver.org
But this is not that I want, especially when I do not have privileges to open http ports on the node. Is there possibility to let Chrome running longer with other flags? I check a lot of them, but did not found appropriate one. Or is there any other methods to postpone exit of the headless Chrome?
You could try this answer https://stackoverflow.com/a/46424041/4830701
Copy pasted here for reference
Use the binary /opt/google/chrome/chrome directly not google-chrome which points to bash script /usr/bin/google-chrome.
Taken from comments in
https://developers.google.com/web/updates/2017/04/headless-chrome#screenshots
Related
I am trying to convert a series of html files to pdfs and then send the generated pdf file. However, if i start headless chrome like
...chrome.exe --headless --disable-gpu --print-to-pdf=file01.pdf source01.html
The chrome process exits immediately even though it could take some time before the pdf is generated. Is there a way or flag to instruct chrome.exe to wait for the pdf to be saved before exiting?
p.s.
Is there also a way to see html/js erros on the page?
p.p.s
I have given the simplest example.
What I actually run has
--run-all-compositor-stages-before-draw
--virtual-time-budget=10000 # Need to render some stuff first
--enable-logging
My operating system is Windows 10 64 bits. I use the latest versions of Firefox and Chrome.
I want to save the pre-master keys in order to use them with WireShark.
For this, I have found many tutorials that all recommend using the SSLKEYLOGFILE environment variable.
However, no matter the path I put in this variable, no file is being created by Firefox (normal and developer editions) or Chrome. This is true even when I restart those browsers or the operating system. This is also true when I clear the cache on the browsers.
I found an alternative way of setting the SSLKEYLOGFILE variable with Chrome by launching the browser with the following argument:
--ssl-key-log-file=PATH
And while this argument does create a log file, it is never populated. It remains completely empty.
What makes this problem even more annoying is that I don't know if there's any debug log that would at least let me know what's happening.
Anyone have ideas on how to solve this issue?
Verified in both Chrome and Firefox
Windows 10 64bit [Version 10.0.17763.379]
I would not use the --ssl-key-log-file flag with Chrome. In my testing, it does not have an effect.
Steps to get SSL keylog file
Change your directory to one that you or your programs have access to. I am using the Desktop folder.
C:\> cd $HOME\Desktop
Set the SSLKEYLOGFILE variable. This sets it for the user (HKCU). To set it for the machine (HKLM), add the /m flag to the end of the command.
PS C:\Users\rj\Desktop> SetX SSLKEYLOGFILE "$(get-location)\ssl.log"
Verify that the variable has been set in a separate powershell window (SetX does not apply to the current window).
PS C:\Users\rj\Desktop> Get-ChildItem ENV: | findstr SSLKEYLOGFILE
SSLKEYLOGFILE C:\Users\rj\Desktop\ssl.log
You can also verify that SSLKEYLOGFILE is a user variable by going to the Control Panel > System and Security > System > Advanced System Settings > Advanced tab > Environment Variables > User Variables. You should see a listing like the powershell example where the value is a directory.
Open Chrome/Firefox and go to an https website like https://stackoverflow.com.
You should see an ssl.log show up on your desktop.
Image of end result
Further Reading
2018-12-07, F5, Decrypting SSL traffic: Great article on doing the above, but cross-platform
2019-03-18, Mozilla, NSS Key Log Format
: The SSLKEYLOGFILE format
2018-08-17, Wireshark Bugzilla, Add TLS 1.3 support: Has good test pcap/SSLKEYLOGFILE contents for TLS1.3 (See Comment 83)
You mentioned that you are using Wireshark. If you using it to export a file from a TLS-encrypted stream in a capture, this article may help you.
One thing I ran into is: Chrome doesn't always fully close when you close the window. Sometimes it stays open in the background. Use the Windows Process Explorer to list all the processes and make cure Chrome is closed. Also, after restarting Chrome, you can double click on it in Process Explorer and select the Environment tab to list all the variables set in it's environment. Make sure SSLKEYLOGFILE is shown in that list. Then Chrome will be sending keys to the specified file.
First off all the problem is that the browsers are protected better and that flaw is patched, thats why you cant capture no ssl. I dont know the details. But chrome with --no-sandbox(linux) option on ,when running will enable you to capture ssl/tls key without any issues from the first try, so you can play with decription of packets. Never tried on windows btw..
This worked for me...
set the SSKEYLOG env: variable as normal in Linux.
just point it to your pefererd save location of the sslkey.log
windows system/advance/env vars/user variables.
add SSLKEYLOGfile with the path to your preferred save location C:/user/sslkey.log
close and your done here.
NSS_ALLOW_SSLKEYLOG=1 is what's missing from the Booleans in Firefox's advanced options.
its been also removed from the general release but is still in the dev ops version. just not listed.
download and install the dev ops version of Firefox
open a terminal start it from its exe if on Linux.
windows just open it from the icon.
go to the about:config
create a new Boolean with NSS_ALLOW_SSLKEYLOG=1 and set it to true.
quit
now create a new symbolic/icon link to the new dev version fire fox version
when you run Firefox from the terminal in Linux the sskeylog should be created and start populating.
run it from the task bar icon in Linux and it doesn't make or update the keylog (just in case you dont want it populating every time you open the browser).
in windows, you will have to disable the env var or it will keep populating every time you open any browser that supports sslkey logging.
on that note opera and vivaldi should work with no changes to the advanced option.
lastly.. open wireshark go to
edit/preferences/protocols/tls
and put the path to the sslkey.log in the pre master secret log box.
click ok.
close everything..
start Wireshark
start your browser (from terminal in linux).
you should start seeing sslkeylog start populating and see extra options for decryption in the tabs of Wireshark.
have fun :)
1st solution:
Restart Chrome
2nd solution:
Use Firefox
I'm trying to run Google Chrome from terminal, and while it's running (my chrome extension will be navigating and extracting data) I want it to be invisible to the user. some kind of headless running.
Xvfb doesn't work in that case (the window is opening and visible to the user)
the google-chrome arg --no-startup-window just starts the process but not the browser.
could you please help?
As far as I know, runnig Chrome headless is impossible. You should run Chrome inside any display. I confused a bit by your statement
Xvfb doesn't work in that case (the window is opening and visible to
the user)
Because we are also using Xvfb for this purpose, and it is NOT visible for user - simply because this display is not known for the user. Maybe you are running Chrome not inside of Xvfb, but inside of user's display? Did you export DISPLAY variable for newly created Xvfb display? This code should work perfectly:
Xvfb :99 -ac -screen 0 1280x1024x16
export DISPLAY=:99
chromium
after that, you will have chromium opened on display :99 (inside Xvbf) and not visible for user.
I know how to enable chrome's crash reporting interactively:
Menu > Settings > Show advanced settings > Tick "Automatically send usage statistics and crash reports to Google"
However, I'm using chrome in an automated testing setup where we set a new --user-data-dir for each run (so there are no persisted settings, we just get the defaults). Chrome is crashing occasionally (running our javascript/html application) and I would like to be able to report these crashes.
I thought I could add --enable-crash-reporter (according to the List of Chromium Command Line Switches), but that doesn't seem to do anything. (We already pass --enable-logging and a couple other flags on the command line.)
I have a test that triggered an "Aw, Snap":
But if I download the user-data-dir for that run and and point chrome at it, and then visit chrome://crashes, it says crash dumps are not enabled.
(The chrome_debug.log does not say anything about the Aw-Snap either.)
We're running chrome under Xvfb on Debian. We're driving chrome via the direct webdriver (from Protractor).
According to the "crashpad with content shell" docs, the steps for using crashpad with content shell (a stripped down version of Chrome):
Create some directory for your crash dumps to go to (the docs suggest tmp/crashes on Linux and Mac, %TEMP%\crashes on Windows, and /data/local/tmp/crashes on Android)
Start Chrome with --enable-crash-reporter --crash-dumps-dir=<your_dump_dir> (looks like you got half of this)
So I'd try adding the --crash-dumps-dir flag.
I carefully revised the list of switches at http://peter.sh/experiments/chromium-command-line-switches/#chrome-frame and I couldn't find anything that would launch Chrome in a hidden background process.
The closest I was able to is --keep-alive-for-test + custom packaged app, but the app fails to execute any passed code because (the way it reports) "no window - ChromeHidden".
TL;DR
google-chrome --headless --remote-debugging-port=9222 http://example.com
You'd also need --disable-gpu temporarily.
Tutorial:
https://developers.google.com/web/updates/2017/04/headless-chrome
There's a work in progress: https://code.google.com/p/chromium/issues/detail?id=546953
The main deliverables are:
A library which headless applications can link to to.
A sample application which demonstrates the use of headless APIs.
So it would be possible to create a simple application that runs in console without connecting to display.
Update Apr 18 '16: The work is mainly done. There's a public forum now:
https://groups.google.com/a/chromium.org/forum/#!forum/headless-dev
Documentation is being in progress:
https://chromium.googlesource.com/chromium/src/+/master/headless/README.md
Update Sep 20 '16: It looks like chrome will eventually get the "--headless" parameter:
https://bugs.chromium.org/p/chromium/issues/detail?id=612904
There was a presentation on BlinkOn 6 (June 16/17, 2016)
Update Nov 29 '16: Design doc for --headless flag: https://docs.google.com/document/d/1aIJUzQr3eougZQp90bp4mqGr5gY6hdUice8UPa-Ys90/edit#heading=h.qxqfzv2lj12s
Update Dec 13 '16: --headless flag is expected to be available in Canary builds soon
Update Mar 12 '17: Chrome 57 has a --headless flag working. Waiting for Selenium and other tools to catch up. User guide: https://chromium.googlesource.com/chromium/src/+/lkgr/headless/README.md
This guy managed to run Chrome headlessly by using Xvfb (X virtual frame buffer) to trick Chrome into thinking it was displaying a window:
http://e-method.blogspot.fr/2010/11/google-chrome-with-xvfb-headless-server.html
If you're on Linux you could try that.
So basically you need to install X virtual frame buffer and Google Chrome via:
root#localhost: ~# apt-get install xvfb imagemagick
root#localhost: ~# apt-get install google-chrome
Then run the browser on the display:
root#localhost: ~# xvfb-run --server-args='-screen 0, 1024x768x24' \
google-chrome -start-maximized http://www.example.com \
> & /dev/null &
root#localhost: ~# DISPLAY=:99 import -window root myimage.png
Or you can look at PhantomJS project which is a headless WebKit implementation.
You could set up a linux VM and use xvfb in it.
Installation on debian / ubuntu:
sudo aptitude install xvfb
Start Chrome headless and visit http://example.com :
xvfb-run --server-args='-screen 0, 1024x768x16' google-chrome
-start-maximized http://example.com > /dev/null &
Turns out it starts in headless mode if you start it as a child subprocess. Besides that:
nircmd.exe can do win hide on chrome based on its PID
Autohotkey_L can also start Chrome hidden without a taskbar button
The Chromium Embedded Framework project seems like it might fit your usecase. I don't have personal experience with the project, but I've heard good things, and it has a solid API that you should be able to exploit for your purposes.
I don't have enough reputation to comment yet, but want to let you guys know that the chrome headless mode which Vanuan mentions actually works with Selenium webdriver.
In Java you can pass the flag to chrome through chromeDriver with the following code:
ChromeOptions options = new ChromeOptions();
options.addArguments("--headless");
ChromeDriver chromeDriver = new ChromeDriver(options);
I've recently found this article which mentions several commandline options that seem to do it. Using these keywords I googled out this piece of code which seem to confirm that these options exist.
// Does not automatically open a browser window on startup (used when
// launching Chrome for the purpose of hosting background apps).
const char kNoStartupWindow[] = "no-startup-window";
// Causes Chrome to launch without opening any windows by default. Useful if
// one wishes to use Chrome as an ash server.
const char kSilentLaunch[] = "silent-launch";
I managed to successfuly run Chrome with --no-startup-window and indeed it launched without any windows. It looked like it launched properly, it spawned all typical children, but the website I tried to make it load inside didn't seem to be actually visited. It maybe possible that this headless mode is only for running apps and not for visiting sites headless*), but it looks very promising as the normal worker tree is set up, just no windows.
The second option --silent-launch made chrome process very silent. I didn't notice any children spawned and the process exited promptly. I doubt it'll be usable for this case.
After I failed my attempts with these options, I focused on less sophisticated ways. On the bottom of the list there are two options:
// Specify the initial window position: --window-position=x,y
const char kWindowPosition[] = "window-position";
// Specify the initial window size: --window-size=w,h
const char kWindowSize[] = "window-size";
I ran Chrome with options to move it completely out of the working area:
--window-size=800,600 --window-position=-800,0
and as dirty as it feels, sure it's no true headless, but still the window is out of my sight, and everything done just with chrome's startup options, without external tools sending low-level window-hide messages.
*) yes, I know try to do weird things. Essentially I tried to get rid of Chrome window that's kept by Karma during tests. I know I can switch to PhantomJS, but I specifically needed to run them in Chrome, and the window popping out was.. eh.. disturbing in the long run.
It is currently under development, you can read more information about it from here:
https://chromium.googlesource.com/chromium/src/+/master/headless/README.md
Headless Chromium is a library for running Chromium in a
headless/server environment. Expected use cases include loading web
pages, extracting metadata (e.g., the DOM) and generating bitmaps from
page contents -- using all the modern web platform features provided
by Chromium and Blink.
It currently works on Linux, there is a nice presentation.
I was also able to make chrome headless work with NightwatchJS. Here is the config that let me use it:
"chromeHeadless": {
"desiredCapabilities": {
"browserName": "chrome",
"chromeOptions": {
"args": ["--headless"],
"binary": "/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary"
}
}
}
Chrome 59 has the ability to create instance as headless.
Find the below tutorial
https://www.automation99.com/2017/07/how-to-use-chrome-headless-using.html?m=1