Selenium python library via docker, Chrome error failed to start: exited abnormally - google-chrome

I am trying to run some python scripts with the selenium library from within a docker container based on miniconda/anaconda, but I keep getting this error: selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally. I am also using a python wrapper for xvfb to avoid opening a real Chrome window.
To reproduce this (from a running docker container):
root#304ccd3bae83:/opt# python
Python 3.6.4 |Anaconda, Inc.| (default, Jan 16 2018, 18:10:19)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>>
>>> from selenium import webdriver
>>> from xvfbwrapper import Xvfb
>>>
>>> with Xvfb(width=1366, height=768) as xvfb:
... my_driver = webdriver.Chrome('/opt/chromedriver/2.33/chromedriver')
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "/opt/conda/lib/python3.6/site-packages/selenium/webdriver/chrome/webdriver.py", line 69, in __init__
desired_capabilities=desired_capabilities)
File "/opt/conda/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 151, in __init__
self.start_session(desired_capabilities, browser_profile)
File "/opt/conda/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 240, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/opt/conda/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 308, in execute
self.error_handler.check_response(response)
File "/opt/conda/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally
(Driver info: chromedriver=2.33.506092 (733a02544d189eeb751fe0d7ddca79a0ee28cce4),platform=Linux 4.4.0-116-generic x86_64)
According to this: https://sites.google.com/a/chromium.org/chromedriver/help/chrome-doesn-t-start it seems someone may need to use a stand-alone version of Chrome that works for all users, but I am not sure how the docker build works, I guess the docker image is built as root, and all the code inside it is executed as root, so there should not be any issue with different users controlling Chrome.
This python code works fine on a normal Ubuntu laptop with X windows. I need to carefully pick both the versions of Chrome and chromedriver, at the moment when checking from within the running docker container:
root#304ccd3bae83:/opt# /opt/chromedriver/2.33/chromedriver --version
ChromeDriver 2.33.506092 (733a02544d189eeb751fe0d7ddca79a0ee28cce4)
root#304ccd3bae83:/opt# google-chrome-stable --version
Google Chrome 62.0.3202.75

These options helped solving the issue.
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument("--disable-setuid-sandbox")
One of them is needed when seeing Chrome failed to start: crashed.
Also: make sure there are no zombies (from previous executions) for the chrome-driver process using ps aux | grep chrome-driver to find the PIDs to kill.
Bare in mind that if you are using the Python multiprocessing library to spawn many processes involving their own instance of the Chrome browser, then you can not use Docker (which is supposed to start just one Python process, unless using stuff like supervisor), so you may see: selenium.common.exceptions.WebDriverException: Message: chrome not reachable if you try anyway.

Related

pygame py2app Library not loaded: #loader_path/.dylibs/libSDL-1.2.0.dylib

I wouldn't call myself a programmer and am unfamiliar with Mac and Linux, so any links or pointers to additional background would help greatly. I've started with the most basic pygame I could google, (from https://realpython.com/pygame-a-primer/). The test.py and test.app runs on my VM but not on anyone else's real Macs.
test.py file
import pygame
pygame.init()
screen = pygame.display.set_mode((400, 300))
done = False
while not done:
for event in pygame.event.get():
if event.type == pygame.QUIT:
done = True
pygame.display.flip()
setup.py file
from setuptools import setup
APP = ['test.py']
DATA_FILES = []
OPTIONS = {}
setup(
app=APP,
data_files=DATA_FILES,
options={'py2app': OPTIONS},
setup_requires=['py2app'],
)
command to create the test.app
$ python3 setup.py py2app
The test.app runs fine on the VM, but whenever the test.app runs on a real Mac (tested 2, one ran on Catalina ver???) this error message appears:
Traceback (most recent call last):
File "/Applications/test.app/Contents/Resources/__boot__.py", line 101, in <module>
_run()
File "/Applications/test.app/Contents/Resources/__boot__.py", line 84, in _run
exec(compile(source, path, "exec"), globals(), globals())
File "/Applications/test.app/Contents/Resources/test.py", line 1, in <module>
import pygame
File "pygame/__init__.pyc", line 120, in <module>
File "pygame/base.pyc", line 14, in <module>
File "pygame/base.pyc", line 10, in __load
File "imp.pyc", line 343, in load_dynamic
ImportError: dlopen(/Applications/test.app/Contents/Resources/lib/python3.7/lib-dynload/pygame/base.so, 2): Library not loaded: #loader_path/.dylibs/libSDL-1.2.0.dylib
Referenced from: /Applications/test.app/Contents/Resources/lib/python3.7/lib-dynload/pygame/base.so
Reason: image not found
2020-05-02 11:27:50.309 test[931:15836] test Error
----------------------------
Working environment: VirtualBox 6.1, macOS Sierra (Version 10.12) with the following:
$ pip list
Package Version
----------- ----------
altgraph 0.17
certifi 2020.4.5.1
macholib 1.14
modulegraph 0.18
pip 20.1
py2app 0.21
pygame 1.9.6
setuptools 39.0.1
wheel 0.34.2
$ brew list --versions
freetype 2.10.1
gdbm 1.18.1
jpeg 9d
libmikmod 3.3.11.1
libogg 1.3.4
libpng 1.6.37
libtiff 4.1.0
libvorbis 1.3.6
openssl#1.1 1.1.1g
pkg-config 0.29.2_3
python 3.7.7
readline 8.0.4
sdl 1.2.15_1
sdl2 2.0.12_1
sdl_image 1.2.12_7
sdl_mixer 1.2.12_3
sdl_ttf 2.0.11_1
sqlite 3.31.1
webp 1.1.0
xz 5.2.5
Any ideas or pointers would be greatly appreciated. The actual program is an OhHell card game which supports Windows clients (via pyinstaller), but I'm also trying to support a Mac client to play with my friends on a Mac. Similar to the test.py and test.app (ohhell.py and ohhell.app) runs on the VM Mac but same issue/error occurs with real macs. I've just included the simplest code snippet that I was able to google that reproduced the exact error message.
-john
Workaround Solution: Linking to a dynamic library on a Mac with full path and What path does #loader_path resolve to?. Using the '$ install_name_tool -change "#loader_path/.dylibs/libSDL-1.2.0.dylib" "/Applications/test.app/Contents/Frameworks/libSDL-1.2.0.dylib" base.so' resolved the Library not loaded: loader_path/.dylibs/libSDL-1.2.0.dylib errors. I assume this hard codes the path, but with my friends it's fine to tell them to make sure it resides in the /Applications folder.
However a secondary problem came up, 'pygame.error: File is not a Windows BMP file'. I used py2app, so the pygame delete and reinstall threads didn't seem like a feasible solution (pygame unnecesary w/py2app), so switching the images to uncompressed bitmap (256 colors) worked.
For those curious, I did try Pyinstaller for Mac but that presented a whole new set of problems.

Windows 10 Rtree installation successful from .whl file, but error when running code

I am running Python 3.7, 64bit on Windows 10 and trying desperately to get Rtree running. I use the package Rtree-0.9.1-cp37-cp37m-win_amd64.whl from Christoph Gohlke (https://www.lfd.uci.edu/~gohlke/pythonlibs/).
I have tried for very long to get it to work, but keep on getting the following error message when running a script that uses geopandas.
Traceback (most recent call last):
File "C:\Python37\lib\site-packages\rtree\core.py", line 90, in <module>
rt = ctypes.CDLL(os.path.join(here, 'spatialindex_c.dll'))
File "C:\Python37\lib\ctypes\__init__.py", line 364, in __init__
self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] The specified module could not be found
The installation of the whl-package should include the libspatialindex files, but they are not found when running the code. I tried to use Python 2.7 first to run it, then installed Python 3.7. I've checked all the dependencies and checked whether the "spatialindex_c.dll" files are at the right place, but nothing helps. Would be great to get an answer on that.

How do you overcome the "RuntimeError: Gamma ramp size is reported as 0." without changing the window type when using psychopy?

I have developed a program to play videos in psychopy using a machine with Ubuntu 16.04, a NVIDIA GPU and associated driver. The program works perfectly fine on this machine. The program is rather large but of note, it uses visual.Window(fullscr=True) which as default uses the pyglet backend and also uses visual.MovieStim3. I am now trying to run this program on a different machine with Ubuntu 18.04.1 LTS and integrated intel graphics (HD Graphics 620 (Kaby Lake GT2)) and am having problems.
new machine driver info below:
*-display
description: VGA compatible controller
product: Intel Corporation
vendor: Intel Corporation
physical id: 2
bus info: pci#0000:00:02.0
version: 02
width: 64 bits
clock: 33MHz
capabilities: pciexpress msi pm vga_controller bus_master cap_list rom
configuration: driver=i915 latency=0
resources: irq:128 memory:ee000000-eeffffff memory:d0000000-dfffffff ioport:f000(size=64) memory:c0000-dffff
Here is the traceback when I run the program:
Traceback (most recent call last):
File "/home/adf/mxj719/experiments/video_sorting/video_sorting.py", line 456, in <module>
start_sorting(av_original_csv, user, usr_csv, bonus, last_video)
File "/home/adf/mxj719/experiments/video_sorting/video_sorting.py", line 357, in start_sorting
win = visual.Window(fullscr=True)
File "/home/adf/mxj719/.conda/envs/conda_psychopy/lib/python2.7/site-packages/psychopy/visual/window.py", line 375, in __init__
self.backend = backends.getBackend(win=self, *args, **kwargs)
File "/home/adf/mxj719/.conda/envs/conda_psychopy/lib/python2.7/site-packages/psychopy/visual/backends/__init__.py", line 32, in getBackend
return Backend(win, *args, **kwargs)
File "/home/adf/mxj719/.conda/envs/conda_psychopy/lib/python2.7/site-packages/psychopy/visual/backends/pygletbackend.py", line 227, in __init__
self._origGammaRamp = self.getGammaRamp()
File "/home/adf/mxj719/.conda/envs/conda_psychopy/lib/python2.7/site-packages/psychopy/visual/backends/pygletbackend.py", line 326, in getGammaRamp
return getGammaRamp(self.screenID, self.xDisplay)
File "/home/adf/mxj719/.conda/envs/conda_psychopy/lib/python2.7/site-packages/psychopy/visual/backends/gamma.py", line 120, in getGammaRamp
rampSize = getGammaRampSize(screenID, xDisplay=xDisplay)
File "/home/adf/mxj719/.conda/envs/conda_psychopy/lib/python2.7/site-packages/psychopy/visual/backends/gamma.py", line 285, in getGammaRampSize
raise RuntimeError("Gamma ramp size is reported as 0.")
RuntimeError: Gamma ramp size is reported as 0.
Segmentation fault
This is a common problem it seems:
https://discourse.psychopy.org/t/gamma-problem-in-v1-90-2/4549
But each solution involves either switching the visual.Window type to pygame (which is now archaic for psychopy) or glfw (which is not a simple setup and is very new and not well documented). I would like a solution that allows me to continue using pyglet.
Another solution given involves ensuring that xf86-video-intel is installed to give a sensible LUT size, I have tried this and it seems that this driver is installed as default on Ubuntu 18, so this does not work for me.
Someone else mentions that I could comment out the RuntimeError in the source code but I could not find those lines of code where they pointed it out. (/usr/lib/python2.7/dist-packages/psychopy/visual/backends/gamma.py).
Please also note that I have tried both the python2.7 and python3.5 psychopy install instructions with anaconda as outlined here:
http://psychopy.org/installation.html
Another solution given involves ensuring that xf86-video-intel is installed to give a sensible LUT size, I have tried this and it seems that this driver is installed as default on Ubuntu 18, so this does not work for me.
You might need to set the driver in a configuration file. To do this, edit your /etc/X11/xorg.conf.d/20-intel.conf file (or create it if it does not exist) to contain the following lines:
Section "Device"
Identifier "Intel Graphics"
Driver "intel"
EndSection

selenium.common.exceptions.WebDriverException: Message: Service

I had a trouble when i use selenium to control my Chrome.
Here is my code:
from selenium import webdriver
driver = webdriver.Chrome()
When i tried to operate it ,it runs successfully at first,the Chrome pop on the screen. However, it shut down at the few seconds.
Traceback (most recent call last):
File "<pyshell#3>", line 1, in <module>
driver = webdriver.Chrome('C:\Program Files (x86)\Google\Chrome\chrome.exe')
File "C:\Users\35273\AppData\Local\Programs\Python\Python35\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 62, in __init__
self.service.start()
File "C:\Users\35273\AppData\Local\Programs\Python\Python35\lib\site-packages\selenium\webdriver\common\service.py", line 86, in start
self.assert_process_still_running()
File "C:\Users\35273\AppData\Local\Programs\Python\Python35\lib\site-packages\selenium\webdriver\common\service.py", line 99, in assert_process_still_running
% (self.path, return_code)
selenium.common.exceptions.WebDriverException: Message: Service C:\Program Files (x86)\Google\Chrome\chrome.exe unexpectedly exited. Status code was: 0
You need to provide the path of chromedriver...download from http://chromedriver.storage.googleapis.com/index.html?path=2.24/...unzip it and provide path to it in... webdriver.chrome ("path to chromedriver")
I explain the things here:
from selenium import webdriver
driver = webdriver.Chrome("C:\Program Files (x86)\Google\Chrome\Application\chrome.exe")
This is the error if i run the above code:
C:\Python27\python.exe C:/Users/Gaurav.Gaurav-PC/PycharmProjects/Learning/StackOverflow/SeleniumQuestion/test123.py
Traceback (most recent call last):
File "C:/Users/Gaurav.Gaurav-PC/PycharmProjects/Learning/StackOverflow/SeleniumQuestion/test123.py", line 4, in <module>
driver = webdriver.Chrome("C:\Program Files (x86)\Google\Chrome\Application\chrome.exe")
File "C:\Python27\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 62, in __init__
self.service.start()
File "C:\Python27\lib\site-packages\selenium\webdriver\common\service.py", line 86, in start
self.assert_process_still_running()
File "C:\Python27\lib\site-packages\selenium\webdriver\common\service.py", line 99, in assert_process_still_running
% (self.path, return_code)
selenium.common.exceptions.WebDriverException: Message: Service C:\Program Files (x86)\Google
\Chrome\Application\chrome.exe unexpectedly exited. Status code was: 0
Which is same as mentioned by #Weiziyoung in original problem.
The solution is as I mentioned you need to provide the path to chromedriver in place of chrome browser like
driver = webdriver.Chrome("E:\Jars\chromedriver.exe")
It will resolve the problem
Download the appropriate driver:http://chromedriver.storage.googleapis.com/index.html
turn up E:\Python37\Scripts <--Paste the chromedriver.exe file in
i faced the same error (behave.step_registry.AmbiguousStep: #given('Lunch chrome Browser') has already been defined in existing step #given('Lunch chrome Browser') at features/steps/OrangeHRMlogin.py:4). i think it takes previous program where it mentioned "Lunch chrome browser". so i did some changes like in 2nd program i cahnged at (U'chrome browser').change the browser name or the Phase and it runs successfully.
the original chrome browser is opened instead of chrome driver that's the reason of that error download chrome driver and give it's path

Error "cannot find Chrome binary" in windows 8 AMD64 while trying to launch selenium web-driver API

I'm currently working with Chrome extension of selenium web-driver in windows 8 AMD64. I got an issue launching Chrome; It gives me this:
Traceback (most recent call last):
File "D:/java/code/SVN/OMC/trunk/Test/Scripts/PMS\elements\UILoader.py", line 164, in __init__
self.driver = webdriver.Chrome(executable_path="F:\\driver\\chromedriver.exe")
File "C:\Python27\lib\site-packages\selenium-2.42.1-py2.7.egg\selenium\webdriver\chrome\webdriver.py", line 65, in __init__
keep_alive=True)
File "C:\Python27\lib\site-packages\selenium-2.42.1-py2.7.egg\selenium\webdriver\remote\webdriver.py", line 73, in __init__
self.start_session(desired_capabilities, browser_profile)
File "C:\Python27\lib\site-packages\selenium-2.42.1-py2.7.egg\selenium\webdriver\remote\webdriver.py", line 121, in start_session
'desiredCapabilities': desired_capabilities,
File "C:\Python27\lib\site-packages\selenium-2.42.1-py2.7.egg\selenium\webdriver\remote\webdriver.py", line 173, in execute
self.error_handler.check_response(response)
File "C:\Python27\lib\site-packages\selenium-2.42.1-py2.7.egg\selenium\webdriver\remote\errorhandler.py", line 164, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: u'unknown error: cannot find Chrome binary\n (Driver info: chromedriver=2.9.248315,platform=Windows NT 6.2 x86_64)'
I also have put IEDriverServer.exe in the same path and it works fine; this path has been added to the system PATH. I also tried to launch it from another computer with windows 7 AMD64 and it works fine there.
I googled a bit and I saw this; but when I checked the path it specified for chrome.exe, I couldn't find the executable file for chrome. I tried to update Google Chrome, but it gave me error code 1.
I also tried this:
opts = webdriver.ChromeOptions()
opts.binary_location(value = "C:\\WebDrivers\\chromedriver.exe")
driver = webdriver.Chrome(chrome_options = opts)
What should I do?
The last line of the error is the important one:
selenium.common.exceptions.WebDriverException: Message: u'unknown error: cannot find Chrome binary\n (Driver info: chromedriver=2.9.248315,platform=Windows NT 6.2 x86_64)'
The ChromeDriver has been found, and it is reporting that it cannot find the chrome browser executable. Make sure that is installed and on the current path. Some additional information is on the Selenium Wiki.
I solved the problem using ChromeOptions; I addressed the chrome's binary file from ProgramFiles directory. This is the code I used:
opts = webdriver.ChromeOptions()
opts.binary_location(value = "C:\\ProgramFiles\\Google\\Chrome\\chrome.exe")
driver = webdriver.Chrome(chrome_options = opts)
for me it was
opts = webdriver.ChromeOptions()
opts.binary_location = "path/to/Chrome.exe"
driver = webdriver.Chrome(chrome_options=opts)
which fixed the problem
you must set the executable_path where you install the chrome,this is ruby style.python may be the same in my opinion.
Selenium::WebDriver::Chrome::Service.executable_path="D:/Program Files (x86)/Chrome/chrome.exe"