We have a Docker file that worked as late as 22 December 2020, but all of a sudden it crashes in runtime if we build the same Docker file again and the exception is:
PuppeteerSharp.ProcessException: Failed to launch Base! /app/.local-chromium/Linux-706915/chrome-linux/chrome: error while loading shared libraries: libX11.so.6: cannot open shared object file: No such file or directory
This is the relevant part of the docker file:
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1-buster-slim AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443
#Excluded since it is not relevant
#####################
#PUPPETEER RECIPE
#####################
RUN apt-get update && apt-get install -y \
apt-transport-https \
ca-certificates \
curl \
gnupg \
--no-install-recommends \
&& curl -sSL https://dl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& echo "deb https://dl.google.com/linux/chrome/deb/ stable main" > /etc/apt/sources.list.d/google-chrome.list \
&& apt-get update && apt-get install -y \
google-chrome-beta \
fontconfig \
fonts-ipafont-gothic \
fonts-wqy-zenhei \
fonts-thai-tlwg \
fonts-kacst \
fonts-symbola \
fonts-noto \
fonts-freefont-ttf \
--no-install-recommends \
&& apt-get purge --auto-remove -y curl gnupg \
&& rm -rf /var/lib/apt/lists/*
#####################
#END PUPPETEER RECIPE
#####################
ENV PUPPETEER_EXECUTABLE_PATH "/usr/bin/google-chrome-beta"
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "Our.File.dll"]
I'm by no means experienced Docker/Linux developer, but we have this is production working well for almost a year now.
When searching for the problem we have found many things to try. Among the things I have tried and all failed are these:
https://stackoverflow.com/a/64293743/6743788
Manually adding dependencies (tried it before and after our RUN apt-get above):
RUN apt-get update && apt-get install -y \
gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 \
libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 \
libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 \
libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 \
ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget
This suggestion was first found here: https://medium.com/#ssmak/how-to-fix-puppetteer-error-while-loading-shared-libraries-libx11-xcb-so-1-c1918b75acc3
When watching the build output, we noticed that most of the dependencies already existed with the latest version.
Tried to specify an older version of Chrome (we have tried with a couple of different versions):
#####################
#PUPPETEER RECIPE
#####################
ARG CHROME_VERSION="81.0.4044.138-1"
RUN apt-get update && apt-get -f install && apt-get -y install wget gnupg2 apt-utils
RUN wget --no-verbose -O /tmp/chrome.deb http://dl.google.com/linux/chrome/deb/pool/main/g/google-chrome-stable/google-chrome-stable_${CHROME_VERSION}_amd64.deb \
&& apt-get update \
&& apt-get install -y /tmp/chrome.deb --no-install-recommends --allow-downgrades fonts-ipafont-gothic fonts-wqy-zenhei fonts-thai-tlwg fonts-kacst fonts-freefont-ttf \
&& rm /tmp/chrome.deb
#####################
#END PUPPETEER RECIPE
#####################
Tried 3 plus 2 together
Also tried to add libgbm-dev to the dependency list because we found that somewhere.
I have tried to verify that the files exist in the docker file by running these commands (and their output) in the container:
root#5c47052da1d8:/app# dpkg-query -L libx11-6
/.
/usr
/usr/lib
/usr/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/libX11.so.6.3.0
/usr/share
/usr/share/doc
/usr/share/doc/libx11-6
/usr/share/doc/libx11-6/NEWS.Debian.gz
/usr/share/doc/libx11-6/NEWS.gz
/usr/share/doc/libx11-6/changelog.Debian.gz
/usr/share/doc/libx11-6/changelog.gz
/usr/share/doc/libx11-6/copyright
/usr/share/lintian
/usr/share/lintian/overrides
/usr/share/lintian/overrides/libx11-6
/usr/lib/x86_64-linux-gnu/libX11.so.6
root#5c47052da1d8:/app# ls -la /usr/lib/x86_64-linux-gnu/libX11.so.6
lrwxrwxrwx 1 root root 15 Sep 11 16:16 /usr/lib/x86_64-linux-gnu/libX11.so.6 -> libX11.so.6.3.0
root#5c47052da1d8:/app# ldd libX11.so.6
ldd: ./libX11.so.6: No such file or directory
root#5c47052da1d8:/app# ldd /usr/lib/x86_64-linux-gnu/libX11.so.6
linux-vdso.so.1 (0x00007ffc432b3000)
libxcb.so.1 => /usr/lib/x86_64-linux-gnu/libxcb.so.1 (0x00007fe8b0ad2000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fe8b0acd000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe8b090c000)
libXau.so.6 => /usr/lib/x86_64-linux-gnu/libXau.so.6 (0x00007fe8b0708000)
libXdmcp.so.6 => /usr/lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007fe8b0502000)
/lib64/ld-linux-x86-64.so.2 (0x00007fe8b0c45000)
libbsd.so.0 => /usr/lib/x86_64-linux-gnu/libbsd.so.0 (0x00007fe8b04e8000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fe8b04dc000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fe8b04bb000)
I have read https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md
Any help would be greatly appreciated, because I've got no clue how to solve this or what to try next.
So, I found the problem and document it here if it happens to someone else. It turned out to be how Visual Studio does the building now.
If I right-click and build the Dockerfile in Visual Studio the result will be correct just as my investigations above showed. The problem is that when I wanted to test the image, I run it by F5 (or Ctrl+F5) in VS and in that case Visual Studio does not build my Dockerfile by default. I thought that it used my recently build result (cached), but it actually uses another cached result. For performance reason it builds the project locally and take that result and adds it to the aspnet:3.1-buster-slim image, which means that my custom dependencies are not added.
This behaviour can be controlled by setting in the project file. The default value of if is Fast which does not use my Dockerfile, but setting it to Regular does, on the cost of slower start up.
Documentation of this and other settings can be found here: https://learn.microsoft.com/en-us/visualstudio/containers/container-msbuild-properties?view=vs-2019
Related
I need to test angular dart components in chrome. Test should be executed in gitlab ci job. How can I achive this?
To achive this you can:
Create mew docker image with chrome and dart
Upload this image to gitlab container registry
Use this image in gitlab pipeline job
Here is Docker file:
FROM google/dart:2.5.0
USER root
# Install deps + add Chrome Stable + purge all the things
RUN apt-get update && apt-get install -y \
apt-transport-https \
ca-certificates \
curl \
gnupg \
unzip \
zip \
--no-install-recommends \
&& curl -sSL https://dl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& echo "deb [arch=amd64] https://dl.google.com/linux/chrome/deb/ stable main" > /etc/apt/sources.list.d/google-chrome.list \
&& apt-get update && apt-get install -y \
google-chrome-stable \
--no-install-recommends \
&& apt-get purge --auto-remove -y curl gnupg \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /
RUN mkdir chromedriver && cd chromedriver \
&& wget https://chromedriver.storage.googleapis.com/2.35/chromedriver_linux64.zip \
&& unzip chromedriver_linux64.zip \
&& rm chromedriver_linux64.zip \
&& ln -s /usr/bin/google-chrome-stable /usr/bin/chrome
ENV CHROME_DRIVER_PATH=/chromedriver/chromedriver
And here is job:
build_web:
stage: client_build
image: registry.gitlab.com/your_org/your_proj/image_name
script:
- pub get
- pub run build_runner test --fail-on-severe --define "build_web_compilers|entrypoint=compiler=dart2js" --delete-conflicting-outputs -- -p chrome
- pub run build_runner build --define "build_web_compilers|entrypoint=compiler=dart2js" --delete-conflicting-outputs --output web:build
only:
- master
I'm currently facing an issue in installing Google Chrome in my docker - this set up was working yesterday but as of today I'm getting this error -
This is how I'm installing Chrome
ENV CHROME_VERSION "google-chrome-stable"
RUN apt-get update
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list \
&& apt-get -qqy install \
${CHROME_VERSION:-google-chrome-stable} \
&& rm /etc/apt/sources.list.d/google-chrome.list \
&& rm -rf /var/lib/apt/lists/*
This throws an error
W: Failed to fetch http://deb.debian.org/debian/dists/jessie-updates/main/binary-amd64/Packages 404 Not Found
E: Some index files failed to download. They have been ignored, or old ones used instead.
If I remove the apt-get update part, then the above error doesn't come but the google-chrome-stable is not found
ENV CHROME_VERSION "google-chrome-stable"
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list \
&& apt-get -qqy install \
${CHROME_VERSION:-google-chrome-stable} \
&& rm /etc/apt/sources.list.d/google-chrome.list \
&& rm -rf /var/lib/apt/lists/*
Then the error is
E: Unable to locate package google-chrome-stable
Further , I found a link which recommends removing jessie - https://lists.debian.org/debian-devel-announce/2019/03/msg00006.html
How can I configure to remove both the errors since this was working all fine yesterday and my docker build was successful.
http://deb.debian.org/debian/dists/jessie-updates/main/binary-amd64/Packages delivers a 404 indeed. I don't know why that is but you are not the only person affected: https://github.com/docker-library/official-images/issues/3551
So as a workaround you have to comment out the line containing that URL in the sources.list before running apt-get update to make sure that it doesn't fail. I used sed for that (sed -i -- 's&deb http://deb.debian.org/debian jessie-updates main&#deb http://deb.debian.org/debian jessie-updates main&g').
So I could install chrome successfully by modifying your Dockerfile to look like:
FROM debian:jessie
ENV CHROME_VERSION "google-chrome-stable"
RUN sed -i -- 's&deb http://deb.debian.org/debian jessie-updates main&#deb http://deb.debian.org/debian jessie-updates main&g' /etc/apt/sources.list \
&& apt-get update && apt-get install wget -y
ENV CHROME_VERSION "google-chrome-stable"
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list \
&& apt-get update && apt-get -qqy install ${CHROME_VERSION:-google-chrome-stable}
CMD /bin/bash
Is possible to run puppeteer on my Debian environment ? I don't have the desktop environment installed on it.
When I run my example.js script I have a Error: Failed to launch chrome! libX11-xcb.so.1: cannot open shared object file: No such file or directory
Thx
Ok I found the solution:
Install the Debian required dependencies to use chromium:
apt-get install gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget
And launch the browser without the sandbox:
example.js:
const puppeteer = require('puppeteer');
(async() => {
const browser = await puppeteer.launch({args: ['--no-sandbox']});
const page = await browser.newPage();
await page.goto('https://example.com');
await page.screenshot({path: 'example.png'});
browser.close();
})();
Maybe some useful documentation here:
https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md#running-puppeteer-in-docker
The use of --no-sandbox is described as "strongly discouraged", but there are hints about running Puppeteer in a Docker container which should solve the missing shared libraries issues.
The Dockerfile suggested in the above mentioned link is:
FROM node:12-slim
# Install latest chrome dev package and fonts to support major charsets (Chinese, Japanese, Arabic, Hebrew, Thai and a few others)
# Note: this installs the necessary libs to make the bundled version of Chromium that Puppeteer
# installs, work.
RUN apt-get update \
&& apt-get install -y wget gnupg \
&& wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
&& apt-get update \
&& apt-get install -y google-chrome-stable fonts-ipafont-gothic fonts-wqy-zenhei fonts-thai-tlwg fonts-kacst fonts-freefont-ttf libxss1 \
--no-install-recommends \
&& rm -rf /var/lib/apt/lists/*
# If running Docker >= 1.13.0 use docker run's --init arg to reap zombie processes, otherwise
# uncomment the following lines to have `dumb-init` as PID 1
# ADD https://github.com/Yelp/dumb-init/releases/download/v1.2.2/dumb-init_1.2.2_x86_64 /usr/local/bin/dumb-init
# RUN chmod +x /usr/local/bin/dumb-init
# ENTRYPOINT ["dumb-init", "--"]
# Uncomment to skip the chromium download when installing puppeteer. If you do,
# you'll need to launch puppeteer with:
# browser.launch({executablePath: 'google-chrome-stable'})
# ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD true
# Install puppeteer so it's available in the container.
RUN npm i puppeteer \
# Add user so we don't need --no-sandbox.
# same layer as npm install to keep re-chowned files from using up several hundred MBs more space
&& groupadd -r pptruser && useradd -r -g pptruser -G audio,video pptruser \
&& mkdir -p /home/pptruser/Downloads \
&& chown -R pptruser:pptruser /home/pptruser \
&& chown -R pptruser:pptruser /node_modules
# Run everything after as non-privileged user.
USER pptruser
CMD ["google-chrome-stable"]
Then the suggested Docker build and run commands are:
docker build -t puppeteer-chrome-linux .
docker run -i --init --rm --cap-add=SYS_ADMIN \
--name puppeteer-chrome puppeteer-chrome-linux \
node -e "`cat yourscript.js`"
I tested the process described here above myself , and after that received a no usable sandbox error (therefore the missing shared object error appeared to be solved by it).
Concerning the sandbox related error, suggested resolution from the same source than mentioned here above, is enabling user namespace cloning.
sudo sysctl -w kernel.unprivileged_userns_clone=1
I am trying to install google chrome in docker build with following standard way:
ARG CHROME_VERSION="google-chrome-stable"
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list \
&& apt-get update -qqy \
&& apt-get -qqy install \
${CHROME_VERSION:-google-chrome-stable} \
&& rm /etc/apt/sources.list.d/google-chrome.list \
&& rm -rf /var/lib/apt/lists/* /var/cache/apt/*
But my proxy does not allow google.com so it fails. Is there other way to install google chrome in ubuntu? I don't want to host any file in network so if there is another source (e.g. github) where I can find debian packages for chrome then I can just get that and run that with dpkg. Or, any other idea?
Thanks a lot.
I found a mirror in my network. Replacing source with my mirror, job was done!
I am trying to run tests in a Docker, which currently run on a Jenkins slave, so bare metal. To make it more portable I try to get the Robot Framework running inside it, but so far with no luck.
This is my Dockerfile to create the robot image:
FROM ubuntu:16.04
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update && \
apt-get install -y --no-install-recommends \
dbus \
libgtk2.0-0 \
libgconf-2-4 \
libnss3 \
nginx \
python3 \
python3-pip \
xvfb
# google chrome requirements
RUN apt-get install -y --no-install-recommends \
fonts-liberation \
gconf-service \
libappindicator1 \
lsb-release \
wget \
libasound2 \
ibatk-bridge2.0-0 \
libgtk-3-0 \
libxss1 \
libxtst6 \
xdg-utils
COPY google-chrome-stable_current_amd64.deb /tmp
RUN dpkg --install /tmp/google-chrome-stable_current_amd64.deb && \
apt-get install -f -y && \
rm /tmp/google-chrome-stable_current_amd64.deb
# update pip & install robot framework
RUN pip3 install --upgrade pip \
setuptools && \
pip3 install robotframework \
robotframework-selenium2library \
robotframework-xvfb
COPY chromedriver /usr/local/bin
RUN chmod +x /usr/local/bin/chromedriver
WORKDIR /usr/src/robot
CMD /etc/init.d/nginx start && python3 -m robot --include ready BasicTest.robot
Then I run the test image by calling:
$ docker run -ti --rm -v "$PWD/src/test/robot-framework":/usr/src/robot -v "$PWD/dist":/var/www/html:ro --add-host databasehost:10.10.10.10 robot
src/test/robot-framework contains the BasicTest.robot file and the dist folder is the created dist folder for an Angular-based project I want to test.
The messages I get, after waiting a longer time, look like this:
Correct Login :: *Description:* | FAIL |
Setup failed:
WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally
(Driver info: chromedriver=2.35.528139 (47ead77cb35ad2a9a83248b292151462a66cd881),platform=Linux 4.9.60-linuxkit-aufs x86_64)
I start the virtual display in the BasicTest.robot by:
Start Virtual Display 1024 768
${chrome_options}= Evaluate sys.modules['selenium.webdriver'].ChromeOptions() sys, selenium.webdriver
Call Method ${chrome_options} add_argument headless
Call Method ${chrome_options} add_argument disable-gpu
Call Method ${chrome_options} add_argument no-sandbox
Open Browser #{url} #{browser} --auto-ssl-client-auth
I am not sure what is the purpose of your project, but here is my Dockerfile that uses puppeteer that is a Google Lib that wraps chrome_headless and allow you to use it more easily.
Here you can find an official example and here is the official documentation of puppeteer with great community suppport.
This is my Docker file that runs a npm script after building a container with puppeteer, node and chromium dependencies.
# Most part taken from https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md
FROM node:8-slim
# # Manually install missing shared libs for Chromium.
RUN apt-get update && \
apt-get install -yq gconf-service libasound2 libatk1.0-0 libc6
libcairo2 libcups2 libdbus-1-3 \
libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0
libglib2.0-0 libgtk-3-0 libnspr4 \
libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1
libxcb1 libxcomposite1 \
libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2
libxrender1 libxss1 libxtst6 \
ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release
xdg-utils wget
COPY . /app/
WORKDIR app
# Install deps for server.
RUN npm install -g
# Install puppeteer so it can be required by user code that gets run in
# server.js. Cache bust so we always get the latest version of puppeteer when
# building the image.
ARG CACHEBUST=1
RUN npm install puppeteer#0.13.0
# Add pptr user.
RUN groupadd -r pptruser && useradd -r -g pptruser -G audio,video pptruser \
&& mkdir -p /home/pptruser/Downloads \
&& chown -R pptruser:pptruser /home/pptruser \
&& chown -R pptruser:pptruser /app
# Run user as non privileged.
USER pptruser
EXPOSE 9222
CMD ["npm", "start"]
Inside that npm start script you can run your tests I guess.
The trick about using puppeteer inside a Docker container is that Docker does not install chromium dependencies automatically and then it fails when you try to use it.
Despite the fact of using puppeteer or not, that container will allow you to use chrome_headless inside and in my opinion is less heavy than the other examples that I found inside the documentation.
For running just type
docker run -d -i --rm --cap-add=SYS_ADMIN --network=${yourNetwork} --name ${image_name} ${container_name}
Hope that helps