Use Dependabot to check Dependencies in Bash scripts and Dockerfiles - dependabot

I would like some notifications from Dependabot when new versions are available. These are my two scenarios:
#1 I'm starting a docker container in a bash script like this.
local DOCKER_IMAGE="hashcorp/terraform:1.3.6"
docker run --rm \
--volume /etc/passwd:/etc/passwd:ro \
--volume /etc/group:/etc/group:ro \
--user "$(id -u):$(id -g)" \
--volume /etc/timezone:/etc/timezone:ro \
--volume /etc/localtime:/etc/localtime:ro \
--volume "$(pwd):$(pwd)" \
--workdir "$(pwd)" \
--env "GITHUB_TOKEN=$TOKEN" \
--env "TF_VAR_bw_client_id=$BW_CLIENT_ID" \
--env "TF_VAR_bw_client_secret=$BW_CLIENT_SECRET" \
--env "TF_VAR_bw_password=$BW_MASTER_PASS" \
"$DOCKER_IMAGE" "$#"
For other Dependencies like Docker Images I use Dependabot to raise a PR when I should update my base image. Is there a way to have Dependabot inspect bash scripts like the above snippet as well? I'd like a notification when there is a new terraform version.
#2 I'm installing Bitwarden CLI into a custom terraform image (needed for the Bitwarden Provider. This works fine. But since I pin the Bitwarden CLI version in my Dockerfile I would like Dependabot to raise PR for the Bitwarden CLI as well.
FROM hashicorp/terraform:1.3.6
LABEL maintainer="sebastian#sommerfeld.io"
# Install basics
RUN apk update \
&& apk --no-cache add curl=7.86.0-r1 \
&& apk --no-cache add unzip=6.0-r13
# Install Bitwarden CLI + dependencies
ARG BW_VERSION="2022.11.0"
RUN apk --no-cache add libc6-compat=1.2.3-r4 \
&& apk --no-cache add gcompat=1.1.0-r0 \
&& apk --no-cache add libgcc=12.2.1_git20220924-r4 \
&& apk --no-cache add libstdc++=12.2.1_git20220924-r4 \
&& rm -rf /var/cache/apk/* \
&& curl -sL https://github.com/bitwarden/clients/releases/download/cli-v${BW_VERSION}/bw-linux-${BW_VERSION}.zip -o bw.zip \
&& unzip bw.zip \
&& chmod +rx bw \
&& rm bw.zip \
&& mv bw /usr/local/bin
Is it possible to use Dependabot for these scenarios as well?

Related

Docker mysql via MariaDB with Supervisor

Somewhat a year ago, I came up with the idea of extending my Docker knowledge to begin with creating a sort of multi-platform server image for development purposes, since then, I figured out how to get Nginx and PHP-fpm running in a stable environment. This all is based on a Debian image. Now since a couple one week ago, I wanted to add MySQL functionality to the image. At first, I tried the normal MySQL(-server) image and after trying to fix errors about why it couldn't run in my image, I switched to using MariaDB - I even had changed the Docker image of MySQL to fit to my needs (Replaced CMD ["mysqld"] for a supervisord.conf executable since my project is using several services of course). Now, I'm trying to figure it out for days but it is still not running. At the moment, I've chosen to use https://hub.docker.com/_/mariadb (second: 10.4.12-bionic, 10.4-bionic, 10-bionic, bionic, 10.4.12, 10.4, 10, latest) with my image.
I've just created a mariadb copy on time of writing, but replaced directly executing mysqld (working). When this topic is created, it didn't worked with a supervisord and that works as supposed to be now.
I have a docker-compose.yml where it will be started, here the code:
version: "3"
services:
db:
container_name: mariadb
image: mariadb
build: .
restart: on-failure
ports:
- 3306:3306
environment:
- MYSQL_ROOT_PASSWORD=test123
networks:
- local-network
networks:
local-network:
driver: bridge
Then, I will execute docker-compose up -d or with the (--build) parameter.
The Dockerfile behind that is:
FROM debian:buster-slim
ENV DEBIAN_FRONTEND noninteractive
ENV GOSU_VERSION 1.12
ENV MARIADB_VERSION 10.4
ENV GPG_KEYS \
199369E5404BD5FC7D2FE43BCBCB082A1BB943DB \
177F4010FE56CA3336300305F1656F24C74CD1D8
# add our user and group first to make sure their IDs get assigned consistently, regardless of whatever dependencies get added
RUN groupadd -r mysql && useradd -r -g mysql mysql
RUN apt-get update && apt-get install --no-install-recommends --no-install-suggests -q -y \
wget \
ca-certificates \
gnupg \
gnupg1 \
gnupg2 \
dirmngr \
pwgen \
tzdata \
xz-utils
# Get Gosu for easy stepdown from root (to avoid sudo/su miscommunications)
# https://github.com/tianon/gosu/releases
RUN set -eux; \
savedAptMark="$(apt-mark showmanual)"; \
dpkgArch="$(dpkg --print-architecture | awk -F- '{ print $NF }')"; \
wget -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$dpkgArch"; \
wget -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$dpkgArch.asc"; \
export GNUPGHOME="$(mktemp -d)"; \
gpg --batch --keyserver hkps://keys.openpgp.org --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4; \
gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu; \
gpgconf --kill all; \
rm -rf "$GNUPGHOME" /usr/local/bin/gosu.asc; \
apt-mark auto '.*' > /dev/null; \
[ -z "$savedAptMark" ] || apt-mark manual $savedAptMark > /dev/null; \
apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false; \
chmod +x /usr/local/bin/gosu; \
gosu --version; \
gosu nobody true
RUN mkdir /docker-entrypoint-initdb.d
RUN set -ex; \
export GNUPGHOME="$(mktemp -d)"; \
for key in $GPG_KEYS; do \
gpg --batch --keyserver ha.pool.sks-keyservers.net --recv-keys "$key"; \
done; \
gpg --batch --export $GPG_KEYS > /etc/apt/trusted.gpg.d/mariadb.gpg; \
command -v gpgconf > /dev/null && gpgconf --kill all || :; \
rm -r "$GNUPGHOME"; \
apt-key list
# Add MariaDB repo
RUN set -e;\
echo "deb http://downloads.mariadb.com/MariaDB/mariadb-$MARIADB_VERSION/repo/debian buster main" > /etc/apt/sources.list.d/mariadb.list; \
{ \
echo 'Package: *'; \
echo 'Pin: release o=MariaDB'; \
echo 'Pin-Priority: 999'; \
} > /etc/apt/preferences.d/mariadb
# Install MariaDB and set custom requirements
RUN set -ex; \
{ \
echo "mariadb-server" mysql-server/root_password password 'unused'; \
echo "mariadb-server" mysql-server/root_password_again password 'unused'; \
} | debconf-set-selections; \
apt-get update && apt-get install --no-install-recommends --no-install-suggests -y -q \
mariadb-server \
mariadb-backup \
socat; \
# comment out any "user" entires in the MySQL config ("docker-entrypoint.sh" or "--user" will handle user switching)
sed -ri 's/^user\s/#&/' /etc/mysql/my.cnf /etc/mysql/conf.d/*; \
# making sure that the correct permissions are set
mkdir -p /var/lib/mysql /var/run/mysqld; \
chown -R mysql:mysql /var/lib/mysql /var/run/mysqld; \
# comment out a few problematic configuration values
find /etc/mysql/ -name '*.cnf' -print0 \
| xargs -0 grep -lZE '^(bind-address|log)' \
| xargs -rt -0 sed -Ei 's/^(bind-address|log)/#&/'; \
# don't reverse lookup hostnames, they are usually another container
echo '[mysqld]\nskip-host-cache\nskip-name-resolve' > /etc/mysql/conf.d/docker.cnf
# Setup the Supervisor
RUN apt-get update && apt-get install supervisor -y \
&& mkdir -p /var/log/supervisor
COPY /supervisord.conf /etc/supervisor/conf.d/supervisord.conf
RUN chmod +x /etc/supervisor/conf.d/supervisord.conf
VOLUME /var/lib/mysql
COPY /docker-entrypoint.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/docker-entrypoint.sh \
&& ln -s /usr/local/bin/docker-entrypoint.sh /
ENTRYPOINT ["docker-entrypoint.sh"]
EXPOSE 3306 33060
# call and execute the supervisor after build
CMD ["/usr/bin/supervisord", "-c", "/etc/supervisor/conf.d/supervisord.conf"]
After a couple days of working on fixing the image I thought that the supervisord was the issue, it couldn't run because of that or something. Well, here is the supervisord:
[supervisord]
logfile=/var/log/supervisord.log
nodaemon=true
user=root
[program:mysql]
command=/usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql
process_name=mysqld
priority=1
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stdout_events_enabled=true
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
stderr_events_enabled=true
autorestart=true
user=mysql
What happens next when the image has been build is that mysql will be executed by the supervisor. But, the problem is that I wanted to use the entrypoint from https://github.com/mariadb-corporation/mariadb-server-docker/tree/master/10.4 - I'm not very well known in Bash, so it will take some time to practice things there. Anyway, the docker-entrypoint has not been executed the first time, the database will not be initialized. What I can do, is creating an own shell script to initialize it. Tested that and it worked, but why can't I just use the default entrypoint as the first choise?
Is it going wrong at some point between Supervisord commands - docker-entrypoint with mysql connection points or something?
I really hope that someone can help me out.
Edit [04/26/2020]: Described the latest situation from now on, database not initializing, no message, notes or warnings from the entrypoint script.
Regards,
Colin
The MySQL service should run as root user, but later that's the mysql user whiche tries to access to the "socket". So, the socket directory should be accessible by mysql user but Superviser runs the mysql service as root user.
I fixed this issue by creating and gave right permission to the MySQL socket directory in my Dockerfile:
ARG MARIADB_MYSQL_SOCKET_DIRECTORY='/var/run/mysqld'
RUN mkdir -p $MARIADB_MYSQL_SOCKET_DIRECTORY && \
chown root:mysql $MARIADB_MYSQL_SOCKET_DIRECTORY && \
chmod 774 $MARIADB_MYSQL_SOCKET_DIRECTORY
then configured the Supervisor like this:
[program:mariadb]
command=/usr/sbin/mysqld
autorestart=true
user=root

How to execute angular dart test in chrome in gitlab-ci

I need to test angular dart components in chrome. Test should be executed in gitlab ci job. How can I achive this?
To achive this you can:
Create mew docker image with chrome and dart
Upload this image to gitlab container registry
Use this image in gitlab pipeline job
Here is Docker file:
FROM google/dart:2.5.0
USER root
# Install deps + add Chrome Stable + purge all the things
RUN apt-get update && apt-get install -y \
apt-transport-https \
ca-certificates \
curl \
gnupg \
unzip \
zip \
--no-install-recommends \
&& curl -sSL https://dl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& echo "deb [arch=amd64] https://dl.google.com/linux/chrome/deb/ stable main" > /etc/apt/sources.list.d/google-chrome.list \
&& apt-get update && apt-get install -y \
google-chrome-stable \
--no-install-recommends \
&& apt-get purge --auto-remove -y curl gnupg \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /
RUN mkdir chromedriver && cd chromedriver \
&& wget https://chromedriver.storage.googleapis.com/2.35/chromedriver_linux64.zip \
&& unzip chromedriver_linux64.zip \
&& rm chromedriver_linux64.zip \
&& ln -s /usr/bin/google-chrome-stable /usr/bin/chrome
ENV CHROME_DRIVER_PATH=/chromedriver/chromedriver
And here is job:
build_web:
stage: client_build
image: registry.gitlab.com/your_org/your_proj/image_name
script:
- pub get
- pub run build_runner test --fail-on-severe --define "build_web_compilers|entrypoint=compiler=dart2js" --delete-conflicting-outputs -- -p chrome
- pub run build_runner build --define "build_web_compilers|entrypoint=compiler=dart2js" --delete-conflicting-outputs --output web:build
only:
- master

installing chrome in docker file

I am trying to install google chrome in docker build with following standard way:
ARG CHROME_VERSION="google-chrome-stable"
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list \
&& apt-get update -qqy \
&& apt-get -qqy install \
${CHROME_VERSION:-google-chrome-stable} \
&& rm /etc/apt/sources.list.d/google-chrome.list \
&& rm -rf /var/lib/apt/lists/* /var/cache/apt/*
But my proxy does not allow google.com so it fails. Is there other way to install google chrome in ubuntu? I don't want to host any file in network so if there is another source (e.g. github) where I can find debian packages for chrome then I can just get that and run that with dpkg. Or, any other idea?
Thanks a lot.
I found a mirror in my network. Replacing source with my mirror, job was done!

How do I resolve WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally in Docker?

I am trying to run tests in a Docker, which currently run on a Jenkins slave, so bare metal. To make it more portable I try to get the Robot Framework running inside it, but so far with no luck.
This is my Dockerfile to create the robot image:
FROM ubuntu:16.04
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update && \
apt-get install -y --no-install-recommends \
dbus \
libgtk2.0-0 \
libgconf-2-4 \
libnss3 \
nginx \
python3 \
python3-pip \
xvfb
# google chrome requirements
RUN apt-get install -y --no-install-recommends \
fonts-liberation \
gconf-service \
libappindicator1 \
lsb-release \
wget \
libasound2 \
ibatk-bridge2.0-0 \
libgtk-3-0 \
libxss1 \
libxtst6 \
xdg-utils
COPY google-chrome-stable_current_amd64.deb /tmp
RUN dpkg --install /tmp/google-chrome-stable_current_amd64.deb && \
apt-get install -f -y && \
rm /tmp/google-chrome-stable_current_amd64.deb
# update pip & install robot framework
RUN pip3 install --upgrade pip \
setuptools && \
pip3 install robotframework \
robotframework-selenium2library \
robotframework-xvfb
COPY chromedriver /usr/local/bin
RUN chmod +x /usr/local/bin/chromedriver
WORKDIR /usr/src/robot
CMD /etc/init.d/nginx start && python3 -m robot --include ready BasicTest.robot
Then I run the test image by calling:
$ docker run -ti --rm -v "$PWD/src/test/robot-framework":/usr/src/robot -v "$PWD/dist":/var/www/html:ro --add-host databasehost:10.10.10.10 robot
src/test/robot-framework contains the BasicTest.robot file and the dist folder is the created dist folder for an Angular-based project I want to test.
The messages I get, after waiting a longer time, look like this:
Correct Login :: *Description:* | FAIL |
Setup failed:
WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally
(Driver info: chromedriver=2.35.528139 (47ead77cb35ad2a9a83248b292151462a66cd881),platform=Linux 4.9.60-linuxkit-aufs x86_64)
I start the virtual display in the BasicTest.robot by:
Start Virtual Display 1024 768
${chrome_options}= Evaluate sys.modules['selenium.webdriver'].ChromeOptions() sys, selenium.webdriver
Call Method ${chrome_options} add_argument headless
Call Method ${chrome_options} add_argument disable-gpu
Call Method ${chrome_options} add_argument no-sandbox
Open Browser #{url} #{browser} --auto-ssl-client-auth
I am not sure what is the purpose of your project, but here is my Dockerfile that uses puppeteer that is a Google Lib that wraps chrome_headless and allow you to use it more easily.
Here you can find an official example and here is the official documentation of puppeteer with great community suppport.
This is my Docker file that runs a npm script after building a container with puppeteer, node and chromium dependencies.
# Most part taken from https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md
FROM node:8-slim
# # Manually install missing shared libs for Chromium.
RUN apt-get update && \
apt-get install -yq gconf-service libasound2 libatk1.0-0 libc6
libcairo2 libcups2 libdbus-1-3 \
libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0
libglib2.0-0 libgtk-3-0 libnspr4 \
libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1
libxcb1 libxcomposite1 \
libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2
libxrender1 libxss1 libxtst6 \
ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release
xdg-utils wget
COPY . /app/
WORKDIR app
# Install deps for server.
RUN npm install -g
# Install puppeteer so it can be required by user code that gets run in
# server.js. Cache bust so we always get the latest version of puppeteer when
# building the image.
ARG CACHEBUST=1
RUN npm install puppeteer#0.13.0
# Add pptr user.
RUN groupadd -r pptruser && useradd -r -g pptruser -G audio,video pptruser \
&& mkdir -p /home/pptruser/Downloads \
&& chown -R pptruser:pptruser /home/pptruser \
&& chown -R pptruser:pptruser /app
# Run user as non privileged.
USER pptruser
EXPOSE 9222
CMD ["npm", "start"]
Inside that npm start script you can run your tests I guess.
The trick about using puppeteer inside a Docker container is that Docker does not install chromium dependencies automatically and then it fails when you try to use it.
Despite the fact of using puppeteer or not, that container will allow you to use chrome_headless inside and in my opinion is less heavy than the other examples that I found inside the documentation.
For running just type
docker run -d -i --rm --cap-add=SYS_ADMIN --network=${yourNetwork} --name ${image_name} ${container_name}
Hope that helps

docker official dind build (docker:latest) with chromiuim

I've been trying for the last two days to get chromuim installed and running on docker:latest docker image. (docker in docker).
I have tried multiple docker files:
from docker:latest
RUN apk add --no-cache python py2-pip curl bash chromuim ttf-freefont xvfb nodejs nodejs-npm udev
RUN curl -sSL https://sdk.cloud.google.com | bash
ENV PATH $PATH:~/google-cloud-sdk/bin
RUN pip install docker-compose
RUN npm install -g #angular/cli swagger
ENV CHROME_BIN=/usr/bin/chromium-browser
This installed chrome 57, which doesn't support headless.
So I suspect I can run this with xvbf, but running this chrome fails with:
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted
[8:8:1124/085514.600081:FATAL:zygote_host_impl_linux.cc(182)] Check failed: ReceiveFixedMessage(fds[0], kZygoteBootMessage, sizeof(kZygoteBootMessage), &boot_pid).
Aborted (core dumped)
So I tried to install chrome 61 (which supported headless).
But for that you need to update the Dockerfile to use edge.
I tried to upgrade / or install 61 right away. I always get fonts missing.
The closest I got was adjusting my dockerfile to use lighthose one
from docker:latest
RUN apk add --no-cache python py2-pip curl bash xvfb nodejs nodejs-npm udev
RUN curl -sSL https://sdk.cloud.google.com | bash
ENV PATH $PATH:~/google-cloud-sdk/bin
RUN pip install docker-compose
RUN npm install -g #angular/cli swagger
ENV CHROME_BIN=/usr/bin/chromium-browser
USER root
RUN echo "http://dl-2.alpinelinux.org/alpine/edge/main" > /etc/apk/repositories
RUN echo "http://dl-2.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories
RUN echo "http://dl-2.alpinelinux.org/alpine/edge/testing" >> /etc/apk/repositories
#-----------------
# Set ENV and change mode
#-----------------
ENV LIGHTHOUSE_CHROMIUM_PATH /usr/bin/chromium-browser
ENV TZ "Europe/Berlin"
ENV DEBIAN_FRONTEND noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN true
ENV SCREEN_WIDTH 750
ENV SCREEN_HEIGHT 1334
ENV SCREEN_DEPTH 24
ENV DISPLAY :99.0
ENV PATH /lighthouse/node_modules/.bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
ENV GEOMETRY "$SCREEN_WIDTH""x""$SCREEN_HEIGHT""x""$SCREEN_DEPTH"
RUN echo $TZ > /etc/timezone
#-----------------
# Add packages
#-----------------
RUN apk -U --no-cache update
RUN apk -U --no-cache add \
zlib-dev \
chromium \
freetype \
ttf-opensans \
xvfb \
wait4ports \
xorg-server \
dbus \
ttf-freefont \
mesa-dri-swrast
# Minimize size
RUN apk del --purge --force curl make gcc g++ python linux-headers binutils-gold gnupg git zlib-dev apk-tools libc-utils
RUN rm -rf /var/lib/apt/lists/* \
/var/cache/apk/* \
/usr/share/man \
/tmp/* \
/usr/lib/node_modules/npm/man \
/usr/lib/node_modules/npm/doc \
/usr/lib/node_modules/npm/html \
/usr/lib/node_modules/npm/scripts
VOLUME /lighthouse/output
ADD xvfb-chromium.sh /chromium-xvfb.sh
RUN chmod +x /chromium-xvfb.sh
xvfb-chromium.sh (althought not need, as you can docker run /bin/bash into the container)
#!/bin/sh
_kill_procs() {
kill -TERM $chromium
wait $chromium
kill -TERM $xvfb
}
parameters=$#
# We need to test if /var/run/dbus exists, since script will fail if it does not
[ ! -e /var/run/dbus ] && mkdir /var/run/dbus
/usr/bin/dbus-daemon --system
# Setup a trap to catch SIGTERM and relay it to child processes
trap _kill_procs SIGTERM
TMP_PROFILE_DIR=`mktemp -d -t chromium.XXXXXX`
export CHROME_DEBUGGING_PORT=9222
# Start Xvfb
Xvfb ${DISPLAY} -ac +iglx -screen 0 ${GEOMETRY} -nolisten tcp & xvfb=$!
printf "Starting xvfb window server..."
while [ 1 -gt $xvfb ]; do printf "..."; sleep 1; done
printf "xvfb started\n\n"
#printf "Starting chromium, with debugger on port $CHROME_DEBUGGING_POST...\n\n"
# --disable-webgl \
$CHROME_BIN \
--no-sandbox \
--user-data-dir=${TMP_PROFILE_DIR} \
--start-maximized \
--remote-debugging-port=${CHROME_DEBUGGING_PORT} \
--no-first-run "about:blank" &
#chromium=$!
#wait4ports tcp://127.0.0.1:$CHROME_DEBUGGING_PORT
printf "\n\n==============================\nlaunching lighthouse run\n==============================\n\n"
#wait $chromium
wait $xvfb
Then I got another error:
Error relocating /usr/lib/chromium/chrome: FT_Set_Default_Properties: symbol not found
Not sure how to solve this, any help would be appreciated.
you could try this link https://github.com/c0b/chrome-in-docker
It downloads a google-chrome Linux version from chrome channels, either stable, or beta, or developer version;It turns google-chrome into a headless browser,