Run statsd as a daemon on EC2 instances programatically - amazon-elastic-beanstalk

EDIT: My goal is to be able to emit metrics from my spring-boot application and have them sent to a Graphite server. For that I am trying to set up statsd. If you can suggest a cleaner approach, that would be better.
I have a Beanstalk application which requires statsd to run as a background process. I was able to specify commands and packages through ebextensions config file as follows:
packages:
yum:
git: []
commands:
01_nodejs_install:
command: sudo yum -y install nodejs npm --enablerepo=epel
ignoreErrors: true
02_mkdir_statsd:
command: mkdir /home/ec2-user/statsd
03_fetch_statsd:
command: git clone https://github.com/etsy/statsd.git /home/ec2-user/statsd
ignoreErrors: true
04_run_statsd:
command: node stats.js exampleConfig.js
cwd: /home/ec2-user/statsd
When I try to deploy the application to a new environment, the EC2 node never comes up fully. I logged in to check what might be going on and noticed in /var/log/cfn-init.log that 01_nodejs_install, 02_mkdir_statsd and 03_fetch_statsd were executed successfully. So I guess the system was stuck on the fourth command (04_run_statsd).
2016-05-24 01:25:09,769 [INFO] Yum installed [u'git']
2016-05-24 01:25:37,751 [INFO] Command 01_nodejs_install succeeded
2016-05-24 01:25:37,755 [INFO] Command 02_mkdir_statsd succeeded
2016-05-24 01:25:38,700 [INFO] Command 03_fetch_statsd succeeded
cfn-init.log (END)
I need help with the following:
If there is a better way to install and run statsd while instantiating an environment, I would appreciate if you could provide details on that approach. This current scheme seems hacky.
If this is the approach I need to stick with, how can I run the fourth command so that statsd can be run as a background process?

Tried a few things and found that the following ebextensions configs work:
packages:
yum:
git: []
commands:
01_nodejs_install:
command: sudo yum -y install nodejs npm --enablerepo=epel
ignoreErrors: true
02_mkdir_statsd:
command: mkdir /home/ec2-user/statsd
03_fetch_statsd:
command: git clone https://github.com/etsy/statsd.git /home/ec2-user/statsd
ignoreErrors: true
04_change_config:
command: cat exampleConfig.js | sed 's/2003/<graphite server port>/g' | sed 's/graphite.example.com/my.graphite.server.hostname/g' > config.js
cwd: /home/ec2-user/statsd
05_run_statsd:
command: setsid node stats.js config.js >/dev/null 2>&1 < /dev/null &
cwd: /home/ec2-user/statsd
Note that I added another command (04_change_config) so that I may configure my own Graphite server and port in statsd configs. This change is not needed to address the original question, though.
The actual run command uses setsid to run the command as a daemon.

Related

podman machine init fails with unhelpful output

I am running the command
podman machine init
in the /entrypoint.sh script which I reference in my Dockerfile:
# Container image that runs your code
FROM ubuntu:latest
# Copies your code file from your action repository to the filesystem path `/` of the container
COPY entrypoint.sh /entrypoint.sh
RUN apt-get update -y
RUN apt-get install -y git gcc python3-dev musl-dev libffi-dev podman
# Code file to execute when the docker container starts up (`entrypoint.sh`)
ENTRYPOINT ["/entrypoint.sh"]
The Dockerfile and the entrypoint.sh scripts are in my github repo where I have github actions configured. The problem I see my my log output from the Github action is this ...
2022-11-29T04:56:58.6855699Z + podman machine init
2022-11-29T04:56:59.1867174Z Downloading VM image: fedora-coreos-37.20221106.2.1-qemu.x8…
2022-11-29T04:56:59.3660034Z [1A[JDownloading VM image: fedora-coreos-37.20221106.2.1-qemu.x8…
...
2022-11-29T04:57:06.5082410Z [1A[JDownloading VM image: fedora-coreos-37.20221106.2.1-qemu.x8…
2022-11-29T04:57:06.9133659Z Extracting compressed file
2022-11-29T04:57:36.0586753Z Error: exit status 1
Extracting compressed file? Did I run out of disk space? What? I am not even sure where to begin to debug this.

Use headless wayland / sway when testing on GitHub Actions

I am trying to run a test suite on GitHub Actions for a package that wraps utilities for calling the clipboard on a variety of platforms. While I have managed to get headless testing set up for a Linux system using X11, based on running xvfb, I am struggling to find documentation for how to set up a headless Wayland-based system for testing the utility wl-clipboard.
The current action I'm running installs sway, creates the required XDG_RUNTIME_DIR, and then runs sway. I suspect I am not starting sway correctly because I can't seem to get it to startup and stay running in the background while the rest of the tests run.
- name: Install wayland
if: ${{ matrix.config.clip_type == 'wayland' }}
run: |
mkdir $XDG_RUNTIME_DIR
chown $USER $XDG_RUNTIME_DIR
chmod 0700 $XDG_RUNTIME_DIR
sudo apt-get update
sudo apt-get purge x11-*
sudo apt-get install sway meson libwayland-dev
echo $XDG_RUNTIME_DIR
ls -la $XDG_RUNTIME_DIR
sway -d -V
cd $GITHUB_WORKSPACE/..
git clone https://github.com/bugaevc/wl-clipboard.git
cd wl-clipboard
meson build
cd build/
sudo ninja install
wl-copy --primary
wl-paste --primary
cd $GITHUB_WORKSPACE
env:
XDG_RUNTIME_DIR: /home/runner/work/clipr/xdg
WLR_BACKENDS: headless
WLR_LIBINPUT_NO_DEVICES: 1
WAYLAND_DISPLAY: wayland-1
GTK_USE_PORTAL: 0
See the verbose logs from sway. Running sway like this in the foreground, it just hangs indefinitely. Naively trying to run sway in the background using nohup sway & results in the later calls to the utilities saying Failed to connect to a Wayland server.
Any suggestions for getting a headless Wayland server up and running?

Unable to run startup script when creating instance on Google Cloud Platform

I have a simple startup script which looks like so:
#!/usr/bin/env bash
sudo apt update
sudo apt install -y ruby-full ruby-bundler build-essential
And create VM instance on GCP like so:
$ gcloud compute instances create test-app --boot-disk-size=10GB --image-family ubuntu-1604-lts --image-project=ubuntu-os-cloud --machine-type=g1-small --zone europe-west1-b --tags test-server --restart-on-failure --metadata-from-file startup-script=startup.sh
My startup.sh is executable. I set its rights like so:
$ chmod +x startup.sh
When however I enter the shell of my newly created instance and check bundler:
test-app:~$ bundle -v
I get these messages:
The program 'bundle' is currently not installed...
So, what is wrong with that and how can I fix it? PS. If I run all my commands just from inside the instance shell, it's all ok, so there is some problem with using startup script on GCP.
I tested with your use case, But the bundle package was installed without making any changes.
Output:
bundle -v
Bundler version 1.11.2
You can check VM serial console log output to verify if start-up script ran. Check the VM instance to verify if the package is installed using the commands below:
sudo apt list --installed | grep -i bundle
sudo egrep bundle /var/log/dpkg.log
In addition, check the gem list bundle

An error occured when creating an image using Dockerfile

I created a Dockerfile to create an image to run web based application. When I running that file and it tries to collect mysqlclient==1.3.7 ,following error is occured.
"mysql_config raise EnvironmentError("%s not found" %(mysql_config.path,))
EnvironmentError: mysql_config not found
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-ME9Fq7/mysqlclient/
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command. "
This is Dockerfile
############################################################
# Dockerfile to build Python WSGI Application Containers
# Based on Ubuntu
############################################################
# Set the base image to Ubuntu
FROM ubuntu:latest
# File Author / Maintainer
MAINTAINER Brian
# Add the application resources URL
#RUN echo "deb http://archive.ubuntu.com/ubuntu/ $(lsb_release -sc) main universe" >> /etc/apt/sources.list
# Update the sources list
RUN apt-get update -y;
# Install basic applications
RUN apt-get install -y tar git curl nano wget dialog net-tools build-essential
# Install Python and Basic Python Tools
RUN apt-get install -y python python3-dev python-distribute python-pip libssl-dev
# Copy the application folder inside the container
ADD /WebApp /WebApp
#RUN pip install mysql
# Get pip to download and install requirements:
RUN pip install -r /WebApp/requirements.txt
# Expose ports
EXPOSE 80
# Set the default directory where CMD will execute
WORKDIR /WebApp
# Set the default command to execute
# when creating a new container
# i.e. using CherryPy to serve the application
CMD ./start.sh
How can I solve this problem?
I believe that you are trying to install the mysql driver for Python, but you don't have MySQL installed. Try to install MySQL before installing mysql driver:
RUN apt-get install mysql-server
Hope it helps!

Why SSH connection to docker container is not working?

So i have this Dockerfile:
FROM debian:squeeze
MAINTAINER Name < email : >
# Update the repository sources list
RUN apt-get update
# Install apache, PHP, and supplimentary programs. curl and lynx-cur are for debugging the container.
RUN DEBIAN_FRONTEND=noninteractive apt-get -y install apache2 build-essential php5 mysql-server openssh-server libapache2-mod-php5 php5-mysql php5-gd php-pear php-apc php5-curl curl lynx-cur
# Enable apache mods.
RUN a2enmod php5
RUN a2enmod rewrite
# Manually set up the apache environment variables
ENV APACHE_RUN_USER www-data
ENV APACHE_RUN_GROUP www-data
ENV APACHE_LOG_DIR /var/log/apache2
ENV APACHE_LOCK_DIR /var/lock/apache2
ENV APACHE_PID_FILE /var/run/apache2.pid
EXPOSE 80
# Copy site into place.
ADD www /var/www/site
# Update the default apache site with the config we created.
ADD apache-config.conf /etc/apache2/sites-enabled/000-default.conf
# start mysqld and apache
EXPOSE 3306
RUN mkdir /var/run/sshd
RUN echo 'root:123' | chpasswd
RUN sed -i 's/PermitRootLogin without-password/PermitRootLogin yes/' /etc/ssh/sshd_config
EXPOSE 22
CMD bash -c ' (mysqld &); /usr/sbin/apache2ctl -D FOREGROUND;/usr/sbin/sshd -D'
it builds up, no problem,MySQL and Apache start and work fine but the ssh won't work and i don't know why. openssh-server is installed.
i tried starting it up like this:
#startup.sh file
#/bin/bash
sshd
+
ADD ./startup.sh /opt/startup.sh
ENTRYPOINT ["/opt/startup.sh"]
and many other,i'm stuck.
What am i doing wrong?
you are starting apache in the foreground, hence the apachectl process will never give back the hand to the shell that started it and thus the /usr/sbin/sshd -D will never be called (unless you kill apache).
The following instruction will start both mysql and apache in the background and then sshd in the foreground:
CMD bash -c ' (mysqld &); /usr/sbin/apache2ctl start;/usr/sbin/sshd -D'
While such a CMD statement is ok for tests I would advise using a different approach for running multiple processes in a single docker container:
supervisor
phusion/baseimage
Replace below lines of code in the docker file,
RUN mkdir /var/run/sshd
RUN echo 'root:123' | chpasswd
RUN sed -i 's/PermitRootLogin without-password/PermitRootLogin yes/' /etc/ssh/sshd_config
Using these codes
RUN apt-get install -y openssh-server
RUN echo 'root:password' |chpasswd
RUN mkdir -p /var/run/sshd
this works for me.
Note: Use ssh only for debugging purpose, it is not a good practice at all.