Packer - sudo/su condition for builders - packer

I just started with Packer, and want to build AMI and Docker images.
Problem is - that Docker's Ubuntu image have no sudo, thus - build failed with an error:
docker: /tmp/script_2085.sh: 3: /tmp/script_2085.sh: sudo: not found
Is there any way to add some "conditions" for a shell provisioner to run cmds with or without sudo, depending on builder?
Currently - my template have:
...
"provisioners": [{
"type": "shell",
"inline": [
"sleep 30",
"sudo apt-get update",
"sudo apt-get install -y nginx"
]
}]
...
Add something like:
if which sudo; then sudo apt-get update && sudo apt-get install -y nginx; else apt-get update && apt-get install -y nginx; fi
Looks too weird solution... I'm sure - Packer has facility to do it in a better way.

Why don't you just install sudo in the docker box.
You could do this one of two ways:
Running a script to install sudo, only on your docker image
You can use the only parameter in the provisioner to install sudo on an image:
...
"provisioners": [{
"type": "shell",
"inline": [
"apt-get install sudo"
],
"only": ["docker"]
},
... other provisioners here ...
]
...
More info in the packer docs, here, under "Run On Specific Builds".
Installing sudo at the beginning of your provision script
At the beginning of your first provision script, add the command to install sudo if it is not already installed. Use the following (taken from here):
dpkg -l | grep -qw sudo || apt-get install sudo
to check the list of installed packages and install if sudo is not present.
Personally, I'd be inclined to go for the first option, since it explicitly states that something should be done only on certain boxes. No room for confusion.
Parting notes
Bear in mind that your provisioners will run in the order you define them in your packer file. So make sure you install sudo in the first one.
You may also need to run apt-get update before you do the install.

Related

Installing mysql client on EC2 via CloudFormation

To install MySQL client on my EC2 instance I do the following:
# Update all packages
sudo yum update
# Install mysql client
sudo yum install mysql
I would like to know how can I provision that using CloudFormation when I create my EC2 instance?
Is there any samples which I can refer to?
To run a simple script on an EC2 instance provisioned using CloudFormation, you use the UserData property of the AWS::EC2::Instance resource.
Example:
Description: Run a bash script using the UserData property.
Mappings:
# amzn-ami-hvm-2016.09.1.20161221-x86_64-gp2
RegionMap:
us-east-1:
"64": "ami-9be6f38c"
Resources:
Instance:
Type: AWS::EC2::Instance
Properties:
ImageId: !FindInMap [ RegionMap, !Ref "AWS::Region", 64]
InstanceType: m3.medium
UserData:
"Fn::Base64":
!Sub |
#!/bin/bash
# Update all packages
yum -y update
# Install mysql client
yum -y install mysql
Just want to share also on top of the answer above.
I later found out my EC2 is deployed via EB and UserData does not seem to work.
In that case, I have to use .ebextension's container_commands.
So posting it here hopefully it will help someone as well:
files:
"/usr/local/bin/eb_mysql.sh":
mode: "000755"
owner: root
group: root
content: |
#!/bin/bash
yum -y update
yum -y install mysql
container_commands:
040_install_mysql_client:
command: "chmod a+rx /usr/local/bin/eb_mysql.sh && /usr/local/bin/eb_mysql.sh"
Install MySQL Client:
sudo yum install -y https://dev.mysql.com/get/mysql57-community-release-el7-11.noarch.rpm
sudo rpm --import https://repo.mysql.com/RPM-GPG-KEY-mysql-2022
sudo yum install -y mysql-community-client

Run statsd as a daemon on EC2 instances programatically

EDIT: My goal is to be able to emit metrics from my spring-boot application and have them sent to a Graphite server. For that I am trying to set up statsd. If you can suggest a cleaner approach, that would be better.
I have a Beanstalk application which requires statsd to run as a background process. I was able to specify commands and packages through ebextensions config file as follows:
packages:
yum:
git: []
commands:
01_nodejs_install:
command: sudo yum -y install nodejs npm --enablerepo=epel
ignoreErrors: true
02_mkdir_statsd:
command: mkdir /home/ec2-user/statsd
03_fetch_statsd:
command: git clone https://github.com/etsy/statsd.git /home/ec2-user/statsd
ignoreErrors: true
04_run_statsd:
command: node stats.js exampleConfig.js
cwd: /home/ec2-user/statsd
When I try to deploy the application to a new environment, the EC2 node never comes up fully. I logged in to check what might be going on and noticed in /var/log/cfn-init.log that 01_nodejs_install, 02_mkdir_statsd and 03_fetch_statsd were executed successfully. So I guess the system was stuck on the fourth command (04_run_statsd).
2016-05-24 01:25:09,769 [INFO] Yum installed [u'git']
2016-05-24 01:25:37,751 [INFO] Command 01_nodejs_install succeeded
2016-05-24 01:25:37,755 [INFO] Command 02_mkdir_statsd succeeded
2016-05-24 01:25:38,700 [INFO] Command 03_fetch_statsd succeeded
cfn-init.log (END)
I need help with the following:
If there is a better way to install and run statsd while instantiating an environment, I would appreciate if you could provide details on that approach. This current scheme seems hacky.
If this is the approach I need to stick with, how can I run the fourth command so that statsd can be run as a background process?
Tried a few things and found that the following ebextensions configs work:
packages:
yum:
git: []
commands:
01_nodejs_install:
command: sudo yum -y install nodejs npm --enablerepo=epel
ignoreErrors: true
02_mkdir_statsd:
command: mkdir /home/ec2-user/statsd
03_fetch_statsd:
command: git clone https://github.com/etsy/statsd.git /home/ec2-user/statsd
ignoreErrors: true
04_change_config:
command: cat exampleConfig.js | sed 's/2003/<graphite server port>/g' | sed 's/graphite.example.com/my.graphite.server.hostname/g' > config.js
cwd: /home/ec2-user/statsd
05_run_statsd:
command: setsid node stats.js config.js >/dev/null 2>&1 < /dev/null &
cwd: /home/ec2-user/statsd
Note that I added another command (04_change_config) so that I may configure my own Graphite server and port in statsd configs. This change is not needed to address the original question, though.
The actual run command uses setsid to run the command as a daemon.

Could not install mysql-server inside docker container

I want to configure a container with MySQL, thereafter I will add a java application to the container. But, I could not install MySQL-server inside the container.
here is my dockerfile:
FROM ubuntu:latest
RUN apt-get -y update
RUN apt-get -y install mysql-server --fix-missing --fix-broken
EXPOSE 3306
CMD ["/usr/bin/mysqld_safe"]
I got the following error:
Errors were encountered while processing:
/var/cache/apt/archives/mysql-server-5.5_5.5.44-0ubuntu0.14.04.1_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
2015/09/01 13:17:36 The command [/bin/sh -c apt-get -y install mysql-server --fix-missing --fix-broken] returned a non-zero code: 100
What's probably happening is the apt-get install is asking for input, and failing. What happens if you just run those commands interactively in the base container? e.g. launch a shell with docker run -ti --rm ubuntu:latest /bin/bash
One thing you can try is to use the DEBIAN_FRONTEND=noninteractive environment variable, e.g.
RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get -yq install mysql-server
I would suggest you use the official mysql docker image. When running that you have to provide the root password as an environment variable during the docker run command. That would be better.

Docker and MySQL can't connect

I can't for the life of me figure this out. I feel like I'm really close, but im just not quite there yet. Here's my Dockerfile:
FROM ubuntu:14.04
MAINTAINER Oscar Godson
RUN apt-get update
RUN apt-get install -y nodejs-legacy
RUN apt-get install -y npm
RUN apt-get install -y mysql-server
RUN apt-get install -y mysql-client
COPY . /src
WORKDIR /src
RUN service mysql start;\
mysql -u root < /src/bin/demo-data.sql;\
npm install
EXPOSE 4000
CMD ["node", "index.js"]
If i go into interactive mode and go into bash in that RUN service... line at the end I can see that mysql is running, the demo data was imported and everything works including starting my Node app with node index.js. If I then try to do another RUN or the CMD you see below it no longer can access MySQL. It's like it stops running.
The error I get Error: connect ECONNREFUSED which happens when it's down or the data isn't there. From me playing with it it looks like MySQL isn't running if I try to access it after the RUN where I start it.
P.S. The MySQL DB here is purely for development. On production we use AWS RDS, so thats why its using root without any password and importing data that way.
The problem was that RUN shouldn't be used for long running processes, however, you can only use one CMD option. For some reason it didn't click with me until a friend pointed it out, but I could just make a "installer.sh" script and call that with CMD. Now it's not losing the MySQL context. My update Dockerfile looks something like this:
FROM ubuntu:14.04
MAINTAINER Oscar Godson
RUN apt-get update
RUN apt-get install -y nodejs-legacy
RUN apt-get install -y npm
RUN apt-get install -y mysql-server
RUN apt-get install -y mysql-client
COPY . /src
WORKDIR /src
RUN cd /src;npm install
EXPOSE 4000
CMD ./bin/installer.sh
Installer than has
service mysql start
mysql -u root < ./bin/demo-data.sql
node index.js

How do I do a clean re-install of zoneminder?

Because doing this:-
sudo apt-get remove --purge zoneminder
sudo apt-get remove --purge mysql-server mysql-client mysql-common
sudo rm -rf /var/lib/mysql
sudo apt-get remove --purge apache2*
sudo apt-get clean
sudo apt-get update
sudo apt-get upgrade
sudo rm -r /var/www/*
sudo rm -r /etc/apache2/*
sudo rmdir /var/www
sudo rmdir /etc/apache2
sudo rm -r /opt/zm
sudo rm /etc/apache2/conf.d/zoneminder.conf
sudo dpkg --configure -a
reboot hardware
sudo apt-get install zoneminder
produces this error:-
* Starting web server apache2 apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1 for ServerName
[ OK ]
Setting up libapache2-mod-php5 (5.4.6-1ubuntu1.4) ...
Creating config file /etc/php5/apache2/php.ini with new version
* Restarting web server apache2 apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1 for ServerName
... waiting apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1 for ServerName
[ OK ]
Setting up apache2 (2.2.22-6ubuntu2.3) ...
Setting up zoneminder (1.25.0-1.1ubuntu1) ...
Starting ZoneMinder: Can't open config file '/opt/zm/etc/zm.conf': No such file or directory at /usr/local/share/perl/5.14.2/ZoneMinder/Config.pm line 100
BEGIN failed--compilation aborted at /usr/local/share/perl/5.14.2/ZoneMinder/Config.pm line 100.
Compilation failed in require at /usr/local/share/perl/5.14.2/ZoneMinder.pm line 33.
BEGIN failed--compilation aborted at /usr/local/share/perl/5.14.2/ZoneMinder.pm line 33.
Compilation failed in require at /usr/bin/zmpkg.pl line 37.
BEGIN failed--compilation aborted at /usr/bin/zmpkg.pl line 37.
failure
invoke-rc.d: initscript zoneminder, action "start" failed.
dpkg: error processing zoneminder (--configure):
subprocess installed post-installation script returned error exit status 2
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place
Processing triggers for ureadahead ...
Errors were encountered while processing:
zoneminder
E: Sub-process /usr/bin/dpkg returned an error code (1)
Originally zoneminder installed "first time" Ok on this 32bit "mint 14" machine using just the single apt-get install zoneminder command from here:-
http://www.tuxradar.com/content/build-your-own-surveillance-zoneminder
but when I tried to upgrade a couple of days ago to version 1.26.2 by compiling manually, I messed it up.
How can I remove zoneminder completely so that the original "clean" install works again?
No need to do any of the above, from this link:-
http://pjpramod.blogspot.co.uk/2010/01/starting-zoneminder-cant-open-config.html
So I did this:-
sudo rm -r /usr/local/share/perl/5.14.2/*
and then the install completes OK - but the popups no longer work (don't open at all or open in the same window), so don't do that.
Finally I abandoned re-installing the pre-built package (1.25.0 - or was it 1.25.2?) and downloaded the 1.26.3 tar file and unpacked it and cd'ed into it...
...and ./configure was missing...
...but if you run bootstrap.sh
./bootstrap.sh
then that (and some other things) magically appear.
Then did this:
sudo ./configure --prefix=/opt/zm --with-mysql=/usr --with-ffmpeg=/usr --with-webuser=www-data --with-webgroup=www-data --disable-debug --disable-crashtrace --enable-mmap=yes --with-webdir=/var/www --with-cgidir=/usr/lib/cgi-bin ZM_SSL_LIB=openssl
and it configured OK.
then this
sudo make
followed by this:-
sudo make install
and it installed OK but when I tried to run zoneminder I got database errors so did something like this to delete zm:-
mysql -uroot -pyourPassWord
mysql > drop database zm;
mysql > quit;
and this to recreate it (I know nothing about databases, this is monkey-see, monkey-do from here):-
mysql -u root -pyourPassWord
>create database zm;
>exit
mysql -u root -pyourPassWord zm < db/zm_create.sql
mysql -u root -pyourPassWord zm
>grant select,insert,update,delete on zm.* to 'zmuser'#localhost identified by 'zmpass';
>quit
mysqladmin -u root -pyourPassWord reload
and it then worked OK (I am running zm like this:-)
sudo /etc/init.d/zm start
And while trying to figure out what link between apache and zm did (and how apache works etc) I discovered that you only need to do this:-
localhost/index.php
to bring up zoneminder, working OK, in my browser - Yee Haw! Not bad for two day's effort and one divorce.
But I could only get zmpkg.pl to work by doing this:-
sudo /opt/zm/bin/zmpkg.pl
and not this:-
sudo zmpkg.pl
Because, I learn, that that $PATH is defined "globally" for sudo and therefore my adding /opt/zm/bin in my .bashrc file had no effect so then, rather than put zmpkg.pl into /usr/local/sbin (or one of those paths in "global" $PATH) I insert a link instead like this:-
sudo ln -s /opt/zm/bin/zmpkg.pl /usr/local/sbin/zmpkg.pl
and it works! so now e.g.
sudo zmpkg.pl status
works again.
Does anyone know how all this should be done? E.g. so that
localhost/zm
works again?
Update on 20th December 2013: I just upgraded to verion 1.26.5 successfully on this mint 14 system by repeating some of the above instructions i.e.
1) Stop zoneminder.
2) Download version 1.26.5
3) Ran the ./bootstrap.sh script (as mentioned above).
4) Ran the ./configure... script (as mentioned above).
5) Ran the make (as mentioned above - but got errors)...
So I read the README and copied and pasted this from it (line 24):
root#host:~# aptitude install -y apache2 mysql-server php5 php5-mysql build-essential libmysqlclient-dev libssl-dev libbz2-dev libpcre3-dev libdbi-perl libarchive-zip-perl libdate-manip-perl libdevice-serialport-perl libmime-perl libpcre3 libwww-perl libdbd-mysql-perl libsys-mmap-perl yasm subversion automake autoconf libjpeg8-dev libjpeg8 apache2-mpm-prefork libapache2-mod-php5 php5-cli libphp-serialization-perl libgnutls-dev libjpeg8-dev libavcodec-dev libavformat-dev libswscale-dev libavutil-dev libv4l-dev libtool ffmpeg libnetpbm10-dev libavdevice-dev libmime-lite-perl dh-autoreconf dpatch;
and then did a
sudo make clean
and then repeated 4) and 5) above and it compiled OK.
6) Did the "sudo make install" OK (i.e. on top of the old version I guess).
7) Then did all the rest of the "as mentioned above" instructions to delete zm from the database (after trying to zm unsuccessfully) and re-inserted it and then it started OK via the "zmpkg.pl start" script as version 1.26.5
8) As before I then had to manually reset up all cameras and password and users and zones but at least it all worked "first time"
I would suggest that you have to uninstall all the packages which include LAMP(apache, mysql, php) and zoneminder.
use this command to remove these packages.
sudo aptitude purge <pkg-name>
Then you have to go to zoneminder manual installation directory (say /usr/src). and use command
sudo make clean
sudp updatedb
After this you can start normal install procedure.