Crontab in Docker with flask application - mysql

I'm trying to create cronjob within the docker container, But the it doesn't work. Below is my code
Dockerfile
FROM python:3
LABEL image for a very management application
# We copy just the requirements.txt first to leverage Docker cache
COPY ./requirements.txt /app/requirements.txt
WORKDIR /app
RUN apt-get install -y default-libmysqlclient-dev
RUN pip3 install -r requirements.txt
RUN apt-get update && apt-get -y install cron
RUN touch /var/log/cron.log
COPY crontab /etc/cron.d/cjob
RUN chmod 0644 /etc/cron.d/cjob
ENV PYTHONUNBUFFERED 1
CMD cron -f
EXPOSE 5000
COPY . .
CMD ["cron", "-f"]
CMD [ "python3", "app.py" ]
crontab
*/5 * * * * root /test.py >> /logfile
test.py
from datetime import datetime
print("Our test works at", datetime.now())
I'm tring to run cronjob with the
docker-compose up
Is anything wrong i'm doing, Is their any other way for cronjob to work. Please can anyone help me. Thank you all

Since you can mainly run one CMD you will need to use the ENTRYPOINT directive. I can give you an example of how I made it work.
The theory behind the solution is that in the base image during build time you prepare all the necessary documents:
copy in the crontab file and apply it
Copy in the additional launch script
set ENTRYPOINT to the launch script
run CMD and start your app
Example:
COPY token-renewal-cron /etc/cron.d/token-renewal-cron
RUN crontab /etc/cron.d/token-renewal-cron
where crontab logic is inside token-renewal-cron
Then you need to copy the launch script into the /usr/local/bin/ directory
COPY launch.sh /usr/local/bin/launch.sh
RUN chmod 0777 /usr/local/bin/launch.sh #0777 as long as it works for POC :)
Where in our case launch.sh contains:
cron # this starts the cron process
TMP # some commands we need to run next to cron command (we prepare app env here)
# Append CMD from Dockerfile NOTE this is important so you can then use CMD after ENTRYPOINT inside DOCKERFILE
exec "$#"
Then at the end of the DOCKERFILE you just prepare ENTRYPOINT and CMD commands
#Launch our entrypoint script
ENTRYPOINT ["launch.sh"]
#Launch our application
CMD ["dotnet", "APPLICATION.dll"]

Related

Run bash script after MySQL Docker container starts (every time, not just the initial time)

I am trying to get a bash script to run when my MySQL container starts. Not the initial time when there are no databases to create, but subsequent times (so placing the files in docker-entrypoint-initdb.d will not work).
My objective is to re-build my container with some database upgrade scripts (schema changes, etc). The thought being I deploy the container with the initial scripts and deploy subsequent updates with my database upgrades as the application ages. It seems like this would be an easy task, but I am having trouble.
Most of the things I have tried came from suggestions I found googling. Here are things I have tried with no success:
Modify the entrypoint.sh (and /usr/local/bin/docker-entrypoint.sh) in the Dockerfile build to add in a call to my script.
This does not even seem to be called, which I suspect is a sign, but my database starts (also note it creates my schema fine the first time)
I do this with a RUN sed in my Dockerfile and have confirmed my changes exist after the container starts
Tried running my script on startup by:
adding a script to /etc/r.d/rc.local
adding a restart cron job (well, I tried, but the Oracle Linux distro doesn’t have it)
— Modifying the /etc/bashrc
— Adding a script to /etc/profile.d/
— Appending to /etc/profie.d/sh.local
Tried adding a command to my docker-compose.yml, but it said that wasn’t found.
My actual database upgrade script works great when I log in to the container manually and execute it. All of my experiments above have been just touching a file or echoing to a file as a proof of concept. Once I get that working, I'll add in the logic to wait for MySQL to start and then run my actual script.
Dockerfile:
FROM mysql:8.0.32
VOLUME /var/lib/mysql
## these are my experiments
RUN sed -i '/main "$#"/a echo "run the script here" > /usr/tmp/XXX' /entrypoint.sh
RUN sed -i '/main "$#"/a echo "run the script here" > /usr/tmp/XXX' /usr/local/bin/docker-entrypoint.sh
RUN echo "touch /usr/tmp/XXX" >> /etc/profile.d/sh.local
RUN sed -i '/doublesourcing/a echo “run the script here > /usr/tmp/XXX' etc/bashrc
I build and run it using:
docker build -t mysql-database -f Dockerfile .
docker run -it --rm -d -p 3306:3306 --name database -v ~/Docker-Volume-Share/database:/var/lib/mysql mysql-database
Some other information that may be useful
I am using a volume on the host. I’ve run my experiments with an existing schema as well as by deleting this directory so it starts fresh
I am using mysql:8.0.32 as the image (Oracle Linux Server release 8.7)
Docker version 20.10.22, build 3a2c30b
Host OS is macOS 13.2.1
Thanks in advance for any tips and guidance!
It sounds like you are trying to run a script after the MySQL container has started and the initial setup has been completed. Here are a few suggestions:
1-Use a custom entrypoint script
You can create a custom entrypoint script that runs after the default entrypoint script included in the MySQL container image. In your Dockerfile, copy your custom entrypoint script into the container and set it as the entrypoint. Here's an example:
FROM mysql:8.0.32
COPY custom-entrypoint.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/custom-entrypoint.sh
ENTRYPOINT ["custom-entrypoint.sh"]
In your custom entrypoint script, you can check if the database already exists and run your upgrade script if it does. Here's an example:
#!/bin/bash
set -e
# Run the default entrypoint script
/docker-entrypoint.sh "$#"
# Check if the database already exists
if mysql -uroot -p"$MYSQL_ROOT_PASSWORD" -e "use my_database"; then
# Run your upgrade script
/path/to/upgrade-script.sh
fi
2-Use a Docker Compose file
If you're using Docker Compose, you can specify a command to run after the container has started. Here's an example:
version: '3'
services:
database:
image: mysql:8.0.32
volumes:
- ~/Docker-Volume-Share/database:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: mypassword
command: >
bash -c "
/docker-entrypoint.sh mysqld &
while ! mysqladmin ping -hlocalhost --silent; do sleep 1; done;
/path/to/upgrade-script.sh
"
This command runs the default entrypoint script in the background, waits for MySQL to start, and then runs your upgrade script.
I hope these suggestions help you achieve your goal!

Create a Dockerfile with ubuntu image and run mysql in it

I am trying to set up a Dockerfile with an ubuntu image that runs a mysql server (I know I should use the mysql image instead, but I need to do it this way for an assignment). I am having a lot of trouble designing the Dockerfile. My Dockerfile looks like this right now:
FROM ubuntu:latest
EXPOSE 3306
VOLUME [ "/var/lib/mysql" ]
RUN apt update && \
apt install -y mysql-server && \
apt-get install -y gosu
ENV MYSQL_ROOT_PASSWORD=secret
COPY ./wordpress.sql /docker-entrypoint-initdb.d/
COPY ./docker-entrypoint.sh /usr/local/bin/
COPY ./entrypoint.sh /
RUN chmod +x entrypoint.sh && \
chmod +x /docker-entrypoint-initdb.d/wordpress.sql && \
chmod +x /usr/local/bin/docker-entrypoint.sh
ENTRYPOINT ["docker-entrypoint.sh"]
CMD [ "mysqld" ]
(I am running docker build -t sql . to build the image, and docker run --name sql_cont -v sql_vol:/var/lib/mysql sql_img to run the container)
I have a folder where I have the docker-entrypoint.sh file and the entrypoint.sh file, because when I install mysql-server, these files are not created by default as If I was just using the mysql docker image.
I just need a Dockerfile from an ubuntu image that runs mysql and just stays waiting ready for connections, so, If anybody know how to refactor this Dockerfile or just give me the answer I'd much appreciate it.
If I got What is the difference between CMD and ENTRYPOINT in a Dockerfile? right, the argument in CMD will be passed to the ENTRYPOINT. Don’t know about your ENTRYPOINT script, but I expect it to just ignore parameters.
I would suggest to:
not replace the default ENTRYPOINT (defaults to /bin/sh -c)
start the docker-entrypoint.sh from CMD
Append mysqld to docker-ENTRYPOINT.sh

Create docker container, execute a script and delete the container

I want a command that, in first, will create a docker container, then execute a sql command on my database and then delete the container when it's done.
For the moment I have this Dockerfile :
FROM ubuntu:18.04
# copy database-file.sh
WORKDIR /database-file
RUN mkdir database-file
COPY * /database-file/
# Update and install mysql-client
RUN apt-get update && apt-get -y install mysql-client
# Prepare database-prepare script to be use
RUN chmod +x ./database-prepare.sh
VOLUME /app/log
# Exec my script
ENTRYPOINT ["./database-prepare.sh"]
I took an Ubuntu image, I don't know if it's the best but for the moment, it's ok i think. I just want to test.
So there is my script :
### Global ./database-prepare.sh ###
#!/bin/bash
# Get service name by Docker secret.
service=$(cat /run/secrets/<society_name>_DATABASE_PREPARE__SERVICE)
# Format the service name.
serviceName=${service//./-}
# Clone the repo of my service
git clone --recurse-submodules git#bitbucket.org:<society_name>/<society_name>.binding.$service.git /$serviceName
# CD in the right folder
cd /$serviceName/build/
# Prepare my script and execute it
chmod +x database-prepare.sh
./database-prepare.sh
Then each service have his own database-prepapre.sh that will create a new user in my database :
### Service ./database-prepare.sh ###
#!/bin/bash
# Get password with Docker secret
<society_name>_MSQL_ROOT_PWD=$(cat /run/secrets/<society_name>_MYSQL_PWD) # My Mysql password
<society_name>_BINDING_ABEEWAY_GENERIC_DBS__<society_name>_DATA__PWD=$(cat /run/secrets/<society_name>_BINDING_ABEEWAY_GENERIC_DBS__<society_name>_DATA__PWD) # My password for this user
mysql -h <society_name>-sql-server.mysql.database.azure.com --user='administrateur#<society_name>-sql-server' --ssl --password=$<society_name>_MYSQL_ROOT_PWD << EOF
DROP USER IF EXISTS '<society_name>.binding.abeeway.generic'#'%';
CREATE USER '<society_name>.binding.abeeway.generic'#'%' IDENTIFIED BY '$<society_name>_BINDING_ABEEWAY_GENERIC_DBS__<society_name>_DATA__PWD';
GRANT SELECT, INSERT ON <society_name>_data.ms_gpsposition TO '<society_name>.binding.abeeway.generic'#'%';
GRANT SELECT, INSERT ON <society_name>_data.ms_log TO '<society_name>.binding.abeeway.generic'#'%';
GRANT SELECT, INSERT ON <society_name>_data.ms_temperature TO '<society_name>.binding.abeeway.generic'#'%';
EOF
And know here is my issue :
./database-prepare.sh: line 6: unexpected EOF while looking for matching `"'
./database-prepare.sh: line 13: syntax error: unexpected end of file
So my error is in the file of my Service, where I have "End Of File" text. I try to execute this code, directly in a running container and there is no problem with the EOF... I don't really know where I make a mistake, I'm a novice in Dockerfile, and I think my mistake it's maybe caused by the Image I took, or I maybe I forgot to install a useful tool to make my script work correctly. Or just my script isn't good .. I don't know.
I hope you have enough information to help me, tell me if you need more.
Thank you very much !
Ps: If you know how to delete my service after he executed his command, I would be grateful ! I work on swarm cluster
I finally find a solution, I rewrite my Dockerfile with another image :
FROM alpine:3.7 # <----- Changed
WORKDIR /database-file
RUN mkdir database-file
COPY * /database-file/
RUN apt-get update && apt-get -y install mysql-client # <----- Deleted
RUN apk add mysql-client # <----- Added
RUN apk add git # <----- Added
RUN apk add openssh-client # <----- Added
RUN chmod +x ./database-prepare.sh
VOLUME /app/log
ENTRYPOINT ["sh","./database-prepare.sh"] # <----- Changed
I think the real problem was my ENTRYPOINT, after reflection I think my old ENTRYPOINT doesn't call correctly my script.
I hope this will help another person.

How can mysql db be restored in dockerfile

I'm trying to restore MySQL DB to a ubuntu docker container which has Apache and MySQL services. Here's my docker file
FROM ubuntu
RUN apt-get update -y
ENV DATABASE_SERVER 'IP'
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get install apache2 -y && apt-get install php7.4 -y && apt-get install mysql-server >
ENV APACHE_RUN_USER www-data
ENV APACHE_RUN_GROUP www-data
ENV APACHE_LOG_DIR /var/log/apache2
ENV APACHE_RUN_DIR /var/www/html/
COPY ./startup.sh /var/www/
COPY ./db_test.php /var/www/html
COPY ./my_sql_secure.sh /var/www/
COPY ./backup.sql /var/www/html/
RUN bash /var/www/my_sql_secure.sh
COPY ./restoredb.sh /var/www/
CMD bash /var/www/startup.sh
EXPOSE 80
Here's my startup.sh
apache2 -DFOREGROUND | service mysql start | mysql -uroot sentrifugo < /var/www/html/backup.sql
If I run startup.sh without "mysql -uroot sentrifugo < /var/www/html/backup.sql" , the script properly brings up mysql service but when I run with it doesn't run.
From what i know CMD accepts only two commands and running the restoredb.sh after startup.sh replaces it.
I just want to restore the mysql Database and run mysql and apache in foreground. I can't use docker-compose as per the requirement I have.
Could someone please tell me what can be done to achieve it.
Thanks a lot in advance
It really depends on what image you are building FROM, assuming that it is an official MySQL image, you just COPY your backup.sql into the seed folder:
When a container is started for the first time, a new database with the specified name will be created and initialized with the provided configuration variables. Furthermore, it will execute files with extensions .sh, .sql and .sql.gz that are found in /docker-entrypoint-initdb.d. Files will be executed in alphabetical order. You can easily populate your mysql services by mounting a SQL dump into that directory and provide custom images with contributed data. SQL files will be imported by default to the database specified by the MYSQL_DATABASE variable.
(from the MySQL page on DockerHub )
So, change:
COPY ./backup.sql /var/www/html/
...to:
COPY ./backup.sql /docker-entrypoint-initdb.d/
If you are using a custom image, and it seems that you may be, the you might want to have an ENTRYPOINT that executes that import script on startup.
Also, your startup.sh might work with some changes:
service mysql start && \
mysql -uroot sentrifugo < /var/www/html/backup.sql;
apache2 -DFOREGROUND
This will start the MySQL service first, then populate the DB. Then, finally, start up Apache.
One last thing, the preferred form of CMD is the exec form. This would have your CMD look like this:
CMD ["/bin/bash","/var/www/startup.sh"]
(ref: https://docs.docker.com/engine/reference/builder/#cmd)

Error : The command '/bin/sh returned a non-zero code: 1

When I am trying to build one of my projects by running a script written by previous team in my ubuntu 16.04
sudo ./build
I am getting error :
Step 8/24 : RUN service mysql start
---> Running in 3djjk653642d
* Starting MySQL database server mysqld
...fail!
The command '/bin/sh -c service mysql start' returned a non-zero code: 1
My Dockerfile looks like:
COPY schema.sql /tmp/schema.sql
### User with ALL accesses (winter/toor)
RUN service mysql start
RUN mysql < /tmp/schema.sql
RUN mysql -e "CREATE USER 'winter'#'%' IDENTIFIED BY 'toor'"
RUN service mysql start && mysql -e "GRANT ALL ON its.* TO 'winter'#'%'"
Please ,any help ?
RUN statements in a Dockerfile are used to run a command which will have some effect on the filesystem, that is then saved in another layer.
It's not normal to start a service like this, as the state of the memory (where the service is running) is not stored in the image, it can only be running in a running container.
The normal way to do stuff like this would be to write a bash script, (called start.sh, or something similar), copy it into the image and then run from an ENTRYPOINT / CMD line at the end of the Dockerfile. This will be run when the container is created in a docker run ... command
start.sh:
service mysql start
mysql < /tmp/schema.sql
mysql -e "CREATE USER 'winter'#'%' IDENTIFIED BY 'toor'"
service mysql start && mysql -e "GRANT ALL ON its.* TO 'winter'#'%'"
Dockerfile:
COPY schema.sql /tmp/schema.sql
COPY start.sh /
ENTRYPOINT ["/start.sh"]
Have a read here for some information on the difference between ENTRYPOINT & CMD and when each should be used.
Better still - use the official MySQL image from Docker hub. Through the use of environment variables, you could probably achieve all you require.
For me the error was:
yum -y install nginx' returned a non-zero code: 1
This docker file helped me:
FROM centos:7
MAINTAINER linuxtechlab
LABEL Remarks="This is a dockerfile example for Centos system"
RUN yum -y update
RUN yum -y install httpd
RUN yum clean all
RUN yum -y install nginx
EXPOSE 80
#ENV HOME /root
#WORKDIR /root
#ENTRYPOINT ["ping"]
#CMD ["google.com"]