I am building an Alpine based image of a Django application with MariaDB and I can't figure out which dependency I should add to my Dockerfile so that my app could properly connect to the DB.
django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module.
Did you install mysqlclient?
Well, I thought I did. From what I read in this article, in this discussion, mariadb-dev is in Alpine the equivalent of default-libmysqlclient-dev in Debian based system. Furthermore, the mysql-client package in Alpine is merely a dummy package (containing mariadb-dev, mariadb-client, etc etc). Here is the Dockerfile:
# pull official base image
FROM python:3.7-alpine
# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# set work directory
WORKDIR /usr/src/cms
# install mysqlclient
RUN apk update \
&& apk add --virtual build-deps gcc python3-dev musl-dev \
&& apk add --no-cache mariadb-dev\
&& apk del build-deps
# install dependencies
RUN pip install --upgrade pip
RUN pip install pipenv
COPY ./Pipfile /usr/src/cms/Pipfile
RUN pipenv install --skip-lock --system --dev
# copy entrypoint.sh
COPY ./entrypoint.sh /usr/src/cms/entrypoint.sh
# copy project
COPY . /usr/src/cms/
# run entrypoint.sh
ENTRYPOINT ["/usr/src/cms/entrypoint.sh"]
I tried to add mariadb-client, to use mysql-client instead.
I also tried to add RUN pip install django-mysql. Nothing seems to change. I successfully build Postgres Django apps without any problem but, when it comes to MySQl vs MariaDB // Debian vs Alpine, I feel confused. Any insight?
It seems you're missing the MySQLdb Python module, for which you should install the mysqlclient Python package: pip install mysqlclient.
On Alpine, pip will build mysqlclient from source, so you'll need gcc and musl-dev for this setup step, hence you'll need to postpone apk del build-deps to after Python modules are installed.
Fixed Dockerfile snippet:
RUN apk update \
&& apk add --virtual build-deps gcc python3-dev musl-dev \
&& apk add --no-cache mariadb-dev
...
RUN pip install mysqlclient
RUN apk del build-deps
Mainly you need to install mariadb-connector-c-dev package. But only this package will give compilation errors. So additionally you will need to add gcc and musl-dev packages into Dockerfile. This will make Django and MySQL work within alpine image.
FROM python:3.8-alpine
RUN apk add gcc musl-dev mariadb-connector-c-dev
Related
I have a container file that uses the base image of fedora 37. I want to install some python libraries and some of them require gssapi. I know that gssapi requires krb5-devel. I have a step to install that and I can confirm that it works by calling krb5-config. However, while building the container (using Kaniko) it hit the issue of incompleted installation of gssapi because krb5-config was not found.
I could build my container fine locally using podman/buildar but when pushing my code to GitLab and using Kaniko to build the container, it failed. I doubt if it is because of Kaniko.
Here is my container file:
FROM fedora:37
USER root
WORKDIR /opt/workspace
RUN dnf install -y createrepo_c \
krb5-devel \
python3-pip \
python3-devel \
tree \
diffutils \
git \
gcc
RUN krb5-config
COPY *requirements.txt .
RUN pip3 install --no-cache-dir -r requirements.txt -r test_requirements.txt
I also tried to install heimdal-devel and did the symblink RUN ln -s /usr/bin/heimdal-krb5-config /usr/bin/krb5-config but it doesn't help at all.
I'm trying to run a release build of my kitura (2.7) app with mysql on the official swift-ubuntu (latest, 5.0.1) image with the following commands.
docker build --no-cache -t my-app-build -f Dockerfile-tools .
docker run -v $PWD:/swift-project -w /swift-project my-app-build /swift-utils/tools-utils.sh build release
First command one is working as expected. Second one is giving a warning:
warning: you may be able to install mysqlclient using your system-packager: apt-get install libmysqlclient-dev
Tried to install the lib but nothing changed...
Can someone help me?
Thanks in advance!
The issue appears to be related to the version of Ubuntu and the resulting level of MySQL that is installed. As the base container is running Ubuntu 14.04 when MySQL installs you get version 5.5 which does not ship the required configuration for pkg-config to find the include paths needed to build your application.
I have been able to get a simple Kitura application which uses SwiftKueryMySQL to build under docker by updating my Dockerfile-tools file with two changes:
1) Update the FROM to:
FROM swift:5.0.1
2) Add some required packages:
# Install system level packages
RUN apt-get update && apt-get install -y sudo libcurl4-openssl-dev openssl libssl-dev pkg-config libmysqlclient-dev
With these updates your build should succeed. I will look into a longer term solution to the issue.
Dockerfile
FROM python:3.6.5-alpine3.7
WORKDIR /app
ADD . /app
RUN pip install --no-cache-dir -r requirements.txt
EXPOSE 10021
CMD ["python", "manage.py", "runserver", "0.0.0.0:10021"]
Requirements
openpyxl==2.4.11
requests==2.18.4
Django==2.0.2
mysqlclient==1.3.12
Build Error
Collecting mysqlclient==1.3.12 (from -r requirements.txt (line 4))
Downloading https://mirrors.aliyun.com/pypi/packages/6f/86/bad31f1c1bb0cc99e88ca2adb7cb5c71f7a6540c1bb001480513de76a931/mysqlclient-1.3.12.tar.gz (89kB)
Complete output from command python setup.py egg_info:
/bin/sh: mysql_config: not found
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-er3m_9t6/mysqlclient/setup.py", line 17, in <module>
metadata, options = get_config()
File "/tmp/pip-install-er3m_9t6/mysqlclient/setup_posix.py", line 44, in get_config
libs = mysql_config("libs_r")
File "/tmp/pip-install-er3m_9t6/mysqlclient/setup_posix.py", line 26, in mysql_config
raise EnvironmentError("%s not found" % (mysql_config.path,))
OSError: mysql_config not found
Thank #danblack for mariadb-connector-c-dev which helped me solve the OSError: mysql_config not found error.
New Dockerfile
FROM python:3.6.5-alpine3.7
WORKDIR /app
ADD . /app
RUN apk add --no-cache build-base && apk add --no-cache mariadb-connector-c-dev && pip install --no-cache-dir -r requirements.txt && apk del build-base
EXPOSE 10021
CMD ["python", "manage.py", "runserver", "0.0.0.0:10021"]
New Problem
But a new problem occurred, Docker printed nothing when docker run ... ran, no logs, no errors. It should have printed logs like the following
Performing system checks...
System check identified no issues (0 silenced).
September 04, 2018 - 15:56:26
Django version 2.0.2, using settings 'xxx.settings'
Starting development server at http://0.0.0.0:10021/
Quit the server with CTRL-BREAK.
Can someone show me a complete Dockerfile? Please help! Thank you so much.
You'd need to install the mariadb-connector-c-dev alpine package, which provides mysql_config that the python packages uses to identify which libraries it needs link against.
As per #danblack, mainly you need to install mariadb-connector-c-dev package. But only this package will give compilation errors. So additionally you will need to add gcc and musl-dev packages into Dockerfile. This will make Django and MySQL work within alpine image.
FROM python:3.8-alpine
RUN apk add gcc musl-dev mariadb-connector-c-dev
The trick is to temporarily install mariadb-dev which is only needed during build and then delete it. You can save some bandwitdh by installing the runtime deps before deleting the build deps.
Something like:
RUN apk add --no-cache --virtual .build-deps mariadb-dev ... \
&& pip install ...\
&& apk add --virtual .runtime-deps mariadb-client-libs \
&& apk del .build-deps
Since mysql_config is required to compile python mysql dependencies and also it has 127MB in size so keeping that in Docker container will increase container size to overcome this problem we can install and after successfully running pip install we can safely remove
apk add --no-cache mariadb-dev build-base
above command will install mysql dependencies then we can run pip install
pip3 install -r requirements.txt
once pip is done we can safely remove
apk del mariadb-dev build-base
In my case run with:
FROM python:3.8-alpine
RUN apk add mysql mysql-client gcc musl-dev mariadb-connector-c-dev
Adding mysql and mysql-client
So, I want to use Mariadb. There is this Connector-C for it.
https://downloads.mariadb.org/connector-c/
How do I install it? Quiet frankly, the documentation for it is horrible. Even the src file for 3.0.5 is linked to 3.0.4 page.
I did not find a way to install the binary, and the documentation for building from src is quiet vague. I would prefer to know how to install both ways (binary and build from source)
I'm using CentOS7 64bit.
The easiest way to install it would be to use the MariaDB package repository.
curl -sS https://downloads.mariadb.com/MariaDB/mariadb_repo_setup | sudo bash
sudo yum -y install MariaDB-devel
As for building from source, these steps should work on CentOS 7.
sudo yum -y install git gcc openssl-devel make cmake
git clone https://github.com/MariaDB/mariadb-connector-c.git
mkdir build && cd build
cmake ../mariadb-connector-c/ -DCMAKE_INSTALL_PREFIX=/usr
make
sudo make install
And for Ubuntu 20.04...
sudo apt-get install libmariadb3 libmariadb-dev
Raspberry Pi OS
cd to preferred build location. Then install (thanks to #markusjm!):
sudo apt install git gcc make cmake libssl-dev
git clone https://github.com/MariaDB/mariadb-connector-c.git
mkdir build && cd build
cmake ../mariadb-connector-c/ -DCMAKE_INSTALL_PREFIX=/usr
make
sudo make install
Then add installation directory to LD_LIBRARY_PATH. Note: my installation directory is /usr/lib/mariadb. If you cannot find this after your installation, search for e.g. libmariadb.so, a file that should reside in your installation folder.
export LD_LIBRARY_PATH=/usr/lib/mariadb:$LD_LIBRARY_PATH
Afterwards you can finally pip3 install mariadb, if, like me, you tried to do that in the first place.
After you download MariaDB Connector/C, untar and cd. Then mv the executable first.
sudo mv -f bin/mariadb_config /usr/bin/
Now you can execute mariadb_config and will know where to put header and library files to build wheel for mariadb.
For example,
Ubuntu 18.04
sudo mv -f include/mariadb /usr/local/include/
sudo mv -f lib/mariadb /usr/local/lib/
CentOS 7 & Ubuntu 20.04
sudo mv -f include/mariadb /usr/include/
sudo mv -f lib/mariadb /usr/lib/
Finally, you could pip install mariadb. (Or, export CFLAGS=-std=c99 may help.)
After, in the case you cannot import mariadb,
export LD_LIBRARY_PATH=/PATH/TO/where/you/mv/lib/mariadb
I created a Dockerfile to create an image to run web based application. When I running that file and it tries to collect mysqlclient==1.3.7 ,following error is occured.
"mysql_config raise EnvironmentError("%s not found" %(mysql_config.path,))
EnvironmentError: mysql_config not found
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-ME9Fq7/mysqlclient/
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command. "
This is Dockerfile
############################################################
# Dockerfile to build Python WSGI Application Containers
# Based on Ubuntu
############################################################
# Set the base image to Ubuntu
FROM ubuntu:latest
# File Author / Maintainer
MAINTAINER Brian
# Add the application resources URL
#RUN echo "deb http://archive.ubuntu.com/ubuntu/ $(lsb_release -sc) main universe" >> /etc/apt/sources.list
# Update the sources list
RUN apt-get update -y;
# Install basic applications
RUN apt-get install -y tar git curl nano wget dialog net-tools build-essential
# Install Python and Basic Python Tools
RUN apt-get install -y python python3-dev python-distribute python-pip libssl-dev
# Copy the application folder inside the container
ADD /WebApp /WebApp
#RUN pip install mysql
# Get pip to download and install requirements:
RUN pip install -r /WebApp/requirements.txt
# Expose ports
EXPOSE 80
# Set the default directory where CMD will execute
WORKDIR /WebApp
# Set the default command to execute
# when creating a new container
# i.e. using CherryPy to serve the application
CMD ./start.sh
How can I solve this problem?
I believe that you are trying to install the mysql driver for Python, but you don't have MySQL installed. Try to install MySQL before installing mysql driver:
RUN apt-get install mysql-server
Hope it helps!