error when docker build python django app in alpine linux - mysql

Dockerfile
FROM python:3.6.5-alpine3.7
WORKDIR /app
ADD . /app
RUN pip install --no-cache-dir -r requirements.txt
EXPOSE 10021
CMD ["python", "manage.py", "runserver", "0.0.0.0:10021"]
Requirements
openpyxl==2.4.11
requests==2.18.4
Django==2.0.2
mysqlclient==1.3.12
Build Error
Collecting mysqlclient==1.3.12 (from -r requirements.txt (line 4))
Downloading https://mirrors.aliyun.com/pypi/packages/6f/86/bad31f1c1bb0cc99e88ca2adb7cb5c71f7a6540c1bb001480513de76a931/mysqlclient-1.3.12.tar.gz (89kB)
Complete output from command python setup.py egg_info:
/bin/sh: mysql_config: not found
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-er3m_9t6/mysqlclient/setup.py", line 17, in <module>
metadata, options = get_config()
File "/tmp/pip-install-er3m_9t6/mysqlclient/setup_posix.py", line 44, in get_config
libs = mysql_config("libs_r")
File "/tmp/pip-install-er3m_9t6/mysqlclient/setup_posix.py", line 26, in mysql_config
raise EnvironmentError("%s not found" % (mysql_config.path,))
OSError: mysql_config not found
Thank #danblack for mariadb-connector-c-dev which helped me solve the OSError: mysql_config not found error.
New Dockerfile
FROM python:3.6.5-alpine3.7
WORKDIR /app
ADD . /app
RUN apk add --no-cache build-base && apk add --no-cache mariadb-connector-c-dev && pip install --no-cache-dir -r requirements.txt && apk del build-base
EXPOSE 10021
CMD ["python", "manage.py", "runserver", "0.0.0.0:10021"]
New Problem
But a new problem occurred, Docker printed nothing when docker run ... ran, no logs, no errors. It should have printed logs like the following
Performing system checks...
System check identified no issues (0 silenced).
September 04, 2018 - 15:56:26
Django version 2.0.2, using settings 'xxx.settings'
Starting development server at http://0.0.0.0:10021/
Quit the server with CTRL-BREAK.
Can someone show me a complete Dockerfile? Please help! Thank you so much.

You'd need to install the mariadb-connector-c-dev alpine package, which provides mysql_config that the python packages uses to identify which libraries it needs link against.

As per #danblack, mainly you need to install mariadb-connector-c-dev package. But only this package will give compilation errors. So additionally you will need to add gcc and musl-dev packages into Dockerfile. This will make Django and MySQL work within alpine image.
FROM python:3.8-alpine
RUN apk add gcc musl-dev mariadb-connector-c-dev

The trick is to temporarily install mariadb-dev which is only needed during build and then delete it. You can save some bandwitdh by installing the runtime deps before deleting the build deps.
Something like:
RUN apk add --no-cache --virtual .build-deps mariadb-dev ... \
&& pip install ...\
&& apk add --virtual .runtime-deps mariadb-client-libs \
&& apk del .build-deps

Since mysql_config is required to compile python mysql dependencies and also it has 127MB in size so keeping that in Docker container will increase container size to overcome this problem we can install and after successfully running pip install we can safely remove
apk add --no-cache mariadb-dev build-base
above command will install mysql dependencies then we can run pip install
pip3 install -r requirements.txt
once pip is done we can safely remove
apk del mariadb-dev build-base

In my case run with:
FROM python:3.8-alpine
RUN apk add mysql mysql-client gcc musl-dev mariadb-connector-c-dev
Adding mysql and mysql-client

Related

Getting "OSError: mysql_config not found" when trying to build a Docker image for my Python 3.7/Django project [duplicate]

This is the error I get
(mysite)zjm1126#zjm1126-G41MT-S2:~/zjm_test/mysite$ pip install mysql-python
Downloading/unpacking mysql-python
Downloading MySQL-python-1.2.3.tar.gz (70Kb): 70Kb downloaded
Running setup.py egg_info for package mysql-python
sh: mysql_config: not found
Traceback (most recent call last):
File "<string>", line 14, in <module>
File "/home/zjm1126/zjm_test/mysite/build/mysql-python/setup.py", line 15, in <module>
metadata, options = get_config()
File "setup_posix.py", line 43, in get_config
libs = mysql_config("libs_r")
File "setup_posix.py", line 24, in mysql_config
raise EnvironmentError("%s not found" % (mysql_config.path,))
EnvironmentError: mysql_config not found
Complete output from command python setup.py egg_info:
sh: mysql_config: not found
Traceback (most recent call last):
File "<string>", line 14, in <module>
File "/home/zjm1126/zjm_test/mysite/build/mysql-python/setup.py", line 15, in <module>
metadata, options = get_config()
File "setup_posix.py", line 43, in get_config
libs = mysql_config("libs_r")
File "setup_posix.py", line 24, in mysql_config
raise EnvironmentError("%s not found" % (mysql_config.path,))
EnvironmentError: mysql_config not found
----------------------------------------
Command python setup.py egg_info failed with error code 1
Storing complete log in /home/zjm1126/.pip/pip.log
(mysite)zjm1126#zjm1126-G41MT-S2:~/zjm_test/mysite$ pip install mysql-python
Downloading/unpacking mysql-python
Running setup.py egg_info for package mysql-python
sh: mysql_config: not found
Traceback (most recent call last):
File "<string>", line 14, in <module>
File "/home/zjm1126/zjm_test/mysite/build/mysql-python/setup.py", line 15, in <module>
metadata, options = get_config()
File "setup_posix.py", line 43, in get_config
libs = mysql_config("libs_r")
File "setup_posix.py", line 24, in mysql_config
raise EnvironmentError("%s not found" % (mysql_config.path,))
EnvironmentError: mysql_config not found
Complete output from command python setup.py egg_info:
sh: mysql_config: not found
Traceback (most recent call last):
File "<string>", line 14, in <module>
File "/home/zjm1126/zjm_test/mysite/build/mysql-python/setup.py", line 15, in <module>
metadata, options = get_config()
File "setup_posix.py", line 43, in get_config
libs = mysql_config("libs_r")
File "setup_posix.py", line 24, in mysql_config
raise EnvironmentError("%s not found" % (mysql_config.path,))
EnvironmentError: mysql_config not found
----------------------------------------
Command python setup.py egg_info failed with error code 1
Storing complete log in /home/zjm1126/.pip/pip.log
What can I do to resolve this?
It seems mysql_config is missing on your system or the installer could not find it.
Be sure mysql_config is really installed.
For example on Debian/Ubuntu you must install the package:
sudo apt-get install libmysqlclient-dev
Maybe the mysql_config is not in your path, it will be the case when you compile by yourself
the mysql suite.
Update: For recent versions of debian/ubuntu (as of 2018) it is
sudo apt install default-libmysqlclient-dev
In Mac OS, I simply ran this in terminal to fix:
export PATH=$PATH:/usr/local/mysql/bin
This is the quickest fix I found - it adds it to the path, but I think you're better off adding it permanently (ie add it to /etc/paths) if you plan to install MySQL-python in another environment.
(tested in OSX Mountain Lion)
There maybe various answers for the above issue, below is a aggregated solution.
For Ubuntu:
$ sudo apt update
$ sudo apt install python-dev
$ sudo apt install python-MySQLdb
For CentOS:
$ yum install python-devel mysql-devel
apt-get install libmysqlclient-dev python-dev
Seemed to do the trick.
If you are on MAC
Install this globally
brew install mysql
then export path like this
export PATH=$PATH:/usr/local/mysql/bin
Than globally or in your venv whatever you like
pip install MySQL-Python
Note: globally for python3 as Mac can have both python2 & 3
pip3 install MySQL-Python
You can use the MySQL Connector/Python
Installation via PyPip
pip install mysql-connector-python
Further information can be found on the MySQL Connector/Python 1.0.5 beta announcement blog.
On Launchpad there's a good example of how to add-, edit- or remove data with the library.
For centos users:
yum install -y mysql-devel python-devel python-setuptools
then
pip install MySQL-python
If this solution doesn't work, and print gcc compile error like:
_mysql.c:29:20: error: Python.h: No such file or directory
You need to specify the path of Python.h, like this:
pip install --global-option=build_ext --global-option="-I/usr/include/python2.6" MySQL-python
I was trying to install mysql-python on an Amazon EC2 Linux instance and I had to install these :
yum install mysql mysql-devel mysql-common mysql-libs gcc
But then I got this error :
_mysql.c:29:20: fatal error: Python.h: No such file or directory
So I installed :
yum install python-devel
And that did the trick.
For anyone that is using MariaDB instead of MySQL, the solution is to install the libmariadbclient-dev package and create a symbolic link to the config file with the correct name.
For example this worked for me:
ln -s /usr/bin/mariadb_config /usr/bin/mysql_config
For Linux
this works for me
yum install python-devel mysql-devel
Try sudo apt-get build-dep python-mysqldb
OSX Mavericks
Due to changes within osx mavericks & xcode development tools you may get the error on installation
clang: error: unknown argument: '-mno-fused-madd' [-Wunused-command-line-argument-hard-error-in-future]
therefore use :
sudo ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future pip install mysql-python
for mariadb install libmariadbclient-dev instead of libmysqlclient-dev
sudo apt-get install libmariadbclient-dev
You should install the mysql first:
yum install python-devel mysql-community-devel -y
Then you can install mysqlclient:
pip install mysqlclient
I had the same problem in the Terraform:light container. It is based on Alpine.
There you have to install mariadb-dev with:
apk add mariadb-dev
But that one is not enough because also all the other dependencies are missed:
apk add python2 py2-pip gcc python2-dev musl-dev
sometimes the error depends on the actual cause. we had a case where mysql-python was installed through the python-mysqldb debian package.
a developer who didn't know this, accidentally ran pip uninstall mysql-python and then failed to recover with pip install mysql-python giving the above error.
pip uninstall mysql-python had destroyed the debian package contents, and of course pip install mysql-python failed because the debian package didn't need any dev files.
the correct solution in that case was apt-get install --reinstall python-mysqldb which restored mysql-python to its original state.
Sequence to be followed.
pip install mysqlclient
sudo apt-get install python3-dev libmysqlclient-dev
pip install configparser
sudo cp /usr/lib/python3.6/configparser.py /usr/lib/python3.6/ConfigParser.py
Then try to install the MYSQL-python again. That Worked for me
On Mac:
brew install mysql-client
locate mysql
mdfind mysql | grep bin
then add to path
export PATH=$PATH:/usr/local/Cellar/mysql-client/8.0.23/bin/
or permanently
echo "export PATH=$PATH:/usr/local/Cellar/mysql-client/8.0.23/bin/" >> ~/.zshrc
source ~/.zshrc
Had a similar issue trying to install on OS X Server 10.6.8. Here's what I had to do.
Using:
MySQL-python 1.2.4b4 (source)
MySQL-5.6.19 (binary installer)
Python 2.7 (binary installer)
NOTE: Installing in virtualenv...
Unzip source, open 'distribute_setup.py' and edit DEFAULT_VERSION to use the latest version of distribute tools, like so:
DEFAULT_VERSION = "0.6.49"
Save. Open 'site.cfg' file and uncomment the path to mysql_config so it looks something like (reference your own path to mysql_config):
# The path to mysql_config.
# Only use this if mysql_config is not on your PATH, or you have some weird
# setup that requires it.
mysql_config = /usr/local/mysql/bin/mysql_config
Now clean, build and make will not fail with the 'mysql_config' not found error.
Hope this helps someone else trying to make use of their old xserves :-)
Your sudo path does not know about your local path... go into superuser mode, add the path, and install it from there.
sudo su
export PATH=$PATH:/usr/local/mysql/bin/
pip install mysql-python
exit
And you're up and running on OSX. Now you have an updated global python.
if you install MySQL-python in your virtual env, you should check the pip version, if the version is older than 9.0.1, please update it
pip install --upgrade pip
write on 27 Aug, 2021
my macos version is BigSur(14.4), and mysql version is mysql8. It seems like in the higher version of mysql, they delete the my_config.h files but change to the mysql.h.
so your guys should run the following shell:
cd /usr/local/include/mysql
cp mysql.h my_config.h
after that, run pip install MySQL-python and everything will be ok.
on MacOS Mojave, mysql_config is found at /usr/local/bin/ rather than /usr/local/mysql/bin as pointed above, so no need to add anything to path.
In my case my database is running on container and my flask app is running on another container when i tried updating code app got broke with error
Complete output from command python setup.py egg_info:
/bin/sh: 1: mysql_config: not found
/bin/sh: 1: mariadb_config: not found
/bin/sh: 1: mysql_config: not found
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-bya8e734/mysqlclient/setup.py", line 15, in <module>
metadata, options = get_config()
File "/tmp/pip-build-bya8e734/mysqlclient/setup_posix.py", line 65, in get_config
libs = mysql_config("libs")
File "/tmp/pip-build-bya8e734/mysqlclient/setup_posix.py", line 31, in mysql_config
raise OSError("{} not found".format(_mysql_config_path))
OSError: mysql_config not found
Key in stack trace is
/bin/sh: 1: mysql_config: not found
because where my flask app is running doesn't have mysql client properly configured so first i installed mysql server and then install
sudo apt-get install mysql-server-5.7 -y
Then started MySQL
mansoor#LARC-mansur:~/Documents/clients/HR/DevopsSimulator/web$ sudo systemctl start mysql
Then install flask-mysql package and this time it worked
mansoor#LARC-mansur:~/Documents/clients/HR/DevopsSimulator/web$ sudo pip3 install flask-mysqldb
This is different case but posting here because may be someone else in the world facing same issue

Docker Alpine: Error loading MySQLdb module

I am building an Alpine based image of a Django application with MariaDB and I can't figure out which dependency I should add to my Dockerfile so that my app could properly connect to the DB.
django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module.
Did you install mysqlclient?
Well, I thought I did. From what I read in this article, in this discussion, mariadb-dev is in Alpine the equivalent of default-libmysqlclient-dev in Debian based system. Furthermore, the mysql-client package in Alpine is merely a dummy package (containing mariadb-dev, mariadb-client, etc etc). Here is the Dockerfile:
# pull official base image
FROM python:3.7-alpine
# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# set work directory
WORKDIR /usr/src/cms
# install mysqlclient
RUN apk update \
&& apk add --virtual build-deps gcc python3-dev musl-dev \
&& apk add --no-cache mariadb-dev\
&& apk del build-deps
# install dependencies
RUN pip install --upgrade pip
RUN pip install pipenv
COPY ./Pipfile /usr/src/cms/Pipfile
RUN pipenv install --skip-lock --system --dev
# copy entrypoint.sh
COPY ./entrypoint.sh /usr/src/cms/entrypoint.sh
# copy project
COPY . /usr/src/cms/
# run entrypoint.sh
ENTRYPOINT ["/usr/src/cms/entrypoint.sh"]
I tried to add mariadb-client, to use mysql-client instead.
I also tried to add RUN pip install django-mysql. Nothing seems to change. I successfully build Postgres Django apps without any problem but, when it comes to MySQl vs MariaDB // Debian vs Alpine, I feel confused. Any insight?
It seems you're missing the MySQLdb Python module, for which you should install the mysqlclient Python package: pip install mysqlclient.
On Alpine, pip will build mysqlclient from source, so you'll need gcc and musl-dev for this setup step, hence you'll need to postpone apk del build-deps to after Python modules are installed.
Fixed Dockerfile snippet:
RUN apk update \
&& apk add --virtual build-deps gcc python3-dev musl-dev \
&& apk add --no-cache mariadb-dev
...
RUN pip install mysqlclient
RUN apk del build-deps
Mainly you need to install mariadb-connector-c-dev package. But only this package will give compilation errors. So additionally you will need to add gcc and musl-dev packages into Dockerfile. This will make Django and MySQL work within alpine image.
FROM python:3.8-alpine
RUN apk add gcc musl-dev mariadb-connector-c-dev

pip install mysqlclient on amazon linux

sudo pip install mysqlclient is failing with the famous mysql_config path error
The database is a remote mysql RDS. I tried to install a local mysql using yum (not that i need it) but that didn't help with the error
[ec2-user#ip-172-31-95-95 /]$ sudo pip install mysqlclient
Collecting mysqlclient
Using cached mysqlclient-1.3.12.tar.gz
Complete output from command python setup.py egg_info:
sh: mysql_config: command not found
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-opVvwg/mysqlclient/setup.py", line 17, in <module>
metadata, options = get_config()
File "setup_posix.py", line 44, in get_config
libs = mysql_config("libs_r")
File "setup_posix.py", line 26, in mysql_config
raise EnvironmentError("%s not found" % (mysql_config.path,))
EnvironmentError: mysql_config not found
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-opVvwg/mysqlclient/
[ec2-user#ip-172-31-95-95 /]$
[ec2-user#ip-172-31-95-95 /]$ cat /etc/*-release
NAME="Amazon Linux AMI"
VERSION="2017.09"
ID="amzn"
ID_LIKE="rhel fedora"
VERSION_ID="2017.09"
PRETTY_NAME="Amazon Linux AMI 2017.09"
ANSI_COLOR="0;33"
CPE_NAME="cpe:/o:amazon:linux:2017.09:ga"
HOME_URL="http://aws.amazon.com/amazon-linux-ami/"
Amazon Linux AMI release 2017.09
[ec2-user#ip-172-31-95-95 /]$
First you have to do some initial setup for the compiler of mysql on the os, at least from what I understand.
Make sure you have compiler collection
sudo yum install gcc
Make sure mysql is configured for development purposes
sudo yum install -y mysql-devel
Try to reinstall
pip install mysqlclient==1.3.12
Hope this helps since it seemed to do the trick for me. I came across this here where I found for the equivalent to ubuntu's sudo apt-get install libmysqlclient-dev

An error occured when creating an image using Dockerfile

I created a Dockerfile to create an image to run web based application. When I running that file and it tries to collect mysqlclient==1.3.7 ,following error is occured.
"mysql_config raise EnvironmentError("%s not found" %(mysql_config.path,))
EnvironmentError: mysql_config not found
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-ME9Fq7/mysqlclient/
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command. "
This is Dockerfile
############################################################
# Dockerfile to build Python WSGI Application Containers
# Based on Ubuntu
############################################################
# Set the base image to Ubuntu
FROM ubuntu:latest
# File Author / Maintainer
MAINTAINER Brian
# Add the application resources URL
#RUN echo "deb http://archive.ubuntu.com/ubuntu/ $(lsb_release -sc) main universe" >> /etc/apt/sources.list
# Update the sources list
RUN apt-get update -y;
# Install basic applications
RUN apt-get install -y tar git curl nano wget dialog net-tools build-essential
# Install Python and Basic Python Tools
RUN apt-get install -y python python3-dev python-distribute python-pip libssl-dev
# Copy the application folder inside the container
ADD /WebApp /WebApp
#RUN pip install mysql
# Get pip to download and install requirements:
RUN pip install -r /WebApp/requirements.txt
# Expose ports
EXPOSE 80
# Set the default directory where CMD will execute
WORKDIR /WebApp
# Set the default command to execute
# when creating a new container
# i.e. using CherryPy to serve the application
CMD ./start.sh
How can I solve this problem?
I believe that you are trying to install the mysql driver for Python, but you don't have MySQL installed. Try to install MySQL before installing mysql driver:
RUN apt-get install mysql-server
Hope it helps!

Unable to install the Elastic Beanstalk CLI on Ubuntu

I am trying to install the Elastic Beanstalk CLI (awsebcli) on a fresh Ubuntu 14.04 (on Linux subsystem for Windows) using sudo pip install awsebcli, but launching the eb command just returns the following error:
flavien#XPS-FLAVIEN:~$ eb
Traceback (most recent call last):
File "/usr/local/bin/eb", line 6, in <module>
from pkg_resources import load_entry_point
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3036, in <module>
#_call_aside
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3020, in _call_aside
f(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3049, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 654, in _build_master
ws.require(__requires__)
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 968, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 854, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'termcolor==1.1.0' distribution was not found and is required by awsebcli
Any idea what might be going wrong?
You can reinstall awsebcli with below command if you have issues after installing it:
sudo pip3 install awsebcli --force-reinstall --upgrade
Once its installed check where its installed:
which eb
$ /usr/local/bin/eb #i got eb installed in this path
Next set the path:
export PATH=/usr/local/bin:$PATH
Then run
eb --version
EB CLI 3.14.6 (Python 3.5.2) #this is my installed version
First install the pip separably and try with this command
pip install --upgrade --user awsebcli
I had awsebcli being installed for Python 2.7 but for some reason running eb needed them for Python 3.x so doing this worked :
$ sudo -H pip3 install --upgrade --user awsebcli
After installing all the required wheels eb worked fine :
$ eb --version
EB CLI 3.14.3 (Python 3.5.2)
Also don't forget to add ~/.local/bin to your PATH variable in ~/.bash_profile
The previous answer helped me to figure this one out.
My detail, I had to install a newer version of python than 2.7, one that supported the --trusted-host switch to allow me to get all of my dependencies:
What I ran:
pip --cert zxroot.pem --trusted-host pypi.python.org --proxy [ProxyServer] install --upgrade --user awsebcli
Once I ran that reinstall using the --upgrade switch, eb finally worked:
eb --version
EB CLI 3.10.5 (Python 3.3.1)
I know this is very specific to my particulars but it might help someone else.
Good luck.
The problem is you are missing quite a few applications required by the scripts to compile.
The following installation steps are required prior to running the EB CLI scripts.
As I'm a windows user I created a clean VirtualBox VM install of Ubuntu 18.04.2
Update Ubuntu
sudo apt update
sudo apt upgrade
sudo reboot
Install curl,wget
sudo apt install curl
sudo apt install wget
Install zlib
sudo apt-get install zlib1g-dev
Install libffi
sudo apt-get install libffi libffi5-dev
Install OpenSSL
sudo apt-get install libssl-dev
Install gcc
sudo apt-get install build-essential
sudo apt install libx11-dev
gcc --version
make -v
Install Python 3.7
sudo apt install software-properties-common
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt install python3.7
python3.7 --version
alias python='python3.7'
Install Git
sudo apt install git
After this run the EB CLI
You should see these 5 successful progress steps
Creating exclusive virtualenv for EBCLI
Activating virtualenv
Installing EBCLI
Creating EB wrappers
Finishing up
Success!
Try This:
sudo chown -R username:username ~/.local/
# add to ./*shrc
export PATH=$PATH:~/.local/bin/
pip install --upgrade --user awsebcli
eb --version
#EB CLI 3.10.1 (Python 2.7.1)