I created a Dockerfile to create an image to run web based application. When I running that file and it tries to collect mysqlclient==1.3.7 ,following error is occured.
"mysql_config raise EnvironmentError("%s not found" %(mysql_config.path,))
EnvironmentError: mysql_config not found
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-ME9Fq7/mysqlclient/
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command. "
This is Dockerfile
############################################################
# Dockerfile to build Python WSGI Application Containers
# Based on Ubuntu
############################################################
# Set the base image to Ubuntu
FROM ubuntu:latest
# File Author / Maintainer
MAINTAINER Brian
# Add the application resources URL
#RUN echo "deb http://archive.ubuntu.com/ubuntu/ $(lsb_release -sc) main universe" >> /etc/apt/sources.list
# Update the sources list
RUN apt-get update -y;
# Install basic applications
RUN apt-get install -y tar git curl nano wget dialog net-tools build-essential
# Install Python and Basic Python Tools
RUN apt-get install -y python python3-dev python-distribute python-pip libssl-dev
# Copy the application folder inside the container
ADD /WebApp /WebApp
#RUN pip install mysql
# Get pip to download and install requirements:
RUN pip install -r /WebApp/requirements.txt
# Expose ports
EXPOSE 80
# Set the default directory where CMD will execute
WORKDIR /WebApp
# Set the default command to execute
# when creating a new container
# i.e. using CherryPy to serve the application
CMD ./start.sh
How can I solve this problem?
I believe that you are trying to install the mysql driver for Python, but you don't have MySQL installed. Try to install MySQL before installing mysql driver:
RUN apt-get install mysql-server
Hope it helps!
Related
I was trying to install and configure apache airflow on dev Hadoop cluster of a three nodes with below configurations/version:
Operating System: Red Hat Enterprise Linux Server 7.7
python 3.7.3
anaconda 2
spark 2.45
a)sudo yum install gcc gcc-c++ -y
b)sudo yum install libffi-devel mariadb-devel cyrus-sasl-devel -y
c)pip install 'apache-airflow[all]'
d)airflow initdb -- airflow.cfgfile was created with SQLlite
Then I followed below set of commands to configure it with mysql
a) rpm -Uvh https://repo.mysql.com/mysql80-community-release-el7-3.noarch.rpm
b) sed -i 's/enabled=1/enabled=0/' /etc/yum.repos.d/mysql-community.repo
c) yum --enablerepo=mysql80-community install mysql-community-server
d) systemctl start mysqld.service
Done below things at mysql
a) CREATE DATABASE airflow CHARACTER SET utf8 COLLATE utf8_unicode_ci;
b) create user 'airflow'#'localhost' identified by 'Airflow123';
c) grant all privileges on * . * to 'airflow'#'localhost';
here are some details from my airflow.cfg file
broker_url = sqla+mysql://airflow:airflow#localhost:3306/airflow
result_backend = db+mysql://airflow:airflow#localhost:3306/airflow
sql_alchemy_conn = mysql://airflow:Airflow123#localhost:3306/airflow
executor = CeleryExecutor
I am getting below error while running airflow initdb commands
ImportError: /home/xyz/anaconda2/envs/python3.7.2/lib/python3.7/site-packages/_mysql.cpython-37m-x86_64-linux-gnu.so: symbol mysql_real_escape_string_quote,
version libmysqlclient_18 not defined in file libmysqlclient.so.18 with link time reference
have set up the .bashrc file as:
export AIRFLOW_HOME=~/airflow
here's my directory created:
[xyz#innolx5984 airflow]$ pwd
/home/xyz/airflow
When I look for this file "libmysqlclient" I have found these many instances.
[xyz#innolx5984 airflow]$ find /home/xyz/ -name "*libmysqlclient*"
/home/xyz/anaconda2/pkgs/mysql-connector-c-6.1.11-h597af5e_1/lib/libmysqlclient.so
/home/xyz/anaconda2/pkgs/mysql-connector-c-6.1.11-h597af5e_1/lib/libmysqlclient.a
/home/xyz/anaconda2/pkgs/mysql-connector-c-6.1.11-h597af5e_1/lib/libmysqlclient.so.18
/home/xyz/anaconda2/pkgs/mysql-connector-c-6.1.11-h597af5e_1/lib/libmysqlclient.so.18.4.0
/home/xyz/anaconda2/lib/libmysqlclient.a
/home/xyz/anaconda2/lib/libmysqlclient.so
/home/xyz/anaconda2/lib/libmysqlclient.so.18
/home/xyz/anaconda2/lib/libmysqlclient.so.18.4.0
Just adding few more details in case it helps.
[xyz#innolx5984 airflow]$ mysql_config
Usage: /home/xyz/an
aconda2/bin/mysql_config [OPTIONS]
Options:
--cflags [-I/home/xyz/anaconda2/include ]
--cxxflags [-I/home/xyz/anaconda2/include ]
--include [-I/home/xyz/anaconda2/include]
--libs [-L/home/xyz/anaconda2/lib -lmysqlclient ]
--libs_r [-L/home/xyz/anaconda2/lib -lmysqlclient ]
--plugindir [/home/xyz`/anaconda2/lib/plugin]
--socket [/tmp/mysql.sock]
--port [0]
--version [6.1.11]
--variable=VAR VAR is one of:
pkgincludedir [/home/xyz/anaconda2/include]
pkglibdir [/home/xyz/anaconda2/lib]
plugindir [/home/xyz/anaconda2/lib/plugin]
Looking for some help and suggestion to resolve this
issue. I am not too sure whether heading into right direction.
Follow these steps to install Apache Airflow with MySQL using Anaconda3
1) Install Pre-requisites
yum install gcc gcc-c++ -y
yum install libffi-devel mariadb-devel cyrus-sasl-devel -y
dnf install redhat-rpm-config
2) Install Anaconda3 (comes with Python 3.7.6)
yum install libXcomposite libXcursor libXi libXtst libXrandr alsa-lib mesa-libEGL libXdamage mesa-libGL libXScrnSaver
wget https://repo.anaconda.com/archive/Anaconda3-2020.02-Linux-x86_64.sh
chmod +x Anaconda3-2020.02-Linux-x86_64.sh
./Anaconda3-2020.02-Linux-x86_64.sh
Make sure you do conda initialize when prompted during installation.
This will make sure the correct version of python and pip are used in the subsequent steps.
3) Install Apache Airflow
pip install apache-airflow[mysql,celery]
You can add other subpackages as required. I have included only the ones required for Airflow to use MySQL database as backend.
4) Initialize Airflow
export AIRFLOW_HOME=~/airflow
airflow initdb
From here, I have mimicked the steps you have followed to configure MySQL Server
5) Install MySQL Server
rpm -Uvh https://repo.mysql.com/mysql80-community-release-el7-3.noarch.rpm
sed -i 's/enabled=1/enabled=0/' /etc/yum.repos.d/mysql-community.repo
yum --enablerepo=mysql80-community install mysql-server
systemctl start mysqld.service
6) Login to MySQL and configure database for Airflow
mysql> CREATE DATABASE airflow CHARACTER SET utf8 COLLATE utf8_unicode_ci;
mysql> CREATE user 'airflow'#'localhost' identified by 'Airflow123';
mysql> GRANT ALL privileges on *.* to 'airflow'#'localhost';
7) Update Airflow configuration file (~/airflow/airflow.cfg)
sql_alchemy_conn = mysql://airflow:Airflow123#localhost:3306/airflow
executor = CeleryExecutor
8) Initialize Airflow
airflow initdb
I am building an Alpine based image of a Django application with MariaDB and I can't figure out which dependency I should add to my Dockerfile so that my app could properly connect to the DB.
django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module.
Did you install mysqlclient?
Well, I thought I did. From what I read in this article, in this discussion, mariadb-dev is in Alpine the equivalent of default-libmysqlclient-dev in Debian based system. Furthermore, the mysql-client package in Alpine is merely a dummy package (containing mariadb-dev, mariadb-client, etc etc). Here is the Dockerfile:
# pull official base image
FROM python:3.7-alpine
# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# set work directory
WORKDIR /usr/src/cms
# install mysqlclient
RUN apk update \
&& apk add --virtual build-deps gcc python3-dev musl-dev \
&& apk add --no-cache mariadb-dev\
&& apk del build-deps
# install dependencies
RUN pip install --upgrade pip
RUN pip install pipenv
COPY ./Pipfile /usr/src/cms/Pipfile
RUN pipenv install --skip-lock --system --dev
# copy entrypoint.sh
COPY ./entrypoint.sh /usr/src/cms/entrypoint.sh
# copy project
COPY . /usr/src/cms/
# run entrypoint.sh
ENTRYPOINT ["/usr/src/cms/entrypoint.sh"]
I tried to add mariadb-client, to use mysql-client instead.
I also tried to add RUN pip install django-mysql. Nothing seems to change. I successfully build Postgres Django apps without any problem but, when it comes to MySQl vs MariaDB // Debian vs Alpine, I feel confused. Any insight?
It seems you're missing the MySQLdb Python module, for which you should install the mysqlclient Python package: pip install mysqlclient.
On Alpine, pip will build mysqlclient from source, so you'll need gcc and musl-dev for this setup step, hence you'll need to postpone apk del build-deps to after Python modules are installed.
Fixed Dockerfile snippet:
RUN apk update \
&& apk add --virtual build-deps gcc python3-dev musl-dev \
&& apk add --no-cache mariadb-dev
...
RUN pip install mysqlclient
RUN apk del build-deps
Mainly you need to install mariadb-connector-c-dev package. But only this package will give compilation errors. So additionally you will need to add gcc and musl-dev packages into Dockerfile. This will make Django and MySQL work within alpine image.
FROM python:3.8-alpine
RUN apk add gcc musl-dev mariadb-connector-c-dev
I'm trying to run a release build of my kitura (2.7) app with mysql on the official swift-ubuntu (latest, 5.0.1) image with the following commands.
docker build --no-cache -t my-app-build -f Dockerfile-tools .
docker run -v $PWD:/swift-project -w /swift-project my-app-build /swift-utils/tools-utils.sh build release
First command one is working as expected. Second one is giving a warning:
warning: you may be able to install mysqlclient using your system-packager: apt-get install libmysqlclient-dev
Tried to install the lib but nothing changed...
Can someone help me?
Thanks in advance!
The issue appears to be related to the version of Ubuntu and the resulting level of MySQL that is installed. As the base container is running Ubuntu 14.04 when MySQL installs you get version 5.5 which does not ship the required configuration for pkg-config to find the include paths needed to build your application.
I have been able to get a simple Kitura application which uses SwiftKueryMySQL to build under docker by updating my Dockerfile-tools file with two changes:
1) Update the FROM to:
FROM swift:5.0.1
2) Add some required packages:
# Install system level packages
RUN apt-get update && apt-get install -y sudo libcurl4-openssl-dev openssl libssl-dev pkg-config libmysqlclient-dev
With these updates your build should succeed. I will look into a longer term solution to the issue.
I am trying to install the Elastic Beanstalk CLI (awsebcli) on a fresh Ubuntu 14.04 (on Linux subsystem for Windows) using sudo pip install awsebcli, but launching the eb command just returns the following error:
flavien#XPS-FLAVIEN:~$ eb
Traceback (most recent call last):
File "/usr/local/bin/eb", line 6, in <module>
from pkg_resources import load_entry_point
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3036, in <module>
#_call_aside
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3020, in _call_aside
f(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3049, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 654, in _build_master
ws.require(__requires__)
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 968, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 854, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'termcolor==1.1.0' distribution was not found and is required by awsebcli
Any idea what might be going wrong?
You can reinstall awsebcli with below command if you have issues after installing it:
sudo pip3 install awsebcli --force-reinstall --upgrade
Once its installed check where its installed:
which eb
$ /usr/local/bin/eb #i got eb installed in this path
Next set the path:
export PATH=/usr/local/bin:$PATH
Then run
eb --version
EB CLI 3.14.6 (Python 3.5.2) #this is my installed version
First install the pip separably and try with this command
pip install --upgrade --user awsebcli
I had awsebcli being installed for Python 2.7 but for some reason running eb needed them for Python 3.x so doing this worked :
$ sudo -H pip3 install --upgrade --user awsebcli
After installing all the required wheels eb worked fine :
$ eb --version
EB CLI 3.14.3 (Python 3.5.2)
Also don't forget to add ~/.local/bin to your PATH variable in ~/.bash_profile
The previous answer helped me to figure this one out.
My detail, I had to install a newer version of python than 2.7, one that supported the --trusted-host switch to allow me to get all of my dependencies:
What I ran:
pip --cert zxroot.pem --trusted-host pypi.python.org --proxy [ProxyServer] install --upgrade --user awsebcli
Once I ran that reinstall using the --upgrade switch, eb finally worked:
eb --version
EB CLI 3.10.5 (Python 3.3.1)
I know this is very specific to my particulars but it might help someone else.
Good luck.
The problem is you are missing quite a few applications required by the scripts to compile.
The following installation steps are required prior to running the EB CLI scripts.
As I'm a windows user I created a clean VirtualBox VM install of Ubuntu 18.04.2
Update Ubuntu
sudo apt update
sudo apt upgrade
sudo reboot
Install curl,wget
sudo apt install curl
sudo apt install wget
Install zlib
sudo apt-get install zlib1g-dev
Install libffi
sudo apt-get install libffi libffi5-dev
Install OpenSSL
sudo apt-get install libssl-dev
Install gcc
sudo apt-get install build-essential
sudo apt install libx11-dev
gcc --version
make -v
Install Python 3.7
sudo apt install software-properties-common
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt install python3.7
python3.7 --version
alias python='python3.7'
Install Git
sudo apt install git
After this run the EB CLI
You should see these 5 successful progress steps
Creating exclusive virtualenv for EBCLI
Activating virtualenv
Installing EBCLI
Creating EB wrappers
Finishing up
Success!
Try This:
sudo chown -R username:username ~/.local/
# add to ./*shrc
export PATH=$PATH:~/.local/bin/
pip install --upgrade --user awsebcli
eb --version
#EB CLI 3.10.1 (Python 2.7.1)
I can't for the life of me figure this out. I feel like I'm really close, but im just not quite there yet. Here's my Dockerfile:
FROM ubuntu:14.04
MAINTAINER Oscar Godson
RUN apt-get update
RUN apt-get install -y nodejs-legacy
RUN apt-get install -y npm
RUN apt-get install -y mysql-server
RUN apt-get install -y mysql-client
COPY . /src
WORKDIR /src
RUN service mysql start;\
mysql -u root < /src/bin/demo-data.sql;\
npm install
EXPOSE 4000
CMD ["node", "index.js"]
If i go into interactive mode and go into bash in that RUN service... line at the end I can see that mysql is running, the demo data was imported and everything works including starting my Node app with node index.js. If I then try to do another RUN or the CMD you see below it no longer can access MySQL. It's like it stops running.
The error I get Error: connect ECONNREFUSED which happens when it's down or the data isn't there. From me playing with it it looks like MySQL isn't running if I try to access it after the RUN where I start it.
P.S. The MySQL DB here is purely for development. On production we use AWS RDS, so thats why its using root without any password and importing data that way.
The problem was that RUN shouldn't be used for long running processes, however, you can only use one CMD option. For some reason it didn't click with me until a friend pointed it out, but I could just make a "installer.sh" script and call that with CMD. Now it's not losing the MySQL context. My update Dockerfile looks something like this:
FROM ubuntu:14.04
MAINTAINER Oscar Godson
RUN apt-get update
RUN apt-get install -y nodejs-legacy
RUN apt-get install -y npm
RUN apt-get install -y mysql-server
RUN apt-get install -y mysql-client
COPY . /src
WORKDIR /src
RUN cd /src;npm install
EXPOSE 4000
CMD ./bin/installer.sh
Installer than has
service mysql start
mysql -u root < ./bin/demo-data.sql
node index.js