django AWS EB CLI Unable to deploy with mysqlclient - mysql

Question

I have been attempting to deploy my Django project up on AWS ElasticBeanstalk via EB CLI n my local Pycharm project with the built in terminal.
Having setup IAM/group (through aws web console) I migrated and ran server locally (Pycharm terminal) and all works fine, and then created env/instance and requirements.txt and deployed using EB CLI in Pycharm and this also worked fine.
I setup a database connection on RDS (AWS web Console) then changed settings to connect to mysql, pip installed mysqlclient, migrate and ran locally (Pycharm terminal) and again all works fine.
However, when I try to deply the project again with mysqlclient installed I get errors and I just cannot figure out why.
Platform: arn:aws:elasticbeanstalk:us-west-2::platform/Python 3.7 running on 64bit Amazon Linux 2/3.3.7
Requirements.txt
asgiref==3.4.1
cement==2.8.2
certifi==2021.10.8
charset-normalizer==2.0.7
colorama==0.4.3
Django==3.2.9
django-storages==1.12.3
future==0.16.0
idna==3.3
jmespath==0.10.0
mysqlclient
pathspec==0.5.9
python-dateutil==2.8.2
pytz==2021.3
PyYAML==5.4.1
requests==2.26.0
s3transfer==0.5.0
semantic-version==2.8.5
six==1.14.0
sqlparse==0.4.2
termcolor==1.1.0
typing-extensions==3.10.0.2
urllib3==1.26.7
wcwidth==0.1.9
.ebextensions/django.config
option_settings:
aws:elasticbeanstalk:container:python:
WSGIPath: awsdeploy.wsgi:application
aws:elasticbeanstalk:environment:proxy:staticfiles:
/static: static
container_commands:
01_collectstatic:
command: "source /var/app/venv/staging-LQM1lest/bin/activate && python manage.py collectstatic --noinput --clear"
02_migrate:
command: "source /var/app/venv/staging-LQM1lest/bin/activate && python manage.py migrate --noinput"
leader_only: true
.ebextensions/packages.config
packages:
yum:
python3-devel: []
mariadb-devel: []
EB Error Logs
Collecting mysqlclient==2.0.3
Using cached mysqlclient-2.0.3.tar.gz (88 kB)
2021/11/11 02:10:40.712344 [ERROR] An error occurred during execution of command [app-deploy] - [InstallDependency]. Stop running the command. Error: fail to install dependencies with requirements.txt file with error Command /bin/sh -c /var/app/venv/staging-LQM1lest/bin/pip install -r requirements.txt failed wi
th error exit status 1. Stderr: ERROR: Command errored out with exit status 1:
command: /var/app/venv/staging-LQM1lest/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-isj_ea8s/mysqlclient_f221eb2ed3c44d6e868da4539588d2a4/setup.py'"'"'; __file__='"'"'/tmp/pip-install-isj_ea8s/mysqlclient_f221eb2ed3c44d6e868da4539588d2a4/setup.py'"'"';f = ge
tattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-hzv5pj6h
cwd: /tmp/pip-install-isj_ea8s/mysqlclient_f221eb2ed3c44d6e868da4539588d2a4/
Complete output (15 lines):
/bin/sh: mysql_config: command not found
/bin/sh: mariadb_config: command not found
/bin/sh: mysql_config: command not found
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-isj_ea8s/mysqlclient_f221eb2ed3c44d6e868da4539588d2a4/setup.py", line 15, in <module>
metadata, options = get_config()
File "/tmp/pip-install-isj_ea8s/mysqlclient_f221eb2ed3c44d6e868da4539588d2a4/setup_posix.py", line 70, in get_config
libs = mysql_config("libs")
File "/tmp/pip-install-isj_ea8s/mysqlclient_f221eb2ed3c44d6e868da4539588d2a4/setup_posix.py", line 31, in mysql_config
raise OSError("{} not found".format(_mysql_config_path))
OSError: mysql_config not found
mysql_config --version
mariadb_config --version
mysql_config --libs
----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/3c/df/59cd2fa5e48d0804d213bdcb1acb4d08c403b61c7ff7ed4dd4a6a2deb3f7/mysqlclient-2.0.3.tar.gz#sha256=f6ebea7c008f155baeefe16c56cd3ee6239f7a5a9ae42396c2f1860f08a7c432 (from https://pypi.org/simple/mysqlclient/) (requires-python:>=3.5). Command errored out
with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement mysqlclient==2.0.3 (from versions: 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.5, 1.3.6, 1.3.7, 1.3.8, 1.3.9, 1.3.10, 1.3.11rc1, 1.3.11, 1.3.12, 1.3.13, 1.3.14, 1.4.0rc1, 1.4.0rc2, 1.4.0rc3, 1.4.0, 1.4.1, 1.4.2, 1.4.2.post1, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 2
.0.0, 2.0.1, 2.0.2, 2.0.3, 2.1.0rc1)
ERROR: No matching distribution found for mysqlclient==2.0.3
WARNING: You are using pip version 21.2.4; however, version 21.3.1 is available.
You should consider upgrading via the '/var/app/venv/staging-LQM1lest/bin/python -m pip install --upgrade pip' command.
2021/11/11 02:10:40.712360 [INFO] Executing cleanup logic
2021/11/11 02:10:40.712489 [INFO] CommandService Response: {"status":"FAILURE","api_version":"1.0","results":[{"status":"FAILURE","msg":"Engine execution has encountered an error.","returncode":1,"events":[{"msg":"Instance deployment failed to install application dependencies. The deployment failed.","timestamp
":1636596640,"severity":"ERROR"},{"msg":"Instance deployment failed. For details, see 'eb-engine.log'.","timestamp":1636596640,"severity":"ERROR"}]}]}
2021/11/11 02:10:40.712674 [INFO] Platform Engine finished execution on command: app-deploy
I don't understand why prior to installing mysqlclient it's managed to deploy install all my packages from requirements file successfully but fails when mysqlclient was added - what am I missing?
Help is much appreciated, happy to share any other details if needed. Thanks
Edit
I came across the answer on this link: [https://stackoverflow.com/questions/67344766/error-encountered-while-installing-dependencies-for-flask-mysql-application-usin][1]
And created the .ebextensions/packages.config as suggested and removed version from mysqlclient within my requirements.txt file - updated above code.
Now mysqlclient appears to have installed succssfully but now get the following error:
New EB Error Logs
An error occurred during execution of command [app-deploy] - [PostBuildEbExtension]. Stop running the command. Error: container commands build failed. Please refer to /var/log/cfn-init.log for more details.
I'm also not sure how to check this log /var/log/cfn-init.log, when I go to the web console I can't see any eb instances listed or any logs.