airflow 2.2.3 sqlalchemy compatible libraries - sqlalchemy

Question

Which of the following library versions work well with Airflow 2.2.3 ?
Tried few options but none worked. Fails when initializing db
solution: downgrading no result
pip install Flask==1.0.4
pip3 install marshmallow-sqlalchemy==0.17.1
solution: no result
pip3 install marshmallow-sqlalchemy==0.17.1
pip3 install SQLAlchemy==1.3.23
pip3 install flask-sqlalchemy
errors:
flask-appbuilder 3.4.3 requires marshmallow-sqlalchemy<0.27.0,>=0.22.0, but you'll have marshmallow-sqlalchemy 0.17.1 which is incompatible.
apache-airflow 2.2.3 requires flask<2.0,>=1.1.0, but you'll have flask 1.0.4 which is incompatible.
apache-airflow 2.2.3 requires werkzeug>=1.0.1,~=1.0, but you'll have werkzeug 0.16.0 which is incompatible.
higher versions when used produuce different error.
- name: "pip : ensure correct werkzeug package present"
pip:
name: "werkzeug"
version: "{{ airflow_werkzeug_version | default('0.16.0') }}"
state: present
virtualenv: "{{ airflow_venv_path }}"
virtualenv_command: "{{ airflow_venv_cmd }}"
environment:
AIRFLOW_GPL_UNIDECODE: "yes"
SLUGIFY_USES_TEXT_UNIDECODE: "yes"
become: true
become_user: "{{ airflow_username }}"
- name: "venv - ensure python sqlalchemy dependencies "
pip:
name: "Flask-SQLAlchemy"
version: "2.4.4"
state: present
virtualenv: "{{ airflow_venv_path }}"
virtualenv_command: "{{ airflow_venv_cmd }}"
environment:
AIRFLOW_GPL_UNIDECODE: "yes"
SLUGIFY_USES_TEXT_UNIDECODE: "yes"
become: true
become_user: "{{ airflow_username }}"
- name: "venv - ensure python sqlalchemy dependencies "
pip:
name: "SQLAlchemy"
version: "1.3.23"
state: present
virtualenv: "{{ airflow_venv_path }}"
virtualenv_command: "{{ airflow_venv_cmd }}"
environment:
AIRFLOW_GPL_UNIDECODE: "yes"
SLUGIFY_USES_TEXT_UNIDECODE: "yes"
become: true
become_user: "{{ airflow_username }}"
- name: "venv - ensure python sqlalchemy dependencies "
pip:
name: "wtforms"
version: "2.3.3"
state: present
virtualenv: "{{ airflow_venv_path }}"
virtualenv_command: "{{ airflow_venv_cmd }}"
environment:
AIRFLOW_GPL_UNIDECODE: "yes"
SLUGIFY_USES_TEXT_UNIDECODE: "yes"
become: true
become_user: "{{ airflow_username }}"

Answer 1

Look at constraint files. You do not have to think about the right versions of dependencies as long as you use golden set of constraints which are generated by Airflow maintainers with every release.
See Airflow documentation about this:
https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html
You should use constraints when you install airflow - this way the right versions will be automatically found and installed by PIP.
But you can also - if you want manually set those dependencies or even automate version retrieval. For example here you have the set of "golden" constraints for Airflow 2.2.2 and Python 3.7:
https://raw.githubusercontent.com/apache/airflow/constraints-2.2.3/constraints-3.7.txt