What is the difference between Uvicorn and Gunicorn+Uvicorn? - gunicorn

Question

What is the difference between deploying FastAPI apps dockerized using Uvicorn and Tiangolo's Gunicorn+Uvicorn? And why do my results show that I get a better result when deploying only using Uvicorn than Gunicorn+Uvicorn?
When I searched in Tiangolo's documentation, it says:
You can use Gunicorn to manage Uvicorn and run multiple of these concurrent processes. That way, you get the best of concurrency and parallelism.
From this, can I assume that using this Gunicorn will get a better result?
This is my testing using JMeter. I deployed my script to Google Cloud Run, and this is the result:
Using Python and Uvicorn:
Using Tiangolo's Gunicorn+Uvicorn:
This is my Dockerfile for Python (Uvicorn):
FROM python:3.8-slim-buster
RUN apt-get update --fix-missing
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y libgl1-mesa-dev python3-pip git
RUN mkdir /usr/src/app
WORKDIR /usr/src/app
COPY ./requirements.txt /usr/src/app/requirements.txt
RUN pip3 install -U setuptools
RUN pip3 install --upgrade pip
RUN pip3 install -r ./requirements.txt --use-feature=2020-resolver
COPY . /usr/src/app
CMD ["python3", "/usr/src/app/main.py"]
This is my Dockerfile for Tiangolo's Gunicorn+Uvicorn:
FROM tiangolo/uvicorn-gunicorn-fastapi:python3.8-slim
RUN apt-get update && apt-get install wget gcc -y
RUN mkdir -p /app
WORKDIR /app
COPY ./requirements.txt /app/requirements.txt
RUN python -m pip install --upgrade pip
RUN pip install --no-cache-dir -r /app/requirements.txt
COPY . /app
You can see the error from Tiangolo's Gunicorn+Uvicorn. Is it caused by Gunicorn?
Edited.
So, in my case, I using lazy load method to load my Machine Learning model. This is my class to load the model.
class MyModelPrediction:
# init method or constructor
def __init__(self, brand):
self.brand = brand
# Sample Method
def load_model(self):
pathfile_model = os.path.join("modules", "model/")
brand = self.brand.lower()
top5_brand = ["honda", "toyota", "nissan", "suzuki", "daihatsu"]
if brand not in top5_brand:
brand = "ex_Top5"
with open(pathfile_model + f'{brand}_all_in_one.pkl', 'rb') as file:
brand = joblib.load(file)
else:
with open(pathfile_model + f'{brand}_all_in_one.pkl', 'rb') as file:
brand = joblib.load(file)
return brand
And, this is my endpoint for my API.
#router.post("/predict", response_model=schemas.ResponsePrediction, responses={422: schemas.responses_dict[422], 400: schemas.responses_dict[400], 500: schemas.responses_dict[500]}, tags=["predict"], response_class=ORJSONResponse)
async def detect(
*,
# db: Session = Depends(deps.get_db_api),
car: schemas.Car = Body(...),
customer_id: str = Body(None, title='Customer unique identifier')
) -> Any:
"""
Predict price for used vehicle.\n
"""
global list_detections
try:
start_time = time.time()
brand = car.dict()['brand']
obj = MyModelPrediction(brand)
top5_brand = ["honda", "toyota", "nissan", "suzuki", "daihatsu"]
if brand not in top5_brand:
brand = "non"
if usedcar.price_engine_4w[brand]:
pass
else:
usedcar.price_engine_4w[brand] = obj.load_model()
print("Load success")
elapsed_time = time.time() - start_time
print(usedcar.price_engine_4w)
print("ELAPSED MODEL TIME : ", elapsed_time)
list_detections = await get_data_model(**car.dict())
if list_detections is None:
result_data = None
else:
result_data = schemas.Prediction(**list_detections)
result_data = result_data.dict()
except Exception as e: # noqa
raise HTTPException(
status_code=500,
detail=str(e),
)
else:
if result_data['prediction_price'] == 0:
raise HTTPException(
status_code=400,
detail="The system cannot process your request",
)
else:
result = {
'code': 200,
'message': 'Successfully fetched data',
'data': result_data
}
return schemas.ResponsePrediction(**result)

Answer 1

Gunicorn is an application server supports the WSGI standard. This means that Gunicorn can serve applications written in frameworks such as Flask or Django (more so for versions released before 2021). The way it works is that it creates and maintains their operability a configurable number of application instances (aka workers) that serve requests from clients. Gunicorn itself is not compatible with FastAPI because FastAPI uses the fresh ASGI standard.
Uvicorn is an app server supports the ASGI protocol. However, it's capabilities as a process (workers) manager leave much to be desired.
But Uvicorn has a Gunicorn-compatible worker class.
Using that combination, Gunicorn would act as a process manager, listening on the port and the IP. And it would transmit the communication to the worker processes running the Uvicorn class.
And then the Gunicorn-compatible Uvicorn worker class would be in charge of converting the data sent by Gunicorn to the ASGI standard for FastAPI to use it.
If you have a cluster of machines with Kubernetes, Docker Swarm or another similar complex system to manage distributed containers on multiple machines, then you will probably want to handle replication at the cluster level instead of using a process manager (like Gunicorn with workers) in each container.
One of those distributed container management systems like Kubernetes normally has some integrated way of handling replication of containers while still supporting load balancing for the incoming requests. All at the cluster level.
In those cases, you would probably want to build a Docker image from scratch, installing your dependencies, and running a single Uvicorn process instead of running something like Gunicorn with Uvicorn workers.