Interaction between pydantic models/schemas in fastAPI Tutorial - sqlalchemy

I follow the FastAPI Tutorial and am not quite sure what the exact relationship between the proposed data objects is.
We have the models.py file:
from sqlalchemy import Boolean, Column, ForeignKey, Integer, String
from sqlalchemy.orm import relationship
from .database import Base
class User(Base):
__tablename__ = "users"
id = Column(Integer, primary_key=True, index=True)
email = Column(String, unique=True, index=True)
hashed_password = Column(String)
is_active = Column(Boolean, default=True)
items = relationship("Item", back_populates="owner")
class Item(Base):
__tablename__ = "items"
id = Column(Integer, primary_key=True, index=True)
title = Column(String, index=True)
description = Column(String, index=True)
owner_id = Column(Integer, ForeignKey("users.id"))
owner = relationship("User", back_populates="items")
and the schemas.py file:
from typing import List, Union
from pydantic import BaseModel
class ItemBase(BaseModel):
title: str
description: Union[str, None] = None
class ItemCreate(ItemBase):
pass
class Item(ItemBase):
id: int
owner_id: int
class Config:
orm_mode = True
class UserBase(BaseModel):
email: str
class UserCreate(UserBase):
password: str
class User(UserBase):
id: int
is_active: bool
items: List[Item] = []
class Config:
orm_mode = True
Those classes are then used to define db queries like in the crud.py file:
from sqlalchemy.orm import Session
from . import models, schemas
def get_user(db: Session, user_id: int):
return db.query(models.User).filter(models.User.id == user_id).first()
def get_user_by_email(db: Session, email: str):
return db.query(models.User).filter(models.User.email == email).first()
def get_users(db: Session, skip: int = 0, limit: int = 100):
return db.query(models.User).offset(skip).limit(limit).all()
def create_user(db: Session, user: schemas.UserCreate):
fake_hashed_password = user.password + "notreallyhashed"
db_user = models.User(email=user.email, hashed_password=fake_hashed_password)
db.add(db_user)
db.commit()
db.refresh(db_user)
return db_user
def get_items(db: Session, skip: int = 0, limit: int = 100):
return db.query(models.Item).offset(skip).limit(limit).all()
def create_user_item(db: Session, item: schemas.ItemCreate, user_id: int):
db_item = models.Item(**item.dict(), owner_id=user_id)
db.add(db_item)
db.commit()
db.refresh(db_item)
return db_item
and in the FastAPI code main.py:
from typing import List
from fastapi import Depends, FastAPI, HTTPException
from sqlalchemy.orm import Session
from . import crud, models, schemas
from .database import SessionLocal, engine
models.Base.metadata.create_all(bind=engine)
app = FastAPI()
# Dependency
def get_db():
db = SessionLocal()
try:
yield db
finally:
db.close()
#app.post("/users/", response_model=schemas.User)
def create_user(user: schemas.UserCreate, db: Session = Depends(get_db)):
db_user = crud.get_user_by_email(db, email=user.email)
if db_user:
raise HTTPException(status_code=400, detail="Email already registered")
return crud.create_user(db=db, user=user)
#app.get("/users/", response_model=List[schemas.User])
def read_users(skip: int = 0, limit: int = 100, db: Session = Depends(get_db)):
users = crud.get_users(db, skip=skip, limit=limit)
return users
#app.get("/users/{user_id}", response_model=schemas.User)
def read_user(user_id: int, db: Session = Depends(get_db)):
db_user = crud.get_user(db, user_id=user_id)
if db_user is None:
raise HTTPException(status_code=404, detail="User not found")
return db_user
#app.post("/users/{user_id}/items/", response_model=schemas.Item)
def create_item_for_user(
user_id: int, item: schemas.ItemCreate, db: Session = Depends(get_db)
):
return crud.create_user_item(db=db, item=item, user_id=user_id)
#app.get("/items/", response_model=List[schemas.Item])
def read_items(skip: int = 0, limit: int = 100, db: Session = Depends(get_db)):
items = crud.get_items(db, skip=skip, limit=limit)
return items
Now from what I understand:
the models data classes define the sql tables
the schemas data classes define the api that FastAPI uses to interact with the database
they must be convertible into each other so that the set-up works
What I don't understand:
in crud.create_user_item I expected the return type to be schemas.Item, since that return type is used by FastAPI again
according to my understanding the response model of #app.post("/users/{user_id}/items/", response_model=schemas.Item) in the main.py is wrong, or how can I understand the return type inconsistency?
however inferring from the code, the actual return type must be models.Item, how is that handled by FastAPI?
what would be the return type of crud.get_user?

I'll go through your bullet points one by one.
The models data classes define the sql tables
Yes. More precisely, the ORM classes that map to actual database tables are defined in the models module.
the schemas data classes define the api that FastAPI uses to interact with the database
Yes and no. The Pydantic models in the schemas module define the data schemas relevant to the API, yes. But that has nothing to do with the database yet. Some of these schemas define what data is expected to be received by certain API endpoints for the request to be considered valid. Others define what the data returned by certain endpoints will look like.
they must be convertible into each other so that the set-up works
While the database table schemas and the API data schemas are usually very similar, that is not necessarily the case. In the tutorial however, they correspond quite neatly, which allows succinct CRUD code, like this:
db_item = models.Item(**item.dict(), owner_id=user_id)
Here item is a Pydantic model instance, i.e. one of your API data schemas schemas.ItemCreate containing data you decided is necessary for creating a new item. Since its fields (their names and types) correspond to those of the database model models.Item, the latter can be instantiated from the dictionary representation of the former (with the addition of the owner_id).
in crud.create_user_item I expected the return type to be schemas.Item, since that return type is used by FastAPI again
No, this is exactly the magic of FastAPI. The function create_user_item return an instance of models.Item, i.e. the ORM object as constructed from the database (after calling session.refresh on it). And the API route handler function create_item_for_user actually does return that same object (of class models.Item).
However, the #app.post decorator takes that object and uses it to construct an instance of the response_model you defined for that route, which is schemas.Item in this case. This is why you set this in your schemas.Item model:
class Config:
orm_mode = True
This allows an instance of that class to be created via the .from_orm method. That all happens behind the scenes and again depends on the SQLAlchemy model corresponding to the Pydantic model with regards to field names and types. Otherwise validation fails.
according to my understanding the response model [...] is wrong
No, see above. The decorated route function actually deals with the schemas.Item model.
however inferring from the code, the actual return type must be models.Item
Yes, see above. The return type of the undecorated route handler function create_item_for_user is in fact models.Item. But its return type is not the response model.
I assume that to reduce confusion the documentation example does not annotate the return type of those route functions. If it did, it would look like this:
#app.post("/users/{user_id}/items/", response_model=schemas.Item)
def create_item_for_user(
user_id: int, item: schemas.ItemCreate, db: Session = Depends(get_db)
) -> models.Item:
return crud.create_user_item(db=db, item=item, user_id=user_id)
It may help to remember that a decorator is just syntactic sugar for a function that takes a callable (/function) as argument and returns a function. Typically the returned function actually internally calls the function passed to it as argument and does additional things before and/or after that call. I could rewrite the route above like this and it would be exactly the same:
def create_item_for_user(
user_id: int, item: schemas.ItemCreate, db: Session = Depends(get_db)
) -> models.Item:
return crud.create_user_item(db=db, item=item, user_id=user_id)
create_item_for_user = app.post(
"/users/{user_id}/items/", response_model=schemas.Item
)(create_item_for_user)
what would be the return type of crud.get_user?
That would be models.User because that is the database model and is what the first method of that query returns.
db.query(models.User).filter(models.User.id == user_id).first()
This is then again returned by the read_user API route function in the same fashion as I explained above for models.Item.
Hope this helps.

Related

Serialization by pydantic in fastapi

Relation in sqlalchemy
I'll ask this question separately, since it's a continuation of this one, but with a different content.
I want to serialize a join and then serialize it with pydantic, but it doesn't work.
If possible, I want to return it as if it were a single object.
{
id: 1,
parent_column: test1,
child_column: test2
}
You can create a Pydantic model with the information you want, so the model will take care of the serialization to JSON format.
from pydantic import BaseModel
class MyResponse(BaseModel):
id: int
parent: str
child: str
You just have to create a response from your model by providing it with the data in the requested format.
data_json = MyResponse(id= my_data.id, parent=my_data.parent, child=my_data.child)
Below is an example given with a fastAPI endpoint that would take an ID to retrieve a specific data and return a JSON response with a specific template.
from pydantic import BaseModel
from fastapi import status
class MyResponse(BaseModel):
id: int
parent: str
child: str
# router.get('/data',
status_code=status.HTTP_200_OK,
response_model=MyResponse)
def get_data(parent_id:int):
try:
data = get_data(parent_id)
except Exception:
raise HTTPException(status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,detail="Server Error")
return MyResponse(id=data.id, parent=data.parent, child=data.child)
Afterwards, it is possible to link Pydantic models with sqlalchemy via the orm_mode of the Pydantic class. More information here: https://fastapi.tiangolo.com/tutorial/sql-databases/#use-pydantics-orm_mode

SQLAlchemy 1.4 abstracting MetaData.reflect into function not returning MetaData object?

I have a class that acts as a PostgreSQL database interface. It has a number of methods which do things with the MetaData, such as get table names, drop tables, etc.
These methods keep calling the same two lines to set up MetaData. I am trying to tidy this up by abstracting this MetaData setup into its own function which is initiated when the class is instantiated, but this isn't working, as the function keeps returning NoneType instead of the MetaData instance.
Here is an example of the class, BEFORE adding the MetaData function:
class Db:
def __init__(self, config):
self.engine = create_async_engine(ENGINE, echo=True, future=True)
self.session = sessionmaker(self.engine, expire_on_commit=False, class_=AsyncSession)
def get_table_names(self):
meta = MetaData()
meta.reflect(bind=sync_engine)
meta = self.meta()
return meta.tables.keys()
This works well, returns a list of table keys:
dict_keys(['user', 'images', 'session'])
When I try to shift the MetaData call into its own function like so:
class Db:
def __init__(self, config):
self.engine = create_async_engine(ENGINE, echo=True, future=True)
self.session = sessionmaker(self.engine, expire_on_commit=False, class_=AsyncSession)
self.meta = self.get_metadata()
def get_metadata(self):
meta = MetaData()
return meta.reflect(bind=sync_engine)
def get_table_names(self):
return self.meta.tables.keys()
It returns this error:
in get_table_names return self.meta.tables.keys()
AttributeError: 'NoneType' object has no attribute 'tables'
How can I achieve this sort of functionality by calling self.meta() from within the various class methods?
Reflect alters the current metadata in-place. So you can just return the meta variable explicitly.
class Db:
# ...
def get_metadata(self):
meta = MetaData()
meta.reflect(bind=sync_engine)
return meta
# ...
Although it might be better to do this in a factory function, like def db_factory(config): and inject these things already prepped in the class constructor, def __init__(self, metadata, engine, session):. Just a thought.
Just wanted to post an answer, as with someone else's help I was able to solve this. The code should look like this:
class Db:
def __init__(self, config):
self.engine = create_async_engine(ENGINE, echo=True, future=True)
self.session = sessionmaker(self.engine, expire_on_commit=False, class_=AsyncSession)
self._meta = MetaData()
#property
def meta(self):
self._meta.reflect(bind=sync_engine)
return self._meta
def get_table_names(self):
return self.meta.tables.keys()

Django throws error on model field for wrong database

I have two databases set up for my Django project, a MySQL and a Postgres.
I have a database router in place and everything has been working fine, until I changed a CharField with max_length of 300 in a Postgres model to unique.
That model is in Postgres, but now Django throws this error about MySQL:
[postgres_app.TestModel.slug]: (mysql.E001) MySQL does not allow unique CharFields to have a max_length > 255.
I double checked my database router and everything looked fine. I also have to mention again the router was doing its job correctly before and picked the correct database for read, write, and migration. Here is my router:
POSTGRES_APPS = ['postgres_app']
class PostgresRouter(object):
""" Controls which apps can have access to the postgres instance. """
def db_for_read(self, model, **hints):
""" Called on every read action on a model. """
if model._meta.app_label in POSTGRES_APPS:
return 'postgres'
return None
def db_for_write(self, model, **hints):
""" Called on every write action on a model. """
if model._meta.app_label in POSTGRES_APPS:
return 'postgres'
return None
def allow_relation(self, obj1, obj2, **hints):
""" Called only on migrations. Only allow relations if both models are in POSTGRES_APPS. """
return {obj1._meta.app_label, obj2._meta.app_label}.issubset(set(POSTGRES_APPS)) or None
def allow_migrate(self, db, app_label, model_name=None, **hints):
""" Called only on migrations. """
if db == 'postgres':
return app_label in POSTGRES_APPS
else:
return app_label not in POSTGRES_APPS
and this is my model:
class TestModel(models.Model):
slug = models.CharField(max_length=300, blank=True, null=True, unique=True)
So the question is whether it is a bug in Django or not, and if not, then how can I tell Django it is looking at the wrong database.

django rest framework - request context key error

I have two models (Like and News). I am using django-rest-framework to make a web api out of it.
class Like(models.Model):
user = models.ForeignKey(User)
content_type = models.ForeignKey(ContentType)
object_id = models.PositiveIntegerField()
content_object = GenericForeignKey('content_type', 'object_id')
class News(models.Model):
user = models.ForeignKey(User)
title = models.CharField(max_length=150)
...
likes = GenericRelation(Like)
A Like object has its own user field to store who liked the News object. Now to check if a specific user exists in any of the likes of a News object, I am getting request.user from a SerializerMethodField.
class NewsSerializer(serializers.HyperlinkedModelSerializer):
user = UserSerializer()
likes_count = serializers.IntegerField(source='likes.count', read_only=True)
user_in_likes = serializers.SerializerMethodField()
class Meta:
model = News
fields = ('user', 'title', 'body', 'article_image', 'pub_date', 'likes_count', 'user_in_likes')
def get_user_in_likes(self, obj):
user = self.context['request'].user
what = obj.likes.filter(user=user).exists()
return what
When I go the /news/ url, I get the json objects including the user_in_likes to true/false for each news object.
However, I have another serialzer for different model which imports NewsSerializer class and other similar serializers:
class ActivityObjectRelatedField(serializers.RelatedField):
def to_representation(self, value):
if isinstance(value, User):
serializer = UserSerializer(value)
elif isinstance(value, Job):
serializer = JobSerializer(value)
elif isinstance(value, News):
serializer = NewsSerializer(value)
elif isinstance(value, Tender):
serializer = TenderSerializer(value)
else:
raise Exception('Unexpected type of tagged object')
return serializer.data
class ActivitySerializer(serializers.HyperlinkedModelSerializer):
actor = ActivityObjectRelatedField(read_only=True)
target = ActivityObjectRelatedField(read_only=True)
class Meta:
model = Activity
fields = ('url', 'actor', 'verb', 'target', 'pub_date')
Now if I visit /activities/, to get the activities objects I am getting an error:
KeyError at /activities/
'request'
And it points to the line of SerializerMethod of NewsSerialize class where self.context['request'].user is used.
Exception Location: /vagrant/myproject/news/serializers.py in get_user_in_likes, line 25
Again if I visit /news/ url, everything is fine and I get news objects. What am I missing here? Why is request not being recognized in the ActivitiesSerializer class? Please help me solve this problem. Thank you.
You are getting this error because you are not passing request in the context when instantiating the NewsSerializer class or other similar serializers in the to_representation() method.
You are manually instantiating the particular serializer class in to_representation() method. So, after instantiation, that particular serializer does not have access to ActivitySerializer's context thereby leading to the error.
You can pass the ActivitySerializer's context during instantiation of the relevant serializer in the to_representation() method.
class ActivityObjectRelatedField(serializers.RelatedField):
def to_representation(self, value):
if isinstance(value, User):
serializer = UserSerializer(value, context=self.context) # pass context
elif isinstance(value, Job):
serializer = JobSerializer(value, context=self.context) # pass context
elif isinstance(value, News):
serializer = NewsSerializer(value, context=self.context) # pass context
elif isinstance(value, Tender):
serializer = TenderSerializer(value, context=self.context) # pass context
else:
raise Exception('Unexpected type of tagged object')
return serializer.data
It seems like you don't populate the context dictionary of NewsSerializer with your request in the /activities/ view.
You probably use a class based view provided by Django REST Framework which populates this dictionary for you (see the get_serializer_context() method) and passes it to the Serializer instance. That's why it's automatically available to your serializer in your /news/ view.
In your /activities/ view, though, the context is passed to ActivitySerializer and isn't (automatically) propagated further from there. That's why there's no request key in your context dictionary of NewsSerializer. You would need to pass your request object to the NewsSerializer instance. To pass extra context to a Serializer you can add a context parameter containing a dictionary when instantiating (see the DRF documentation).

RESTful interface in Flask and issues serializing

I'm learning Backbone.js and Flask (and Flask-sqlalchemy). I chose Flask because I read that it plays well with Backbone implementing RESTful interfaces. I'm currently following a course that uses (more or less) this model:
class Tasks(db.Model):
id = db.Column(db.Integer, primary_key=True)
title = db.Column(db.String(80), unique=True)
completed = db.Column(db.Boolean, unique=False, default=False)
def __init__(self, title, completed):
self.title = title
self.completed = completed
def json_dump(self):
return dict(title=self.title, completed=self.completed)
def __repr__(self):
return '<Task %r>' % self.title
I had to add a json_dump method in order to send JSON to the browser. Otherwise, I would get errors like object is not JSON serializable, so my first question is:
Is there a better way to do serialization in Flask? It seems that some objects are serializable but others aren't, but in general, it's not as easy as I expected.
After a while, I ended up with the following views to take care of each type of request:
#app.route('/tasks')
def tasks():
tasks = Tasks.query.all()
serialized = json.dumps([c.json_dump() for c in tasks])
return serialized
#app.route('/tasks/<id>', methods=['GET'])
def get_task(id):
tasks = Tasks.query.get(int(id))
serialized = json.dumps(tasks.json_dump())
return serialized
#app.route('/tasks/<id>', methods=['PUT'])
def put_task(id):
task = Tasks.query.get(int(id))
task.title = request.json['title']
task.completed = request.json['completed']
db.session.add(task)
db.session.commit()
serialized = json.dumps(task.json_dump())
return serialized
#app.route('/tasks/<id>', methods=['DELETE'])
def delete_task(id):
task = Tasks.query.get(int(id))
db.session.delete(task)
db.session.commit()
serialized = json.dumps(task.json_dump())
return serialized
#app.route('/tasks', methods=['POST'])
def post_task():
task = Tasks(request.json['title'], request.json['completed'])
db.session.add(task)
db.session.commit()
serialized = json.dumps(task.json_dump())
return serialized
In my opinion, it seems a bit verbose. Again, what is the proper way to implement them? I have seen some extensions that offer RESTful interfaces in Flask but those look quite complex to me.
Thanks
I would use a module to do this, honestly. We've used Flask-Restless for some APIs, you might take a look at that:
https://flask-restless.readthedocs.org/en/latest/
However, if you want build your own, you can use SQLAlchemy's introspection to output your objects as key/value pairs.
http://docs.sqlalchemy.org/en/rel_0_7/core/schema.html#metadata-reflection
Something like this, although I always have to triple-check I got the syntax right, so take this as a guide more than working code.
#app.route('/tasks')
def tasks():
tasks = Tasks.query.all()
output = []
for task in tasks:
row = {}
for field in Tasks.__table__.c:
row[str(field)] = getattr(task, field, None)
output.append(row)
return jsonify(data=output)
I found this question which might help you more. I'm familiar with SQLAlchemy 0.7 and it looks like 0.8 added some nicer introspection techniques:
SQLAlchemy introspection
Flask provides jsonify function to do this. Check out its working here.
Your json_dump method is right though code can be made concise. See this code snippet
#app.route('/tasks')
def tasks():
tasks = Tasks.query.all()
return jsonify(data=[c.json_dump() for c in tasks])