I am developing above database with many to many relationship and a additionally I need add in intermediate table a relationship many to one but I can't get this las relationship with others.
What's a proper way to define many-to-many relationships in a pydantic model with extra data as relationship.
models.py:
class Devices(Base):
__tablename__ = "devices"
id = Column(Integer, primary_key=True, unique=True, index=True)
name = Column(String(255))
description = Column(String(255), nullable=True)
status_id = Column(Integer, ForeignKey('status.id'))
status = relationship("Status", backref="devices")
protocols = relationship("Protocols", secondary="device_protocols", back_populates='device')
class Status(Base):
__tablename__ = "status"
id = Column(Integer, primary_key=True, unique=True, index=True)
name = Column(String(255))
description = Column(String(255), nullable=True)
class Protocols(Base):
__tablename__ = "protocols"
id = Column(Integer, primary_key=True, unique=True, index=True)
name = Column(String(255))
device = relationship("Devices", secondary="device_protocols", back_populates='protocols')
class DeviceProtocols(Base):
__tablename__ = "device_protocols"
device_id = Column(Integer, ForeignKey('devices.id'), primary_key=True)
protocol_id = Column(Integer, ForeignKey('protocols.id'), primary_key=True)
protocol_status_id = Column(Integer, ForeignKey('status.id'), nullable=True)
protocol_status = relationship("Status", backref="protocol_status")
Schemas:
class DeviceBase (BaseModel):
name: str
class Config:
orm_mode = True
class DeviceRead (DeviceBase):
id: str
description: str | None = None
status: StatusReadSimple | None = None
protocols: list[ProtocolSimple]
class ProtocolBase (BaseModel):
name: str
class Config:
orm_mode = True
class ProtocolSimple(ProtocolBase):
id: str
class StatusBase (BaseModel):
name: str
description: str | None = None
class Config:
orm_mode = True
class StatusReadSimple(StatusBase):
id: str
How do I need to develop the schemas so that the device returns the intermediate table with the protocol and its status?
Actual response:
{
"name": "device1",
"id": "3",
"description": "my device",
"status": {
"name": "OK",
"description": "Connection OK",
"id": "1"
},
"protocols": [
{
"name": "ethernet",
"id": "1"
},
{
"name": "ethercat",
"id": "2"
}
]
}
Expected response or similar:
{
"name": "device1",
"id": "3",
"description": "my device",
"status": {
"name": "OK",
"description": "Connection OK",
"id": "1"
},
"protocols": [
{
"protocol:"{
"name": "ethernet",
"id": "1"
},
"protocol_status":{
"id":1,
"name": "OK"
}
},
{
"protocol:"{
"name": "ethercat",
"id": "2"
},
"protocol_status":{
"id":2,
"name": "NOK"
}
}
]
}
TL;DR
Use a direct relationship from device to device_protocol, alias the protocols field on your Pydantic model with the name of that relationship and annotate it with a corresponding device_protocol Pydantic model. Then add protocol and status relationships to the device_protocol models (DB and Pydantic).
Pre
Your naming is a bit inconsistent, especially regarding singular/plural forms. In the following I am going to use slightly different table names (e.g. Device instead of Devices) and field names (status instead of protocol_status) that I think make more sense semantically.
I will also rename DeviceProtocols to DeviceProtocolAssociation to be very clear about the purpose of this table. You'll see why later.
Also, you should avoid using the legacy backref parameter.
Problem: Missing ORM relationship fields
This is not something that I would try to solve purely via the Pydantic models. It is possible (e.g. via complex custom validators), but I would suggest that there is a much more elegant way to do this, if we adjust the database models (specifically their relationship attributes) instead.
The problem is that your Device database model does not have an attribute capturing the intermediary DeviceProtocolAssociation model directly. You only set that table (device_protocol) as the secondary argument to get direct access to the related Protocol instances via protocol.
But if you want a Device to have an attribute listing the associated Protocols as well as the Status of those Device-Protocol pairs, you actually do need a relationship attribute to that intermediary DeviceProtocolAssociation model. After all, that model/table is exactly where that connection is made.
Then, in your DeviceProtocolAssociation you'll have to add a protocol relationship to the associated Protocol, just like you have a status relationship to the associated Status.
Solution
To make this work, and be consistent with what you already have, you'll need a few additional components.
relationship from Device directly to DeviceProtocolAssociation
Let's call this attribute device_protocol_associations. Since one Device can have many DeviceProtocolAssociations, that will be a list of instances of that model.
relationship from DeviceProtocolAssociation to Protocol
We'll call that protocol and it will just hold an instance of the associated Protocol.
association_proxy from Device to DeviceProtocolAssociation.protocol
Instead of having a relationship from Device to Protocol (via the secondary parameter), we will define an association proxy to the protocol field of each associated DeviceProtocolAssociation via the new device_protocol_associations relationship. We will call that attribute protocols to replace the old attribute. It will work the same as before.
Optionally do the same with the Protocol/Status fields
Depending on whether or not that is useful for you, you could again define a direct relationship from Protocol to DeviceProtocolAssociation and add an association proxy called devices. And you could do that for Status too.
New model definitions
from sqlalchemy import Column, ForeignKey, Integer, String
from sqlalchemy.ext.associationproxy import association_proxy
from sqlalchemy.orm import declarative_base, relationship
Base = declarative_base()
class DeviceProtocolAssociation(Base):
__tablename__ = "device_protocol"
device_id = Column(Integer, ForeignKey("device.id"), primary_key=True)
device = relationship("Device", back_populates="device_protocol_associations")
protocol_id = Column(Integer, ForeignKey("protocol.id"), primary_key=True)
protocol = relationship("Protocol", back_populates="device_protocol_associations")
status_id = Column(Integer, ForeignKey("status.id"), nullable=True)
status = relationship("Status", back_populates="device_protocol_associations")
class Device(Base):
__tablename__ = "device"
id = Column(Integer, primary_key=True)
name = Column(String(255))
status_id = Column(Integer, ForeignKey("status.id"))
status = relationship("Status", back_populates="devices")
device_protocol_associations = relationship(DeviceProtocolAssociation, back_populates="device")
protocols = association_proxy("device_protocol_associations", "protocol")
class Protocol(Base):
__tablename__ = "protocol"
id = Column(Integer, primary_key=True)
name = Column(String(255))
device_protocol_associations = relationship(DeviceProtocolAssociation, back_populates="protocol")
devices = association_proxy("device_protocol_associations", "device")
class Status(Base):
__tablename__ = "status"
id = Column(Integer, primary_key=True)
name = Column(String(255))
devices = relationship("Device", back_populates="status")
device_protocol_associations = relationship(DeviceProtocolAssociation, back_populates="status")
To create some test data for demo purposes, I'll add the following function underneath:
def create_test_data() -> Device:
status_ok = Status(id=1, name="OK")
device = Device(id=42, name="device1", status=status_ok)
device.device_protocol_associations.append(
DeviceProtocolAssociation(
protocol=Protocol(id=1, name="ethernet"),
status=status_ok,
)
)
device.device_protocol_associations.append(
DeviceProtocolAssociation(
protocol=Protocol(id=2, name="ethercat"),
status=Status(id=69, name="Not OK"),
)
)
return device
Adjusted Pydantic models
This is again just to demonstrate. You can transfer this to the inheritance structure you already have.
from __future__ import annotations
from pydantic import BaseModel as _BaseModel, Field
class BaseModel(_BaseModel):
class Config:
orm_mode = True
class ProtocolStatusModel(BaseModel):
"""Corresponds to `DeviceProtocolAssociation`, but no need for `device`"""
protocol: ProtocolModel
status: StatusModel
class DeviceModel(BaseModel):
id: int
name: str
status: StatusModel
protocols: list[ProtocolStatusModel] = Field(alias="device_protocol_associations")
class ProtocolModel(BaseModel):
id: int
name: str
class StatusModel(BaseModel):
id: int
name: str
As you can see, the DeviceModel has a customized protocols field with an alias of device_protocol_associations. That means when we parse a Device ORM object, that field will get its value from the device_protocol_associations list.
Demo
ProtocolStatusModel.update_forward_refs()
DeviceModel.update_forward_refs()
def main() -> None:
db_device = create_test_data()
output_device = DeviceModel.from_orm(db_device)
print(output_device.json(indent=4))
if __name__ == "__main__":
main()
Output
{
"id": 42,
"name": "device1",
"status": {
"id": 1,
"name": "OK"
},
"protocols": [
{
"protocol": {
"id": 1,
"name": "ethernet"
},
"status": {
"id": 1,
"name": "OK"
}
},
{
"protocol": {
"id": 2,
"name": "ethercat"
},
"status": {
"id": 69,
"name": "Not OK"
}
}
]
}
Related
I am trying to create an endpoint where, having a User entity, I can add / remove existing Group entities to user.groups many-to-many field. But when I try to do it, django-rest-framework tries to create new group objects instead of finding existing ones.
I have defined two serializers where UserSerializer has a nested GroupSerializer:
class GroupSerializer(serializers.ModelSerializer):
class Meta:
model = Group
fields = ['id', 'name']
class UserSerializer(serializers.ModelSerializer):
class Meta:
model = User
fields = ['id', 'username', 'email', 'groups']
groups = GroupSerializer(many=True)
def update(self, instance, validated_data):
data = validated_data.copy()
groups = data.pop('groups', [])
for key, val in data.items():
setattr(instance, key, val)
instance.groups.clear()
for group in groups:
instance.groups.add(group)
return instance
def create(self, validated_data):
data = validated_data.copy()
groups = data.pop('groups', [])
instance = self.Meta.model.objects.create(**data)
for group in groups:
instance.groups.add(group)
return instance
When I send a JSON through a PUT REST call (from django-rest-framework web interface):
{
"id": 6,
"username": "user5#example.com",
"email": "user5#example.com",
"groups": [
{
"id": 1,
"name": "AAA"
}
]
}
I expect serializer to find the Group with given id and add it to User. But instead, it tries to create a new user group and fails with duplicate key error:
{
"groups": [
{
"name": [
"group with this name already exists."
]
}
]
}
I searched over the internet and debugged myself and found no solution to this use case.
The create and update methods inside UserSerializerclass are never reached.
Edit: as asked, here are my views and urls:
class UserViewSet(viewsets.ModelViewSet):
queryset = User.objects.all().order_by('-date_joined')
serializer_class = UserSerializer
class GroupViewSet(viewsets.ModelViewSet):
queryset = Group.objects.all()
serializer_class = GroupSerializer
Urls:
router = routers.DefaultRouter()
router.register(r'users', views.UserViewSet)
router.register(r'groups', views.GroupViewSet)
urlpatterns = [
path('', include(router.urls)),
path('api-auth/', include('rest_framework.urls', namespace='rest_framework'))
]
This seems to be the validation error due to a nested serializer model that contains unique constraint, see this post. According to the article, DRF did not handle this condition since it's hard to realize if the serializer is a nested serializer within another one. And that's why the create() and update() never been reached since the validation is done before calling them.
The way to work around this is to remove the uniqueness validator manually in GroupSerializer as follow:
class GroupSerializer(serializers.ModelSerializer):
class Meta:
model = Group
fields = ['id', 'name']
extra_kwargs = {
'name': {'validators': []},
}
BTW, there are some points that can be improved or should be corrected in your update() and create() code. Firstly, you didn't do instance.save() so the instance won't be update after the whole process done. Second, the groups are just a list of dictionary, and you should not add object that way. The following are the modification based on your OP code:
def update(self, instance, validated_data):
data = validated_data.copy()
groups = data.pop('groups', [])
for key, val in data.items():
setattr(instance, key, val)
instance.save() # This will indeed update DB values
group_ids = [g['id'] for g in groups]
instance.groups.clear()
instance.groups.add(*group_ids) # Add all groups once. Also you can replace these two lines with
# instance.groups.set(group_ids)
return instance
Though am late, here is how I did it, adding to Tsang-Yi Shen answer. However this worked for me because I was using django-role-permissions https://django-role-permissions.readthedocs.io/en/stable/setup.html
For the group serializer am only interested in the name.
from rolepermissions.roles import assign_role
class GroupModelSerializer(serializers.ModelSerializer):
name = serializers.SerializerMethodField()
class Meta:
model = Group
fields = ['name']
extra_kwargs = {
'name': {'validators': []},
}
def get_name(self, obj):
"""
This method modifies the way the name field is returned in a get request.
"""
return [group.name for group in obj.objects.all()]
In the UserSerializer, I modify the groups field to be returned as a list of groups a user belongs to, and received as a list instead of an OrderedDict in the JSON payload for creating a User, which can look something like this:
{
"email": "testuser#example.com",
"username": "testuser",
"name": "Test User",
"password": "password123",
"groups": ["doctor", "nurse", ]
}
class UserSerializer(serializers.ModelSerializer):
password = serializers.CharField(write_only=True)
groups = ListField(required=False, default=[], write_only=True)
user_groups = serializers.SerializerMethodField(read_only=True)
class Meta:
model = User
fields = ["id", "email", "username", "name", "password", "groups", "user_groups"]
depth = 2
def get_user_groups(self, obj):
"""
This method modifies the way the `groups` field is returned in a get request.
"""
return [group.name for group in obj.groups.all()]
def create(self, validated_data):
password = validated_data.pop('password')
gropus_to_add_user = validated_data.pop("groups")
user = User(**validated_data)
user.set_password(password)
user.save()
for group_name in groups_to_add_user:
# The assign_role function adds the user to a group.
# In this case group and roles may mean the same thing.
assign_role(user, group_name)
return user
You will have something like this when creating a user, here am using Swagger:
actually, I wanted to send nested JSON with an image to creating user profile image in Django, and the main issues addressed to I cant fill the nested objects
this is my json that successfully post in raw mode :
{
"id": 1,
"bio": "salam manam",
"user": {
"username": "amirlesani",
"first_name": "",
"password":"somepasword"
"last_name": "",
"email": ""
},
"user_type": {
"user_type": "br"
}
}
But when I want to fill up the form data same as that way its shown an error like this
user_type: this field is required!
user: this field is required!
serlializers:
class UserProfileSerializer(serializers.HyperlinkedModelSerializer):
user = UserSerializer()
user_type = UserTypeSerializer()
images = ProfileImageSerializer(source='profileimage_set', many=True, read_only=True)
class Meta:
model = UserProfile
fields = ('id', 'bio', 'user', 'images', 'user_type')
def create(self, validated_data):
usertype = validated_data.pop('user_type')
type = UserTypeSerializer.create(UserTypeSerializer(), validated_data=usertype)
user_data = validated_data.pop('user')
user = UserSerializer.create(UserSerializer(), validated_data=user_data)
userprofile = UserProfile.objects.create(user=user, bio=validated_data.pop('bio'), user_type=type)
images_data = self.context.get('view').request.FILES
for image_data in images_data.values():
ProfileImage.objects.create(userprofile=userprofile, image=image_data, user_type=type)
userprofile.save()
return userprofile
i found the solution , u must distinct the imageUploader from the nested json , and connect ur images to ur data with a foreign key
Hi am new to django rest framework.Currently am working on a thing which is contains a json response like this :
{
"students": [
{
"first_name": <str: student first name>,
"last_name": <str: student last name>,
"unique_id": <str: student unique id>,
"current_teachers": [
{
"first_name": <str: teacher first name>,
"last_name": <str: teacher last name>
},
...
]
},
...more...
]
}
This is my serializer:
class StudentFilterSerializer(serializers.ModelSerializer):
class Meta:
model = Student
fields = ("first_name", "last_name", "unique_id",)
class StudentFilterTeacherSerializer(serializers.ModelSerializer):
first_name = serializers.CharField(source='teacher.first_name')
last_name = serializers.CharField(source='teacher.last_name')
class Meta:
model = TeacherClass
fields = ("teacher","first_name","last_name")
class FilterStudentsSerializer(serializers.Serializer):
students = StudentFilterSerializer(many=True)
current_teachers = StudentFilterTeacherSerializer(many=True, required=False)
Now how to edit my serializer to achieve the json response.
Cause this is my current json structure:
serializer = FilterStudentsSerializer()
serializer.data
{'students': [], 'current_teachers': []}
It would be easier to provide correct answer if we could see how your models are implemented. Without knowing the fields connection, I will still try
Here's two kind of solutions:
First:
class StudentFilterTeacherSerializer(serializers.ModelSerializer):
first_name = serializers.CharField(source='teacher.first_name')
last_name = serializers.CharField(source='teacher.last_name')
class Meta:
model = TeacherClass
fields = ("first_name","last_name")
class StudentFilterSerializer(serializers.ModelSerializer):
current_teachers = SerializerMethodField(required=False)
class Meta:
model = Student
fields = ("first_name", "last_name", "unique_id", "current_teachers")
def current_teachers(self, student):
# Get all teachers for this student
# I don't know how your models are created but you'll get the idea
# assuming you have "related_name" setup between student and teachers
teachers = student.teachers.all()
return StudentFilterTeacherSerializer(teachers, many=True).data
and second (probably not the one you want, still for your knowledge):
Note: This will return all fields for teachers.
class StudentFilterSerializer(serializers.ModelSerializer):
class Meta:
model = Student
fields = ("first_name", "last_name", "unique_id",)
# if you somehow have teachers field in 'student' model then you can also do
depth = 1
Happy to help.
Let me know if it works or you don't understand anything.
My issue is related to the django-rest-framework and is about how to group elements.
This is my serializers.py
from collaborativeAPP.models import *
from rest_framework import serializers
class VocabSerializer(serializers.ModelSerializer):
term_word = serializers.CharField(source='term.word',read_only=True)
kwdGroup = serializers.StringRelatedField()
class Meta:
model = Vocab
fields = ('id','term_word', 'meaning','kwdGroup')
class TermSerializer(serializers.ModelSerializer):
word = serializers.CharField(read_only=True)
class Meta:
model = Term
fields = ('url', 'word')
The following JSON it's the actual result:
{"results":[
{
"id": 5,
"term_word": "word1",
"meaning": "Text1"
"kwdGroup": "A"
},
{
"id": 6,
"term_word": "word2",
"meaning": "Text2"
"kwdGroup": "A"
},
{
"id": 7,
"term_word": "word3",
"meaning": "Text3"
"kwdGroup": "A"
}
]}
As you can notice kwdGroup is a repetitive element that I which to group.
I would like to group by kwdGroup:
{"A":[
{
"id": 5,
"term_word": "word1",
"meaning": "Text1"
},
{
"id": 6,
"term_word": "word2",
"meaning": "Text2"
},
{
"id": 7,
"term_word": "word3",
"meaning": "Text3"
}
]
}
I'm looking for answers on http://www.django-rest-framework.org/ on the API guide but I'm having difficulties finding an approach to lead with it.
Do you share this same issue? Do you have any suggestions on how can i do this? Do you have any example that deals with element grouping using django-rest-framework?
Thanks in advance.
Let's assume that the kwdGroup field is the relation field to a model called KeyWordGroup.
The default ListSerializer uses the to_representation method to render a list of the serialized objects rest_framework :
class ListSerializer(BaseSerializer):
...
def to_representation(self, data):
"""
List of object instances -> List of dicts of primitive datatypes.
"""
# Dealing with nested relationships, data can be a Manager,
# so, first get a queryset from the Manager if needed
iterable = data.all() if isinstance(data, models.Manager) else data
return [
self.child.to_representation(item) for item in iterable
]
We can modify the ListSerializer to group the results for example:
class VocabListSerializer(serializers.ListSerializer):
def to_representation(self, data):
iterable = data.all() if isinstance(data, models.Manager) else data
return {
kwdGroup: super().to_representation(Vocab.objects.filter(kwdGroup=kwdGroup))
for kwdGroup in KeyWordGroup.objects.all()
}
We can then use the modified VocabListSerializer with the VocabSerializer.
class VocabSerializer(serializers.Serializer):
...
class Meta:
list_serializer_class = VocabListSerializer
One way to achieve this is to use a SerializerMethodField. The below might be slightly different than your use case, but you can adopt accordingly. There are other ways of achieving this as well, including overwriting to_representation methods, but they rely on messing with the inner workings of DRF more than is relevant here.
models.py
class Dictionary(Model):
id = PrimaryKey
class Word(Model):
dictionary = ForeignKey(Dictionary, related_name='words')
word = Charfield()
group = Charfield()
serializers.py
class WordSerializer(serializers.ModelSerializer):
word = serializers.CharField(read_only=True)
class Meta:
model = Word
fields = ('word',)
class DictionarySerializer(serializers.ModelSerializer):
group_a = serializers.SerializerMethodField()
group_b = serializers.SerializerMethodField()
def get_group_a(self, instance):
return WordSerializer(instance.words.filter(group='a'), many=True).data
def get_group_b(self, instance):
return WordSerializer(instance.words.filter(group='b'), many=True).data
class Meta:
model = Dictionary
fields = ('group_a', 'group_b')
An example
>>> my_dictionary = Dictionary.objects.create()
>>> Word.objects.bulk_create(
Word(word='arrow', group='a' dictionary=my_dictionary),
Word(word='apple', group='a' dictionary=my_dictionary),
Word(word='baby', group='b' dictionary=my_dictionary),
Word(word='banana', group='b' dictionary=my_dictionary)
)
>>> serializer = DictionarySerializer(my_dictionary)
>>> print serializer.data
{
'group_a': {
'word': 'arrow',
'word': 'apple'
},
'group_b': {
'word': 'baby',
'word': 'banana'
},
}
I defined a nested model Product as follow. Each Product can belong to a lot of Productlist.
class Product(models.Model):
product_id = models.AutoField(primary_key=True)
product_name = models.CharField(max_length=50)
class Productlist(models.Model):
productlist_id = models.AutoField(primary_key=True)
productlist_name = models.CharField(max_length=50)
product = models.ManyToManyField(Product, related_name='productlists')
The corresponding serializers are:
class ProductlistSerializer(serializers.ModelSerializer):
class Meta:
model = Productlist
fields = ('productlist_id', 'productlist_name',)
class ProductSerializer(serializers.ModelSerializer):
productlists = ProductlistSerializer(many=True, required=False)
class Meta:
model = Product
fields = ('product_id', 'product_name', 'product lists')
def create(self, validated_data):
#some codes
When I POST a new Product (url(r'^api/products/$', views.ProductEnum.as_view()), I would like to update the product lists for adding the new product to the corresponding product lists. The JSON file I prefer to use is:
{
"product_name": "product1"
"productlist": [
{
"productlist_id": 1,
"productlist_name": "list1",
},
{
"productlist_id": 2,
"productlist_name": list2"
}
]
}
The problem is that I cannot get the productlist_id from validated_data. In Django Rest Framework, you always need to call to_internal_value() for deserializing data and generate validated_data. After some degugging, I checked the code of DRF and find the following snippets in to_internal_value():
def to_internal_value(self, data):
"""
Dict of native values <- Dict of primitive datatypes.
"""
if not isinstance(data, dict):
message = self.error_messages['invalid'].format(
datatype=type(data).__name__
)
raise ValidationError({
api_settings.NON_FIELD_ERRORS_KEY: [message]
})
ret = OrderedDict()
errors = OrderedDict()
fields = [
field for field in self.fields.values()
if (not field.read_only) or (field.default is not empty)
]
for field in fields:
validate_method = getattr(self, 'validate_' + field.field_name, None)
primitive_value = field.get_value(data)
try:
validated_value = field.run_validation(primitive_value)
if validate_method is not None:
validated_value = validate_method(validated_value)
except ValidationError as exc:
errors[field.field_name] = exc.detail
except DjangoValidationError as exc:
errors[field.field_name] = list(exc.messages)
except SkipField:
pass
else:
set_value(ret, field.source_attrs, validated_value)
if errors:
raise ValidationError(errors)
return ret
Please notice the to_internal_value's fields has ignored the IntegerField(read_only=True) for it cannot satisfy the following condition:
fields = [
field for field in self.fields.values()
if (not field.read_only) or (field.default is not empty)
]
So the validated_data will just have the following data:
{
"product_name": "product1"
"productlist": [
{
"productlist_name": "list1",
},
{
"productlist_name": list2"
}
]
}
How could I get the primary key of product list? Thanks in advance!
After some digging, I found that the read_only fields are only for output presentation. You can find the similar question on the offcial github link of Django REST Framework.
So the solution is overriding the read_only field in the serializer as follow:
class ProductlistSerializer(serializers.ModelSerializer):
productlist_id = serializers.IntegerField(read_only=False)
class Meta:
model = Productlist
fields = ('productlist_id', 'productlist_name',)