Related
I am developing above database with many to many relationship and a additionally I need add in intermediate table a relationship many to one but I can't get this las relationship with others.
What's a proper way to define many-to-many relationships in a pydantic model with extra data as relationship.
models.py:
class Devices(Base):
__tablename__ = "devices"
id = Column(Integer, primary_key=True, unique=True, index=True)
name = Column(String(255))
description = Column(String(255), nullable=True)
status_id = Column(Integer, ForeignKey('status.id'))
status = relationship("Status", backref="devices")
protocols = relationship("Protocols", secondary="device_protocols", back_populates='device')
class Status(Base):
__tablename__ = "status"
id = Column(Integer, primary_key=True, unique=True, index=True)
name = Column(String(255))
description = Column(String(255), nullable=True)
class Protocols(Base):
__tablename__ = "protocols"
id = Column(Integer, primary_key=True, unique=True, index=True)
name = Column(String(255))
device = relationship("Devices", secondary="device_protocols", back_populates='protocols')
class DeviceProtocols(Base):
__tablename__ = "device_protocols"
device_id = Column(Integer, ForeignKey('devices.id'), primary_key=True)
protocol_id = Column(Integer, ForeignKey('protocols.id'), primary_key=True)
protocol_status_id = Column(Integer, ForeignKey('status.id'), nullable=True)
protocol_status = relationship("Status", backref="protocol_status")
Schemas:
class DeviceBase (BaseModel):
name: str
class Config:
orm_mode = True
class DeviceRead (DeviceBase):
id: str
description: str | None = None
status: StatusReadSimple | None = None
protocols: list[ProtocolSimple]
class ProtocolBase (BaseModel):
name: str
class Config:
orm_mode = True
class ProtocolSimple(ProtocolBase):
id: str
class StatusBase (BaseModel):
name: str
description: str | None = None
class Config:
orm_mode = True
class StatusReadSimple(StatusBase):
id: str
How do I need to develop the schemas so that the device returns the intermediate table with the protocol and its status?
Actual response:
{
"name": "device1",
"id": "3",
"description": "my device",
"status": {
"name": "OK",
"description": "Connection OK",
"id": "1"
},
"protocols": [
{
"name": "ethernet",
"id": "1"
},
{
"name": "ethercat",
"id": "2"
}
]
}
Expected response or similar:
{
"name": "device1",
"id": "3",
"description": "my device",
"status": {
"name": "OK",
"description": "Connection OK",
"id": "1"
},
"protocols": [
{
"protocol:"{
"name": "ethernet",
"id": "1"
},
"protocol_status":{
"id":1,
"name": "OK"
}
},
{
"protocol:"{
"name": "ethercat",
"id": "2"
},
"protocol_status":{
"id":2,
"name": "NOK"
}
}
]
}
TL;DR
Use a direct relationship from device to device_protocol, alias the protocols field on your Pydantic model with the name of that relationship and annotate it with a corresponding device_protocol Pydantic model. Then add protocol and status relationships to the device_protocol models (DB and Pydantic).
Pre
Your naming is a bit inconsistent, especially regarding singular/plural forms. In the following I am going to use slightly different table names (e.g. Device instead of Devices) and field names (status instead of protocol_status) that I think make more sense semantically.
I will also rename DeviceProtocols to DeviceProtocolAssociation to be very clear about the purpose of this table. You'll see why later.
Also, you should avoid using the legacy backref parameter.
Problem: Missing ORM relationship fields
This is not something that I would try to solve purely via the Pydantic models. It is possible (e.g. via complex custom validators), but I would suggest that there is a much more elegant way to do this, if we adjust the database models (specifically their relationship attributes) instead.
The problem is that your Device database model does not have an attribute capturing the intermediary DeviceProtocolAssociation model directly. You only set that table (device_protocol) as the secondary argument to get direct access to the related Protocol instances via protocol.
But if you want a Device to have an attribute listing the associated Protocols as well as the Status of those Device-Protocol pairs, you actually do need a relationship attribute to that intermediary DeviceProtocolAssociation model. After all, that model/table is exactly where that connection is made.
Then, in your DeviceProtocolAssociation you'll have to add a protocol relationship to the associated Protocol, just like you have a status relationship to the associated Status.
Solution
To make this work, and be consistent with what you already have, you'll need a few additional components.
relationship from Device directly to DeviceProtocolAssociation
Let's call this attribute device_protocol_associations. Since one Device can have many DeviceProtocolAssociations, that will be a list of instances of that model.
relationship from DeviceProtocolAssociation to Protocol
We'll call that protocol and it will just hold an instance of the associated Protocol.
association_proxy from Device to DeviceProtocolAssociation.protocol
Instead of having a relationship from Device to Protocol (via the secondary parameter), we will define an association proxy to the protocol field of each associated DeviceProtocolAssociation via the new device_protocol_associations relationship. We will call that attribute protocols to replace the old attribute. It will work the same as before.
Optionally do the same with the Protocol/Status fields
Depending on whether or not that is useful for you, you could again define a direct relationship from Protocol to DeviceProtocolAssociation and add an association proxy called devices. And you could do that for Status too.
New model definitions
from sqlalchemy import Column, ForeignKey, Integer, String
from sqlalchemy.ext.associationproxy import association_proxy
from sqlalchemy.orm import declarative_base, relationship
Base = declarative_base()
class DeviceProtocolAssociation(Base):
__tablename__ = "device_protocol"
device_id = Column(Integer, ForeignKey("device.id"), primary_key=True)
device = relationship("Device", back_populates="device_protocol_associations")
protocol_id = Column(Integer, ForeignKey("protocol.id"), primary_key=True)
protocol = relationship("Protocol", back_populates="device_protocol_associations")
status_id = Column(Integer, ForeignKey("status.id"), nullable=True)
status = relationship("Status", back_populates="device_protocol_associations")
class Device(Base):
__tablename__ = "device"
id = Column(Integer, primary_key=True)
name = Column(String(255))
status_id = Column(Integer, ForeignKey("status.id"))
status = relationship("Status", back_populates="devices")
device_protocol_associations = relationship(DeviceProtocolAssociation, back_populates="device")
protocols = association_proxy("device_protocol_associations", "protocol")
class Protocol(Base):
__tablename__ = "protocol"
id = Column(Integer, primary_key=True)
name = Column(String(255))
device_protocol_associations = relationship(DeviceProtocolAssociation, back_populates="protocol")
devices = association_proxy("device_protocol_associations", "device")
class Status(Base):
__tablename__ = "status"
id = Column(Integer, primary_key=True)
name = Column(String(255))
devices = relationship("Device", back_populates="status")
device_protocol_associations = relationship(DeviceProtocolAssociation, back_populates="status")
To create some test data for demo purposes, I'll add the following function underneath:
def create_test_data() -> Device:
status_ok = Status(id=1, name="OK")
device = Device(id=42, name="device1", status=status_ok)
device.device_protocol_associations.append(
DeviceProtocolAssociation(
protocol=Protocol(id=1, name="ethernet"),
status=status_ok,
)
)
device.device_protocol_associations.append(
DeviceProtocolAssociation(
protocol=Protocol(id=2, name="ethercat"),
status=Status(id=69, name="Not OK"),
)
)
return device
Adjusted Pydantic models
This is again just to demonstrate. You can transfer this to the inheritance structure you already have.
from __future__ import annotations
from pydantic import BaseModel as _BaseModel, Field
class BaseModel(_BaseModel):
class Config:
orm_mode = True
class ProtocolStatusModel(BaseModel):
"""Corresponds to `DeviceProtocolAssociation`, but no need for `device`"""
protocol: ProtocolModel
status: StatusModel
class DeviceModel(BaseModel):
id: int
name: str
status: StatusModel
protocols: list[ProtocolStatusModel] = Field(alias="device_protocol_associations")
class ProtocolModel(BaseModel):
id: int
name: str
class StatusModel(BaseModel):
id: int
name: str
As you can see, the DeviceModel has a customized protocols field with an alias of device_protocol_associations. That means when we parse a Device ORM object, that field will get its value from the device_protocol_associations list.
Demo
ProtocolStatusModel.update_forward_refs()
DeviceModel.update_forward_refs()
def main() -> None:
db_device = create_test_data()
output_device = DeviceModel.from_orm(db_device)
print(output_device.json(indent=4))
if __name__ == "__main__":
main()
Output
{
"id": 42,
"name": "device1",
"status": {
"id": 1,
"name": "OK"
},
"protocols": [
{
"protocol": {
"id": 1,
"name": "ethernet"
},
"status": {
"id": 1,
"name": "OK"
}
},
{
"protocol": {
"id": 2,
"name": "ethercat"
},
"status": {
"id": 69,
"name": "Not OK"
}
}
]
}
I am building an e-commerce website using Django, my models is like bellow :
class ProductAttribute(models.Model):
product=models.ForeignKey(Product,on_delete=models.CASCADE)
attributes_values = models.ManyToManyField(AttributeValue,verbose_name="Liste des attributs")
stock = models.PositiveIntegerField()
price = models.PositiveIntegerField(verbose_name="Prix")
image = models.ImageField(blank=True,null=True,upload_to="products")
class AttributeValue(models.Model):
attribute=models.ForeignKey(Attribute,on_delete=models.CASCADE,verbose_name="Attribut")
value = models.CharField(max_length=50,verbose_name="Valeur")
class Attribute(models.Model):
name = models.CharField(max_length=50,verbose_name="Nom")
my view.py
def getatts(request,product_id):
products_with_attributes=ProductAttribute.objects.filter(product__id=product_id)
res=#..... missing code to get attributes with values
return res
In the front end i want to retrieve attributes of a particular product to get them by order, to use them in select (ex:size,color choices) , for example if the query set of ProductAttribute is like:
[{id:1,product:1,attributes_values:[3,4],...},{id:1,product:1,attributes_values:[5,6],...}]
the result in JSON would be like so:
{
result:[
{
key: "color", // attribute.name
values: [
{id: 1, value: "Red",
choices:{
key:"size", // second attribute.name
values:[{id:3,value:"L"},{id:4,value:"XL"}]
}
},
{id: 2, value: "Black",
choices:{
key:"size",
values:[{id:5,value:"M"},{id:6,value:"XXL"}]
}
},
]
}
]
}
Note: I am using MYSQL as database
this is a dirty way of doing it and it is static way (max two attribute values) is there any way to do it using Django ORM:
products=ProductAttribute.objects.filter(product__id=id)
res={}
keys=[]
values=[]
for attribute_value in products.first().attributes_values.all():
keys.append({"id":attribute_value.attribute.id,"name":attribute_value.attribute.name})
res["id"]=keys[0]["id"]
res["name"]=keys[0]["name"]
# print(res)
for p in products:
attributes_values=p.attributes_values.all()
# print([ { "id":attv.id,"value":attv.value, "attribute_id":attv.attribute.id, "attribute_name":attv.attribute.name } for attv in attributes_values ])
for attv in attributes_values:
if attv.attribute.id==res["id"]:
exists=False
for v in values:
if v["id"]==attv.id:
exists=True
if not exists:
if len(keys)>1:
first_attribute={ "id":attv.id,"value":attv.value}
first_attribute["sub"]={"id":keys[1]["id"],"name":keys[1]["name"],"values":[]}
for pp in products:
for attv2 in pp.attributes_values.filter(productattribute__id__in= products.filter(attributes_values__id=attv.id).values("id")):
if attv2.attribute.id!=res["id"]:
exists2=False
for sub_value in first_attribute["sub"]["values"]:
if sub_value["id"]==attv2.id:
exists2=True
if not exists2:
first_attribute["sub"]["values"].append({"id":attv2.id,"value":attv2.value})
# first_attribute["sub"]["values"]
# p.attributes_values.all()[1]
values.append(first_attribute)
else:
values.append({ "id":attv.id,"value":attv.value,"sub":{}})
print(attv.attribute.id)
res["values"]=values
print(res)
I am trying to create an endpoint where, having a User entity, I can add / remove existing Group entities to user.groups many-to-many field. But when I try to do it, django-rest-framework tries to create new group objects instead of finding existing ones.
I have defined two serializers where UserSerializer has a nested GroupSerializer:
class GroupSerializer(serializers.ModelSerializer):
class Meta:
model = Group
fields = ['id', 'name']
class UserSerializer(serializers.ModelSerializer):
class Meta:
model = User
fields = ['id', 'username', 'email', 'groups']
groups = GroupSerializer(many=True)
def update(self, instance, validated_data):
data = validated_data.copy()
groups = data.pop('groups', [])
for key, val in data.items():
setattr(instance, key, val)
instance.groups.clear()
for group in groups:
instance.groups.add(group)
return instance
def create(self, validated_data):
data = validated_data.copy()
groups = data.pop('groups', [])
instance = self.Meta.model.objects.create(**data)
for group in groups:
instance.groups.add(group)
return instance
When I send a JSON through a PUT REST call (from django-rest-framework web interface):
{
"id": 6,
"username": "user5#example.com",
"email": "user5#example.com",
"groups": [
{
"id": 1,
"name": "AAA"
}
]
}
I expect serializer to find the Group with given id and add it to User. But instead, it tries to create a new user group and fails with duplicate key error:
{
"groups": [
{
"name": [
"group with this name already exists."
]
}
]
}
I searched over the internet and debugged myself and found no solution to this use case.
The create and update methods inside UserSerializerclass are never reached.
Edit: as asked, here are my views and urls:
class UserViewSet(viewsets.ModelViewSet):
queryset = User.objects.all().order_by('-date_joined')
serializer_class = UserSerializer
class GroupViewSet(viewsets.ModelViewSet):
queryset = Group.objects.all()
serializer_class = GroupSerializer
Urls:
router = routers.DefaultRouter()
router.register(r'users', views.UserViewSet)
router.register(r'groups', views.GroupViewSet)
urlpatterns = [
path('', include(router.urls)),
path('api-auth/', include('rest_framework.urls', namespace='rest_framework'))
]
This seems to be the validation error due to a nested serializer model that contains unique constraint, see this post. According to the article, DRF did not handle this condition since it's hard to realize if the serializer is a nested serializer within another one. And that's why the create() and update() never been reached since the validation is done before calling them.
The way to work around this is to remove the uniqueness validator manually in GroupSerializer as follow:
class GroupSerializer(serializers.ModelSerializer):
class Meta:
model = Group
fields = ['id', 'name']
extra_kwargs = {
'name': {'validators': []},
}
BTW, there are some points that can be improved or should be corrected in your update() and create() code. Firstly, you didn't do instance.save() so the instance won't be update after the whole process done. Second, the groups are just a list of dictionary, and you should not add object that way. The following are the modification based on your OP code:
def update(self, instance, validated_data):
data = validated_data.copy()
groups = data.pop('groups', [])
for key, val in data.items():
setattr(instance, key, val)
instance.save() # This will indeed update DB values
group_ids = [g['id'] for g in groups]
instance.groups.clear()
instance.groups.add(*group_ids) # Add all groups once. Also you can replace these two lines with
# instance.groups.set(group_ids)
return instance
Though am late, here is how I did it, adding to Tsang-Yi Shen answer. However this worked for me because I was using django-role-permissions https://django-role-permissions.readthedocs.io/en/stable/setup.html
For the group serializer am only interested in the name.
from rolepermissions.roles import assign_role
class GroupModelSerializer(serializers.ModelSerializer):
name = serializers.SerializerMethodField()
class Meta:
model = Group
fields = ['name']
extra_kwargs = {
'name': {'validators': []},
}
def get_name(self, obj):
"""
This method modifies the way the name field is returned in a get request.
"""
return [group.name for group in obj.objects.all()]
In the UserSerializer, I modify the groups field to be returned as a list of groups a user belongs to, and received as a list instead of an OrderedDict in the JSON payload for creating a User, which can look something like this:
{
"email": "testuser#example.com",
"username": "testuser",
"name": "Test User",
"password": "password123",
"groups": ["doctor", "nurse", ]
}
class UserSerializer(serializers.ModelSerializer):
password = serializers.CharField(write_only=True)
groups = ListField(required=False, default=[], write_only=True)
user_groups = serializers.SerializerMethodField(read_only=True)
class Meta:
model = User
fields = ["id", "email", "username", "name", "password", "groups", "user_groups"]
depth = 2
def get_user_groups(self, obj):
"""
This method modifies the way the `groups` field is returned in a get request.
"""
return [group.name for group in obj.groups.all()]
def create(self, validated_data):
password = validated_data.pop('password')
gropus_to_add_user = validated_data.pop("groups")
user = User(**validated_data)
user.set_password(password)
user.save()
for group_name in groups_to_add_user:
# The assign_role function adds the user to a group.
# In this case group and roles may mean the same thing.
assign_role(user, group_name)
return user
You will have something like this when creating a user, here am using Swagger:
My issue is related to the django-rest-framework and is about how to group elements.
This is my serializers.py
from collaborativeAPP.models import *
from rest_framework import serializers
class VocabSerializer(serializers.ModelSerializer):
term_word = serializers.CharField(source='term.word',read_only=True)
kwdGroup = serializers.StringRelatedField()
class Meta:
model = Vocab
fields = ('id','term_word', 'meaning','kwdGroup')
class TermSerializer(serializers.ModelSerializer):
word = serializers.CharField(read_only=True)
class Meta:
model = Term
fields = ('url', 'word')
The following JSON it's the actual result:
{"results":[
{
"id": 5,
"term_word": "word1",
"meaning": "Text1"
"kwdGroup": "A"
},
{
"id": 6,
"term_word": "word2",
"meaning": "Text2"
"kwdGroup": "A"
},
{
"id": 7,
"term_word": "word3",
"meaning": "Text3"
"kwdGroup": "A"
}
]}
As you can notice kwdGroup is a repetitive element that I which to group.
I would like to group by kwdGroup:
{"A":[
{
"id": 5,
"term_word": "word1",
"meaning": "Text1"
},
{
"id": 6,
"term_word": "word2",
"meaning": "Text2"
},
{
"id": 7,
"term_word": "word3",
"meaning": "Text3"
}
]
}
I'm looking for answers on http://www.django-rest-framework.org/ on the API guide but I'm having difficulties finding an approach to lead with it.
Do you share this same issue? Do you have any suggestions on how can i do this? Do you have any example that deals with element grouping using django-rest-framework?
Thanks in advance.
Let's assume that the kwdGroup field is the relation field to a model called KeyWordGroup.
The default ListSerializer uses the to_representation method to render a list of the serialized objects rest_framework :
class ListSerializer(BaseSerializer):
...
def to_representation(self, data):
"""
List of object instances -> List of dicts of primitive datatypes.
"""
# Dealing with nested relationships, data can be a Manager,
# so, first get a queryset from the Manager if needed
iterable = data.all() if isinstance(data, models.Manager) else data
return [
self.child.to_representation(item) for item in iterable
]
We can modify the ListSerializer to group the results for example:
class VocabListSerializer(serializers.ListSerializer):
def to_representation(self, data):
iterable = data.all() if isinstance(data, models.Manager) else data
return {
kwdGroup: super().to_representation(Vocab.objects.filter(kwdGroup=kwdGroup))
for kwdGroup in KeyWordGroup.objects.all()
}
We can then use the modified VocabListSerializer with the VocabSerializer.
class VocabSerializer(serializers.Serializer):
...
class Meta:
list_serializer_class = VocabListSerializer
One way to achieve this is to use a SerializerMethodField. The below might be slightly different than your use case, but you can adopt accordingly. There are other ways of achieving this as well, including overwriting to_representation methods, but they rely on messing with the inner workings of DRF more than is relevant here.
models.py
class Dictionary(Model):
id = PrimaryKey
class Word(Model):
dictionary = ForeignKey(Dictionary, related_name='words')
word = Charfield()
group = Charfield()
serializers.py
class WordSerializer(serializers.ModelSerializer):
word = serializers.CharField(read_only=True)
class Meta:
model = Word
fields = ('word',)
class DictionarySerializer(serializers.ModelSerializer):
group_a = serializers.SerializerMethodField()
group_b = serializers.SerializerMethodField()
def get_group_a(self, instance):
return WordSerializer(instance.words.filter(group='a'), many=True).data
def get_group_b(self, instance):
return WordSerializer(instance.words.filter(group='b'), many=True).data
class Meta:
model = Dictionary
fields = ('group_a', 'group_b')
An example
>>> my_dictionary = Dictionary.objects.create()
>>> Word.objects.bulk_create(
Word(word='arrow', group='a' dictionary=my_dictionary),
Word(word='apple', group='a' dictionary=my_dictionary),
Word(word='baby', group='b' dictionary=my_dictionary),
Word(word='banana', group='b' dictionary=my_dictionary)
)
>>> serializer = DictionarySerializer(my_dictionary)
>>> print serializer.data
{
'group_a': {
'word': 'arrow',
'word': 'apple'
},
'group_b': {
'word': 'baby',
'word': 'banana'
},
}
new to ember js, and working on an app using ember-data. If I test with same data using FixtureAdapter, everything populates in the html template ok. When I switch to RESTAdapter, the data looks like it's coming back ok, but the models are not being populated in the template? Any ideas? Here's the code:
App.Store = DS.Store.extend({
revision:12,
//adapter: 'DS.FixtureAdapter'
adapter: DS.RESTAdapter.extend({
url:'http://bbx-dev.footballamerica.com/builderapirequest/bat'
})
});
App.Brand = DS.Model.extend({
name: DS.attr('string'),
numStyles: DS.attr('string'),
vendorId: DS.attr('string')
});
App.BrandsRoute = Ember.Route.extend({
setupController:function(controller){
},
model:function(){
return App.Brand.find();
}
});
And here is the data coming back, but not being inserted into the template!
returnValue: [{numStyles:1, name:Easton, vendorId:6043}, {numStyles:1, name:Louisville Slugger, vendorId:6075},…]
0: {numStyles:1, name:Easton, vendorId:6043}
1: {numStyles:1, name:Louisville Slugger, vendorId:6075}
2: {numStyles:1, name:Rawlings, vendorId:6109}
3: {numStyles:7, name:BWP Bats , vendorId:6496}
4: {numStyles:1, name:DeMarini, vendorId:W002}
status: "ok"
And here is the template:
{{#each brand in model.returnValue }}
<div class="brand-node"{{action select brand}}>
<h2>{{brand.name}}</h2>
<p>{{brand.numStyles}} Styles</p>
</div>
{{/each}}
Any help would be greatly appreciated! I'm not getting any errors, and the data seems to be coming back ok, just not getting into the template. Not sure if the returned dataset needs "id" param?
I am also using the Store congfig to alter the find() from plural to singular:
DS.RESTAdapter.configure("plurals", {
brand: "brand"
});
The way the API was written, its expecting "brand" and not "brands"... maybe its something to do with this??
Thanks in advance.
You have stated:
Not sure if the returned dataset needs "id" param?
Yes you are guessing right, you data coming back from the backend need's an id field set. And if the id field name is different then id you should also define this in ember like so:
App.Store = DS.Store.extend({
revision:12,
//adapter: 'DS.FixtureAdapter'
adapter: DS.RESTAdapter.extend({
url:'http://bbx-dev.footballamerica.com/builderapirequest/bat'
}),
serializer: DS.RESTSerializer.extend({
primaryKey: function (type) {
return '_my_super_custom_ID'; // Only needed if your id field name is different than 'id'
}
})
});
I suppose your Fixtures have an id defined thus it works, right?
Note: you don't need to define the id field at all explicitly, ember add's automatically the id field to a model, so your model is correct.
Here a website that is still a WIP but can be good reference for this conventions
and as stated there:
The document MUST contain an id key.
And this is how your JSON should look like for a collection of records:
{
"brands": [{
"id": "1",
"numStyles": "1",
"name": "Easton",
"vendorId" :"6043"
}, {
"id": "2",
"numStyles": "4",
"name": "Siemens",
"vendorId": "6123"
}/** and so on**/]
}
Note: as you have shown you JSON root is called returnValue this should be called brand or brands if you are not adapting the plurals. See here for reference for the JSON root I'm talking about.
Hope it helps