DRF Nested Objects serialization and creation - json

I'm trying to create objects just by getting a json that have the objects nested in it,for example, I have a model with only one field that is name and im getting this json
{"name":"category1",
"children":[{
"name":"category1.1",
"children":[]},
{"name":"category1.2",
"children":[{"name":"category1.2.1",
"children":[]}]
}
]
}
What i'm trying to achieve is to read this,and create category objects that reference its parent or its children,
I've tried multiple solutions inspired from these answers but I can't seem to get close to getting the job done(Django rest framework nested self-referential objects -- How to create multiple objects (related) with one request in DRF?)
I've tried having the model with only name and a foreign key that reference the category itself,and I added recursive field to my serializer like this:
class RecursiveField(serializers.Serializer):
def to_representation(self, value):
serializer = self.parent.parent.__class__(value, context=self.context)
return serializer.data
class CategorySerializer(serializers.ModelSerializer):
subcategories = RecursiveField(many=True,allow_null=True)
class Meta:
model = Category
fields = ['name','subcategories']
def create(self, validated_data):
category = None
if len(validated_data['subcategories'])==0:
category = Category.objects.create(name=validated_data['name'])
else:
for i in range(len(validated_data['subcategories'])):
child = validated_data['subcategories']
child_list = list(child.items())
subcat = child_list[1]
if len(subcat)==0:
subcategory = Category.objects.create(name=child.get('name'))
category = Category.objects.create(name=validated_data['name'],children=subcategory)
return category
The best I got with this solution is being able to create the parent object,but I wasn't able to get the children objects instead I got an empty OrderedDict() (I only tried this solution to see if i can access the children but apparently i can't,I get an empty OrderedDict() in child variable)
Am I looking at this from the wrong prespective? or is my model architecture not suitable for this?
if not,what am i doing wrong

I think I found a solution that solves all the problems in my case,It's not the most optimized and if someone can share a better one I'm happy to check it out,but here is:
I went for thick view instead of a thick serializer,I kept the model as it is with 2 fields,name field and foreign key that references the parent,as for the serializer:
class CategorySerializer(serializers.Serializer):
name = serializers.CharField()
children = serializers.ListField()
I still have to add validators for the list field,as for the view I went for a recursive function that goes through all the children and adds them:
def post(self,request):
serializer = self.get_serializer(data=request.data)
serializer.is_valid(raise_exception=True)
parent = serializer
#creating the parent object
category = Category.objects.create(name=parent.validated_data['name'])
#saving the parent object's id
parent_id = category.id
#Checking if the item has children
children = parent.validated_data['children']
#if the parent doesn't have children nothing will happen
if children == None:
pass
#Call the recursion algorithm to create all the children
else:
self.recursion_creation(children,parent_id)
return Response({
"id":parent_id,
})
def recursion_creation(self,listOfChildren,parentId):
#This will go through all the children of the parent and create them,and create their children
for k in range(len(listOfChildren)):
name = listOfChildren[k].get("name")
#create the child
category = Category.objects.create(name=name,parent_id=parentId)
categoryId = category.id
#save it's id
children = listOfChildren[k].get("children")
#if this child doesn't have children the recursion will stop
if children==None:
pass
#else it will apply the same algorithm and goes through it's children and create them and check their children
else:
self.recursion_creation(children,categoryId)
I'm looking forward to any improvements or other answers that solve the problem.

Related

Update relationship in SQLAlchemy

I have this kind of model:
class A(Base):
id = Column(UUID(as_uuid=True), primary_key=True, server_default=text("uuid_generate_v4()"))
name = Column(String, nullable=False, unique=True)
property = Column(String)
parent_id = Column(UUID(as_uuid=True), ForeignKey(id, ondelete="CASCADE"))
children = relationship(
"A", cascade="all,delete", backref=backref("parent", remote_side=[id])
)
An id is created automatically by the server
I have a relationship from a model to itself (parent and children).
In the background I run a task that periodically receives a message with id of parent and list of pairs (name, property) of children. I would like to update the parent's children in table (Defined by name). Is there a way to do so without reading all children, see which one is missing (name not present is message), need to be updated (name exists but property has changed) or new (name not present in db)?
Do I need to set name to be my primary key and get rid of the UUID?
Thanks
I'd do a single query and compare the result against the message you receive. That way it's easier to handle both additions, removals and updates.
msg_parent_id = 5
msg_children = [('name', 'property'), ('name2', 'property2')]
stmt = select(A).where(A.parent_id == msg_parent_id)
children = session.execute(stmt).scalars()
# Example of determining what to change
name_map = {row.name: row for row in children}
for child_name, child_prop in msg_children:
# Child exists
if child_name in name_map:
# Edit child
if name_map[child_name].property != child_prop:
print(child_name, 'has changed to', property)
del name_map[child_name]
# Add child
else:
print(child_name, 'was added')
# Remove child
for child in name_map.values():
print(child, 'was removed')
Do I need to set name to be my primary key and get rid of the UUID?
Personally I'd add a unique constraint on the name, but still have a separate ID column for the sake of relationships.
Edit for a more ORM orientated way. I believe you can already use A.children = [val1, val2], which is really what you need.
In the past I have used this answer on how to intercept the call, parse the input data, and fetch the existing record from the database if it exists. As part of the that call you could update the property of that record.
Finally use a cascade on the relationship to automatically delete records when parent_id is set to None.

Apply similar options in SQL alchemy as contains_eager by default to query without specifying relationship path

The following pytest passes, as expected:
from sqlalchemy.orm import contains_eager
#pytest.fixture(scope="function", autouse=True)
def populate_db(test_db: Session):
"""Populate db with a single parent with two children that go to different schools."""
parent_1 = Parent(uuid="1")
child_1 = Child(uuid="2", parent_uuid="1", school="north")
child_2 = Child(uuid="3", parent_uuid="1", school="south")
test_db.add(parent_1)
test_db.commit()
test_db.add(child_1)
test_db.add(child_2)
test_db.commit()
def test_eager(db: Session):
"""Query for parents based off of weather or their child goes to school 'north'."""
parent = db.query(Parent).join(Child).filter(Child.school == "north").first()
# both of the parent children relationships are returned
assert len(parent.children) == 2
parent = db.query(Parent).join(Child).options(contains_eager(Parent.children)).filter(Child.school == "north").first()
# Only one of the parent child relationship returned
assert len(parent.children) == 1
What I would like, is for the behaviour of contains_eager to be applied by default (so in this case the first assertion would be false, since only the 1 parent.children relationship would be returned)
Also, I have many instances where I would want this to be applied, so would prefer not to have to define the relationship path each time .options(contains_eager(Parent.children))
For example if I added a School model, I wouldn't want to be required to update the options with School.children.

SQLAlchemy clone table row with relations

Following on from this question SQLAlchemy: Modification of detached object.
This makes a copy of the object fine, but it loses any many-to-many relationships the original object had. Is there a way to copy the object and any many-to-many relationships as well?
Cheers!
I got this to work by walking the object graph and doing the expunge(), make_transient() and id = None steps on each object in the graph as described in SQLAlchemy: Modification of detached object.
Here is my sample code. The agent has at most one campaign.
from sqlalchemy.orm.session import make_transient
def clone_agent(id):
s = app.db.session
agent = s.query(Agent).get(id)
c = None
# You need to get child before expunge agent, otherwise the children will be empty
if agent.campaigns:
c = agent.campaigns[0]
s.expunge(c)
make_transient(c)
c.id = None
s.expunge(agent)
agent.id = None
# I have unique constraint on the following column.
agent.name = agent.name + '_clone'
agent.externalId = - agent.externalId # Find a number that is not in db.
make_transient(agent)
s.add(agent)
s.commit() # Commit so the agent will save to database and get an id
if c:
assert agent.id
c.agent_id = agent.id # Attach child to parent (agent_id is a foreign key)
s.add(c)
s.commit()

Association Proxy SQLAlchemy

This source details how to use association proxies to create views and objects with values of an ORM object.
However, when I append an value that matches an existing object in the database (and said value is either unique or a primary key), it creates a conflicting object so I cannot commit.
So in my case is this only useful as a view, and I'll need to use ORM queries to retrieve the object to be appended.
Is this my only option or can I use merge (I may only be able to do this if it's a primary key and not a unique constraint), OR set up the constructor such that it will use an existing object in the database if it exists instead of creating a new object?
For example from the docs:
user.keywords.append('cheese inspector')
# Is translated by the association proxy into the operation:
user.kw.append(Keyword('cheese inspector'))
But I'd like to to be translated to something more like: (of course the query could fail).
keyword = session.query(Keyword).filter(Keyword.keyword == 'cheese inspector').one()
user.kw.append(keyword)
OR ideally
user.kw.append(Keyword('cheese inspector'))
session.merge() # retrieves identical object from the database, or keeps new one
session.commit() # success!
I suppose this may not even be a good idea, but it could be in certain use cases :)
The example shown on the documentation page you link to is a composition type of relationship (in OOP terms) and as such represents the owns type of relationship rather then uses in terms of verbs. Therefore each owner would have its own copy of the same (in terms of value) keyword.
In fact, you can use exactly the suggestion from the documentation you link to in your question to create a custom creator method and hack it to reuse existing object for given key instead of just creating a new one. In this case the sample code of the User class and creator function will look like below:
def _keyword_find_or_create(kw):
keyword = Keyword.query.filter_by(keyword=kw).first()
if not(keyword):
keyword = Keyword(keyword=kw)
# if aufoflush=False used in the session, then uncomment below
#session.add(keyword)
#session.flush()
return keyword
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String(64))
kw = relationship("Keyword", secondary=lambda: userkeywords_table)
keywords = association_proxy('kw', 'keyword',
creator=_keyword_find_or_create, # #note: this is the
)
I recently ran into the same problem. Mike Bayer, creator of SQLAlchemy, refered me to the “unique object” recipe but also showed me a variant that uses an event listener. The latter approach modifies the association proxy so that UserKeyword.keyword temporarily points to a plain string and only creates a new Keyword object if the keyword doesn't already exist.
from sqlalchemy import event
# Same User and Keyword classes from documentation
class UserKeyword(Base):
__tablename__ = 'user_keywords'
# Columns
user_id = Column(Integer, ForeignKey(User.id), primary_key=True)
keyword_id = Column(Integer, ForeignKey(Keyword.id), primary_key=True)
special_key = Column(String(50))
# Bidirectional attribute/collection of 'user'/'user_keywords'
user = relationship(
User,
backref=backref(
'user_keywords',
cascade='all, delete-orphan'
)
)
# Reference to the 'Keyword' object
keyword = relationship(Keyword)
def __init__(self, keyword=None, user=None, special_key=None):
self._keyword_keyword = keyword_keyword # temporary, will turn into a
# Keyword when we attach to a
# Session
self.special_key = special_key
#property
def keyword_keyword(self):
if self.keyword is not None:
return self.keyword.keyword
else:
return self._keyword_keyword
#event.listens_for(Session, "after_attach")
def after_attach(session, instance):
# when UserKeyword objects are attached to a Session, figure out what
# Keyword in the database it should point to, or create a new one
if isinstance(instance, UserKeyword):
with session.no_autoflush:
keyword = session.query(Keyword).\
filter_by(keyword=instance._keyword_keyword).\
first()
if keyword is None:
keyword = Keyword(keyword=instance._keyword_keyword)
instance.keyword = keyword

Constraining a django Many-to-Many Relationship using unique_together

I think this may be more SQL than Django but Django is what I'm working in. What I am trying to do is to come up with a object model which can have many properties but is constrained to only 1 property type per object.
Say we have 3 property types:
is_cool
is_happy
is_mean
Suppose I have an object (MyObject) which can have * (0-All) of these properties applied to it but only one of each.
So I think this is diagramed as follows (please correct me if I'm wrong):
In Django I am stuggling with this constraint. I want it at the db level i.e using unique_together.
Here is what I have..
PROP_VALUE_CHOICES = (("URL", "url"),
("Boolean", "bool"),
("String", "char"),
("Person", "person"))
class PropertyType(models.Model):
name = models.CharField(max_length=32)
value_type = models.CharField(max_length=32, choices=PROP_VALUE_CHOICES)
class Property(models.Model):
type = models.ForeignKey(PropertyType)
value = models.CharField(max_length=32)
class MyObjectA(models.Model):
properties = models.ManyToManyField(Property, related_name="MyObjectA")
class MyObjectB(models.Model):
properties = models.ManyToManyField(Property, related_name="MyObjectB")
So the questions:
Is the above picture the correct way to document what I'm trying to accomplish.
My model is not complete - what am I missing and where do I apply the unique together constraint on the Object name and property type.
BTW - This is similar to this post but they used a through which I'm not sure I need??
Thanks!!
In case anyone really is looking for this answer...
Using Abstract Base Class I created the following structure which should work. Granted it no longer represents the picture completely but does solve the problem.
PROP_VALUE_CHOICES = (("URL", "url"),
("Boolean", "bool"),
("String", "char"),
("Person", "person"))
class PropertyType(models.Model):
name = models.CharField(max_length=32)
value_type = models.CharField(max_length=32, choices=PROP_VALUE_CHOICES)
class Property(models.Model):
type = models.ForeignKey(PropertyType, unique=True, related_name="%(app_label)s_%(class)s_related")
value = models.CharField(max_length=32)
class Meta:
abstract = True
class ObjectAProperties(Property): pass
class ObjectA(models.Model):
properties = models.ManyToManyField(Property, through="ObjectAProperties")
class ObjectBProperties(Property): pass
class ObjectB(models.Model):
properties = models.ManyToManyField(Property, through="ObjectBProperties")
Posted in case I need this again in the future!