I'm trying to implement the following with Flask+SQLAlchemy:
I have two database models containing information about bee apiaries and bee hives. I would like to add feature to somehow connect both of them to Sensor model. Those can be attached to one of apiaries or one of bee hives. Here is what I have.
class Apiary(db.Model):
__tablename__ = 'apiary'
# ... fields ...
beehives = db.relationship("BeeHive", backref=db.backref('apiary', lazy='dynamic'), lazy='dynamic')
class BeeHive(db.Model)
__tablename__ = 'beehive'
# ... fields ...
apiary_id = db.Column(db.Integer(), db.ForeignKey('apiary.id'))
class SensorType(db.Model):
__tablename__ = 'sensor_type'
id = db.Column(db.Integer(), primary_key=True)
title = db.Column(db.Unicode(32), unique=True)
sensors = db.relationship('Sensor', backref=db.backref('sensor_type', lazy='dynamic'), lazy='dynamic')
class Sensor(db.Model):
__tablename__ = 'sensor'
id = db.Column(db.Integer(), primary_key=True)
serial = db.Column(UUID(), unique=True)
sensor_type_id = db.Column(db.Integer(), db.ForeignKey('sensor_type.id'))
readings = db.relationship('SensorReading', backref=db.backref('sensor', lazy='dynamic'), lazy='dynamic')
class SensorReading(db.Model):
__tablename__ = 'sensor_reading'
id = db.Column(db.BigInteger(), primary_key=True)
value = # TODO
timestamp = db.Column(db.DateTime(), default=db.func.now())
I was surfing through the internet, reading SQLAlchemy documentation and found something about "polymorphic loading". I have good feeling that this is probably what I was searching for, but don't know how to implement it in my case. I have seen similar thing in "Django world" and they call it "GenericForeignKey".
UPDATE: I have found SQLAlchemy examples about this type of association. Can anyone advice me which of those would be optimum approach? discriminator_on_related, generic_fk, table_per_association or table_per_related? Which of those will be the least headache in further expanding application? Cascading delete?
After two days of experiments I have came to final conclusion. Examples have been taken from this source.
"Discriminator on association" (candidate for answer):
(+) has backward reference
(?) can have 1 parent object
(-) complexity
"Generic Foreign Key":
(+) low complexity
(+) has backward reference
(?) can have 1 parent object
(-) programmer's code must take care of cascading actions
"table per association":
(+) multiple parents
(+) shared table stays intact
(?) number of tables raises with number of associated tables
(-) no backward reference
"table per related" (candidate for answer):
(+) every associated object is in same table
(+) can have multiple tables
(-) associated objects are somehow separated for each foreign table
Answer: "Discriminator on association" as Sensor do not have ability of superposition and therefore no need for multiple parents.
Related
Say that I have this structure:
class Distinct_Alert(models.Model):
entities = models.ManyToManyField(to='Entity', through='Entity_To_Alert_Map')
objects = Distinct_Alert_Manager()
has_unattended = models.BooleanField(default=True)
latest_alert_datetime = models.DateTimeField(null=True)
class Entity_To_Alert_Map(models.Model):
entity = models.ForeignKey(Entity, on_delete=models.CASCADE)
distinct_alert = models.ForeignKey(Distinct_Alert, on_delete=models.CASCADE)
entity_alert_relationship_label = models.TextField()
class Entity(models.Model):
label = models.CharField(max_length=700, blank=False)
related_entities = models.ManyToManyField('self')
identical_entities = models.ManyToManyField('self')
objects = Entity_Manager()
disregarding the other fields, what I'm trying to do is get all the unique entities from a selection of distinct alerts. So say that I pull 3 distinct alerts, and each of them have 4 entities in its manytomany entity field, say that across them, a couple are shared, I want to get only the distinct ones.
I'm doing this:
ret_list = map(lambda x: x.get_dictionary(), itertools.chain(*[alert.entities.all() for alert in
Distinct_Alert.objects.filter(
has_unattended=True,
entities__related_entities__label=threat_group_label)]))
return [dict(t) for t in set([tuple(d.items()) for d in ret_list])]
But as I imagine, this isn't optimal at all since I end up pulling a lot of dupes and then deduping at the end. I've tried pulling the distinct value entities but that pulls me a Long that's used as a key to map the entities to the distinct alert table. Any way to improve this?
Can you try this?
entity_ids = EntityToAlertMap.objects.filter(distinct_alert__has_unattended=True,
entity__related_entities__label=thread_group_label)) \
.values_list('entity', flat=True).distinct()
Entity.objects.filter(id__in=entity_ids)
Django doc about values_list.
I'm writing an elections application. In the process, I've defined an Election model and a Candidate model.
Note: I'm using Django version 1.3.7, Python 2.7.1.
One of Election's methods,
Election.count_first_place(self)
is intended to count the number of first place votes each candidate receives and update the candidates' numVotes attribute. But for some reason they all stay at zero, no matter the ballots.
Note: I'm implementing STV so each ballot contains an array(ballot.voteArray) of Candidates in order of most preferred (position zero) to least preferred (position n). I've implemented this list with a PickledObjectField (see link).
models.py
class Candidate(models.Model):
election = models.ForeignKey("Election")
numVotes = models.FloatField(blank=True)
class Ballot(models.Model):
election = models.ForeignKey("Election", related_name = "ballot_set")
voteArray = PickledObjectField(null=True,blank=True)
class Election(models.Model):
position = models.CharField(max_length = 50)
candidates = models.ManyToManyField(Candidate,related_name="elections_in",null=True,blank=True)
def count_first_place(self):
#retrieve all of the ballots cast in this election
ballots = Ballot.objects.filter(election = self)
for ballot in ballots.all():
# the first element of a ballot's voteArray is a Candidate object
first_place_choice = ballot.voteArray[0]
first_place_choice.numVotes += 1
first_place_choice.save()
ballot.save()
self.save()
Here is what happens when I run a test:
Note: I realize that I am saving way more often than is necessary. Just being absolutely sure while I test this thing that it saves when it needs to.
elec = Election(position="Student Body President")
elec.save()
j = Candidate(election=elec,numVotes=0)
j.save()
e = Candidate(election=elec,numVotes=0)
e.save()
b = Candidate(election=elec,numVotes=0)
b.save()
elec.candidates.add(j,e,b)
elec.save()
ballot1 = Ballot(election=elec,voteArray=[j,e,b])
ballot1.save()
ballot2 = Ballot(election=elec,voteArray=[j,b,e])
ballot2.save()
ballot3 = Ballot(election=elec,voteArray[e,b,j])
ballot3.save()
So after this bit, j has two 2 place votes, and e has 1. But when I run
elec.count_first_place()
j still has zero votes, as do e and b.
What's up with that????
This is a very strange table structure. Pickling other model instances is a very bad idea: the pickled versions will not update when their database rows do. Really you should be storing an array of candidate IDs, or even better create a many-to-many relationship from Ballot to Candidate with a through table indicating position.
But I think your problem is simpler than that. You say that the objects still have zero votes: that is because you have not updated those particular instances. Again, there is no direct relationship between a Django instance and the database row, other than on loading and saving. You'll need to reload the objects from the database to see any updates.
All I trying to do is simple blog website using Pyramid, SQLAlchemy. The form module I have chosen is Deform which uses Coland. So I have for now two fields in my form: name and url. Url creates by transliteration the name field, but it's nevermind. So I don't wanna have two articles with the same urls. I need somehow make the validator with Colland I think. But the problem is the validator performs per field, but not per Model record. I mean if I'd make validator for url field, I dont have information in my method about another fields, such as id or name, so I couldn't perform the validation.
For now I have there couple of strings I created for two hours =)
from slugify import slugify
def convertUrl(val):
return slugify(val) if val else val
class ArticleForm(colander.MappingSchema):
name = colander.SchemaNode(colander.String())
url = colander.SchemaNode(colander.String(),
preparer=convertUrl)
Actually, I thought I should perform such validation on a model level, i.e. in SQLAlchemy model, but of course futher rules don't work, because such rules exist mainly for making SQL scripts (CREATE TABLE):
class Article(TBase, Base):
""" The SQLAlchemy declarative model class for a Article object. """
__tablename__ = 'article'
id = Column(Integer, primary_key=True)
name = Column(Text, unique=True)
url = Column(Text, unique=True)
Actually my question doesn't refer neither to Deform nor to Colander, this validation must be performed at SQLAlchemy level, here's what i've come to:
#validates('url')
def validate_url_unique(self, key, value):
check_unique = DBSession.query(Article)\
.filter(and_(Article.url == value, Article.id != self.id)).first()
if check_unique:
# Doesn't work
raise ValueError('Something went wrong')
# Neither doesn't work
# assert not check_unique
return value
This source details how to use association proxies to create views and objects with values of an ORM object.
However, when I append an value that matches an existing object in the database (and said value is either unique or a primary key), it creates a conflicting object so I cannot commit.
So in my case is this only useful as a view, and I'll need to use ORM queries to retrieve the object to be appended.
Is this my only option or can I use merge (I may only be able to do this if it's a primary key and not a unique constraint), OR set up the constructor such that it will use an existing object in the database if it exists instead of creating a new object?
For example from the docs:
user.keywords.append('cheese inspector')
# Is translated by the association proxy into the operation:
user.kw.append(Keyword('cheese inspector'))
But I'd like to to be translated to something more like: (of course the query could fail).
keyword = session.query(Keyword).filter(Keyword.keyword == 'cheese inspector').one()
user.kw.append(keyword)
OR ideally
user.kw.append(Keyword('cheese inspector'))
session.merge() # retrieves identical object from the database, or keeps new one
session.commit() # success!
I suppose this may not even be a good idea, but it could be in certain use cases :)
The example shown on the documentation page you link to is a composition type of relationship (in OOP terms) and as such represents the owns type of relationship rather then uses in terms of verbs. Therefore each owner would have its own copy of the same (in terms of value) keyword.
In fact, you can use exactly the suggestion from the documentation you link to in your question to create a custom creator method and hack it to reuse existing object for given key instead of just creating a new one. In this case the sample code of the User class and creator function will look like below:
def _keyword_find_or_create(kw):
keyword = Keyword.query.filter_by(keyword=kw).first()
if not(keyword):
keyword = Keyword(keyword=kw)
# if aufoflush=False used in the session, then uncomment below
#session.add(keyword)
#session.flush()
return keyword
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String(64))
kw = relationship("Keyword", secondary=lambda: userkeywords_table)
keywords = association_proxy('kw', 'keyword',
creator=_keyword_find_or_create, # #note: this is the
)
I recently ran into the same problem. Mike Bayer, creator of SQLAlchemy, refered me to the “unique object” recipe but also showed me a variant that uses an event listener. The latter approach modifies the association proxy so that UserKeyword.keyword temporarily points to a plain string and only creates a new Keyword object if the keyword doesn't already exist.
from sqlalchemy import event
# Same User and Keyword classes from documentation
class UserKeyword(Base):
__tablename__ = 'user_keywords'
# Columns
user_id = Column(Integer, ForeignKey(User.id), primary_key=True)
keyword_id = Column(Integer, ForeignKey(Keyword.id), primary_key=True)
special_key = Column(String(50))
# Bidirectional attribute/collection of 'user'/'user_keywords'
user = relationship(
User,
backref=backref(
'user_keywords',
cascade='all, delete-orphan'
)
)
# Reference to the 'Keyword' object
keyword = relationship(Keyword)
def __init__(self, keyword=None, user=None, special_key=None):
self._keyword_keyword = keyword_keyword # temporary, will turn into a
# Keyword when we attach to a
# Session
self.special_key = special_key
#property
def keyword_keyword(self):
if self.keyword is not None:
return self.keyword.keyword
else:
return self._keyword_keyword
#event.listens_for(Session, "after_attach")
def after_attach(session, instance):
# when UserKeyword objects are attached to a Session, figure out what
# Keyword in the database it should point to, or create a new one
if isinstance(instance, UserKeyword):
with session.no_autoflush:
keyword = session.query(Keyword).\
filter_by(keyword=instance._keyword_keyword).\
first()
if keyword is None:
keyword = Keyword(keyword=instance._keyword_keyword)
instance.keyword = keyword
I think this may be more SQL than Django but Django is what I'm working in. What I am trying to do is to come up with a object model which can have many properties but is constrained to only 1 property type per object.
Say we have 3 property types:
is_cool
is_happy
is_mean
Suppose I have an object (MyObject) which can have * (0-All) of these properties applied to it but only one of each.
So I think this is diagramed as follows (please correct me if I'm wrong):
In Django I am stuggling with this constraint. I want it at the db level i.e using unique_together.
Here is what I have..
PROP_VALUE_CHOICES = (("URL", "url"),
("Boolean", "bool"),
("String", "char"),
("Person", "person"))
class PropertyType(models.Model):
name = models.CharField(max_length=32)
value_type = models.CharField(max_length=32, choices=PROP_VALUE_CHOICES)
class Property(models.Model):
type = models.ForeignKey(PropertyType)
value = models.CharField(max_length=32)
class MyObjectA(models.Model):
properties = models.ManyToManyField(Property, related_name="MyObjectA")
class MyObjectB(models.Model):
properties = models.ManyToManyField(Property, related_name="MyObjectB")
So the questions:
Is the above picture the correct way to document what I'm trying to accomplish.
My model is not complete - what am I missing and where do I apply the unique together constraint on the Object name and property type.
BTW - This is similar to this post but they used a through which I'm not sure I need??
Thanks!!
In case anyone really is looking for this answer...
Using Abstract Base Class I created the following structure which should work. Granted it no longer represents the picture completely but does solve the problem.
PROP_VALUE_CHOICES = (("URL", "url"),
("Boolean", "bool"),
("String", "char"),
("Person", "person"))
class PropertyType(models.Model):
name = models.CharField(max_length=32)
value_type = models.CharField(max_length=32, choices=PROP_VALUE_CHOICES)
class Property(models.Model):
type = models.ForeignKey(PropertyType, unique=True, related_name="%(app_label)s_%(class)s_related")
value = models.CharField(max_length=32)
class Meta:
abstract = True
class ObjectAProperties(Property): pass
class ObjectA(models.Model):
properties = models.ManyToManyField(Property, through="ObjectAProperties")
class ObjectBProperties(Property): pass
class ObjectB(models.Model):
properties = models.ManyToManyField(Property, through="ObjectBProperties")
Posted in case I need this again in the future!