I have this table structure in SQLAlchemy:
User(Base)
__tablename__ = 'users'
id = Column(Integer, primary_key = True)
addresses = relationship("Address", backref="user")
Address(Base)
__talbename__ = 'addresses'
id = Column(Integer, primary_key = True)
user_id = Column(Integer, ForeignKey(users.id))
So a Many (addresses) to One (user) relationship.
My question is now how can I easily add a an address to a user without deleting the addresses already stored.
In pure SQL I would just insert a row in the address table with a foreign key to the right user.
This is how I'm doing it right now in SQLAlchemy (method in User Class):
def add_address(self, address):
adds = self.addresses
adds.append(address)
self.addresses = adds
So What I'm basically doing is first finding all the addresses, then appending the list to overwrite the list again with the extra address.
I was wondering if there is a more efficient way to do this?
You can define another relationship between User and Address, which will not be loaded using lazy='noload' (see Setting Noload):
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key = True)
addresses = relationship("Address", backref="user")
addresses_noload = relationship("Address", lazy='noload')
def add_address(self, address):
adds = self.addresses_noload # #note: use the relationship which does not load items from DB
adds.append(address)
Also see sqlalchemy add child in one-to-many relationship for similar question.
Related
Each of my mapped class contains created_by and updated_by audit properties that I would like to set automatically upon INSERT and UPDATE of respective objects.
class User(Base):
__tablename__ = 'user'
id = Column(BigInteger, primary_key=True)
name = Column(Text, nullable=False)
...
class Address(Base):
__tablename__ = 'address'
id = Column(BigInteger, primary_key=True)
street = Column(Text, nullable=False)
...
created_by = Column(BigInteger) # references user.id
updated_by = Column(BigInteger) # references user.id
...
Is there a way to handle this centrally in SQLAlchemy? I looked at the events but it appears it needs to be setup for every single mapped class individually (note the SomeClass in the decorator).
#event.listens_for(SomeClass, 'before_insert')
def on_insert(mapper, connection, target):
target.created_by = context["current_user"] # I want to be able to do this not just for 'SomeClass' but for all mapped classes
#event.listens_for(SomeClass, 'before_update')
def on_update(mapper, connection, target):
target.updated_by = context["current_user"] # I want to be able to do this not just for 'SomeClass' but for all mapped classes
One solution here is to use the default parameters in the Column class provided by sqlalchemy. You can actually pass a callable to both default (to execute when first created) and onupdate to execute whenever updated.
def get_current_user():
return context["user"].id
class Address(Base):
__tablename__ = 'address'
...
created_by = Column(default = get_current_user)
updated_by = Column(default = get_current_user, onupdate=get_current_user)
Managed to figure it out, though somewhat concerned about using a dunder method __subclasses__() on declarative_base. If there is a better alternative do suggest.
def on_insert(mapper, connection, target):
target.created_by = context["user"].id
target.updated_at = datetime.utcnow()
def on_update(mapper, connection, target):
target.updated_by = context["user"].id
target.updated_at = datetime.utcnow()
Base.metadata.create_all()
mapped_classes = Base.__subclasses__()
for mapped_class in mapped_classes:
event.listen(mapped_class, 'before_insert', on_insert)
event.listen(mapped_class, 'before_update', on_update)
The context being referred to here is actually starlette-context
I have employee table and location table. All locations data is loaded into the table first. Then employee table will be loaded later using data from source system.
Tables:
Locations
location_id(pk)| code(unique)| city | country
Employees
emp_id(pk)| name| email | phone | location_id
Models(SQLAlchemy):
class Employee(Base):
__tablename__ = 'employee'
id = Column(Integer, primary_key=True)
location_id = Column(Integer, ForeignKey('location.id'))
location = relationship("Location")
class Location(Base):
__tablename__ = 'location'
id = Column(Integer, primary_key=True)
Employees data from source will have location code but while inserting into employee table I need location_id. I will be getting many records from source as part of API call. Is there a way I can use the location code and insert the data into employee table without DB call.
I have these two approaches - need a more optimized one.
Make call to DB using code and get location id for each employee.
Load all locations in memory in a map structure and use it to get location id for code.
Expecting something like
emp = Employee(name="a",email="a__#__.com",phone="123")
emp.location = Location(code="L1")
db.session.add(emp)
db.session.commit()
This should create employee record with location id corresponding to L1 code.
Maintaining a mapping of location codes to ids or location objects is an entirely reasonable strategy if locations are not being added or removed from the database too frequently.
If you need to control the size of the cache you could use a function decorated with functools.lru_cache:
import functools
import sqlalchemy as sa
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import orm
Base = declarative_base()
class Employee(Base):
__tablename__ = 'employee'
id = sa.Column(sa.Integer, primary_key=True)
location_id = sa.Column(sa.Integer, sa.ForeignKey('location.id'))
location = orm.relationship("Location")
class Location(Base):
__tablename__ = 'location'
id = sa.Column(sa.Integer, primary_key=True)
code = sa.Column(sa.String)
# Set echo=True on the engine so we can se the queries.
engine = sa.create_engine('sqlite:///', echo=True)
Base.metadata.drop_all(bind=engine, checkfirst=True)
Base.metadata.create_all(bind=engine)
session_factory = orm.sessionmaker(bind=engine)
Session = orm.scoped_session(session_factory)
#functools.lru_cache
def get_location_by_code(code):
# We can proxy the session factory to avoid having to pass
# the session to this function (otherwise the session is part
# of the cache key, which we don't want).
# We could add logic to handle a missing code.
return Session.query(Location).filter_by(code=code).one()
# Add a location to the db
session = Session()
session.add(Location(code='L1'))
session.commit()
Session.remove()
# Add some employees to the database.
# Observe that we only query for location once.
session = Session()
for _ in range(3):
employee = Employee()
employee.location = get_location_by_code('L1')
session.add(employee)
session.commit()
Session.remove()
There are 3 tables: Account, Role, User. Both Role and User have a foreign key account_id that points to Account.
A user can have multiple roles, hence the roles_users table which acts as the secondary relation table between Role and User.
The Account table is a tenant table for our app, it is used to separate different customers.
Note that all tables have (besides Account) have composite primary keys with account_id. This is done for a few reasons, but let's say it's done to keep everything consistent.
Now if I have a simple secondary relationship (User.roles - the one that is commented out) all works as expected. Well kind of.. it throws a legitimate warning (though I believe it should be an error):
SAWarning: relationship 'User.roles' will copy column role.account_id to column roles_users.account_id, which conflicts with relationship(s): 'User.roles' (copies user.account_id to roles_users.account_id). Consider applying viewonly=True to read-only relationships, or provide a primaryjoin condition marking writable columns with the foreign() annotation.
That's why I created the second relation User.roles - the one that is not commented out. Querying works as expected which has 2 conditions on join and everything. However I get this error when I try to save some roles on the user:
sqlalchemy.orm.exc.UnmappedColumnError: Can't execute sync rule for source column 'roles_users.role_id'; mapper 'Mapper|User|user' does not map this column. Try using an explicit `foreign_keys` collection which does not include destination column 'role.id' (or use a viewonly=True relation).
As far as I understand it, SA is not able to figure out how to save the secondary because it has a custom primaryjoin and secondaryjoin so it proposes to use viewonly=True which has the effect of just ignoring the roles relation when saving the model.
The question is how to save the roles for a user without having to do it by hand (the example is commented out in the code). In the real app we have many secondary relationships and we're saving them in many places. It would be super hard to rewrite them all.
Is there a solution to keep using User.roles = some_roles while keeping the custom primaryjoin and secondaryjoin below?
The full example using SA 1.1.9:
from sqlalchemy import create_engine, Column, Integer, Text, Table, ForeignKeyConstraint, ForeignKey, and_
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import foreign, relationship, Session
Base = declarative_base()
class Account(Base):
__tablename__ = 'account'
id = Column(Integer, primary_key=True)
roles_users = Table(
'roles_users', Base.metadata,
Column('account_id', Integer, primary_key=True),
Column('user_id', Integer, primary_key=True),
Column('role_id', Integer, primary_key=True),
ForeignKeyConstraint(['user_id', 'account_id'], ['user.id', 'user.account_id']),
ForeignKeyConstraint(['role_id', 'account_id'], ['role.id', 'role.account_id']),
)
class Role(Base):
__tablename__ = 'role'
id = Column(Integer, primary_key=True)
account_id = Column(Integer, ForeignKey('account.id'), primary_key=True)
name = Column(Text)
def __str__(self):
return '<Role {} {}>'.format(self.id, self.name)
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
account_id = Column(Integer, ForeignKey('account.id'), primary_key=True)
name = Column(Text)
# This works as expected: It saves data in roles_users
# roles = relationship(Role, secondary=roles_users)
# This custom relationship - does not work
roles = relationship(
Role,
secondary=roles_users,
primaryjoin=and_(foreign(Role.id) == roles_users.c.role_id,
Role.account_id == roles_users.c.account_id),
secondaryjoin=and_(foreign(id) == roles_users.c.user_id,
account_id == roles_users.c.account_id))
engine = create_engine('sqlite:///')
Base.metadata.create_all(engine)
session = Session(engine)
# Create our account
a = Account()
session.add(a)
session.commit()
# Create 2 roles
u_role = Role()
u_role.id = 1
u_role.account_id = a.id
u_role.name = 'user'
session.add(u_role)
m_role = Role()
m_role.id = 2
m_role.account_id = a.id
m_role.name = 'member'
session.add(m_role)
session.commit()
# Create 1 user
u = User()
u.id = 1
u.account_id = a.id
u.name = 'user'
# This does not work
u.roles = [u_role, m_role]
session.add(u)
session.commit()
# Works as expected
i = roles_users.insert()
i = i.values([
dict(account_id=a.id, role_id=u_role.id, user_id=u.id),
dict(account_id=a.id, role_id=m_role.id, user_id=u.id),
])
session.execute(i)
# re-fetch user from db
u = session.query(User).first()
for r in u.roles:
print(r)
NOTE: Switching primaryjoin with secondaryjoin does not help.
Solution for posterity sake - switch foreign wrappers and careful with primary vs secondary joins:
Instead of this:
roles = relationship(
Role,
secondary=roles_users,
primaryjoin=and_(foreign(Role.id) == roles_users.c.role_id,
Role.account_id == roles_users.c.account_id),
secondaryjoin=and_(foreign(id) == roles_users.c.user_id,
account_id == roles_users.c.account_id))
Do this:
roles = relationship(
Role,
secondary=roles_users,
primaryjoin=and_(id == foreign(roles_users.c.user_id), account_id == foreign(roles_users.c.account_id)),
secondaryjoin=and_(Role.id == foreign(roles_users.c.role_id), Role.account_id == roles_users.c.account_id),
)
How can I implement a self-referential many-to-many relationship that is effectively the union of two other relationships?
The relationship should return all FacebookFriendship models that exist between a user and other users in the network. A user may have a FacebookFriendship that points to another existing user, but due to FB API outages, privacy controls, etc, the mirror FBFriendship might not exist for the existing user to this user.
# This class is necessary for python-social-auth
# A UserSocialAuth model only exists for users who are in the network
class UserSocialAuth(_AppSession, Base, SQLAlchemyUserMixin):
"""Social Auth association model"""
__tablename__ = 'social_auth_usersocialauth'
__table_args__ = (UniqueConstraint('provider', 'uid'),)
id = Column(Integer, primary_key=True)
provider = Column(String(32))
uid = Column(String(UID_LENGTH))
extra_data = Column(JSONType())
user_id = Column(
Integer, ForeignKey(User.id), nullable=False, index=True)
user = relationship(
User,
backref=backref('social_auth', lazy='dynamic')
)
This relationship finds FacebookFriendship models that point from this user to any existing user.
facebook_friendships = relationship(
FacebookFriendship,
primaryjoin=and_(
user_id == FacebookFriendship.user_id,
provider == 'facebook'
),
secondary=FacebookFriendship.__table__,
secondaryjoin=uid == FacebookFriendship.fb_uid_friend,
foreign_keys=[provider, user_id, uid],
viewonly=True,
uselist=True,
lazy='dynamic',
)
This relationship finds FacebookFriendship models that point to this user.
other_facebook_friendships = relationship(
FacebookFriendship,
primaryjoin=and_(
uid == FacebookFriendship.fb_uid_friend,
provider == 'facebook'
),
foreign_keys=[provider, uid],
viewonly=True,
uselist=True,
lazy='dynamic',
)
I was able to express the union query using the hybrid_property decorator, but this prevents usage of comparators like any() or from using association proxies, at least from what I can tell.
# Can I rewrite this using relationship()?
#hybrid_property
def all_facebook_friendships(self):
return self.facebook_friendships.union(
self.other_facebook_friendships).correlate(
FacebookFriendship)
# FBFriendship models are created for every friend that a user has,
# regardless of whether they're in the network or not.
class FacebookFriendship(Base):
__tablename__ = u'user_fb_friend'
user_id = Column(Integer, sa.ForeignKey(User.id), primary_key=True)
user = relationship(
User, backref=backref('facebook_friendships', lazy='dynamic'),
primaryjoin=User.id == user_id)
fb_uid_friend = Column(sa.String(length=255), primary_key=True)
In the end, I'd like to query this relationship like any other InstrumentedAttribute:
UserSocialAuth.query.filter(UserSocialAuth.all_facebook_friendships.any()).all()
and define an association_proxy on the User model:
User.all_facebook_friends = association_proxy('all_facebook_friendships', 'user')
Sorry for the length of this question, but I've trialed & errored to no avail for days now.
Related:
How can I achieve a self-referencing many-to-many relationship on the SQLAlchemy ORM back referencing to the same attribute?
How to create relationship many to many in SQLAlchemy (python, flask) for model User to itself
Using zzzeek's solution linked above, I created a self-referential M2M relationship by using a select statement as the "secondary" argument to relationship().
friendship_union = select([
FacebookFriendship.dater_id,
cast(FacebookFriendship.fb_uid_friend, Integer()).label(
'fb_uid_friend')
]).union(
select([
cast(FacebookFriendship.fb_uid_friend, Integer()),
FacebookFriendship.dater_id]
)
).alias()
cls.all_fb_friendships = relationship(
UserSocialAuth,
secondary=friendship_union,
primaryjoin=UserSocialAuth.user_id == friendship_union.c.dater_id,
secondaryjoin=and_(
UserSocialAuth.provider == 'facebook',
cast(UserSocialAuth.uid, Integer() ) == friendship_union.c.fb_uid_friend,
),
viewonly=True
)
I'm trying to make a self-referential many-to-many relationship (it means that Line can have many parent lines and many child lines) in sqlalchemy like this:
Base = declarative_base()
class Association(Base):
__tablename__ = 'association'
prev_id = Column(Integer, ForeignKey('line.id'), primary_key=True)
next_id = Column(Integer, ForeignKey('line.id'), primary_key=True)
class Line(Base):
__tablename__ = 'line'
id = Column(Integer, primary_key = True)
text = Column(Text)
condition = Column(Text)
action = Column(Text)
next_lines = relationship(Association, backref="prev_lines")
class Root(Base):
__tablename__ = 'root'
name = Column(String, primary_key = True)
start_line_id = Column(Integer, ForeignKey('line.id'))
start_line = relationship('Line')
But I get the following error:
sqlalchemy.exc.ArgumentError: Could not determine join condition between parent/
child tables on relationship Line.next_lines. Specify a 'primaryjoin' expressio
n. If 'secondary' is present, 'secondaryjoin' is needed as well.
Do you know how I could remedy this?
You should just need:
prev_lines = relationship(
Association,
backref="next_lines",
primaryjoin=id==Association.prev_id)
Since this specifies the next_lines back reference there is no need to have a next_lines relationship.
You can also do this using the remote_side parameter to a relationship: http://www.sqlalchemy.org/trac/browser/examples/adjacency_list/adjacency_list.py