Creating a self-referencing M2M relationship in SQLAlchemy (+Flask) - sqlalchemy

While trying to learn Flask, I am building a simple Twitter clone. This would include the ability for a User to follow other Users. I am trying to set up a relational database through SQLAlchemy to allow this.
I figured I would need a self-referencing many-to-many relationship on the User. Following from the SQLAlchemy documentation I arrived at:
#imports omitted
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///twitclone.db'
db = SQLAlchemy(app)
Base = declarative_base()
user_to_user = Table("user_to_user", Base.metadata,
Column("follower_id", Integer, ForeignKey("user.id"), primary_key=True),
Column("followed_id", Integer, ForeignKey("user.id"), primary_key=True)
)
class User(db.Model):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String, unique=False)
handle = Column(String, unique=True)
password = Column(String, unique=False)
children = relationship("tweet")
following = relationship("user",
secondary=user_to_user,
primaryjoin=id==user_to_user.c.follower_id,
secondaryjoin=id==user_to_user.c.followed_id,
backref="followed_by"
)
#Tweet class goes here
db.create_all()
if __name__ == "__main__":
app.run()
Running this code results in the database being created without any error messages. However, the whole part (table) connecting a user to a user is simply omitted. This is the definition of the User table:
CREATE TABLE user (
id INTEGER NOT NULL,
name VARCHAR,
handle VARCHAR,
password VARCHAR,
PRIMARY KEY (id),
UNIQUE (handle)
)
Why does SQLAlchemy not create the self-referential relationship for the User?
note: I am new to both Flask and SQLAlchemy and could be missing something obvious here.

Ok, it seems I mixed up two different styles of using SQLAlchemy with Flask: the declarative extension of SQLAlchemy and flask-sqlalchemy extension. Both are similar in capabilities with the difference being that the flask extension has some goodies like session handling. This is how I rewrote my code to strictly make use of flask-sqlalchemy.
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
from datetime import datetime
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///kwek.db'
db = SQLAlchemy(app)
#Table to handle the self-referencing many-to-many relationship for the User class:
#First column holds the user who follows, the second the user who is being followed.
user_to_user = db.Table('user_to_user',
db.Column("follower_id", db.Integer, db.ForeignKey("user.id"), primary_key=True),
db.Column("followed_id", db.Integer, db.ForeignKey("user.id"), primary_key=True)
)
class User(db.Model):
__tablename__ = 'user'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(64), unique=False)
handle = db.Column(db.String(16), unique=True)
password = db.Column(db.String, unique=False)
kweks = db.relationship("Kwek", lazy="dynamic")
following = db.relationship("User",
secondary=user_to_user,
primaryjoin=id==user_to_user.c.follower_id,
secondaryjoin=id==user_to_user.c.followed_id,
backref="followed_by"
)
def __repr__(self):
return '<User %r>' % self.name
class Kwek(db.Model):
__tablename__ = 'kwek'
id = db.Column(db.Integer, primary_key=True)
content = db.Column(db.String(140), unique=False)
post_date = db.Column(db.DateTime, default=datetime.now())
user_id = db.Column(db.Integer, db.ForeignKey('user.id'))
def __repr__(self):
return '<Kwek %r>' % self.content
if __name__ == "__main__":
app.run()

Related

SQLalchemy mutually dependent foreign key constraints

I’m trying to define 2 entities like this:
class User(Base):
id = Column(Integer, primary_key=True)
name = Column(String(256), index=True, unique=True)
main_token_id = Column(ForeignKey('token.id'), nullable=False)
main_token = relationship('Token', uselist=False)
tokens = relationship('Token', back_populates="user", foreign_keys=['token.id'])
class Token(Base):
id = Column(Integer, primary_key=True)
user_id = Column(ForeignKey('user.id'), nullable=False)
user: User = relationship("user", back_populates="tokens")
I want the user to have access to the collection of all his tokens and I also want him to have a special, main token. I want to ensure that the user has just one main token and I need integrity provided by the foreign key. By both of them actually.
I have read Cascading deletes in mutually dependent tables in SQLAlchemy but I don't feel it helps. I would like to have the integrity from both sides.
How can I make this work? If the design is flawed how can I rephrase this so that I may keep my integrity guarantees?
A kludge I have used to sort of solve this problem before is to create a column like precedence = Column(Integer, nullable=False) on tokens. Then set a unique constraint like UniqueConstraint('user_id', 'precedence'). Then set that integer manually when you create the tokens. The token with precedence 0 or the lowest precedence is the main token.
Here is an example. I'm sure some sqlalchemy geniuses can perform the precedence swap without 3 updates but I think in most cases that doesn't come up very often. There is a way to defer the unique constraint within a transaction but I guess sqlite does not support that yet.
This relies on your application not clearing the main token from precedence 0, ie. no integrity check to prevent that.
from sqlalchemy import (
create_engine,
UnicodeText,
Integer,
String,
ForeignKey,
UniqueConstraint,
update,
)
from sqlalchemy.schema import (
Table,
Column,
MetaData,
)
from sqlalchemy.sql import select
from sqlalchemy.orm import declarative_base, relationship
from sqlalchemy.orm import Session
from sqlalchemy.exc import IntegrityError
Base = declarative_base()
engine = create_engine("sqlite://", echo=False)
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String(256), index=True, unique=True)
tokens = relationship('Token', backref="user", cascade="all, delete-orphan", order_by='Token.precedence')
main_token = relationship('Token', primaryjoin='and_(User.id == Token.user_id, Token.precedence == 0)', viewonly=True, uselist=False)
class Token(Base):
__tablename__ = 'tokens'
id = Column(Integer, primary_key=True)
precedence = Column(Integer, nullable=False)
user_id = Column(ForeignKey('users.id'), nullable=False)
__table_args__ = (UniqueConstraint('precedence', 'user_id', name='tokens_user_precedence'),)
Base.metadata.create_all(engine)
with Session(engine) as session:
user = User(name='tokenizer')
session.add(user)
main_token = Token(user=user, precedence=0)
session.add(main_token)
session.add(Token(user=user, precedence=1))
session.commit()
assert session.query(Token).first()
assert session.query(User).first()
assert session.query(User).first().tokens
assert session.query(User).first().tokens[0] == main_token
# This viewonly relationship seems to be working.
assert session.query(User).first().main_token == main_token
# We don't want this so don't do this, no integrity checks here!!
main_token.precedence = 100
session.commit()
assert not session.query(User).first().main_token
# Put it back now.
main_token.precedence = 0
session.commit()
assert session.query(User).first().main_token
# Now check tokens are cleared.
session.delete(user)
session.commit()
assert not session.query(Token).all()
assert not session.query(User).all()
with Session(engine) as session:
# Try making 2 main tokens.
user = User(name='tokenizer')
session.add(user)
main_token = Token(user=user, precedence=0)
main_token2 = Token(user=user, precedence=0)
session.add_all([main_token, main_token2])
try:
session.commit()
except IntegrityError as e:
pass
else:
assert False, 'Exception should have occurred.'
with Session(engine) as session:
# Try swapping the tokens.
user = User(name='tokenizer')
session.add(user)
main_token = Token(user=user, precedence=0)
session.add(main_token)
other_token = Token(user=user, precedence=1)
session.add(other_token)
session.commit()
old_precedence = other_token.precedence
main_token.precedence = -1
session.flush()
other_token.precedence = 0
session.flush()
main_token.precedence = old_precedence
session.commit()
user.tokens[0] == other_token
user.tokens[1] == main_token
user.main_token == other_token
session.commit()

SqlAlchemy error: Foreign key could not find table

So I'm building an app and I'm trying to save new changes to my database but when I try to commit the changes in the flask using db.session.commit() it returns me the following error:
sqlalchemy.exc.NoReferencedTableError: Foreign key associated with column 'products.country_id' could not find table 'countries' with which to generate a foreign key to target column 'id'
In my models.py I have the following:
from app import db
from . import db
from datetime import datetime
def now():
return datetime.now()
class Countries(db.Model):
__tablename__ = 'countries'
__table_args__ = {'schema': 'products_data'}
id = db.Column(db.Integer, primary_key=True, unique=True, autoincrement=True)
name = db.Column(db.String(255))
code = db.Column(db.String(45))
def __repr__(self):
return f'Id {self.id}'
class Categories(db.Model):
__tablename__ = 'categories'
__table_args__ = {'schema': 'products_data'}
id = db.Column(db.Integer, primary_key=True, unique=True, autoincrement=True)
name = db.Column(db.String(255))
def __repr__(self):
return f'Id {self.id}'
class Brands(db.Model):
__tablename__ = 'brands'
__table_args__ = {'schema': 'products_data'}
id = db.Column(db.Integer, primary_key=True, unique=True, autoincrement=True)
name = db.Column(db.String(255))
logo = db.Column(db.String(5000))
feed = db.Column(db.String(5000))
feed_type = db.Column(db.String(45))
category_id = db.Column(db.Integer, db.ForeignKey('categories.id'))
country_id = db.Column(db.Integer, db.ForeignKey('countries.id'))
awinmid = db.Column(db.Integer)
def __repr__(self):
return f'Id {self.id}'
class Products(db.Model):
__tablename__ = 'products'
__table_args__ = {'schema': 'products_data'}
id = db.Column(db.Integer, primary_key=True, unique=True, autoincrement=True)
name = db.Column(db.String(255))
url = db.Column(db.Text)
category_id = db.Column(db.Integer, db.ForeignKey('categories.id'))
country_id = db.Column(db.Integer, db.ForeignKey('countries.id'))
price = db.Column(db.Float)
currency = db.Column(db.String(45))
discount_price = db.Column(db.Float)
shipping = db.Column(db.Float)
brand_id = db.Column(db.Integer, db.ForeignKey('brands.id'))
Am I doing anything wrong when associating a column in products with a foreign key? This is the first time I encounter this error so I'm really lost on what to do right now.
To fix I just added the schema to the db.ForeignKey and it worked
Example:
db.ForeignKey('products_data.countries.id')
PS:
Not my idea. Just wanted to post the answer in case someone visits the post later with the same problem.
Gord Thompson thanks for the help!
First of all, I don t see any table Categories. Secondly, you copy pasted your schema from the Products table into your Countries one.
PS: By default sqlalchemy gives the tables the name of the class (lower cased). So your __tablename__='products' does nothing actually.
EDIT:
The problem with your code lies in how you set the __table_args__ attribute. You assign an object to it, which by their specifications is wrong.
Take a look at the following example and modify your code accordingly
__table_args__ = ({'schema': 'products_data'})
Also for further reference, take a look at this https://docs.sqlalchemy.org/en/13/orm/extensions/declarative/table_config.html

Implementation of One-to-One relationship in Flask SQLAlchemy

I am inexperienced programmer. I'd like to add one-to-one relationship CurrencyDefault (that value will be assigned to the field in FlaskForm) between User and Currency:
class User(UserMixin, db.Model):
id = db.Column(db.Integer, db.Sequence('user_id_seq'), primary_key=True)
username = db.Column(db.String(64), index=True, unique=True)
email = db.Column(db.String(64), index=True, unique=True)
pswd_hash = db.Column(db.String(128))
expenses = db.relationship('Expense', backref='user')
currency = db.relationship('Currency', backref='user')
curr_default = ?
# ...
class Currency(db.Model):
id = db.Column(db.Integer, db.Sequence('expense_id_seq'), primary_key=True)
abbr = db.Column(db.String(10))
name = db.Column(db.String(64))
user_id = db.Column(db.Integer, db.ForeignKey('user.id'))
default = ?
# ...
What I want to achieve is to assign to each user.id one currency.id (one-to-one)
I'd like to ask for some advice what is the best way to make it.
After considering the problem I have some ideas like:
Association Table with uselist=False relationship,
Create a new class CurrencyDefault(id, user_id, currency_id),
Or maybe there is other, better way to achieve it?
I'm very curious of your point of view on this problem.
Implementing solution:
This is how my classes look like now:
class User(UserMixin, db.Model):
id = db.Column(db.Integer, db.Sequence('user_id_seq'), primary_key=True)
username = db.Column(db.String(64), index=True, unique=True)
email = db.Column(db.String(64), index=True, unique=True)
pswd_hash = db.Column(db.String(128))
currency_default_choice = db.Column(db.Integer, b.ForeignKey('currency.id'))
expenses = db.relationship('Expense', backref='user')
currency = db.relationship('Currency', backref='user', foreign_keys="Currency.user_id")
# currency_default = db.relationship(
# 'Currency',
# foreign_keys='User.currency_default_choice',
# backref='currency_default',
# uselist=False,
# )
# ...
class Currency(db.Model):
id = db.Column(db.Integer, db.Sequence('expense_id_seq'), primary_key=True)
abbr = db.Column(db.String(10), b.ForeignKey('currency_official_abbr.abbr'))
name = db.Column(db.String(64))
user_id = db.Column(db.Integer, db.ForeignKey('user.id'))
currency_default = db.relationship(
'User',
foreign_keys='User.currency_default_choice',
backref='currency_default',
uselist=False,
)
The first problem I find is that I can set Currency.id object created by other_user as currency_default_choice. How to restrict currency_default_choice only to the <Currency> that was created by this user?
What is the difference between setting relationship having foreign key in User class (currency_default_choice = db.Column(db.Integer, b.ForeignKey('currency.id'))) with:
class Currency(db.Model):
# ...
currency_default = db.relationship(
'User',
foreign_keys='User.currency_default_choice',
backref='currency_default',
uselist=False,
)
and setting this relationship on User side with:
class User(db.Model):
# ...
currency_default = db.relationship(
'Currency',
foreign_keys='User.currency_default_choice',
backref='currency_default',
uselist=False,
)
Ad.2. What seems to me is that there is no difference between these two ways because the backref parameter implicates bidirectional behavior so it doesn't matter if I placed db.relationship() in User or Currency class. Is it correct?
Using Python shell I added value to the User.currency_default
>>> app = create_app()
>>> app.app_context().push()
>>> admin = User.query.filter_by(username='admin').first()
<User(id= 1, username = admin, email = admin#admin.com)
>>> currency = Currency.query.filter_by(user=admin)
>>> currency
<flask_sqlalchemy.BaseQuery object at 0x03EA05D0>
>>> currency[0].id
1
>>> admin.currency_default = currency[0]
>>> db.session.commit()
>>> currency[0].currency_default
<User(id= 1, username = admin, email = admin#admin.com)
>>> admin.currency_default_choice
1
and then using Admin Panel after running flask run I wanted to remove introduced value but I got error that I don't understand. Why there is circular dependency between (Currency.currency_default),(User.currency_default) and (User.currency)? I don't understand what is happening. How to fix it?
sqlalchemy.exc.CircularDependencyError: Circular dependency detected.
(ProcessState(OneToManyDP(Currency.currency_default),
<Currency at 0x46542b0>, delete=False),
ProcessState(ManyToOneDP(User.currency_default),
<User at 0x4669b10>, delete=False),
SaveUpdateState(<Currency at 0x46542b0>),
ProcessState(OneToManyDP(User.currency), <User at 0x4669b10>, delete=False),
SaveUpdateState(<User at 0x4669b10>))

Flask-Migrate sqlalchemy.exc.NoReferencedTableError: Foreign key associated with column

I am using Flask-Migrate in my application, with the following models:
listpull/models.py
from datetime import datetime
from listpull import db
class Job(db.Model):
id = db.Column(db.Integer, primary_key=True)
list_type_id = db.Column(db.Integer, db.ForeignKey('listtype.id'),
nullable=False)
list_type = db.relationship('ListType',
backref=db.backref('jobs', lazy='dynamic'))
record_count = db.Column(db.Integer, nullable=False)
status = db.Column(db.Integer, nullable=False)
sf_job_id = db.Column(db.Integer, nullable=False)
created_at = db.Column(db.DateTime, nullable=False)
compressed_csv = db.Column(db.LargeBinary)
def __init__(self, list_type, created_at=None):
self.list_type = list_type
if created_at is None:
created_at = datetime.utcnow()
self.created_at = created_at
def __repr__(self):
return '<Job {}>'.format(self.id)
class ListType(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(80), unique=True, nullable=False)
def __init__(self, name):
self.name = name
def __repr__(self):
return '<ListType {}>'.format(self.name)
run.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from listpull import app, manager
manager.run()
listpull/__init__.py
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
from flask.ext.script import Manager
from flask.ext.migrate import Migrate, MigrateCommand
from mom.client import SQLClient
from smartfocus.restclient import RESTClient
app = Flask(__name__)
app.config.from_object('config')
db = SQLAlchemy(app)
migrate = Migrate(app, db)
manager = Manager(app)
manager.add_command('db', MigrateCommand)
...
import listpull.models
import listpull.views
I initialized the database using ./run.py db init and then I run ./run.py db migrate and I get the following error:
sqlalchemy.exc.NoReferencedTableError: Foreign key associated with column 'job.list_type_id' could not find table 'listtype' with which to generate a foreign key to target column 'id'
What am I doing wrong here?
You are letting Flask-SQLAlchemy choose the names for your tables. If I remember correctly, for a class called ListType the table name will be list_type (or something similar), not the listtype that you specified in your foreign key.
My recommendation is that you specify your own table names using __tablename__, that way they are explicit in the code and not magically determined for you. For example:
class Job(db.Model):
__tablename__ = 'jobs'
id = db.Column(db.Integer, primary_key=True)
list_type_id = db.Column(db.Integer, db.ForeignKey('list_types.id'),
nullable=False)
# ...
class ListType(db.Model):
__tablename__ = 'list_types'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(80), unique=True, nullable=False)
# ...
I was looking for a solution for similar information problem you deal with for a long time as well... at the end, the solution i found was add into every "table-class" the line __tablename__ = '<your_table_name>'
it seems to help flask to find your table
In my case, the issue was that i accidentally called the foreign key by the class name vs the table name ...
Like: db.Column(db.Integer, db.ForeignKey('ClassListTypes.id') vs db.Column(db.Integer, db.ForeignKey('list_types.id')

After I create my tables using SQLAlchemy, how can I add additional columns to it?

This is my file so far:
from sqlalchemy import create_engine, ForeignKey
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship, backref
from sqlalchemy import Column, Integer, String
from sqlalchemy import Table, Text
engine = create_engine('mysql://root:ababab#localhost/alctest',
echo=False)
Base = declarative_base()
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key = True)
name = Column(String(100))
fullname = Column(String(100))
password = Column(String(100))
addresses = relationship("Address", order_by="Address.id", backref="user")
def __init__(self, name, fullname, password):
self.name = name
self.fullname = fullname
self.password = password
def __repr__(self):
return "<User('%s','%s', '%s')>" % (self.name, self.fullname, self.password)
class Address(Base):
__tablename__ = 'addresses'
id = Column(Integer, primary_key = True)
email_address = Column(String(100), nullable=False)
#foreign key, must define relationship
user_id = Column(Integer, ForeignKey('users.id'))
user = relationship("User", backref = backref('addresses',order_by=id))
Base.metadata.create_all(engine)
This file is pretty simple. It creates a User and Address tables. After I run this file, the tables are created.
But now I want to add a column to "User". How can I do that? What do I have to do?
You can add column with Table.append_column method.
test = Column('test', Integer)
User.__table__.append_column(test)
But this will not fire the ALTER TABLE command to add that column in database. As per doc given for append_column that command you have to run manually after adding that column in model.
Short answer: You cannot: AFAIK, currently there is no way to do it from sqlalchemy directly.
Howerever, you can use sqlalchemy-migrate for this if you change your model frequently and have different versions rolled out to production. Else it might be an overkill and you may be better off generating the ALTER TABLE ... scripts manually.