My mobileapp is going to be used by different schools( with different db but same structure).Logged in user(parent) will be connected with multiple db(if their two child in different schools) using DynamicDbRouter. Now the problem is django router forgets the DB while pointing to ForeignKey.
views.py
class StudentFees(LoginRequiredMixin,TemplateView):
template_name = 'student_fees.html'
def get_context_data(self,**kwargs):
context = super(StudentFees,self).get_context_data(**kwargs)
schl_id=kwargs['schl_id']
school_id=School.objects.get(id=kwargs['schl_id'])
with in_database(school_id) :
context['classes'] = StudentSection.objects.filter(student_detail=kwargs['student_id'])
context['invoices'] = Invoice.objects.filter(student_master=kwargs['student_id'],status=1)
print context
return context
Here student_detail & student_master are foreign key fields.
routers.py
class in_database(object):
def __init__(self, client, read=True, write=False):
...
database = {'ENGINE':'django.db.backends.mysql','NAME':client.db_name,'USER':client.username,'PASSWORD':client.password,'HOST':client.host,'PORT':client.port}
...
Instead of pointing to client_db.model it points to default_db.model while accessing foreign keys.
I can able to see context details in console.. but values not redering in student_fees.html. It throws error like
ProgrammingError :
default_db.studentsection' doesn't exist
Related
I have also asked this question in sqlalchemy discussion group.
I am using alembic to apply a migration for a postgresql view using an async engine. This is successfully applied to the database.
I have the following declarative mapped class to the view, defined as:
class MailingListView(Base):
"""View for mailing labels.
After metata reflection from db -> model expecting columns to
be available on this class.
"""
__tablename__ = "mailing_list_view"
# Specify the column override from the underlying view that is the primary key
id = Column(UUID(as_uuid=True), primary_key=True)
# Expecting these columns below to be mapped in this class after
# metadata reflection. Currently have to uncomment these
# to manually synchronise with view!
#
# addressee = Column(String)
# street = Column(String)
# town = Column(String)
# county = Column(String)
# postcode = Column(String)
# cursor = Column(String)
I am reflecting the views using the following:
def use_inspector(conn):
inspector = inspect(conn)
return inspector.get_view_names()
views = await connection.run_sync(use_inspector)
# I can see the table columns in __table__.c.keys()
# after the reflection below has run
await connection.run_sync(
target_metadata.reflect,
only=views,
views=True,
extend_existing=True,
)
After applying migrations and performing the above reflection I can see that my mapped model has the underlying table columns updated with those defined in the underlying view.
obj = MailingListView()
obj.__table__.c.keys()
However, the properties of my mapped class are not updated after reflection, raising an exception:
obj = MailingListView()
obj.town = "town" # this raises an exception with unavailable property
How is it possible for a postgresql db (asyncpg) + async sqlalchemy to:
Synchronise the columns of a declarative model with its underlying table after metadata reflection?
Currently, I have to manually specify the columns in the declarative model.
Using inspectdb, I have imported a legacy database, that contains entities with composite primary keys, in django . The database schema contains about 200 different entities and inspectdb is quite handy in that situation.
This is the schema in mysql:
CREATE TABLE `mymodel` (
`id` bigint(20) unsigned NOT NULL DEFAULT '0',
`siteid` bigint(20) unsigned NOT NULL DEFAULT '0',
...
PRIMARY KEY (`siteid`,`id`),
...
Following the autogenerated model in django (imported using python manager.py inspectdb)
class Mymodel(models.Model):
id = models.PositiveBigIntegerField()
siteid = models.PositiveBigIntegerField(primary_key=True)
...
class Meta:
managed = False
db_table = 'mymodel'
unique_together = (('siteid', 'id'),
I have registered all models in the admin site using the following approach:
from django.contrib import admin
from django.apps import apps
app = apps.get_app_config('appname')
for model_name, model in app.models.items():
admin.site.register(model)
After all the work is done, I navigate to the admin site and click on any object in the "mymodel" section and the following exception will be returned:
appname.models.Content.MultipleObjectsReturned: get() returned more than one Mymodel-- it returned more than 20!
Obviously, (this is what it seems to me at least) admin is using the siteid to get the object, tough it should use the unique_together from the Meta class.
Any suggestions how I can achieve to solve this with a general configuration and get the admin site module to query using the unique_together?
Yes you can solve this problem but you put a little more effort.
First you separate model-admin class for model Mymodel and customize model-admin class method:
Since django admin build change url in ChangeList class, So we can create a custom Changelist class like MymodelChangelist and pass id field value as a query params. We will use id field value to getting object.
Override get_object() method to use custom query for getting object from queryset
Override get_changelist() method of model-admin to set your custom Changelist class
Override save_model() method to save object explicitly.
admin.py
class MymodelChangelist(ChangeList):
# override changelist class
def url_for_result(self, result):
id = getattr(result, 'id')
pk = getattr(result, self.pk_attname)
url = reverse('admin:%s_%s_change' % (self.opts.app_label,
self.opts.model_name),
args=(quote(pk),),
current_app=self.model_admin.admin_site.name)
# Added `id` as query params to filter queryset to get unique object
url = url + "?id=" + str(id)
return url
#admin.register(Mymodel)
class MymodelAdmin(admin.ModelAdmin):
list_display = [
'id', 'siteid', 'other_model_fields'
]
def get_changelist(self, request, **kwargs):
"""
Return the ChangeList class for use on the changelist page.
"""
return MymodelChangelist
def get_object(self, request, object_id, from_field=None):
"""
Return an instance matching the field and value provided, the primary
key is used if no field is provided. Return ``None`` if no match is
found or the object_id fails validation.
"""
queryset = self.get_queryset(request)
model = queryset.model
field = model._meta.pk if from_field is None else model._meta.get_field(from_field)
try:
object_id = field.to_python(object_id)
# get id field value from query params
id = request.GET.get('id')
return queryset.get(**{'id': id, 'siteid': object_id})
except (model.DoesNotExist, ValidationError, ValueError):
return None
def save_model(self, request, obj, form, change):
cleaned_data = form.cleaned_data
if change:
id = cleaned_data.get('id')
siteid = cleaned_data.get('siteid')
other_fields = cleaned_data.get('other_fields')
self.model.objects.filter(id=id, siteid=siteid).update(other_fields=other_fields)
else:
obj.save()
Now you can update any objects and also add new object. But, On addition one case you can't add- siteid which is already added because of primary key validation
I have two databases (both Mysql) that have exactly the same tables, and I want to copy some data from one to another using Sqlalchemy.
I can copy simple objects following the answer given in this question:
Cannot move object from one database to another
The problem is when the object has dependencies from another table, and I want to copy the dependencies as well.
So to make it more clear, this is my model (the same for both databases but using a different bind_key that points to a different database):
db1 = SQLAlchemy()
Class Payment(db.Model):
__tablename__ = 'payments'
__bind_key__ = 'db1'
id = db1.Column(db.Integer, primary_key=True)
paymethod_id = db1.Column(db.Integer(), db1.ForeignKey(PaymentMethod.id))
payment_method = db1.relationship(PaymentMethod)
What I would like to do is the following:
from models1 import Payment as Payment1
from models2 import Payment as Payment2
# query from one database
payment1 = db1.session.query(Payment1).first()
# create and add it to the other database
payment2 = Payment2(**payment1.__dict__.copy())
db2.session.add(payment)
db2.session.commit()
But in this case the foreign key fails because I don't have the PaymentMethod stored yet.
Is there a different approach to do that or I would have to do this procedure for every dependency of my object and be sure that I store the children beforehand?
Any help is appreciated :)
I came up with a solution that remaps the object to the right model and stores all its children. You call the method save_obj and pass the object you want to map. It will then retrieve a table with the same name but then from the model you want to remap the object to and it will recursively do the same for all its children. You have to define the right model in the method get_model.
To run this is necessary to disable autoflush to prevent committing before the object is correctly formed and it is also necessary to commit after calling the method. I'm using flask-sqlalchemy.
Hope this can help or give some insight to someone that faces a similar problem :)
def save_obj(obj, checked=[]):
if obj in checked:
# if the object was already converted, retrieve the right object
model = get_model(obj.__mapper__.mapped_table.name)
return get_obj(obj, model)
checked.append(obj)
children = []
relations = obj.__mapper__.relationships.items()
# get all the relationships of this model (foreign keys)
for relation in relations:
model = get_model(relation[1].table.name)
if model:
# remove the cascade option for this object, so the children are not stored automatically in the session
relation[1]._cascade = CascadeOptions('')
child = getattr(obj, relation[0])
if not child:
continue
# if the child is a set of children
if isinstance(child, list):
new_children = []
for ch in copy(child):
# convert the child
new_child = save_obj(ch, checked)
new_children.append(new_child)
children.append((relation[0], new_children))
else:
new_child = save_obj(child, checked)
children.append((relation[0], new_child))
# get the model of the object passed
model = get_model(obj.__mapper__.mapped_table.name)
new_obj = get_obj(obj, model)
# set all the children in this object
for child in children:
if child[1]:
setattr(new_obj, child[0], child[1])
checked.append(new_obj)
session.add(new_obj)
return new_obj
def get_model(table_name):
# get the right model for this object
for table in db.Model._decl_class_registry.values():
if hasattr(table, '__tablename__') and table.__tablename__ == table_name:
return table
return None
def create_new_obj(obj, model):
params = obj.__dict__.copy()
params.pop('_sa_instance_state')
return model(**params)
def get_obj(child, model):
# check if the object is already stored in the db
child_in_db = session.query(model).get(child.id)
if child_in_db:
return child_in_db
# check if the object is already in the session
for s in session.new:
if type(s) == model and s.id == child.id:
return s
return create_new_obj(child, model)
I have this in my models.py
class Page(models.Model)
#fields
class News(Page)
#no fields
class NewsComment(models.Model)
news = models.Foreignkey(news)
name = models.CharField(max_length=128)
email = models.EmailField(max_length=75)
comment = models.TextField()
Every time I am trying this:
page = get_object_or_404(News, id=page_id)
and then
comment, created = NewsComment.objects.get_or_create(news=page, name=name, email=email, comment=text)
I get this error:
(1452, 'Cannot add or update a child row: a foreign key constraint fails (myproject_db.main_newscomment, CONSTRAINT news_id_refs_page_ptr_id_5a5b8a6204eece43 FOREIGN KEY (news_id) REFERENCES main_news (page_ptr_id))')
What am I doing wrong?
(PS: I am using MySQL with InnoDB storage engine)
If the News model has no fields you should implement the inheritance using a proxy model. It will lead to much simpler database schema, and much simpler and faster (!) queries. It will also eliminate most problems dealing with how model inheritance is implemented on the database level.
class Page(models.Model)
#fields
class News(Page)
class Meta:
proxy = True
I defined some WTForms forms in an application that uses SQLALchemy to manage database operations.
For example, a form for managing Categories:
class CategoryForm(Form):
name = TextField(u'name', [validators.Required()])
And here's the corresponding SQLAlchemy model:
class Category(Base):
__tablename__= 'category'
id = Column(Integer, primary_key=True)
name = Column(Unicode(255))
def __repr__(self):
return '<Category %i>'% self.id
def __unicode__(self):
return self.name
I would like to add a unique constraint on the form validation (not on the model itself).
Reading the WTForms documentation, I found a way to do it with a simple class:
class Unique(object):
""" validator that checks field uniqueness """
def __init__(self, model, field, message=None):
self.model = model
self.field = field
if not message:
message = u'this element already exists'
self.message = message
def __call__(self, form, field):
check = self.model.query.filter(self.field == field.data).first()
if check:
raise ValidationError(self.message)
Now I can add that validator to the CategoryForm like this:
name = TextField(u'name', [validators.Required(), Unique(Category, Category.name)])
This check works great when the user tries to add a category that already exists \o/
BUT it won't work when the user tries to update an existing category (without changing the name attribute).
When you want to update an existing category : you'll instantiate the form with the category attribute to edit:
def category_update(category_id):
""" update the given category """
category = Category.query.get(category_id)
form = CategoryForm(request.form, category)
The main problem is I don't know how to access the existing category object in the validator which would let me exclude the edited object from the query.
Is there a way to do it? Thanks.
In the validation phase, you will have access to all the fields. So the trick here is to pass in the primary key into your edit form, e.g.
class CategoryEditForm(CategoryForm):
id = IntegerField(widget=HiddenInput())
Then, in the Unique validator, change the if-condition to:
check = self.model.query.filter(self.field == field.data).first()
if 'id' in form:
id = form.id.data
else:
id = None
if check and (id is None or id != check.id):
Although this is not a direct answer I am adding it because this question is flirting with being an XY Problem. WTForms primary job is to validate that the content of a form submission. While a decent case could be made that verifying that a field's uniqueness could be considered the responsibility of the form validator, a better case could be made that this is the responsibility of the storage engine.
In cases where I have be presented with this problem I have treated uniqueness as an optimistic case, allowed it to pass form submission and fail on a database constraint. I then catch the failure and add the error to the form.
The advantages are several. First it greatly simplifies your WTForms code because you do not have to write complex validation schemes. Secondly, it could improve your application's performance. This is because you do not have to dispatch a SELECT before you attempt to INSERT effectively doubling your database traffic.
The unique validator needs to use the new and the old data to compare first before checking if the data is unique.
class Unique(object):
...
def __call__(self, form, field):
if field.object_data == field.data:
return
check = DBSession.query(model).filter(field == data).first()
if check:
raise ValidationError(self.message)
Additionally, you may want to squash nulls too. Depending on if your truly unique or unique but allow nulls.
I use WTForms 1.0.5 and SQLAlchemy 0.9.1.
Declaration
from wtforms.validators import ValidationError
class Unique(object):
def __init__(self, model=None, pk="id", get_session=None, message=None,ignoreif=None):
self.pk = pk
self.model = model
self.message = message
self.get_session = get_session
self.ignoreif = ignoreif
if not self.ignoreif:
self.ignoreif = lambda field: not field.data
#property
def query(self):
self._check_for_session(self.model)
if self.get_session:
return self.get_session().query(self.model)
elif hasattr(self.model, 'query'):
return getattr(self.model, 'query')
else:
raise Exception(
'Validator requires either get_session or Flask-SQLAlchemy'
' styled query parameter'
)
def _check_for_session(self, model):
if not hasattr(model, 'query') and not self.get_session:
raise Exception('Could not obtain SQLAlchemy session.')
def __call__(self, form, field):
if self.ignoreif(field):
return True
query = self.query
query = query.filter(getattr(self.model,field.id)== form[field.id].data)
if form[self.pk].data:
query = query.filter(getattr(self.model,self.pk)!=form[self.pk].data)
obj = query.first()
if obj:
if self.message is None:
self.message = field.gettext(u'Already exists.')
raise ValidationError(self.message)
To use it
class ProductForm(Form):
id = HiddenField()
code = TextField("Code",validators=[DataRequired()],render_kw={"required": "required"})
name = TextField("Name",validators=[DataRequired()],render_kw={"required": "required"})
barcode = TextField("Barcode",
validators=[Unique(model= Product, get_session=lambda : db)],
render_kw={})
Looks like what you are looking for can easily be achieved with ModelForm which is built to handle forms that are strongly coupled with models (the category model in your case).
To use it:
...
from wtforms_components import Unique
from wtforms_alchemy import ModelForm
class CategoryForm(ModelForm):
name = TextField(u'name', [validators.Required(), Unique(Category, Category.name)])
It will verify unique values while considering the current value in the model. You can use the original Unique validator with it.
This worked for me, simple and easy:
Make sure that every time when a new row created in DB it must have unique name in colomn_name_in_db otherwise it will not work.
class SomeForm(FlaskForm):
id = IntegerField(widget=HiddenInput())
fieldname = StringField('Field name', validators=[DataRequired()])
...
def validate_fieldname(self, fieldname):
names_in_db = dict(Model.query.with_entities(Model.id,
Model.colomn_name_in_db).filter_by(some_filtes_if_needed).all())
if fieldname.data in names_in_db.values() and names_in_db[int(self.id)] != fieldname.data:
raise ValidationError('Name must be unique')