I want to either update a row in the database, if it exists, or create it if it doesn't.
I have a class that first sets the instance variable user:
self.user = models.User.query.filter_by(entity=self.entityUrl).first()
# can be None if not found
Then, later on in another class method I do this:
if self.user is not None:
self.user.foo = bar # etc. Change the attributes of self.user
else:
self.user = models.User(bar, ... ) # create a new model
db.session.add(self.user)
db.session.commit()
Problem is, the corresponding row in the database doesn't get updated. I've also tried this approach:
if self.user is not None:
self.user.foo = bar
else:
self.user = models.User(bar, ... )
db.session.add(self.user) # add it to the session either way
db.session.commit()
Here, the db.session.add() call fails with sqlalchemy.exc.InvalidRequestError: Object '<User at 0x7f4918172890>' is already attached to session '1' (this is '2')
And the first thing I tried was to delete the existing model in all cases, then create a new one, i.e.:
if self.user is not None:
db.session.delete(self.user)
self.user = models.User(bar, ... )
db.session.add(self.user)
db.session.commit()
In this case the db.session.delete() call fails with the same already attached to session '1' message as above.
Why is the object attached to a different session and not the same one? How do I do this correctly?
Make sure the foo attribute in your class existes. Next, maybe there is something wrong in the way you use it. Because I see you use "self.user....". Try the simplest thing first. Then step by step.
The following code is wrong:
if self.user is not None:
self.user.foo = bar
else:
self.user = models.User(bar, ... )
db.session.add(self.user) # add it to the session either way
db.session.commit()
NO need to db.session.add, if you want to update the record.
To update existing record using Flask-SQLAlchemy, you do not need to re-create the entire User object and add it to the session. You just update the specific field (e.g. foo) and thats it. You can then do the db commit.
You can do your exact requirement as below:
Step 1: query the existing user object
user = models.User.query.filter_by(entity=self.entityUrl).first()
Step 2:
if user is not None:
user.foo = bar
else:
user = User(...)
db.session.add(user)
Step 3: commit the db session.
db.session.commit()
Related
I'm working on a project using Flask and a PostgreSQL database, with SQLAlchemy.
I have Group objects which have a list of User IDs who are members of the group. For some reason, when I try to add an ID to a group, it will not save properly.
If I try members.append(user_id), it doesn't seem to work at all. However, if I try members += [user_id], the id will show up in the view listing all the groups, but if I restart the server, the added value(s) is (are) not there. The initial values, however, are.
Related code:
Adding group to the database initially:
db = SQLAlchemy(app)
# ...
g = Group(request.form['name'], user_id)
db.session.add(g)
db.session.commit()
The Group class:
from flask.ext.sqlalchemy import SQLAlchemy
from sqlalchemy.dialects.postgresql import ARRAY
class Group(db.Model):
__tablename__ = "groups"
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(128))
leader = db.Column(db.Integer)
# list of the members in the group based on user id
members = db.Column(ARRAY(db.Integer))
def __init__(self, name, leader):
self.name = name
self.leader = leader
self.members = [leader]
def __repr__(self):
return "Name: {}, Leader: {}, Members: {}".format(self.name, self.leader, self.members)
def add_user(self, user_id):
self.members += [user_id]
My test function for updating the Group:
def add_2_to_group():
g = Group.query.all()[0]
g.add_user(2)
db.session.commit()
return redirect(url_for('show_groups'))
Thanks for any help!
As you have mentioned, the ARRAY datatype in sqlalchemy is immutable. This means it isn’t possible to add new data into array once it has been initialised.
To solve this, create class MutableList.
from sqlalchemy.ext.mutable import Mutable
class MutableList(Mutable, list):
def append(self, value):
list.append(self, value)
self.changed()
#classmethod
def coerce(cls, key, value):
if not isinstance(value, MutableList):
if isinstance(value, list):
return MutableList(value)
return Mutable.coerce(key, value)
else:
return value
This snippet allows you to extend a list to add mutability to it. So, now you can use the class above to create a mutable array type like:
class Group(db.Model):
...
members = db.Column(MutableList.as_mutable(ARRAY(db.Integer)))
...
You can use the flag_modified function to mark the property as having changed. In this example, you could change your add_user method to:
from sqlalchemy.orm.attributes import flag_modified
# ~~~
def add_user(self, user_id):
self.members += [user_id]
flag_modified(self, 'members')
To anyone in the future: so it turns out that arrays through SQLAlchemy are immutable. So, once they're initialized in the database, they can't change size. There's probably a way to do this, but there are better ways to do what we're trying to do.
This is a hacky solution, but what you can do is:
Store the existing array temporarily
Set the column value to None
Set the column value to the existing temporary array
For example:
g = Group.query.all()[0]
temp_array = g.members
g.members = None
db.session.commit()
db.session.refresh(g)
g.members = temp_array
db.session.commit()
In my case it was solved by using the new reference for storing a object variable and assiging that new created variable in object variable.so, Instead of updating the existing objects variable it will create a new reference address which reflect the changes.
Here in Model,
Table: question
optional_id = sa.Column(sa.ARRAY(sa.Integer), nullable=True)
In views,
option_list=list(question.optional_id if question.optional_id else [])
if option_list:
question.optional_id.clear()
option_list.append(obj.id)
question.optional_id=option_list
else:
question.optional_id=[obj.id]
I have two databases (both Mysql) that have exactly the same tables, and I want to copy some data from one to another using Sqlalchemy.
I can copy simple objects following the answer given in this question:
Cannot move object from one database to another
The problem is when the object has dependencies from another table, and I want to copy the dependencies as well.
So to make it more clear, this is my model (the same for both databases but using a different bind_key that points to a different database):
db1 = SQLAlchemy()
Class Payment(db.Model):
__tablename__ = 'payments'
__bind_key__ = 'db1'
id = db1.Column(db.Integer, primary_key=True)
paymethod_id = db1.Column(db.Integer(), db1.ForeignKey(PaymentMethod.id))
payment_method = db1.relationship(PaymentMethod)
What I would like to do is the following:
from models1 import Payment as Payment1
from models2 import Payment as Payment2
# query from one database
payment1 = db1.session.query(Payment1).first()
# create and add it to the other database
payment2 = Payment2(**payment1.__dict__.copy())
db2.session.add(payment)
db2.session.commit()
But in this case the foreign key fails because I don't have the PaymentMethod stored yet.
Is there a different approach to do that or I would have to do this procedure for every dependency of my object and be sure that I store the children beforehand?
Any help is appreciated :)
I came up with a solution that remaps the object to the right model and stores all its children. You call the method save_obj and pass the object you want to map. It will then retrieve a table with the same name but then from the model you want to remap the object to and it will recursively do the same for all its children. You have to define the right model in the method get_model.
To run this is necessary to disable autoflush to prevent committing before the object is correctly formed and it is also necessary to commit after calling the method. I'm using flask-sqlalchemy.
Hope this can help or give some insight to someone that faces a similar problem :)
def save_obj(obj, checked=[]):
if obj in checked:
# if the object was already converted, retrieve the right object
model = get_model(obj.__mapper__.mapped_table.name)
return get_obj(obj, model)
checked.append(obj)
children = []
relations = obj.__mapper__.relationships.items()
# get all the relationships of this model (foreign keys)
for relation in relations:
model = get_model(relation[1].table.name)
if model:
# remove the cascade option for this object, so the children are not stored automatically in the session
relation[1]._cascade = CascadeOptions('')
child = getattr(obj, relation[0])
if not child:
continue
# if the child is a set of children
if isinstance(child, list):
new_children = []
for ch in copy(child):
# convert the child
new_child = save_obj(ch, checked)
new_children.append(new_child)
children.append((relation[0], new_children))
else:
new_child = save_obj(child, checked)
children.append((relation[0], new_child))
# get the model of the object passed
model = get_model(obj.__mapper__.mapped_table.name)
new_obj = get_obj(obj, model)
# set all the children in this object
for child in children:
if child[1]:
setattr(new_obj, child[0], child[1])
checked.append(new_obj)
session.add(new_obj)
return new_obj
def get_model(table_name):
# get the right model for this object
for table in db.Model._decl_class_registry.values():
if hasattr(table, '__tablename__') and table.__tablename__ == table_name:
return table
return None
def create_new_obj(obj, model):
params = obj.__dict__.copy()
params.pop('_sa_instance_state')
return model(**params)
def get_obj(child, model):
# check if the object is already stored in the db
child_in_db = session.query(model).get(child.id)
if child_in_db:
return child_in_db
# check if the object is already in the session
for s in session.new:
if type(s) == model and s.id == child.id:
return s
return create_new_obj(child, model)
I have this application where I want to deploy some web interface to a syslog server.
So the syslogserver does write his stuff into a mysql database. I already have build
some parts for the application except for this specific part where I want to build a dropdown select form, to select the hosttables inside the database.
Actually I am using flask, flask-sqlalchemy and wtforms. So I tried to implement this over the 'QuerySelectField', somehow I only get a dropdown with no table name entries shown.
I should mention that the tables inside the database itself are created on the fly. For my model I used the automap_base() Feature from sqlalchemy:
model.py
Base = automap_base()
engine = create_engine("mysql://sumuser:tehpass#127.0.0.1/syslog")
Base.prepare(engine, reflect=True)
session = Session(engine)
This is whats inside my forms:
forms.py
def factoryHelper():
return session.query("information_schema.tables.table_name from information_schema.tables where information_schema.tables.table_name like 'messages_hostname0'")
class HostSelectForm(Form):
title = TextField('Fooblah')
hostTables = QuerySelectField(query_factory=factoryHelper,allow_blank=True)
and this inside the views:
views.py
#app.route('/index/', defaults={'page':1})
#app.route('/index/page/<int:page>')
def index(page):
form = HostSelectForm()
count = session.execute("select host,facility,level,msg from messages_hostname0").rowcount
pagination = Pagination(page, PER_PAGE, count)
return render_template('index.html', pagination=pagination, form=form)
So is there anyway I can create a dropdown menu from dynamically created table names? Also if I use the automap feature? Thanks in advance.
Somehow i managed to solve this issue with this in the model.py:
def reflectTables():
for i in Base.classes.items():
yield i[0]
def stripTables():
tablelist = []
datelist = []
re0 = r"^(?P<prefix>[a-zA-Z]*)\_(?P<mid>[a-zA-Z,0-9]*)\_(?P<suffix>[0-9]*)"
myre = re.compile(re0, re.MULTILINE)
for x,table in enumerate(reflectTables()):
striptablename = re.match(myre, table.strip("\n"))
if striptablename:
tablelist.append((x, striptablename.group(2)))
datelist.append((x, striptablename.group(3)))
return dicht(tablelist,datelist)
the forms.py:
AVAILABLE_CHOICES = stripTables()
class HostSelectForm(Form):
tableSelect = SelectField('LogHost', choices=AVAILABLE_CHOICES, default=0)
and finnaly inside the views.py:
if request.method == "GET":
count = session.query("* from mytable_monitory_counts")
items = session.execute("select * from mytable_%s_%s limit %s, %s" % \
(tableslector[int(request.values['tableSelect'])][1],\
datelist[int(request.values['tableSelect'])][1], my_start_range, PER_PAGE)).fetchall()
pagination = Pagination(page=page, total=count.count(), search=search, record_name='hosts')
if not items and page != 1:
in_error()
return render_template('index.html', pagination=pagination, form=form, items=items)
I am starting to learn rails and have run into a problem. I am writing a simple application (similar to the twitter tutorials I have seen) where a user logs in and creates a new post.
When a user logs in, I am setting the session information as follows
session[:id] = authorized_user.id
session[:email] = authorized_user.email
So now I have the ID of the user logged in. Upon login, the user is brought to a form where they can submit a new post (3 fields.) When user clicks submit, I want to create a new record with the data they entered, and associate the record to that user (User ID). I am not exactly sure how to do this.
Below is the code on the controller:
def create
#Used for creating new status posts
#Need to get the ID of the user logged in
#user = AdminUser.find(session[:id])
#Instantiate new object using form parameters
#post = Post.new(post_params)
#post.AdminUser = #user # THIS IS THE LINE NOT WORKING
#Save the object
if #post.save
#If save succeeds, redirect to the index action
flash[:notice] = "Status has been saved"
redirect_to(:action => 'index')
else
#If the save fails, redisplay the form so user can fix problems
render('new')
end
end
Here is the Private method for post_params
def post_params
#Defining the params that are allowed to be passed with forms.
params.require(:post).permit(:post_status, :post_title, :post_content)
end
The record is saved but the UserID for the record is NULL.
My first instict was to try to pass UserID as a post parameter, but i think this is a potential security risk, so I am trying to figure out an alternate way. I am sure it is something simple and I am just missing it.
Attributes
Firstly,
#post.AdminUser = #user # THIS IS THE LINE NOT WORKING
You should use snake_case for your attribute names (you're using CamelCase). Calling an attribute AdminUser has all sorts of potential issues which will arise down the line.
Call it admin_user or admin_id or something similar
--
Params
Secondly,
I want to create a new record with the data they entered, and
associate the record to that user (User ID)
If you're trying to save a "dependent" record for an object (for example, saving a post for a user), you'll have to assign the user_id record yourself, and pass it through the params, like so:
#app/controllers/posts_controller.rb
Class PostsController < ApplicationController
def create
#post = Post.new(post_params)
#post.save
end
private
def post_params
params.require(:post).permit(:title, :body).merge({user_id: authorized_user.id})
end
end
When you create an element in your app, you're basically just taking data from the params hash & sending to the model to save. This is done using the strong_params functionality introduced in Rails 4:
def post_params
params.require(:post).permit(:title, :body).merge({user_id: authorized_user.id})
end
As you can see from my example above, you basically need to be able to send through the user_id / admin_id / AdminUser value through to the model (so it can save)
You can also do this by setting the attribute as the example below:
#app/controllers/posts_controller.rb
def create
#post = Post.new(post_params)
#post.user_id = authorized_user.id
#post.save
end
private
def post_params
params.require(:post).permit(:title, :body, :user_id)
end
--
You should also look at the difference between authentication & autorhization for better definition of your logged-in user object :)
Rewrite the line like this, taking UserID as the column name in posts table
#post.UserID = #user.id
I defined some WTForms forms in an application that uses SQLALchemy to manage database operations.
For example, a form for managing Categories:
class CategoryForm(Form):
name = TextField(u'name', [validators.Required()])
And here's the corresponding SQLAlchemy model:
class Category(Base):
__tablename__= 'category'
id = Column(Integer, primary_key=True)
name = Column(Unicode(255))
def __repr__(self):
return '<Category %i>'% self.id
def __unicode__(self):
return self.name
I would like to add a unique constraint on the form validation (not on the model itself).
Reading the WTForms documentation, I found a way to do it with a simple class:
class Unique(object):
""" validator that checks field uniqueness """
def __init__(self, model, field, message=None):
self.model = model
self.field = field
if not message:
message = u'this element already exists'
self.message = message
def __call__(self, form, field):
check = self.model.query.filter(self.field == field.data).first()
if check:
raise ValidationError(self.message)
Now I can add that validator to the CategoryForm like this:
name = TextField(u'name', [validators.Required(), Unique(Category, Category.name)])
This check works great when the user tries to add a category that already exists \o/
BUT it won't work when the user tries to update an existing category (without changing the name attribute).
When you want to update an existing category : you'll instantiate the form with the category attribute to edit:
def category_update(category_id):
""" update the given category """
category = Category.query.get(category_id)
form = CategoryForm(request.form, category)
The main problem is I don't know how to access the existing category object in the validator which would let me exclude the edited object from the query.
Is there a way to do it? Thanks.
In the validation phase, you will have access to all the fields. So the trick here is to pass in the primary key into your edit form, e.g.
class CategoryEditForm(CategoryForm):
id = IntegerField(widget=HiddenInput())
Then, in the Unique validator, change the if-condition to:
check = self.model.query.filter(self.field == field.data).first()
if 'id' in form:
id = form.id.data
else:
id = None
if check and (id is None or id != check.id):
Although this is not a direct answer I am adding it because this question is flirting with being an XY Problem. WTForms primary job is to validate that the content of a form submission. While a decent case could be made that verifying that a field's uniqueness could be considered the responsibility of the form validator, a better case could be made that this is the responsibility of the storage engine.
In cases where I have be presented with this problem I have treated uniqueness as an optimistic case, allowed it to pass form submission and fail on a database constraint. I then catch the failure and add the error to the form.
The advantages are several. First it greatly simplifies your WTForms code because you do not have to write complex validation schemes. Secondly, it could improve your application's performance. This is because you do not have to dispatch a SELECT before you attempt to INSERT effectively doubling your database traffic.
The unique validator needs to use the new and the old data to compare first before checking if the data is unique.
class Unique(object):
...
def __call__(self, form, field):
if field.object_data == field.data:
return
check = DBSession.query(model).filter(field == data).first()
if check:
raise ValidationError(self.message)
Additionally, you may want to squash nulls too. Depending on if your truly unique or unique but allow nulls.
I use WTForms 1.0.5 and SQLAlchemy 0.9.1.
Declaration
from wtforms.validators import ValidationError
class Unique(object):
def __init__(self, model=None, pk="id", get_session=None, message=None,ignoreif=None):
self.pk = pk
self.model = model
self.message = message
self.get_session = get_session
self.ignoreif = ignoreif
if not self.ignoreif:
self.ignoreif = lambda field: not field.data
#property
def query(self):
self._check_for_session(self.model)
if self.get_session:
return self.get_session().query(self.model)
elif hasattr(self.model, 'query'):
return getattr(self.model, 'query')
else:
raise Exception(
'Validator requires either get_session or Flask-SQLAlchemy'
' styled query parameter'
)
def _check_for_session(self, model):
if not hasattr(model, 'query') and not self.get_session:
raise Exception('Could not obtain SQLAlchemy session.')
def __call__(self, form, field):
if self.ignoreif(field):
return True
query = self.query
query = query.filter(getattr(self.model,field.id)== form[field.id].data)
if form[self.pk].data:
query = query.filter(getattr(self.model,self.pk)!=form[self.pk].data)
obj = query.first()
if obj:
if self.message is None:
self.message = field.gettext(u'Already exists.')
raise ValidationError(self.message)
To use it
class ProductForm(Form):
id = HiddenField()
code = TextField("Code",validators=[DataRequired()],render_kw={"required": "required"})
name = TextField("Name",validators=[DataRequired()],render_kw={"required": "required"})
barcode = TextField("Barcode",
validators=[Unique(model= Product, get_session=lambda : db)],
render_kw={})
Looks like what you are looking for can easily be achieved with ModelForm which is built to handle forms that are strongly coupled with models (the category model in your case).
To use it:
...
from wtforms_components import Unique
from wtforms_alchemy import ModelForm
class CategoryForm(ModelForm):
name = TextField(u'name', [validators.Required(), Unique(Category, Category.name)])
It will verify unique values while considering the current value in the model. You can use the original Unique validator with it.
This worked for me, simple and easy:
Make sure that every time when a new row created in DB it must have unique name in colomn_name_in_db otherwise it will not work.
class SomeForm(FlaskForm):
id = IntegerField(widget=HiddenInput())
fieldname = StringField('Field name', validators=[DataRequired()])
...
def validate_fieldname(self, fieldname):
names_in_db = dict(Model.query.with_entities(Model.id,
Model.colomn_name_in_db).filter_by(some_filtes_if_needed).all())
if fieldname.data in names_in_db.values() and names_in_db[int(self.id)] != fieldname.data:
raise ValidationError('Name must be unique')