I think this may be more SQL than Django but Django is what I'm working in. What I am trying to do is to come up with a object model which can have many properties but is constrained to only 1 property type per object.
Say we have 3 property types:
is_cool
is_happy
is_mean
Suppose I have an object (MyObject) which can have * (0-All) of these properties applied to it but only one of each.
So I think this is diagramed as follows (please correct me if I'm wrong):
In Django I am stuggling with this constraint. I want it at the db level i.e using unique_together.
Here is what I have..
PROP_VALUE_CHOICES = (("URL", "url"),
("Boolean", "bool"),
("String", "char"),
("Person", "person"))
class PropertyType(models.Model):
name = models.CharField(max_length=32)
value_type = models.CharField(max_length=32, choices=PROP_VALUE_CHOICES)
class Property(models.Model):
type = models.ForeignKey(PropertyType)
value = models.CharField(max_length=32)
class MyObjectA(models.Model):
properties = models.ManyToManyField(Property, related_name="MyObjectA")
class MyObjectB(models.Model):
properties = models.ManyToManyField(Property, related_name="MyObjectB")
So the questions:
Is the above picture the correct way to document what I'm trying to accomplish.
My model is not complete - what am I missing and where do I apply the unique together constraint on the Object name and property type.
BTW - This is similar to this post but they used a through which I'm not sure I need??
Thanks!!
In case anyone really is looking for this answer...
Using Abstract Base Class I created the following structure which should work. Granted it no longer represents the picture completely but does solve the problem.
PROP_VALUE_CHOICES = (("URL", "url"),
("Boolean", "bool"),
("String", "char"),
("Person", "person"))
class PropertyType(models.Model):
name = models.CharField(max_length=32)
value_type = models.CharField(max_length=32, choices=PROP_VALUE_CHOICES)
class Property(models.Model):
type = models.ForeignKey(PropertyType, unique=True, related_name="%(app_label)s_%(class)s_related")
value = models.CharField(max_length=32)
class Meta:
abstract = True
class ObjectAProperties(Property): pass
class ObjectA(models.Model):
properties = models.ManyToManyField(Property, through="ObjectAProperties")
class ObjectBProperties(Property): pass
class ObjectB(models.Model):
properties = models.ManyToManyField(Property, through="ObjectBProperties")
Posted in case I need this again in the future!
Related
I wrote some table ORM in SqlAlchemy, and I want to share common functionality between them, this is my table ORM:
class RetailCampaignTemp(Base, Serialize, PrimaryField):
__tablename__ = 'retail_campaign_temp'
clicks = Column(INTEGER(11))
id = Column(INTEGER(11), primary_key=True)
Here are my custom functions:
class Serialize(object):
def Serialize(self):
return {c: getattr(self, c) for c in inspect(self).attrs.keys()}
class PrimaryField(object):
#declared_attr
def GetPrimaryField(cls):
yield from (column for column in cls.columns if column.primary_key)
When I call RetailCampaignTemp.GetPrimaryField() it say 'generator' object is not callable, what does this mean?
The culprit here is the #declared_attr decorator, which creates a class property, not a class method - that is, the expression RetailCampaignTemp.GetPrimaryField is already running the function body and returning the resulting generator, and RetailCampaignTemp.GetPrimaryField() is trying to call this generator.
This decorator is designed to be used to dynamically create sqlalchemy mapping/table declarations, e.g. by returning a relationship. From your example code it doesn't look like this is the case, so is there any reason you're not just using python's builtin #classmethod instead? This would make your call RetailCampaignTemp.GetPrimaryField() valid.
Hi stack overflow community! This is my first question here, but I tried to find an answer beforehand. Right now I am working on loading data from json file like so (I actually have a json file named persons.json, not API) with the use of peewee to SQLite DB. As you can see, the json file has multiple nested dicts. My peewee model is as follows:
import json
import sqlite3
from peewee import *
db = SqliteDatabase('persons.sqlite3')
class Person(Model):
gender = CharField()
name = CharField()
location = CharField()
email = CharField()
login = CharField()
dob = CharField()
registered = CharField()
phone = CharField()
cell = CharField()
id_ = CharField()
picture = CharField()
nat = CharField()
count_dob = IntegerField()
And this is how I load all the data from json file to SQLite DB:
db.connect()
db.create_tables([Person])
with open('persons.json', encoding='utf8') as persons:
persons_data = json.load(persons)
for person in persons_data['results']:
p = Person(gender=person['gender'], name=person['name'], location=person['location'], email=person['email'],
login=person['login'], dob=person['dob'], registered=person['registered'], phone=person['phone'],
cell=person['cell'], id_=person['id'], picture=person['picture'], nat=person['nat'])
My question is, do you think the variables in my model are correctly defined (basically every single one of them as a CharField) ? The thing is, later whenever I query the DB and I need to access some of these nested dictionaries, they are actually a string, which I can convert with the use of ast.literal_eval back to dict, but I don't think it looks nice. I thought of a solution - for all the 'dictionary type' variables in my model ('location', 'dob' etc), instead of using CharField() I could probably use JSONField() - not sure how to do that though. Could you please advise on that one?
Relational databases do not support "nesting". It's foundational. Anything that is nested should probably be in a separate table or in its own column as a flat/scalar value.
https://en.wikipedia.org/wiki/Database_normalization
In SqlAlchemy I use:
price = Column(Numeric(18, 5))
in various placed throught my app. When I get a number formatted in swedish, with a comma instead of a dot (0,34 instead of 0.34) and try to change the price column the number gets set to 0.00000.
To solve this I have this code:
obj.price = price.replace(',','.')
But having this all over the code makes it pretty ugly and the risk is that I forget one place. Would it be possible to have some kind of generic converter function which gets called before a value is converted from a string to a Numeric? And that I have that in one place only.
Check the validates decorator of SQLAlchemy: http://docs.sqlalchemy.org/en/rel_1_0/orm/mapped_attributes.html
A quick way to add a “validation” routine to an attribute is to use
the validates() decorator. An attribute validator can raise an
exception, halting the process of mutating the attribute’s value, or
can change the given value into something different.
In your case the code could look similar to:
from sqlalchemy.orm import validates
class Obj(Base):
__tablename__ = 'obj'
id = Column(Integer, primary_key=True)
price = Column(Numeric(18, 5))
#validates('price')
def validate_price(self, key, price):
if ',' in price:
return float(price.replace(',','.'))
else:
return float(price)
All I trying to do is simple blog website using Pyramid, SQLAlchemy. The form module I have chosen is Deform which uses Coland. So I have for now two fields in my form: name and url. Url creates by transliteration the name field, but it's nevermind. So I don't wanna have two articles with the same urls. I need somehow make the validator with Colland I think. But the problem is the validator performs per field, but not per Model record. I mean if I'd make validator for url field, I dont have information in my method about another fields, such as id or name, so I couldn't perform the validation.
For now I have there couple of strings I created for two hours =)
from slugify import slugify
def convertUrl(val):
return slugify(val) if val else val
class ArticleForm(colander.MappingSchema):
name = colander.SchemaNode(colander.String())
url = colander.SchemaNode(colander.String(),
preparer=convertUrl)
Actually, I thought I should perform such validation on a model level, i.e. in SQLAlchemy model, but of course futher rules don't work, because such rules exist mainly for making SQL scripts (CREATE TABLE):
class Article(TBase, Base):
""" The SQLAlchemy declarative model class for a Article object. """
__tablename__ = 'article'
id = Column(Integer, primary_key=True)
name = Column(Text, unique=True)
url = Column(Text, unique=True)
Actually my question doesn't refer neither to Deform nor to Colander, this validation must be performed at SQLAlchemy level, here's what i've come to:
#validates('url')
def validate_url_unique(self, key, value):
check_unique = DBSession.query(Article)\
.filter(and_(Article.url == value, Article.id != self.id)).first()
if check_unique:
# Doesn't work
raise ValueError('Something went wrong')
# Neither doesn't work
# assert not check_unique
return value
How do I update an HSTORE field with Flask-Admin?
The regular ModelView doesn't show the HSTORE field in Edit view. It shows nothing. No control at all. In list view, it shows a column with data in JSON notation. That's fine with me.
Using a custom ModelView, I can change the HSTORE field into a TextAreaField. This will show me the HSTORE field in JSON notation when in edit view. But I cannot edit/update it. In list view, it still shows me the object in JSON notation. Looks fine to me.
class MyView(ModelView):
form_overrides = dict(attributes=fields.TextAreaField)
When I attempt to save/edit the JSON, I receive this error:
sqlalchemy.exc.InternalError
InternalError: (InternalError) Unexpected end of string
LINE 1: UPDATE mytable SET attributes='{}' WHERE mytable.id = ...
^
'UPDATE mytable SET attributes=%(attributes)s WHERE mytable.id = %(mytable_id)s' {'attributes': u'{}', 'mytable_id': 14L}
Now -- using code, I can get something to save into the HSTORE field:
class MyView(ModelView):
form_overrides = dict(attributes=fields.TextAreaField)
def on_model_change(self, form, model, is_created):
model.attributes = {"a": "1"}
return
This basically overrides the model and put this object into it. I can then see the object in the List view and the Edit view. Still not good enough -- I want to save/edit the object that the user typed in.
I tried to parse and save the content from the form into JSON and back out. This doesn't work:
class MyView(ModelView):
form_overrides = dict(attributes=fields.TextAreaField)
def on_model_change(self, form, model, is_created):
x = form.data['attributes']
y = json.loads(x)
model.attributes = y
return
json.loads(x) says this:
ValueError ValueError: Expecting property name: line 1 column 1 (char
1)
and here are some sample inputs that fail:
{u's': u'ff'}
{'s':'ff'}
However, this input works:
{}
Blank also works
This is my SQL Table:
CREATE TABLE mytable (
id BIGSERIAL UNIQUE PRIMARY KEY,
attributes hstore
);
This is my SQA Model:
class MyTable(Base):
__tablename__ = u'mytable'
id = Column(BigInteger, primary_key=True)
attributes = Column(HSTORE)
Here is how I added the view's to the admin object
admin.add_view(ModelView(models.MyTable, db.session))
Add the view using a custom Model View
admin.add_view(MyView(models.MyTable, db.session))
But I don't do those views at the same time -- I get a Blueprint name collision error -- separate issue)
I also attempted to use a form field converter. I couldn't get it to actually hit the code.
class MyModelConverter(AdminModelConverter):
def post_process(self, form_class, info):
raise Exception('here I am') #but it never hits this
return form_class
class MyView(ModelView):
form_overrides = dict(attributes=fields.TextAreaField)
The answer gives you a bit more then asked
Fist of all it "extends" hstore to be able to store actually JSON, not just key-value
So this structure is also OK:
{"key":{"inner_object_key":{"Another_key":"Done!","list":["no","problem"]}}}
So, first of all your ModelView should use custom converter
class ExtendedModelView(ModelView):
model_form_converter=CustomAdminConverter
Converter itself should know how to use hstore dialect:
class CustomAdminConverter(AdminModelConverter):
#converts('sqlalchemy.dialects.postgresql.hstore.HSTORE')
def conv_HSTORE(self, field_args, **extra):
return DictToHstoreField(**field_args)
This one as you can see uses custom WTForms field which converts data in both directions:
class DictToHstoreField(TextAreaField):
def process_data(self, value):
if value is None:
value = {}
else:
for key,obj in value.iteritems():
if (obj.startswith("{") and obj.endswith("}")) or (obj.startswith("[") and obj.endswith("]")):
try:
value[key]=json.loads(obj)
except:
pass #
self.data=json.dumps(value)
def process_formdata(self, valuelist):
if valuelist:
self.data = json.loads(valuelist[0])
for key,obj in self.data.iteritems():
if isinstance(obj,dict) or isinstance(obj,list):
self.data[key]=json.dumps(obj)
if isinstance(obj,int):
self.data[key]=str(obj)
The final step will be to actual use this data in application
I did not make it in common nice way for SQLalchemy, since was used with flask-restful, so I have only adoption for flask-restful in one direction, but I think it's easy to get the idea from here and do the rest.
And if your case is simple key-value storage so nothing additionaly should be done, just use it as is.
But if you want to unwrap JSON somewhere in code, it's simple like this whenever you use it, just wrap in function
if (value.startswith("{") and value.endswith("}")) or (value.startswith("[") and value.endswith("]")):
value=json.loads(value)
Creating dynamical field for actual nice non-JSON way for editing of data also possible by extending FormField and adding some javascript for adding/removing fields, but this is whole different story, in my case I needed actual json storage, with blackjack and lists :)
Was working on postgres JSON datatype. The above solution worked great with a minor modifications.
Tried
'sqlalchemy.dialects.postgresql.json.JSON',
'sqlalchemy.dialects.postgresql.JSON',
'dialects.postgresql.json.JSON',
'dialects.postgresql.JSON'
The above versions did not work.
Finally the following change worked
#converts('JSON')
And changed class DictToHstoreField to the following:
class DictToJSONField(fields.TextAreaField):
def process_data(self, value):
if value is None:
value = {}
self.data = json.dumps(value)
def process_formdata(self, valuelist):
if valuelist:
self.data = json.loads(valuelist[0])
else:
self.data = '{}'
Although, this is might not be the answer to your question, but by default SQLAlchemy's ORM doesn't detect in-place changes to HSTORE field values. But fortunately there's a solution: SQLAlchemy's MutableDict type:
from sqlalchemy.ext.mutable import MutableDict
class MyClass(Base):
__tablename__ = 'mytable'
id = Column(Integer, primary_key=True)
attributes = Column(MutableDict.as_mutable(HSTORE))
Now when you change something in-place:
my_object.attributes.['some_key'] = 'some value'
The hstore field will be updated after session.commit().