I'm beginner in Djanog and trying to display values that distinct result with foreinkey.
Here are my env and example model.
Django 1.8
Mysql5
Python2.7
class a_group(models.Model):
num = models.AutoField(primary_key=True)
title = models.CharField(max_length=50)
def __unicode__(self):
return self.title
class b_group(models.Model):
no = models.AutoField(primary_key=True)
group = models.ForeignKey(a_group)
And then I tried to distinct with group field like this.
g = b_group.objects.values('group').distinct()
But, as mentioned at here https://docs.djangoproject.com/en/dev/ref/models/querysets/#values , it only return pk, not title.
Is there anyway to get title field value also?
You can also refer to fields on related models with reverse relations through OneToOneField,ForeignKey and ManyToManyField attributes, you can do as follow:
g = b_group.objects.values('group__title').distinct()
to access a field of related model Django use by convention double underscore.
Related
Hi stack overflow community! This is my first question here, but I tried to find an answer beforehand. Right now I am working on loading data from json file like so (I actually have a json file named persons.json, not API) with the use of peewee to SQLite DB. As you can see, the json file has multiple nested dicts. My peewee model is as follows:
import json
import sqlite3
from peewee import *
db = SqliteDatabase('persons.sqlite3')
class Person(Model):
gender = CharField()
name = CharField()
location = CharField()
email = CharField()
login = CharField()
dob = CharField()
registered = CharField()
phone = CharField()
cell = CharField()
id_ = CharField()
picture = CharField()
nat = CharField()
count_dob = IntegerField()
And this is how I load all the data from json file to SQLite DB:
db.connect()
db.create_tables([Person])
with open('persons.json', encoding='utf8') as persons:
persons_data = json.load(persons)
for person in persons_data['results']:
p = Person(gender=person['gender'], name=person['name'], location=person['location'], email=person['email'],
login=person['login'], dob=person['dob'], registered=person['registered'], phone=person['phone'],
cell=person['cell'], id_=person['id'], picture=person['picture'], nat=person['nat'])
My question is, do you think the variables in my model are correctly defined (basically every single one of them as a CharField) ? The thing is, later whenever I query the DB and I need to access some of these nested dictionaries, they are actually a string, which I can convert with the use of ast.literal_eval back to dict, but I don't think it looks nice. I thought of a solution - for all the 'dictionary type' variables in my model ('location', 'dob' etc), instead of using CharField() I could probably use JSONField() - not sure how to do that though. Could you please advise on that one?
Relational databases do not support "nesting". It's foundational. Anything that is nested should probably be in a separate table or in its own column as a flat/scalar value.
https://en.wikipedia.org/wiki/Database_normalization
I have SQLAlchemy models on top of the MySQL db. I need to query almost all models (string or text fields) and find everything that contains a specific substring. And also, apply common filtering like object_type=type1. For exsmple:
class Model1(Model):
name = Column(String(100), nullable=False, unique=True)
version = Column(String(100))
description = Column(String(100))
updated_at = Column(TIMESTAMP(timezone=True))
# other fields
class Model2(Model):
name = Column(String(100), nullable=False, unique=True)
version = Column(String(100))
description = Column(String(100))
updated_at = Column(TIMESTAMP(timezone=True))
# other fields
class Model3(Model):
name = Column(String(100), nullable=False, unique=True)
version = Column(String(100))
description = Column(String(100))
updated_at = Column(TIMESTAMP(timezone=True))
# other fields
And then do query something like:
db.query(
Model1.any_of_all_columns.contains('sub_string') or
Model2.any_of_all_columns.contains('sub_string') or
Model3.any_of_all_columns.contains('sub_string')
).all()
Is it possible to build such an ORM query in one SQL to the db and dynamically add Model(table) names and columns?
For applying common filtering for all the columns, you can subscribe to sqlachemy events as following:
#event.listens_for(Query, "before_compile", retval=True)
def before_compile(query):
for ent in query.column_descriptions:
entity = ent['entity']
if entity is None:
continue
inspect_entity_for_mapper = inspect(ent['entity'])
mapper = getattr(inspect_entity_for_mapper, 'mapper', None)
if mapper and has_tenant_id:
query = query.enable_assertions(False).filter(
ent['entity’].object == object)
return query
This function will be called whenever you do Model.query() and add filter for your object.
I eventually gave up and did one big loop in which I make a separate request for each model:
from sqlalchemy import or_
def db_search(self, model, q, object_ids=None, status=None, m2m_ids=None):
"""
Build a query to the db for given model using 'q' search substring
and filter it by object ids, its status and m2m related model.
:param model: a model object which columns will be used for search.
:param q: the query substring we are trying to find in all
string/text columns of the model.
:param object_ids: list of ids we want to include in the search.
If the list is empty, the search query will return 0 results.
If object_ids is None, we will ignore this filter.
:param status: name of object status.
:param m2m_ids: list of many-to-many related object ids.
:return: sqlalchemy query result.
"""
# Filter out private columns and not string/text columns
string_text_columns = [
column.name for column in model.__table__.columns if
isinstance(column.type, (db.String, db.Text))
and column.name not in PRIVATE_COLUMN_NAMES
]
# Find only enum ForeignKey columns
foreign_key_columns = [
column.name for column in model.__table__.columns if
column.name.endswith("_id") and column.name in ENUM_OBJECTS
)
]
query_result = model.query
# Search in all string/text columns for the required query
# as % LIKE %
if q:
query_result = query_result.join(
# Join related enum tables for being able to search in
*[enum_tables_to_model_map[col]["model_name"] for col in
foreign_key_columns]
).filter(
or_(
# Search 'q' substring in all string/text columns
*[
getattr(model, col_name).like(f"%{q}%")
for col_name in string_text_columns
],
# Search 'q' substring in the enum tables
*[
enum_tables_to_model_map[col]["model_field"]
.like(f"%{q}%") for col in foreign_key_columns
]
)
)
# Apply filter by object ids if given and it's not None.
# If the object ids filter exist but it's empty, we should
# return an empty result
if object_ids is not None:
query_result = query_result.filter(model.id.in_(object_ids))
# Apply filter by status if given and if the model has the status
# column
if status and 'status_id' in model.__table__.columns:
query_result = query_result.filter(model.status_id == status.id)
if m2m_ids:
query_result = query_result.filter(
model.labels.any(Label.id.in_(m2m_ids)))
return query_result.all()
And call it:
result = {}
for model in db.Model._decl_class_registry.values():
# Search only in the public tables
# sqlalchemy.ext.declarative.clsregistry._ModuleMarker object
# located in the _decl_class_registry that is why we check
# instance type and whether it is subclass of the db.Model
if isinstance(model, type) and issubclass(model, db.Model) \
and model.__name__ in PUBLIC_MODEL_NAMES:
query_result = self.db_search(
model, q, object_ids.get(model.__name__), status=status,
m2m_ids=m2m_ids)
result[model.__tablename__] = query_result
This is far from the best solution, but it works for me.
I am facing issues with "makeflag" field which is bit(1) type in my database(MySQL). I have tried using booleanField and bit1booleanfield with below syntax. But i am getting error with both. when i try POST request with json data on this model,
I get error as
"Data too long for column" on passing 1 or 0 as value.
And when i give true or false as value, then i get 400 Bad Request.
Can someone please help me understand how can i post data using django and json for bit field (of mysql).
makeflag=models.BooleanField(db_column='MakeFlag', default=1)
makeflag=Bit1BooleanField()
My model is the next:
class Product(models.Model):
productid = models.AutoField(db_column='ProductID', primary_key=True)
name = models.CharField(db_column='Name', max_length=50)
productnumber = models.CharField(db_column='ProductNumber', max_length=25)
makeflag = models.TextField(db_column='MakeFlag', max_length=1)
color = models.CharField(db_column='Color', max_length=15, blank=True)
safetystocklevel = models.SmallIntegerField(db_column='SafetyStockLevel')
reorderpoint = models.SmallIntegerField(db_column='ReorderPoint')
standardcost = models.FloatField(db_column='StandardCost')
You probably need to use django-mysql for mysql specific functionality. Have a look docs for bit here
Say that I have this structure:
class Distinct_Alert(models.Model):
entities = models.ManyToManyField(to='Entity', through='Entity_To_Alert_Map')
objects = Distinct_Alert_Manager()
has_unattended = models.BooleanField(default=True)
latest_alert_datetime = models.DateTimeField(null=True)
class Entity_To_Alert_Map(models.Model):
entity = models.ForeignKey(Entity, on_delete=models.CASCADE)
distinct_alert = models.ForeignKey(Distinct_Alert, on_delete=models.CASCADE)
entity_alert_relationship_label = models.TextField()
class Entity(models.Model):
label = models.CharField(max_length=700, blank=False)
related_entities = models.ManyToManyField('self')
identical_entities = models.ManyToManyField('self')
objects = Entity_Manager()
disregarding the other fields, what I'm trying to do is get all the unique entities from a selection of distinct alerts. So say that I pull 3 distinct alerts, and each of them have 4 entities in its manytomany entity field, say that across them, a couple are shared, I want to get only the distinct ones.
I'm doing this:
ret_list = map(lambda x: x.get_dictionary(), itertools.chain(*[alert.entities.all() for alert in
Distinct_Alert.objects.filter(
has_unattended=True,
entities__related_entities__label=threat_group_label)]))
return [dict(t) for t in set([tuple(d.items()) for d in ret_list])]
But as I imagine, this isn't optimal at all since I end up pulling a lot of dupes and then deduping at the end. I've tried pulling the distinct value entities but that pulls me a Long that's used as a key to map the entities to the distinct alert table. Any way to improve this?
Can you try this?
entity_ids = EntityToAlertMap.objects.filter(distinct_alert__has_unattended=True,
entity__related_entities__label=thread_group_label)) \
.values_list('entity', flat=True).distinct()
Entity.objects.filter(id__in=entity_ids)
Django doc about values_list.
All I trying to do is simple blog website using Pyramid, SQLAlchemy. The form module I have chosen is Deform which uses Coland. So I have for now two fields in my form: name and url. Url creates by transliteration the name field, but it's nevermind. So I don't wanna have two articles with the same urls. I need somehow make the validator with Colland I think. But the problem is the validator performs per field, but not per Model record. I mean if I'd make validator for url field, I dont have information in my method about another fields, such as id or name, so I couldn't perform the validation.
For now I have there couple of strings I created for two hours =)
from slugify import slugify
def convertUrl(val):
return slugify(val) if val else val
class ArticleForm(colander.MappingSchema):
name = colander.SchemaNode(colander.String())
url = colander.SchemaNode(colander.String(),
preparer=convertUrl)
Actually, I thought I should perform such validation on a model level, i.e. in SQLAlchemy model, but of course futher rules don't work, because such rules exist mainly for making SQL scripts (CREATE TABLE):
class Article(TBase, Base):
""" The SQLAlchemy declarative model class for a Article object. """
__tablename__ = 'article'
id = Column(Integer, primary_key=True)
name = Column(Text, unique=True)
url = Column(Text, unique=True)
Actually my question doesn't refer neither to Deform nor to Colander, this validation must be performed at SQLAlchemy level, here's what i've come to:
#validates('url')
def validate_url_unique(self, key, value):
check_unique = DBSession.query(Article)\
.filter(and_(Article.url == value, Article.id != self.id)).first()
if check_unique:
# Doesn't work
raise ValueError('Something went wrong')
# Neither doesn't work
# assert not check_unique
return value