SQLAlchemy: Join table with object's children - sqlalchemy

I have three object types with corresponding tables:
class Order:
suborders = relationship('Suborder', lazy='dynamic')
class Suborder:
...
class PurchaseOrder:
suborder = relationship('Suborder', foreign_keys=[suborder_id], lazy='joined')
Now I need to get all PurchaseOrder instances matching Order's suborders. In pure SQL I'd write something like this:
SELECT po.*
FROM purchase_orders AS po JOIN suborders AS so ON po.suborder_id = so.id
WHERE so.order_id = 'order-007'
How do I do it using SQLAlchemy? I tried this:
o = Order.query.get('order-007')
PurchaseOrder.query.join(o.suborders)
But this gave me an error:
AttributeError: 'AppenderQuery' object has no attribute 'is_selectable'
What is the right way to do it?

Getting required data with just one query:
query = (
PurchaseOrder.query
.join(Suborder)
.filter(Suborder.order_id == 'order-007')
)
If you already have the Order instance o, you can do the following:
o = Order.query.get('order-007') # already have the order instance
query = (
session.query(PurchaseOrder)
.join(Suborder)
.with_parent(o)
)
, but you will still need to join on Suborder.
But again, the first one would correspond to the SQL query you provided as the SQL implementation.

Related

Using a cartesian product in LINQ and Entity framework to combine 3 tables

before I return "set.select" I would like to include fields from another table but I canot join this table because it has no fields in common with the other two tables. How may I adjust my code below to achieve this? Iam using vs2012 sql and in MVC c#
var set =
(from m in managers
from t in context.tblCompany
join tsc in context.tblStyling on t.ccID equals tsc.ccID
select new { tsc.ccID,LogoIcon = tsc.Icon , tsc.style1, tsc.style2, t.Desc })
.ToList();
return set.Select(c => new Settings(c.ccID, c.style1, c.style2, c.Desc, c.LogoIcon , m.firstName , m.lastName));
Okay, it looks like you want to join context.tblCompany and context.tblStyle and get the cross product of the resulting set and managers. If that is correct, then you are already there. You just need to include the fields from manager that you want in your select statement:
var set =
(from m in managers
from t in context.tblCompany
join tsc in context.tblStyling on t.ccID equals tsc.ccID
select new
{
tsc.ccID,
LogoIcon = tsc.Icon,
tsc.style1,
tsc.style2,
t.Desc,
m.firstName,
m.lastName })
.ToList();
return set;

Eager loading hierarchical children with explicit self-joins and contains_eager in SQLAlchemy

Given the following relationships:
- 1 MasterProduct parent -> many MasterProduct children
- 1 MasterProduct child -> many StoreProducts
- 1 StoreProduct -> 1 Store
I have defined the following declarative models in SQLAlchemy:
class MasterProduct(Base):
__tablename__ = 'master_products'
id = Column(Integer, primary_key=True)
pid = Column(Integer, ForeignKey('master_products.id'))
children = relationship('MasterProduct', join_depth=1,
backref=backref('parent', remote_side=[id]))
store_products = relationship('StoreProduct', backref='master_product')
class StoreProduct(Base):
__tablename__ = 'store_products'
id = Column(Integer, primary_key=True)
mid = Column(Integer, ForeignKey('master_products.id'))
sid = Column(Integer, ForeignKey('stores.id'))
timestamp = Column(DateTime)
store = relationship('Store', uselist=False)
class Store(Base):
__tablename__ = 'stores'
id = Column(Integer, primary_key=True)
My goal is to replicate the following query in SQLAlchemy with eager loading:
SELECT *
FROM master_products mp_parent
INNER JOIN master_products mp_child ON mp_child.pid = mp_parent.id
INNER JOIN store_products sp1 ON sp1.mid = mp_child.id
LEFT JOIN store_products sp2
ON sp1.mid = sp2.mid AND sp1.sid = sp2.sid AND sp1.timestamp < sp2.timestamp
WHERE mp_parent.id = 6752 AND sp2.id IS NULL
The query selects all MasterProduct children for parent 6752 and all
corresponding store products grouped by most recent timestamp using a NULL
self-join (greatest-n-per-group). There are 82 store products returned from the
query, with 14 master product children.
I've tried the following to no avail:
mp_child = aliased(MasterProduct)
sp1 = aliased(StoreProduct)
sp2 = aliased(StoreProduct)
q = db.session.query(MasterProduct).filter_by(id=6752) \
.join(mp_child, MasterProduct.children) \
.join(sp1, mp_child.store_products) \
.outerjoin(sp2, and_(sp1.mid == sp2.mid, sp1.sid == sp2.sid, sp1.timestamp < sp2.timestamp)) \
.filter(sp2.id == None) \
.options(contains_eager(MasterProduct.children, alias=mp_child),
contains_eager(MasterProduct.children, mp_child.store_products, alias=sp1))
>>> mp_parent = q.first() # the query below looks ok!
SELECT <all columns from master_products, master_products_1, and store_products_1>
FROM master_products INNER JOIN master_products AS master_products_1 ON master_products.id = master_products_1.pid INNER JOIN store_products AS store_products_1 ON master_products_1.id = store_products_1.mid LEFT OUTER JOIN store_products AS store_products_2 ON store_products_1.mid = store_products_2.mid AND store_products_1.sid = store_products_2.sid AND store_products_1.timestamp < store_products_2.timestamp
WHERE master_products.id = %s AND store_products_2.id IS NULL
LIMIT %s
>>> mp_parent.children # only *one* child is eagerly loaded (expected 14)
[<app.models.MasterProduct object at 0x2463850>]
>>> mp_parent.children[0].id # this is correct, 6762 is one of the children
6762L
>>> mp_parent.children[0].pid # this is correct
6752L
>>> mp_parent.children[0].store_products # only *one* store product is eagerly loaded (expected 7 for this child)
[<app.models.StoreProduct object at 0x24543d0>]
Taking a step back and simplifying the query to eagerly load just the children
also results in only 1 child being eagerly loaded instead of all 14:
mp_child = aliased(MasterProduct)
q = db.session.query(MasterProduct).filter_by(id=6752) \
.join(mp_child, MasterProduct.children)
.options(contains_eager(MasterProduct.children, alias=mp_child))
However, when I use a joinedload, joinedload_all, or subqueryload, all
14 children are eagerly loaded, i.e.:
q = db.session.query(MasterProduct).filter_by(id=6752) \
.options(joinedload_all('children.store_products', innerjoin=True))
So the problem seems to be populating MasterProduct.children from the
explicit join using contains_eager.
Can anyone spot the error in my ways or help point me in the right direction?
OK what you might observe in the SQL is that there's a "LIMIT 1" coming out. That's because you're using first(). We can just compare the first two queries, the contains eager, and the joinedload:
join() + contains_eager():
SELECT master_products_1.id AS master_products_1_id, master_products_1.pid AS master_products_1_pid, master_products.id AS master_products_id, master_products.pid AS master_products_pid
FROM master_products JOIN master_products AS master_products_1 ON master_products.id = master_products_1.pid
WHERE master_products.id = ?
LIMIT ? OFFSET ?
joinedload():
SELECT anon_1.master_products_id AS anon_1_master_products_id, anon_1.master_products_pid AS anon_1_master_products_pid, master_products_1.id AS master_products_1_id, master_products_1.pid AS master_products_1_pid
FROM (SELECT master_products.id AS master_products_id, master_products.pid AS master_products_pid
FROM master_products
WHERE master_products.id = ?
LIMIT ? OFFSET ?) AS anon_1 JOIN master_products AS master_products_1 ON anon_1.master_products_id = master_products_1.pid
you can see the second query is quite different; because first() means a LIMIT is applied, joinedload() knows to wrap the "criteria" query in a subquery, apply the limit to that, then apply the JOIN afterwards. In the join+contains_eager case, the LIMIT is applied to the collection itself and you get the wrong number of rows.
Just changing the script at the bottom to this:
for q, query_label in queries:
mp_parent = q.all()[0]
I get the output it says you're expecting:
[explicit join with contains_eager] children=3, store_products=27
[joinedload] children=3, store_products=27
[joinedload_all] children=3, store_products=27
[subqueryload] children=3, store_products=27
[subqueryload_all] children=3, store_products=27
[explicit joins with contains_eager, filtered by left-join] children=3, store_products=9
(this is why getting a user-created example is so important)

Making a subrequest in SQLAlchemy

I try to do the follwoing request in SQLAlchemy (ORM) :
SELECT id, ref_prog FROM stepvand_1c_1t.equipment_day_hour
WHERE id IN (SELECT id FROM stepvand_1c_1t.equipment WHERE equipment_type='L')
I did :
subq = session.query(Equipment)
subq = subq.filter(Equipment.equipment_type == "L").subquery()
query = session.query(EquipmentDayHour)
query = query.filter(EquipmentDayHour.id.in_(subq))
But that doesn't work...
Python tells me that the subrequest has too many columns.
I think you should only change one line of your sample code:
# error: includes all columns of Equipment
`subq = session.query(Equipment)`
# correct: include only ID column
`subq = session.query(Equipment.id)`
However, I believe that you can do this without subquery:
query = (session.query(EquipmentDayHour).
# version-1: if you have a relationship between EquipmentDayHour and Equipment
join(Equipment).
# version-2: if you do not have such relationship
#join(Equipment, EquipmentDayHour.id==Equipment.id).
filter(Equipment.equipment_type == "L")
)

Comparing two collections of enums with Hibernate or SQL

I have an entity 'Parent' which has a Set of enums
private Set<MyEnum> myEnums = EnumSet.noneOf(MyEnum.class);
#CollectionOfElements(targetElement=MyEnum.class)
#JoinTable
(name="PARENT_MY_ENUM",
joinColumns=#JoinColumn(name="PARENT_ID"))
#Enumerated(EnumType.STRING)
#Column (name="MY_ENUM", nullable=false)
public Set<MyEnum> getMyEnums(){
return myEnums;
}
public MyEnum {
ENUM_A,
ENUM_B,
ENUM_C,
ENUM_D;
}
Now I want to search for this entity with a collection of MyEnums. Only entities where all enums are set as in the search collection should be returned.
So if entity A has ENUM_A, ENUM_B and ENUM_C and entity B has ENUM_B, ENUM_C, ENUM_D a search with the search collectoin ENUM_A, ENUM_B, ENUM_C should only return entity A. A search for ENUM_B and ENUM_C should return nothing.
How would I do that in Hibernate?
if I do
select p from Parent p where p.myEnums IN (:searchCollection) and size(p.myEnums) = size(:searchCollection)
then this would return both entities for the first search.
Any ideas?
Update: I got a step further by figuring out how to do it in MySQL but applying this to Hibernate generates invalid SQL.
You would use a subquery with EXISTS for it like:
WHERE EXISTS(
SELECT pa.PARENT_ID, count(pme.MY_ENUM) FROM PARENT pa, PARENT_MY_ENUM pme
where pa.PARENT_ID = pme.PARENT_ID
AND pme.MY_ENUM IN ('ENUM_A','ENUM_B')
GROUP BY pa.PARENT_ID HAVING count(pme.MY_ENUM) = 2
)
But when I try to do the same in Hibernate:
select pa.ParentId, count(pa.myEnums) from Parent pa
WHERE pa.myEnums IN ('ENUM_A','ENUM_B')
GROUP BY pa.ParentId HAVING count(pa.myEnums) = 2
Hiberante creates this SQL statement:
select pa.CONTAINER_RELEASE_REFERENCE_ID as col_0_0_, count(.) as col_1_0_ from PARENT pa, PARENT_MY_ENUM enum1, PARENT_MY_ENUM enum2, PARENT_MY_ENUM enum3
where pa.PARENT_ID=enum1.PARENT_ID and pa.PARENT_ID=enum2.PARENT_ID and pa.PARENT_ID=enum3.PARENT_ID
and (. in ('ENUM_A' , 'ENUM_B'))
group by pa.PARENT_ID having count(.)=2
MySQL complains about the '.', Where is that coming from and why is Hibernate using 3 joins to PARENT_MY_ENUM?
Is this a Hibernate bug or what am I doing wrong?
Give the following a try for you exists subselect
select pa.ParentId, count(en) from Parent pa join pa.myEnums as en
WHERE en IN ('ENUM_A','ENUM_B')
GROUP BY pa.ParentId HAVING count(en) = 2
Otherwise, I wonder if something like that might not do the job
select p from Parent p join p.myEnums em
where (:results) = elements(em)
or
select p from Parent p join p.myEnums em
where (:results) in elements(em)
I think you can do that in java. Execute your initially proposed query, iterate the result and exclude (iterator.remove()) the false positives. It should be O(n), and I believe MySQL will require the same time to filter your result.

Django ORM Creates Phantom Alias in SQL Join

I am executing the following code (names changed to protect the innocent, so the model structure might seem weird):
memberships =
models.Membership.objects.filter(
degree__gt=0.0,
group=request.user.get_profile().group
)
exclude_count =
memberships.filter(
member__officerships__profile=request.user.get_profile()
).count()
if exclude_officers_with_profile:
memberships = memberships.exclude(
member__officerships__profile=request.user.get_profile()
)
total_count = memberships.count()
which at this point results in:
OperationalError at /
(1054, "Unknown column 'U1.id' in 'on clause'")
The SQL generated is:
SELECT
COUNT(*)
FROM
`membership`
WHERE (
`membership`.`group_id` = 2 AND
`membership`.`level` > 0.0 AND
NOT (
`membership`.`member_id`
IN (
SELECT
U2.`member_id`
FROM
`membership` U0 INNER JOIN `officers` U2
ON (U1.`id` = U2.`member_id`)
WHERE U2.`profile_id` = 2
)
)
)
It appears that the SQL Join's ON statement is referencing an alias that hasn't been defined. Any ideas why!? I dropped my MySQL database and re-synced the tables from my models to try and ensure that there weren't any inconsistencies there.
The structure of the models I'm using are:
class Membership(models.Model):
member = models.ForeignKey(Member, related_name='memberships')
group = models.ForeignKey(Group, related_name='memberships')
level = models.FloatField(default=0.0)
class Member(models.Model):
name = models.CharField(max_length=32)
class Officer(models.Model):
member = models.ForeignKey(Member, related_name='officerships')
profile = models.ForeignKey(UserProfile)
class UserProfile(models.Model)
group = models.ForeignKey(Group)
class Group(models.Model)
pass
I think this may be a symptom of:
http://code.djangoproject.com/ticket/9188
which was fixed as of django revision 9589, I think. Now how to figure out which revision I'm working from...
Confirmed. When I implemented the change referenced in the ticket above:
http://code.djangoproject.com/changeset/9589
my error went away.