I have a rails join table between 2 models superhero and superpower. Now I have 3 different superpower id and I want all the superheroes which have all the selected superpowers
To do that I'm trying to do the following:
matches = Superhero.all
matches = matches.joins(:superpowers).where('superpowers.id = ?', 17).where('superpowers.id = ?', 12).where('superpowers.id = ?', 6)
But this gives me an empty object even though I have superheroes which have all the given superpowers in my join table
The query generated from the above is:
SELECT "superheroes".* FROM "superheroes" INNER JOIN "superheroes_superpowers" ON "superheroes_superpowers"."superhero_id" = "superheroes"."id" INNER JOIN "superpowers" ON "superpowers"."id" = "superheroes_superpowers"."superpower_id" WHERE (superpowers.id = 17) AND (superpowers.id = 17) AND (superpowers.id = 12) AND (superpowers.id = 6)
So weirdly it tries to check for the superpower with id 17 twice (but it shouldn't affect the result I think) and the rest of the query seems to be correct.
try using an in clause
superpowers_ids = [17,12,6]
matches = Superhero.all
matches = matches.joins(:superpowers).where('superpowers.id in (?)', superpowers_ids)
Superhero.joins(:superpowers).where(superpowers: { id: [17,12,6] } )
This gives the following SQL query (formatted for readibility):
SELECT "superheros".*
FROM "superheros"
INNER JOIN "superhero_superpowers" ON "superhero_superpowers"."superhero_id" = "superheros"."id"
INNER JOIN "superpowers" ON "superpowers"."id" = "superhero_superpowers"."superpower_id"
WHERE "superpowers"."id" IN (17, 12, 6)
Related
I'm trying to filter product collection with multiple OR filter with this code :
$values = ['xxx','xxx'];
$filters = [];
$filters[] = $this->_filterBuilder
->setField('field_a')->setConditionType('in')
->setValue($values)
->create();
$filters[] = $this->_filterBuilder
->setField('field_b')
->setConditionType('in')
->setValue($values)
->create();
// two more filter like that
$filterGroup = $this->_filterGroupBuilder
->setFilters($filters)
->create();
$searchCriteria = $this->_searchCriteriaBuilder
->setFilterGroups([$filterGroup])
->create();
$products = $this->_productRepository->getList($searchCriteria)->getItems();
Problem is collection return 0 result instead of two. After analyze sql query generated by Magento eav table are joined with INNER JOIN like that :
INNER JOIN `catalog_product_entity_text` AS `at_field_b` ON (`at_field_b`.`row_id` = `e`.`row_id`) AND (`at_field_b`.`attribute_id` = '204') AND (`at_field_b`.`store_id` = 0)
If on raw sql query I execute it by replacing INNER JOIN by LEFT JOIN it works, i've got my results.
So my question is how can I "force" magento to left join instead of inner ? Or maybe it's a pure coincidence and real reason isn't the left/inner join
I didn't precise but field_a,field_b,etc... aren't not required so they could be empty for products
As someone who constantly has problems with databases, I'm trying to look for a solution that optimizes my database calls. Scenario is this, I'm trying to get all rows of items directly owned by the user, but also I want to get the rows of items where the user roles acts as a guest. My current query works, but I think it isn't the best forged solution for what I'm trying to achieve. This get me the results I want, but I fear when there are hundreds of rows this will get terribly slow
set #UsrId = 23;
(SELECT Eventos.Fecha, Eventos.InformacionDelEvento, Eventos.PosicionGpsSiNo, Eventos.InformacionGps, Dispositivos.Nombre, CatalogoDeEventos.Descripcion
FROM Eventos, CatalogoDeEventos, Dispositivos, DispositivosxUsuario
WHERE Eventos.IdEvento = CatalogoDeEventos.Id
AND Dispositivos.IdentificadorUnico = Eventos.IdDispositivo
AND DispositivosxUsuario.IdDispositivo = Dispositivos.Id
AND Dispositivos.Invisible = 0
AND DispositivosxUsuario.IdUsuario = #UsrId
ORDER BY Fecha DESC)
UNION
(SELECT Eventos.Fecha, Eventos.InformacionDelEvento, Eventos.PosicionGpsSiNo, Eventos.InformacionGps, Dispositivos.Nombre, CatalogoDeEventos.Descripcion
FROM Eventos, CatalogoDeEventos, Dispositivos, DispositivosxUsuario, Seguidores
WHERE Eventos.IdEvento = CatalogoDeEventos.Id
AND Dispositivos.IdentificadorUnico = Eventos.IdDispositivo
AND DispositivosxUsuario.IdDispositivo = Dispositivos.Id
AND Dispositivos.Invisible = 0
AND Seguidores.IdUsuario = DispositivosxUsuario.IdUsuario
AND Seguidores.IdSeguidor = #UsrId
ORDER BY Fecha DESC)
So both queries have this same block
SELECT Eventos.Fecha, Eventos.InformacionDelEvento, Eventos.PosicionGpsSiNo, Eventos.InformacionGps, Dispositivos.Nombre, CatalogoDeEventos.Descripcion
FROM Eventos, CatalogoDeEventos, Dispositivos, DispositivosxUsuario
WHERE Eventos.IdEvento = CatalogoDeEventos.Id
AND Dispositivos.IdentificadorUnico = Eventos.IdDispositivo
AND DispositivosxUsuario.IdDispositivo = Dispositivos.Id
AND Dispositivos.Invisible = 0
They differ on the lines above this, so far I haven't figured a way of getting both results on a single call
this should work
SELECT Eventos.Fecha, Eventos.InformacionDelEvento, Eventos.PosicionGpsSiNo, Eventos.InformacionGps, Dispositivos.Nombre, CatalogoDeEventos.Descripcion
FROM Eventos, CatalogoDeEventos, Dispositivos, DispositivosxUsuario, Seguidores
WHERE Eventos.IdEvento = CatalogoDeEventos.Id
AND Dispositivos.IdentificadorUnico = Eventos.IdDispositivo
AND DispositivosxUsuario.IdDispositivo = Dispositivos.Id
AND Dispositivos.Invisible = 0
AND Seguidores.IdUsuario = DispositivosxUsuario.IdUsuario
AND (
(Seguidores.IdUsuario = DispositivosxUsuario.IdUsuario
AND Seguidores.IdSeguidor = #UsrId)
OR DispositivosxUsuario.IdUsuario = #UsrId)
ORDER BY Fecha DESC
as was mentioned in the comments, this effectively left joins on Sequidores and that would probably have been more obvious if the query was written with the newer join syntax.
Re-written to use newer join syntax
SELECT e.Fecha, e.InformacionDelEvento, e.PosicionGpsSiNo, e.InformacionGps, d.Nombre, cde.Descripcion
FROM Eventos e
INNER JOIN CatalogoDeEventos cde ON e.IdEvento = cde.Id
INNER JOIN Dispositivos d ON d.IdentificadorUnico = e.IdDispositivo
INNER JOIN DispositivosxUsuario du ON du.IdDispositivo = d.Id
LEFT JOIN Seguidores s ON s.IdUsuario = du.IdUsuario
WHERE d.Invisible = 0
AND (Seguidores.IdSeguidor = #UsrId
OR DispositivosxUsuario.IdUsuario = #UsrId)
ORDER BY Fecha DESC
So here is the issue. I'm trying to write a new fillrate report because the one built in is not good enough... I'm trying to run a single select statement to return both, a count of how many times an item was ordered for a specific month, and then also a count of how many times it was invoiced/shipped in full.
This code is obviously wrong, I also currently have it restricted to only look at AUG of 2015, but that is just to simplify results during testing.
I can't figure out how to do the 2nd count... This is what I was trying (brain stuck on old for each loop logic):
select inv_mast.item_id,
inv_mast.item_desc,
"YEAR" = year(oe_line.required_date),
"MONTH" = month(oe_line.required_date),
"ORDERS" = count(1),
"HITS" = (
select count(1)
from invoice_line
where invoice_line.order_no = oe_line.order_no
and invoice_line.oe_line_number = oe_line.line_no
and invoice_line.qty_shipped = oe_line.qty_ordered
)
from oe_line,
inv_mast,
inv_loc
where inv_mast.inv_mast_uid = oe_line.inv_mast_uid
and inv_mast.delete_flag = 'N'
and inv_mast.inv_mast_uid = inv_loc.inv_mast_uid
and inv_loc.location_id = '101'
and year(oe_line.required_date) = '2015'
and month(oe_line.required_date) = '8'
group by inv_mast.item_id,
inv_mast.item_desc,
year(oe_line.required_date),
month(oe_line.required_date)
order by inv_mast.item_id
To me it would seem like you could rewrite the query to use a left join on the invoice_line table instead. Without any proper test data I can't guarantee it is correct, but I think it should be.
Besides the left join I also changed to explicit joins and moved the aliases as I don't think MySQL supports the alias = column syntax.
select inv_mast.item_id,
inv_mast.item_desc,
year(o.required_date) as "YEAR",
month(o.required_date) as "MONTH",
count(1) as "ORDERS",
count(invoice_line.order_no) as "HITS"
from oe_line o
join inv_mast on inv_mast.inv_mast_uid = o.inv_mast_uid
join inv_loc on inv_mast.inv_mast_uid = inv_loc.inv_mast_uid
left join invoice_line on invoice_line.order_no = o.order_no
and invoice_line.oe_line_number = o.line_no
and invoice_line.qty_shipped = o.qty_ordered
where inv_mast.delete_flag = 'N'
and inv_loc.location_id = '101'
and year(o.required_date) = '2015'
and month(o.required_date) = '8'
group by inv_mast.item_id,
inv_mast.item_desc,
year(o.required_date),
month(o.required_date)
order by inv_mast.item_id;
I have a view with the following definition:
select
`cb_trans_detail`.`numero_partida` AS `numero_partida`,
`cb_trans_head`.`fecha_partida` AS `fecha_partida`,
`cb_trans_detail`.`concepto_partida` AS `concepto_partida`,
`cb_cuenta`.`nombre_cuenta` AS `nombre_cuenta`,
`cb_cuenta`.`codigo_mayor` AS `codigo_mayor`,
`cb_mayor`.`nombre_mayor` AS `nombre_mayor`,
`cb_mayor`.`categoria` AS `categoria`,
`cb_categoria`.`nombre` AS `nombre`,
`cb_categoria`.`presentacion` AS `presentacion`,
`cb_trans_detail`.`codigo_cuenta` AS `codigo_cuenta`,
sum(`cb_trans_detail`.`debito_partida`) AS `Debitos`,
sum(`cb_trans_detail`.`credito_partida`) AS `Creditos`,
`cb_cuenta`.`saldo_inicial` AS `saldo_inicial`,
((`cb_cuenta`.`saldo_inicial` +
sum(`cb_trans_detail`.`debito_partida`)) -
sum(`cb_trans_detail`.`credito_partida`)) AS `Saldo`,
concat(`cb_mayor`.`categoria`,`cb_cuenta`.`codigo_mayor`,`cb_trans_detail`.`codigo_cuenta`)
AS `Codigo`
from
((((`cb_trans_detail` join `cb_cuenta`
on(((`cb_trans_detail`.`codigo_cuenta` = `cb_cuenta`.`codigo_cuenta`)
and (`cb_trans_detail`.`categoria` = `cb_cuenta`.`categoria`)
and (`cb_trans_detail`.`codigo_mayor` = `cb_cuenta`.`codigo_mayor`))))
join `cb_mayor` on(((`cb_cuenta`.`codigo_mayor` = `cb_mayor`.`codigo_mayor`)
and (`cb_cuenta`.`categoria` = `cb_mayor`.`categoria`))))
join `cb_categoria` on(((`cb_mayor`.`categoria` = `cb_categoria`.`categoria`)
and (`cb_trans_detail`.`categoria` = `cb_categoria`.`categoria`))))
left join `cb_trans_head` on((`cb_trans_detail`.`numero_partida` =
`cb_trans_head`.`numero_partida`)))
where
(`cb_categoria`.`presentacion` = '1')
group by concat(`cb_mayor`.`categoria`,`cb_cuenta`.`codigo_mayor`,
`cb_trans_detail`.`codigo_cuenta`)
If I select from this view as follows:
SELECT
`balance_general_view`.`numero_partida`,
`balance_general_view`.`fecha_partida`,
`balance_general_view`.`concepto_partida`,
`balance_general_view`.`nombre_cuenta`,
`balance_general_view`.`codigo_mayor`,
`balance_general_view`.`nombre_mayor`,
`balance_general_view`.`categoria`,
`balance_general_view`.`nombre`,
`balance_general_view`.`presentacion`,
`balance_general_view`.`codigo_cuenta`,
`balance_general_view`.`Debitos`,
`balance_general_view`.`Creditos`,
`balance_general_view`.`saldo_inicial`,
`balance_general_view`.`Saldo`,
`balance_general_view`.`Codigo`
FROM
`balance_general_view`
WHERE
`balance_general_view`.`fecha_partida` BETWEEN '2014-01-01' AND '2014-01-31'
this yields a different result than if I execute the query as follows:
select
`cb_trans_detail`.`numero_partida` AS `numero_partida`,
`cb_trans_head`.`fecha_partida` AS `fecha_partida`,
`cb_trans_detail`.`concepto_partida` AS `concepto_partida`,
`cb_cuenta`.`nombre_cuenta` AS `nombre_cuenta`,
`cb_cuenta`.`codigo_mayor` AS `codigo_mayor`,
`cb_mayor`.`nombre_mayor` AS `nombre_mayor`,
`cb_mayor`.`categoria` AS `categoria`,
`cb_categoria`.`nombre` AS `nombre`,
`cb_categoria`.`presentacion` AS `presentacion`,
`cb_trans_detail`.`codigo_cuenta` AS `codigo_cuenta`,
sum(`cb_trans_detail`.`debito_partida`) AS `Debitos`,
sum(`cb_trans_detail`.`credito_partida`) AS `Creditos`,
`cb_cuenta`.`saldo_inicial` AS `saldo_inicial`,
((`cb_cuenta`.`saldo_inicial` + sum(`cb_trans_detail`.`debito_partida`)) - sum(`cb_trans_detail`.`credito_partida`)) AS `Saldo`,
concat(`cb_mayor`.`categoria`,`cb_cuenta`.`codigo_mayor`,`cb_trans_detail`.`codigo_cuenta`) AS `Codigo`
from
((((`cb_trans_detail` join `cb_cuenta` on(((`cb_trans_detail`.`codigo_cuenta` = `cb_cuenta`.`codigo_cuenta`) and (`cb_trans_detail`.`categoria` = `cb_cuenta`.`categoria`) and (`cb_trans_detail`.`codigo_mayor` = `cb_cuenta`.`codigo_mayor`)))) join `cb_mayor` on(((`cb_cuenta`.`codigo_mayor` = `cb_mayor`.`codigo_mayor`) and (`cb_cuenta`.`categoria` = `cb_mayor`.`categoria`)))) join `cb_categoria` on(((`cb_mayor`.`categoria` = `cb_categoria`.`categoria`) and (`cb_trans_detail`.`categoria` = `cb_categoria`.`categoria`)))) left join `cb_trans_head` on((`cb_trans_detail`.`numero_partida` = `cb_trans_head`.`numero_partida`)))
where
(`cb_categoria`.`presentacion` = '1') and `cb_trans_head`.`fecha_partida` BETWEEN '2014-01-01' and '2014-01-31'
group by
concat(`cb_mayor`.`categoria`,`cb_cuenta`.`codigo_mayor`,`cb_trans_detail`.`codigo_cuenta`)
My question is: How to get the result I need using the view and filtering programatically instead of hard-coding the where condition? Thank you. If you need the individual table definitions let me know. Much appreciated.
If you use a left join to a table X and a condition in the WHERE-clause to this table X, you change your left join to an inner one. If you want to restrict the results by the values of this left joined table, you must use this condition in the ON clause of the left join instead:
...
LEFT JOIN
cb_trans_head
ON
cb_trans_detail.numero_partida = cb_trans_head.numero_partida
AND
cb_trans_head.fecha_partida BETWEEN '2014-01-01' and '2014-01-31'
...
If there's another one that I overlook, tread it the same way.
Given the following relationships:
- 1 MasterProduct parent -> many MasterProduct children
- 1 MasterProduct child -> many StoreProducts
- 1 StoreProduct -> 1 Store
I have defined the following declarative models in SQLAlchemy:
class MasterProduct(Base):
__tablename__ = 'master_products'
id = Column(Integer, primary_key=True)
pid = Column(Integer, ForeignKey('master_products.id'))
children = relationship('MasterProduct', join_depth=1,
backref=backref('parent', remote_side=[id]))
store_products = relationship('StoreProduct', backref='master_product')
class StoreProduct(Base):
__tablename__ = 'store_products'
id = Column(Integer, primary_key=True)
mid = Column(Integer, ForeignKey('master_products.id'))
sid = Column(Integer, ForeignKey('stores.id'))
timestamp = Column(DateTime)
store = relationship('Store', uselist=False)
class Store(Base):
__tablename__ = 'stores'
id = Column(Integer, primary_key=True)
My goal is to replicate the following query in SQLAlchemy with eager loading:
SELECT *
FROM master_products mp_parent
INNER JOIN master_products mp_child ON mp_child.pid = mp_parent.id
INNER JOIN store_products sp1 ON sp1.mid = mp_child.id
LEFT JOIN store_products sp2
ON sp1.mid = sp2.mid AND sp1.sid = sp2.sid AND sp1.timestamp < sp2.timestamp
WHERE mp_parent.id = 6752 AND sp2.id IS NULL
The query selects all MasterProduct children for parent 6752 and all
corresponding store products grouped by most recent timestamp using a NULL
self-join (greatest-n-per-group). There are 82 store products returned from the
query, with 14 master product children.
I've tried the following to no avail:
mp_child = aliased(MasterProduct)
sp1 = aliased(StoreProduct)
sp2 = aliased(StoreProduct)
q = db.session.query(MasterProduct).filter_by(id=6752) \
.join(mp_child, MasterProduct.children) \
.join(sp1, mp_child.store_products) \
.outerjoin(sp2, and_(sp1.mid == sp2.mid, sp1.sid == sp2.sid, sp1.timestamp < sp2.timestamp)) \
.filter(sp2.id == None) \
.options(contains_eager(MasterProduct.children, alias=mp_child),
contains_eager(MasterProduct.children, mp_child.store_products, alias=sp1))
>>> mp_parent = q.first() # the query below looks ok!
SELECT <all columns from master_products, master_products_1, and store_products_1>
FROM master_products INNER JOIN master_products AS master_products_1 ON master_products.id = master_products_1.pid INNER JOIN store_products AS store_products_1 ON master_products_1.id = store_products_1.mid LEFT OUTER JOIN store_products AS store_products_2 ON store_products_1.mid = store_products_2.mid AND store_products_1.sid = store_products_2.sid AND store_products_1.timestamp < store_products_2.timestamp
WHERE master_products.id = %s AND store_products_2.id IS NULL
LIMIT %s
>>> mp_parent.children # only *one* child is eagerly loaded (expected 14)
[<app.models.MasterProduct object at 0x2463850>]
>>> mp_parent.children[0].id # this is correct, 6762 is one of the children
6762L
>>> mp_parent.children[0].pid # this is correct
6752L
>>> mp_parent.children[0].store_products # only *one* store product is eagerly loaded (expected 7 for this child)
[<app.models.StoreProduct object at 0x24543d0>]
Taking a step back and simplifying the query to eagerly load just the children
also results in only 1 child being eagerly loaded instead of all 14:
mp_child = aliased(MasterProduct)
q = db.session.query(MasterProduct).filter_by(id=6752) \
.join(mp_child, MasterProduct.children)
.options(contains_eager(MasterProduct.children, alias=mp_child))
However, when I use a joinedload, joinedload_all, or subqueryload, all
14 children are eagerly loaded, i.e.:
q = db.session.query(MasterProduct).filter_by(id=6752) \
.options(joinedload_all('children.store_products', innerjoin=True))
So the problem seems to be populating MasterProduct.children from the
explicit join using contains_eager.
Can anyone spot the error in my ways or help point me in the right direction?
OK what you might observe in the SQL is that there's a "LIMIT 1" coming out. That's because you're using first(). We can just compare the first two queries, the contains eager, and the joinedload:
join() + contains_eager():
SELECT master_products_1.id AS master_products_1_id, master_products_1.pid AS master_products_1_pid, master_products.id AS master_products_id, master_products.pid AS master_products_pid
FROM master_products JOIN master_products AS master_products_1 ON master_products.id = master_products_1.pid
WHERE master_products.id = ?
LIMIT ? OFFSET ?
joinedload():
SELECT anon_1.master_products_id AS anon_1_master_products_id, anon_1.master_products_pid AS anon_1_master_products_pid, master_products_1.id AS master_products_1_id, master_products_1.pid AS master_products_1_pid
FROM (SELECT master_products.id AS master_products_id, master_products.pid AS master_products_pid
FROM master_products
WHERE master_products.id = ?
LIMIT ? OFFSET ?) AS anon_1 JOIN master_products AS master_products_1 ON anon_1.master_products_id = master_products_1.pid
you can see the second query is quite different; because first() means a LIMIT is applied, joinedload() knows to wrap the "criteria" query in a subquery, apply the limit to that, then apply the JOIN afterwards. In the join+contains_eager case, the LIMIT is applied to the collection itself and you get the wrong number of rows.
Just changing the script at the bottom to this:
for q, query_label in queries:
mp_parent = q.all()[0]
I get the output it says you're expecting:
[explicit join with contains_eager] children=3, store_products=27
[joinedload] children=3, store_products=27
[joinedload_all] children=3, store_products=27
[subqueryload] children=3, store_products=27
[subqueryload_all] children=3, store_products=27
[explicit joins with contains_eager, filtered by left-join] children=3, store_products=9
(this is why getting a user-created example is so important)