"NOT IN" for Active Record - mysql

I have a MySQL query that I am trying to chain a "NOT IN" at the end of it.
Here is what it looks like in ruby using Active Record:
not_in = find_by_sql("SELECT parent_dimension_id FROM relations WHERE relation_type_id = 6;").map(&:parent_dimension_id)
joins('INNER JOIN dimensions ON child_dimension_id = dimensions.id')
.where(relation_type_id: model_relation_id,
parent_dimension_id: sub_type_ids,
child_dimension_id: model_type)
.where.not(parent_dimension_id: not_in)
So the SQL query I'm trying to do looks like this:
INNER JOIN dimensions ON child_dimension_id = dimensions.id
WHERE relations.relation_type_id = 5
AND relations.parent_dimension_id
NOT IN(SELECT parent_dimension_id FROM relations WHERE relation_type_id = 6);
Can someone confirm to me what I should use for that query?
do I chain on where.not ?

If you really do want
SELECT parent_dimension_id
FROM relations
WHERE relation_type_id = 6
as a subquery, you just need to convert that SQL to an ActiveRecord relation:
Relation.select(:parent_dimension_id).where(:relation_type_id => 6)
then use that as a value in a where call the same way you'd use an array:
not_parents = Relation.select(:parent_dimension_id).where(:relation_type_id => 6)
Relation.joins('...')
.where(relation_type_id: model_relation_id, ...)
.where.not(parent_dimension_id: not_parents)
When you use an ActiveRecord relation as a value in a where and that relation selects a single column:
r = M1.select(:one_column).where(...)
M2.where(:column => r)
ActiveRecord is smart enough to inline r's SQL as an in (select one_column ...) rather than doing two queries.
You could probably replace your:
joins('INNER JOIN dimensions ON child_dimension_id = dimensions.id')
with a simpler joins(:some_relation) if your relations are set up too.

You can feed where clauses with values or arrays of values, in which case they will be translated into in (?) clauses.
Thus, the last part of your query could contain a mapping:
.where.not(parent_dimension_id:Relation.where(relation_type_id:6).map(&:parent_dimension_id))
Or you can prepare a statement
.where('parent_dimension_id not in (?)', Relation.where(relation_type_id:6).map(&:parent_dimension_id) )
which is essentially exactly the same thing

Related

How to use the distinct method in Rails with Arel Table?

I am looking to run the following query in Rails (I have used the scuttle.io site to convert my SQL to rails-friendly syntax):
Here is the original query:
SELECT pools.name AS "Pool Name", COUNT(DISTINCT stakings.user_id) AS "Total Number of Users Per Pool" from stakings
INNER JOIN pools ON stakings.pool_id = pools.id
INNER JOIN users ON users.id = stakings.user_id
INNER JOIN countries ON countries.code = users.country
WHERE countries.kyc_flow = 1
GROUP BY (pools.name);
And here is the scuttle.io query:
<%Staking.select(
[
Pool.arel_table[:name].as('Pool_Name'), Staking.arel_table[:user_id].count.as('Total_Number_of_Users_Per_Pool')
]
).where(Country.arel_table[:kyc_flow].eq(1)).joins(
Staking.arel_table.join(Pool.arel_table).on(
Staking.arel_table[:pool_id].eq(Pool.arel_table[:id])
).join_sources
).joins(
Staking.arel_table.join(User.arel_table).on(
User.arel_table[:id].eq(Staking.arel_table[:user_id])
).join_sources
).joins(
Staking.arel_table.join(Country.arel_table).on(
Country.arel_table[:code].eq(User.arel_table[:country])
).join_sources
).group(Pool.arel_table[:name]).each do |x|%>
<p><%=x.Pool_Name%><p>
<p><%=x.Total_Number_of_Users_Per_Pool%>
<%end%>
Now, as you may notice, sctuttle.io does not include the distinct parameter which I need. How in the world can I use distinct here without getting errors such as "method distinct does not exist for Arel Node?" or just syntax errors?
Is there any way to write the above query using rails ActiveRecord? I am sure there is, but I am really not sure how.
Answer
The Arel::Nodes::Count class (an Arel::Nodes::Function) accepts a boolean value for distinctness.
def initialize expr, distinct = false, aliaz = nil
super(expr, aliaz)
#distinct = distinct
end
The #count expression is a shortcut for the same and also accepts a single argument
def count distinct = false
Nodes::Count.new [self], distinct
end
So in your case you could use either of the below options
Arel::Nodes::Count.new([Staking.arel_table[:user_id]],true,'Total_Number_of_Users_Per_Pool')
# OR
Staking.arel_table[:user_id].count(true).as('Total_Number_of_Users_Per_Pool')
Suggestion 1:
The Arel you have seems a bit overkill. Given the natural relationships you should be able to simplify this a bit e.g.
country_table = Country.arel_table
Staking
.joins(:pools,:users)
.joins( Arel::Nodes::InnerJoin(
country_table,
country_table.create_on(country_table[:code].eq(User.arel_table[:country])))
.select(
Pool.arel_table[:name],
Staking.arel_table[:user_id].count(true).as('Total_Number_of_Users_Per_Pool')
)
.where(countries: {kyc_flow: 1})
.group(Pool.arel_table[:name])
Suggestion 2: Move this query to your controller. The view has no business making database calls.

How to access columns of subqueries with jooq?

i am having troubles understanding how to access columns from a subquery (MySQL). Here is my code:
Personne personne = Personne.PERSONNE.as("personne");
Evenement evenement = Evenement.EVENEMENT.as("evenement");
Genealogie genealogie = Genealogie.GENEALOGIE.as("genealogie");
Lieu lieu = Lieu.LIEU.as("lieu");
SelectField<?>[] select = { DSL.countDistinct(personne.ID).as("countRs"), lieu.LIBELLE.as("libelleRs"),
lieu.ID.as("idVille") };
Table<?> fromPersonne = evenement.innerJoin(personne).on(personne.ID.eq(evenement.IDPERS))
.innerJoin(genealogie).on(genealogie.ID.eq(personne.IDGEN)).innerJoin(lieu)
.on(lieu.ID.eq(evenement.IDLIEU));
Table<?> fromFamille = evenement.innerJoin(personne).on(personne.IDFAM.eq(evenement.IDFAM))
.innerJoin(genealogie).on(genealogie.ID.eq(personne.IDGEN)).innerJoin(lieu)
.on(lieu.ID.eq(evenement.IDLIEU));
GroupField[] groupBy = { lieu.ID };
Condition condition = //conditionally build, not relevant i think
result = create.select(DSL.asterisk())
.from(create.select(select).from(fromPersonne).where(condition).groupBy(groupBy)
.union(create.select(select).from(fromFamille).where(condition).groupBy(groupBy)))
// i would like something like this but i don't know how: .groupBy(groupBy).fetch();
Basicly what i have is:
SELECT
*
FROM(
(SELECT
countRs, libelleRs, idVille
FROM
fromPersonne
WHERE
-- conditions
GROUP BY lieu.ID)
UNION
(SELECT
countRs, libelleRs, idVille
FROM
fromFamille
WHERE
-- conditions
GROUP BY lieu.ID)
)GROUP BY lieu.ID -- this is where i need help
In a plain MySQL query i would just give an alias to the union and then make a reference to the column i want to group by using the alias but it seems like it does not work like this with JOOQ.
I just need to group the results of the subqueries together but i don't know how to make a reference to the subqueries columns... I am sure i would have to reference my subqueries in objects outside of that "main select" to be able to access the columns or something along those lines but i am lost in all the object types.
You have to assign your derived table to a local variable and dereference columns from it, e.g.
Table<?> t = table(
select(...).from(...).groupBy(...).unionAll(select(...).from(...).groupBy(...))
).as("t");
Field<Integer> tId = t.field(lieu.ID);

INNER JOIN Results from Select Statement using Doctrine QueryBuilder

Can you use Doctrine QueryBuilder to INNER JOIN a temporary table from a full SELECT statement that includes a GROUP BY?
The ultimate goal is to select the best version of a record. I have a viewVersion table that has multiple versions with the same viewId value but different timeMod. I want to find the version with the latest timeMod (and do a lot of other complex joins and filters on the query).
Initially people assume you can do a GROUP BY viewId and then ORDER BY timeMod, but ORDER BY has no effect on GROUP BY, and MySQL will return random results. There are a ton of answers out there (e.g. here) that explain the problem with using GROUP and offer a solution, but I am having trouble interpreting the Doctrine docs to find a way to implement the SQL with Doctrine QueryBuilder (if it's even possible). Why don't I just use DQL? I may have to, but I have a lot of dynamic filters and joins that are much easier to do with QueryBuilder, so I wanted to see if that's possible.
Sample MySQL to Reproduce in Doctrine QueryBuilder
SELECT vv.*
FROM view_version vv
#inner join only returns where the result sets overlap, i.e. one record
INNER JOIN (
SELECT MAX(timeMod) maxTimeMod, viewId
FROM view_version
GROUP BY viewId
) version ON version.viewId = vv.viewId AND vv.timeMod = version.maxTimeMod
#join other tables for filter, etc
INNER JOIN view v ON v.id = vv.viewId
INNER JOIN content_type c ON c.id = v.contentTypeId
WHERE vv.siteId=1
AND v.contentTypeId IN (2)
ORDER BY vv.title ASC;
Theoretical Solution via Query Builder (not working)
I am thinking that the JOIN needs to inject a DQL statement, e.g.
$em = $this->getDoctrine()->getManager();
$viewVersionRepo = $em->getRepository('GutensiteCmsBundle:View\ViewVersion');
$queryMax = $viewVersionRepo->createQueryBuilder()
->addSelect('MAX(timeMod) AS timeModMax')
->addSelect('viewId')
->groupBy('viewId');
$queryBuilder = $viewVersionRepo->createQueryBuilder('vv')
// I tried putting the query in a parenthesis, to no avail
->join('('.$queryMax->getDQL().')', 'version', 'WITH', 'vv.viewId = version.viewId AND vv.timeMod = version.timeModMax')
// Join other Entities
->join('e.view', 'view')
->addSelect('view')
->join('view.contentType', 'contentType')
->addSelect('contentType')
// Perform random filters
->andWhere('vv.siteId = :siteId')->setParameter('siteId', 1)
->andWhere('view.contentTypeId IN(:contentTypeId)')->setParameter('contentTypeId', $contentTypeIds)
->addOrderBy('e.title', 'ASC');
$query = $queryBuilder->getQuery();
$results = $query->getResult();
My code (which may not match the above example perfectly) outputs:
SELECT e, view, contentType
FROM Gutensite\CmsBundle\Entity\View\ViewVersion e
INNER JOIN (
SELECT MAX(v.timeMod) AS timeModMax, v.viewId
FROM Gutensite\CmsBundle\Entity\View\ViewVersion v
GROUP BY v.viewId
) version WITH vv.viewId = version.viewId AND vv.timeMod = version.timeModMax
INNER JOIN e.view view
INNER JOIN view.contentType contentType
WHERE e.siteId = :siteId
AND view.contentTypeId IN (:contentTypeId)
ORDER BY e.title ASC
This Answer seems to indicate that it's possible in other contexts like IN statements, but when I try the above method in the JOIN, I get the error:
[Semantical Error] line 0, col 90 near '(SELECT MAX(v.timeMod)': Error: Class '(' is not defined.
A big thanks to #AdrienCarniero for his alternative query structure for sorting the highest version with a simple JOIN where the entity's timeMod is less than the joined table timeMod.
Alternative Query
SELECT view_version.*
FROM view_version
#inner join to get the best version
LEFT JOIN view_version AS best_version ON best_version.viewId = view_version.viewId AND best_version.timeMod > view_version.timeMod
#join other tables for filter, etc
INNER JOIN view ON view.id = view_version.viewId
INNER JOIN content_type ON content_type.id = view.contentTypeId
WHERE view_version.siteId=1
# LIMIT Best Version
AND best_version.timeMod IS NULL
AND view.contentTypeId IN (2)
ORDER BY view_version.title ASC;
Using Doctrine QueryBuilder
$em = $this->getDoctrine()->getManager();
$viewVersionRepo = $em->getRepository('GutensiteCmsBundle:View\ViewVersion');
$queryBuilder = $viewVersionRepo->createQueryBuilder('vv')
// Join Best Version
->leftJoin('GutensiteCmsBundle:View\ViewVersion', 'bestVersion', 'WITH', 'bestVersion.viewId = e.viewId AND bestVersion.timeMod > e.timeMod')
// Join other Entities
->join('e.view', 'view')
->addSelect('view')
->join('view.contentType', 'contentType')
->addSelect('contentType')
// Perform random filters
->andWhere('vv.siteId = :siteId')->setParameter('siteId', 1)
// LIMIT Joined Best Version
->andWhere('bestVersion.timeMod IS NULL')
->andWhere('view.contentTypeId IN(:contentTypeId)')->setParameter('contentTypeId', $contentTypeIds)
->addOrderBy('e.title', 'ASC');
$query = $queryBuilder->getQuery();
$results = $query->getResult();
In terms of performance, it really depends on the dataset. See this discussion for details.
TIP: The table should include indexes on both these values (viewId and timeMod) to speed up results. I don't know if it would also benefit from a single index on both fields.
A native SQL query using the original JOIN method may be better in some cases, but compiling the query over an extended range of code that dynamically creates it, and getting the mappings correct is a pain. So this is at least an alternative solution that I hope helps others.

How to pass data dynamically to mysql query

I have following query,
SELECT t_subject.subject, SUM( t_skilllist.skill_level ) AS total_skill, t_users.first_name,
t_skilllist.skill_level
FROM `t_skilllist`
JOIN t_subject ON t_subject.id = t_skilllist.subject_id
JOIN t_users ON t_users.id = t_skilllist.user_id
WHERE t_subject.subject = 'html'
GROUP BY t_users.first_name
ORDER BY total_skill DESC
LIMIT 0 , 30
I want to display subject and skill level for each student. But, for one subject I can do that with above query. As an example for html it works. However, I want to pass more than one subject to the query dynamically. I tried to combined subjects with AND operator but it return empty result set.
How to solve this? How to pass more than two subjects to the query? I am using PHP as server side scripting language.
You can use the IN() clause.
WHERE t_subject.subject IN ('html', 'php', 'and', 'a', 'lot', 'more')

NHibernate INNER JOIN on a SubQuery

I would like to do a subquery and then inner join the result of that to produce a query. I want to do this as I have tested an inner join query and it seems to be far more performant on MySql when compared to a straight IN subquery.
Below is a very basic example of the type of sql I am trying to reproduce.
Tables
ITEM
ItemId
Name
ITEMRELATIONS
ItemId
RelationId
Example Sql I would Like to create
Give me the COUNT of RELATIONs for ITEMs having a name of 'bob':
select ir.itemId, count(ir.relationId)
from ItemRelations ir
inner join (select itemId from Items where name = 'bob') sq
on ir.itemId = sq.itemId
group by ir.itemId
The base Nhibernate QueryOver
var bobItems = QueryOver.Of<Item>(() => itemAlias)
.Where(() => itemAlias.Name == "bob")
.Select(Projections.Id());
var bobRelationCount = session.QueryOver<ItemRelation>(() => itemRelationAlias)
.Inner.Join(/* Somehow join the detached criteria here on the itemId */)
.SelectList(
list =>
list.SelectGroup(() => itemRelationAlias.ItemId)
.WithAlias(() => itemRelationCountAlias.ItemId)
.SelectCount(() => itemRelationAlias.ItemRelationId)
.WithAlias(() => itemRelationCountAlias.Count))
.TransformUsing(Transformers.AliasToBean<ItemRelationCount>())
.List<ItemRelationCount>();
I know it may be possible to refactor this into a single query, however the above is merely as simple example. I cannot change the detached QueryOver, as it is handed to my bit of code and is used in other parts of the system.
Does anyone know if it is possible to do an inner join on a detached criteria?
MySql 5.6.5 has addressed the performance issue related to the query structure.
See here: http://bugs.mysql.com/bug.php?id=42259
No need for me to change the output format of my NHibernate queries anymore. :)