In my schema, User hasMany GameSession, GameSession hasMany Game.
For a particular user, I'm trying to select the most recent game session which has three games, each of which has a non-null score.
This comment looks like it does almost exactly what I want, but it's not working for me. It suggests this code:
ModelA.findAll({
include: [ModelB],
having: 'count(ModelB.id) > 0'
});
I think the above comment might be out of date, because unless I wrap the having clause in an array I get Error: where: "raw query" has been removed, please use where ["raw query", [replacements]].
So I have this JS:
user.getGameSessions({
include: [
{
model: models.Game,
where: {
score: {
$ne: null
}
}
}
],
having: ['count(Games.id) = 3'],
order: [['updatedAt', 'DESC']],
limit: 1
}).then(function(gameSessions) {
res.json(gameSessions);
});
And this is the query it is making:
SELECT
`GameSession`.*,
`Games`.`id` AS `Games.id`,
...
FROM (
SELECT
`GameSession`.`id`,
...
`GameSession`.`UserId`
FROM `GameSessions` AS `GameSession`
WHERE `GameSession`.`UserId` = 33 AND (
SELECT `GameSessionId`
FROM `Games` AS `Game`
WHERE (
`Game`.`GameSessionId` = `GameSession`.`id`
AND `Game`.`score` IS NOT NULL
)
LIMIT 1
) IS NOT NULL
HAVING count(Games.id) = 3
ORDER BY `GameSession`.`updatedAt` DESC
LIMIT 1
) AS `GameSession`
INNER JOIN `Games` AS `Games`
ON `GameSession`.`id` = `Games`.`GameSessionId`
AND `Games`.`score` IS NOT NULL
ORDER BY `GameSession`.`updatedAt` DESC;
But running this gives the following error: SequelizeDatabaseError: ER_BAD_FIELD_ERROR: Unknown column 'Games.id' in 'having clause'. I've also tried using singular Game.id in the having clause, since I'm not always certain when to use the plural and when to use the singular, and I get a similar error.
What am I doing wrong? How can I make this work?
Is it possible to limit an include?
Currently no. I think there's a feature request open
for it (although i couldn't find it with a cursory search. It's not
trivial to implement since it requires a join on a subquery.
I think because MySQL is case sensitive your Games.id is Games.Id or Games.ID.
If I were you to save yourself some grief in the future, just make every table and column uppercase or all lower case. Don't do camel casing or things like that! It becomes really hard to debug.
So your code here is probably:
having: 'count(Games.id) = 3',
should be:
having: 'count(Games.Id) = 3',
Edit:
Have you tried putting backticks around your count? Like this:
having: 'count(`Games`.`id`) = 3',
Related
I am trying to filter one table Payments by a field on the associated table Invoices.
Using the function matching() on the query object filters correctly but causes duplicate rows. It seemed like the solution was using distinct(), but calling distinct(Payments.id) results in an invalid query. I'm doing the following in a controller action.
$conditions = [
'Payments.is_deleted =' => false
];
$args = [
'conditions' => $conditions,
'contain' => ['Invoices', 'Invoices.Clients'],
];
$payments = $this->Payments->find('all', $args);
if($issuer) {
// This causes duplicate rows
$payments->matching('Invoices', function ($q) use ($issuer) {
return $q->where(['Invoices.issuer_id' => $issuer['id']]);
});
// $payments->distinct('Payments.id'); // Causes a mysql error
}
Am I correct in thinking that distinct() is what I need, and if so any idea what's missing to make it work?
I'm getting the following mysql error when uncommenting the line above:
Error: SQLSTATE[42000]: Syntax error or access violation: 1055 Expression #8 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'InvoicesPayments.id' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by
Full query:
SELECT
PAYMENTS.ID AS `PAYMENTS__ID`,
PAYMENTS.CREATED AS `PAYMENTS__CREATED`,
PAYMENTS.MODIFIED AS `PAYMENTS__MODIFIED`,
PAYMENTS.DATE_REGISTERED AS `PAYMENTS__DATE_REGISTERED`,
PAYMENTS.USER_ID AS `PAYMENTS__USER_ID`,
PAYMENTS.AMOUNT AS `PAYMENTS__AMOUNT`,
PAYMENTS.IS_DELETED AS `PAYMENTS__IS_DELETED`,
INVOICESPAYMENTS.ID AS `INVOICESPAYMENTS__ID`,
INVOICESPAYMENTS.INVOICE_ID AS `INVOICESPAYMENTS__INVOICE_ID`,
INVOICESPAYMENTS.PAYMENT_ID AS `INVOICESPAYMENTS__PAYMENT_ID`,
INVOICESPAYMENTS.PART_AMOUNT AS `INVOICESPAYMENTS__PART_AMOUNT`,
INVOICES.ID AS `INVOICES__ID`,
INVOICES.CREATED AS `INVOICES__CREATED`,
INVOICES.MODIFIED AS `INVOICES__MODIFIED`,
INVOICES.IS_PAID AS `INVOICES__IS_PAID`,
INVOICES.IS_DELETED AS `INVOICES__IS_DELETED`,
INVOICES.CLIENT_ID AS `INVOICES__CLIENT_ID`,
INVOICES.ISSUER_ID AS `INVOICES__ISSUER_ID`,
INVOICES.NUMBER AS `INVOICES__NUMBER`,
INVOICES.SUBTOTAL AS `INVOICES__SUBTOTAL`,
INVOICES.TOTAL AS `INVOICES__TOTAL`,
INVOICES.DATE_REGISTERED AS `INVOICES__DATE_REGISTERED`,
INVOICES.CURRENCY AS `INVOICES__CURRENCY`,
INVOICES.RECEIVER_NAME AS `INVOICES__RECEIVER_NAME`,
INVOICES.RECEIVER_RFC AS `INVOICES__RECEIVER_RFC`,
INVOICES.EMAIL_SENDER AS `INVOICES__EMAIL_SENDER`,
INVOICES.PDF_PATH AS `INVOICES__PDF_PATH`
FROM
PAYMENTS PAYMENTS
INNER JOIN
INVOICES_PAYMENTS INVOICESPAYMENTS
ON PAYMENTS.ID = (
INVOICESPAYMENTS.PAYMENT_ID
)
INNER JOIN
INVOICES INVOICES
ON (
INVOICES.ISSUER_ID = :C0
AND INVOICES.ID = (
INVOICESPAYMENTS.INVOICE_ID
)
)
WHERE
(
PAYMENTS.IS_DELETED = :C1
AND PAYMENTS.DATE_REGISTERED >= :C2
AND PAYMENTS.DATE_REGISTERED <= :C3
)
GROUP BY
PAYMENT_ID
ORDER BY
PAYMENTS.DATE_REGISTERED ASC
That behavior is expected, as matching will use an INNER join, and yes, grouping is how you avoid duplicates:
As this function will create an INNER JOIN, you might want to consider calling distinct on the find query as you might get duplicate rows if your conditions don’t exclude them already. This might be the case, for example, when the same users comments more than once on a single article.
Cookbook > Database Access & ORM > Query Builder > Loading Associations > Filtering by Associated Data
As the error message states, your MySQL server is configured to use the strict only_full_group_by mode, where your query is invalid. You can either disable that strict mode as mentioned by Akash prajapati (which can come with its own problems, as MySQL is then allowed to pretty much pick values of a group at random), or you could change how you query things in order to conform to the strict mode.
In your case where you need to group on the primary key, you could simply switch to using innerJoinWith() instead, unlike matching() this will not add any fields of that association to the SELECT list, and things should be fine in strict mode, as everything else is functionally dependent:
In cases where you would group on a key that would break functional dependency detection, one way to solve that could for example be to use a subquery for filtering, one that only selects that key, something along the lines of this:
$conditions = [
'Payments.is_deleted =' => false
];
$payments = $this->Payments
->find()
->contain(['Invoices.Clients']);
if($issuer) {
$matcherQuery = $this->Payments
->find()
->select(['Payments.some_other_field'])
->where($conditions)
->matching('Invoices', function ($q) use ($issuer) {
return $q->where(['Invoices.issuer_id' => $issuer['id']]);
})
->distinct('Payments.some_other_field');
$payments->where([
'Payments.some_other_field IN' => $matcherQuery
]);
} else {
$payments->where($conditions);
}
This will result in a query similar to this, where the outer query can then select all the fields you want:
SELECT
...
FROM
payments
WHERE
payments.some_other_field IN (
SELECT
payments.some_other_field
FROM
payments
INNER JOIN
invoices_payments ON
payments.id = invoices_payments.payment_id
INNER JOIN
invoices ON
invoices.issuer_id = ...
AND
invoices.id = invoices_payments.invoice_id
WHERE
payments.is_deleted = ...
GROUP BY
payments.some_other_field
)
The problem with sql_mode value in mysql so you need to set the sql_mode value as blank and then you can try and working fine for you.
SET GLOBAL sql_mode=(SELECT REPLACE(##sql_mode,'ONLY_FULL_GROUP_BY',''));
Please let me know still anything else.
I had the same issue, but was too afraid to set the sql_mode as mentioned by #Akash and also too much in a hurry to restructure the query. So I decided to use the inherited Collection method indexBy()
https://book.cakephp.org/4/en/core-libraries/collections.html#Cake\Collection\Collection::indexBy
$resultSetFromYourPaymentsQuery = $resultSetFromYourPaymentsQuery->indexBy('id');
It worked like a charm and it is DB independent.
EDIT: After some more tinkering, this might not be practical for all use cases. Replacing matching with innerJoinWith as proposed in the accepted answer will probably solve it in more generalized manner.
Tag.joins(:quote_tags).group('quote_tags.tag_id').order('count desc').select('count(tags.id) AS count, tags.id, tags.name')
Build query:
SELECT count(tags.id) AS count, tags.id, tags.name FROM `tags` INNER JOIN `quote_tags` ON `quote_tags`.`tag_id` = `tags`.`id` GROUP BY quote_tags.tag_id ORDER BY count desc
Result:
[#<Tag id: 401, name: "different">, ... , #<Tag id: 4, name: "family">]
It not return count column for me. How can I get it?
Have you tried calling the count method on one of the returned Tag objects? Just because inspect doesn't mention the count doesn't mean that it isn't there. The inspect output:
[#<Tag id: 401, name: "different">, ... , #<Tag id: 4, name: "family">]
will only include things that the Tag class knows about and Tag will only know about the columns in the tags table: you only have id and name in the table so that's all you see.
If you do this:
tags = Tag.joins(:quote_tags).group('quote_tags.tag_id').order('count desc').select('count(tags.id) AS count, tags.id, tags.name')
and then look at the counts:
tags.map(&:count)
You'll see the array of counts that you're expecting.
Update: The original version of this answer mistakenly characterized select and subsequent versions ended up effectively repeating the current version of the other answer from #muistooshort. I'm leaving it in it's current state because it has the information about using raw sql. Thanks to #muistooshort for pointing out my error.
Although your query is in fact working as explained by the other answer, you can always execute raw SQL as an alternative.
There are a variety of select_... methods you can choose from, but I would think you'd want to use select_all. Assuming the build query that you implicitly generated was correct, you can just use that, as in:
ActiveRecord::Base.connection.select_all('
SELECT count(tags.id) AS count, tags.id, tags.name FROM `tags`
INNER JOIN `quote_tags` ON `quote_tags`.`tag_id` = `tags`.`id`
GROUP BY quote_tags.tag_id
ORDER BY count desc')
See http://api.rubyonrails.org/classes/ActiveRecord/ConnectionAdapters/DatabaseStatements.html for information on the various methods you can choose from.
I want to show posts that are tagged with the current locale first. I thought that the MySQL FIELD function would be a nice way to solve this.
However,
Post.includes(:tags).order("FIELD(tags.name, '#{I18n.locale}') DESC").first
results in
Mysql2::Error: Unknown column 'tags.name' in 'order clause': SELECT spree_posts`.* FROM 'spree_posts' ORDER BY FIELD(tags.name, 'en') DESC LIMIT 1
If I try
Post.includes(:tags).order("FIELD(tags.name, '#{I18n.locale}') DESC").where.not(tags: { name: 'WTF?' } )
it returns the posts according to the FIELD method correctly, but does a join with tags, hence not returning all posts without tags :(
I think this might be a bug in ActiveRecord? It's definitely not the behaviour I expect.
If you want to use includes and ensure Rails loads the table in the main query, you should use references:
Post.includes(:tags).references(:tags).
order("FIELD(tags.name, '#{I18n.locale}') DESC").first
SELECT spree_posts`.* FROM 'spree_posts' ORDER BY FIELD(tags.name,
'en') DESC LIMIT 1
There is no tags table in this query, the table is not loaded, because includes is a smart method, that generates JOIN only when included table is used in WHERE clause.
You need to explicitly define LEFT JOIN:
Post.joins('LEFT JOIN tags ON tags.post_id = post.id').order("FIELD(tags.name, '#{I18n.locale}') DESC").first
(by default joins generates INNER JOIN)
I've already looked at all similar solved questions but I've not found the solution yet.
So In this wordpress there are houses and circuits (custom post types). A house can have several circuits attached (repeater custom field ). The query gets dinamically built to retrieve all houses associated to any circuit from a list of IDS.
This works, but it repeats a house when it has more than one circuit that matches. For example:
SELECT DISTINCT *
FROM reoask5_posts p
INNER JOIN reoask5_postmeta pm_ci ON p.ID = pm_ci.post_id
WHERE p.post_type = 'casa'
AND p.post_status = 'publish'
AND pm_ci.meta_key LIKE 'circuito|__|_ci' ESCAPE '|'
AND (pm_ci.meta_value = 194 OR pm_ci.meta_value = 189)
...retrieves houseA which has circuito_0_ci = 194 and retrieves two times houseB which has circuito_0_ci = 194 and circuito_1_ci = 189 associated.
So what should I modify to filter out duplicates? What I'm doing wrong?
UPDATE1: More info
The table schema for posts (p)
The table schema for postmeta (pm_ci)
Still doesn't work well. Selectin specific columns from post database does filter out duplicates. But it fails in looping through the results like a normal wordpress loop. Even if I select ALL the columns:
SELECT DISTINCT p.id, p.post_author, p.post_date, p.post_date_gmt, p.post_content, p.post_title, p.post_excerpt, p.post_status, p.comment_status, p.ping_status, p.post_password, p.post_name, p.to_ping, p.pinged, p.post_modified, p.post_modified_gmt, p.post_content_filtered, p.post_parent, p.guid, p.menu_order, p.post_type, p.post_mime_type, p.comment_count
It gives me errors:
Notice: Undefined property: stdClass::$ID in .... line 250
Notice: Undefined property: stdClass::$ID in .... line 251
// etc...
This the simplified php code that comes after defining the query. All of these calls fail when I select specific columns:
$total_posts= $wpdb->get_results($query);
foreach ($total_posts as $post){
$post->post_name;
get_permalink($post->ID);
$coordenadas = get_field( "coords", $post->ID);
// ...
}
If I add all the 4 columns from metaposts to the SELECT line:
... p.comment_count, pm_ci.meta_id, pm_ci.post_id, pm_ci.meta_key, pm_ci.meta_value
It still gives me errors when trying to use those functions, that work when I use SELECT *
And I don't know how would I be supposed to use group by in this case. Still learning.
UPDATE2: Found the solution
The reason for the errors was because SELECT DISTINCT p.id should read SELECT DISTINCT p.ID . The SQL query returned the ID in a case insensitive manner, storing it in key 'id' , but the wordpress functions need that key in capitals ('ID') in order to work.
Thank you!
Your using DISTINCT * . This means that if one value in any one column is different (from a different row) it will be shown.
What you should instead do, is write distinct followed by the specific columns your interested in.
So, if your only interested in the specific houses -
Example:
select distinct houses
FROM ...
Using your Format:
select distinct
p.post_type
FROM ...
You could also use GROUP BY to count the number of different circuits a house has.
I'm using this kind of queries with different parameters :
EXPLAIN SELECT SQL_NO_CACHE `ilan_genel`.`id` , `ilan_genel`.`durum` , `ilan_genel`.`kategori` , `ilan_genel`.`tip` , `ilan_genel`.`ozellik` , `ilan_genel`.`m2` , `ilan_genel`.`fiyat` , `ilan_genel`.`baslik` , `ilan_genel`.`ilce` , `ilan_genel`.`parabirimi` , `ilan_genel`.`tarih` , `kgsim_mahalleler`.`isim` AS mahalle, `kgsim_ilceler`.`isim` AS ilce, (
SELECT `ilanresimler`.`resimlink`
FROM `ilanresimler`
WHERE `ilanresimler`.`ilanid` = `ilan_genel`.`id`
LIMIT 1
) AS resim
FROM (
`ilan_genel`
)
LEFT JOIN `kgsim_ilceler` ON `kgsim_ilceler`.`id` = `ilan_genel`.`ilce`
LEFT JOIN `kgsim_mahalleler` ON `kgsim_mahalleler`.`id` = `ilan_genel`.`mahalle`
WHERE `ilan_genel`.`ilce` = '703'
AND `ilan_genel`.`durum` = '1'
AND `ilan_genel`.`kategori` = '1'
AND `ilan_genel`.`tip` = '9'
ORDER BY `ilan_genel`.`id` DESC
LIMIT 225 , 15
and this is what i get in explain section:
these are the indexes that i already tried to use:
any help will be deeply appreciated what kind of index will be the best option or should i use another table structure ?
You should first simplify your query to understand your problem better. As it appears your problem is constrained to the ilan_gen1 table, the following query would also show you the same symptoms.:
SELECT * from ilan_gene1 WHERE `ilan_genel`.`ilce` = '703'
AND `ilan_genel`.`durum` = '1'
AND `ilan_genel`.`kategori` = '1'
AND `ilan_genel`.`tip` = '9'
So the first thing to do is check that this is the case. If so, the simpler question is simply why does this query require a file sort on 3661 rows. Now the 'hepsi' index sort order is:
ilce->mahelle->durum->kategori->tip->ozelik
I've written it that way to emphasise that it is first sorted on 'ilce', then 'mahelle', then 'durum', etc. Note that your query does not specify the 'mahelle' value. So the best the index can do is lookup on 'ilce'. Now I don't know the heuristics of your data, but the next logical step in debugging this would be:
SELECT * from ilan_gene1 WHERE `ilan_genel`.`ilce` = '703'`
Does this return 3661 rows?
If so, you should be able to see what is happening. The database is using the hepsi index, to the best of it's ability, getting 3661 rows back then sorting those rows in order to eliminate values according to the other criteria (i.e. 'durum', 'kategori', 'tip').
The key point here is that if data is sorted by A, B, C in that order and B is not specified, then the best logical thing that can be done is: first a look up on A then a filter on the remaining values against C. In this case, that filter is performed via a file sort.
Possible solutions
Supply 'mahelle' (B) in your query.
Add a new index on 'ilan_gene1' that doesn't require 'mahelle', i.e. A->C->D...
Another tip
In case I have misdiagnosed your problem (easy to do when I don't have your system to test against), the important thing here is the approach to solving the problem. In particular, how to break a complicated query into a simpler query that produces the same behaviour, until you get to a very simple SELECT statement that demonstrates the problem. At this point, the answer is usually much clearer.