Yii Framework and HAS_ONE relation with millions of rows - mysql

Good afternoon.
I have a model called 'Cliente' and another called 'Acct'. The ratio is 1 'Cliente' for many 'Acct'. When I use a has_one relationship, it fetches all the millions of 'Acct' to pick only one of these results.
Statement of the model in relation 'Cliente':
'accts' => [
self::HAS_MANY,
'Acct',
'cliente_id',
],
'lastAcct' => [
self::HAS_ONE,
'Acct',
'cliente_id',
'order' => 'acct.id DESC',
],

In Yii (Yii1 as well as Yii2), creating a "Has One" relationship does not automatically apply a LIMIT 1 to the query. You can read more about the reasoning behind it here: https://github.com/yiisoft/yii/pull/2113
You should manually add the limit clause, like so:
'lastAcct' => [
self::HAS_ONE,
'Acct',
'cliente_id',
'order' => 'acct.id DESC',
'limit' => '1'
],

Thanks for your answer, sir.
This 'limit' to enter the main query and not only in relation, for example, if you search thousands of 'clients' with the last 'acct' in each will not work, will get only one 'cliente'.
Complementing ...
To solve this I use a subquery, for example:
LEFT OUTER JOIN `radacct` `acct` ON ((acct.username = t.login) AND (acct.radacctid = (SELECT radacctid FROM `radacct` `acct_subquery` WHERE acct_subquery.username = t.login GROUP BY acct_subquery.username ORDER BY acct_subquery.radacctid DESC LIMIT 1)))
But what worries me is that after this subquery be too cool with millions of results, it is a must do and ERP reports, weekly, monthly, yearly and since he started the company.

Related

CakePHP 4: How to add count of association results to where conditions

I'm trying to add the count results of an association to the where conditions, for example:
$findQuery = $this->Products->find('all');
$findQuery->leftJoinWith('Synonyms', function($q) {
return $q->where(['Synonyms.title LIKE' => '%TEST%']);
});
$findQuery->select([
'amount_of_matching_synonyms' => $findQuery->func()->count('Synonyms.id')
]);
$findQuery->where([
'OR' => [
'Products.description LIKE' => '%TEST%',
'amount_of_matching_synonyms >' => 0
]
]);
What happens now is that I'm getting 1 result returned with the 'amount_of_matching_synonyms' field. But this appears to have a cumulated result of all the records it should return.
Please help me out!
You should first figure out how to do these things in plain SQL, it will then be much easier to translate things to the query builder.
Counting related data requires joining in the realted data and creating groups on which aggregate functions can be used, and you're missing the latter. Furthermore you cannot use aggregates in the WHERE clause, as grouping happens after the WHERE clause is applied, you would have to use the HAVING clause instead.
The basic SQL to filter on such counting would look something like this:
SELECT
COUNT(synonyms.id) amount_of_matching_synonyms
FROM
products
LEFT JOIN
synonyms ON synonyms.id = synonyms.product_id
GROUP BY
products.id
HAVING
amount_of_matching_synonyms > 0
Translating this into the query builder would be fairly simple, you'd just need group() and having(), something like this:
$findQuery = $this->Products
->find()
->select([
'Products.description',
'amount_of_matching_synonyms' => $findQuery->func()->count('Synonyms.id')
])
->leftJoinWith('Synonyms', function(\Cake\ORM\Query $q) {
return $q->where(['Synonyms.title LIKE' => '%TEST%']);
})
->group('Products.id')
->having([
'OR' => [
'Products.description LIKE' => '%TEST%',
'amount_of_matching_synonyms >' => 0
],
]);
Note that you need to select the description, otherwise the condition in the having clause would fail.
The resulting SQL would look something like this:
SELECT
products.description,
COUNT(synonyms.id) amount_of_matching_synonyms
FROM
products
LEFT JOIN
synonyms ON
synonyms.product_id = products.id
AND
synonyms.title LIKE '%TEST%'
GROUP BY
products.id
HAVING
products.description LIKE '%TEST%'
OR
amount_of_matching_synonyms > 0
See also
MySQL 5.7 Reference Manual / Functions and Operators / Aggregate Functions
Cookbook > Database Access & ORM > Query Builder > Aggregates - Group and Having
Cookbook > Database Access & ORM > Query Builder > Subqueries

Find all distinct field values for a Model that occur N times in CakePHP

So I have a model FeaturedListing that has a field date which is a mysql date field. There will be multiple FeaturedListings that have the same date value.
I want to find all dates that have N or more FeaturedListings on it. I think I'd have to group by date and then find which ones have N or more in there group and get the value that was grouped on. Could any one give me any pointers to accomplish that. A raw sql query may be required.
Edit: Thanks to the answers below it got me going on the right track and I finally have a solution I like. This has some extra conditions specific to my application but I think its pretty clear. This snippet finds all dates after today that have at least N featured listings on them.
$dates = $this->find('list', array(
'fields' => array('FeaturedListing.date'),
'conditions' => array('FeaturedListing.date >' => date('Y-m-d') ),
'group' => array('FeaturedListing.date HAVING COUNT(*) >= $N')
)
);
I then make a call to array_values() to remove the index from the returned list and flatten it to an array of date strings.
$dates = array_values($dates);
No need to go to raw SQL, you can achieve this easily in cake ($n is the variable that holds N):
$featuredListings = $this->FeaturedListing->find('all', array(
'fields' => array('FeaturedListing.date'),
'group' => array('FeaturedListing.date HAVING COUNT(*)' => $n),
));
In "raw" SQL you would use group by and having:
select `date`
from FeaturedListings fl
group by `date`
having count(*) >= N;
If you want the listings on these dates, you need to join this back to the original data. Here is one method:
select fl.*
from FeaturedListings fl join
(select `date`
from FeaturedListings fl
group by `date`
having count(*) >= N
) fld
on fl.`date` = fld.`date`

CakePHP 2.2.4: Why does this Union $this->Model->query not work?

I'm in the view action of my PhotosController.php. What I want to do is given the id of the current photo I am viewing, create a carousel of photos containing the two photos before and two photos after the current photo with the current photo in the middle (5 in total).
I was pointed to this solution but I can't seem to convert it to CakePHP using $this->Photo->query.
My controller
$this->set('photos', $this->Photo->query("
SELECT id, file FROM photos WHERE id <= $id AND page_id = $page_id ORDER BY id DESC LIMIT 3
UNION ALL
SELECT id, file FROM photos WHERE id > $id AND page_id = $page_id ORDER BY id ASC LIMIT 2
"));
Unfortunately, when I don't see anything when I turn debugging on. id, file, and page_id are all columns in the photos table. Both #id and $page_id are passed to the action from the router. Is my syntax wrong?
EDIT: If I remove the UNION ALL and the second SELECT statement, then the query works fine so it's not an issue with the model not being loaded because it is.
EDIT (workaround): For now I'm doing two queries which is not ideal.
$this->set('photos_before', $this->Photo->find('all', array(
'conditions' => array(
'Photo.page_id' => $page_id,
'Photo.id <' => $id
),
'order' => array('Photo.id ASC'),
'limit' => 2
)));
$this->set('photos_after', $this->Photo->find('all', array(
'conditions' => array(
'Photo.page_id' => $page_id,
'Photo.id >' => $id
),
'order' => array('Photo.id ASC'),
'limit' => 2
)));
I have a contain before hand to only return the fields and associated models I need.
Below is what I want to be displayed and it currently works using the two queries above but I am hoping this can be achieved with a single, Cake-friendly query
My guess is that your original query is invalid SQL. Afaik UNIONS cannot contain multiple 'order by' clauses. As a workaround you may consider to rewrite it to use subqueries like this:
SELECT * FROM (SELECT id, file FROM photos WHERE id <= $id AND page_id = $page_id ORDER BY id DESC LIMIT 3) AS suba
UNION ALL
SELECT * FROM (SELECT id, file FROM photos WHERE id > $id AND page_id = $page_id ORDER BY id ASC LIMIT 2) AS subb
Although I serious think a query like this is far from optimal. Of course, I don't know the way your application works, but it seems that a standard pagination query, with a OFFSET/LIMIT is a more logical approach.
Please take my comment below your question into account; using model->query does NOT automatically handle sanitisation/escaping to prevent SQL injections!
You have to load model as
$this->loadModel('Photo');
Before executing query.
You should create a VIEW in MySQL and then use that as a model, and do a traditional CakePHP find on that.
Read up on creating views in MySQL and then create a model based on that view name.

Using get_user in wordpress with sorting

I have wp_users table which has a column ordering. I came to know that get_users() returns all the users.
I am using it like get_users('orderby=ordering')
I got help form this link
But unfortunately it is not sorting on ordering column.
Any help?
You should first take a look at the users table from the database.
The command you try is good, but the argument you use for ordering might be wrong. You should order by a column from the users table, for example user name, or user id's..
On the link you mentioned I've found these:
orderby - Sort by 'ID', 'login', 'nicename', 'email', 'url', 'registered', 'display_name', or 'post_count'.
order - ASC (ascending) or DESC (descending).
Some working examples:
Get users by nicename:
$users = get_users('orderby=nicename');
Other examples:
Display users sorted by Post Count, Descending order
$user_query = new WP_User_Query( array ( 'orderby' => 'post_count', 'order' => 'DESC' ) );
Display users sorted by registered, Ascending order
$user_query = new WP_User_Query( array ( 'orderby' => 'registered', 'order' => 'ASC' ) );

Rails query on multiple primary keys with conditions on association

Is there a way in Active Record to construct a single query that will do a conditional join for multiple primary keys?
Say I have the following models:
Class Athlete < ActiveRecord::Base
has_many :workouts
end
Class Workout < ActiveRecord::Base
belongs_to :athlete
named_scope :run, :conditions => {:type => "run"}
named_scope :best, :order => "time", :limit => 1
end
With that, I could generate a query to get the best run time for an athlete:
Athlete.find(1).workouts.run.best
How can I get the best run time for each athlete in a group, using a single query?
The following does not work, because it applies the named scopes just once to the whole array, returning the single best time for all athletes:
Athlete.find([1,2,3]).workouts.run.best
The following works. However, it is not scalable for larger numbers of Athletes, since it generates a separate query for each Athlete:
[1,2,3].collect {|id| Athlete.find(id).workouts.run.best}
Is there a way to generate a single query using the Active Record query interface and associations?
If not, can anyone suggest a SQL query pattern that I can use for find_by_SQL? I must confess I am not very strong at SQL, but if someone will point me in the right direction I can probably figure it out.
To get the Workout objects with the best time:
athlete_ids = [1,2,3]
# Sanitize the SQL as we need to substitute the bind variable
# this query will give duplicates
join_sql = Workout.send(:santize_sql, [
"JOIN (
SELECT a.athlete_id, max(a.time) time
FROM workouts a
WHERE a.athlete_id IN (?)
GROUP BY a.athlete_id
) b ON b.athlete_id = workouts.athlete_id AND b.time = workouts.time",
athlete_ids])
Workout.all(:joins => join_sql, :conditions => {:athlete_id => })
If you require just the best workout time per user then:
Athlete.max("workouts.time", :include => :workouts, :group => "athletes.id",
:conditions => {:athlete_id => [1,2,3]}))
This will return a OrderedHash
{1 => 300, 2 => 60, 3 => 120}
Edit 1
The solution below avoids returning multiple workouts with same best time. This solution is very efficient if athlete_id and time columns are indexed.
Workout.all(:joins => "LEFT OUTER JOIN workouts a
ON workouts.athlete_id = a.athlete_id AND
(workouts.time < b.time OR workouts.id < b.id)",
:conditions => ["workouts.athlete_id = ? AND b.id IS NULL", athlete_ids]
)
Read this article to understand how this query works. Last check (workouts.id < b.id) in the JOIN ensures only one row is returned when there are more than one matches for the best time. When there are more than one match to the best time for an athlete, the workout with the highest id is returned(i.e. the last workout).
Certainly following will not work
Athlete.find([1,2,3]).workouts.run.best
Because Athlete.find([1,2,3]) returns an array and you can't call Array.workouts
You can try something like this:
Workout.find(:first, :joins => [:athlete], :conditions => "athletes.id IN (1,2,3)", :order => 'workouts.time DESC')
You can edit the conditions according to your need.