I use ActiveDateProvider and I use query like this:
Foo::find()->joinWith('relation');
And Foo's relation is hasMany, so there is more than one record joined to every Foo.
In that case the pagination doesn't work properly (for example in GridView it says: Showing 1-6 of 17 items. even though the pagination is set to 10 and there are actually only 10 records of Foo in the database)
What can I do to make the pagination work properly (counting only Foo, not the joined records)
Sorry for late reply, but I recently stuck into the issue and found out that distinct() will do the trick for you.
Foo::find()->joinWith('relation')->distinct();
Related
I'm trying to write an eloquent query which returns up to 3 rows per group (using groupBy). This post is similar but I'm not sure how to apply that solution here.
I have an eloquent model Deal for my MySQL table deal. Each deal can have one of four types, so ultimately there should be at most four groups (or less if there are no deals of a certain type).
I have tried Deal::where('status', 'active')->groupBy('type')->get(); and this returns an array of four objects (one deal for each of the four types), however I am expecting 12 deals, 3 per group (and they do exist in the DB). Similarly, trying Deal::where('status', 'active')->groupBy('type')->take(3)->get(); returns an array of three objects (one group is now missing).
I've accomplished this so far by making four separate DB queries, one for each of the four types but I'd prefer it if this can be done in one. Any thoughts? Thank you!
One of the way you can achieve what you are trying to do is:
Deal:where('status', 'active')
->get()
->groupBy('type')
->map(function($deal) {
return $deal->take(3);
});
Of course, you need to check whether the query returns null or empty objects and please note that the query returns all the active Deal(s) (the filtering happens on the Collection) which might be non-efficient if there are lots of Deal(s)
You should call groupBy on the collection, not on the query:
Deal::where('status', 'active')->get()->groupBy('type');
To restrict the query itself to only return the maximum of 3 per group, refer to the article you linked to in your original question.
Trying to do a simple Model.all.page(1)
But whenever .page is called, it creates a SQL COUNT call. (My actual code is more complex than above, but simplified for ease of reading.) Is there a way to prevent .page from calling a SQL count? I'm dealing with millions of products and this calls is making the page refresh take an extra 2 seconds to load. I already have my own custom count which is instant so I don't need this .page count.
Edit: Def not using .all. Bad example sorry.
Heres a really simple example that is basically my code in a nutshell:
Product.limit(1).page(1)
With my real code SQL produces: (1495.3ms) SELECT COUNT(*) FROM 'products' LEFT OUTER JOIN...
I have joins on the products table that I don't need to be counted, hence the fact I have my own count methods I want to use and don't need .page to produce it's own count.
When you call Model.all.page(1) you are getting back an array instead of an ActiveRecord relation.
Try just calling Model.page(1) and you should get what you want... If what you want is:
Model.page(1)
# results in SELECT "models".* FROM "models" LIMIT 30 OFFSET 0
Edit:
So the issue ended up being in the will_paginate gem as it was calling count on the query to know the total number of entries so it can get an accurate number of pages. However will_paginate does provide an option to the paginate method which allows you to pass in a custom total_entries count which is useful if you have a massive table and don't care to get the precise number of pages for every record that matches the query.
You can pass in the option like so:
Model.paginate(:page => params[:page], :per_page => 30, :total_entries => 100)
You are concerned about doing a COUNT query, yet you are selecting ALL records from your database by doing Model.all? Are you joking right now?
Also you need to provide code in order to get help. We cant read your mind, we cant make up what code you might have. Especially when you say "my actual code is more complex than above". Don't try to simplify issues or hide code that you THINK is irrelevant.
What does your code look like? What does your log look like, specifically query time and total page time (rendering and ActiveRecord split out). You need to give more information.
I have searched for a solution for this problem, but haven't found it (yet), probably because I don't quite know how to explain it properly myself. If it is posted somewhere already, please let me know.
What I have is three databases that are related to each other; main, pieces & groups. Basically, the main database contains the most elementary/ most used information from a post and the pieces database contains data that is associated with that post. The groups database contains all of the (long) names of the groups a post in the main database can be 'posted in'. A post can be posted in multiple groups simultaneously. When a new post is added to my site, I check the pieces too see if there are any duplicates (check if the post has been posted already). In order to make the search for duplicates more effective, I only check the pieces that are posted in the same group(s).
Hopefully you're still with me, cause here's where it starts to get really confusing I think (let me know if I need to specify things more clearly): right now, both the main and the pieces database contain the full name of the group(s) (basically I'm not using the groups database at all). What I want to do is replace the names of those groups with their associated IDs from the groups database. For example, I want to change this:
from:
MAIN_table:
id | group_posted_in
--------|---------------------------
1 | group_1, group_5
2 | group_15, group_75
3 | group_1, group_215
GROUPS_table:
id | group_name
--------|---------------------------
1 | group_1
2 | group_2
3 | group_3
etc...
into:
MAIN_table:
id | group_posted_in
--------|---------------------------
1 | 1,5
2 | 15,75
3 | 1,215
Or something similar to this. However, This format specifically causes issues as the following query will return all of the rows (from the example), instead of just the one I need:
SELECT * FROM main_table WHERE group = '5'
I either have to change the query to something like this:
...WHERE group = '5' OR group = '5,%' OR group = '%,5,%' OR group = '%,5'
Or I have to change the database structure from Comma Separated Values to something like this: [15][75]. The accompanying query would be simpler, but it somehow seems like a cumbersome solution to me. Additionally, (simple) joins will not be easy/ possible at all. It will always require me to run a separate query to fetch the names of the groups--whether a user searches for posts in a specific group (in which case, I first have to run a query to fetch the id's, then to search for the associated posts), or whether it is to display them (first the posts, then another query to match the groups).
So, in conclusion: I suppose I know there is a solution to this problem, but my gut tells me that it is not the right/ best way to do it. So, I suppose the question that ties this post together is:
What is the correct method to connect the group database to the others?
For a many-to-many relationship, you need to create a joining table. Rather than storing a list of groups in a single column, you should split that column out into multiple rows in a separate table. This will allow you to perform set based functions on them and will significantly speed up the database, as well as making it more robust and error proof.
Main
MainID ...
Group
GroupID GroupName
GroupsInMain
GroupsInMainID MainID(FK) GroupID(FK)
So, for MainID 1, you would have GroupsInMain records:
1,1,1
2,1,5
This associates groups 1 and 5 with MainID 1
FK in this case means a Foreign Key (i.e. a reference to a primary key in another table). You'd probably also want to add a unique constraint to GroupsInMain on MainID and GroupID, since you'd never want the same values for the pairing to show up more than once.
Your query would then be:
select GroupsInMain.MainID, Group.GroupName
from Group, GroupsInMain
where Group.GroupID=GroupsInMain.GroupID
and Group.GroupID=5
I'm having an issue with a certain requirement to one of my Homework Assignments. I am required to take a list of students and print out all of the students with credit hours of 12 or more. The Credit hours are stored in a separate table, and referenced through a third table
basically, a students table, a classes table with hours, and an enrolled table matching student id to Course id
I used a SUM aggregate grouped by First name from the tables and that all works great, but I don't quite understand how to filter out the people with less than 12 hours, since the SQL doesn't know how many hours each person is taking until it's done with the query.
my string looks like this
'SELECT Students.Fname, SUM(Classes.Crhrs) AS Credits
FROM Students, Classes, Enrolled
WHERE Students.ID = Enrolled.StudentID AND Classes.ID = Enrolled.CourseID
GROUP BY Students.Fname;'
It works fine and shows the grid in the Delphi Project, but I don't know where to go from here to filter the results, since each query run deletes the previous.
Since it's a homework exercise, I'm going to give a very short answer: look up the documentation for HAVING.
Beside getting the desired result directly from SQL as Martijn suggested, Delphi datasets have ways to filter data on the "client side" also. Check the Filter property and the OnFilter record.
Anyway, remember it is usually better to apply the best "filter" on the database side using the proper SQL, and then use client side "filters" only to allow for different views on an already obtained data set, without re-querying the same data, thus saving some database resources and bandwidth (as long as the data on the server didn't change meanwhile...)
I have two tables, that is joined in some way. I've spent the last hour googling around, not finding any concrete answers that worked in my case.
The case is, table 1 is called Clients, and table 2 is called Projects. I need to list all the client names, followed by number of projects related to that project and the project title.
For example:
Client 1 (2 projects)
- Project 1 title
- Project 2 title
Client 2 (0 projects)
Client 3 (1 project)
- Project 1 title
How is this doable, in the simplest and easiest way?
There are basically three parts to your query.
Joining the tables
Grouping the output by project
Counting the projects per client
The first is easy. It involves finding the client ID field in each table and use an JOIN clause specifying the two columns (one in each table) to correlate on. This will give you one row per project that also contains information for the matching client. This is almost what you're asking for.
It is tricky to put the second and third together in the one query and not one I would recommend. If you are going to be putting this in a program, then you can easily post-process the result from the query. To do this, you need to add an ORDER BY clause to specify sorting by client. That will put all the projects for each client in subsequent rows.
Now you can write a loop to process the output. As it does to, it has to watch for two things:
when the client ID changes
counting the projects
By doing this, it can easily display a "group header" for each client, and a "group footer" for the number of projects.
You don't say anything about your app, but I'd recommend separating querying from displaying. Let SQL get the data for you and then let something else do presentation.