i'm having an issue with how eloquent is formulation a query that i have no access to. When doing something like
$model->where('something')
->distinct()
->paginate();
eloquent runs a query to get the total count, and the query looks something like
select count(*) as aggregate from .....
The problem is that if you use distinct in the query, you want something like
select count(distinct id) as aggregate from .....
to get the correct total. Eloquent is not doing that though, thus returning wrong totals. The only way to get the distinct in count is to pass an argument through the query builder like so ->count('id') in which case it will add it. Problem is that this query is auto-generated and i have no control over it.
Is there a way to trick it into adding the distinct on the count query?
P.S Digging deep into the builders code we find an IF statement asking for a field on the count() method in order to add the distinct property to the count. Illuminate\Database\Query\Grammars\BaseGrammar#compileAggregate
if ($query->distinct && $column !== '*')
{
$column = 'distinct '.$column;
}
return 'select '.$aggregate['function'].'('.$column.') as aggregate';
P.S.1 I know that in SQL you could do a group by, but since i'm eager loading stuff it is not a good idea cause it will add a IN (number of id's found) to each of the other queries which slows things down significantly.
I faced the exact same problem and found two solutions:
The bad one:
$results = $model->groupBy('foo.id')->paginate();
It works but it will costs too much memory (and time) if you have a high number of rows (it was my case).
The better one:
$ids = $model->distinct()->pluck('foo.id');
$results = $query = $model->whereIn('foo.id', $ids)->paginate();
I tried this with 100k results, and had no problem at all.
This seems to be a wider problem, discussed here:
https://github.com/laravel/framework/issues/3191
https://github.com/laravel/framework/pull/4088
Untill the fixes are shipped with one of the next Laravel releases, you can always try using the raw expressions, like below (I didnt test it, but should work)
$stuff = $model->select(DB::raw('distinct id as did'))
->where('whatever','=','whateverelse')
->paginate();
Reference: http://laravel.com/docs/queries#raw-expressions
$model->where('something')->distinct()->count('id')->paginate();
Related
I have two tables.
rooms with id, title
chats with id, content, room_id, created_at
My goal is fetch rooms and orderBy created_at of chats in desc mode. It's a simple query, yet it's giving a headache in Laravel. I cannot use the eloquent ORM way and do a sort in with method. Thus, I had to use a join.
My query looks like this:
$items = $items->select("rooms.*")->join("chats", "chats.room_id", "=", "rooms.id")->orderBy("chats.created_at", "desc")->groupBy('rooms.id');
Result:
I'm getting the data without any order for created_at.
removing groupBy will result in a correct order, but because it's missing the groupby, the room will exist as much as it has chats. So it's not a great approach.
What am I missing? If i need to include other information, please let me know in the comments.
Just like the comment said, I should use an aggregate function in the query while using a groupby function. Basically, it will look like this
$items = $items->select("rooms.*")
->selectRaw("max(chats.created_at) as latest_created_at")
->join("chats", "chats.room_id", "=", "rooms.id")
->orderBy("latest_created_at", "desc")
->groupBy('rooms.id');
Then, it will be ordered correctly. I'll leave the thread, maybe someone will use the same clumsy search keywords as mine and see this question :)
I'm trying to get the most recent record for each candidate_id from a ìnterviews` table.
This is what I want to achive:
I'm using Eloquent on laravel and have already tried this methods (with and without eloquent):
$candidates = DB::table('interviews')->select('interviews.*', 'i2.*')
->leftJoin('interviews as i2',
function ($join) {
$join->on('interviews.candidate_id', '=', 'i2.candidate_id');
$join->on('interviews.created_at', '<', 'i2.created_at');
}
)
->whereNull('i2.candidate_id')
->get();
and with eloquent I've tried this:
$candidates = Interview::leftJoin('interviews as i2',
function ($join) {
$join->on('interviews.candidate_id', '=', 'i2.candidate_id');
$join->on('interviews.created_at', '<', 'i2.created_at');
}
)->whereNull('i2.candidate_id')
->get();
If I change get() to toSql() I have exactly the same query that's shown on the above image, but running on laravel I'm getting always these results (this using the first method, with query builder):
Anyone know why I get this results? Is hard to understand that laravel is doing the same query that I do in HeidiSql but I get diferent results :(
Any tip?
Thanks in advance!
Because you are using ->select('interviews.*', 'i2.*') combined with ->whereNull('i2.candidate_id') I am assuming the second select parameter is overriding all fields on the interviews table with nulls, try reversing the order to ->select('i2.*','interviews.*') or not use the i2.* at all.
This is because the output ignores the alias and only uses the fieldname as element key in the returned collection.
Hope it works.
Perfect case scenario you pick the exact columns you want from each of the joined tables for e.g. it may go like this: table1.id,table1.column1,table1.column2,table2.column2 as smth_so_it_doesnt_override
the code below is executed but it brings a wrong records(all records in my table) , it's like he doesn't take on consideration the clause where
$result = $this->Posts->query('SELECT * FROM POSTS WHERE id=1');
I know that I can do it easily with find() but for some reasons I want to write the sql statement and to have the right results
Thanks for helping me.
query() method does not take any parameter. you can use it like
$data= $this->Posts->query()
->where(['id'=>1])
->execute()
->fetchAll();
I want to do something like the following:
Post.count.explain # doesn't work
This fails because EXPLAIN is a method on Relation and Post.count isn't a relation. It's just a regular integer that is the result of the query. So how could a count query be EXPLAINed?
Here's a form that generates the exact same SQL query, but returns a Relation to call explain on:
Post.select('count(*)').explain
Both generate the SQL
SELECT COUNT(*) FROM `posts`
...so the query plan should be the same.
From ActiveRecord::Relation#explain, we can make that method accept a block.
module ExplainBlock
def explain_block(&block)
exec_explain(collecting_queries_for_explain { instance_exec(&block) })
end
end
ActiveRecord::Relation.include(ExplainBlock)
Then Post.all.explain_block { count } .
The COUNT shouldn't affect the query plan, since the only difference it does is to tell the database to fetch the row data, but the rows need to be found anyway with/without the COUNT.
I'm trying to basically do a search through the database with Perl to tell if there is an item with a certain ID. This search can return no rows, but it can also return one.
I have the following code:
my $th = $dbh->prepare(qq{SELECT bi_exim_id FROM bounce_info WHERE bi_exim_id = '$exid'});
$th->execute();
if ($th->fetch()->[0] != $exid) {
...
Basically, this tries to see if the ID was returned and if it's not, continue with the script. But it is throwing a Null array reference error on the $th->fetch()->[0] thing.
How can i just simply check to see if it returned rows or now?
The DBD::mysql driver has a the rows() method that can return the count of the results:
$sth = $dbh->prepare( ... );
$sth->execute;
$rows = $sth->rows;
This is database-driver specific, so it might not work in other drivers, or it might work differently in other drivers.
my $th = $dbh->prepare(qq{SELECT bi_exim_id FROM bounce_info WHERE bi_exim_id = '$exid'});
$th->execute();
my $found = 0;
while ($th->fetch())
{
$found = 1;
}
Your query won't return anything if the row doesn't exist, so you can't de-reference the fetch.
Update: you might want to re-write that as
my $found = $th->fetch();
Why don't you just "select count(*) ..."??
my ($got_id) = $dbh->selectrow_array("SELECT count(*) from FROM bounce_info WHERE bi_exim_id = '$exid'");
Or to thwart Little Bobby Tables:
my $q_exid = $dbh->quote($exid);
my ($got_id) = $dbh->selectrow_array("SELECT count(*) from FROM bounce_info WHERE bi_exim_id = $q_exid");
Or if you're going to execute this a lot:
my $sth = $dbh->prepare("SELECT count(*) from FROM bounce_info WHERE bi_exim_id = ?");
....save $sth (or use prepare_cached()) and then later
my ($got_id) = $dbh->selectrow_array($sth, undef, $exid);
Change select to always return something? This should work in Sybase, dunno about other DBs.
my $th = $dbh->prepare(qq{SELECT count(*) FROM bounce_info WHERE bi_exim_id = '$exid'});
$th->execute();
if ($th->fetch()->[0]) {
....
}
In general, I'm not sure why people are so afraid of exceptions. You catch them and move on.
my $sth = prepare ...
$sth->execute;
my $result = eval { $sth->fetchrow_arrayref->[1] };
if($result){ say "OH HAI. YOU HAVE A RESULT." }
else { say "0 row(s) returned." }
In this case, though, Paul's answer is best.
Also, $sth->rows doesn't usually work until you have fetched every row. If you want to know how many rows match, then you have to actually ask the database engine the question you want to know the answer to; namely select count(1) from foo where bar='baz'.
The only reliable way to find out how many rows are returned by a SELECT query is to fetch them all and count them. As stated in the DBI documentation:
Generally, you can only rely on a row
count after a non-SELECT execute (for
some specific operations like UPDATE
and DELETE), or after fetching all the
rows of a SELECT statement.
For SELECT
statements, it is generally not
possible to know how many rows will be
returned except by fetching them all.
Some drivers will return the number of
rows the application has fetched so
far, but others may return -1 until
all rows have been fetched. So use of
the rows method or $DBI::rows with
SELECT statements is not recommended.
However, when you get down to it, you almost never need to know that in advance anyhow. Just loop on while ($sth->fetch) to process each row or, for the special case of a query that will only return zero or one rows,
if ($sth->fetch) {
# do stuff for one row returned
} else {
# do stuff for no rows returned
}
This article may users of MySQL prior to version 8:
http://www.arraystudio.com/as-workshop/mysql-get-total-number-of-rows-when-using-limit.html
Luckily since MySQL 4.0.0 you can use SQL_CALC_FOUND_ROWS option in your query which will tell MySQL to count total number of rows disregarding LIMIT clause. You still need to execute a second query in order to retrieve row count, but it’s a simple query and not as complex as your query which retrieved the data.
Usage is pretty simple. In you main query you need to add SQL_CALC_FOUND_ROWS option just after SELECT and in second query you need to use FOUND_ROWS() function to get total number of rows. Queries would look like this:
SELECT SQL_CALC_FOUND_ROWS name, email FROM users WHERE name LIKE 'a%' LIMIT 10;
SELECT FOUND_ROWS();
The only limitation is that you must call second query immediately after the first one because SQL_CALC_FOUND_ROWS does not save number of rows anywhere.
Although this solution also requires two queries it’s much more faster, as you execute the main query only once.
You can read more about SQL_CALC_FOUND_ROWS and FOUND_ROWS() in MySQL docs.
Sadly, as of 8.0.17:
The SQL_CALC_FOUND_ROWS query modifier and accompanying FOUND_ROWS() function are deprecated as of MySQL 8.0.17 ...
Details here.