I'm building some analytical stat graphs for an administrator dashboard using Laravel. I'm having a lot of fun. Though I'm having a bit of difficulty understanding how to utilize the SQL queries and GroupBy function to achieve some stuff.
End goal: retreive the count of users who were created and groupBy the week they registered.
So far I'm using this:
DB::table("users")
->select(DB::raw("COUNT(*) as count"))
->orderBy("created_at")
->groupBy(function ($query)
{
return \Carbon\Carbon::parse($query->created_at)->format("W");
})
->get();
I'm getting an error in Tinker (which is where I do a majority of my testing at the moment it's something like this:
PHP warning: strtolower() expects parameter 1 to be string, object given in
/Users/Ian/sites/bangerz-army/vendor/laravel/framework/src/Illuminate/Database/Grammar.php on line 58
I had gotten a similar query to work in PHP but for performance reasons I'd prefer to keep as much of the math in SQL as possible.
/ Laravel 5.5
Try using DB::raw() in your group by
DB::table("users")
->select(DB::raw("COUNT(*) as count, WEEK(created_at)"))
->orderBy(DB::raw('WEEK(created_at)'))
->groupBy(DB::raw('WEEK(created_at)'))
->get();
Related
I'm developing an API using NestJS & TypeORM to fetch data from a MySQL DB. Currently I'm trying to get all the instances of an entity (HearingTonalTestPage) and all the related entities (e.g. Frequency). I can get it using createQueryBuilder:
const queryBuilder = await this.hearingTonalTestPageRepo
.createQueryBuilder('hearing_tonal_test_page')
.innerJoinAndSelect('hearing_tonal_test_page.hearingTest', 'hearingTest')
.innerJoinAndSelect('hearingTest.page', 'page')
.innerJoinAndSelect('hearing_tonal_test_page.frequencies', 'frequencies')
.innerJoinAndSelect('frequencies.frequency', 'frequency')
.where(whereConditions)
.orderBy(`page.${orderBy}`, StringToSortType(pageFilterDto.ascending));
The problem here is that this will produce a SQL query (screenshot below) which will output a line per each related entity (Frequency), when I want to output a line per each HearingTonalTestPage (in the screenshot example, 3 rows instead of 12) without losing its relations data. Reading the docs, apparently this can be easily achieved using the relations option with .find(). With QueryBuilder I see some relation methods, but from I've read, under the hood it will produce JOINs, which of course I want to avoid.
So the 1 million $ question here is: is it possible with CreateQueryBuilder to load the relations after querying the main entities (something similar to .find({ relations: { } }) )? If yes, how can I achieve it?
I am not an expert, but I had a similar case and using:
const qb = this.createQueryBuilder("product");
// apply relations
FindOptionsUtils.applyRelationsRecursively(qb, ["createdBy", "updatedBy"], qb.alias, this.metadata, "");
return qb
.orderBy("product.id", "DESC")
.limit(1)
.getOne();
it worked for me, all relations are correctly loaded.
ref: https://github.com/typeorm/typeorm/blob/master/src/find-options/FindOptionsUtils.ts
You say that you want to avoid JOINs, and are seeking an analogue of find({relations: {}}), but, as the documentation says, find({relations: {}}) uses under the hood, expectedly, LEFT JOINs. So when we talk about query with relations, it can't be without JOIN's.
Now about the problem:
The problem here is that this will produce a SQL query (screenshot
below) which will output a line per each related entity (Frequency),
when I want to output a line per each HearingTonalTestPage
Your query looks fine. And the result of the query, also, is ok. I think that you expected to have as a result of the query something similar to json structure(when the relation field contains all the information inside itself instead of creating new rows and spread all its values on several rows). But that is how the SQL works. By the way, getMany() method should return 3 HearingTonalTestPage objects, not 12, so what the SQL query returns should not worry you.
The main question:
is it possible with CreateQueryBuilder to load the relations after
querying the main entities
I did't get what do you mean by saying "after querying the main entities". Can you provide more context?
I am switching an application from PHP/MYSQL to Express and am using knex to connect to the MYSQL database. In one of my queries I use a statement like such (I have shortened it for brevity.)
SELECT ROUND(AVG(Q1),2) AS Q1 FROM reviews WHERE id=? AND active='1'
I am able to use ROUND if I use knex.raw but I am wondering if there is a way to write this using query builder. Using query builder makes dealing with the output on the view side so much easier than trying to navigate the objects returned from the raw query.
Here is what I have so far in knex.
let id = req.params.id;
knex('reviews')
//Can you wrap a ROUND around the average? Or do a Round at all?
.avg('Q1 as Q1')
.where('id', '=', id)
Thanks so much!
You can use raw inside select. In this case:
knex('reviews')
.select(knex.raw('ROUND(AVG(Q1),2) AS Q1'))
Check the docs here for more examples and good practices when dealing with raw statements.
I have a Question model from very large table of questions (600,000 records), with relation to Customer,Answer and Product models. Relations are irrelevant to this question but I mentioned them to clarify I need to use Eloquent. When I call Question::with('customer')->get(); it runs smoothly and fast.
But there is another table in which I have question_ids of all questions which should not be shown (for specific reasons).
I tried this code:
// omitted product ids, about 95,000 records
$question_ids_in_product = DB::table('question_to_product')
->pluck('product_id')->all();
$questions = Question::with('customer')
->whereNotIn('product_id', $question_ids_in_product)
->paginate($perPage)->get();
It takes so much time and shows this error:
SQLSTATE[HY000]: General error: 1390 Prepared statement contains too many placeholders
and sometimes Fatal error: Maximum execution time of 30 seconds exceeded
When I run it with plain sql query:
SELECT * FROM questions LEFT JOIN customers USING (customer_id)
WHERE question_id NOT IN (SELECT question_id FROM question_to_product)
it takes only 80 milliseconds
How can I use Eloquent in this situation?
You can use whereRaw method:
$questions = Question::with('customer')
->whereRaw('question_id NOT IN (SELECT question_id FROM question_to_product)')
->paginate($perPage)->get();
But ideally as you found out this is a better sollution:
Question::with('customer')->whereNotIn('question_id',
function ($query) {
$query->from('question_to_product') ->select('question_id');
}
);
Difference?
When you will migrate your database to another database the whereRaw might not work as you put in raw statements.
That is why we have Eloquent ORM which handles these transitions and build the appropriate queries to run.
No performance impact because the SQL is the same (for MySQL)
P.S: For better debugging try installing this debug bar
refer from https://laravel.com/docs/5.4/queries#where-clauses
$users = DB::table('questions')
->leftJoin('customers', 'curtomer.id', '=', 'question.user_id')
->whereNotIn('question_id', [1, 2, 3])
->get();
It'll work 100%. When you query getting longer to response like more than 30 seconds when you are using whereNotIn. Use this Query Syntax.
$order = Order::on($databaseCredentials['database'])
->whereRaw('orders_id NOT IN (SELECT orders_id FROM orders)')
->skip($page)
->take(10)
->orderBy('orders.updated_at', 'ASC')
->paginate(10);
I'm trying to get the most recent record for each candidate_id from a ìnterviews` table.
This is what I want to achive:
I'm using Eloquent on laravel and have already tried this methods (with and without eloquent):
$candidates = DB::table('interviews')->select('interviews.*', 'i2.*')
->leftJoin('interviews as i2',
function ($join) {
$join->on('interviews.candidate_id', '=', 'i2.candidate_id');
$join->on('interviews.created_at', '<', 'i2.created_at');
}
)
->whereNull('i2.candidate_id')
->get();
and with eloquent I've tried this:
$candidates = Interview::leftJoin('interviews as i2',
function ($join) {
$join->on('interviews.candidate_id', '=', 'i2.candidate_id');
$join->on('interviews.created_at', '<', 'i2.created_at');
}
)->whereNull('i2.candidate_id')
->get();
If I change get() to toSql() I have exactly the same query that's shown on the above image, but running on laravel I'm getting always these results (this using the first method, with query builder):
Anyone know why I get this results? Is hard to understand that laravel is doing the same query that I do in HeidiSql but I get diferent results :(
Any tip?
Thanks in advance!
Because you are using ->select('interviews.*', 'i2.*') combined with ->whereNull('i2.candidate_id') I am assuming the second select parameter is overriding all fields on the interviews table with nulls, try reversing the order to ->select('i2.*','interviews.*') or not use the i2.* at all.
This is because the output ignores the alias and only uses the fieldname as element key in the returned collection.
Hope it works.
Perfect case scenario you pick the exact columns you want from each of the joined tables for e.g. it may go like this: table1.id,table1.column1,table1.column2,table2.column2 as smth_so_it_doesnt_override
i'm having an issue with how eloquent is formulation a query that i have no access to. When doing something like
$model->where('something')
->distinct()
->paginate();
eloquent runs a query to get the total count, and the query looks something like
select count(*) as aggregate from .....
The problem is that if you use distinct in the query, you want something like
select count(distinct id) as aggregate from .....
to get the correct total. Eloquent is not doing that though, thus returning wrong totals. The only way to get the distinct in count is to pass an argument through the query builder like so ->count('id') in which case it will add it. Problem is that this query is auto-generated and i have no control over it.
Is there a way to trick it into adding the distinct on the count query?
P.S Digging deep into the builders code we find an IF statement asking for a field on the count() method in order to add the distinct property to the count. Illuminate\Database\Query\Grammars\BaseGrammar#compileAggregate
if ($query->distinct && $column !== '*')
{
$column = 'distinct '.$column;
}
return 'select '.$aggregate['function'].'('.$column.') as aggregate';
P.S.1 I know that in SQL you could do a group by, but since i'm eager loading stuff it is not a good idea cause it will add a IN (number of id's found) to each of the other queries which slows things down significantly.
I faced the exact same problem and found two solutions:
The bad one:
$results = $model->groupBy('foo.id')->paginate();
It works but it will costs too much memory (and time) if you have a high number of rows (it was my case).
The better one:
$ids = $model->distinct()->pluck('foo.id');
$results = $query = $model->whereIn('foo.id', $ids)->paginate();
I tried this with 100k results, and had no problem at all.
This seems to be a wider problem, discussed here:
https://github.com/laravel/framework/issues/3191
https://github.com/laravel/framework/pull/4088
Untill the fixes are shipped with one of the next Laravel releases, you can always try using the raw expressions, like below (I didnt test it, but should work)
$stuff = $model->select(DB::raw('distinct id as did'))
->where('whatever','=','whateverelse')
->paginate();
Reference: http://laravel.com/docs/queries#raw-expressions
$model->where('something')->distinct()->count('id')->paginate();