Is there a way to speed up the performance of this query.
I have indexes on tswProjectID and tswWeekEdning.
This SQL was generated from my Linq statement which is
what I want to use in my C# code.
Is there a more efficient way to write this?
var qry = (from tsw in TimesheetWeeklies where tsw.TswProjectID == 8263 select tsw).OrderByDescending(x => x.TswWeekEnding).FirstOrDefault();
SELECT TOP (1) [t0].[tswID] AS [TswID]
FROM [TimesheetWeekly] AS [t0]
WHERE [t0].[tswProjectID] = 8263
ORDER BY [t0].[tswWeekEnding] DESC
Try create an index with both columns in it (tswProjectID, tswWeekEnding)
It wont make the query any faster but if you make it a compiled query you could possibly save some time that it takes to build the query if it's done more than once, more info here:
http://msdn.microsoft.com/en-us/library/bb399335.aspx
Related
I am very new to mysql . I had a query, from this query i am getting some stock_ids.In the second query i need to pass those stock_ids. For this i am fetching all the stock_ids in a array.Now my question is in which way can i pass those stock_ids to the second query. For this i have two approaches.
First approach:
$array_cnt = count($stockid_array);
for($i = 0; $i<$array_cnt;$i++)
{
$sql = "select reciever_id,sender_identifier,unique_stock_id,vat_number,tax_number from stocktable where stock_id = '".$stockid_array[$i]."'";
// my while loop
}
Another approach is
$sql = "reciever_id,sender_identifier,unique_stock_id,vat_number,tax_number from stocktable where stock_id in ('1','2','3','4','5','6','7','8','9');
//while loop comes here.
So which approach gives good performance ? Please guide me.
MySQL has a nice optimization for constants in an in list -- it basically sorts the values and uses a binary search. That means that the in is going to be faster. Also, in can take advantage of an index on the column.
In addition, running a single query should be faster than running multiple queries.
So, the in version should be better than running multiple queries with =.
I am new in Code Igniter and like its active record feature now is there any useful steps or tips or any guidness how do i convert my pervoiusly written simple SQL Queries in CI style like this is my perviouly written simple query
SELECT *
FROM hs_albums
WHERE id NOT IN (SELECT album_id
FROM hs_delete_albums
WHERE user_id = 72
AND del_type = 1)
AND ( created = 72
OR club_id IN (SELECT cbs.id
FROM hs_clubs cbs
INNER JOIN hs_club_permissions cbp
ON cbs.id = cbp.club_id
WHERE cbp.user_id = 72
AND cbp.status = 2)
OR group_id IN (SELECT gps.id
FROM hs_groups gps
INNER JOIN hs_group_permissions grp
ON gps.id = grp.group_id
WHERE grp.user_id = 72
AND grp.status = 2)
OR comp_id IN (SELECT cmp.id
FROM hs_companies cmp
INNER JOIN hs_comp_permissions comp
ON cmp.id = comp.comp_id
WHERE comp.user_id = 72
AND comp.status = 2) )
The short answer is: You don't.
CodeIgniter's Active Record implementation is basically a layer on top of SQL that makes writing queries easier by:
Automatically escaping values
Automatically generating the appropriate query syntax for the database, so that the application can be more easily ported between databases (for instance, if you didn't use Active Record to write a query, and then wanted to move from MySQL to PostgreSQL, then you might well need to rewrite the query to make it work with PostgreSQL)
Providing a syntax for queries in PHP directly, thus avoiding the context switching between PHP and SQL.
However, it can't do everything SQL can do, and while I would always try to use ActiveRecord where possible, there comes a point where you're better off forgetting about using it and just using $this->db->query() to write your query directly. In this case, as mamdouh alramadan has said, CodeIgniter doesn't support subqueries so you can't replicate this query using ActiveRecord anyway.
The thing to remember is that ActiveRecord is a tool, not a requirement. If you're using CodeIgniter and aren't using an ORM instead, you should use it for the reasons mentioned above. However, once it starts getting in the way, you should consider whether it would be better practice to write your query manually instead.
So basicly the problem is in query SELECT COUNT(*) which executed in calculateTotalItemCount function in activedataprovider. As i understood it needed for pagination for $itemcount variable. The problem is this query slow for big tables. For my ~30m table it executes 5 seconds.
So there are 2 ways to solve this problem:
1. Disable pagination ('pagination'=>'false') and write own pagination.
2. Rewrite AR count function.
I dont have enough experience/knowledge to acomplish this.
Maybe some one had same issues before and can share his solution.
Atleast for totalItemCount we can use EXPLAIN SELECT *. Its way more faster.
I appreciate any help. Thank you.
If you have a "cheaper" query in raw SQL than the one that active records create automatically, you can also query manually (e.g. through DAO) and set the totalItemCount on your data provider:
$count = Yii::app()->db->createCommand('SELECT COUNT(*)...')->queryScalar();
$provider = new CActiveDataProvider('SomeModel', array(
'totalItemCount' => $count,
'criteria' => $criteria,
...
I don't see how this is possible, but I really, really hate to run my query an extra time just to get the record count so I can build a pager. When I say a "pager" I simply mean the common gizmo with a link for each 10 records for example.
Assuming you are building a query in the selecting event, the best you could do is construct the full query, get and save the count, then take or skip it into the e.result.
And by best I mean, the easiest read code from a single query, rather than two. You'll still be running two separate evaluations on the database though. Use query analyser to see if the statements are a 'Select Count' then a 'Select take' or a dirty big select pared down by LINQ after the retrieve. I think LINQ does the former.
As far as I know it is not possible to return the total count and the items retrieved by skip and take at the same time.
I wrote a custom data source control and view, which caches the count for a short duration. I invalidate the cache whenever the criteria changes that would affect the number of results, but not when the data is paged, or when the data is sorted for instance.
I was concerned about this same question. Here is are the results of my experimenting in Linqpad--the actual behavior of Linq does not create a full new query to the SQL server:
This simple test, in one of my development databases:
var query = from p in HrPersons select p;
var x = query.Skip(20).Take(10).Dump();
var t = query.Count().Dump();
generates the following actual SQL queries:
SELECT [t1].[company] AS [Company], [t1].[processLevel] AS [ProcessLevel], [t1].[emplId] AS [EmplId], [t1].[sn] AS [Sn], [t1].[givenName] AS [GivenName], [t1].[middleInitial] AS [MiddleInitial], [t1].[nickName] AS [NickName], [t1].[formerName] AS [FormerName], [t1].[ssn] AS [Ssn], [t1].[cn] AS [Cn], [t1].[costCenter] AS [CostCenter], [t1].[title] AS [Title], [t1].[status] AS [Status], [t1].[batch] AS [Batch], [t1].[rowversion] AS [Rowversion], [t1].[id] AS [Id], [t1].[source] AS [Source]
FROM (
SELECT ROW_NUMBER() OVER (ORDER BY [t0].[company], [t0].[processLevel], [t0].[emplId], [t0].[sn], [t0].[givenName], [t0].[middleInitial], [t0].[nickName], [t0].[formerName], [t0].[ssn], [t0].[cn], [t0].[costCenter], [t0].[title], [t0].[status], [t0].[batch], [t0].[rowversion], [t0].[id], [t0].[source]) AS [ROW_NUMBER], [t0].[company], [t0].[processLevel], [t0].[emplId], [t0].[sn], [t0].[givenName], [t0].[middleInitial], [t0].[nickName], [t0].[formerName], [t0].[ssn], [t0].[cn], [t0].[costCenter], [t0].[title], [t0].[status], [t0].[batch], [t0].[rowversion], [t0].[id], [t0].[source]
FROM [HrPerson] AS [t0]
) AS [t1]
WHERE [t1].[ROW_NUMBER] BETWEEN #p0 + 1 AND #p0 + #p1
ORDER BY [t1].[ROW_NUMBER]
GO
SELECT COUNT(*) AS [value]
FROM [HrPerson] AS [t0]
So while there is a second SQL query, it is a trivial one that only requests the total count. I believe this is reasonable and acceptable as a pattern.
I'm using LINQ to Entities to display paged results. But I'm having issues with the combination of Skip(), Take() and OrderBy() calls.
Everything works fine, except that OrderBy() is assigned too late. It's executed after result set has been cut down by Skip() and Take().
So each page of results has items in order. But ordering is done on a page handful of data instead of ordering of the whole set and then limiting those records with Skip() and Take().
How do I set precedence with these statements?
My example (simplified)
var query = ctx.EntitySet.Where(/* filter */).OrderByDescending(e => e.ChangedDate);
int total = query.Count();
var result = query.Skip(n).Take(x).ToList();
One possible (but a bad) solution
One possible solution would be to apply clustered index to order by column, but this column changes frequently, which would slow database performance on inserts and updates. And I really don't want to do that.
EDIT
I ran ToTraceString() on my query where we can actually see when order by is applied to the result set. Unfortunately at the end. :(
SELECT
-- columns
FROM (SELECT
-- columns
FROM (SELECT -- columns
FROM ( SELECT
-- columns
FROM table1 AS Extent1
WHERE EXISTS (SELECT
-- single constant column
FROM table2 AS Extent2
WHERE (Extent1.ID = Extent2.ID) AND (Extent2.userId = :p__linq__4)
)
) AS Project2
limit 0,10 ) AS Limit1
LEFT OUTER JOIN (SELECT
-- columns
FROM table2 AS Extent3 ) AS Project3 ON Limit1.ID = Project3.ID
UNION ALL
SELECT
-- columns
FROM (SELECT -- columns
FROM ( SELECT
-- columns
FROM table1 AS Extent4
WHERE EXISTS (SELECT
-- single constant column
FROM table2 AS Extent5
WHERE (Extent4.ID = Extent5.ID) AND (Extent5.userId = :p__linq__4)
)
) AS Project6
limit 0,10 ) AS Limit2
INNER JOIN table3 AS Extent6 ON Limit2.ID = Extent6.ID) AS UnionAll1
ORDER BY UnionAll1.ChangedDate DESC, UnionAll1.ID ASC, UnionAll1.C1 ASC
My workaround solution
I've managed to workaround this problem. Don't get me wrong here. I haven't solved precedence issue as of yet, but I've mitigated it.
What I did?
This is the code I've used until I get an answer from Devart. If they won't be able to overcome this issue I'll have to use this code in the end.
// get ordered list of IDs
List<int> ids = ctx.MyEntitySet
.Include(/* Related entity set that is needed in where clause */)
.Where(/* filter */)
.OrderByDescending(e => e.ChangedDate)
.Select(e => e.Id)
.ToList();
// get total count
int total = ids.Count;
if (total > 0)
{
// get a single page of results
List<MyEntity> result = ctx.MyEntitySet
.Include(/* related entity set (as described above) */)
.Include(/* additional entity set that's neede in end results */)
.Where(string.Format("it.Id in {{{0}}}", string.Join(",", ids.ConvertAll(id => id.ToString()).Skip(pageSize * currentPageIndex).Take(pageSize).ToArray())))
.OrderByDescending(e => e.ChangedOn)
.ToList();
}
First of all I'm getting ordered IDs of my entities. Getting only IDs is well performant even with larger set of data. MySql query is quite simple and performs really well. In the second part I partition these IDs and use them to get actual entity instances.
Thinking of it, this should perform even better than the way I was doing it at the beginning (as described in my question), because getting total count is much much quicker due to simplified query. The second part is practically very very similar, except that my entities are returned rather by their IDs instead of partitioned using Skip and Take...
Hopefully someone may find this solution helpful.
I haven't worked directly with Linq to Entities, but it should have a way to hook specific stored procedures into certain locations when needed. (Linq to SQL did.) If so, you could turn this query into a stored procedure, doing exacly what is required, and doing it efficiently.
Assuming from you comment the persisting the values in a List is not acceptable:
There's no way to completely minimize the iterations, as you intended (and as I would have tried too, living in hope). Cutting the iterations down by one would be nice. Is it possible to just get the Count once and cache/session it? Then you could:
int total = ctx.EntitySet.Count; // Hopefully you can not repeat doing this.
var result = ctx.EntitySet.Where(/* filter */).OrderBy(/* expression */).Skip(n).Take(x).ToList();
Hopefully you can cache the Count somehow, or avoid needing it every time. Even if you can't, this is the best you can do.
Could you please create a sample illusrating the problem and send it to us (support * devart * com, subject "EF: Skip, Take, OrderBy")?
Hope we will be able to help you.
You can also contact us using our forums or contact form.
Are you absolutely certain the ordering is off? What does the SQL look like?
Can you reorder your code as follows and post the output?
// Redefine your queries.
var query = ctx.EntitySet.Where(/* filter */).OrderBy(e => e.ChangedDate);
var skipped = query.Skip(n).Take(x);
// let's look at the SQL, shall we?
var querySQL = query.ToTraceString();
var skippedSQL = skipped.ToTraceString();
// actual execution of the queries...
int total = query.Count();
var result = skipped.ToList();
Edit:
I'm absolutely certain. You can check my "edit" to see trace result of my query with skipped trace result that is imperative in this case. Count is not really important.
Yeah, I see it. Wow, that's a stumper. Might even be an outright bug. I note you're not using SQL Server... what DB are you using? Looks like it might be MySQl.
One way:
var query = ctx.EntitySet.Where(/* filter */).OrderBy(/* expression */).ToList();
int total = query.Count;
var result = query.Skip(n).Take(x).ToList();
Convert it to a List before skipping. It's not too efficient, mind you...