I have researched this to death. There does seem to be some answers here already, but frankly the answers confuse me. So I'll try to make this as basic as possible.
If I have a table in a database, "People" with a name column(string) and an age(int) column. Then I do something like this:
List<int> ages = new List<int>();
ages.Add(39);
ages.Add(40);
var result = from p in People
where ages.Contains((int)p.Age)
select p;
The syntax is passing, but returns nothing (both in VS2010 and LinqPad). Shouldn't this make a SQL statement with a where clause containing the ages in my list?
Simple answer anyone? (Ideally if you could modify my example and make it work.. that would be great!)
if you are doing it as Linq2SQL, it appears you are doing it correctly in order to ensure the proper projections for SQL Server. You may try writing it a slightly different way, like:
var result = from p in People
where ages.Select(a => a).Contains(p.Age)
select p;
You say that it returns nothing. Are you sure that there are matching records to return? Also, are you using result anywhere? The query won't execute if you don't bind it to something or call ToList() to some other projecting interaction.
Sample from comments:
var ints = new List<int> { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 };
var Merchandiser = (new DataClasses1DataContext()).Merchandisers;
var result = from m in Merchandiser
where ints.Contains(m.Id)
select m;
foreach (var item in result)
{
Console.WriteLine(item.Name);
}
Console.ReadLine();
Related
Currently when I need to run a query that will be used w/ paging I do it something like this:
//Setup query (Typically much more complex)
var q = ctx.People.Where(p=>p.Name.StartsWith("A"));
//Get total result count prior to sorting
int total = q.Count();
//Apply sort to query
q = q.OrderBy(p => p.Name);
q.Select(p => new PersonResult
{
Name = p.Name
}.Skip(skipRows).Take(pageSize).ToArray();
This works, but I wondered if it is possible to improve this to be more efficient while still using linq? I couldn't think of a way to combine the count w/ the data retrieval in a single trip to the DB w/o using a stored proc.
The following query will get the count and page results in one trip to the database, but if you check the SQL in LINQPad, you'll see that it's not very pretty. I can only imagine what it would look like for a more complex query.
var query = ctx.People.Where (p => p.Name.StartsWith("A"));
var page = query.OrderBy (p => p.Name)
.Select (p => new PersonResult { Name = p.Name } )
.Skip(skipRows).Take(pageSize)
.GroupBy (p => new { Total = query.Count() })
.First();
int total = page.Key.Total;
var people = page.Select(p => p);
For a simple query like this, you could probably use either method (2 trips to the database, or using GroupBy to do it in 1 trip) and not notice much difference. For anything complex, I think a stored procedure would be the best solution.
Jeff Ogata's answer can be optimized a little bit.
var results = query.OrderBy(p => p.Name)
.Select(p => new
{
Person = new PersonResult { Name = p.Name },
TotalCount = query.Count()
})
.Skip(skipRows).Take(pageSize)
.ToArray(); // query is executed once, here
var totalCount = results.First().TotalCount;
var people = results.Select(r => r.Person).ToArray();
This does pretty much the same thing except it won't bother the database with an unnecessary GROUP BY. When you are not certain your query will contain at least one result, and don't want it to ever throw an exception, you can get totalCount in the following (albeit less cleaner) way:
var totalCount = results.FirstOrDefault()?.TotalCount ?? query.Count();
Important Note for People using EF Core >= 1.1.x && < 3.0.0:
At the time I was looking for solution to this and this page is/was Rank 1 for the google term "EF Core Paging Total Count".
Having checked the SQL profiler I have found EF generates a SELECT COUNT(*) for every row that is returned. I have tired every solution provided on this page.
This was tested using EF Core 2.1.4 & SQL Server 2014. In the end I had to perform them as two separate queries like so. Which, for me at least, isn't the end of the world.
var query = _db.Foo.AsQueryable(); // Add Where Filters Here.
var resultsTask = query.OrderBy(p => p.ID).Skip(request.Offset).Take(request.Limit).ToArrayAsync();
var countTask = query.CountAsync();
await Task.WhenAll(resultsTask, countTask);
return new Result()
{
TotalCount = await countTask,
Data = await resultsTask,
Limit = request.Limit,
Offset = request.Offset
};
It looks like the EF Core team are aware of this:
https://github.com/aspnet/EntityFrameworkCore/issues/13739
https://github.com/aspnet/EntityFrameworkCore/issues/11186
I suggest making two queries for the first page, one for the total count and one for the first page or results.
Cache the total count for use as you move beyond the first page.
I've stumbled upon a very strange LINQ to SQL behaviour / bug, that I just can't understand.
Let's take the following tables as an example: Customers -> Orders -> Details.
Each table is a subtable of the previous table, with a regular Primary-Foreign key relationship (1 to many).
If I execute the follow query:
var q = from c in context.Customers
select (c.Orders.FirstOrDefault() ?? new Order()).Details.Count();
Then I get an exception: Could not format node 'Value' for execution as SQL.
But the following queries do not throw an exception:
var q = from c in context.Customers
select (c.Orders.FirstOrDefault() ?? new Order()).OrderDateTime;
var q = from c in context.Customers
select (new Order()).Details.Count();
If I change my primary query as follows, I don't get an exception:
var q = from r in context.Customers.ToList()
select (c.Orders.FirstOrDefault() ?? new Order()).Details.Count();
Now I could understand that the last query works, because of the following logic:
Since there is no mapping of "new Order()" to SQL (I'm guessing here), I need to work on a local list instead.
But what I can't understand is why do the other two queries work?!?
I could potentially accept working with the "local" version of context.Customers.ToList(), but how to speed up the query?
For instance in the last query example, I'm pretty sure that each select will cause a new SQL query to be executed to retrieve the Orders. Now I could avoid lazy loading by using DataLoadOptions, but then I would be retrieving thousands of Order rows for no reason what so ever (I only need the first row)...
If I could execute the entire query in one SQL statement as I would like (my first query example), then the SQL engine itself would be smart enough to only retrieve one Order row for each Customer...
Is there perhaps a way to rewrite my original query in such a way that it will work as intended and be executed in one swoop by the SQL server?
EDIT:
(longer answer for Arturo)
The queries I provided are purely for example purposes. I know they are pointless in their own right, I just wanted to show a simplistic example.
The reason your example works is because you have avoided using "new Order()" all together. If I slightly modify your query to still use it, then I still get an exception:
var results = from e in (from c in db.Customers
select new { c.CustomerID, FirstOrder = c.Orders.FirstOrDefault() })
select new { e.CustomerID, Count = (e.FirstOrder != null ? e.FirstOrder : new Order()).Details().Count() }
Although this time the exception is slightly different - Could not format node 'ClientQuery' for execution as SQL.
If I use the ?? syntax instead of (x ? y : z) in that query, I get the same exception as I originaly got.
In my real-life query I don't need Count(), I need to select a couple of properties from the last table (which in my previous examples would be Details). Essentially I need to merge values of all the rows in each table. Inorder to give a more hefty example I'll first have to restate my tabels:
Models -> ModelCategoryVariations <- CategoryVariations -> CategoryVariationItems -> ModelModuleCategoryVariationItemAmounts -> ModelModuleCategoryVariationItemAmountValueChanges
The -> sign represents a 1 -> many relationship. Do notice that there is one sign that is the other way round...
My real query would go something like this:
var q = from m in context.Models
from mcv in m.ModelCategoryVariations
... // select some more tables
select new
{
ModelId = m.Id,
ModelName = m.Name,
CategoryVariationName = mcv.CategoryVariation.Name,
..., // values from other tables
Categories = (from cvi in mcv.CategoryVariation.CategoryVariationItems
let mmcvia = cvi.ModelModuleCategoryVariationItemAmounts.SingleOrDefault(mmcvia2 => mmcvia2.ModelModuleId == m.ModelModuleId) ?? new ModelModuleCategoryVariationItemAmount()
select new
{
cvi.Id,
Amount = (mmcvia.ModelModuleCategoryVariationItemAmountValueChanges.FirstOrDefault() ?? new ModelModuleCategoryVariationItemAmountValueChange()).Amount
... // select some more properties
}
}
This query blows up at the line let mmcvia =.
If I recall correctly, by using let mmcvia = new ModelModuleCategoryVariationItemAmount(), the query would blow up at the next ?? operand, which is at Amount =.
If I start the query with from m in context.Models.ToList() then everything works...
Why are you looking into only the individual count without selecting anything related to the customer.
You can do the following.
var results = from e in
(from c in db.Customers
select new { c.CustomerID, FirstOrder = c.Orders.FirstOrDefault() })
select new { e.CustomerID, DetailCount = e.FirstOrder != null ? e.FirstOrder.Details.Count() : 0 };
EDIT:
OK, I think you are over complicating your query.
The problem is that you are using the new WhateverObject() in your query, T-SQL doesnt know anyting about that; T-SQL knows about records in your hard drive, your are throwing something that doesn't exist. Only C# knows about that. DON'T USE new IN YOUR QUERIES OTHER THAN IN THE OUTER MOST SELECT STATEMENT because that is what C# will receive, and C# knows about creating new instances of objects.
Of course is going to work if you use ToList() method, but performance is affected because now you have your application host and sql server working together to give you the results and it might take many calls to your database instead of one.
Try this instead:
Categories = (from cvi in mcv.CategoryVariation.CategoryVariationItems
let mmcvia =
cvi.ModelModuleCategoryVariationItemAmounts.SingleOrDefault(
mmcvia2 => mmcvia2.ModelModuleId == m.ModelModuleId)
select new
{
cvi.Id,
Amount = mmcvia != null ?
(mmcvia.ModelModuleCategoryVariationItemAmountValueChanges.Select(
x => x.Amount).FirstOrDefault() : 0
... // select some more properties
}
Using the Select() method allows you to get the first Amount or its default value. I used "0" as an example only, I dont know what is your default value for Amount.
I am following Phil Haack's example on using jQuery Grid with ASP.NET MVC. I have it working and it works well...except for one minor problem. When I sort the columns by something other than the ID, the JSON data returned from the server is very...well...wrong. Here's is my Controller method.
[HttpPost]
public ActionResult PeopleData(string sidx, string sord, int page, int rows)
{
int pageIndex = Convert.ToInt32(page) - 1;
int pageSize = rows;
int totalRecords = repository.FindAllPeople().Count();
int totalPages = (int)Math.Ceiling((float)totalRecords / (float)pageSize);
var people = repository.FindAllPeople()
.OrderBy(sidx + " " + sord)
.Skip(pageIndex * pageSize)
.Take(pageSize);
var jsonData = new
{
total = totalPages,
page = page,
records = totalRecords,
rows = (
from person in people
select new
{
i = person.PersonID,
cell = new List<string> { SqlFunctions.StringConvert((double) person.PersonID), person.PersonName }
}
).ToArray()
};
return Json(jsonData);
}
When I sort by PersonID in the jsGrid table, I get this data back (I just used the name of the current ID as the name - e.g. 1, One; 2, Two, etc.)
{"total":1,"page":1,"records":6,"rows":[{"i":1,"cell":[" 1","One"]},{"i":2,"cell":[" 2","Two"]},{"i":3,"cell":[" 3","Three"]},{"i":4,"cell":[" 4","Four"]},{"i":5,"cell":[" 5","Five"]},{"i":6,"cell":[" 6","Six"]}]}
When I sort by PersonName, however, every other row has the order (the ID vs. the name) flipped around. So when I show it in the table, the PersonName is in the ID column and the ID is in the person column. Here is the JSON result.
{"total":1,"page":1,"records":6,"rows":[{"i":5,"cell":[" 5","Five"]},{"i":4,"cell":["Four"," 4"]},{"i":1,"cell":[" 1","One"]},{"i":6,"cell":["Six"," 6"]},{"i":3,"cell":[" 3","Three"]},{"i":2,"cell":["Two"," 2"]}]}
Anybody have any insight into what I've done wrong that causes this to happen?
Update
So, I have learned that, what is happening, is that my array values are flipping for every other item in the array. For example...if I populate my database with:
[A, B, C]
then for every even-numbered result (or odd, if you're counting from 0), my data is coming back:
[C, B, A]
So, ultimately, my JSON row data is something like:
[A, B, C]
[C, B, A]
[A, B, C]
[C, B, A]
...etc
This is always happening and always consistent. I am going a bit crazy trying to figure out what's going on because it seems like it should be something simple.
I have the same problem with my data which are INT type.
If elements in my queue (A,B,C) are NVARCHAR type I do not have this problem.
So problem is obviously in SqlFunction.StringConvert function.
Try to use the method described here. If you use fields instead of properties in the repository.FindAllPeople() you should look at the commented part of the code where are used FieldInfo and GetField instead of PropertyInfo and GetProperty.
I found the solution here: linq to entities orderby strange issue
The issue ultimately stems from the fact that Linq to Entities has trouble handling strings. When I was using the SqlFunctions.StringConvert method, this was incorrectly performing the conversion (although, I must admit that I don't fully understand why the order was then switched around).
In either case, per the above post, the solution for fixing the problem was to do the selection locally so that I could "force" Linq to Entities to work with strings properly. From this, my final code is:
var people = repository.FindAllPeople()
.OrderBy(sidx + " " + sord)
.Skip(pageIndex * pageSize)
.Take(pageSize);
// Due to a problem with Linq to Entities working with strings,
// all string work has to be done locally.
var local = people.AsEnumerable();
var rowData = local.Select(person => new
{
id = person.PersonID,
cell = new List<string> {
person.PersonID.ToString(),
person.PersonName
}
}
).ToArray();
var jsonData = new
{
total = totalPages,
page = page,
records = totalRecords,
rows = rowData
};
return Json(jsonData);
I'm struggling to get what I thought would be a simple LINQ-to-SQL query to work. I can construct the query ok and can verify that the SQL it generates is correct but when executing get the dreaded "System.NotSupportedException: Queries with local collections are not supported" exception.
I've simplified my query but in short the query below works:
var query = from asset in context where new[] { 1, 2, 3 }.Contains(asset.code) select asset
this query also works:
var query = from asset in context where new List<int>() { 1, 2, 3 }.Contains(asset.code) select asset
But the query below will fail when an attempt is made to get the result set:
List<int> myList = new List<int>(){1, 2, 3};
var query = from asset in context where myList.Contains(asset.code) select asset
Anyone solved this sort of issue?
What you posted should work, leading me to believe you didn't actually post the broken code.
Make sure that the MyList variable is a List<int> and not an IList<int>... if the variable type is IList<int>, you'd get that exception.
This works fine for me in LINQPad:
List<int> myList = new List<int>(){1, 2, 3}; /* Fixed your compiler error here */
var query = from asset in assets where myList.Contains(asset.code) select asset;
As such, I assume you aren't actually using a List for myList, but rather a generic IEnumerable or something similar?
Please try pasting your "unsimplified" version, as your simplication corrects your error.
I am trying to get the records from the 'many' table of a one-to-many relationship and add them as a list to the relevant record from the 'one' table.
I am also trying to do this in a single database request.
Code derived from Linq to Sql - Populate JOIN result into a List almost achieves the intended result, but makes one database request per entry in the 'one' table which is unacceptable. That failing code is here:
var res = from variable in _dc.GetTable<VARIABLE>()
select new { x = variable, y = variable.VARIABLE_VALUEs };
However if I do a similar query but loop through all the results, then only a single database request is made. This code achieves all goals:
var res = from variable in _dc.GetTable<VARIABLE>()
select variable;
List<GDO.Variable> output = new List<GDO.Variable>();
foreach (var v2 in res)
{
List<GDO.VariableValue> values = new List<GDO.VariableValue>();
foreach (var vv in v2.VARIABLE_VALUEs)
{
values.Add(VariableValue.EntityToGDO(vv));
}
output.Add(EntityToGDO(v2));
output[output.Count - 1].VariableValues = values;
}
However the latter code is ugly as hell, and it really feels like something that should be do-able in a single linq query.
So, how can this be done in a single linq query that makes only a single database query?
In both cases the table is set to preload using the following code:
_dc = _db.CreateLinqDataContext();
var loadOptions = new DataLoadOptions();
loadOptions.LoadWith<VARIABLE>(v => v.VARIABLE_VALUEs);
_dc.LoadOptions = loadOptions;
I am using .NET 3.5, and the database back-end was generated using SqlMetal.
This link may help
http://msdn.microsoft.com/en-us/vcsharp/aa336746.aspx
Look under join operators. You'll probably have to change from using extension syntax other syntax too. Like this,
var = from obj in dc.Table
from obj2 in dc.Table2
where condition
select