I have following query against EF whereby mysql was used:
var query = from r in context.myContext
where r.clmn1.CompareTo("2015-11-19 00:00:00") > 0)
orderby r.someColumn
select r;
return query;
The number of returned rows is as expected. however some values of the property r.clmn2 repeat itself in the result of the query. For example I could not find clmn2 == 220011 because it was "overwritten" by the value 220033 (The value 220033 is correct and expected but should not "overwrite" other values). Strangely enough, when I add this condition to the query I get it in the result (of course then only and only this value) which means that the first condition is also valid for clmn2:
var query = from r in context.myContext
where r.clmn1.CompareTo("2015-11-19 00:00:00") > 0) && r.clmn2.Equals("220011")
orderby r.someColumn
select r;
return query;
The same query (the first one) works at DB-level and returns all values (will not be overwritten)
SELECT * FROM myContext.myTable
WHERE r.clmn1 > ("2015-11-19 00:00:00")
ORDER BY r.someColumn
It should be a problem of EF. I hope someone could help me!
Thanks in Advance.
I have prefixed the column/property clmn2 with [key] atribute in the generated entity class so that it is now a part of the multiple key, i.e., with other columns/properties. It works and i get all values from DB. Maybe cus this property comes from a DB-view, Visual Studio could not recognize it as a primary key as done by other properties.
Related
I've been stuck on a GORM issue for about a full day now. I need to be able to filter a messages table on any of 4 things: sender, recipient, keyword, and date range. It also has to paginate. Filtering by sender and recipient is working, and so is pagination. So far this is the query that I have come up with, but it does not seem to work for date ranges or keywords.
Here is how I am selecting from MySQL
db.Preload("Thread").Where(query).Scopes(Paginate(r)).Find(&threadMessages)
I am creating the query like this:
var query map[string]interface{}
Then based on which parameters I am passed, I update the query like this by adding new key values to the map:
query = map[string]interface{}{"user_id": sender, "recipient_id": recipient}
For dates it does not seem to work if I try something like this:
query = map[string]interface{}{"created_at > ?": fromDate}
And for a LIKE condition is also does not seem to work:
query = map[string]interface{}{"contents LIKE ?": keyword}
The reason I chose this approach is that I could not seem to get optional inputs to work in .Where since it takes a string with positional parameters and null positional parameters seem to cause MySQL to return an empty array. Has anyone else dealt with a complicated GORM issue like this? Any help is appreciated at this point.
Passing the map[string]interface{} into Where() only appears to work for Equals operations, or IN operations (if a slice is provided as the value instead).
One way to achieve what you want, is to construct a slice of clause.Expression, and append clauses to the slice when you need to. Then, you can simply pass in all of the clauses (using the ... operator to pass in the whole slice) into db.Clauses().
clauses := make([]clause.Expression, 0)
if mustFilterCreatedAt {
clauses = append(clauses, clause.Gt{Column: "created_at", fromDate})
}
if mustFilterContents {
clauses = append(clauses, clause.Like{Column: "contents", Value: keyword})
}
db.Preload("Thread").Clauses(clauses...).Scopes(Paginate(r)).Find(&threadMessages)
Note: If you're trying to search for content that contains keyword, then you should concatenate the wildcard % onto the ends of keyword, otherwise LIKE behaves essentially the same as =:
clause.Like{Column: "contents", Value: "%" + keyword + "%"}
My final solution to this was to create dynamic Where clauses based on which query params were sent from the client like this:
fields := []string{""}
values := []interface{}{}
If, for example, there is a keyword param:
fields = []string{"thread_messages.contents LIKE ?"}
values = []interface{}{"%" + keyword + "%"}
And to use the dynamic clauses in the below query:
db.Preload("Thread", "agency_id = ?", agencyID).Preload("Thread.ThreadUsers", "agency_id = ?", agencyID).Joins("JOIN threads on thread_messages.thread_id = threads.id").Where("threads.agency_id = ?", agencyID).Where(strings.Join(fields, " AND "), values...).Scopes(PaginateMessages(r)).Find(&threadMessages)
I am pretty new to hibernate again, so this might be a noobish question ;).
Without to_days, but clustered by timestamp it works like this:
CriteriaQuery<Tuple> query = criteriaBuilder.createQuery(Tuple.class);
Root<Session> sessionRoot = query.from(Session.class);
query.multiselect(
sessionRoot.get("time").alias("time"),
criteriaBuilder.count(sessionRoot).alias("count")
);
query.groupBy(sessionRoot.get("time"));
List<Tuple> results = this.executeQuery(query);
So I recieve:
time|count
13721938721|1
13721938722|2
13721938723|3
13721938724|4
13721938725|2
13721938726|1
13721938727|4
But this are all sessioncounts for each millisecond, but I need those clustered by day and not by timestamp: thus I use to_days in plain mysql.
In mysql I perform this query:
SELECT TO_DAYS(`time`) AS `days`, COUNT(*) as `count` FROM sessions WHERE 1 GROUP BY `days`
This gives me:
days|count
777594|123
777595|60
777596|61
777597|74
But I have no idea, yet: how to achieve the same thing with javax.persistence.criteria.CriteriaBuilder and CriteriaQuery in hibernate?
I dont know how to do it with criteriaBuilder, but i do know how in Hibernate 4 criteria api:
query.setProjection(
Projections.sqlProjection(
"TO_DAYS(time) as days",
new String[]{"days"},
new Type[]{StandardBasicTypes.INTEGER}
)
);
sqlProjection allows you to cast or convert data types, but careful, using a projection will only retrieve the fileds you specify in it, and the resulting list will come up like this:
List<Object[]> results = this.executeQuery(query);
But you can make hibernate do a alias match with the properties using a result transformer:
query.setResultTransformer(new AliasToBeanResultTransformer(Session.class));
and the list comes out like it normally does:
List<Session.class> results = this.executeQuery(query);
Sorry i could not provide a criteriaBuilder solution, but i hope this gets you in the right track.
After some investigation, it turned out, that HQL does not support TO_DAYS. Since I want to make it possible for MySQL and other databases, this is my final solution:
Query q = entityManager.createQuery("SELECT concat(day(e.time), '-', month(e.time), '-', year(e.time)) AS days, COUNT(*) FROM Event e GROUP BY concat(day(e.time), '-', month(e.time), '-', year(e.time))");
The result is:
3-5-2012|980
4-5-2012|200
10-6-2012|123
12-6-2012|144
13-11-2012|500
So afterwards I convert all ugly date strings into proper milliseconds in java and have the data, which I need.
I have a table with rows containing the following numbers:
4191808.51
3035280.22
3437796.06
4013772.33
1740652.56
0
The sum of that table is 16419309.68.
When I do a Linq SUM query on that table, it returns a whole different number: 19876858.14
The code I use to do the sum is as follows:
IEnumerable<MyData> data = _myRepository.GetMatching(myValue);
decimal sumSales = data.Sum(x => x.Sales);
What could be causing this? I suspect some max decimal value but couldn't find info on that
can you please examine your
IEnumerable<MyData> data = _myRepository.GetMatching(myValue);
in the debugger after the execution. I'm suspecting that you are selecting some different set of data - not what you have shown in the sample. I attempted to recreated you situation LinqPad, but constantly getting the correct answer.
decimal[] data = {
4191808.51m
,3035280.22m
,3437796.06m
,4013772.33m
,1740652.56m
,0m
};
decimal sumSales = data.Sum();
sumSales.Dump();
and getting: 16419309.68
I've got a membership table that records whether a user is a member of a list. When an update to a user's membership occurs a new record is written and the previous records are left as is which allows a history of their memberships to be maintained. To get a user's membership status involves selecting their most recent entry.
An example of some user list membership data is below. The aim is to find a LINQ expression that groups by list and user but only returns the row with most recently inserted record.
List Name, Username, Comment, ExpiresOn, Inserted
Test List, joeb, second update, 2012-03-13 16:55:03, 2012-01-31 22:28:40
Test List, joeb, first update, 2012-02-13 16:55:01, 2012-01-31 22:28:39
Test List, joeb, initial, 2012-01-13 16:55:02, 2012-01-31 22:28:38
An SQL query illustrates how the current list membership status can be extracted.
select ulm2.ID, ulm2.ExpiresOn, ulm2.Comment, ulm2.Inserted
from UserListMembership as ulm1
left outer join UserListMembership ulm2 on ulm1.id = ulm2.id
group by ulm1.userlistid, ulm1.userid;
The question is how to write a LINQ expression for the query that doesn't use a nested FirstOrDefault call which causes my MySQL entity framework provider to throw a "System.NotSupportedException: Specified method is not supported." exception?
Update:
Below is my failing LINQ expression, it throws the "Specified method is not supported" once the FirstOrDefault call is added.
var query = from mem in db.UserListMemberships
where
mem.User.UserUsernames.Any(y => y.Username.ToLower() == username.ToLower())
&& mem.UserList.Account.Subscriptions.Any(x => x.ID == subscriptionID)
&& mem.ExpiresOn > utcNow
group mem by new { mem.UserListID, mem.UserID } into g
select new { UserListMembership = (from mem2 in db.UserListMemberships where mem2.UserListID == g.Key.UserListID && mem2.UserID == g.Key.UserID orderby mem2.Inserted descending select mem2).FirstOrDefault() };
return query.Select(a => a.UserListMembership).ToList();
Without posting code it's hard for anyone to see what your issue is.
Are you not able to just order the results by your date field and then select the first record? Something like users.OrderByDescending(u => u.Inserted).FirstOrDefault()?
I've stumbled upon a very strange LINQ to SQL behaviour / bug, that I just can't understand.
Let's take the following tables as an example: Customers -> Orders -> Details.
Each table is a subtable of the previous table, with a regular Primary-Foreign key relationship (1 to many).
If I execute the follow query:
var q = from c in context.Customers
select (c.Orders.FirstOrDefault() ?? new Order()).Details.Count();
Then I get an exception: Could not format node 'Value' for execution as SQL.
But the following queries do not throw an exception:
var q = from c in context.Customers
select (c.Orders.FirstOrDefault() ?? new Order()).OrderDateTime;
var q = from c in context.Customers
select (new Order()).Details.Count();
If I change my primary query as follows, I don't get an exception:
var q = from r in context.Customers.ToList()
select (c.Orders.FirstOrDefault() ?? new Order()).Details.Count();
Now I could understand that the last query works, because of the following logic:
Since there is no mapping of "new Order()" to SQL (I'm guessing here), I need to work on a local list instead.
But what I can't understand is why do the other two queries work?!?
I could potentially accept working with the "local" version of context.Customers.ToList(), but how to speed up the query?
For instance in the last query example, I'm pretty sure that each select will cause a new SQL query to be executed to retrieve the Orders. Now I could avoid lazy loading by using DataLoadOptions, but then I would be retrieving thousands of Order rows for no reason what so ever (I only need the first row)...
If I could execute the entire query in one SQL statement as I would like (my first query example), then the SQL engine itself would be smart enough to only retrieve one Order row for each Customer...
Is there perhaps a way to rewrite my original query in such a way that it will work as intended and be executed in one swoop by the SQL server?
EDIT:
(longer answer for Arturo)
The queries I provided are purely for example purposes. I know they are pointless in their own right, I just wanted to show a simplistic example.
The reason your example works is because you have avoided using "new Order()" all together. If I slightly modify your query to still use it, then I still get an exception:
var results = from e in (from c in db.Customers
select new { c.CustomerID, FirstOrder = c.Orders.FirstOrDefault() })
select new { e.CustomerID, Count = (e.FirstOrder != null ? e.FirstOrder : new Order()).Details().Count() }
Although this time the exception is slightly different - Could not format node 'ClientQuery' for execution as SQL.
If I use the ?? syntax instead of (x ? y : z) in that query, I get the same exception as I originaly got.
In my real-life query I don't need Count(), I need to select a couple of properties from the last table (which in my previous examples would be Details). Essentially I need to merge values of all the rows in each table. Inorder to give a more hefty example I'll first have to restate my tabels:
Models -> ModelCategoryVariations <- CategoryVariations -> CategoryVariationItems -> ModelModuleCategoryVariationItemAmounts -> ModelModuleCategoryVariationItemAmountValueChanges
The -> sign represents a 1 -> many relationship. Do notice that there is one sign that is the other way round...
My real query would go something like this:
var q = from m in context.Models
from mcv in m.ModelCategoryVariations
... // select some more tables
select new
{
ModelId = m.Id,
ModelName = m.Name,
CategoryVariationName = mcv.CategoryVariation.Name,
..., // values from other tables
Categories = (from cvi in mcv.CategoryVariation.CategoryVariationItems
let mmcvia = cvi.ModelModuleCategoryVariationItemAmounts.SingleOrDefault(mmcvia2 => mmcvia2.ModelModuleId == m.ModelModuleId) ?? new ModelModuleCategoryVariationItemAmount()
select new
{
cvi.Id,
Amount = (mmcvia.ModelModuleCategoryVariationItemAmountValueChanges.FirstOrDefault() ?? new ModelModuleCategoryVariationItemAmountValueChange()).Amount
... // select some more properties
}
}
This query blows up at the line let mmcvia =.
If I recall correctly, by using let mmcvia = new ModelModuleCategoryVariationItemAmount(), the query would blow up at the next ?? operand, which is at Amount =.
If I start the query with from m in context.Models.ToList() then everything works...
Why are you looking into only the individual count without selecting anything related to the customer.
You can do the following.
var results = from e in
(from c in db.Customers
select new { c.CustomerID, FirstOrder = c.Orders.FirstOrDefault() })
select new { e.CustomerID, DetailCount = e.FirstOrder != null ? e.FirstOrder.Details.Count() : 0 };
EDIT:
OK, I think you are over complicating your query.
The problem is that you are using the new WhateverObject() in your query, T-SQL doesnt know anyting about that; T-SQL knows about records in your hard drive, your are throwing something that doesn't exist. Only C# knows about that. DON'T USE new IN YOUR QUERIES OTHER THAN IN THE OUTER MOST SELECT STATEMENT because that is what C# will receive, and C# knows about creating new instances of objects.
Of course is going to work if you use ToList() method, but performance is affected because now you have your application host and sql server working together to give you the results and it might take many calls to your database instead of one.
Try this instead:
Categories = (from cvi in mcv.CategoryVariation.CategoryVariationItems
let mmcvia =
cvi.ModelModuleCategoryVariationItemAmounts.SingleOrDefault(
mmcvia2 => mmcvia2.ModelModuleId == m.ModelModuleId)
select new
{
cvi.Id,
Amount = mmcvia != null ?
(mmcvia.ModelModuleCategoryVariationItemAmountValueChanges.Select(
x => x.Amount).FirstOrDefault() : 0
... // select some more properties
}
Using the Select() method allows you to get the first Amount or its default value. I used "0" as an example only, I dont know what is your default value for Amount.