LINQ join and group - linq-to-sql

How to expand this query:
public Dictionary<int, List<TasksInDeal>> FindAllCreatedTasks()
{
return (from taskInDeal in db.TasksInDeals
where taskInDeal.Date > DateTime.Now && taskInDeal.Date < DateTime.Now.AddDays(7)
group taskInDeal by taskInDeal.CreatedByUserID
into groupedDemoClasses
select groupedDemoClasses).ToDictionary(gdc => gdc.Key, gdc => gdc.ToList());
}
into something like this:
public Dictionary<int, List<TaskForNotification>> FindAllCreatedTasks()
{
return (from taskInDeal in db.TasksInDeals
join user in db.Users on taskInDeal.CreatedByUserID equals user.UserID
where taskInDeal.Date > DateTime.Now && taskInDeal.Date < DateTime.Now.AddDays(7)
group taskInDeal by taskInDeal.CreatedByUserID
into groupedDemoClasses
select new TaskForNotification
{
Email = user.Email,
TaskInDealField1 = taskInDeal.TaskInDealField1,
TaskInDealField2 = taskInDeal.TaskInDealField2,
TaskInDealField3 = taskInDeal.TaskInDealField3,
...
}
).ToDictionary(gdc => gdc.Key, gdc => gdc.ToList());
}
So, to first query I need to join email from other table.

// do the date logic up front, not in the database.
DateTime now = DateTime.Now
DateTime weekFromNow = now.AddDays(7);
// pull the joined rows out of the database.
var rows =
(
from taskInDeal in db.TasksInDeals
where taskInDeal.Date > now && taskInDeal.Date < weekFromNow
join user in db.Users
on taskInDeal.CreatedByUserID equals user.UserID
select new {TaskInDeal = taskInDeal, UserEmail = user.Email}
).ToList();
// shape the rows in memory
Dictionary<int, List<TaskForNotification>> result =
(
from row in rows
let taskForNotification = new TaskForNotification
{
Email = row.UserEmail,
TaskInDealField1 = row.TaskInDeal.TaskInDealField1,
TaskInDealField2 = row.TaskInDeal.TaskInDealField2,
TaskInDealField3 = row.TaskInDeal.TaskInDealField3,
...
}
group taskForNotification by row.TaskInDeal.CreatedByUserID
// without an "into", group by ends the query.
).ToDictionary(g => g.Key, g => g.ToList());
When you group, bear this in mind. Groups in SQL have only keys and aggregates. Groups in LINQ have keys, aggregates and elements! If you ask the database for groups, and then ask for the elements - SQL couldn't provide you with those elements in a single query. You'll wind up automatically repeatedly re-querying using the group's key as a filter.

Related

Query in asp.net core

I have a query table like this, people let me ask how can I get the Positions data of the current user when I get the userid to query with the Document table.
var claimsIdentity = _httpContextAccessor.HttpContext.User.Identity as ClaimsIdentity;
var userId = claimsIdentity.FindFirst(ClaimTypes.NameIdentifier)?.Value.ToString();
var query = from c in _context.Documents
join u in _context.Users on c.UserID equals u.Id
join p in _context.Positions on u.Id equals p.UserID
where c.UserID.ToString() == userId
select new { c, u, p };
The data you query is almost enough, but it contains duplicate entries of Document and Position. If you want the final query to be put in a single object like this:
{
User = ...,
Documents = ...,
Positions = ...
}
You just need to project it using Linq-to-object (because all the data is loaded and ready for projection on the client):
var result = (from document in _context.Documents
join user in _context.Users on document.UserID equals user.Id
join position in _context.Positions on user.Id equals position.UserID
where document.UserID.ToString() == userId
select new { document, user, position }).AsEnumerable()
.GroupBy(e => e.user.Id)
.Select(g => new {
User = g.First().user,
Documents = g.GroupBy(e => e.document.Id)
.Select(e => e.First().document),
Positions = g.GroupBy(e => e.position.Id)
.Select(e => e.First().position)
}).FirstOrDefault();
If you don't want to fetch the user info, you don't need to join that DbSet but instead join the two Document and Position directly like this:
var result = (from document in _context.Documents
join position in _context.Positions on document.UserID equals position.UserID
where document.UserID.ToString() == userId
select new { document, position }).AsEnumerable()
.GroupBy(e => e.document.UserID)
.Select(g => new {
Documents = g.GroupBy(e => e.document.Id)
.Select(e => e.First().document),
Positions = g.GroupBy(e => e.position.Id)
.Select(e => e.First().position)
}).FirstOrDefault();
Note that I suppose your Document and Position both have its own primary key property of Id (adjust that to your actual design).
Finally, usually if your User entity type exposes navigation collection properties to the Document and Position. We can have a better (but equal) query like this:
var user = _context.Users
.Include(e => e.Documents)
.Include(e => e.Positions)
.FirstOrDefault(e => e.Id.ToString() == userId);
It's much simpler because all the joining internally translated by the EFCore. The magic is embedded right into the design of navigation collection properties.
I would like to talk about the important note of the condition UserID.ToString() == userId or Id.ToString() == userId. You should avoid that because it would be translated into a query that breaks the using of index for filtering. Instead try parsing for an int userId first (looks like it's a string in your case) and use that parsed int directly for comparison in the query, like this:
if(!int.TryParse(userId, out var intUserId)){
//return or throw exception
}
//here we have an user id of int, use it directly in your query
var user = _context.Users
.Include(e => e.Documents)
.Include(e => e.Positions)
.FirstOrDefault(e => e.Id == intUserId);
That applies similarly to other queries as well.

Spring Boot: How to create dynamic query involving 'join' and 'group by' using predicate

My post request body will be like
{
"queryCondition":[
{
"filter":"status",
"filterlist":["Closed","New","Resolved"...]
},
{
"filter":"assigned_team",
"filterlist":["A","B","C"...]
},
{
"filter":"assigned_to",
"filterlist":["ram","govind","ajith"...]
},
{
"filter":"duration",
"filterlist":["2020-02-01","2020-05-01"....]
}
....
....
],
"durationField":"created_date"
}
I receive the columns(filter) and values(filterlist) dynamically with which I need to build this query.
SELECT * FROM tickets
WHERE ticket_id IN (SELECT ticket_id FROM Tickets WHERE created_date >= '2020-02-01') AND created_date '2020-05-01'
AND status IN ('Closed','Resolved','New')
AND assigned_team IN ('A' , 'B', 'C')
AND assigned_to IN ('ram','govind','ajith');
I built this query dynamically using Predicate and it is working fine.
#Override
public List<Tickets> conditionedQuery(QueryCondition queryCondition) {
CriteriaBuilder cb = entityManager.getCriteriaBuilder();
CriteriaQuery<Tickets> query = cb.createQuery(Tickets.class);
Root<Tickets> ticket = query.from(Tickets.class);
List<Predicate> predicatessub = new ArrayList<>();
for(FilterConditions fc:queryCondition.getQueryCondition()) {
if(fc.getFilter().equals("duration")) {
Predicate ps = cb.greaterThanOrEqualTo(ticket.get(queryCondition.getDurationField()), fc.getFilterlist()[0]);
Predicate pe = cb.lessThan(ticket.get(queryCondition.getDurationField()), fc.getFilterlist()[1]);
predicatessub.add(cb.and(ps,pe));
}else
{
List<Predicate> predicates = new ArrayList<>();
for(int i=0; i<fc.getFilterlist().length; i++) {
Predicate p = cb.equal(ticket.get(fc.getFilter()),fc.getFilterlist()[i]);
predicates.add(p);
}
predicatessub.add(cb.or(predicates.toArray(new Predicate[predicates.size()])));
}
}
query.select(ticket)
.where(cb.and(predicatessub.toArray(new Predicate[predicatessub.size()])));
return entityManager.createQuery(query)
.getResultList();
}
QueryCondition.class
public class QueryCondition {
private List<FilterConditions> filterCondition;
private String durationField;
}
FilterConditions.class
public class FilterConditions {
private String filter;
private String[] filterlist;
}
Now I would like to build a quite more complex query involving joins and group by. Below is the sample query like what I would like to build using predicate.
SELECT
YEAR(pt.created_date),
MONTH(pt.created_date),
pt.assigned_team,
COUNT(tk.ticket_id)
FROM
(SELECT
*
FROM
tickets
WHERE
ticket_id IN (SELECT
ticket_id
FROM
Tickets
WHERE
resolved_date >= '2020-02-01')
AND resolved_date < '2020-05-01'
and assigned_team IN ('A' , 'B', 'C')) pt
LEFT JOIN
(SELECT
*
FROM
tickets
WHERE
status IN ('Closed','Resolved','New')
AND assigned_to IN ('ram','govind','ajith')) tk ON tk.ticket_id = pt.ticket_id
GROUP BY YEAR(pt.created_date) , MONTH(pt.created_date), pt.assigned_team order by
pt.assigned_team,YEAR(pt.created_date),MONTH(pt.created_date) asc;
Kindly advise how this can be achieved with Predicate or is there any other simpler way than Predicate.
Maybe this works:
First create a list with all field your model but removing associations. After pass this list in CriteriaQuery.
List<Expression<?>> groupByList = new ArrayList<>();
// ticket => Root<Tickets>
ticket.getModel().getAttibutes().stream()
.filter(a -> !a.isAssociation())
.forEach(a -> groupByList.add(ticket.get(a.getName())));
query.select(ticket)
.where(cb.and(predicatessub.toArray(new Predicate[predicatessub.size()])))
.groupBy(groupByList); // <== add group by
If you have join just do the same thing creating a cast with object EntityTypeImpl<?>. ex:
// import org.hibernate.metamodel.model.domain.internal.EntityTypeImpl
// join => Join<?,?>
((EntityTypeImpl<MyEntity>) join.getModel())
.getDeclaredAttributes().stream()
.filter(a -> !a.isAssociation())
.forEach(a -> groupByList.add(join.get(a.getName())));

Convert complex nested selects in Entity Framework query

I need to create a table in View by this View Model:
public class ApplicationContentViewModel
{
public BPMSPARS.Models.MySql.application application {get; set;}
public BPMSPARS.Models.MySql.content content { get; set; }
public BPMSPARS.Models.MySql.app_delegation app_delegation { get; set; }
}
But the query for creating new Table is very complex.
I use this query in MySQL, and I can get correct results by using it.
SELECT APP_UID, (SELECT CON_VALUE FROM content WHERE CON_CATEGORY = 'PRO_TITLE' AND CON_ID =
(SELECT PRO_UID from app_delegation WHERE del_thread_status='open' and USR_UID = '00000000000000000000000000000001' AND APP_UID = '9134216305aaaea1b67c4e2096663219')) AS TASK_NAME,
(SELECT CON_VALUE FROM content WHERE CON_CATEGORY = 'TAS_TITLE' AND CON_ID =
(SELECT TAS_UID from app_delegation WHERE del_thread_status='open' and USR_UID = '00000000000000000000000000000001' AND APP_UID = '9134216305aaaea1b67c4e2096663219')) AS PROCESS_NAME FROM app_delegation
WHERE del_thread_status='open' and USR_UID = '00000000000000000000000000000001' AND APP_UID = '9134216305aaaea1b67c4e2096663219'
But, I have to convert this query in linq or EF in MVC.
How Can I write This Query in Entity Framework query?
And How Can I display results in View?
Your SQL query seems (very) peculiar to me, as it is quite redundant. I am going to assume the sub-queries return a single value and enforce it with LINQ.
First I pulled out the common sub-query over app_delegation:
var USR_APP_Delegation = from a in app_delegation
where a.del_thread_status == "open" &&
a.USR_UID == "00000000000000000000000000000001" &&
a.APP_UID == "9134216305aaaea1b67c4e2096663219"
select a;
In LINQ it is easy to combine the two UID queries into one query:
var UIDs = (from a in USR_APP_Delegation
select new { a.PRO_UID, a.TAS_UID })
.Single();
Now you can do the name subqueries:
var TASK_NAME = (from c in content
where c.CON_CATEGORY == "PRO_TITLE" &&
c.CON_ID == UIDs.PRO_UID
select c.CON_VALUE)
.Single();
var PROCESS_NAME = (from c in content
where c.CON_CATEGORY == "TAS_TITLE" &&
c.CON_ID == UIDs.TAS_UID
select c.CON_VALUE)
.Single();
Then you can put all the queries together for the final result:
var ans = (from a in USR_APP_Delegation
select new {
a.APP_UID,
TASK_NAME,
PROCESS_NAME
})
.Single();
Again, this makes it obvious that your e.g. returning APP_UID when you know exactly what it is, and you are combining TASK_NAME and PROCESS_NAME into a query for no real advantage.
I would suggest using join against content makes a much more understandable query (even in SQL) and makes it clearer what is being returned:
var names = from a in app_delegation
join cpro in content on new { CON_ID = a.PRO_UID, CON_CATEGORY = "PRO_TITLE" } equals new { cpro.CON_ID, cpro.CON_CATEGORY }
join ctas in content on new { CON_ID = a.PRO_UID, CON_CATEGORY = "TAS_TITLE" } equals new { ctas.CON_ID, ctas.CON_CATEGORY }
where a.del_thread_status == "open" &&
a.USR_UID == "00000000000000000000000000000001" &&
a.APP_UID == "9134216305aaaea1b67c4e2096663219"
select new {
a.APP_UID,
Task_Name = ctas.CON_VALUE,
Process_Name = cpro.CON_VALUE
};

LINQ to SQL: rows to columns

I would like to have a single LINQ to SQL query to count 2 entities from the same table. E.g. Count number of employees and managers from table Personnel.
Example:
var q = from p in db.Personnel
where p.PersonType == 'Manager' || p.PersonType == 'Employee'
select new
{ NoOfPersonnel = p.Count(p => p.PersonType == 'Employee'), //Wrong way
NoOfManagers = p.Count(p => p.PersonType == 'Manager') //Wrong way
}
How can I do it?
Try this:
var list = from employee in db.Personnel
where employee.PersonType == "Manager" || employee.PersonType == "Employee"
group employee by employee.PersonType
into temp
select new { PersonType = temp.Key, Count = temp.Count() };

Extremely slow join with join buffer

I am having a problem with a complex query with multiple joins. When running EXPLAIN:
Query
explain
select ud.id from user_detail ud
cross join ticket t
cross join guest_list gl
cross join event e
cross join venue v
where t.guest_list = gl.id and gl.event = e.id and e.venue = v.id
and (ud.account = 10 or ud.venue = 10 or ud.event = 10 or ud.guest_list = 10 or t.reference_user = 10 and (ud.guest_list=t.guest_list or ud.event = gl.event or ud.venue = e.venue or ud.account = v.account) and (t.guest_list = 10))
I get this:
id, select_type, table, type, rows, extra
1, SIMPLE, v, index, 2, "Using index"
1, SIMPLE, e, ref, 2, "Using where; using index"
1, SIMPLE, gl, ref, 1, "Using where; using index"
1, SIMPLE, t, ref, 418, "Using where"
1, SIMPLE, ud, ALL, 44028, "Using where; Using join buffer"
The data model is like this:
Account <1-> Venue <1-> Event <1-> GuestList <1-> Ticket
UserDetail has an account, venue, event or guest list as a parent.
And what I am trying to do with this query is to get all of the UserDetail that has one of the specific account/venue/event/guestlist as a parent, OR that has a guestlist as a parent that has a ticket that has the reference_user field set to a specific user.
Hibernate criteria
public List<UserDetail> listUserDetails(final Collection<UserDetailNode> anyOfNodes, final User orTicketReferenceUser, final Collection<GuestList> andAnyOfGuestlistsForTicketReferenceUser, final Collection<User> anyOfUsers, final Date fromLastModificationDate, final Date toLastModificationDate, final Boolean deletedNodes, final Boolean deletedUsers, final Boolean deletedUserDetails) {
final CriteriaBuilder cb = entityManager.getCriteriaBuilder();
final CriteriaQuery<UserDetail> cq = cb.createQuery(UserDetail.class);
final Root<UserDetail> userDetail = cq.from(UserDetail.class);
Predicate criteria = cb.conjunction();
if (anyOfNodes != null || orTicketReferenceUser != null) {
Predicate subCriteria = cb.disjunction();
if (anyOfNodes != null) {
Predicate anyOfNodesCriteria = cb.disjunction();
Collection<Account> anyOfAccounts = null;
Collection<Venue> anyOfVenues = null;
Collection<Event> anyOfEvents = null;
Collection<GuestList> anyOfGuestLists = null;
final Set<UserDetailNode> anyOfNodesWithParents = new HashSet<UserDetailNode>();
for (UserDetailNode node : anyOfNodes) {
while (node != null) {
anyOfNodesWithParents.add(node);
node = node.getParentNode();
}
}
for (final UserDetailNode node : anyOfNodesWithParents) {
if (node instanceof Account) {
if (anyOfAccounts == null) anyOfAccounts = new ArrayList<Account>();
anyOfAccounts.add((Account)node);
}
else if (node instanceof Venue) {
if (anyOfVenues == null) anyOfVenues = new ArrayList<Venue>();
anyOfVenues.add((Venue)node);
}
else if (node instanceof Event) {
if (anyOfEvents == null) anyOfEvents = new ArrayList<Event>();
anyOfEvents.add((Event)node);
}
else if (node instanceof GuestList) {
if (anyOfGuestLists == null) anyOfGuestLists = new ArrayList<GuestList>();
anyOfGuestLists.add((GuestList)node);
}
}
if (anyOfAccounts != null) anyOfNodesCriteria = cb.or(anyOfNodesCriteria, cb.or(userDetail.get("account").in(anyOfAccounts)));
if (anyOfVenues != null) anyOfNodesCriteria = cb.or(anyOfNodesCriteria, cb.or(userDetail.get("venue").in(anyOfVenues)));
if (anyOfEvents != null) anyOfNodesCriteria = cb.or(anyOfNodesCriteria, cb.or(userDetail.get("event").in(anyOfEvents)));
if (anyOfGuestLists != null) anyOfNodesCriteria = cb.or(anyOfNodesCriteria, cb.or(userDetail.get("guestList").in(anyOfGuestLists)));
subCriteria = cb.or(subCriteria, anyOfNodesCriteria);
}
if (orTicketReferenceUser != null && (andAnyOfGuestlistsForTicketReferenceUser == null || !andAnyOfGuestlistsForTicketReferenceUser.isEmpty())) {
final Root<Ticket> ticket = cq.from(Ticket.class);
Predicate ticketCriteria = cb.equal(ticket.get("referenceUser"), orTicketReferenceUser);
ticketCriteria = cb.and(ticketCriteria, cb.or(cb.equal(userDetail.get("guestList"), ticket.get("guestList")), cb.equal(userDetail.get("event"), ticket.get("guestList").get("event")), cb.equal(userDetail.get("venue"), ticket.get("guestList").get("event").get("venue")), cb.equal(userDetail.get("account"), ticket.get("guestList").get("event").get("venue").get("account"))));
if (andAnyOfGuestlistsForTicketReferenceUser != null) ticketCriteria = cb.and(ticketCriteria, ticket.get("guestList").in(andAnyOfGuestlistsForTicketReferenceUser));
subCriteria = cb.or(subCriteria, ticketCriteria);
}
criteria = cb.and(criteria, subCriteria);
}
if (anyOfUsers != null) {
if (anyOfUsers.isEmpty()) return new ArrayList<UserDetail>();
criteria = cb.and(criteria, userDetail.get("user").in(anyOfUsers));
}
if (fromLastModificationDate != null) criteria = cb.and(criteria, cb.greaterThanOrEqualTo(userDetail.<Date>get("lastModificationDate"), fromLastModificationDate));
if (toLastModificationDate != null) criteria = cb.and(criteria, cb.lessThanOrEqualTo(userDetail.<Date>get("lastModificationDate"), toLastModificationDate));
cq.select(userDetail).distinct(true).where(criteria);
return entityManager.createQuery(cq).getResultList();
}
From what I can see the last row is the problem, but how can I fix this? This query is auto-generated by hibernate, so I am not sure how much I can alter it.
Your over-use of cross-join Cartesian joins doesn't make sense... What is it you are actually looking for. Since your "OR" clauses are all based on this value of 10, but then doing an implicit join to the ticket table by the guest_list id -- and finally REQUIRING the t.guest_list = 10 ?
Since all your inner joins are ALSO looking at the original user detail table having same value as result of the join. Your kicker is that the FINAL "AND" is specifically looking for guest_list = 10. I would immediately start with this as the basis and OR the others... I might consider the following:
select STRAIGHT_JOIN
ud.id
from
ticket t
JOIN user_detail ud
ON t.guest_list = ud.guest_list
where
t.guest_list = 10
AND ( ud.account = 10
or ud.venue = 10
or ud.event = 10 )
You make a reference to a "Reference_User = 10", but what is that context... is that like one user detail has a guest? and that guest can be associated with the same user detail event/venue/account?
By providing some sample of the details, and clarification of what you are hoping to get will get you much further ahead...