Extremely slow join with join buffer - mysql

I am having a problem with a complex query with multiple joins. When running EXPLAIN:
Query
explain
select ud.id from user_detail ud
cross join ticket t
cross join guest_list gl
cross join event e
cross join venue v
where t.guest_list = gl.id and gl.event = e.id and e.venue = v.id
and (ud.account = 10 or ud.venue = 10 or ud.event = 10 or ud.guest_list = 10 or t.reference_user = 10 and (ud.guest_list=t.guest_list or ud.event = gl.event or ud.venue = e.venue or ud.account = v.account) and (t.guest_list = 10))
I get this:
id, select_type, table, type, rows, extra
1, SIMPLE, v, index, 2, "Using index"
1, SIMPLE, e, ref, 2, "Using where; using index"
1, SIMPLE, gl, ref, 1, "Using where; using index"
1, SIMPLE, t, ref, 418, "Using where"
1, SIMPLE, ud, ALL, 44028, "Using where; Using join buffer"
The data model is like this:
Account <1-> Venue <1-> Event <1-> GuestList <1-> Ticket
UserDetail has an account, venue, event or guest list as a parent.
And what I am trying to do with this query is to get all of the UserDetail that has one of the specific account/venue/event/guestlist as a parent, OR that has a guestlist as a parent that has a ticket that has the reference_user field set to a specific user.
Hibernate criteria
public List<UserDetail> listUserDetails(final Collection<UserDetailNode> anyOfNodes, final User orTicketReferenceUser, final Collection<GuestList> andAnyOfGuestlistsForTicketReferenceUser, final Collection<User> anyOfUsers, final Date fromLastModificationDate, final Date toLastModificationDate, final Boolean deletedNodes, final Boolean deletedUsers, final Boolean deletedUserDetails) {
final CriteriaBuilder cb = entityManager.getCriteriaBuilder();
final CriteriaQuery<UserDetail> cq = cb.createQuery(UserDetail.class);
final Root<UserDetail> userDetail = cq.from(UserDetail.class);
Predicate criteria = cb.conjunction();
if (anyOfNodes != null || orTicketReferenceUser != null) {
Predicate subCriteria = cb.disjunction();
if (anyOfNodes != null) {
Predicate anyOfNodesCriteria = cb.disjunction();
Collection<Account> anyOfAccounts = null;
Collection<Venue> anyOfVenues = null;
Collection<Event> anyOfEvents = null;
Collection<GuestList> anyOfGuestLists = null;
final Set<UserDetailNode> anyOfNodesWithParents = new HashSet<UserDetailNode>();
for (UserDetailNode node : anyOfNodes) {
while (node != null) {
anyOfNodesWithParents.add(node);
node = node.getParentNode();
}
}
for (final UserDetailNode node : anyOfNodesWithParents) {
if (node instanceof Account) {
if (anyOfAccounts == null) anyOfAccounts = new ArrayList<Account>();
anyOfAccounts.add((Account)node);
}
else if (node instanceof Venue) {
if (anyOfVenues == null) anyOfVenues = new ArrayList<Venue>();
anyOfVenues.add((Venue)node);
}
else if (node instanceof Event) {
if (anyOfEvents == null) anyOfEvents = new ArrayList<Event>();
anyOfEvents.add((Event)node);
}
else if (node instanceof GuestList) {
if (anyOfGuestLists == null) anyOfGuestLists = new ArrayList<GuestList>();
anyOfGuestLists.add((GuestList)node);
}
}
if (anyOfAccounts != null) anyOfNodesCriteria = cb.or(anyOfNodesCriteria, cb.or(userDetail.get("account").in(anyOfAccounts)));
if (anyOfVenues != null) anyOfNodesCriteria = cb.or(anyOfNodesCriteria, cb.or(userDetail.get("venue").in(anyOfVenues)));
if (anyOfEvents != null) anyOfNodesCriteria = cb.or(anyOfNodesCriteria, cb.or(userDetail.get("event").in(anyOfEvents)));
if (anyOfGuestLists != null) anyOfNodesCriteria = cb.or(anyOfNodesCriteria, cb.or(userDetail.get("guestList").in(anyOfGuestLists)));
subCriteria = cb.or(subCriteria, anyOfNodesCriteria);
}
if (orTicketReferenceUser != null && (andAnyOfGuestlistsForTicketReferenceUser == null || !andAnyOfGuestlistsForTicketReferenceUser.isEmpty())) {
final Root<Ticket> ticket = cq.from(Ticket.class);
Predicate ticketCriteria = cb.equal(ticket.get("referenceUser"), orTicketReferenceUser);
ticketCriteria = cb.and(ticketCriteria, cb.or(cb.equal(userDetail.get("guestList"), ticket.get("guestList")), cb.equal(userDetail.get("event"), ticket.get("guestList").get("event")), cb.equal(userDetail.get("venue"), ticket.get("guestList").get("event").get("venue")), cb.equal(userDetail.get("account"), ticket.get("guestList").get("event").get("venue").get("account"))));
if (andAnyOfGuestlistsForTicketReferenceUser != null) ticketCriteria = cb.and(ticketCriteria, ticket.get("guestList").in(andAnyOfGuestlistsForTicketReferenceUser));
subCriteria = cb.or(subCriteria, ticketCriteria);
}
criteria = cb.and(criteria, subCriteria);
}
if (anyOfUsers != null) {
if (anyOfUsers.isEmpty()) return new ArrayList<UserDetail>();
criteria = cb.and(criteria, userDetail.get("user").in(anyOfUsers));
}
if (fromLastModificationDate != null) criteria = cb.and(criteria, cb.greaterThanOrEqualTo(userDetail.<Date>get("lastModificationDate"), fromLastModificationDate));
if (toLastModificationDate != null) criteria = cb.and(criteria, cb.lessThanOrEqualTo(userDetail.<Date>get("lastModificationDate"), toLastModificationDate));
cq.select(userDetail).distinct(true).where(criteria);
return entityManager.createQuery(cq).getResultList();
}
From what I can see the last row is the problem, but how can I fix this? This query is auto-generated by hibernate, so I am not sure how much I can alter it.

Your over-use of cross-join Cartesian joins doesn't make sense... What is it you are actually looking for. Since your "OR" clauses are all based on this value of 10, but then doing an implicit join to the ticket table by the guest_list id -- and finally REQUIRING the t.guest_list = 10 ?
Since all your inner joins are ALSO looking at the original user detail table having same value as result of the join. Your kicker is that the FINAL "AND" is specifically looking for guest_list = 10. I would immediately start with this as the basis and OR the others... I might consider the following:
select STRAIGHT_JOIN
ud.id
from
ticket t
JOIN user_detail ud
ON t.guest_list = ud.guest_list
where
t.guest_list = 10
AND ( ud.account = 10
or ud.venue = 10
or ud.event = 10 )
You make a reference to a "Reference_User = 10", but what is that context... is that like one user detail has a guest? and that guest can be associated with the same user detail event/venue/account?
By providing some sample of the details, and clarification of what you are hoping to get will get you much further ahead...

Related

Dynamic where clause using Linq expression having join with multiple tables

i have a linq query but in where clause is conditional. if eve.EventType is null then it will not include in where clause. How can we do with Linq lambda expression
var data= (from reg in product
join se in _order on reg.EventSessionId equals se.EventSessionId
join eve in Event on se.EventId equals eve.EventId
where eve.EventType == (EventType)eventType &&
((!string.IsNullOrEmpty(eve.EventName) && eve.EventName.Contains(SearchText, StringComparison.OrdinalIgnoreCase))
select (new OrderHistory
{
RegistrationId = reg.RegistrationId,
EventName = eve.EventName,
EventSesionName = se.EventSesionName,
})).ToList();
Thanks
This may help you.You should check null at first of your expression.
var data= (from reg in product
join se in _order on reg.EventSessionId equals se.EventSessionId
join eve in Event on se.EventId equals eve.EventId
where eve.EventType != null && eve.EventType == (EventType)eventType &&
((!string.IsNullOrEmpty(eve.EventName) && eve.EventName.Contains(SearchText, StringComparison.OrdinalIgnoreCase))
select (new OrderHistory
{
RegistrationId = reg.RegistrationId,
EventName = eve.EventName,
EventSesionName = se.EventSesionName,
})).ToList();

Convert complex nested selects in Entity Framework query

I need to create a table in View by this View Model:
public class ApplicationContentViewModel
{
public BPMSPARS.Models.MySql.application application {get; set;}
public BPMSPARS.Models.MySql.content content { get; set; }
public BPMSPARS.Models.MySql.app_delegation app_delegation { get; set; }
}
But the query for creating new Table is very complex.
I use this query in MySQL, and I can get correct results by using it.
SELECT APP_UID, (SELECT CON_VALUE FROM content WHERE CON_CATEGORY = 'PRO_TITLE' AND CON_ID =
(SELECT PRO_UID from app_delegation WHERE del_thread_status='open' and USR_UID = '00000000000000000000000000000001' AND APP_UID = '9134216305aaaea1b67c4e2096663219')) AS TASK_NAME,
(SELECT CON_VALUE FROM content WHERE CON_CATEGORY = 'TAS_TITLE' AND CON_ID =
(SELECT TAS_UID from app_delegation WHERE del_thread_status='open' and USR_UID = '00000000000000000000000000000001' AND APP_UID = '9134216305aaaea1b67c4e2096663219')) AS PROCESS_NAME FROM app_delegation
WHERE del_thread_status='open' and USR_UID = '00000000000000000000000000000001' AND APP_UID = '9134216305aaaea1b67c4e2096663219'
But, I have to convert this query in linq or EF in MVC.
How Can I write This Query in Entity Framework query?
And How Can I display results in View?
Your SQL query seems (very) peculiar to me, as it is quite redundant. I am going to assume the sub-queries return a single value and enforce it with LINQ.
First I pulled out the common sub-query over app_delegation:
var USR_APP_Delegation = from a in app_delegation
where a.del_thread_status == "open" &&
a.USR_UID == "00000000000000000000000000000001" &&
a.APP_UID == "9134216305aaaea1b67c4e2096663219"
select a;
In LINQ it is easy to combine the two UID queries into one query:
var UIDs = (from a in USR_APP_Delegation
select new { a.PRO_UID, a.TAS_UID })
.Single();
Now you can do the name subqueries:
var TASK_NAME = (from c in content
where c.CON_CATEGORY == "PRO_TITLE" &&
c.CON_ID == UIDs.PRO_UID
select c.CON_VALUE)
.Single();
var PROCESS_NAME = (from c in content
where c.CON_CATEGORY == "TAS_TITLE" &&
c.CON_ID == UIDs.TAS_UID
select c.CON_VALUE)
.Single();
Then you can put all the queries together for the final result:
var ans = (from a in USR_APP_Delegation
select new {
a.APP_UID,
TASK_NAME,
PROCESS_NAME
})
.Single();
Again, this makes it obvious that your e.g. returning APP_UID when you know exactly what it is, and you are combining TASK_NAME and PROCESS_NAME into a query for no real advantage.
I would suggest using join against content makes a much more understandable query (even in SQL) and makes it clearer what is being returned:
var names = from a in app_delegation
join cpro in content on new { CON_ID = a.PRO_UID, CON_CATEGORY = "PRO_TITLE" } equals new { cpro.CON_ID, cpro.CON_CATEGORY }
join ctas in content on new { CON_ID = a.PRO_UID, CON_CATEGORY = "TAS_TITLE" } equals new { ctas.CON_ID, ctas.CON_CATEGORY }
where a.del_thread_status == "open" &&
a.USR_UID == "00000000000000000000000000000001" &&
a.APP_UID == "9134216305aaaea1b67c4e2096663219"
select new {
a.APP_UID,
Task_Name = ctas.CON_VALUE,
Process_Name = cpro.CON_VALUE
};

How toPerform a left join in linq pad -

How can I get linq pad to run my left join as show below?
var query =
from s in db.CDBLogsHeaders
.OrderByDescending(g => g.LogDateTime)
from sc in db.StyleColors
.Where(stylecolor => stylecolor.id == (int?)s.StyleColorID)
.DefaultIfEmpty()
from c in db.StyleHeaders
.Where(styleHeader => styleHeader.id == (int?)s.StyleHeaderID)
.DefaultIfEmpty()
select new
{
CDBLogsHeaderId = s.Id,
Merchandiser = c.Merchandiser,
Material = s.Material,
Season = s.Season,
LogsHeaderLogType = s.LogType,
PushFromTo = s.PushFromTo,
LinePlan = s.LinePlan,
QuikRefNumber = s.QuikRefNumber,
PLMOrigin = s.PLM_Origin,
SeasonOriginal = c.SeasonOriginal,
SeasonCurrent = c.SeasonCurrent,
StyleHeaderId = c.Id,
StyleCode = c.StyleCode,
StyleColorsColorCode = sc.ColorCode
};
query.Dump();
The sql that linq pad creates runs perfectly in Management Studio but linq-pad doesn't display any rows and gives this error
InvalidOperationException: The null value cannot be assigned to a
member with type System.Int32 which is a non-nullable value type.
How can I get linqpad to work so I can play with it?
In your anonymous type, make sure your ints are returning ints.
Change
StyleHeaderId = c.Id,
To
StyleHeaderId = (c.Id == null ? 0 : c.Id),

Help building up LINQ2SQL query which has joins

I have this:
var q = (from order in db.Orders
from payment in db.Payments
.Where(x => x.ID == order.paymentID)
.DefaultIfEmpty()
from siteUser in db.SiteUsers
.Where(x => x.siteUserID == order.siteUserID)
.DefaultIfEmpty()
where siteUser.siteUserID != null
select new
{
order.orderID,
order.dateCreated,
payment.totalAmount,
siteUser.firstName,
siteUser.lastName
});
I want to add on to it like this:
switch (_qs["sort"])
{
case "0":
q = q.OrderByDescending(x => x.dateCreated);
break;
case "1":
q = q.OrderBy(x => x.dateCreated);
break; ...
I've done this before with a single table, but the multiple tables in the first code block force me to specify a select statement which causes it to be an anonymous type. How can this be done?
Note: I even tried to make a class with the properties that i'm selecting and casting the query to this type, still a no go.
Not sure I understand the question but the code you pasted looks valid to me.
I checked:
var q = (
from order in db.Orders
join payment in db.Payments on
order.paymentID equals payment.ID into payments
from payment in payments.DefaultIfEmpty()
join siteUser in db.SiteUsers on
order.siteUserID equals siteUser.siteUserID into siteUsers
from siteUser in siteUsers.DefaultIfEmpty()
where siteUser.siteUserID != null
select
new
{
order.orderID,
order.dateCreated,
payment.totalAmount,
siteUser.firstName,
siteUser.lastName
});
switch (sort)
{
case "0":
q = q.OrderByDescending(x => x.dateCreated);
break;
case "1":
q = q.OrderBy(x => x.dateCreated);
break;
}
var restult = q.ToList();
This works.

LINQ join and group

How to expand this query:
public Dictionary<int, List<TasksInDeal>> FindAllCreatedTasks()
{
return (from taskInDeal in db.TasksInDeals
where taskInDeal.Date > DateTime.Now && taskInDeal.Date < DateTime.Now.AddDays(7)
group taskInDeal by taskInDeal.CreatedByUserID
into groupedDemoClasses
select groupedDemoClasses).ToDictionary(gdc => gdc.Key, gdc => gdc.ToList());
}
into something like this:
public Dictionary<int, List<TaskForNotification>> FindAllCreatedTasks()
{
return (from taskInDeal in db.TasksInDeals
join user in db.Users on taskInDeal.CreatedByUserID equals user.UserID
where taskInDeal.Date > DateTime.Now && taskInDeal.Date < DateTime.Now.AddDays(7)
group taskInDeal by taskInDeal.CreatedByUserID
into groupedDemoClasses
select new TaskForNotification
{
Email = user.Email,
TaskInDealField1 = taskInDeal.TaskInDealField1,
TaskInDealField2 = taskInDeal.TaskInDealField2,
TaskInDealField3 = taskInDeal.TaskInDealField3,
...
}
).ToDictionary(gdc => gdc.Key, gdc => gdc.ToList());
}
So, to first query I need to join email from other table.
// do the date logic up front, not in the database.
DateTime now = DateTime.Now
DateTime weekFromNow = now.AddDays(7);
// pull the joined rows out of the database.
var rows =
(
from taskInDeal in db.TasksInDeals
where taskInDeal.Date > now && taskInDeal.Date < weekFromNow
join user in db.Users
on taskInDeal.CreatedByUserID equals user.UserID
select new {TaskInDeal = taskInDeal, UserEmail = user.Email}
).ToList();
// shape the rows in memory
Dictionary<int, List<TaskForNotification>> result =
(
from row in rows
let taskForNotification = new TaskForNotification
{
Email = row.UserEmail,
TaskInDealField1 = row.TaskInDeal.TaskInDealField1,
TaskInDealField2 = row.TaskInDeal.TaskInDealField2,
TaskInDealField3 = row.TaskInDeal.TaskInDealField3,
...
}
group taskForNotification by row.TaskInDeal.CreatedByUserID
// without an "into", group by ends the query.
).ToDictionary(g => g.Key, g => g.ToList());
When you group, bear this in mind. Groups in SQL have only keys and aggregates. Groups in LINQ have keys, aggregates and elements! If you ask the database for groups, and then ask for the elements - SQL couldn't provide you with those elements in a single query. You'll wind up automatically repeatedly re-querying using the group's key as a filter.