How do I make this complex query run faster? - mysql

This query is taking between 20-40 seconds to run. I need to speed it up greatly if possible.
SELECT DISTINCT a.category, a.key
FROM system_permissions AS a, system_permission_to_role AS b,
system_user_to_role AS c, system_users AS d
WHERE
(
(
a.system_permission_id=b.system_permission_id
AND (b.system_role_id=c.system_role_id || c.system_role_id = 0)
AND a.system_permission_id NOT IN (
SELECT system_permission_id FROM system_permission_exclusions AS f
WHERE d.system_user_id=f.system_user_id
)
AND c.system_user_id=d.system_user_id
)
OR a.system_permission_id IN (
SELECT system_permission_id
FROM system_permission_inclusions AS g
WHERE d.system_user_id=g.system_user_id
)
)
AND d.ldap_objectguid = '?';
The reason behind doing it this way is that I am creating exclusion and inclusion tables for permissions that fall outside of the standard defined roles, so first I need to exclude ones that are part of the role but exist in the exclusion table, then I need to add ones that are NOT part of their role, but exist in the inclusion table.
I am open to the idea of redesigning the tables also.

Does this work?
SELECT DISTINCT P.category, P.key
FROM system_users U
LEFT OUTER JOIN system_permission_inclusions PI ON PI.system_user_id = U.system_user_id
INNER JOIN system_user_to_role UR ON UR.system_user_id = U.system_user_id
INNER JOIN system_permission_to_role PR ON PR.system_role_id = UR.system_role_id
INNER JOIN system_permissions P ON P.system_permission_id = PR.system_permission_id OR P.system_permission_id = PI.system_permission_id
WHERE U.ldap_objectguid = '?'
AND P.system_permission_id NOT IN (SELECT system_permission_id FROM system_permission_exclusions WHERE system_user_id = U.system_user_id)

Related

Want to short the query

I have the query, which is giving correct result, but, I am sure there are other way to do so, having same conditions repeated.
Can anybody help me to reduce the complexity of the query.
Query using these mysql parameters:-
SELECT avai.account_visit_account_info_pk AS Account_ID,
mb.NAME AS Client_Name,
mb.fullname AS Client_Full_Name,
avai.account_name AS Account_Name,
mc.NAME AS Asset_City,
Format(( bfd.finance_value ), 'en_IN') AS Reserve_Price,
Format(( bfd.finance_value ) * 10 / 100, 'en_IN') AS EMD_Value,
Ifnull(Concat(CASE
WHEN mpc.parent = 4 THEN 'Residential'
WHEN mpc.parent = 5 THEN 'Commercial'
WHEN mpc.parent = 6 THEN 'Industrial'
WHEN mpc.parent = 7 THEN 'Agricultural'
END, '/', mpc.category_name), mpc.category_name) Asset_Category,
Concat(ud.first_name, ' ', ud.last_name) AS ADM_Name,
Concat(udd.first_name, ' ', udd.last_name) AS MKT_Name,
mcc.NAME AS ADM_City,
ms.NAME AS ADM_State,
mz.NAME AS ADM_Zone,
bec.e_auction_from AS Auction_Date,
bfdd.finance_value AS Sold_Price
FROM account_branch_visit abv
JOIN mst_product_category mpc
ON mpc.mst_product_category_pk = abv.mst_product_category_pk
JOIN mst_bank mb
ON abv.mst_bank_pk = mb.mst_bank_pk
JOIN banking_financial_details bfd
ON abv.account_branch_visit_pk = bfd.account_branch_visit_pk
AND bfd.mst_financial_pk IN ( 33 )
LEFT JOIN banking_financial_details bfdd
ON abv.account_branch_visit_pk = bfdd.account_branch_visit_pk
AND bfd.mst_financial_pk IN ( 38 )
JOIN mst_city mc
ON mc.mst_city_pk = avai.mst_city_pk
JOIN mst_city mcc
ON mcc.mst_city_pk = avai.mst_city_pk
JOIN mst_state ms
ON ms.mst_state_pk = mcc.mst_state_pk
JOIN mst_zone mz
ON mz.mst_zone_pk = ms.mst_zone_pk
JOIN case_allocation ca
ON ca.account_branch_visit_pk = avai.account_branch_visit_pk
AND ca.mst_activity_pk = 21
JOIN case_allocation caa
ON caa.account_branch_visit_pk = avai.account_branch_visit_pk
AND caa.mst_activity_pk = 18
JOIN user_detail ud
ON ud.user_detail_pk = ca.assignedto
JOIN user_detail udd
ON udd.user_detail_pk = caa.assignedto
JOIN banking_event_calender bec
ON bec.account_branch_visit_pk = avai.account_branch_visit_pk
AND ( abv.closed_reasons_pk IS NULL
OR abv.closed_reasons_pk = 16 )
AND abv.isdeleted = '0'
WHERE avai.account_branch_visit_pk = '1301';
I do not know what the exact intent of the query is, so I will provide some technical nuances, without actually understanding your data model or goal. The select clause provides you some columns and you probably need it. So, what I'm looking for are duplicate table joins. Some of them are necessary, some of them are unnecessary.
banking_financial_details
You join and left join this table with different ideas. You use both of them, so I assume this is necessary.
mst_city
This is obviously unnecessarily duplicating:
JOIN mst_city mc
ON mc.mst_city_pk = avai.mst_city_pk
JOIN mst_city mcc
ON mcc.mst_city_pk = avai.mst_city_pk
Remove the second JOIN and ON clauses from the above and replace all usages of mcc to mc in the query.
case_allocation
You join this table twice, but with different ids and you then join the corresponding user_detail to both and both user_detail instances are being used, so this is probably necessary.
user_detail
Since this duplicated join seems to be used in the select, it's probably necessary.
Summary
We have found an unnecessary join that can be removed. Further shortening of the query may be possible, but we would need to know more about your task and database to determine further improvements.

Mysql tekes too much time to excute sql query, based on multiple join

My Sql query takes more time to execute from mysql database server . There are number of tables are joined with sb_tblproperty table. sb_tblproperty is main table that contain more than 1,00,000 rows . most of table contain 50,000 rows.
How to optimize my sql query to fast execution. I have also used indexing.
indexing Explain - query - structure
SELECT `t1`.`propertyId`, `t1`.`projectId`,
`t1`.`furnised`, `t1`.`ownerID`, `t1`.`subType`,
`t1`.`fors`, `t1`.`size`, `t1`.`unit`,
`t1`.`bedrooms`, `t1`.`address`, `t1`.`dateConfirm`,
`t1`.`dateAdded`, `t1`.`floor`, `t1`.`priceAmount`,
`t1`.`priceRate`, `t1`.`allInclusive`, `t1`.`booking`,
`t1`.`bookingRate`, `t1`.`paidPercetage`,
`t1`.`paidAmount`, `t1`.`is_sold`, `t1`.`remarks`,
`t1`.`status`, `t1`.`confirmedStatus`, `t1`.`source`,
`t1`.`companyName` as company, `t1`.`monthly_rent`,
`t1`.`per_sqft`, `t1`.`lease_duration`,
`t1`.`lease_commencement`, `t1`.`lock_in_period`,
`t1`.`security_deposit`, `t1`.`security_amount`,
`t1`.`total_area_leased`, `t1`.`lease_escalation_amount`,
`t1`.`lease_escalation_years`, `t2`.`propertyTypeName` as
propertyTypeName, `t3`.`propertySubTypeName` subType,
`t3`.`propertySubTypeId` subTypeId, `Owner`.`ContactName`
ownerName, `Owner`.`companyName`, `Owner`.`mobile1`,
`Owner`.`otherPhoneNo`, `Owner`.`mobile2`,
`Owner`.`email`, `Owner`.`address` as caddress,
`Owner`.`contactType`, `P`.`projectName` as project,
`P`.`developerName` as developer, `c`.`name` as city,
if(t1.projectId="", group_concat( distinct( L.locality)),
group_concat( distinct(L2.locality))) as locality, `U`.`firstname`
addedBy, `U1`.`firstname` confirmedBy
FROM `sb_tblproperty` as t1
JOIN `sb_contact` Owner ON `Owner`.`id` = `t1`.`ownerID`
JOIN `tbl_city` C ON `c`.`id` = `t1`.`city`
JOIN `sb_propertytype` t2 ON `t1`.`propertyType`= `t2`.`propertyTypeId`
JOIN `sb_propertysubtype` t3 ON `t1`.`subType` =`t3`.`propertySubTypeId`
LEFT JOIN `sb_tbluser` U ON `t1`.`addedBy` = `U`.`userId`
LEFT JOIN`sb_tbluser` U1 ON `t1`.`confirmedBy` = `U1`.`userId`
LEFT JOIN `sb_tblproject` P ON `P`.`id` = `t1`.`projectId` LEFT
JOIN `sb_tblpropertylocality` PL ON `t1`.`propertyId` = `PL`.`propertyId`
LEFT JOIN `sa_localitiez` L ON `L`.`id` = `PL`.`localityId`
LEFT JOIN `sb_tblprojectlocality` PROL ON `PROL`.`projectId` = `P`.`id`
LEFT JOIN `sa_localitiez` L2 ON `L2`.`id` = `PROL`.`localityId`
LEFT JOIN `sb_tblfloor` F
ON `F`.`floorName` =`t1`.`floor`
WHERE `t1`.`is_sold` != '1' GROUP BY `t1`.`propertyId`
ORDER BY `t1`.`dateConfirm`
DESC LIMIT 1000
Please provide the EXPLAIN.
Meanwhile, try this:
SELECT ...
FROM (
SELECT propertyId
FROM sb_tblproperty
WHERE `is_sold` = 0
ORDER BY `dateConfirm` DESC
LIMIT 1000
) AS x
JOIN `sb_tblproperty` as t1 ON t1.propertyId = x.propertyId
JOIN `sb_contact` Owner ON `Owner`.`id` = `t1`.`ownerID`
JOIN `tbl_city` C ON `c`.`id` = `t1`.`city`
...
LEFT JOIN `sb_tblfloor` F ON `F`.`floorName` =`t1`.`floor`
ORDER BY `t1`.`dateConfirm` DESC -- yes, again
Together with
INDEX(is_sold, dateConfirm)
How can t1.projectId="" ? Isn't projectId the PRIMARY KEY? (This is one of many reasons for needing the SHOW CREATE TABLE.)
If my suggestion leads to "duplicate" rows (that is, multiple rows with the same propertyId), don't simply add back the GROUP BY propertyId. Instead figure out why, and avoid the need for the GROUP BY. (That is probably the performance issue.)
A likely case is the GROUP_CONCAT. A common workaround is to change from
GROUP_CONCAT( distinct( L.locality)) AS Localities,
...
LEFT JOIN `sa_localitiez` L ON `L`.`id` = `PL`.`localityId`
to
( SELECT GROUP_CONCAT(distinct locality)
FROM sa_localitiez
WHERE id = PL.localityId ) AS Localities
...
# and remove the JOIN

Simple query issue with multiple tables and mismatching IDs

I'm having trouble with a simple MySQL Query.
Here is the query:
SELECT distinct e.E_CODE, s.S_CODE, p.P_ID, p.P_NAME, p.P_FIRSTNAME, p.P_STATUS, e.E_BOSS, tp.TP_TITLE
from event_participation ep, worker p, type_participation tp, event e, section s
where ep.P_ID = p.P_ID
and s.S_ID = e.S_ID
and ep.TP_ID = tp.TP_ID
and e.E_CODE = ep.E_CODE
The problem is that ep.TP_ID sometimes has a value set to zero while tp.TP_ID has nothing with a zero ID. It's auto-increment and starts at 1 and so on.
The result is obviously that this query does not return records when the ep.TP_ID = 0 and there is no match in tp.TP_ID.
So I'm trying to figure out a way to get those results in there anyway. I was thinking of using a LEFT JOIN statement but couldn't figure out a proper way to insert it into the query.
Any advice on this matter would be greatly appreciated.
First of all, I advice you to use some general type for event_participation records without type; But, unless to take that decision, supposing you want to get all matching records between all tables but also get results with no type, you can use the following query:
SELECT DISTINCT e.E_CODE, s.S_CODE, p.P_ID, p.P_NAME, p.P_FIRSTNAME, p.P_STATUS, e.E_BOSS, tp.TP_TITLE
FROM event_participation ep
JOIN worker p ON (ep.P_ID = p.P_ID)
JOIN event e ON (e.E_CODE = ep.E_CODE)
JOIN section s ON (s.S_ID = e.S_ID)
LEFT JOIN type_participation tp ON (ep.TP_ID = tp.TP_ID)
SELECT DISTINCT e.E_CODE
, s.S_CODE
, p.P_ID
, p.P_NAME
, p.P_FIRSTNAME
, p.P_STATUS
, e.E_BOSS
, tp.TP_TITLE
FROM event_participation ep
JOIN worker p
ON p.P_ID = ep.P_ID
JOIN event e
ON e.E_CODE = ep.E_CODE
JOIN section s
ON s.S_ID = e.S_ID
LEFT
JOIN type_participation tp
ON tp.TP_ID = ep.TP_ID;

Rails - How to force associations to use alias table name

p = Patient.find(30)
p.patient_problems
The above code generates the following query
SELECT `patient_problem`.* FROM `patient_problem` WHERE `patient_problem`.`patient_id` = 30 AND (`patient_problem`.`record_status_id` = 1)
But is there any way to assign/use alias table_name like
p.patient_problems(:alias=>'p1') # just for Ex.. This code will not work
p.patient_problems(:alias=>'p2') # just for Ex.. This code will not work
So it will generate the following queries
SELECT `p1`.* FROM `patient_problem` AS `p1` WHERE `p1`.`patient_id` = 30 AND (`p1`.`record_status_id` = 1)
SELECT `p2`.* FROM `patient_problem` AS `p2` WHERE `p2`.`patient_id` = 30 AND (`p2`.`record_status_id` = 1)
Additional Info
My problem is when I try to use joins
p.patient_problems(:all,:joins=>joins)
I get this error
ActionView::Template::Error (Mysql2::Error: Not unique table/alias: 'patient_problem': SELECT `patient_problem`.* FROM `patient_problem` LEFT OUTER JOIN party on party.id = patient_problem.patient_id
LEFT OUTER JOIN party_identifier on party.id = party_identifier.party_id
LEFT OUTER JOIN blood_type on blood_type.id = party.blood_type_id
LEFT OUTER JOIN education_level on education_level.id = party.education_level_id
LEFT OUTER JOIN religion on religion.id = party.religion_id
LEFT OUTER JOIN living_arrangement on living_arrangement.id = party.living_arrangement_id
LEFT OUTER JOIN patient_problem patient_problem on patient_problem.patient_id = party.id and patient_problem.record_status_id = 1
left join (select user_type,username,user_id,auditable_id from (select MAX(id) id from audits where audits.auditable_type = 'PatientProblem' and user_type is not null group by auditable_id ) t inner join audits v on v.id=t.id ) entered_by1 on entered_by1.auditable_id = patient_problem.id
left outer join user user1 on entered_by1.user_id = user1.id
left outer join party as party_user1 on party_user1.id = user1.person_id
LEFT OUTER JOIN patient_patient_search patient_patient_search1 on patient_patient_search1.patient_id = party.id
left join search search1 on patient_patient_search1.patient_search_id = search1.id
and patient_patient_search1.patient_search_id = '75' WHERE `patient_problem`.`patient_id` = 45 AND (`patient_problem`.`record_status_id` = 1) AND ( (patient_problem.occurrence_date > '2013-01-01 00:00:00' and patient_problem.occurrence_date < '2013-06-30 23:59:59' and patient_problem.patient_problem_status_id in (5) and patient_problem.code is not null and patient_problem.code in ('10725009') ) and ( patient_patient_search1.patient_search_id in (75.0) ) ))
Ofcourse I could do some string manipulation on the generated joins query and set alias to patient_problem. But I thought setting alias for associations would be more cleaner since the joins query generated are unpredictable(in my scenario)
I am not sure what the variable joins is or how it was constructed. To alias tables in a join build your query like
Rails 3
PatientProblem.joins("as p1 OUTER JOIN patient_problem as p2 on ...")
or
PatientProblem.find(:all, :joins => "as p1 OUTER JOIN patient_problem as p2 ON ...")
you can make singleton methods for that and write the query one time and use may time like
def self.p1
#your active record query here.
end
and call like
PatientProblem.p1
Update
You can simply change the table name in your code:
Patient.table_name="p2"
I'm not sure if this would break anything else though ... so good luck!
Orignal Answer
One solution may be to define a separate model for each type of patient_problem and then do something like this:
class PatientProblem2 < ActiveRecord::Base
self.set_table_name "p2"
...
end
Another solution may be to use the ActiveRecord query interface which will allows for significant query flexibility:
http://guides.rubyonrails.org/active_record_querying.html
Perhaps you can be more specific on the nature problem you are trying to solve.

MySQL view with 5 table join is not returning any rows

Here is my view creation.. Which is not returning any rows..
CREATE VIEW TOP AS
(
SELECT
a.USER_ID, a.USER_NAME, a.PASSWORD, a.FIRST_NAME, a.LAST_NAME, a.CONTACT_NUMBER, a.EMERGENCY_NUMBER, a.CITIZEN_SHIP, a.VISA_TYPE,
b.PLACE_CODE, b.PLACE_NAME, b.AIR_AVAILIBILITY, b.TRAIN_AVAILIBILITY, b.ROAD_AVAILIBILITY,
c.CLIENT_ID, c.CLIENT_NAME, c.CLIENT_ADDRESS, c.CLIENT_LOCATION,
d.PROJECT_CODE, d.PROJECT_NAME,
e.REQUEST_ID, e.REQUEST_STATUS, e.REQUEST_FOR, e.REQUEST_MODE, e.REQUEST_TYPE, e.TRAVEL_FROM, e.TRAVEL_TO, e.TRAVEL_DATE, e.TRAVEL_TICKET_BY, e.ASSIGNMENT_PLACE,
e.ASSIGNMENT_COUNTRY, e.TRAVEL_PURPOSE, e.ASSIGNMENT_DURATION, e.ASSIGNMENT_START_DATE, e.ASSIGNMENT_ACCOMODATION_BY, e.ACCOMODATION_BILLABLE, e.TRAVEL_BILLABLE, e.ASSIGNMENT_BILLABLE, e.NSHORE_PROJECT_BILLABLE
FROM USERS a, PLACES b, CLIENTS c, PROJECTS d, REQUESTS e
WHERE
a.USER_ID = e.USER_ID
AND b.PLACE_CODE = e.TRAVEL_FROM
AND b.PLACE_CODE = e.TRAVEL_TO
AND c.CLIENT_ID = d.CLIENT_ID
AND d.PROJECT_CODE = e.PROJECT_CODE
);
I think the problem is in these two lines..
AND b.PLACE_CODE = e.TRAVEL_FROM
AND b.PLACE_CODE = e.TRAVEL_TO
where I'm trying to reference to the same table more than once. Sorry if I'm asking really a very basic question. Im completely new to databases.
Looks like you are trying to get separate details for the place TRAVEL_FROM and the place TRAVEL_TO. To do that, you need to join twice against the same table with different aliases. I'm going to replace your implicit (comma-separated FROM) joins with explicit INNER JOINs which are preferred:
CREATE VIEW TOP AS
(
SELECT
a.USER_ID, a.USER_NAME, a.PASSWORD, a.FIRST_NAME, a.LAST_NAME, a.CONTACT_NUMBER, a.EMERGENCY_NUMBER, a.CITIZEN_SHIP, a.VISA_TYPE,
/* one set of columns for TRAVEL_FROM, each with its own alias */
b_from.PLACE_CODE AS PLACE_CODE_from, b_from.PLACE_NAME AS PLACE_NAME_from, b_from.AIR_AVAILIBILITY AS AIR_AVAILABILITY_from, b_from.TRAIN_AVAILIBILITY AS TRAIN_AVAILABILITY_from, b_from.ROAD_AVAILIBILITY AS ROAD_AVAILABILITY_from,
/* and one set for TRAVEL_TO, each with its own alias*/
b_to.PLACE_CODE AS PLACE_CODE_to, b_to.PLACE_NAME AS PLACE_NAME_to, b_to.AIR_AVAILIBILITY AS AIR_AVAILABILITY_to, b_to.TRAIN_AVAILIBILITY AS TRAIN_AVAILABILITY_to, b_to.ROAD_AVAILIBILITY AS ROAD_AVAILABILITY_to,
c.CLIENT_ID, c.CLIENT_NAME, c.CLIENT_ADDRESS, c.CLIENT_LOCATION,
d.PROJECT_CODE, d.PROJECT_NAME,
e.REQUEST_ID, e.REQUEST_STATUS, e.REQUEST_FOR, e.REQUEST_MODE, e.REQUEST_TYPE, e.TRAVEL_FROM, e.TRAVEL_TO, e.TRAVEL_DATE, e.TRAVEL_TICKET_BY, e.ASSIGNMENT_PLACE,
e.ASSIGNMENT_COUNTRY, e.TRAVEL_PURPOSE, e.ASSIGNMENT_DURATION, e.ASSIGNMENT_START_DATE, e.ASSIGNMENT_ACCOMODATION_BY, e.ACCOMODATION_BILLABLE, e.TRAVEL_BILLABLE, e.ASSIGNMENT_BILLABLE, e.NSHORE_PROJECT_BILLABLE
FROM
/* implicit joins replaced with preferred explicit joins */
USERS a
INNER JOIN REQUESTS e ON a.USER_ID = e.USER_ID
/* Join frist against PLACES for TRAVEL_FROM */
INNER JOIN PLACES b_from ON b_from.PLACE_CODE = e.TRAVEL_FROM
/* And again against PLACES for TRAVEL_TO */
INNER JOIN PLACES b_to ON b_to.PLACE_CODE = e.TRAVEL_TO
INNER JOIN PROJECTS d ON d.PROJECT_CODE = e.PROJECT_CODE
INNER JOIN CLIENTS c ON c.CLIENT_ID = d.CLIENT_ID
);