multiple nested queries - mysql

I am currently trying to optimize a report using mysql and openreports for accounting use. Although I'm sure there are better tools for the job, these are what I have to work with at the moment. I am doing the following calculated fields:
client,
number of orders,
client cost,
client adjustments,
adjusted client price (client cost - client adjustments),
Vendor cost,
vendor adjustments,
adjusted vendor price (vendor cost - vendor adjustments),
margin ((client billed - vendor billed)/client billed),
adj margin ((adj client price-adj vendor price)/adj client)
The tables are quite large and I have to join 4 tables for each inner join. The report that was in place before has a union and about 20 nested selects in each side of the union all containing at least four joins and counts. Since I'm doing calculations based on previous fields, each nested select gets progressively larger containing all of the nested selects, counts, and calculations from the prior calculations. The query is now 400 lines and very expensive and taxing to the system especially on larger clients.
I realize there are better tools for the job but I was wondering what the ideal fix would be for the situation. Could I create a user defined variable within the query and use it later? Would this work if I had multiple queries per client? I can't include the entire 400 line query in this post but I can supply any additional details that wold be helpful. Any insight would be greatly appreciated.
select office_1.name as 'Client'
,count(distinct(property_1.id)) as 'Total Billed Orders',
(select format(coalesce(sum(serviceprice_2.amount),0),2)
from cap.service_price serviceprice_2
inner join cap.service service_2 on service_2.id = serviceprice_2.service_id
inner join cap.service_area servicearea_2 on servicearea_2.id = service_2.service_area_id
inner join cap.property property_2 on property_2.id = servicearea_2.property_id
inner join cap.office office_2 on office_2.id = property_2.client_id
where serviceprice_2.price_context = 'CLIENT'
and serviceprice_2.price_type_id = 236
&& service_2.date_client_billed between '2012-09-01' and now()
and service_2.gl_code_ap='4700-70-000'
and service_2.date_cancelled is null
and office_2.id = office_1.id) as 'Client Price'
/* other calculations between here */
from cap.service service_1
inner join cap.service_area servicearea_1 on servicearea_1.id = service_1.service_area_id
inner join cap.property property_1 on property_1.id = servicearea_1.property_id
inner join cap.office office_1 on office_1.id = property_1.client_id
inner join cap.office vendor_1 on vendor_1.id = service_1.vendor_id
where
service_1.date_client_billed between '2012-09-01' and now()
and service_1.date_cancelled is null
and office_1.id = 26377
and service_1.gl_code_ap='4700-70-000'
group by office_1.id
;

Using views can simplify the query although there is no effect on performance.
create view view1 as
select *
from
cap.service_price serviceprice_2
inner join cap.service service_2 on service_2.id = serviceprice_2.service_id
inner join cap.service_area servicearea_2 on servicearea_2.id = service_2.service_area_id
inner join cap.property property_2 on property_2.id = servicearea_2.property_id
inner join cap.office office_2 on office_2.id = property_2.client_id
;
create view view2 as
select *
from
cap.service service_1
inner join cap.service_area servicearea_1 on servicearea_1.id = service_1.service_area_id
inner join cap.property property_1 on property_1.id = servicearea_1.property_id
inner join cap.office office_1 on office_1.id = property_1.client_id
inner join cap.office vendor_1 on vendor_1.id = service_1.vendor_id
Now the query is more manageable:
select #client_price := format(coalesce(sum(serviceprice_2.amount),0),2)
from view1
where serviceprice_2.price_context = 'CLIENT'
and serviceprice_2.price_type_id = 236
&& service_2.date_client_billed between '2012-09-01' and now()
and service_2.gl_code_ap='4700-70-000'
and service_2.date_cancelled is null
and office_2.id = office_1.id
;
select
office_1.name as 'Client'
,count(distinct(property_1.id)) as 'Total Billed Orders',
#client_price
/* other calculations between here */
from view2
where
service_1.date_client_billed between '2012-09-01' and now()
and service_1.date_cancelled is null
and office_1.id = 26377
and service_1.gl_code_ap='4700-70-000'
group by office_1.id

Related

Want to short the query

I have the query, which is giving correct result, but, I am sure there are other way to do so, having same conditions repeated.
Can anybody help me to reduce the complexity of the query.
Query using these mysql parameters:-
SELECT avai.account_visit_account_info_pk AS Account_ID,
mb.NAME AS Client_Name,
mb.fullname AS Client_Full_Name,
avai.account_name AS Account_Name,
mc.NAME AS Asset_City,
Format(( bfd.finance_value ), 'en_IN') AS Reserve_Price,
Format(( bfd.finance_value ) * 10 / 100, 'en_IN') AS EMD_Value,
Ifnull(Concat(CASE
WHEN mpc.parent = 4 THEN 'Residential'
WHEN mpc.parent = 5 THEN 'Commercial'
WHEN mpc.parent = 6 THEN 'Industrial'
WHEN mpc.parent = 7 THEN 'Agricultural'
END, '/', mpc.category_name), mpc.category_name) Asset_Category,
Concat(ud.first_name, ' ', ud.last_name) AS ADM_Name,
Concat(udd.first_name, ' ', udd.last_name) AS MKT_Name,
mcc.NAME AS ADM_City,
ms.NAME AS ADM_State,
mz.NAME AS ADM_Zone,
bec.e_auction_from AS Auction_Date,
bfdd.finance_value AS Sold_Price
FROM account_branch_visit abv
JOIN mst_product_category mpc
ON mpc.mst_product_category_pk = abv.mst_product_category_pk
JOIN mst_bank mb
ON abv.mst_bank_pk = mb.mst_bank_pk
JOIN banking_financial_details bfd
ON abv.account_branch_visit_pk = bfd.account_branch_visit_pk
AND bfd.mst_financial_pk IN ( 33 )
LEFT JOIN banking_financial_details bfdd
ON abv.account_branch_visit_pk = bfdd.account_branch_visit_pk
AND bfd.mst_financial_pk IN ( 38 )
JOIN mst_city mc
ON mc.mst_city_pk = avai.mst_city_pk
JOIN mst_city mcc
ON mcc.mst_city_pk = avai.mst_city_pk
JOIN mst_state ms
ON ms.mst_state_pk = mcc.mst_state_pk
JOIN mst_zone mz
ON mz.mst_zone_pk = ms.mst_zone_pk
JOIN case_allocation ca
ON ca.account_branch_visit_pk = avai.account_branch_visit_pk
AND ca.mst_activity_pk = 21
JOIN case_allocation caa
ON caa.account_branch_visit_pk = avai.account_branch_visit_pk
AND caa.mst_activity_pk = 18
JOIN user_detail ud
ON ud.user_detail_pk = ca.assignedto
JOIN user_detail udd
ON udd.user_detail_pk = caa.assignedto
JOIN banking_event_calender bec
ON bec.account_branch_visit_pk = avai.account_branch_visit_pk
AND ( abv.closed_reasons_pk IS NULL
OR abv.closed_reasons_pk = 16 )
AND abv.isdeleted = '0'
WHERE avai.account_branch_visit_pk = '1301';
I do not know what the exact intent of the query is, so I will provide some technical nuances, without actually understanding your data model or goal. The select clause provides you some columns and you probably need it. So, what I'm looking for are duplicate table joins. Some of them are necessary, some of them are unnecessary.
banking_financial_details
You join and left join this table with different ideas. You use both of them, so I assume this is necessary.
mst_city
This is obviously unnecessarily duplicating:
JOIN mst_city mc
ON mc.mst_city_pk = avai.mst_city_pk
JOIN mst_city mcc
ON mcc.mst_city_pk = avai.mst_city_pk
Remove the second JOIN and ON clauses from the above and replace all usages of mcc to mc in the query.
case_allocation
You join this table twice, but with different ids and you then join the corresponding user_detail to both and both user_detail instances are being used, so this is probably necessary.
user_detail
Since this duplicated join seems to be used in the select, it's probably necessary.
Summary
We have found an unnecessary join that can be removed. Further shortening of the query may be possible, but we would need to know more about your task and database to determine further improvements.

SQL Query Count With JOIN

I've the following SQL Query which runs perfectly fine but now i want to calculate the count based on the following scenario:
SELECT d.vseverity, v.vulnstatus, v.vtitleid, d.vtitle
FROM vulnsummary v
JOIN project p ON v.projid = p.projid
AND v.stagename = p.currentstage
JOIN datasets d ON v.vtitleid = d.datasetid
The current Output is:
Now i want to show the count like this way:
High (Open) - 2
High (Closed) - 0
Medium (Open) - 1
Medium (Closed) - 0
Low (Open) - 3
Low (Closed) - 1
Please help me to solve this query, Thank You
You need to CROSS JOIN the distinct sets of severity and status values and then LEFT JOIN that to your table to allow you to count the values of each severity/status combination. Without sample data it's hard to be certain but something like this should work:
SELECT sv.vseverity, st.vulnstatus, COUNT(v.vseverity) AS count
FROM (
SELECT DISTINCT vseverity
FROM datasets
) sv
CROSS JOIN (
SELECT DISTINCT vulnstatus
FROM vulnsummary
) st
LEFT JOIN (
SELECT d.vseverity, v.vulnstatus
FROM vulnsummary v
JOIN project p ON v.projid = p.projid
AND v.stagename = p.currentstage
JOIN datasets d ON v.vtitleid = d.datasetid
) v ON v.vseverity = sv.vseverity AND v.vulnstatus = st.vulnstatus
GROUP BY sv.vseverity, st.vulnstatus
I don't have your full dataset, however, a RIGHT OUTER JOIN to a master volnstatus table will enable (the volnstatus table showing all options i.e. 'Open', 'Closed'). A rough draft example, with only the volnstatus table populated:
SELECT COUNT(s.vulnstatus) CountOf, t.vtype
FROM dbo.vusummary s
RIGHT OUTER JOIN
vusummarytype t
ON s.vulnstatus = t.vtype
GROUP BY t.vtype

MySQL, select, comparing values from 2 tables

in my database I want to select every construction_manual
where storage_room.quantity > component.quantity
But when I use:
SELECT construction_manual.name
FROM construction_manual cm, construction_manual_component cmc, component, storage_room
WHERE cm.ID = cmc.construction_manual_ID
AND cmc.component_ID = component.ID
AND component.storage_room_ID = storage_room.ID
AND storage_room.quantity > component.quantity
It will select every construction_manual.name as long as the first component in the storage_room has enough quantity.
So let's say...
construction_manual.1 needs
component.A -> quantity 5
component.B -> quantity 10
and in the storage room there is:
component.A -> quantity 6
component.B -> quantity 0
Although there is not enough in the storage room, construction_manual.1 will be selected. How do I select only those construction_manuals where there are enough components in the storage_room?
edit:
In my 4 tables are the following datasets. I will get the following result when I use the query mentioned above:database . But I should not be able to construct a table because there are not enough wooden_plates (wooden planks and not wooden plates ofc...ups)
I think what you want to ask is to ignore or exclude any construction manual that does not have enough components to satisfy a build. If that's the case you can do something like the below (a left join would also do it):
select distinct cm.name as construction_manual
from construction_manual cm
join construction_manual_component cmc
on cmc.construction_manual_ID = cm.ID
join component c
on c.ID = cmc.component_ID
join storage_room sr
on sr.ID = c.storage_room_ID
where sr.quantity > c.quantity
and cm.ID not in (select cm.ID
from construction_manual cm
join construction_manual_component cmc
on cmc.construction_manual_ID = cm.ID
join component c
on c.ID = cmc.component_ID
join storage_room sr
on sr.ID = c.storage_room_ID
where sr.quantity < c.quantity);

MySQL Creat View with all data where date is max

I want to create a view which combines the data with the maximal date from the tables shown in the picture. These should be grouped by the profileID.
Database ERM
The profileIDs are linked to profile.userID.
I tried different approches in my code. The fort one slects the data where date is max, but the join doesn't work. Every profileID will be joined with the same data.
CREATE
ALGORITHM = UNDEFINED
DEFINER = `b91788dd8d05b5`#`%`
SQL SECURITY DEFINER
VIEW fitchallengersql1.profileview AS
Select p.userID,
(SELECT
`bf`.`bodyFat`
FROM
(`fitchallengersql1`.`bodyfatprofile` `bf`
JOIN `fitchallengersql1`.`profile` `p`)
WHERE
((`bf`.`profileID` = `p`.`userID`)
AND (`bf`.`date` = (SELECT
MAX(`fitchallengersql1`.`bodyfatprofile`.`date`)
FROM
`fitchallengersql1`.`bodyfatprofile`)))) AS `bodyFat`,
(SELECT
`bw`.`bodyweight`
FROM
(`fitchallengersql1`.`bodyweightprofile` `bw`
JOIN `fitchallengersql1`.`profile` `p`)
WHERE
((`bw`.`profileID` = `p`.`userID`)
AND (`bw`.`date` = (SELECT
MAX(`fitchallengersql1`.`bodyweightprofile`.`date`)
FROM
`fitchallengersql1`.`bodyweightprofile`)))) AS `bodyWeight`,
(SELECT
`bmi`.`bmi`
FROM
(`fitchallengersql1`.`bmiprofile` `bmi`
JOIN `fitchallengersql1`.`profile` `p`)
WHERE
((`bmi`.`profileID` = `p`.`userID`)
AND (`bmi`.`date` = (SELECT
MAX(`fitchallengersql1`.`bmiprofile`.`date`)
FROM
`fitchallengersql1`.`bmiprofile`)))) AS `bmi`
From profile
In the second one the join works how it should, but I can't figure out a way to select just the data where date is max.
CREATE
ALGORITHM = UNDEFINED
DEFINER = `b91788dd8d05b5`#`%`
SQL SECURITY DEFINER
VIEW `fitchallengersql1`.`profileview` AS
SELECT
`p`.`userID` AS `userID`,
`p`.`privacy` AS `privacy`,
`bs`.`size` AS `bodysize`,
`bw`.`bodyweight` AS `bodyweight`,
`bf`.`bodyFat` AS `bodyfat`,
`bmi`.`bmi` AS `bmi`
FROM
((((`fitchallengersql1`.`profile` `p`
JOIN `fitchallengersql1`.`bodysizeprofile` `bs`)
JOIN `fitchallengersql1`.`bodyweightprofile` `bw`)
JOIN `fitchallengersql1`.`bmiprofile` `bmi`)
JOIN `fitchallengersql1`.`bodyfatprofile` `bf`)
WHERE
((`p`.`userID` = `bs`.`profileID`)
AND (`p`.`userID` = `bw`.`profileID`)
AND (`p`.`userID` = `bmi`.`profileID`)
AND (`p`.`userID` = `bf`.`profileID`))
Hope someone could help me.
Thank you!
fleewe
Hope following query gives what you need. Please follow the pattern and join the rest of the tables. Please note that when the table grows these will definitely have performance issues as this require huge processing.
-- Select the columns that you need
select p.*, lbp.*
from profile p
inner join (
-- get the latest bmiprofile per user profile
select bp1.*
from bmiprofile bp1
inner join (select profileID, max(date) as date from bmiprofile group by profileID) as bp2 on bp1.prfileId = bp2.profileId and bp1.date = bp2.date
) as lbp on lbp.ProfileId = p.userId
-- Join the other tables in similar way
this is only a comment, but I needed formating capability:
Don't place the joining predicates into the where clause if using ANSI join syntax, instead use ON followed by the relevant predicates. e.g.
FROM `fitchallengersql1`.`profile` `p`
JOIN `fitchallengersql1`.`bodysizeprofile` `bs` ON `p`.`userID` = `bs`.`profileID`
JOIN `fitchallengersql1`.`bodyweightprofile` `bw` ON `p`.`userID` = `bw`.`profileID`
JOIN `fitchallengersql1`.`bmiprofile` `bmi` ON `p`.`userID` = `bmi`.`profileID`
JOIN `fitchallengersql1`.`bodyfatprofile` `bf` ON `p`.`userID` = `bf`.`profileID`

Rails - How to force associations to use alias table name

p = Patient.find(30)
p.patient_problems
The above code generates the following query
SELECT `patient_problem`.* FROM `patient_problem` WHERE `patient_problem`.`patient_id` = 30 AND (`patient_problem`.`record_status_id` = 1)
But is there any way to assign/use alias table_name like
p.patient_problems(:alias=>'p1') # just for Ex.. This code will not work
p.patient_problems(:alias=>'p2') # just for Ex.. This code will not work
So it will generate the following queries
SELECT `p1`.* FROM `patient_problem` AS `p1` WHERE `p1`.`patient_id` = 30 AND (`p1`.`record_status_id` = 1)
SELECT `p2`.* FROM `patient_problem` AS `p2` WHERE `p2`.`patient_id` = 30 AND (`p2`.`record_status_id` = 1)
Additional Info
My problem is when I try to use joins
p.patient_problems(:all,:joins=>joins)
I get this error
ActionView::Template::Error (Mysql2::Error: Not unique table/alias: 'patient_problem': SELECT `patient_problem`.* FROM `patient_problem` LEFT OUTER JOIN party on party.id = patient_problem.patient_id
LEFT OUTER JOIN party_identifier on party.id = party_identifier.party_id
LEFT OUTER JOIN blood_type on blood_type.id = party.blood_type_id
LEFT OUTER JOIN education_level on education_level.id = party.education_level_id
LEFT OUTER JOIN religion on religion.id = party.religion_id
LEFT OUTER JOIN living_arrangement on living_arrangement.id = party.living_arrangement_id
LEFT OUTER JOIN patient_problem patient_problem on patient_problem.patient_id = party.id and patient_problem.record_status_id = 1
left join (select user_type,username,user_id,auditable_id from (select MAX(id) id from audits where audits.auditable_type = 'PatientProblem' and user_type is not null group by auditable_id ) t inner join audits v on v.id=t.id ) entered_by1 on entered_by1.auditable_id = patient_problem.id
left outer join user user1 on entered_by1.user_id = user1.id
left outer join party as party_user1 on party_user1.id = user1.person_id
LEFT OUTER JOIN patient_patient_search patient_patient_search1 on patient_patient_search1.patient_id = party.id
left join search search1 on patient_patient_search1.patient_search_id = search1.id
and patient_patient_search1.patient_search_id = '75' WHERE `patient_problem`.`patient_id` = 45 AND (`patient_problem`.`record_status_id` = 1) AND ( (patient_problem.occurrence_date > '2013-01-01 00:00:00' and patient_problem.occurrence_date < '2013-06-30 23:59:59' and patient_problem.patient_problem_status_id in (5) and patient_problem.code is not null and patient_problem.code in ('10725009') ) and ( patient_patient_search1.patient_search_id in (75.0) ) ))
Ofcourse I could do some string manipulation on the generated joins query and set alias to patient_problem. But I thought setting alias for associations would be more cleaner since the joins query generated are unpredictable(in my scenario)
I am not sure what the variable joins is or how it was constructed. To alias tables in a join build your query like
Rails 3
PatientProblem.joins("as p1 OUTER JOIN patient_problem as p2 on ...")
or
PatientProblem.find(:all, :joins => "as p1 OUTER JOIN patient_problem as p2 ON ...")
you can make singleton methods for that and write the query one time and use may time like
def self.p1
#your active record query here.
end
and call like
PatientProblem.p1
Update
You can simply change the table name in your code:
Patient.table_name="p2"
I'm not sure if this would break anything else though ... so good luck!
Orignal Answer
One solution may be to define a separate model for each type of patient_problem and then do something like this:
class PatientProblem2 < ActiveRecord::Base
self.set_table_name "p2"
...
end
Another solution may be to use the ActiveRecord query interface which will allows for significant query flexibility:
http://guides.rubyonrails.org/active_record_querying.html
Perhaps you can be more specific on the nature problem you are trying to solve.