I am working on a rather complex and time critical data conversion.
This is achieved by using a Stored Procedure.
A recent change request required me to update this stored procedure. To do so in a safe manner, I have created a full schema dump including data and have imported this into two different schemas. One called 'original' and one called 'amended'.
Before updating the live database I am trying to compare the output of the original and amended database to identify the data that has been added and that none has been lost.
Unfortunately my test system is running MySQL v 5.7.12-0ubuntu-1 whereas my live server is a CentOS Server running 5.6.30.
A group clause in the original database (identical with live) is raising an issue (that does not exist on live). This error (1055 - Expression #5 of SELECT list is not in GROUP BY clause and contains non-aggregated column) makes total sense and is addressed in the revised database. I just want to run a comparison without having to install the exact same version of MySQL on this development server.
I have tried to remove ONLY_FULL_GROUP_BY from my sql_mode parameter and have matched the sql_mode on both servers to only NO_ENGINE_SUBSTITUTION
Can anyone help? The query is below:
Insert into skeleton_football (
sport_id,
parent_event_id,
event_id,
event_part_id,
back_outcome_id,
lay_outcome_id,
start_time,
market_type,
competition_name,
event_name,
event_part_name,
outcome_name,
outcome_value
)
SELECT
sp.id as `sport_id`,
if (ev.name is null, pev.id, null) as `parent_event_id`,
ev.id as `event_id`,
evp.id as `event_part_id`,
oc.back_outcome_id as back_outcome_id,
oc.lay_outcome_id as lay_outcome_id,
#oc.param_float_1,
#oc.param_float_2,
#oc.param_float_3,
#oc.param_boolean_1,
#oc.param_string_1,
#oc.param_participant_id_1,
#oc.param_participant_id_2,
#oc.param_participant_id_3,
#oc.param_event_part_id_1,
ev.start_time as start_time,
sp.name as market_type,
if (ev.name is null, pev.name, ev.name) as competition_name,
if (sp.name = "football",concat(epah.name, " v ", epaa.name),ev.name) as event_name,
evp.name as event_part_name,
oct.name as outcome_name,
get_outcome_value(
oct.name,
oc.param_participant_id_1,
max(epah.id),
max(epah.name),
max(epaa.id),
max(epaa.name),
opa.id,
opa.name,
oc.param_float_1,
oc.param_float_2,
oc.param_boolean_1
) as outcome_value
FROM raw_event ev
left join raw_event pev on pev.id = ev.parent_id
inner join raw_sport sp on ev.sport_id = sp.id
inner join config_events_filter cef on cef.sport_id = sp.id and (cef.competition_name is null or pev.name like cef.competition_name)
left join outcome_matched oc on ev.id = oc.event_id
inner join raw_event_part evp on evp.id = oc.event_part_id and evp.name in ("ordinary time", "whole match", "whole event")
cross join raw_participant_role prh on prh.name ="home"
cross join raw_participant_role pra on pra.name = "away"
cross join raw_participant_role pro on pro.name not in ("home", "away")
left join raw_event_participant_relation pera on pera.event_id = ev.id and pra.id = pera.participant_role_id
left join raw_event_participant_relation perh on perh.event_id = ev.id and prh.id = perh.participant_role_id
left join raw_event_participant_relation pero on pero.event_id = ev.id and pro.id = pero.participant_role_id
left join raw_outcome_type oct on oc.type_id = oct.id
left join raw_participant epaa on pera.participant_id = epaa.id
left join raw_participant epah on perh.participant_id = epah.id
left join raw_participant epao on pero.participant_id = epao.id
#left join raw_participant epap on per.participant_id = epap.id and pr.name not in ("home", "away")
left join raw_participant opa on oc.param_participant_id_1 = opa.id
where ev.start_time > date_sub('2016-05-11 14:00:00', INTERVAL 1 HOUR)
and ev.start_time < date_add('2016-05-11 14:00:00', INTERVAL 2 DAY)
group by sp.id, ev.id, evp.id, oc.back_unique, if (sp.name = "football",concat(epah.name, " v ", epaa.name),ev.name)
having event_name is not null;
Related
I want to create the custom report on event by 2 monitoring system.
I know about Zabbix API, but think
if I request data through the API, then the performance will not be the same if I perform queries directly to the database Zabbix.
I found a table with information about events,
SELECT eventid, source, object, objectid, clock, value, acknowledged, ns
FROM zabbix.events
Found zabbix.hosts and zabbix.host_inventory.
How is zabbix.events linked to zabbix.hosts?
You can check this link on the Zabbix share :
https://share.zabbix.com/databases/mysql/zabbix-database-model
Alain updates it pretty fast for every new release, and if you want to do some SQL queries, this is a must have.
objectid for trigger events (source=0) is the trigger ID, which in turn has function IDs, which in turn have item IDs, which in turn have host IDs.
The database schema is not documented officially, but you may find some community resources for that - http://zabbix.org/wiki/Database_Schemas. Just keep in mind that's 2.4.3 version. That is likely to be enough to find relations for your case.
information did compilate in this select:
SELECT * FROM events
JOIN triggers ON events.objectid = triggers.triggerid
JOIN functions ON functions.triggerid = triggers.triggerid
JOIN items ON items.itemid = functions.itemid
JOIN hosts ON items.hostid = hosts.hostid
WHERE events.source = 0
AND
LOWER(hosts.host) like 'mysql%'
AND events.clock>=unix_timestamp('2017-09-25 09:55:00')
AND events.clock<=unix_timestamp('2017-09-25 11:00:00')
ORDER BY events.clock DESC
;
This is a query I created to run against the zabbix db. It joins hosts and hostgroups, and then left joins acknowledged or unacknowledged events. once an event is resolved, it will drop off this query. It pulls for enabled or unreachable hosts.
select
h.name as hostname,
h.status as hoststatus,
g.name as groupname,
hi.alias as hostalias,
hi.location as hostlocation,
hi.os as hostos,
dt.idescription as itemdescription,
dt.ikey as itemkey_,
dt.iname as itemname,
dt.hsurl as httpstepurl,
dt.hsname as httpstepname,
dt.tcomments as triggercomments,
dt.tdescription as triggerdescription,
dt.tpriority as triggerpriority,
dt.eventclock as eventclock,
dt.eacknowledged as eventacknowledged
from
hosts h
inner join hosts_groups hg on h.hostid=hg.hostid
inner join groups g on hg.groupid = g.groupid
left join host_inventory hi on h.hostid=hi.hostid
LEFT JOIN
(SELECT
i.hostid as ihostid,
i.itemid as iitemid,
i.description as idescription,
i.key_ as ikey,
i.name as iname,
hs.url as hsurl,
hs.name as hsname,
t.description as tdescription,
t.url as turl,
t.comments as tcomments,
t.priority as tpriority,
from_unixtime(e.clock) as eventclock,
e.acknowledged as eacknowledged
from items i
left join functions f on i.itemid = f.itemid
left join triggers t on f.triggerid = t.triggerid
right join events e on t.triggerid = e.objectid
left join httpstepitem hsi on i.itemid = hsi.itemid
left join httpstep hs on hsi.httpstepid = hs.httpstepid
inner join problem p on e.eventid = p.eventid
WHERE
((e.acknowledged='0' AND i.status='0' AND r_clock='0') OR (e.acknowledged='1' AND i.status='0' AND r_clock='0'))
) dt ON h.hostid = dt.ihostid
where (h.status='2' or h.status='0');
I want to create a view which combines the data with the maximal date from the tables shown in the picture. These should be grouped by the profileID.
Database ERM
The profileIDs are linked to profile.userID.
I tried different approches in my code. The fort one slects the data where date is max, but the join doesn't work. Every profileID will be joined with the same data.
CREATE
ALGORITHM = UNDEFINED
DEFINER = `b91788dd8d05b5`#`%`
SQL SECURITY DEFINER
VIEW fitchallengersql1.profileview AS
Select p.userID,
(SELECT
`bf`.`bodyFat`
FROM
(`fitchallengersql1`.`bodyfatprofile` `bf`
JOIN `fitchallengersql1`.`profile` `p`)
WHERE
((`bf`.`profileID` = `p`.`userID`)
AND (`bf`.`date` = (SELECT
MAX(`fitchallengersql1`.`bodyfatprofile`.`date`)
FROM
`fitchallengersql1`.`bodyfatprofile`)))) AS `bodyFat`,
(SELECT
`bw`.`bodyweight`
FROM
(`fitchallengersql1`.`bodyweightprofile` `bw`
JOIN `fitchallengersql1`.`profile` `p`)
WHERE
((`bw`.`profileID` = `p`.`userID`)
AND (`bw`.`date` = (SELECT
MAX(`fitchallengersql1`.`bodyweightprofile`.`date`)
FROM
`fitchallengersql1`.`bodyweightprofile`)))) AS `bodyWeight`,
(SELECT
`bmi`.`bmi`
FROM
(`fitchallengersql1`.`bmiprofile` `bmi`
JOIN `fitchallengersql1`.`profile` `p`)
WHERE
((`bmi`.`profileID` = `p`.`userID`)
AND (`bmi`.`date` = (SELECT
MAX(`fitchallengersql1`.`bmiprofile`.`date`)
FROM
`fitchallengersql1`.`bmiprofile`)))) AS `bmi`
From profile
In the second one the join works how it should, but I can't figure out a way to select just the data where date is max.
CREATE
ALGORITHM = UNDEFINED
DEFINER = `b91788dd8d05b5`#`%`
SQL SECURITY DEFINER
VIEW `fitchallengersql1`.`profileview` AS
SELECT
`p`.`userID` AS `userID`,
`p`.`privacy` AS `privacy`,
`bs`.`size` AS `bodysize`,
`bw`.`bodyweight` AS `bodyweight`,
`bf`.`bodyFat` AS `bodyfat`,
`bmi`.`bmi` AS `bmi`
FROM
((((`fitchallengersql1`.`profile` `p`
JOIN `fitchallengersql1`.`bodysizeprofile` `bs`)
JOIN `fitchallengersql1`.`bodyweightprofile` `bw`)
JOIN `fitchallengersql1`.`bmiprofile` `bmi`)
JOIN `fitchallengersql1`.`bodyfatprofile` `bf`)
WHERE
((`p`.`userID` = `bs`.`profileID`)
AND (`p`.`userID` = `bw`.`profileID`)
AND (`p`.`userID` = `bmi`.`profileID`)
AND (`p`.`userID` = `bf`.`profileID`))
Hope someone could help me.
Thank you!
fleewe
Hope following query gives what you need. Please follow the pattern and join the rest of the tables. Please note that when the table grows these will definitely have performance issues as this require huge processing.
-- Select the columns that you need
select p.*, lbp.*
from profile p
inner join (
-- get the latest bmiprofile per user profile
select bp1.*
from bmiprofile bp1
inner join (select profileID, max(date) as date from bmiprofile group by profileID) as bp2 on bp1.prfileId = bp2.profileId and bp1.date = bp2.date
) as lbp on lbp.ProfileId = p.userId
-- Join the other tables in similar way
this is only a comment, but I needed formating capability:
Don't place the joining predicates into the where clause if using ANSI join syntax, instead use ON followed by the relevant predicates. e.g.
FROM `fitchallengersql1`.`profile` `p`
JOIN `fitchallengersql1`.`bodysizeprofile` `bs` ON `p`.`userID` = `bs`.`profileID`
JOIN `fitchallengersql1`.`bodyweightprofile` `bw` ON `p`.`userID` = `bw`.`profileID`
JOIN `fitchallengersql1`.`bmiprofile` `bmi` ON `p`.`userID` = `bmi`.`profileID`
JOIN `fitchallengersql1`.`bodyfatprofile` `bf` ON `p`.`userID` = `bf`.`profileID`
Have an access report that shows training programs, and which employees should be but are not trained on that program. This query is fine. Problem is that we want to only display on the report training programs which have more than 10 employees untrained. So we have the total of untrained for each program in a subtotal, but we want to filter on that value.
How can this be done?
EDIT:
Here is pass-through query to SQL Server
SELECT T.ProgramTitle
,T.ProgramCode
,AE.Code AS 'AvantiCode'
,AE.FullName
,AE.FirstName
,AE.LastName
,AE.Department
,C.Position
,AE.Shift
FROM HR_Curriculum C
INNER JOIN HR_Trainings T ON C.TrainingID = T.TrainingID
INNER JOIN HR_EmployeeDetails ED ON C.Position = ED.Postion
INNER JOIN Avanti_Employees AE ON ED.AvantiRecID = AE.RecID
LEFT JOIN HR_Employeetrainings ET ON C.TrainingID = ET.TrainingID
AND ED.AvantiRecID = ET.AvantiRecID
LEFT JOIN HR_TrainingVersion V ON V.VersionID = ET.VersionID
WHERE terminated = 0
AND T.Active = - 1
AND CompletedDate IS NULL
GROUP BY T.ProgramTitle
,T.ProgramCode
,AE.Code
,AE.FullName
,AE.FirstName
,AE.LastName
,AE.Department
,C.Position
,AE.Shift
Order by programtitle
Consider an inline view, using a grouped by table alias with HAVING clause.
Try adding one more inner join:
INNER JOIN
(SELECT TrainingID, ProgramTitle, ProgramCode
FROM HR_Trainings
GROUP BY TrainingID, ProgramTitle, ProgramCode
HAVING Count(TrainingID) > 10) AS Trainings10More
ON Trainings10More.TrainingID = T.TrainingID
p = Patient.find(30)
p.patient_problems
The above code generates the following query
SELECT `patient_problem`.* FROM `patient_problem` WHERE `patient_problem`.`patient_id` = 30 AND (`patient_problem`.`record_status_id` = 1)
But is there any way to assign/use alias table_name like
p.patient_problems(:alias=>'p1') # just for Ex.. This code will not work
p.patient_problems(:alias=>'p2') # just for Ex.. This code will not work
So it will generate the following queries
SELECT `p1`.* FROM `patient_problem` AS `p1` WHERE `p1`.`patient_id` = 30 AND (`p1`.`record_status_id` = 1)
SELECT `p2`.* FROM `patient_problem` AS `p2` WHERE `p2`.`patient_id` = 30 AND (`p2`.`record_status_id` = 1)
Additional Info
My problem is when I try to use joins
p.patient_problems(:all,:joins=>joins)
I get this error
ActionView::Template::Error (Mysql2::Error: Not unique table/alias: 'patient_problem': SELECT `patient_problem`.* FROM `patient_problem` LEFT OUTER JOIN party on party.id = patient_problem.patient_id
LEFT OUTER JOIN party_identifier on party.id = party_identifier.party_id
LEFT OUTER JOIN blood_type on blood_type.id = party.blood_type_id
LEFT OUTER JOIN education_level on education_level.id = party.education_level_id
LEFT OUTER JOIN religion on religion.id = party.religion_id
LEFT OUTER JOIN living_arrangement on living_arrangement.id = party.living_arrangement_id
LEFT OUTER JOIN patient_problem patient_problem on patient_problem.patient_id = party.id and patient_problem.record_status_id = 1
left join (select user_type,username,user_id,auditable_id from (select MAX(id) id from audits where audits.auditable_type = 'PatientProblem' and user_type is not null group by auditable_id ) t inner join audits v on v.id=t.id ) entered_by1 on entered_by1.auditable_id = patient_problem.id
left outer join user user1 on entered_by1.user_id = user1.id
left outer join party as party_user1 on party_user1.id = user1.person_id
LEFT OUTER JOIN patient_patient_search patient_patient_search1 on patient_patient_search1.patient_id = party.id
left join search search1 on patient_patient_search1.patient_search_id = search1.id
and patient_patient_search1.patient_search_id = '75' WHERE `patient_problem`.`patient_id` = 45 AND (`patient_problem`.`record_status_id` = 1) AND ( (patient_problem.occurrence_date > '2013-01-01 00:00:00' and patient_problem.occurrence_date < '2013-06-30 23:59:59' and patient_problem.patient_problem_status_id in (5) and patient_problem.code is not null and patient_problem.code in ('10725009') ) and ( patient_patient_search1.patient_search_id in (75.0) ) ))
Ofcourse I could do some string manipulation on the generated joins query and set alias to patient_problem. But I thought setting alias for associations would be more cleaner since the joins query generated are unpredictable(in my scenario)
I am not sure what the variable joins is or how it was constructed. To alias tables in a join build your query like
Rails 3
PatientProblem.joins("as p1 OUTER JOIN patient_problem as p2 on ...")
or
PatientProblem.find(:all, :joins => "as p1 OUTER JOIN patient_problem as p2 ON ...")
you can make singleton methods for that and write the query one time and use may time like
def self.p1
#your active record query here.
end
and call like
PatientProblem.p1
Update
You can simply change the table name in your code:
Patient.table_name="p2"
I'm not sure if this would break anything else though ... so good luck!
Orignal Answer
One solution may be to define a separate model for each type of patient_problem and then do something like this:
class PatientProblem2 < ActiveRecord::Base
self.set_table_name "p2"
...
end
Another solution may be to use the ActiveRecord query interface which will allows for significant query flexibility:
http://guides.rubyonrails.org/active_record_querying.html
Perhaps you can be more specific on the nature problem you are trying to solve.
I am currently trying to optimize a report using mysql and openreports for accounting use. Although I'm sure there are better tools for the job, these are what I have to work with at the moment. I am doing the following calculated fields:
client,
number of orders,
client cost,
client adjustments,
adjusted client price (client cost - client adjustments),
Vendor cost,
vendor adjustments,
adjusted vendor price (vendor cost - vendor adjustments),
margin ((client billed - vendor billed)/client billed),
adj margin ((adj client price-adj vendor price)/adj client)
The tables are quite large and I have to join 4 tables for each inner join. The report that was in place before has a union and about 20 nested selects in each side of the union all containing at least four joins and counts. Since I'm doing calculations based on previous fields, each nested select gets progressively larger containing all of the nested selects, counts, and calculations from the prior calculations. The query is now 400 lines and very expensive and taxing to the system especially on larger clients.
I realize there are better tools for the job but I was wondering what the ideal fix would be for the situation. Could I create a user defined variable within the query and use it later? Would this work if I had multiple queries per client? I can't include the entire 400 line query in this post but I can supply any additional details that wold be helpful. Any insight would be greatly appreciated.
select office_1.name as 'Client'
,count(distinct(property_1.id)) as 'Total Billed Orders',
(select format(coalesce(sum(serviceprice_2.amount),0),2)
from cap.service_price serviceprice_2
inner join cap.service service_2 on service_2.id = serviceprice_2.service_id
inner join cap.service_area servicearea_2 on servicearea_2.id = service_2.service_area_id
inner join cap.property property_2 on property_2.id = servicearea_2.property_id
inner join cap.office office_2 on office_2.id = property_2.client_id
where serviceprice_2.price_context = 'CLIENT'
and serviceprice_2.price_type_id = 236
&& service_2.date_client_billed between '2012-09-01' and now()
and service_2.gl_code_ap='4700-70-000'
and service_2.date_cancelled is null
and office_2.id = office_1.id) as 'Client Price'
/* other calculations between here */
from cap.service service_1
inner join cap.service_area servicearea_1 on servicearea_1.id = service_1.service_area_id
inner join cap.property property_1 on property_1.id = servicearea_1.property_id
inner join cap.office office_1 on office_1.id = property_1.client_id
inner join cap.office vendor_1 on vendor_1.id = service_1.vendor_id
where
service_1.date_client_billed between '2012-09-01' and now()
and service_1.date_cancelled is null
and office_1.id = 26377
and service_1.gl_code_ap='4700-70-000'
group by office_1.id
;
Using views can simplify the query although there is no effect on performance.
create view view1 as
select *
from
cap.service_price serviceprice_2
inner join cap.service service_2 on service_2.id = serviceprice_2.service_id
inner join cap.service_area servicearea_2 on servicearea_2.id = service_2.service_area_id
inner join cap.property property_2 on property_2.id = servicearea_2.property_id
inner join cap.office office_2 on office_2.id = property_2.client_id
;
create view view2 as
select *
from
cap.service service_1
inner join cap.service_area servicearea_1 on servicearea_1.id = service_1.service_area_id
inner join cap.property property_1 on property_1.id = servicearea_1.property_id
inner join cap.office office_1 on office_1.id = property_1.client_id
inner join cap.office vendor_1 on vendor_1.id = service_1.vendor_id
Now the query is more manageable:
select #client_price := format(coalesce(sum(serviceprice_2.amount),0),2)
from view1
where serviceprice_2.price_context = 'CLIENT'
and serviceprice_2.price_type_id = 236
&& service_2.date_client_billed between '2012-09-01' and now()
and service_2.gl_code_ap='4700-70-000'
and service_2.date_cancelled is null
and office_2.id = office_1.id
;
select
office_1.name as 'Client'
,count(distinct(property_1.id)) as 'Total Billed Orders',
#client_price
/* other calculations between here */
from view2
where
service_1.date_client_billed between '2012-09-01' and now()
and service_1.date_cancelled is null
and office_1.id = 26377
and service_1.gl_code_ap='4700-70-000'
group by office_1.id