This query takes around 2.23seconds and feels a bit slow ... is there anyway to make it faster.
our member.id, member_id, membership_id, valid_to, valid_from has index as well.
select *
from member
where (member.id in ( select member_id from member_membership mm
INNER JOIN membership m ON mm.membership_id = m.id
where instr(organization_chain, 2513) and m.valid_to > NOW() and m.valid_from < NOW() ) )
order by id desc
limit 10 offset 0
EXPLAIN FOR WHAT QUERY DOING: every member has many a member_memberships and and member_memberships connect with another table called membership there we have the membership details. so query will get all members that has valid memberships and where the organization id 2513 exist on member_membership.
Tables as following:
CREATE TABLE `member` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`first_name` varchar(255) DEFAULT NULL,
`last_name` varchar(255) DEFAULT NULL,
PRIMARY KEY (`id`),
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=latin1;
CREATE TABLE `member_membership` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`membership_id` int(11) DEFAULT NULL,
`member_id` int(11) DEFAULT NULL,
`organization_chain` text DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `member_membership_to_membership` (`membership_id`),
KEY `member_membership_to_member` (`member_id`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=latin1;
CREATE TABLE `membership` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(255) DEFAULT NULL,
`valid_to` datetime DEFAULT NULL,
`valid_from` datetime DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `valid_to` (`valid_to`),
KEY `valid_from` (`valid_from`),
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=latin1;
ALTER TABLE `member_membership` ADD CONSTRAINT `member_membership_to_membership` FOREIGN KEY (`membership_id`) REFERENCES `membership` (`id`);
ALTER TABLE `member_membership` ADD CONSTRAINT `member_membership_to_member` FOREIGN KEY (`member_id`) REFERENCES `member` (`id`);
Here with EXPLAIN statement => https://i.ibb.co/xjrcYWR/EXPLAIN.png
Relations
member has many member_membership
membership has manymember_membership
So member_membership is like join for tables member and membership.
Well I found a way to make it less to 800ms ... like this. Is this good way or maybe there is more we can do?
select *
from member
where (member.id in ( select member_id from member_membership mm FORCE INDEX (PRIMARY)
INNER JOIN membership m ON mm.membership_id = m.id
where instr(organization_chain, 2513) and m.valid_to > NOW() and m.valid_from < NOW() ) )
order by id desc
limit 10 offset 0
NEW UPDATE.. and I think this solve the issue.. 15ms :)
I added FORCE INDEX..
The FORCE INDEX hint acts like USE INDEX (index_list), with the addition that a table scan is assumed to be very expensive. In other words, a table scan is used only if there is no way to use one of the named indexes to find rows in the table.
select *
from member
where (member.id in ( select member_id from member_membership mm FORCE INDEX (member_membership_to_member)
INNER JOIN membership m FORCE INDEX (organization_to_membership) ON mm.membership_id = m.id
where instr(organization_chain, 2513) and m.valid_to > NOW() and m.valid_from < NOW() ) )
order by id desc
limit 10 offset 0
How big is organization_chain? If you don't need TEXT, use a reasonably sized VARCHAR so that it could be in an index. Better yet, is there some way to get 2513 in a column by itself?
Don't use id int(11) NOT NULL AUTO_INCREMENT, in a many-to-many table; rather have the two columns in PRIMARY KEY.
Put the ORDER BY and LIMIT in the subquery.
Don't use IN ( SELECT ...), use a JOIN.
I have this query that I think I have indexed them properly. But still get filesort and temporary indexing.
The query is as follow:
SELECT * FROM
(SELECT PIH.timestamp, PIH.practice_id, PIH.timestamp as invoice_num, PIH.custom_invnum,
CEIL(PIH.total_invoice + PIH.tax + PIH.other_bill) as grand_total, PIH.total_invoice, PIH.extra_charge_ph as extra_charge,
PIH.tax, PIH.other_bill, PIH.changed, PIH.source,
PIH.notes, PIH.is_active, PIH.paid as pay,
PIH.covered_amount, IF(PIH.is_active = 1, IF(PIH.total_invoice = 0 OR PIH.total_invoice + PIH.tax + PIH.other_bill - PIH.covered_amount <= PIH.paid, 1, IF(PIH.paid = 0, 0, 2)), '') as invoice_st,
RPP.patient_id, RPP.first_name as pfname, RPP.last_name as plname, RPP.dob as p_dob, RPP.gender as p_gender, RPP.reff_id as p_reff_id, RPP.mobile_number as p_mobile, IF(PIH.group_doctors IS NOT NULL, NULL, D.doc_title) as doc_title, IF(PIH.group_doctors IS NOT NULL,
PIH.group_doctors, D.first_name) as doc_fname, IF(PIH.group_doctors IS NOT NULL, PIH.group_doctors, D.last_name) as doc_lname, IF(PIH.group_doctors IS NOT NULL, NULL, D.spc_dsg) as spc_dsg, PA.username, TL.timestamp as checkout_time, IP.name as ip_name, PMM.timestamp as mcu_id
FROM practice_invoice_header PIH
INNER JOIN practice_invoice_detail PID ON PID.timestamp = PIH.timestamp
AND PID.practice_id = PIH.practice_id
INNER JOIN practice_queue_list PQL ON PQL.encounter_id = PID.encounter_id
AND PQL.practice_place_id = PIH.practice_id
INNER JOIN temp_search_view D ON D.id = PQL.doctor_id
AND D.pp_id = PQL.practice_place_id
INNER JOIN practice_place PP ON PP.id = PIH.practice_id
INNER JOIN ref_practice_patient RPP ON RPP.patient_id = PIH.patient_id
AND RPP.practice_id = PP.parent_id
LEFT JOIN practice_mcu_module PMM ON PMM.id = PID.mcu_module_id
AND PMM.practice_id = PID.practice_id
LEFT JOIN transaction_log TL ON TL.reff_id = PIH.timestamp
AND TL.practice_id = PIH.practice_id
AND TL.activity = "CHK"
LEFT JOIN practice_admin PA ON PA.id = TL.admin_id
LEFT JOIN insurance_plan IP ON IP.id = PIH.insurance_plan_id
WHERE PIH.source <> 'P'
AND PIH.practice_id = 28699
AND PIH.is_active = 1
AND PQL.cal_id >= 201807010
AND PQL.cal_id <= 201807312
GROUP BY PIH.timestamp, PIH.practice_id
) AS U LIMIT 0,20
NOTE: I only show some of the main tables that are used in this query and the ones that sort using filesort/temporary, of course If I post everything it will be too many information.
The query is about list of invoices, and it has the header (practice_invoice_header) and the details (practice_invoice_detail). And this query join with the practice_place table
CREATE TABLE `practice_invoice_header` (
`timestamp` bigint(20) NOT NULL,
`practice_id` int(11) NOT NULL,
`cal_id` int(11) NOT NULL,
`patient_id` int(11) NOT NULL DEFAULT 0,
`source` char(1) NOT NULL COMMENT 'E = ENCOUNTER; P = OTHER (PHARM / LAB)',
`total_invoice` float(30,2) NOT NULL DEFAULT 0.00,
`tax` float(30,2) NOT NULL DEFAULT 0.00,
`other_bill` float(30,2) NOT NULL DEFAULT 0.00,
`changed` float(30,2) NOT NULL DEFAULT 0.00,
`paid` float(30,2) NOT NULL DEFAULT 0.00,
`covered_amount` float(30,2) NOT NULL DEFAULT 0.00,
`notes` varchar(300) DEFAULT NULL,
`custom_invnum` varchar(30) DEFAULT NULL,
`insurance_plan_id` varchar(20) DEFAULT NULL,
`is_active` int(11) NOT NULL DEFAULT 1,
`cancel_reason` varchar(200) DEFAULT NULL,
PRIMARY KEY (`timestamp`,`practice_id`),
KEY `custom_invnum` (`custom_invnum`),
KEY `insurance_plan_id` (`insurance_plan_id`),
KEY `practice_id_3` (`practice_id`,`xxx_reff_id`),
KEY `ph_check_status` (`ph_checked_by`),
KEY `cal_id` (`cal_id`),
KEY `practice_id_5` (`practice_id`,`outpx_id`),
KEY `practice_id_6` (`practice_id`,`cal_id`,`source`,`is_active`),
KEY `total_invoice` (`total_invoice`),
KEY `patient_id` (`patient_id`),
CONSTRAINT `practice_invoice_header_ibfk_1` FOREIGN KEY (`practice_id`)
REFERENCES `practice_place` (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
CREATE TABLE `practice_invoice_detail` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`timestamp` bigint(20) NOT NULL,
`practice_id` int(11) NOT NULL,
`item_id` int(11) NOT NULL,
`item_sub_id` int(11) DEFAULT NULL,
`item_type` char(1) NOT NULL COMMENT 'D = DRUG; P = PROCEDURE; L = LAB',
`item_qty` float NOT NULL,
`item_price` float(22,2) NOT NULL,
`discount` float NOT NULL DEFAULT 0,
`is_active` int(11) NOT NULL DEFAULT 1,
PRIMARY KEY (`id`),
KEY `item_type` (`item_type`),
KEY `timestamp` (`timestamp`,`practice_id`),
KEY `practice_id` (`practice_id`),
KEY `item_id_2` (`item_id`,`item_sub_id`,`item_type`),
KEY `timestamp_2` (`timestamp`,`practice_id`,`item_id`,`item_sub_id`,`item_type`),
KEY `practice_id_3` (`practice_id`,`item_type`),
KEY `the_id` (`id`,`practice_id`) USING BTREE,
KEY `timestamp_3` (`timestamp`,`practice_id`,`item_type`,`item_comission`,
`item_comission_type`, `doctor_id`,`item_id`,`item_sub_id`,`id`) USING BTREE,
KEY `timestamp_4` (`timestamp`,`practice_id`,`item_id`,`item_sub_id`,`item_type`,
`item_comission_2`,`item_comission_2_type`,`doctor_id_2`,`id`) USING BTREE,
KEY `request_id` (`request_id`,`request_practice`),
KEY `timestamp_5` (`timestamp`,`practice_id`,`is_active`),
KEY `practice_id_6` (`practice_id`,`encounter_id`,`is_active`),
KEY `practice_id_7` (`practice_id`,`item_type`,`encounter_id`,`is_active`),
CONSTRAINT `practice_invoice_detail_ibfk_1` FOREIGN KEY (`timestamp`)
REFERENCES `practice_invoice_header` (`timestamp`) ON DELETE CASCADE,
CONSTRAINT `practice_invoice_detail_ibfk_2` FOREIGN KEY (`practice_id`)
REFERENCES `practice_place` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=1447348 DEFAULT CHARSET=latin1
CREATE TABLE `ref_practice_patient` (
`practice_id` int(11) NOT NULL,
`patient_id` int(11) NOT NULL,
`reff_id` varchar(35) DEFAULT NULL,
`is_user` int(11) NOT NULL DEFAULT 0,
`parent_user_id` int(11) NOT NULL DEFAULT 0
PRIMARY KEY (`practice_id`,`patient_id`),
KEY `patient_id` (`patient_id`),
KEY `reff_id` (`reff_id`),
KEY `practice_id` (`practice_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
CREATE TABLE `practice_place` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(75) NOT NULL,
`statement` text DEFAULT NULL,
`address` varchar(200) NOT NULL,
`phone` varchar(15) NOT NULL,
`wa_number` varchar(15) DEFAULT NULL,
`fax` varchar(15) NOT NULL,
`email` varchar(50) NOT NULL,
`is_branch` int(11) NOT NULL,
`parent_id` int(11) NOT NULL,
`editted_by` int(11) DEFAULT NULL,
`editted_date` bigint(20) DEFAULT NULL,
`status` int(11) NOT NULL DEFAULT 1,
PRIMARY KEY (`id`),
KEY `parent_id` (`parent_id`),
KEY `reff_id` (`reff_id`),
) ENGINE=InnoDB AUTO_INCREMENT=29058 DEFAULT CHARSET=latin1
And below is the explain produce by the query, and I highlight the one using filsort (no. 2)
1 PRIMARY ALL NULL NULL NULL NULL 14028
2 DERIVED PP const PRIMARY,parent_id PRIMARY 4 const 1 Using temporary; Using filesort
2 DERIVED PIH ref PRIMARY,practice_id_3,practice_id_5,practice_id_6,practice_id_8,pharm_read,lab_read,rad_read,patient_id
practice_id_5 4 const 7014 Using where
2 DERIVED RPP eq_ref PRIMARY,patient_id,practice_id,practice_id_2,practice_id_3
PRIMARY 8 const,k6064619_lokadok.PIH.patient_id 1
2 DERIVED PID ref timestamp,practice_id,timestamp_2,practice_id_2,practice_id_3,timestamp_3,timestamp_4,practice_id_4,practice_id_5,timestamp_5,practice_id_6,practice_id_7
timestamp 12 k6064619_lokadok.PIH.timestamp,const 1
2 DERIVED PMM eq_ref PRIMARY,id,practice_id
PRIMARY 4 k6064619_lokadok.PID.mcu_module_id 1 Using where
2 DERIVED TL ref reff_id reff_id 12 k6064619_lokadok.PIH.timestamp,const 1 Using where
2 DERIVED PA eq_ref PRIMARY PRIMARY 4 k6064619_lokadok.TL.admin_id 1 Using where
2 DERIVED IP ref PRIMARY,id PRIMARY 22 k6064619_lokadok.PIH.insurance_plan_id 1 Using where
2 DERIVED PQL ref PRIMARY,encounter_id,cal_id_2
encounter_id 5 k6064619_lokadok.PID.encounter_id 2 Using where; Using index
2 DERIVED D ref doc_id,pp_id,id_2,pp_doc doc_id 4 k6064619_lokadok.PQL.doctor_id 1 Using where
I believe I have indexed the parent_id in practice_place table, and also in ref_practice_patient the patient_id and practice_id is PRIMARY.
Why have the outer query? The Optimizer is free to shuffle the result of the inner query, thereby leaving the LIMIT to pick an ordering that you not expecting. At least add ORDER BY, preferably also toss the outer select.
Main Index
Let's analyze the likely place to design an index:
WHERE PIH.source <> 'P'
AND PIH.practice_id = 28699
AND PIH.is_active = 1
AND PQL.cal_id >= 201807010
AND PQL.cal_id <= 201807312
GROUP BY PIH.timestamp, PIH.practice_id
Since there is a mixture of tables involved, it is not possible to have an index that handles all the WHERE.
Since the tests are not all =, it is not possible to reach beyond the WHERE and include columns of the GROUP BY.
So, I see two indexes:
PIH: INDEX(practice_id, is_active, -- in either order
source)
PQL: INDEX(cal_id)
Since we can't get into the GROUP BY, the Optimizer has no choice but to gather all the rows based on WHERE, do some grouping, and do an ORDER BY (as I said, that is missing, but necessary).
Therefor, GROUP BY and the ORDER BY will require one or two temps and filesorts. No, you can't get away from it, at least not without changing the query in some way. (Please note that "filesort" might actually be done in RAM.)
Your extra SELECT layer may be adding an extra temp and filesort.
EXPLAIN fails to point out when there are two sorts. EXPLAIN FORMAT=JSON has that sort of detail.
Other issues...
Having a timestamp in a PRIMARY KEY is risky unless you are sure that two rows can occur with the same timestamp, or there is another column in the PK to assure uniqueness.
Don't use FLOAT for money. It will incur extra rounding errors, and it cannot store more than about 7 significant digits (that' less than $100K to the penny). Don't use float(30,2), it is even worse because you are forcing an extra rounding. Use DECIMAL(30,2), but pick something reasonable, not 30. It takes 14 bytes -- mostly a waste of space.
Whenever you have INDEX(a,b), you don't need INDEX(a); it is redundant and slows down (slightly) INSERTs.
LEFT JOIN transaction_log TL
ON TL.reff_id = PIH.timestamp
AND TL.practice_id = PIH.practice_id
AND TL.activity = "CHK"
needs
INDEX(reff_id, practice_id, activity) -- in any order
Also
INNER JOIN practice_invoice_detail PID ON PID.timestamp = PIH.timestamp
AND PID.practice_id = PIH.practice_id
PIH: INDEX(practice_id, timestamp) -- not the opposite order
PIH: INDEX(practice_id, is_active, timestamp)
INNER JOIN practice_queue_list PQL ON PQL.encounter_id = PID.encounter_id
AND PQL.practice_place_id = PIH.practice_id
PQL: INDEX(encounter_id, cal_id)
PQL: INDEX(encounter_id, practice_place_id, cal_id)
Some discussion...
In a JOIN, EXPLAIN shows one order of working through the tables; it gives you no clues of how things would be if it worked through the tables some other way.
I have attempted to show what index might be needed if PQL were used first, or if PIH were used first -- namely use the WHERE stuff for that table, then
I have attempted to show the optimial index for joining to the other table.
Probably the Optimizer will not start with any table not mentioned in the WHERE clause, but this is not a certainty.
I have not listed the optimal indexes for getting to each of the other tables.
More discussion: http://mysql.rjweb.org/doc.php/index_cookbook_mysql
I have a query that is executed in 35s, which is waaaaay too long.
Here are the 3 tables concerned by the query (each table is approx. 13000 lines long, and should be much longer in the future) :
Table 1 : Domains
CREATE TABLE IF NOT EXISTS `domain` (
`id_domain` int(11) NOT NULL AUTO_INCREMENT,
`domain_domain` varchar(255) NOT NULL,
`projet_domain` int(11) NOT NULL,
`date_crea_domain` int(11) NOT NULL,
`date_expi_domain` int(11) NOT NULL,
`active_domain` tinyint(1) NOT NULL,
`remarques_domain` text NOT NULL,
PRIMARY KEY (`id_domain`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Table 2 : Keywords
CREATE TABLE IF NOT EXISTS `kw` (
`id_kw` int(11) NOT NULL AUTO_INCREMENT,
`kw_kw` varchar(255) NOT NULL,
`clics_kw` int(11) NOT NULL,
`cpc_kw` float(11,3) NOT NULL,
`date_kw` int(11) NOT NULL,
PRIMARY KEY (`id_kw`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Table 3 : Linking between domain and keyword
CREATE TABLE IF NOT EXISTS `kw_domain` (
`id_kd` int(11) NOT NULL AUTO_INCREMENT,
`kw_kd` int(11) NOT NULL,
`domain_kd` int(11) NOT NULL,
`selected_kd` tinyint(1) NOT NULL,
PRIMARY KEY (`id_kd`),
KEY `kw_to_domain` (`kw_kd`,`domain_kd`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
The query is as follows :
SELECT ng.*, kd.*, kg.*
FROM domain ng
LEFT JOIN kw_domain kd ON kd.domain_kd = ng.id_domain
LEFT JOIN kw kg ON kg.id_kw = kd.kw_kd
GROUP BY ng.id_domain
ORDER BY kd.selected_kd DESC, kd.id_kd DESC
Basically, it selects all domains, with, for each one of these domains, the last associated keyword.
Does anyone have an idea on how to optimize the tables or the query ?
The following will get the last keyword, according to your logic:
select ng.*,
(select kw_kd
from kw_domain kd
where kd.domain_kd = ng.id_domain and kd.selected_kd = 1
order by kd.id_kd desc
limit 1
) as kw_kd
from domain ng;
For performance, you want an index on kw_domain(domain_kd, selected_kd, kw_kd). In this case, the order of the fields matters.
You can use this as a subquery to get more information about the keyword:
select ng.*, kg.*
from (select ng.*,
(select kw_kd
from kw_domain kd
where kd.domain_kd = ng.id_domain and kd.selected_kd = 1
order by kd.id_kd desc
limit 1
) as kw_kd
from domain ng
) ng left join
kw kg
on kg.id_kw = ng.kw_kd;
In MySQL, group by can have poor performance, so this might work better, particularly with the right indexes.
I want to subtract between two rows of different table:
I have created a view called leave_taken and table called leave_balance.
I want this result from both table:
leave_taken.COUNT(*) - leave_balance.balance
and group by leave_type_id_leave_type
Code of both table
-----------------View Leave_Taken-----------
CREATE ALGORITHM = UNDEFINED DEFINER=`1`#`localhost` SQL SECURITY DEFINER
VIEW `leave_taken`
AS
select
`leave`.`staff_leave_application_staff_id_staff` AS `staff_leave_application_staff_id_staff`,
`leave`.`leave_type_id_leave_type` AS `leave_type_id_leave_type`,
count(0) AS `COUNT(*)`
from
(
`leave`
join `staff` on((`staff`.`id_staff` = `leave`.`staff_leave_application_staff_id_staff`))
)
where (`leave`.`active` = 1)
group by `leave`.`leave_type_id_leave_type`;
----------------Table leave_balance----------
CREATE TABLE IF NOT EXISTS `leave_balance` (
`id_leave_balance` int(11) NOT NULL AUTO_INCREMENT,
`staff_id_staff` int(11) NOT NULL,
`leave_type_id_leave_type` int(11) NOT NULL,
`balance` int(3) NOT NULL,
`date_added` date NOT NULL,
PRIMARY KEY (`id_leave_balance`),
UNIQUE KEY `id_leave_balance_UNIQUE` (`id_leave_balance`),
KEY `fk_leave_balance_staff1` (`staff_id_staff`),
KEY `fk_leave_balance_leave_type1` (`leave_type_id_leave_type`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=3 ;
------- Table leave ----------
CREATE TABLE IF NOT EXISTS `leave` (
`id_leave` int(11) NOT NULL AUTO_INCREMENT,
`staff_leave_application_id_staff_leave_application` int(11) NOT NULL,
`staff_leave_application_staff_id_staff` int(11) NOT NULL,
`leave_type_id_leave_type` int(11) NOT NULL,
`date` date NOT NULL,
`active` int(11) NOT NULL DEFAULT '1',
`date_updated` date NOT NULL,
PRIMARY KEY (`id_leave`,`staff_leave_application_id_staff_leave_application`,`staff_leave_application_staff_id_staff`),
KEY `fk_table1_leave_type1` (`leave_type_id_leave_type`),
KEY `fk_table1_staff_leave_application1` (`staff_leave_application_id_staff_leave_application`,`staff_leave_application_staff_id_staff`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=32 ;
Well, I still don't think you've provided enough information. It would be very helpful to have some sample data and your expected output (in tabular format). That said, I may have something you can start working with. This query finds all staff members, calculates their current leave (grouped by type), and determines the difference between that and their balance by leave type. Take a look at it, and more importantly (perhaps) the sqlfiddle here that I used which has the sample data in it (very important to determining if this is the correct path for your data).
SELECT
staff.id_staff,
staff.name,
COUNT(`leave`.id_leave) AS leave_count,
leave_balance.balance,
(COUNT(`leave`.id_leave) - leave_balance.balance) AS leave_difference,
`leave`.leave_type_id_leave_type AS leave_type
FROM
staff
JOIN `leave` ON staff.id_staff = `leave`.staff_leave_application_staff_id_staff
JOIN leave_balance ON
(
staff.id_staff = leave_balance.staff_id_staff
AND `leave`.leave_type_id_leave_type = leave_balance.leave_type_id_leave_type
)
WHERE
`leave`.active = 1
GROUP BY
staff.id_staff, leave_type;
Good luck!
Looking at this query there's got to be something bogging it down that I'm not noticing. I ran it for 7 minutes and it only updated 2 rows.
//set product count for makes
$tru->query->run(array(
'name' => 'get-make-list',
'sql' => 'SELECT id, name FROM vehicle_make',
'connection' => 'core'
));
while($tempMake = $tru->query->getArray('get-make-list')) {
$tru->query->run(array(
'name' => 'update-product-count',
'sql' => 'UPDATE vehicle_make SET product_count = (
SELECT COUNT(product_id) FROM taxonomy_master WHERE v_id IN (
SELECT id FROM vehicle_catalog WHERE make_id = '.$tempMake['id'].'
)
) WHERE id = '.$tempMake['id'],
'connection' => 'core'
));
}
I'm sure this query can be optimized to perform better, but I can't think of how to do it.
vehicle_make = 45 rows
taxonomy_master = 11,223 rows
vehicle_catalog = 5,108 rows
All tables have appropriate indexes
UPDATE: I should note that this is a 1-time script so overhead isn't a big deal as long as it runs.
CREATE TABLE IF NOT EXISTS `vehicle_make` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(32) NOT NULL,
`product_count` int(11) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=46 ;
CREATE TABLE IF NOT EXISTS `taxonomy_master` (
`product_id` int(10) NOT NULL,
`v_id` int(10) NOT NULL,
`vehicle_requirement` varchar(255) DEFAULT NULL,
`is_sellable` enum('True','False') DEFAULT 'True',
`programming_override` varchar(25) DEFAULT NULL,
PRIMARY KEY (`product_id`,`v_id`),
KEY `idx2` (`product_id`),
KEY `idx3` (`v_id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
CREATE TABLE IF NOT EXISTS `vehicle_catalog` (
`v_id` int(10) NOT NULL,
`id` int(11) NOT NULL,
`v_make` varchar(255) NOT NULL,
`make_id` int(11) NOT NULL,
`v_model` varchar(255) NOT NULL,
`model_id` int(11) NOT NULL,
`v_year` varchar(255) NOT NULL,
PRIMARY KEY (`v_id`,`v_make`,`v_model`,`v_year`),
UNIQUE KEY `idx` (`v_make`,`v_model`,`v_year`),
UNIQUE KEY `idx2` (`v_id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
Update: The successful query to get what I needed is here....
SELECT
m.id,COUNT(t.product_id) AS CountOf
FROM taxonomy_master t
INNER JOIN vehicle_catalog v ON t.v_id=v.id
INNER JOIN vehicle_make m ON v.make_id=m.id
GROUP BY m.id;
without the tables/columns this is my best guess from reverse engineering the given queries:
UPDATE m
SET product_count =COUNT(t.product_id)
FROM taxonomy_master t
INNER JOIN vehicle_catalog v ON t.v_id=v.id
INNER JOIN vehicle_make m ON v.make_id=m.id
GROUP BY m.name
The given code loops over each make, and then runs a query the counts for each. My answer just does them all in one query and should be a lot faster.
have an index for each of these:
vehicle_make.id cover on name
vehicle_catalog.id cover make_id
taxonomy_master.v_id
EDIT
give this a try:
CREATE TEMPORARY TABLE CountsOf (
id int(11) NOT NULL
, CountOf int(11) NOT NULL DEFAULT 0.00
);
INSERT INTO CountsOf
(id, CountOf )
SELECT
m.id,COUNT(t.product_id) AS CountOf
FROM taxonomy_master t
INNER JOIN vehicle_catalog v ON t.v_id=v.id
INNER JOIN vehicle_make m ON v.make_id=m.id
GROUP BY m.id;
UPDATE taxonomy_master,CountsOf
SET taxonomy_master.product_count=CountsOf.CountOf
WHERE taxonomy_master.id=CountsOf.id;
instead of using nested query ,
you can separated this query to 2 or 3 queries,
and in php insert the result of the inner query to the out query ,
its faster !
#haim-evgi Separating the queries will not increase the speed significantly, it will just shift the load from the DB server to the Web server and create overhead of moving data between the two servers.
I am not sure with the appropriate indexes you run such query 7 minutes. Could you please show the table structure of the tables involved in these queries.
Seems like you need the following indices:
INDEX BTREE('make_id') on vehicle_catalog
INDEX BTREE('v_id') on taxonomy_master