How to optimize MySQL query with multiple identical subqueries and multiple grouping - mysql

Question

Problem statement
The two identical sub-queries (i.e., "a" and "b") are used to derive sub-query "n" which I further aggregate to get the final result. The response time of the query is not optimal, can anyone share some ideas to help optimize? I tried to comebine "a" and "b" as well as "a" and "n" but neither turns out to be dead ends per my knowledge...
select n.businessLiaision,
n.channel,
n.name,
n.dt,
n.ifNew,
count(n.productRowId),
sum(n.totalQty),
sum(n.totalAmount)
from ( select a.productRowId,
a.name,
a.rowId,
a.dt,
a.businessLiaision,
a.channel,
a.ct,
a.totalQty,
a.totalAmount,
case when a.ct = sum(b.ct) then 'true' else 'false' end as 'ifNew'
from ( select d.productRowId,
p.name,
DATE_FORMAT(o.effectiveTime, '%m/%Y') as 'dt',
p.rowId,
p.businessLiaision,
p.channel,
count(*) as 'ct',
sum(d.qty) as 'totalQty',
sum(d.amountPostDiscount) as 'totalAmount'
from transactionParty as p
join transactionOrderHist as o on p.rowId = o.transactionPartyRowId
join transactionOrderDetailHist as d on o.rowId = d.orderRowId
where o.businessType = 'sales'
group by d.productRowId, p.name, DATE_FORMAT(o.effectiveTime, '%m/%Y'), p.rowId, p.businessLiaision, p.channel
) as a
left join ( select d.productRowId,
p.name,
DATE_FORMAT(o.effectiveTime, '%m/%Y') as 'dt',
count(*) as 'ct'
from transactionParty as p
join transactionOrderHist as o on p.rowId = o.transactionPartyRowId
join transactionOrderDetailHist as d on o.rowId = d.orderRowId
where o.businessType = 'sales'
group by d.productRowId, p.name, DATE_FORMAT(o.effectiveTime, '%m/%Y')
) as b on b.productRowId = a.productRowId and b.name = a.name and b.dt <= a.dt
group by a.productRowId, a.name, a.rowId, a.dt, a.ct, a.businessLiaision, a.channel, a.ct, a.totalQty, a.totalAmount
) as n
group by n.businessLiaision, n.channel, n.name, n.dt, n.ifNew
Explain plan result
enter image description here
Table descriptions
transactionParty
CREATE TABLE `transactionParty` (
`rowId` varchar(50) NOT NULL,
`name` varchar(100) DEFAULT NULL,
`code` varchar(20) DEFAULT NULL,
`businessLiaision` varchar(20) DEFAULT NULL,
`type` varchar(20) DEFAULT NULL,
`contractualType` varchar(20) DEFAULT NULL,
`paymentMethod` varchar(20) DEFAULT NULL,
`partyGroup` varchar(20) DEFAULT NULL,
`channel` varchar(20) DEFAULT NULL,
`costCenter` varchar(20) DEFAULT NULL,
`warehouseRowId` varchar(100) DEFAULT NULL,
`taxOption` varchar(20) DEFAULT NULL,
PRIMARY KEY (`rowId`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
transactionOrderHist
CREATE TABLE `transactionOrderHist` (
`rowId` varchar(50) NOT NULL COMMENT '明道记录ID',
`orderId` varchar(50) DEFAULT NULL COMMENT '单据ID',
`orderCreationTime` datetime DEFAULT NULL COMMENT '单据创建时间',
`orderCreator` varchar(20) DEFAULT NULL COMMENT '单据创建人',
`businessType` varchar(20) DEFAULT NULL COMMENT '业务类型',
`businessLiaision` varchar(20) DEFAULT NULL COMMENT '内部业务负责人，如业务员/采购/文员',
`transactionPartyRowId` varchar(50) DEFAULT NULL COMMENT '往来单位',
`outboundWarehouse` varchar(50) DEFAULT NULL COMMENT '发货仓',
`inboundWarehouse` varchar(50) DEFAULT NULL COMMENT '收货仓',
`outboundWarehouseType` varchar(50) DEFAULT NULL,
`inboundWarehouseType` varchar(50) DEFAULT NULL,
`effectiveTime` datetime DEFAULT '0000-00-00 00:00:00' COMMENT '单据过账时间',
`orderEffectuater` varchar(20) DEFAULT NULL COMMENT '单据过账人',
`remark` varchar(200) DEFAULT NULL,
`productCount` int(10) DEFAULT NULL,
`totalUnitCount` int(10) DEFAULT NULL,
`totalCostAmount` decimal(10,0) DEFAULT NULL,
`totalPostDiscountAmount` decimal(10,0) DEFAULT NULL,
`paymentStatus` varchar(50) DEFAULT NULL,
`paymentDate` datetime DEFAULT NULL,
`outboundWarehouseRowId` varchar(50) DEFAULT NULL,
`inboundWarehouseRowId` varchar(50) DEFAULT NULL,
PRIMARY KEY (`rowId`) USING BTREE,
KEY `orderEffectiveDate` (`effectiveTime`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
transactionOrderDetailHist
CREATE TABLE `transactionOrderDetailHist` (
`rowId` varchar(50) NOT NULL,
`orderRowId` varchar(50) DEFAULT NULL,
`productRowId` varchar(50) DEFAULT NULL,
`qty` int(20) DEFAULT NULL,
`price` decimal(20,2) DEFAULT NULL,
`cost` decimal(20,2) DEFAULT NULL,
`amount` decimal(20,2) DEFAULT NULL,
`amountPostDiscount` decimal(20,2) DEFAULT NULL,
`type` varchar(50) DEFAULT NULL,
`effectiveTime` datetime DEFAULT NULL,
`costAmount` decimal(20,2) DEFAULT NULL,
PRIMARY KEY (`rowId`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8

Answer 1

FROM ( SELECT ... )
[LEFT] JOIN ( SELECT ... ) ON ...
is notoriously inefficient. Try to rewrite the query to avoid such.
Please provide EXPLAIN SELECT ...; meanwhile, I will do some guessing.
These indexes may help:
d: INDEX(orderRowId, productRowId, qty, amountPostDiscount)
o: INDEX(businessType, transactionPartyRowId, effectiveTime, rowId)
Looking at the final GROUP BY. It does not necessarily order the rows. If you want a particular order, add an ORDER BY. Be aware that the dt part is not quite in chronological order since it has month before year. The other group-bys are in derived tables, so an ORDER BY would be ignored.
If and b.name = a.name and b.dt <= a.dt is for doing "groupwise-max", that is the performance bottleneck. There are much better ways to do it. See Groupwise-Max .
If the text is in Chinese, you should consider converting to utf8mb4 -- utf8 is missing the 4-byte UTF-8 Chinese characters.
It is OK to have the PRIMARY KEY as VARCHAR(50), but it could be better to use INT. Is some outside process providing the rowId?