MySQL query optimization for extremly slow query - mysql

Question

I'm a web developer and I'm posting for the first time on SO.
Today I'm asking for your help because I already tried all the possibilities with no luck.
I created a SAAS web application that is used by salesman on the ground, it includes an offline version where users don't need to be connected to use it.
As the database is getting bigger, the queries are taking more and more time to be executed.
Today I'm facing a big issue where the query is leading to a timeout when the user tries to display it's result.
So, here's the dump :
SET SQL_MODE = "NO_AUTO_VALUE_ON_ZERO";
SET AUTOCOMMIT = 0;
START TRANSACTION;
SET time_zone = "+00:00";
-- 86 rows
CREATE TABLE t2 (
id_t2 int(11) NOT NULL,
quantite_t2 int(11) NOT NULL,
ca_t2 decimal(10,2) NOT NULL,
date_t2 date NOT NULL,
import_t2 datetime NOT NULL,
id_enseigne int(11) NOT NULL,
id_t3 int(11) NOT NULL,
annee_t2 int(11) NOT NULL,
mois_t2 int(11) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
-- 2012065 rows
CREATE TABLE t1 (
id_t2 int(11) NOT NULL,
id_t0 bigint(20) NOT NULL,
id_t4 bigint(20) NOT NULL,
quantite_t1 int(11) NOT NULL,
ca_t1 decimal(10,2) NOT NULL,
pvc_moyen_t1 float NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
-- 388 rows
CREATE TABLE t4 (
id_t4 int(11) NOT NULL,
lib_t4 text NOT NULL,
libcourt_t4 varchar(255) NOT NULL,
ean_t4 varchar(255) NOT NULL,
pcb_t4 int(11) NOT NULL,
pcb2_t4 int(11) NOT NULL,
fam_t4 varchar(255) NOT NULL,
gam_t4 varchar(255) NOT NULL DEFAULT '0',
stat_t4 int(11) NOT NULL DEFAULT 1,
vmh_t4 decimal(10,2) NOT NULL,
detail_t4 text NOT NULL,
ingr_t4 text NOT NULL,
weight_t4 float NOT NULL,
lifetime_t4 varchar(255) NOT NULL,
pmc1_t4 float NOT NULL,
pmc2_t4 float NOT NULL,
dim_t4 decimal(10,2) NOT NULL,
ordre_t4 int(11) NOT NULL,
created_t4 datetime NOT NULL,
updated_t4 datetime NOT NULL,
updated_img_t4 datetime NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
-- 1 row
CREATE TABLE t3 (
id_t3 int(11) NOT NULL,
nom_t3 text NOT NULL,
stat_t3 int(11) NOT NULL,
created_t3 datetime NOT NULL,
deleted_t3 datetime NOT NULL,
updated_t3 datetime NOT NULL,
ip_create_t3 text NOT NULL,
ip_delete_t3 text NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
ALTER TABLE t2
ADD PRIMARY KEY (id_t2),
ADD KEY annee_t2 (annee_t2,mois_t2,date_t2);
ALTER TABLE t1
ADD PRIMARY KEY (id_t2,id_t0,id_t4);
ALTER TABLE t4
ADD PRIMARY KEY (id_t4,ean_t4),
ADD KEY ean_t4 (ean_t4),
ADD KEY id_t4 (id_t4);
ALTER TABLE t3
ADD PRIMARY KEY (id_t3);
ALTER TABLE t2
MODIFY id_t2 int(11) NOT NULL AUTO_INCREMENT;
ALTER TABLE t4
MODIFY id_t4 int(11) NOT NULL AUTO_INCREMENT;
ALTER TABLE t3
MODIFY id_t3 int(11) NOT NULL AUTO_INCREMENT;
COMMIT;
The executed query below takes like 4 min to execute :
-- execution time 228 seconds
SELECT SUM(t1.ca_t1) AS ca_t4, SUM(t1.quantite_t1) AS qte_t4,
t4.fam_t4, t4.gam_t4, t4.lib_t4, t4.ean_t4, t4.id_t4,
t2.annee_t2, t2.mois_t2, COUNT(t1.id_t0) AS count_mag,
t3.id_t3, t3.nom_t3
FROM t1 t1
INNER JOIN t2 t2 ON t2.id_t2 = t1.id_t2
LEFT JOIN t3 t3 ON t2.id_t3 = t3.id_t3
INNER JOIN t4 t4 ON t1.id_t4 = t4.ean_t4
WHERE t2.date_t2 BETWEEN "2017-05-01" AND "2019-05-01"
GROUP BY t2.annee_t2, t2.mois_t2, t4.id_t4
ORDER BY ca_t4 DESC;
I tried all optimization I know to help me reduce execution time but no success...
The EXPLAIN shows this :
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t2 ALL PRIMARY NULL NULL NULL 86 Using where; Using temporary; Using filesort
1 SIMPLE t3 eq_ref PRIMARY PRIMARY 4 db.t2.id_t3 1
1 SIMPLE t1 ref PRIMARY,id_t2 PRIMARY 4 db.t2.id_t2 11266
1 SIMPLE t4 ALL ean_t4 NULL NULL NULL 388 Using where; Using join buffer (flat, BNL join)
Thank you for your help guys.

Answer 1

This looks like a fairly complex query, with multiple places that will slow it down. However, the first thing I notice is that a double lookup would have to occur in order to use the annee_2 index, which is probably why it is not using it.
Try adding id_t3 to the end of that index on table t2:
(annee_t2,mois_t2,date_t2,id_t3)
This should permit the optimize to use that index.
Run the query again (twice, to populate the buffer cache, report only the 2nd result) and if it doesn't improve sufficiently, post the new EXPLAIN plan.

Answer 2

The GROUP BY is probably improper since it does not include the t3 columns that are not aggergated.
Do you really want 2 years plus 1 day? Perhaps use this:
t2.date_t2 >= "2017-05-01"
AND t2.date_t2 < "2017-05-01" + INTERVAL 2 YEAR
Do not mix datatypes when JOINing -- ON t1.id_t4 = t4.ean_t4:
ean_t4 varchar(255) NOT NULL,
id_t4 bigint(20) NOT NULL,
(There may be other issues, but these should help.)