MariaDB/PostreSQL - Query optimization - mysql

Question

I need to produce some numbers, I have designed a query to get the result desired by my "customers".
This query is based on a table that contains one million records.
I usually use MariaDB for that, and I get a result in ~ 7s.
This execution time is quite suitable but I am looking to optimize again to improve my skills.
After some research, I came across a few posts saying "MySQL is fine, but not on tables> 1M of records, you have to switch on something else" PostgreSQL has been quoted several times.
So I installed PostgreSQL, and copied my tables, indexes and data.
I executed the same query, and I had a result in ~ 12s
I know less PostgreSQL, I think I did not use the specificities inherent to the language.
So for now I stay on MariaDB. Do you have an idea to improve the execution time?
Here my query :
select categorie.cat
,dhu_type.type
,COUNT(DISTINCT(
CASE WHEN dhu.date between '2013-01-01' and '2013-12-31'
THEN dhu.id
END )
) AS "2013"
,COUNT(DISTINCT(
CASE WHEN dhu.date between '2014-01-01' and '2014-12-31'
THEN dhu.id
END )
) AS "2014"
,COUNT(DISTINCT(
CASE WHEN dhu.date between '2015-01-01' and '2015-12-31'
THEN dhu.id
END )
) AS "2015"
,COUNT(DISTINCT(
CASE WHEN dhu.date between '2016-01-01' and '2016-12-31'
THEN dhu.id
END )
) AS "2016"
from dhu
inner join dhu_type on dhu.type_id = dhu_type.id
inner join patient on dhu.patient_id=patient.id
inner join fa on patient.id = fa.patient_id
inner join categorie on categorie.id = fa.cat_id
group by cat,dhu_type.type
I complete my question with a diagram
Here the CREATE TABLE :
/*!40101 SET #OLD_CHARACTER_SET_CLIENT=##CHARACTER_SET_CLIENT */;
/*!40101 SET NAMES utf8 */;
/*!50503 SET NAMES utf8mb4 */;
/*!40014 SET #OLD_FOREIGN_KEY_CHECKS=##FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */;
/*!40101 SET #OLD_SQL_MODE=##SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */;
CREATE TABLE IF NOT EXISTS `categorie` (
`id` tinyint(3) unsigned NOT NULL AUTO_INCREMENT,
`cat` varchar(50) NOT NULL DEFAULT 'neonat',
PRIMARY KEY (`id`,`cat`)
) ENGINE=InnoDB AUTO_INCREMENT=16 DEFAULT CHARSET=utf8;
CREATE TABLE IF NOT EXISTS `cp` (
`id` smallint(5) unsigned NOT NULL AUTO_INCREMENT,
`cp` varchar(5) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `cp` (`cp`)
) ENGINE=InnoDB AUTO_INCREMENT=4096 DEFAULT CHARSET=utf8;
CREATE TABLE IF NOT EXISTS `dhu` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`patient_id` int(10) unsigned NOT NULL,
`date` date NOT NULL,
`type_id` tinyint(3) unsigned NOT NULL,
PRIMARY KEY (`id`),
KEY `FK_dhu_patient` (`patient_id`),
KEY `FK_dhu_dhu_type` (`type_id`),
CONSTRAINT `FK_dhu_dhu_type` FOREIGN KEY (`type_id`) REFERENCES `dhu_type` (`id`),
CONSTRAINT `FK_dhu_patient` FOREIGN KEY (`patient_id`) REFERENCES `patient` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=953590 DEFAULT CHARSET=utf8;
CREATE TABLE IF NOT EXISTS `dhu_import` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`noip` bigint(10) unsigned zerofill NOT NULL,
`date` date NOT NULL,
`cp` varchar(5) NOT NULL,
`type` varchar(4) NOT NULL,
PRIMARY KEY (`id`),
KEY `noip` (`noip`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE TABLE IF NOT EXISTS `dhu_type` (
`id` tinyint(3) unsigned NOT NULL AUTO_INCREMENT,
`type` varchar(4) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `type` (`type`)
) ENGINE=InnoDB AUTO_INCREMENT=8 DEFAULT CHARSET=utf8;
CREATE TABLE IF NOT EXISTS `dpt` (
`dpt` tinyint(3) unsigned DEFAULT NULL,
`abrev` char(3) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE TABLE IF NOT EXISTS `fa` (
`patient_id` int(10) unsigned NOT NULL,
`cat_id` tinyint(3) unsigned NOT NULL,
PRIMARY KEY (`patient_id`,`cat_id`),
KEY `idx_cat_id_pat_id` (`cat_id`,`patient_id`),
CONSTRAINT `FK_fa_patient` FOREIGN KEY (`patient_id`) REFERENCES `patient` (`id`),
CONSTRAINT `FK_fa_categorie` FOREIGN KEY (`cat_id`) REFERENCES `categorie` (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE TABLE IF NOT EXISTS `fa_import` (
`noip` bigint(10) unsigned zerofill NOT NULL,
`cat` varchar(50) NOT NULL,
PRIMARY KEY (`noip`,`cat`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=COMPACT;
CREATE TABLE IF NOT EXISTS `patient` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`noip` bigint(10) unsigned zerofill NOT NULL,
`cp_id` smallint(5) unsigned NOT NULL,
PRIMARY KEY (`id`),
KEY `FK_patient_cp` (`cp_id`),
CONSTRAINT `FK_patient_cp` FOREIGN KEY (`cp_id`) REFERENCES `cp` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=262141 DEFAULT CHARSET=utf8;
/*!40101 SET SQL_MODE=IFNULL(#OLD_SQL_MODE, '') */;
/*!40014 SET FOREIGN_KEY_CHECKS=IF(#OLD_FOREIGN_KEY_CHECKS IS NULL, 1, #OLD_FOREIGN_KEY_CHECKS) */;
/*!40101 SET CHARACTER_SET_CLIENT=#OLD_CHARACTER_SET_CLIENT */;
Here the explain query :
Here a modification improving performance (select categorie.id instead of categorie.cat):
Here the best best query I found thanks #RickJames & #BillKarwin
select categorie.cat
,dhu_type.`type`
,t.`2013`
,t.`2014`
,t.`2015`
,t.`2016`
from ( select fa.cat_id as catid
,dhu.type_id typid
,COUNT(DISTINCT(
CASE WHEN dhu.date between '2013-01-01' and '2013-12-31'
THEN dhu.id
END )
) AS "2013"
,COUNT(DISTINCT(
CASE WHEN dhu.date between '2014-01-01' and '2014-12-31'
THEN dhu.id
END )
) AS "2014"
,COUNT(DISTINCT(
CASE WHEN dhu.date between '2015-01-01' and '2015-12-31'
THEN dhu.id
END )
) AS "2015"
,COUNT(DISTINCT(
CASE WHEN dhu.date between '2016-01-01' and '2016-12-31'
THEN dhu.id
END )
) AS "2016"
from dhu
inner join patient on dhu.patient_id=patient.id
inner join fa on patient.id = fa.patient_id
group by fa.cat_id, dhu.type_id ) t
inner join categorie on t.catid = categorie.id
inner join dhu_type on t.typid = dhu_type.id
order by categorie.cat,dhu_type.`type`

Answer 1

MySQL does just fine with billion-row tables.
Any database engine is at the mercy of the speed of the disk and how much (or little) RAM you have for caching.
The textbooks say to normalize everything, but I suggest that a 4-char type is not worth normalizing. Ditto for the 5-char cp.
Unless you really want output rows with all zeros, add this WHERE dhu.date between '2016-01-01' and '2016-12-31' before the GROUP BY.
Follow my advice here on many:many schema design (fa). This may speed up the query for MySQL. (I don't know if the same principles apply to Postgres.)