Optimizing a MySQL query on big tables - mysql

Question

I have a SQL query with 3 tables joined on a distant MySQL DB- Two of these tables have size of about 18GByte (STEP_RESULT and meas_numericlimit) and then the distant server create a TMP table which takes age (about 25 min) to end
How can I optimize this query ?
select
t1.UUT_NAME,
t1.STATION_NUM,
t1.START_DATE_TIME,
t3.LOW_LIMIT,
t3.DATA,
t3.HIGH_LIMIT,
t3.UNITS,
t2b.STEP_NAME
from
meas_numericlimit t3
inner join STEP_RESULT t2a on t3.ID = t2a.STEP_ID
inner join STEP_RESULT t2b on t2a.STEP_PARENT = t2b.STEP_ID
inner join uut_result t1 on t2b.UUT_RESULT = t1.ID
where
t1.UUT_NAME like 'Variable1-1%' and
t1.STATION_NUM = 'variable2' and
t2b.STEP_NAME = 'variable3' and
t2b.STEP_TYPE = 'constant'
Here the SHOW TABLES and EXPLAIN output queries :
+--------------------+
| Tables_in_spectrum |
+--------------------+
| cal_dates |
| calibrage |
| execution_time |
| meas_numericlimit |
| station_feature |
| step_callexe |
| step_graph |
| step_msgjnl |
| step_msgpopup |
| step_passfail |
| step_result |
| step_seqcall |
| step_stringvalue |
| syst_event |
| uptime |
| users |
| uut_result |
+--------------------+
and
+----+-------------+-------+--------+-------------------------+---------+
| id | select_type | table | type | possible_keys | key |
+----+-------------+-------+--------+-------------------------+---------+
| 1 | SIMPLE | t2a | ALL | NULL | NULL |
| 1 | SIMPLE | t3 | eq_ref | PRIMARY | PRIMARY |
| 1 | SIMPLE | t2b | ALL | NULL | NULL |
| 1 | SIMPLE | t1 | eq_ref | PRIMARY,FK_uut_result_1 | PRIMARY |
+----+-------------+-------+--------+-------------------------+---------+
---------+----------------------+----------- +---------------------------+
key_len | ref | rows | Extra |
---------+----------------------+----------- +---------------------------+
NULL | NULL | 48120004 | |
40 | spectrum.t2a.STEP_ID | 1 | |
NULL | NULL | 48120004 | Using where; Using join
buffer |
40 | spectrum.t2b.UUT_RESULT | 1 | Using where |
-------+----------------------+------------+---------------------------+
Here the SHOW CREATE TABLE :
CREATE TABLE `uut_result` (
`ID` varchar(38) NOT NULL DEFAULT '',
`STATION_NUM` varchar(255) DEFAULT NULL,
`SOFTVER_ODTGEN` varchar(10) DEFAULT NULL,
`HARDVER_ODTGEN` varchar(10) DEFAULT NULL,
`NEXT_CAL_DATE` date DEFAULT NULL,
`UUT_NAME` varchar(255) DEFAULT NULL,
`UUT_SERIAL_NUMBER` varchar(255) DEFAULT NULL,
`UUT_VERSION` varchar(255) DEFAULT NULL,
`USER_LOGIN_NAME` varchar(255) DEFAULT NULL,
`USER_LOGIN_LOGIN` varchar(255) NOT NULL DEFAULT '',
`START_DATE_TIME` datetime DEFAULT NULL,
`EXECUTION_TIME` float DEFAULT NULL,
`UUT_STATUS` varchar(255) DEFAULT NULL,
`UUT_ERROR_CODE` int(11) DEFAULT NULL,
`UUT_ERROR_MESSAGE` varchar(1023) DEFAULT NULL,
`PAT_NAME` varchar(255) NOT NULL DEFAULT '',
`PAT_VERSION` varchar(10) NOT NULL DEFAULT '',
`TEST_LEVEL` varchar(50) DEFAULT NULL,
`INTERFACE_ID` int(10) unsigned NOT NULL DEFAULT '0',
`EXECUTION_MODE` varchar(45) DEFAULT NULL,
`LOOP_MODE` varchar(45) DEFAULT NULL,
`STOP_ON_FAIL` tinyint(4) unsigned NOT NULL DEFAULT '0',
`EXECUTION_COMMENT` text,
PRIMARY KEY (`ID`),
KEY `FK_uut_result_1` (`STATION_NUM`)
) ENGINE=MyISAM DEFAULT CHARSET=latin;
and
CREATE TABLE `meas_numericlimit` (
`ID` varchar(38) NOT NULL DEFAULT '',
`STEP_RESULT` varchar(38) NOT NULL DEFAULT '',
`NAME` varchar(255) DEFAULT NULL,
`COMP_OPERATOR` varchar(30) DEFAULT NULL,
`HIGH_LIMIT` double DEFAULT NULL,
`LOW_LIMIT` double DEFAULT NULL,
`UNITS` varchar(255) DEFAULT NULL,
`DATA` double DEFAULT NULL,
`STATUS` varchar(255) DEFAULT NULL,
`FORMAT` varchar(15) DEFAULT NULL,
`NANDATA` int(11) DEFAULT '0',
PRIMARY KEY (`ID`),
KEY `FK_meas_numericlimit_1` (`STEP_RESULT`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1
and
CREATE TABLE `step_result` (
`ID` varchar(38) NOT NULL DEFAULT '',
`UUT_RESULT` varchar(38) NOT NULL DEFAULT '',
`STEP_PARENT` varchar(38) DEFAULT NULL,
`STEP_NAME` varchar(255) DEFAULT NULL,
`STEP_ID` varchar(38) NOT NULL DEFAULT '',
`STEP_TYPE` varchar(255) DEFAULT NULL,
`STATUS` varchar(255) DEFAULT NULL,
`REPORT_TEXT` text,
`DIAG` text,
`ERROR_OCCURRED` tinyint(1) NOT NULL DEFAULT '0',
`ERROR_CODE` int(11) DEFAULT NULL,
`ERROR_MESSAGE` varchar(1023) DEFAULT NULL,
`MODULE_TIME` float DEFAULT NULL,
`TOTAL_TIME` float DEFAULT NULL,
`NUM_LOOPS` int(11) DEFAULT NULL,
`NUM_PASSED` int(11) DEFAULT NULL,
`NUM_FAILED` int(11) DEFAULT NULL,
`ENDING_LOOP_INDEX` int(11) DEFAULT NULL,
`LOOP_INDEX` int(11) DEFAULT NULL,
`INTERACTIVE_EXENUM` int(11) DEFAULT NULL,
`STEP_GROUP` varchar(30) DEFAULT NULL,
`STEP_INDEX` int(11) DEFAULT NULL,
`ORDER_NUMBER` int(11) DEFAULT NULL,
PRIMARY KEY (`ID`),
KEY `FK_step_result_1` (`UUT_RESULT`),
KEY `IDX_step_parent` (`STEP_PARENT`)
) ENGINE=MyISAM DEFAULT CHARSET=latin
Thank you for your help

Answer 1

What is the value of join_buffer_size? It should not be more than about 1% of RAM. If it is much bigger, you run the risk of swapping, which is especially bad for performance.
One thing jumps out in the EXPLAIN: NULL | 48120004 saying that this is needed: INDEX(STEP_ID);
However, the SELECT and the EXPLAIN do not seem to match. Please double check.
uut_result needs INDEX(station_num, uut_name) -- in that order; replaces just (station_num).
What is varchar(38)? UUIDs are only 36. IPv6 needs 39.
UUIDs are terribly inefficient when the data is too big to be cached. More discussion: http://mysql.rjweb.org/doc.php/uuid
Lots of datatypes could (should) be shrunken -- this shrinkage will cut down on I/O, which will speed up queries. If you provide some sample values for some typical columns, I can give more advice.
For example, STATUS is (usually) a small number of distinct values. That could be represented as a 1-byte ENUM or a 1-byte TINYINT; but maybe your app has hundreds of different status values? If so, "normalizing" it may be the better answer.
DOUBLE takes 8 bytes; FLOAT takes only 4 bytes, but limits precision to only ~7 significant digits -- perhaps that is sufficient?
(Presumably you meant latin1, not latin?)
Also consider switching to InnoDB.
How much RAM do you have? How big (GB) are the tables?