mysql select query optimization of partitioned table in non-cluster environment - mysql

Question

I have select query on a partitioned table with 123 million records which is taking more then 10 minutes to fetch data. My query looks like 'select * from tableName where column1='1.1.1.1' order by timestamp desc';
Table is already indexed on column1.
Any help appreciated.
(From comments)
CREATE TABLE mytable (
column1 varchar(256) NOT NULL,
column2 varchar(100) NOT NULL,
column3 smallint(5) unsigned NOT NULL,
column4 smallint(5) unsigned NOT NULL,
timestamp bigint(20) unsigned NOT NULL,
KEY mytable_idx (column2,timestamp,column3,column4),
KEY ip_addr_index (column1),
KEY ts_idx (timestamp)
) /*!50100 PARTITION BY RANGE ((TIMESTAMP))
(PARTITION p1498800000 VALUES LESS THAN (1498800000) ENGINE = InnoDB,
PARTITION p1500000000 VALUES LESS THAN (1500000000) ENGINE = InnoDB,
PARTITION p1501200000 VALUES LESS THAN (1501200000) ENGINE = InnoDB,
PARTITION p1502400000 VALUES LESS THAN (1502400000) ENGINE = InnoDB,
PARTITION p1503600000 VALUES LESS THAN (1503600000) ENGINE = InnoDB,
PARTITION p1504800000 VALUES LESS THAN (1504800000) ENGINE = InnoDB,
PARTITION p1506000000 VALUES LESS THAN (1506000000) ENGINE = InnoDB
) */

Answer 1

For this query:
select *
from tableName
where column1 = '1.1.1.1'
order by timestamp desc;
You want an index on (column1, timestamp desc). Note: The desc may be ignored in earlier versions of MySQL.

Answer 2

PARTITIONing does not intrinsically provide speed. Please provide SHOW CREATE TABLE so we can discuss whether partitioning actually hurts performance in your case.
INDEX(column1, timestamp) -- In this order
is optimal whether the table is partitioned or not. In particular, that index will work just as good for non-partitioned. (Gordon's comment about DESC has no impact on performance, whether old or new version.)
With 123 million rows, you should keep an eye on datatypes. If you have
column1 VARCHAR(15) CHARACTER SET utf8
then that ipv4_address can be improved from up-to-17 bytes to exactly 4:
BINARY(4)
with suitable conversions on INSERT and SELECT. Making that change would also allow for CDR and other range tests, which are not possible with VARCHAR. Will you need to handle IPv6? I discuss that here.
How many rows match 1.1.1.1? Are there any TEXT columns? What is the PRIMARY KEY? Which Engine? Each of those questions may have an impact on the "10 minutes".
It is important to understand when a "composite" index is better than a single-column index. More discussion: http://mysql.rjweb.org/doc.php/index_cookbook_mysql
after CREATE
Replace this
KEY ip_addr_index (column1)
with
KEY ip_addr_index (column1, timestamp)
Don't create more than one future partition before it is needed. Always have a LESS THAN (MAXVALUE) partition just in case.
IPv4 can live with VARCHAR(15); IPv6 fits in (39) or `BINARY(16) after packing.
For that one query, 7 queries must be done (one per partition); the results put together, then sorted. Without partitioning, it becomes one query, no sort (since the index is already sorted). So, (I believe) that partitioning slows that query down.
When discussing performance in 123M rows, I need to see all the main queries in one sitting in order to advise. Optimizing for one query is all to likely to de-optimize for some other.
There seems to be no reason to use BIGINT for TIMESTAMP. INT UNSIGNED would save 4 bytes per row of data, plus more for the indexes. Perhaps a total savings of 2GB of disk space. That translates into some speedup for some queries.
If timestamp is always used in a "range", then this index (column2,timestamp,column3,column4) is probably in an inefficient order. Please provide the query that benefits from this index so I can further elaborate.