Counting unique rows performance - mysql

Question

I have table with user statistics. Statistics are generated requesting URL.
Under this URL with GET parameters (event_id) is PHP script. Is response with transparent [1x1] GIF.
event is a child and insertion and insertion is a child from campaign etc up to account.
Where are two types of statistics based on statistic_type.
One account have many spaces (space_id).
This is table CREATE:
CREATE TABLE `statistic` (
`id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
`statistic_type` TINYINT(3) UNSIGNED NOT NULL,
`account_id` TINYINT(3) UNSIGNED NOT NULL,
`advertiser_id` SMALLINT(5) UNSIGNED NOT NULL,
`campaign_id` SMALLINT(5) UNSIGNED NOT NULL,
`insertion_id` MEDIUMINT(8) UNSIGNED NOT NULL,
`event_id` INT(10) UNSIGNED NOT NULL,
`space_id` SMALLINT(5) UNSIGNED NULL DEFAULT NULL,
`date` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`stat_platform_id` TINYINT(3) UNSIGNED NULL DEFAULT NULL,
`stat_browser_id` SMALLINT(5) UNSIGNED NULL DEFAULT NULL,
`stat_device_type_id` TINYINT(3) UNSIGNED NULL DEFAULT NULL,
`major_version` TINYINT(4) UNSIGNED NULL DEFAULT NULL,
`uid` CHAR(19) NOT NULL,
`referrer` VARCHAR(255) NULL DEFAULT NULL,
`useragent` VARCHAR(255) NULL DEFAULT NULL,
`ipv4` INT(11) UNSIGNED NULL DEFAULT NULL,
PRIMARY KEY (`id`),
INDEX `IDX_uid` (`uid`),
INDEX `Indeks 2` (`statistic_type`, `account_id`, `advertiser_id`, `campaign_id`, `insertion_id`, `event_id`, `space_id`, `date`)
)
COLLATE='utf8_general_ci'
ENGINE=InnoDB
ROW_FORMAT=COMPRESSED
AUTO_INCREMENT=427891347;
Now I need to get all and unique (based on uid field) number of rows for campaign_id grouped by statistic_type, event_id, space.
All table have now 428067039 rows. It is still growing about 1500000 rows daily.
Result should have up to 100 rows.
Maximum filtered and letter aggregate (Rows with one campaign_id) row should be up to 5000000.
Now I user this query:
SELECT
`sub`.`account_id`,
`sub`.`advertiser_id`,
`sub`.`campaign_id`,
`sub`.`insertion_id`,
`sub`.`event_id`,
`sub`.`space_id`,
SUM(`sub`.actions) as 'actions',
count(1) as 'unique_actions'
FROM (
SELECT
`statistic`.`account_id`,
`statistic`.`advertiser_id`,
`statistic`.`campaign_id`,
`statistic`.`insertion_id`,
`statistic`.`event_id`,
`statistic`.`space_id`,
count(1) as 'actions'
FROM `statistic`
WHERE
(`statistic`.`statistic_type`=1) AND
(`statistic`.`account_id`=3) AND
(`statistic`.`advertiser_id`=679) AND
(`statistic`.`campaign_id`=4475) AND
(`statistic`.`insertion_id`=26841)
GROUP BY
`statistic`.`statistic_type`,
`statistic`.`account_id`,
`statistic`.`advertiser_id`,
`statistic`.`campaign_id`,
`statistic`.`insertion_id`,
`statistic`.`event_id`,
`statistic`.`space_id`,
`statistic`.`uid`
) sub
GROUP BY
`sub`.`statistic_type`,
`sub`.`account_id`,
`sub`.`advertiser_id`,
`sub`.`campaign_id`,
`sub`.`insertion_id`,
`sub`.`event_id`,
`sub`.`space_id`;
When I have 400000 filtered by insertion_id rows it count's longer than 7-10 minutes.
Explain:
+----+-------------+------------+------+---------------+----------+---------+------+-------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+------------+------+---------------+----------+---------+------+-------+----------------------------------------------+
| 1 | PRIMARY | <derived2> | ALL | NULL | NULL | NULL | NULL | 157521 | Using temporary; Using filesort |
| 2 | DERIVED | statistic | ref | Indeks 2 | Indeks 2 | 9 | | 627090 | Using where; Using temporary; Using filesort |
+----+-------------+------------+------+---------------+----------+---------+------+-------+----------------------------------------------+
2 rows in set (2.61 sec)
Maybe I need change index, maybe add new?
Maybe I need add new col and insert some value when user requesting URL ?

Answer 1

Normally I recommend against having long indexes, but in this case, I suggest an 8-column index with all the columns of sub's GROUP BY, in exactly that order. The one you have ends with date; I don't know if it might be useful for something else. Mine ends with uid.
I expect that would speed up the subquery, but have no impact on the outer query.
An additional thought... Since 5 of the columns are constant, can you get away with not carrying them through to the outer query? This would cut down on the bulk of the temp table of about 157521 rows, thereby possibly speeding up the outer part some.