Improve ORDER BY RAND() in large table [duplicate] - mysql

Question

This question already has answers here:
MySQL select 10 random rows from 600K rows fast
(28 answers)
quick selection of a random row from a large table in mysql
(24 answers)
Closed last year.
I have a very large table with about 6.6 million records and I want to select a random sample of 100,000 records
SELECT column FROM table
ORDER BY RAND()
LIMIT 100000
Is EXTREMELY SLOW on each record.
I have not found a solution that works with MySQL/MariaDB to extract a random sample of 100,000 records.
Please advise.
Thank you.

Answer 1

You may try to decrease the amount of rows to be sorted.
Example:
Create source data
mysql> create table test (id int auto_increment primary key, val int);
Query OK, 0 rows affected (0.05 sec)
mysql> set ##cte_max_recursion_depth := 10000000;
Query OK, 0 rows affected (0.00 sec)
mysql> insert into test (val) with recursive cte as ( select 1 num union all select num+1 from cte where num < 6600000 )select rand() * 1000000000 from cte;
Query OK, 6600000 rows affected (1 min 48.62 sec)
Records: 6600000 Duplicates: 0 Warnings: 0
mysql> create table tmp (val int);
Query OK, 0 rows affected (0.05 sec)
Insert without sorting
mysql> insert into tmp select val from test limit 100000;
Query OK, 100000 rows affected (1.93 sec)
Records: 100000 Duplicates: 0 Warnings: 0
Insert with random sorting
mysql> insert into tmp select val from test order by rand() limit 100000;
Query OK, 100000 rows affected (26.31 sec)
Records: 100000 Duplicates: 0 Warnings: 0
Insert with random selection (1.1 is overage coefficient)
mysql> insert into tmp select val from test where rand() < 1.1 * 100000 / 6600000 limit 100000;
Query OK, 100000 rows affected (15.89 sec)
Records: 100000 Duplicates: 0 Warnings: 0
Insert with random selection (1.1 is overage coefficient) and random sorting
mysql> insert into tmp select val from test where rand() < 1.1 * 100000 / 6600000 order by rand() limit 100000;
Query OK, 100000 rows affected (19.26 sec)
Records: 100000 Duplicates: 0 Warnings: 0
Overage coefficient may be adjusted. If you decrease it then you'll improve the query (slightly) but the probability that the amount of output rows will be less than needed 100k rows will increase.