MySQL update with subquery and between stupidly slow - mysql

Question

I have a simple query. All I am trying to do is combine geolocation data into a primary table so I dont need to use a join when pulling data. Yeah a join is fine but I want all the data in a single table...
Here is my query:
> explain UPDATE test AS l
SET l.country_name1 = (
select country_name
from ip2location_db9
where
l.ip >= ip_from
ORDER BY ip_from DESC
LIMIT 1)
******************** 1. row *********************
id: 1
select_type: UPDATE
table: l
partitions:
type: ALL
possible_keys:
key:
key_len:
ref:
rows: 10
filtered: 100.00
Extra:
******************** 2. row *********************
id: 2
select_type: DEPENDENT SUBQUERY
table: ip2location_db9
partitions:
type: index
possible_keys: idx_ip_from,idx_ip_from_to,ip_from_to,ip_from
key: idx_ip_from
key_len: 5
ref:
rows: 1
filtered: 33.33
Extra: Using where; Backward index scan
2 rows in set
Simple right? Except that this query for 10 lines takes 130+ seconds?..
The select statement when run by itself is super fast (0.00 sec) and works perfectly.
explain select country_name
from ip2location_db9
where
ip_from <= INET_ATON('114.160.63.108')
ORDER BY ip_from DESC
LIMIT 1
******************** 1. row *********************
id: 1
select_type: SIMPLE
table: ip2location_db9
partitions:
type: range
possible_keys: idx_ip_from,idx_ip_from_to
key: idx_ip_from
key_len: 5
ref:
rows: 1949595
filtered: 100.00
Extra: Using index condition
1 rows in set
However when I use it with the update statement with only 14 rows in the test table, the query takes a painful 130 seconds.
Tables are set up as follows:
CREATE TABLE `test` (
`ip` int(10) unsigned DEFAULT NULL,
`country_name` varchar(64) COLLATE utf8_bin DEFAULT NULL,
`region_name` varchar(128) COLLATE utf8_bin DEFAULT NULL,
`city_name` varchar(128) COLLATE utf8_bin DEFAULT NULL,
`latitude` double DEFAULT NULL,
`longitude` double DEFAULT NULL,
`zip_code` varchar(30) COLLATE utf8_bin DEFAULT NULL,
`data` varchar(512) DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
CREATE TABLE `ip2location_db9`(
`ip_from` INT(10) UNSIGNED,
`ip_to` INT(10) UNSIGNED,
`country_code` CHAR(2),
`country_name` VARCHAR(64),
`region_name` VARCHAR(128),
`city_name` VARCHAR(128),
`latitude` DOUBLE,
`longitude` DOUBLE,
`zip_code` VARCHAR(30),
INDEX `idx_ip_from` (`ip_from`),
INDEX `idx_ip_to` (`ip_to`),
INDEX `idx_ip_from_to` (`ip_from`, `ip_to`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
Note that when running the update query, the hard drive activity goes crazy high utilization.
See? Nothing fancy. Table data matches. Rows are properly indexed.
Explain shows that is uses the index just as it does when I perform the select by itself:
******************** 1. row *********************
id: 1
select_type: UPDATE
table: l
partitions:
type: ALL
possible_keys:
key:
key_len:
ref:
rows: 14
filtered: 100.00
Extra:
******************** 2. row *********************
id: 2
select_type: DEPENDENT SUBQUERY
table: ip2location_db93
partitions:
type: index
possible_keys: idx_ip_from,idx_ip_from_to
key: idx_ip_from
key_len: 5
ref:
rows: 1
filtered: 33.33
Extra: Using where
2 rows in set
So what could I possibly been doing wrong that would cause the query to take 2 min to run for 14 lines?
Its not a hardware thing. i7 6990, 32gb ram, running off an ssd. Not the fastest in the world but I can manually update 14 rows faster than this query can...
I have spent an excess of time searching trying to find out why this takes so long. I assume that I am just not searching correctly. Perhaps I dont know a specific term or something that would point me in the right direction. I am not a DB guy. Just doing this for a work thing.
Hoping you guys can save my sanity..
Adding more info...
I have tried to make this query work many many ways. Other questions on stack said to avoid the subquery and use a join. Ok that was the first thing I tried but I cant get the query to use the indexes I built.
> explain UPDATE test AS l
JOIN
ip2location_db9 AS h
ON
l.ip <= h.ip_from and l.ip >= h.ip_to
SET
l.country_name1 = h.country_name
******************** 1. row *********************
id: 1
select_type: UPDATE
table: l
partitions:
type: ALL
possible_keys: ip
key:
key_len:
ref:
rows: 10
filtered: 100.00
Extra:
******************** 2. row *********************
id: 1
select_type: SIMPLE
table: h
partitions:
type: ALL
possible_keys: idx_ip_from,idx_ip_to,idx_ip_from_to,ip_from_to,ip_from,ip_to
key:
key_len:
ref:
rows: 3495740
filtered: 11.11
Extra: Range checked for each record (index map: 0x3F)
2 rows in set
Does an update with a join get any easier than that?
Even using force index doesn't get the query to use an index.
Ive tried this query so many ways
UPDATE test
JOIN (
SELECT * FROM ip2location_db9
) AS t1
ON (test.ip <= t1.ip_from and test.ip >= t1.ip_to)
SET test.country_name1 = t1.country_name
UPDATE test l,
(SELECT
*
FROM
ip2location_db9,test
WHERE
test.ip >= ip_from
ORDER BY ip_from DESC
LIMIT 1) AS PreQualified
SET
l.country_name1 = PreQualified.country_name
Nothing works! What am I doing wrong?