How can one break up an UPDATE into subqueries? - mysql

Question

There exists a table:
CREATE TABLE person
(
id INT(10) PRIMARY KEY AUTO_INCREMENT,
nameFirst VARCHAR(255) DEFAULT '?',
nameSecond VARCHAR(255) DEFAULT '',
fatherNameFirst VARCHAR(255) DEFAULT NULL
);
Note: There are actually other columns there, 18 in total, but they are not being used here.
The goal is to set up father's first name from using second name of the child. It can be predicted from the second name (patronymic) in russian language, but not always correctly. So i plan to do what can be done automatically and some will do by hand later.
So the UPDATE done as follows:
UPDATE person AS child
LEFT JOIN
(SELECT DISTINCT nameFirst FROM person) AS parent
ON CONCAT(parent.nameFirst,'овна')=child.nameSecond OR CONCAT(parent.nameFirst,'ович')=child.nameSecond
SET child.fatherNameFirst=parent.nameFirst;
Eventually it will need to run on a table that has >2m entries, for now i have tried with the sample data of 400k. The problem is that after about an hour of my computer using one if its cores at 100% the query has not yet finished.
So i was thinking if i can break it up into subqueries, so these can be set to run one after another, but they should each take 5-10 minutes. This way if i need to do something, i can terminate currently running one and not lose a day of CPU time.
I have attempted to add: WHERE child.id<1000 but either it was still way too long or had little impact (perhaps i misunderstand how MariaDB opens up this update).
In case sample data will actually help somebody understand it better:
select id, nameFirst, nameSecond from person limit 10;
+----+--------------------+----------------------------+
| id | nameFirst | nameSecond |
+----+--------------------+----------------------------+
| 1 | Туликович | |
| 2 | Август | Михайлович |
| 3 | Август | Христианович |
| 4 | Александр | Александрович |
| 5 | Александр | Христьянович |
| 6 | Альберт | Викторович |
| 7 | Альбрехт | Александрович |
| 8 | Амалия | Андреевна |
| 9 | Амалия | Ивановна |
| 10 | Ангелина | Андреевна |
+----+--------------------+----------------------------+
fatherNameFirst is empty at this time.

Answer 1

You could break this down alphabetically by adding where nameFirst like 'A%' to the update query - and then run the query multiple times.

Answer 2

Given this sample data:
CREATE TABLE person
(
id INT(10) PRIMARY KEY AUTO_INCREMENT,
nameFirst VARCHAR(255) DEFAULT '?',
nameSecond VARCHAR(255) DEFAULT '',
fatherNameFirst VARCHAR(255) DEFAULT NULL
) DEFAULT CHARSET=utf8;
INSERT INTO person
(`id`, `nameFirst`, `nameSecond`)
VALUES
(1, 'Туликович', NULL),
(2, 'Август', 'Михайлович'),
(3, 'Август', 'Христианович'),
(4, 'Александр', 'Александрович'),
(5, 'Александр', 'Христьянович'),
(6, 'Альберт', 'Викторович'),
(7, 'Альбрехт', 'Александрович'),
(8, 'Амалия', 'Андреевна'),
(9, 'Амалия', 'Ивановна'),
(10, 'Ангелина', 'Андреевна')
;
with your query you get this EXPLAIN output:
+----+-------------+------------+------+---------------+------+---------+------+------+----------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+------------+------+---------------+------+---------+------+------+----------------------------------------------------+
| 1 | PRIMARY | child | ALL | NULL | NULL | NULL | NULL | 10 | NULL |
| 1 | PRIMARY | <derived2> | ALL | NULL | NULL | NULL | NULL | 10 | Using where; Using join buffer (Block Nested Loop) |
| 2 | DERIVED | person | ALL | NULL | NULL | NULL | NULL | 10 | Using temporary |
+----+-------------+------------+------+---------------+------+---------+------+------+----------------------------------------------------+
That's probably the worst you can get.
Let's see if we can rewrite this. First, there's absolutely no need for this subquery and the DISTINCT.
mysql > explain UPDATE person AS child
-> LEFT JOIN person parent
-> ON CONCAT(parent.nameFirst,'овна')=child.nameSecond OR CONCAT(parent.nameFirst,'ович')=child.nameSecond
-> SET child.fatherNameFirst=parent.nameFirst;
+----+-------------+--------+------+---------------+------+---------+------+------+----------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------+------+---------------+------+---------+------+------+----------------------------------------------------+
| 1 | SIMPLE | child | ALL | NULL | NULL | NULL | NULL | 10 | NULL |
| 1 | SIMPLE | parent | ALL | NULL | NULL | NULL | NULL | 10 | Using where; Using join buffer (Block Nested Loop) |
+----+-------------+--------+------+---------------+------+---------+------+------+----------------------------------------------------+
This eliminates the Using temporary. That's good.
With an index on nameFirst, we can speed this up further.
CREATE INDEX idx_person_nameFirst ON person(nameFirst);
Then explaining again:
+----+-------------+--------+-------+---------------+----------------------+---------+------+------+-----------------------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------+-------+---------------+----------------------+---------+------+------+-----------------------------------------------------------------+
| 1 | SIMPLE | child | ALL | NULL | NULL | NULL | NULL | 10 | NULL |
| 1 | SIMPLE | parent | index | NULL | idx_person_nameFirst | 768 | NULL | 10 | Using where; Using index; Using join buffer (Block Nested Loop) |
+----+-------------+--------+-------+---------------+----------------------+---------+------+------+-----------------------------------------------------------------+
Not yet perfect, but it's using the index. This should speed things up a lot.
From here on, it gets hard to optimize further. You can experiment a bit by adjusting the join buffer size, but I recommend you do this in a session only.
SET SESSION join_buffer_size = <whatever value>;
Every thread connecting to your server uses its own join buffer. That's why you should only test it in a session. When you have very much connections on your server, memory consumption could get out of hand.