Can MySQL automatically and transparently de-duplicate strings? - mysql

Question

In C, the compiler assigns "strings" numeric IDs (4-byte pointers) and only keeps one copy of each string: for char *a="Hello", *b="Hello";, only one copy of "Hello" is stored in memory. This is totally automatic and transparent to the user.
My question is whether MySQL can do the same, i.e, de-duplicate strings automatically and transparently to the user.
Ideally, I would expect it to be an internal storage mechanism of the database, so that (as in case of C) for the user the database would look and behave completely as if it contained actual strings, while in implementation it would only contain pointers.
In my database there are many repeating strings, like this:
`unit`, `building`, `office`, `firstName`, `lastName`
Chicago main production unit | headquarters | accounting | Jane | Smith
Chicago main production unit | office | sales | Jane | Dow
Miami administrative department | headquarters | sales | Mary | Smith
Miami administrative department | office | accounting | Mary | Dow
etc. where strings like 'Miami administrative department' or 'accounting' or 'Smith' are repeated many times in different records.
This increases the size of the database, so that I hit hosting limitations.
An obvious solution is data normalization: to maintain a separate table for names
`id`, `string`
1 | Chicago main production unit
2 | Miami administrative department
3 | headquarters
4 | accounting
5 | Jane
6 | Smith
7 | office
8 | sales
9 | Dow
and then have my table as
`unit_id`, `building_id`, `office_id`, `firstName_id`, `lastName_id`
1 | 3 | 4 | 5 | 6
1 | 7 | 8 | 5 | 9
and translate all strings on input and output. But of course this is very cumbersome.
My question is whether MySQL can do it automatically and transparently for the user: whenever I INSERT a row, it would automatically update the table of strings and only store the ids instead of strings in the table, and same for DELETE, WHERE, etc., so that to the user the table would look exactly the same as if it had strings, but occupy less space.

Answer 1

My question is whether MySQL can do the same.
Although you can certainly achieve the desired result (it is called data normalization) MySQL does not do it implicitly.
Can MySQL do it automatically and transparently for the user?
No, MySQL cannot do it automatically for you - you have to do it yourself. You need to be explicit about it in your queries and DDL statements.
Here is a short demo to show how you can create a lookup table, and then use it in your inserts and selects:
create table lookup(id int, name varchar(10));
create table data(id int, id_lookup int);
insert into lookup(id,name) values (1,'quick');
insert into lookup(id,name) values (2,'brown');
insert into lookup(id,name) values (3,'fox');
insert into data (id, id_lookup)
values (110, (select id from lookup where name = 'quick'));
insert into data (id, id_lookup)
values (120, (select id from lookup where name = 'brown'));
insert into data (id, id_lookup)
values (130, (select id from lookup where name = 'quick'));
insert into data (id, id_lookup)
values (140, (select id from lookup where name = 'fox'));
Now data has these rows:
110 1
120 2
130 1
140 3
To select the name, you need to join to your lookup table:
select d.id, t.name
from data d
join lookup t on t.id=d.id_lookup
Demo on sqlfiddle.
Note: it is uncommon to create a lookup table for all your strings. Commonly you would create a separate lookup table for each kind of strings (i.e. unit_lookup, building_lookup, and so on) or to partition your lookup table with a special lookup code column:
id code name
-- ---- ----
1 unit Chicago
2 unit Miami
3 bldg Headquarters
4 bldg Office