Finding matches between 30,000+ data sets - mysql

Question

For simplicity sake, lets use a case example of 3 colors with corresponding numbers (there is actually 30,000+ different 'colors' and 254 different 'numbers' in real life though)
Red - 0, 1, 2, 3, 10, 15
Green - 0, 2, 3, 20
Blue - 2, 10, 11, 12
I want to find the matches between them (rgb, rg, rb), as well as keep a tally of the number of #s shared between the set:
rgb = 1
rg = 2
rb = 2
Finally it'll need to determine a ratio of the number of #s shared compared to the number of distinct #s in the set.
rgb = 1/9 (since it has a distinct: 0, 1, 2, 3, 10, 11, 12, 15, 20)
rg = 2/7 (0, 1, 2, 3, 10, 15, 20)
rb = 2/8 (0, 1, 2, 3, 10, 11, 12, 15)
So the total output would be
match | # of matches | % |
rgb | 1 | 1/9
rg | 2 | 2/7
rb | 2 | 2/8
The algorithm I was able to come up w/is you have each color in a table and map the numbers associated with it (aka red (table Name), 0, 1, 2, 3, 10, 15 (data)). Then take the color with the most 'numbers' and compare it to every other color's hours, find matches. Once done with that color you can 100% ignore it and move onto the next color and do comparisons with n-1.
Take the example:
1) Select red
2) Does any other color share 0
3) Does any other color share 1
....etc
4) Select blue
5) Does any other color minus red share .....
I know that there has to be a more efficient way to do this, any suggestions?
Thanks for the help.

Answer 1

As there are only 254 (or 255 if your 0-254 comment is correct), then you can express the sets of 'numbers' for each 'colour' as a 256-bit integer. Then the number of shared numbers for r and g is just the bitcount of (r and b) and the number of distinct numbers is the bitcount of (r or b), so using your example,
if R is the bit-set for red, G is the bitset for B etc:
match | # of matches | % |
rgb | bitcount(R and G and B) | bitcount(R and G and B)/bitcount(R or G or B) |
rg | bitcount(R and G) | bitcount(R and G)/bitcount(R or G) |
rb | bitcount(R and B) | bitcount(R and B)/bitcount(R or B) |