Optimal Database Struct - mysql

Question

Im a data lover and created a list of possible item combinations for a widely known mobile game. There are 21.000.000 combinations (useless combos filtered out by logics).
So what i wanna do now is creating a website people can access to see what they need to get the best gear OR whats the best they can do with the gear the have right now.
My Item Database currently looks like this:
CREATE TABLE `items` (
`ID` int(8) unsigned NOT NULL,
`Item1` int(2) unsigned NOT NULL,
`Item2` int(2) unsigned NOT NULL,
`Item3` int(2) unsigned NOT NULL,
`Item4` int(2) unsigned NOT NULL,
`Item5` int(2) unsigned NOT NULL,
`Item6` int(2) unsigned NOT NULL,
`Item7` int(2) unsigned NOT NULL,
`Item8` int(2) unsigned NOT NULL,
PRIMARY KEY (`ID`)
) ENGINE=InnoDB
ID range: 1 - 21.000.000
Every Item is known by its number e.g. 11. First number describes the category and second number the item of this category. For example 34 means Item3 --> 4. Its saved like this as i also have images to show on the website later using this number as identification (34.png).
The Stats Database looks like this right now:
CREATE TABLE stats (
Stat1 FLOAT UNSIGNED NOT NULL,
Stat2 FLOAT UNSIGNED NOT NULL,
Stat3 FLOAT UNSIGNED NOT NULL,
Stat4 FLOAT UNSIGNED NOT NULL,
Stat5 FLOAT UNSIGNED NOT NULL,
Stat6 FLOAT UNSIGNED NOT NULL,
Stat7 FLOAT UNSIGNED NOT NULL,
Stat8 FLOAT UNSIGNED NOT NULL,
ID1 INT UNSIGNED,
ID2 INT UNSIGNED,
ID3 INT UNSIGNED,
ID4 INT UNSIGNED,
ID5 INT UNSIGNED,
ID6 INT UNSIGNED,
ID7 INT UNSIGNED,
ID8 INT UNSIGNED
) ENGINE = InnoDB;
Where Stat* stands for stuff like Attack, Defense, Health, etc. and ID* for the ID of the Item Database. Some Combinations have the same stat combinations over all 8 possible stats, so i grouped them together to save some entries (dunno if that was smart yet). For example one Stat combination can have ID1, ID2 and ID3 filled and another combination just has ID1 (the max is 8 IDs tho, i calced it).
Right now im displaying a huge table sortable by every Stat, and its working fine.
What i want in the future tho is to let the user search for items or exclude certain items from the list. I know i can do this with some join and where-clauses (where items.ID == stats.ID1 OR items.ID == stats.ID2 etc.), but i wonder if my current structure is the smartest solution for this? I try to get the best performance as im running this on my old Pi 2.

Answer 1

When you have very large data-sets that only have a small number of matches, the best performance is often to use a subquery in the FROM or WHERE clause.
SELECT SP.TerritoryID,
SP.BusinessEntityID,
SP.Bonus,
TerritorySummary.AverageBonus
FROM (SELECT TerritoryID,
AVG(Bonus) AS AverageBonus
FROM Sales.SalesPerson
GROUP BY TerritoryID) AS TerritorySummary
INNER JOIN
Sales.SalesPerson AS SP
ON SP.TerritoryID = TerritorySummary.TerritoryID
Copied from here
This effectively creates a virtual table of only those rows that match, then runs the join on the virtual table - a lot like selecting the matching rows into a tmp table, then joining on the tmp table. Running a join on the entire table, although you might think it would be OK, often comes out terrible.
You may also find using a subquery in the WHERE clause works
... where items.id in (select id1 from stats union select id2 from stats)
Or select your matching stats IDs into a tmp table, then indexing the tmp table.
It all depends quite a lot on what your other selection logic is.
It also sounds like you should get some indexes on the stats table. If you're not updating it a lot, then indexing every ID can work OK. Just make sure the unfilled stats IDs have the value NULL