Is it possible to have a foreign key without an index in MySQL 5.6.34? I want that because I created a nullable column in 20M rows with a foreign key to another table. As this is a new feature, only the new rows MAY have this column filled with an actual value, and as you may expect, the cardinality of that index becomes horrible. So, for most of the time, using that index is actually a bad idea. The problem: I have tons of queries that shares this same restriction:
[...] from large_table where tenant_id = ? and nullable_foreign_key_with_index is null and [...]
The issue? MySQL thinks that it's a good idea to use an index_merge/intersect strategy for query resolution. In this case MySQL would do 2 queries in parallel: one with tenant_id (which uses a valid and good index) and another one with nullable_foreign_key_with_index which is bad, almost a "full table scan in parallel" given that the cardinality of this index is <1000 in a table with >20M rows. More details about this "problem" in here
So, what are the pssible solutions? Given that MySQL "forces" a foreign key to have an index attached:
Drop the foreign key and the index. This is bad, because in the case of a bug in the app we may compromise the referential integrity.
FOREIGN_KEY_CHECKS=0; Drop index; FOREIGN_KEY_CHECKS=1. This is bad, because even that the foreign key still exists, MySQL doesn't validade the column anymore to check if the value actually exists. Is that a bug?
Use query hints in all existing queries to make sure that we are only using the old and efficient "tenant_id_index". This is bad because I have to hunt down all existing queries and also remember to use it again when news queries are built.
So, how can I say: "MySQL, don't bother creating an index for this foreign key, but keep validating it's content in the related table, which is indexed by primary key anyway". Am I missing something? The best idea so far is to remove the foreign key and just believe that the app is working as expected, which probably it is, but this would start a classic discussion about having constraints in APP vs DATABASE. Any ideas?
For this query:
from large_table
where tenant_id = ? and
nullable_foreign_key_with_index is null and [...]
Just add the index large_table(tenant_id, nullable_foreign_key_with_index).
MySQL should use this index for the table.
I'm pretty sure you can do this backwards (I would be 100% sure if the comparison were to anything other than NULL, but I'm pretty sure MySQL does the right thing with NULL as well.)
large_table(nullable_foreign_key_with_index, tenant_id)
And MySQL will recognize that this index works for the foreign key and not create any other index.
Q: How can I say: "MySQL, don't bother creating an index for this foreign key, but keep validating it's content in the related table, which is indexed by primary key anyway"
A: No can do. InnoDB requires a suitable index to support the enforcement of foreign key constraint.
Consider the flip side of it... if we are going to DELETE a row in the parent table, then InnoDB needs to check the foreign key constraint.
That means InnoDB needs to check the contents of the child table, to find rows that have a specific value in foreign key column. Essentially equivalent to
SELECT ... FROM child_table c WHERE c.foreign_key_col = ?
And to do that, InnoDB requires that there be an index on child_table that has foreign_key_col as the leading column.
The options suggested in the question (disabling or dropping the foreign key) will work because then InnoDB isn't going enforce the foreign key. But as noted in the question, what this means is that the foreign key isn't enforced. Which defeats the purpose of the foreign key. The application code could be responsible for enforcing referential integrity, or we could write some ug-gghhh-ly triggers (no, we don't want to go there).
As Gordon already noted in his (as usual excellent) answer... the problem isn't really dropping the index on the foreign key column. The actual problem is the inefficient execution plan. And the most likely fix for that is to make sure a more suitable index is available.
Composite indexes are the way to go. An index like this:
... ON child_table (foreign_key_col,tenant_id,...)
would satisfy the requirement of the foreign key, an index with the foreign key column as a leading column. And drop the (now redundant) index on just the singleton foreign_key_col.
This index could also be used to satisfy the query that's using a horrible index merge access plan. (Verify with EXPLAIN.)
Also, consider adding columns (such as foreign_key_col) to the index that has tenant_id as the leading column
... ON child_table (tenant_id,...,foreign_key_col,...)
and drop the redundant index on the singleton tenant_id col.
Summary: Almost always it is better to have a composite index instead of depending on "index merge intersect".
If both columns are tested with = (or IS NULL), it does not matter which order the columns are in the index definition. That is, cardinality is irrelevant.
Related
I have table department that has two column (dept_ID and dept_name)
my another table is login which i create a foreign key for the column dept_name referencing dept_name at table department. so i create an index named index_department, now in login table, i want to create another foreign key for the column eadd which will reference to DIFFERENT table named info_table.
should i create another index for the second foreign key??
another scenario, i want to create a dept_name column at info_table too. can i use the same index 'index_department'??
The general answer is, "It depends."
As #gordon-Linoff commented "You create indexes to meet performance requirements for queries."
Indexes take up space and and take processing time as the have to be maintained. So the case for any given index depends on the trade off between cost and usage. For example if you the data rarely changes, but you look it up a lot you will prefer to have more indexes.
My educated guess is that on the scale you are probably working you do want the indexes on all your foreign keys.
Specifically in mysql you seem to get the index is you formally add the FK constraint. It is discussed here does mysql index foreign key columns automatically
I say formally, because you can have implied foreign key relationships without actually declaring/enforcing the constraints. People sometimes do that to avoid even the cost of the checking/enforcing constraint. The risk is in updates that violate referential integrity. But I'm drifting onto a tangent.
As a side note, there is some pertinent discussion here does a foreign key automatically create an index
In MySQL (at least 5.6, which I am using), indices are automatically created for foreign keys.
I have a small question for performance reasons.
I'm working with symfony and doctrine. I always used annotations in my entities and decided recently to switch to yml files.
So I exported externally all my entities and generated the yml files.
I compared the yml files with the database. There was a diff file generated which drops the primary key on certain tables and then adds them, simply in a different order. These primary keys have multiple columns.
It seems that this happens only when one of the columns is a foreign key.
The question is whether I can execute the change to my database and switch the order of the key columns, or whether it will affect my performance?
Primary keys in MySQL are implemented with unique indexes. Indeed, that's true for most, if not all, SQL dbms nowadays.
The order of columns in an index is significant. Changing the order can certainly change performance.
MySQL can use multiple-column indexes for queries that test all the
columns in the index, or queries that test just the first column, the
first two columns, the first three columns, and so on. If you specify
the columns in the right order in the index definition, a single
composite index can speed up several kinds of queries on the same
table.
There might be a good reason for changing the order. See Using Foreign Key Constraints.
MySQL requires indexes on foreign keys and referenced keys so that
foreign key checks can be fast and not require a table scan. In the
referencing table, there must be an index where the foreign key
columns are listed as the first columns in the same order. Such an
index is created on the referencing table automatically if it does not
exist. This index might be silently dropped later, if you create
another index that can be used to enforce the foreign key constraint.
If your programs are putting the foreign key columns first in the new primary key, this might be the problem they're trying to solve. They're trying to avoid creating both an index on the primary key columns and an additional index on the foreign key columns alone.
That doesn't mean it won't hurt performance of particular queries, though.
There are at least two ways to test this. First, you can bring up a new database, connect your application to it, and run it. Does it seem fast enough?
Second, you can bring up a new database, and run some or all of your queries manually, using EXPLAIN.
Is it possible to add a constraint like
ALTER TABLE `t1` ADD UNIQUE(`col1`, `col2`);
without creating an index? The index wouldn't be used for any queries so it would be a waste of space.
It wouldn't be a problem if inserts and updates would be way slower, because the table doesn't get updated very often.
No, this is not possible. A UNIQUE constraint contains an index definition and I barely imagine how it might be implemented without creating an index (in DBMS terms).
You should realize that indexes are not just 'wizardy' - they are a real data structure, which takes space to be placed, special procedures to be handled e.t.c. A unique constraint, itself, means unique index values, not unique column values.
In my application I usually use my primary keys as a way to access data. However, I've been told in order to increase performance, I should index columns in my table. But I have no idea what columns to index.
Now the Questions:
Is it a good idea to create an index on your primary key?
How would I know what columns to index?
Is it a good idea to create an index on your primary key?
Primary keys are implemented using a unique index automatically in Postgres. You are done here.
The same is true for MySQL. See:
Is the primary key automatically indexed in MySQL?
How would I know what columns to index?
For advice on additional indices, see:
Optimize PostgreSQL read-only tables
Again, the basics are the same for MySQL and Postgres. But Postgres has more advanced features like partial or functional indices if you need them. Start with the basics, though.
Your primary key will already have an index that is created for you automatically by PostgreSQL. You do not need to index the column again.
As far as the rest of the fields go, take a look at the article here on figuring out cardinality:
http://kirk.webfinish.com/2013/08/some-help-to-find-uniqueness-in-a-large-table-with-many-fields/
Fields that are completely unique are candidates, fields that have no uniqueness at all are useless to index. The sweet spot is the cardinality in the middle (.5).
And of course you should take a look at which columns you are using in the WHERE clause. It is useless to index columns that are not a part of your quals.
Primary keys will have an idex only if you formally define them as primary keys. Where most people forget to make indexes are Foriegn keys which are not generally automatically indexed and almost always will be involved in joins and thus indexed. Other candidates for indexes are things you frequently filter data on that have a large number fo possible values, things like names, part numbers, start Dates, etc.
1) Is it a good idea to make your primary key as an Index?(assuming the primary key is unique,an id
All DBMSes I know of will automatically create an index underneath the PK.
In case of MySQL/InnoDB, PK will not just be indexed, but that index will be clustered index.
(BTW, just saying "primary key" implies it is unique, so there is no need to explicitly state "assuming the primary key is unique".)
2) how would I know what columns to index ?
That depends on which queries need to be supported.
But beware that adding indexes is not free and is a matter of engineering tradeoff - while some queries might benefit from an index, some may actually suffer from it. For example:
An index on FOO would significantly speed-up the SELECT * FROM T WHERE FOO = ....
However, the same index would somewhat slow-down the INSERT INTO T VALUES (...).
In most situations you'd favor large speedup in SELECT over small slowdown in INSERT, but that may not always be the case.
Indexing and the database performance in general are a complex topic beyond the scope of a humble StackOverflow post, but if you are interested I warmly recommend reading Use The Index, Luke!.
Your primary key will always be an index.
Always create indexes in columns that help to reduce the search, for example if in the column there are only 3 different values among more than a thousand it is a good sign to make it index.
I have to work with a database to do reporting
The DB is quite big : 416 055 104 rows
Each row is very light though, just booleans and int ids.
Each row is identify by 3 columns, but at my surprise, there is no Primary Key on it.
Only a Clustered Index with a unique constraint.
So Knowing that, I have 2 question.
Could there be ANY good reason for that?
Is there any way I can turn this into a primary key.
Regarding question 2
Creating a new primary key also creates a non-clustered index to associate with (there is already an existing clustered one).
This is not what I am looking for. I want to keep that same index, but also make it a primary key.
Is it possible?
Would that be faster that creating the whole index again? (I hope so)
What could be the consequences? (locks? crash? corrupted data?)
There is little or no difference between a PRIMARY KEY and a UNIQUE constraint on non-nullable columns. So if the columns in question are non-nullable then I suggest you do nothing. The main reason to make a candidate key into a primary key is if you have some software (such as a data modelling tool or other development tool) that expects the key to be identified with a PRIMARY KEY constraint.
Good question.
If you already have a unique index on non nullable columns then you have a candidate key. I'm not aware of any particular benefit of making this an "official" primary key. In fact I have a feeling that not making it a PK will give greater flexibility.
A unique index can allow null
values. A primary key can't.
I believe you can't "mark" an existing index as the primary key. You'd have to drop it and recreate. To avoid stuff, I'd say it'd be good to place a TABLOCKX, HOLDLOCK on the table before doing that.