I've been doing a little research on the MySQL index key for possible database query performance. I have a specific question that hasn't really been answered.
Can you index every field in a table? Or should you just stick to indexing fields that will be in your WHERE clause?
Thanks, sorry if this question has been answered before.
A database index is a data structure that improves the speed of data
retrieval operations on a database table at the cost of additional
writes and storage space to maintain the index data structure. Indexes
are used to quickly locate data without having to search every row in
a database table every time a database table is accessed. Indexes can
be created using one or more columns of a database table, providing
the basis for both rapid random lookups and efficient access of
ordered records.
https://en.wikipedia.org/wiki/Database_index
You should not create INDEX on every fields of table. Use only column which is used for search purpose (i.e WHERE, JOIN).
Note: Index helps to SEARCH faster on table on other hand it has to perform additional INSERT, DELETE and UPDATE so it increase the cost of query.
Consider scenario: You 3 or n queries with 3 or n different fields on same table in this case how to choose index?
It depends on how many times you are executing particular query on table. If you are executing 2 queries very rare then don't create index on that two columns. Create index only on column which is used in query that is executing multiple times.
Related
I have a mysql db table that has fifteen columns. My web api will allow the user to search the table using any of the columns (each column could be used individually or with an combination of other columns).
My question is whether I should create create single instance indexes and then also multiple composite indexes to cover all the search combinations?
Whats the pros and cons?
As I understand the more indexes I create the worst performance for inserting/updates - but this table will only have its rows updated once a day.
PS the table size is in the region of 7 - 8k rows.
I am designing an SQL database (accessed via PHP/MySQL) and have questions about designing the database in a way that helps the website run relatively quickly. My precise question pertains to speed when querying a table with many columns where one column is of type text. I am wondering, if
this table will be frequently queried,
the queries only about 50% of the time will include the text column,
and I specify the columns names so the text column is not returned in those 50% of queries,
will the presence of the text column in the table affect the query speed? As a follow-up, do text columns generally slow down database queries?
Any other general tips on database design to help boost query speed are appreciated, as are any suggestions for books or other references on this topic. Thanks!
afaik there is no difference if you add a text column to a table, as long as you do not use it in the where clause.
if you use it in the where clause it's definately good to have an index on it. Avoid comparsions with like, as they are slower.
I'm not convinced that text columns are much slower than the alternatives.
Specifying the columns to be returned is a good performance choice - as there is no need to move more data than is needed across the wires.
If you get your indexes right you will get far better performance improvements than using text columns will cost.
If you are doing more many more database reads than writes then indexes will improve read speed.
Helping your optimizer by regularly dropping and re-adding indexes will also help as the data shape of your tables will change over time.
The 'text' data type will only slow your queries down if you intend to filter using this column in the WHERE clause of your select statements.
SELECT textColumn
FROM table WHERE varcharColumn LIKE '%Spanner%'
can be optimised more easily than
SELECT varcharColumn
FROM table WHERE textColumn LIKE '%Spanner%'
however
SELECT textColumn
FROM table WHERE integerColumn = 1
performs just as well as
SELECT varcharColumn
FROM table WHERE integerColumn = 1
Some general tips:
As a general rule you should think about how your tables are going to be ordered in your output (by date or alphabetically?) and put an index on that column.
If you are starting out with DB design you should generally have all your tables using an INT Primary Key that is also your IDENTITY column AND Clustered Index. This means that the tables will be physically ordered (on disk) by that column (generally your ID such as PersonID etc), then use Non-clustered indexes on the columns that you are going to filter and order by.
At a later stage when you've built a few DBs I'd recommend you go further into optimising table design by setting your Clustered Index to be the unique column that is most frequently being used to order the table, including using multiple columns as your Clustered Index.
I am currently working on a new system looking at data stored by a CMS about user access logs.
The current table I am looking at extracting data from is currently 5 million rows. This is data spanning about about 11 months. The SQL queries I am making are usually searching on something like uid which is an indexed column.
The question I have out of interest and scalablity is how large does a table need to get when even indexed columns don't speed up searches?
Indexes will always be faster if the table is mostly read. If you expect writes to scale faster than reads, then updating the index may become more expensive than it's worth.
If uid is your primary key, then it will always be indexed and there's really no overhead for this index since MySQL needs a key for each row anyway.
Proper indexes will always speed up queries...that's the point of them. It doesn't matter how large your table is, the point of the index is to provide the DBMS with an avenue of retrieving a subset of a table faster than if it had to read through the entire table row by row.
The database I use is currently mySQL but maybe later MSSQL.
My questing is about how mySQL and msSQL takes care about indexed and nonindexed columns.
Lets say I have a simple table like this:
*table_ID -Auto increase. just a ID, indexed.
*table_user_ID -every user has a unique ID indexed
*table_somOtherID -some data..
*....
Lets say that I have A LOT!! of rows in this table, But the number of rows that every user add to this table is very small (10-100)
And I want to find one o a few specific rows in this table. a row or rows from a specific User(indexed column).
If I use the following WHERE clause:
..... WHERE table_user_ID= 'someID' AND table_someOtherID='anotherValue'.
Will the database first search for the indexed columns, and then search for the "anotherValue" inside of those rows, or how does the database handle this?
I guess the database will increase a lot if I have to index every column in all tables..
But what do you think, is it enough to index those columns that will decrease the number of rows to just ten maybe hundred?
Database optimizers generally work on a cost basis on indexes by looking at all the possible indexes to use based on the query. In your specific case it will see 2 columns - table_user_ID with an index and someOtherID without an index. If you really only have 10-100 rows per userID then the cost of this index will be very low and it will be used. This is because the cardinality is high and the DB can only read the few rows it needs and not touch the other rows for every other user its not interested in. However, if the cost to use the index is very high (very few unique userIDs and many entries per user) it might actually be more efficient to not use the index and scan the whole table to prevent random seeking action as it jumps around the table grabbing rows based on the index.
Once it picks the index then the DB just grabs the rows that match that index (10 to 100 in your case) and try to match them against your other criteria searching for rows where someOtherID='anotherValue'
But the number of rows that every user add to this table is very small (10-100)
You only need to index the user_id. It should give you good performance regardless of your query, as long as it includes the user_id in the filter. Until you have identified other use cases, it will pretty much work as you state
Will the database first search for the indexed columns, and then search for the "anotherValue" inside of those rows, or how does the database handle this?
Yes (in layman terms that is close).
In regards to SQL Server:
The ordering of the indexes are important depending on how you query and how the indexes are structured. If you create an index on the columns
-table_user_id
-table_someotherID
The index is ordered by the table_user_id first. Example:
1-2
1-5
1-6
2-3
2-5
2-6
For the first record on the index, 1 being the table user id, and 2 being some other value.
If you run a query with a where on table_user_id = blah, it will be very fast to use this index, since the table_user_id are indexed in order.
But if you run a query that only uses table_someotherID in the WHERE clause, it might not even use this index, as instead of doing a quick seek in the index for the matching value, it will do a rough scan of the index (which is less efficient than a seek).
Also SQL Server has a INCLUDE feature that associate the columns you want in the SELECT clause to the index you create on the WHERE or JOIN columns.
So to answer your question, it all depends on how you create the indexes and how you query them. You're right not to think about indexing every column, as indexes take up storage and performance hit when you do inserts and updates on the table.
How do you create MySQL index? Multi-column or one column per index?
What's your thinking?
This is something that depends heavily on the data stored in your tables. Is this a db with a single table with queries that are only run against the unique id/PK of the record? If so, then just index a single column, the id column. If not, it may call for indexing across more than one column.
It's a case by case issue that can't be generalized. Indexing can be very tricky and over indexing can actually hurt performance (especially on inserts and updates). Make extensive use of the "EXPLAIN" statement in mysql to see how your index changes affect the way mysql actually queries the data.