I am using MySQL 5.1 version in Windows 2008 server. When I execute below queries:
SELECT * FROM tablename;
It is taking too much time for fetching all the results in that table. This query is listed in the slow query log too while this table has primary key as well as few more index.
I execute below query to check the execution plan:
explain extended select * from tablename;
I found below information:
id=1
select_type=SIMPLE
table=tablename
possible_keys=null
key=null
key_len=null
ref=null
rows=85151
Extra=blank
I thought that it query should use at least primary key by default. Again, I executed below query and found that filtered column has value=100.0
explain extended select * from tablenmae;
Is there any specific reason about why query is not utilizing key?
You are selecting all rows from the table. This is why the whole table (all rows) needs to be scanned.
A key (or index) is only used if you narrow your search (using where). An index is used in that case to pre-select the rows which you want to have without having to actually scan the whole table for the given criteria.
If you don't need to access all the rows at once, try limiting the returned rows using LIMIT.
SELECT * FROM tablename LIMIT 100;
If you want the next 100 rows, use
SELECT * FROM tablename LIMIT 100,100;
and so on.
Other than that approach (referred to as "paging"), there is not much you can do to speed up this query (other than get a faster machine, more RAM, a faster disk, better network if the DMBS is accessed remotely).
If you need to do some processing, consider moving logic (such as filtering) to the DBMS. This can be achieved using the WHERE portion of a query.
Why would it use a key, when there is no filter, nor order? - there's going to be no approach, in this single table query, where a table scan is not going to be at least as fast.
To solve your performance issue, perhaps you have client side processing that could be passed tot he server (after all, you're not really showing 85,151 rows to the end user at once, are you?) - or get a faster disk...
Related
I am trying to speed up a simple SELECT query on a table that has around 2 million entries, in a MariaDB MySQL database. It took over 1.5s until I created an index for the columns that I need, and running it through PhpMyAdmin showed a significant boost in speed (now takes around 0.09s).
The problem is, when I run it through my PHP server (mysqli), the execution time does not change at all. I'm logging my execution time by running microtime() before and after the query, and it takes ~1.5s to run it, regardless of having the index or not (tried removing/readding it to see the difference).
Query example:
SELECT `pair`, `price`, `time` FROM `live_prices` FORCE INDEX
(pairPriceTime) WHERE `time` = '2022-08-07 03:01:59';
Index created:
ALTER TABLE `live_prices` ADD INDEX pairPriceTime (pair, price, time);
Any thoughts on this? Does PHP PDO ignore indexes? Do I need to restart the server in order for it to "acknowledge" that there is a new index? (Which is a problem since I'm using a shared hosting service...)
If that is really the query, then it needs an INDEX starting with the value tested in the WHERE:
INDEX(time)
Or, to make a "covering index":
INDEX(time, pair, price)
However, I suspect that most of your accesses involve pair? If so, then other queries may need
INDEX(pair, time)
especially if you as for a range of times.
To discuss various options further, please provide EXPLAIN SELECT ...
PDO, mysqli, phpmyadmin -- These all work the same way. (A possible exception deals with an implicit LIMIT on phpmyadmin.)
Try hard to avoid the use of FORCE INDEX -- what helps on today's query and dataset may hurt on tomorrow's.
When you see puzzling anomalies in timings, run the query twice. Caching may be the explanation.
The mysql documenation says
The FORCE INDEX hint acts like USE INDEX (index_list), with the addition that a table scan is assumed to be very expensive. In other words, a table scan is used only if there is no way to use one of the named indexes to find rows in the table.
MariaDB documentation Force Index here says this
FORCE INDEX works by only considering the given indexes (like with USE_INDEX) but in addition, it tells the optimizer to regard a table scan as something very expensive. However, if none of the 'forced' indexes can be used, then a table scan will be used anyway.
Use of the index is not mandatory. Since you have only specified one condition - the time, it can choose to use some other index for the fetch. I would suggest that you use another condition for the select in the where clause or add an order by
order by pair, price, time
I ended up creating another index (just for the time column) and it did the trick, running at ~0.002s now. Setting the LIMIT clause had no effect since I was always getting 423 rows (for 423 coin pairs).
Bottom line, I probably needed a more specific index, although the weird part is that the first index worked great on PMA but not through PHP, but the second one now applies to both approaches.
Thank you all for the kind replies :)
I have this query
SELECT id, alias, parent FROM `content`
Is there a way to optimize this query so 'type' is different than 'all'
id - primary, unique
id - index
parent - index
alias - index
....
Note that this query will almost never return more than 1500 rows.
Thank you
Your query is fetching all the rows, so by definition it's going to report "ALL" as the query type in the EXPLAIN report. The only other possibility is the "index" query type, an index-scan that visits every entry in the index. But that's virtually the same cost as a table-scan.
There's a saying that the fastest SQL query is one that you don't run at all, because you get the data some other way.
For example, if the data is in a cache of some type. If your data has no more than 1500 rows, and it doesn't change frequently, it may be a good candidate for putting in memory. Then you run the SQL query only if the cached data is missing.
There are a couple of common options:
The MySQL query cache is an in-memory cache maintained in the MySQL server, and purged automatically when the data in the table changes.
Memcached is a popular in-memory key-value store used frequently by applications that also use MySQL. It's very fast. Another option is Redis, which is similar but is also backed by disk storage.
Turn OFF log_queries_not_using_indexes; it clutters the slow log with red herrings like what you got.
0.00XX seconds -- good enough not to worry.
ALL is actually optimal for fetching multiple columns from 'all' rows of a table.
The orders table has 2m records. There are ~900K unique ship-to-ids.
There is an index on ship_to_id ( the field isint(8)).
The query below takes nearly 10mn to complete. I've run PROCESSLIST which has Command = Query and State = Sending Data.
When I run explain, the existing index is used, and possible_keys is NULL.
Is there anything I should do to speed this query up? Thanks.
SELECT
ship_to_id as customer_id
FROM orders
GROUP BY ship_to_id
HAVING SUM( price_after_discount ) > 0
Does not look like you have a useful index. Try adding an index on price_after_discount, and add a where condition like this:
WHERE price_after_discount > 0
to minimize the number of rows you need to sum as you can obviously discard any that are 0.
Also try running "top" command and look at the io "wait" column while the query is running. If its high, it means your query causes a lot of disk I/O. You can increase various memory buffers if you have the RAM to speed this up (if you're using innodb) or myisam is done through filesystem cacheing. Restarting the server will flush these caches.
If you do not have enough RAM (which you shouldn't need too much for 2M records) then consider a partitioning scheme against maybe ship-to-ids column (if your version of mysql supports it).
If all the orders in that table aren't current (i.e. not going to change again) then you could archive them off into another table to reduce how much data has to be scanned.
Another option is to throw a last_modified timestamp on the table with an index. You could then keep track of when the query is run and store the results in another table (query_results). When it's time to run the query again, you would only need to select the orders that were modified since the last time the query was run, then use that to update the query_results. The logic is a little more complicated, but it should be much faster assuming a low percentage of the orders are updated between query executions.
MySQL will use an index for a group by, at least according to the documentation, as explained here.
To be most useful, all the columns used in the query should be in the index. This prevents the engine from having to reference the original data as well as the index. So, try an index on orders(ship_to_id, price_after_discount).
From the mk-archiver help, we can see there is an option to optimize "seek-then-scan". Any idea how do they do this?
What I'm really looking for is, if I do have a table with one PKey, and queries
SELECT col1,col2 FROM tbl LIMIT 1,10;
SELECT col1,col2 FROM tbl LIMIT 11,20; ...
SELECT col1,col2 FROM tbl LIMIT m,n;
Any way to do this in an optimized way, given m and n are very large values and each select query is initiated in parallel from multiple machines? (will address host/network choking later)
How do others tackle the situation if the table doesn't have a PKey?
*Using MySQL
The default ascending-index optimization causes mk-archiver to
optimize repeated SELECT queries so they seek into the index where the
previous query ended, then scan along it, rather than scanning from
the beginning of the table every time. This is enabled by default
because it is generally a good strategy for repeated accesses.
I believe they are playing directly with the index structures, not relying on SQL. Advantage of access to source code of MySQL. It should be possible to have such an option using SQL, per connection, but with multiple users connect through intermediate (web) servers would be more complicated, if at all possible.
I've linked a MySQL view into MS Access via ODBC, but it's running WAY slow.
It's a simple select, that compares two other selects to find records that are unique to the first select.
SELECT `contacts_onlinedonors`.`contactkey` AS `contactkey`
FROM (`hal9k3-testbed`.`contacts_onlinedonors`
LEFT JOIN `hal9k3-testbed`.`contacts_offlinedonors`
ON(( `contacts_onlinedonors`.`contactkey` =
`contacts_offlinedonors`.`contactkey` )))
WHERE Isnull(`contacts_offlinedonors`.`contactkey`)
The slow query log says it returns 34,000 rows after examining 1.5 Billion. There are only 200,000 in the base table. What the heck?
The field "contactkey" is obviously an index on the table.
First thing to do is to "explain" this query.
See http://dev.mysql.com/doc/refman/5.0/en/explain.html
The idea is to figure out what the mysql server is doing, which indexes it is using, and adding indexes where needed, or rewriting your query so it can use indexes.