I've linked a MySQL view into MS Access via ODBC, but it's running WAY slow.
It's a simple select, that compares two other selects to find records that are unique to the first select.
SELECT `contacts_onlinedonors`.`contactkey` AS `contactkey`
FROM (`hal9k3-testbed`.`contacts_onlinedonors`
LEFT JOIN `hal9k3-testbed`.`contacts_offlinedonors`
ON(( `contacts_onlinedonors`.`contactkey` =
`contacts_offlinedonors`.`contactkey` )))
WHERE Isnull(`contacts_offlinedonors`.`contactkey`)
The slow query log says it returns 34,000 rows after examining 1.5 Billion. There are only 200,000 in the base table. What the heck?
The field "contactkey" is obviously an index on the table.
First thing to do is to "explain" this query.
See http://dev.mysql.com/doc/refman/5.0/en/explain.html
The idea is to figure out what the mysql server is doing, which indexes it is using, and adding indexes where needed, or rewriting your query so it can use indexes.
Related
I am using MySQL 5.1 version in Windows 2008 server. When I execute below queries:
SELECT * FROM tablename;
It is taking too much time for fetching all the results in that table. This query is listed in the slow query log too while this table has primary key as well as few more index.
I execute below query to check the execution plan:
explain extended select * from tablename;
I found below information:
id=1
select_type=SIMPLE
table=tablename
possible_keys=null
key=null
key_len=null
ref=null
rows=85151
Extra=blank
I thought that it query should use at least primary key by default. Again, I executed below query and found that filtered column has value=100.0
explain extended select * from tablenmae;
Is there any specific reason about why query is not utilizing key?
You are selecting all rows from the table. This is why the whole table (all rows) needs to be scanned.
A key (or index) is only used if you narrow your search (using where). An index is used in that case to pre-select the rows which you want to have without having to actually scan the whole table for the given criteria.
If you don't need to access all the rows at once, try limiting the returned rows using LIMIT.
SELECT * FROM tablename LIMIT 100;
If you want the next 100 rows, use
SELECT * FROM tablename LIMIT 100,100;
and so on.
Other than that approach (referred to as "paging"), there is not much you can do to speed up this query (other than get a faster machine, more RAM, a faster disk, better network if the DMBS is accessed remotely).
If you need to do some processing, consider moving logic (such as filtering) to the DBMS. This can be achieved using the WHERE portion of a query.
Why would it use a key, when there is no filter, nor order? - there's going to be no approach, in this single table query, where a table scan is not going to be at least as fast.
To solve your performance issue, perhaps you have client side processing that could be passed tot he server (after all, you're not really showing 85,151 rows to the end user at once, are you?) - or get a faster disk...
I have a query that is running very slowly. The table is was querying has about 100k records and no indexes on most of the columns used in the where clause. I just added indexes on those columns but the query hasn't gotten any faster.
I think this is because when a column is indexed, it's value is written in the index at the time of insertion. I just added the indexes now after all those records were added. So is there a way to "re-run the indexes" on the table?
Edit
Here is the query and explain result:
Oddly enough when I copy the query and run in directly in my SQL manager tool it runs quite fast so may bye the problem is in my application code and not in the query itself.
Mysql keeps consistent indexes. It does not matter if the data is added first, the index is added first, or the data is changed at any time. The same final index will result (assuming the same final data and index type).
Your slow query is not caused by adding the index later. There will be some other reason.
This is an extremely common problem.
Use MySQL explain http://dev.mysql.com/doc/refman/5.0/en/using-explain.html
When you precede a SELECT statement with the keyword EXPLAIN, MySQL displays information from the optimizer about the query execution plan. That is, MySQL explains how it would process the statement, including information about how tables are joined and in which order.
Using these results... verify the index you created is functioning the way you expected.
If not, you will want to tweak your index until you have it working as expected.
You might want to create a new table, create indexes, then insert all elements from old table to new while testing this. It's easier than dropping and re-adding indices a million times.
So I've been struggling to run this query. It takes a really long time.
Its MySQL Innodb. The fields I am using are indexed. Its on a pretty beefy server with around 10gig allocated to the innodb pool config thing.
UPDATE TEMP_account_product p
JOIN products_temp c ON (c.`some_id` = p.`old_someid`)
SET p.`product` = c.id
WHERE p.product IS NULL;
The thing to note here is that both tables contain around 900,000 rows. this line brings back around 800,000 records (WHERE p.product IS NULL;)
I have a feeling I'm kinda screwed here but thought Id try anyway.
I think that possible reasons of slow execution of such type of request can be:
MOST probable - you have an INDEX(es) on updated field and that request is updating lot of rows - in that case MySQL will need to do a lot of work rebuilding that INDEX(es) during UPDATE. In that case just DROP the INDEX(es) before request, and later recreate it (if needed).
JOIN is slow (you can check it by select with that JOIN) - i.e. join is done w/o INDEXES. Add indexes in that case.
slow filtering of WHERE (i.e. MySQL make a full scan to filter), - you can check how fast it is by select with same filter.
I suggest running it in batches, so that you don't need to rely on the query plan to decide not to being the entire result set into memory before it starts doing the updates. Add something like LIMIT 1000 to the query, and then run it until the number of affected rows is zero (technique depends on your environment, but I think it could be done in SQL).
UPDATE, this is not a valid option (as-is).
Sure enough, I overlooked this in the UPDATE docs:
For the multiple-table syntax ... In this case, ORDER BY and LIMIT cannot be used.
is it okay to use the following query? how is the performance?
select * from table where id not in ( 2000 or much more ids here)
my initial test comes up very fast, but I guess it is because I am the only who is using the server right now.
If you have an index it can be very fast.
However there is a bug in MySQL (possibly fixed in MySQL 5.5) where if there is no index, it won't just be slow, it will be incredibly slow. This because the subquery can be detected as a DEPENDENT SUBQUERY (correlated subquery) even when it is not. You can see whether MySQL is using the correct query plan by running EXPLAIN SELECT ... and checking that key is not NULL for your subquery. I have made another post about this bug with some more details:
Why would an IN condition be slower than “=” in sql?
You can also consider rewriting your query to use JOIN instead of IN to avoid this bug.
I have a table of about 800 000 records. Its basically a log which I query often.
I gave condition to query only queries that were entered last month in attempt to reduce the load on a database.
My thinking is
a) if the database goes only through the first month and then returns entries, its good.
b) if the database goes through the whole database + checking the condition against every single record, it's actually worse than no condition.
What is your opinion?
How would you go about reducing load on a dbf?
If the field containing the entry date is keyed/indexed, and is used by the DB software to optimize the query, that should reduce the set of rows examined to the rows matching that date range.
That said, it's a commonly understood that you are better off optimizing queries, indexes, database server settings and hardware, in that order. Changing how you query your data can reduce the impact of a query a millionfold for a query that is badly formulated in the first place, depending on the dataset.
If there are no obvious areas for speedup in how the query itself is formulated (joins done correctly or no joins needed, or effective use of indexes), adding indexes to help your common queries would by a good next step.
If you want more information about how the database is going to execute your query; you can use the MySQL EXPLAIN command to find out. For example, that will tell you if it's able to use an index for the query.