I know that SQL query will use query cache to receive data instead of reprocess all of the data. Here the question I would like to ask,
I working with a server of database and I'm one of the developer that working on it and I need to do performance testing on queries that i handling
If I clear the query cache
example using FLUSH QUERY CACHE; or RESET QUERY CACHE;,
will it affect others developer or it only clears away my local query cache?
If it will affect others, is there any way to clear locally or allow my query won't use the query cache for testing
Two clarifications to begin with:
MySQL query cache is a server-side feature, there's no such thing as "local cache". You're probably confused by the LOCAL keyword in FLUSH command. As docs explain it's just an alias for NO_WRITE_TO_BINLOG (thus it's related to replication and "local" means "this server").
MySQL will only return cached data if you've enabled the feature and either made it default or opted-in with the SQL_CACHE hint. In my experience, most servers do not have it by default.
Let's now answer your question. At The MySQL Query Cache we can read:
The query cache is shared among sessions, so a result set generated by
one client can be sent in response to the same query issued by another
client.
Which makes sense: a cache that cannot reuse stored data is not as useful.
I don't know what you want to test exactly. Your data should always be fresh:
The query cache does not return stale data. When tables are modified,
any relevant entries in the query cache are flushed.
However you might want to get an idea of how long the query takes to run. You can always opt out with the SQL_NO_CACHE keyword:
The server does not use the query cache. It neither checks the query
cache to see whether the result is already cached, nor does it cache
the query result.
Just take into account that a query that runs for the second time might run faster even without cache because part of the data segments might be already loaded into RAM.
Try using the SQL_NO_CACHE option in your query.This will stop MySQL caching the results
SELECT SQL_NO_CACHE * FROM TABLE
With SQL Server for cached data, you can use DBCC DROPCLEANBUFFERS and force a manul CHECKPOINT.
However it works at the Server (instance) level:
Use DBCC DROPCLEANBUFFERS to test queries with a cold buffer cache without shutting down and restarting the server.
To drop clean buffers from the buffer pool, first use CHECKPOINT to produce a cold buffer cache. This forces all dirty pages for the current database to be written to disk and cleans the buffers. After you do this, you can issue DBCC DROPCLEANBUFFERS command to remove all buffers from the buffer pool.
edited*
SQL Query buffer cache is global but not local. If the buffer or query cache is drop, it drops it globally, and will affect all user using the database server.
Related
I would like to run the same query multiple times to see how much time it takes without the aid of cache memory.
Running the command RESET QUERY CACHE seems not to work because the query takes a really short time on the second run even after the cache reset.
What am i missing?
Your OS is caching a whole lot of data. Then the DBMS itself caches a whole of data. The query cache only holds the output of a previous query. The data the query output is created from may all be in RAM. If you want to run the query without any caching then switch off the host between executions or run a query which will read enough data from disk to overwrite all the cache. But it won't be a realistic measure of ho.w your query will perform in the wild.
Two "caches" affects the query. Query Cache and Buffer Cache(buffer_pool with InnoDB and key_buffer with MyISAM). Query cache affects much more than Buffer Cache because it caches the result of your query, so the same query will not execute again.
To avoid Query Cache, uses SQL_NO_CACHE is a good idea, and you can disable Query Cache in my.cnf and restart mysql.
Buffer cache is managed by mysql to cache data in memory so that your query will not read data from disk(SSD or HDD). If you want to clear it, try to set the 'Buffer cache' small enough and fill it with other data(use SELECT).
You can use the SQL_NO_CACHE in the SELECT query.
https://dev.mysql.com/doc/refman/5.5/en/query-cache-in-select.html
Keep it in mind that Query cache has been removed in later versions of MySQL.
Depending on the storage engine, (e.g innodb), it loads table data into memory as part the buffer pool. This part you can't really control on what tables to load and which tables not to load (or at least not easily).
On an ancient version of MySQL (v.5.1), I had a problem with a view being cached, which would never refresh. I tried RESET QUERY CACHE , FLUSH TABLES, SELECT SQL_NO_CACHE..., etc. And nothing worked. Then, I changed the storage engine for the underlying (single) table I was querying in this view, from InnoDB to MyISAM, and it worked as desired implicitly! There was no need to jump through any hoops to clear or prevent caching!
I'm not sure if this was simply a bug with that old version / storage engine? Please leave comments if you have any knowledge to share on the matter.
Do we need to drop OS cache?
I want to read data from disk not from cache I have disabled
1. query_cache_type=OFF
2. query_cache_size=0
even then when i perform select operation for Id =2 , innodb_buffer_pool_reads changes . If i select Id=3 no change for innodb_buffer_pool_reads.
How do I read next value from disk? Is there any other way to verify whether data is being read from the disk?
[Edit] Thank you all for your response.
I m trying to perform reverse engineering , want to test the execution speed of a select query without cache . So want to disable all cache and read data from disk?
Yes, to completely turn off the Query cache, make both of those settings.
To disable the Query cache for a single SELECT, do SELECT SQL_NO_CACHE ....
But... The QC is not the only caching mechanism. For InnoDB, the buffer_pool caches data and indexes. (MyISAM uses its key_cache and the OS's cache)
Typically, the first time you perform a query (after restarting the server), the disk will need to be hit. Typically, that query (or similar queries) performed after that will not need to hit the disk. Because of "caching", MySQL will hit the disk as little as necessary.
If some other connection in modifying the data you are about to SELECT, do not worry. MySQL will make sure to change the cached copy and/or the disk copy. You will always get the correct value.
InnoDB does things in "blocks" (16KB, typically about 100 rows). That is the unit of disk I/O. Ids 1,2,3 are probably in the same block. Again, MySQL takes care of fetches and changes. It will probably read the block once, cache it for a long time, and eventually write it once, even if there are a lot of changes to the rows in the block.
So how does "Durability" happen? Magic. It involves the InnoDB log file and some extra writes that are done to it. That is another topic; it would take much too long to explain it all.
I want to cache data on MySQL
SET GLOBAL query_cache_size = SOME_SIZE;
Is it all the thing required for caching data [efficiently] in MySQL ?
Do I need to add something extra to use the cache efficiently ?
I don't have good knowledge on data caching but still need to use for performance issue, so if I've missed to give some vital info, answer this question assuming the system is in default state.
I don't usually recommend using the MySQL query cache. It sounds great in theory, but unfortunately isn't a great win for caching efficiently, because access to it from queries is governed by a mutex. That means many concurrent queries queue up to get access to the query cache, and this harms more than it helps if you have a lot of concurrent clients.
It even harms INSERT/UPDATE/DELETE, even though these queries don't have result sets, because they purge query results from the query cache if they update the same table(s). And this purging is subject to the same queueing on the mutex.
A better strategy is to use memcached for scalable caching of specific query results, but this requires you to think about what you want to cache and to write application code to access memcached and fail back to MySQL if the data isn't present in the cache. That's more work, but if you do it right it gives better results.
See TANSTAAFL.
There are quite a few settings used for caching different things within MySQL. This is a good guide to optimizing MySQL:
http://www.fromdual.com/mysql-performance-tuning-key
Be careful, the query cache is very specific in what it does:
The query cache stores the text of a
SELECT statement together with the
corresponding result that was sent to
the client. If an identical statement
is received later, the server
retrieves the results from the query
cache rather than parsing and
executing the statement again.
http://dev.mysql.com/doc/refman/5.6/en/query-cache.html
Therefore, if anything in the related tables change, or the query is even reworded, the cache isn't used. So select * from T where id in (1,2) and select * from T where id in (2,1) are different.
SHOW VARIABLES LIKE '%query_cache%';
Will show you the current settings for the cache. But its not as simple as just turning it on, the queries you run need to have result sets that are cacheable and it would take more than this comments box to explain that.
If you have a particular query that you think should be cached then post it and we may be able to determine if it is cacheable.
I have a slow MySQL query in my application that I need to re-write. The problem is, it's only slow on my production server and only when it's not cached. The first time I run it, it will take 12 seconds, then any time after that it'll be 500 milliseconds.
Is there an easy way to test this query without it hitting the query cache so I can see the results of my refactoring?
MySQL supports to prevent caching single queries. Try
SELECT SQL_NO_CACHE field_a, field_b FROM table;
alternatively you can diasble the query cache for the current session:
SET SESSION query_cache_type = OFF;
See http://dev.mysql.com/doc/refman/5.1/en/query-cache.html
To add to johannes's good answer, what I do is
RESET QUERY CACHE;
This has the slight added advantage of not requiring any changes to either the statements I'm executing or the connection.
A trivial thing to do is to alter the statement you're executing somehow, such as put a random number in a comment, because a queries are located in the cache only if they are byte-identical to some previous query.
I'm testing the speed of some queries in MySQL. The database is caching these queries making it difficult for me to get reliable results when testing how fast these queries are.
Is there a way to disable caching for a query?
System: MySQL 4 on Linux webhosting, I have access to PHPMyAdmin.
Thanks
Try using the SQL_NO_CACHE (MySQL 5.7) option in your query.
(MySQL 5.6 users click HERE )
eg.
SELECT SQL_NO_CACHE * FROM TABLE
This will stop MySQL caching the results, however be aware that other OS and disk caches may also impact performance. These are harder to get around.
Another alternative that only affects the current connection:
SET SESSION query_cache_type=0;
Any reference to current date/time will disable the query cache for that selection:
SELECT *,NOW() FROM TABLE
See "Prerequisites and Notes for MySQL Query Cache Use" # http://dev.mysql.com/tech-resources/articles/mysql-query-cache.html
There is also configuration option: query_cache_size=0
To disable the query cache at server startup, set the query_cache_size system variable to 0. By disabling the query cache code, there is no noticeable overhead. If you build MySQL from source, query cache capabilities can be excluded from the server entirely by invoking configure with the --without-query-cache option.
See http://dev.mysql.com/doc/refman/5.1/en/query-cache.html
You can also run the follow command to reset the query cache.
RESET QUERY CACHE
One problem with the
SELECT SQL_NO_CACHE * FROM TABLE
method is that it seems to only prevent the result of your query from being cached. However, if you're querying a database that is actively being used with the query you want to test, then other clients may cache your query, affecting your results. I am continuing to research ways around this, will edit this post if I figure one out.
I'd Use the following:
SHOW VARIABLES LIKE 'query_cache_type';
SET SESSION query_cache_type = OFF;
SHOW VARIABLES LIKE 'query_cache_type';
Using a user-defined variable within a query makes the query resuts uncacheable. I found it a much better indicator than using SQL_NO_CACHE. But you should put the variable in a place where the variable setting would not seriously affect the performance:
SELECT t.*
FROM thetable t, (SELECT #a:=NULL) as init;
Whilst some of the answers are good, there is a major caveat.
The mysql queries may be prevented from being cached, but it won't prevent your underlying O.S caching disk accesses into memory. This can be a major slowdown for some queries especially if they need to pull data from spinning disks.
So whilst it's good to use the methods above, I would also try and test with a different set of data/range each time, that's likely not been pulled from disk into disk/memory cache.
If you want to disable the Query cache set the 'query_cache_size' to 0 in your mysql configuration file . If its set 0 mysql wont use the query cache.
You must change SQL string. Because SQL string is a cache key.
For example, add a timestamp to a SQL comment.
Function for PHP:
function db_RunSQL($SQL, $NoCacheMode=false)
{
$SQL = (($NoCacheMode) ? '/*'.time().'*/ ' : '') . $SQL;
return mysqli_query(db_SavedConnect(), $SQL);
}