I'm currently running a DELETE query that is taking a lot longer than expected (already 10hrs!). I would like to kill it through phpmyadmin processes, however am concerned about what might happen. Will the roll-back he automatically does take a lot of time also? Current query status shows "updating".
It depends on the stage your query is in right now.
But generally rollback takes about equal time, sometimes even more than the original operation.
As per point 2 of this document, it's not really advisable.
Also, be sure to verify your MySQL version as it has a VERY nasty bug with delete/update queries rollback in some versions as per this article
Restarting/killing the MySQL process won't help as rollback will resume upon restart.
Thumb rule is:
Just let it rollback by its own and don't even think about restarting
the DB as it will resume after restart but worse is your DB won't be
accessible meanwhile.
Yes rollback of huge data (i.e. millions of rows) will be considerably slower than its commit operation and even more slow if parallel InnoDB commits are happening in same database.
Related
This application is running on AWS EC2, using RDS/MySQL (5.7).
We have an audit script that is visiting every user in a MySQL user's table. \
For each user, we
Unconditionally start a transaction,
Possibly make some changes the user's record and to other tables.
Unconditionally commit the transaction.
Now, it's likely (and common) in step 2 that no changes were made to any table. The question arose during the code inspection as to the performance impact of Starting/Committing a transaction for many records when no changes were made.
I've read elsewhere that MySQL is optimized for commits, not rollback. But I've yet to find a discussion on the cost of starting/committing transactions when no work was done.
There's no significant cost in short running transactions.
The minimal cost of transactions starts to happen when changes are actually made.
Rollbacks are only expensive if there is data to be changed during the rollback, otherwise its a fairly empty operation.
Committing (and probably rollback) where there are no changes shouldn't incur a penalty.
Keep in mind that as soon as you read an InnoDB table, whether you explicitly "started a transaction" or not, InnoDB does similar work.
The alternative to starting a transaction is relying on autocommit. But autocommit doesn't mean no transaction happens. Autocommit means the transaction starts implicitly when your query touches an InnoDB table, and the transaction commits automatically as soon as the query is done. You can't run more than one statement, and you can't rollback, but otherwise it's the same as an explicit transaction.
You don't really save anything by trying to avoid starting a transaction.
There is most definitely an overhead, and it can be quite significant. I recently worked with a client who had exactly this issue (lots of empty commits due to a bug in their ORM) and their database was burning through about 30% of CPU on empty commits.
Capture a day of slow log with long_query_time=0 (important!), put it through mysqldumpslow or pt-query-digest, and see if you have a query that is just COMMIT; using up a large amount of CPU.
I have a big query in php using MYSQLI_USE_RESULT not to put all the results into the php memory.
Because if I use MYSQLI_STORE_RESULT it will put all of the data into memory for all results, which takes multiple GB of ram, instead of getting row by row.
It returns millions of rows and each row will generate an api request, so the query will be running for days.
In the mean time, I have other mysql queries that update/insert the tables related to the first query, and I think it cause the undo log to grow without stopping.
I setup innodb_undo_tablespaces=2 and innodb_undo_log_truncate = ON
so the undo log is separated from ibdata1, but the undo files are still big until I kill the queries that have been running for days.
I executed "SET SESSION TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;" before running the long running query, hoping that it would prevent undo file to grow, but it didn't.
The other queries that are updating/inserting have autocommit.
In 1-2 day, the undo file is already 40GB large.
The question : how to prevent this undo file to increase ? As I don't want to keep the previous version of the data while the query is running. It's not important if I get updated data instead of the data that was at the time of the query.
Regardless of your transaction isolation level, a given query will always establish a fixed snapshot, which requires the data to be preserved in the state it was when the query started.
In other words, READ-COMMITTED or READ-UNCOMMITTED allow subsequent queries in the same transaction to see updated data, but a single query will never see a changing data set. Thus concurrent updates to data will force old record versions to be copied to the undo log, and those record versions will be preserved there until your long-running query is finished.
READ-UNCOMMITTED doesn't help any more than READ-COMMITTED. In fact, I've never needed to use READ-UNCOMMITTED for any reason. Allowing "dirty reads" of unfinished transactions breaks rules of ACID databases, and leads to anomalies.
The only way to avoid long-lasting growth of your undo log is to finish your query.
The simplest way to achieve this is to use multiple short-running queries, each fetching a subset of the result. Finish each query in a timely way.
Another solution would be to run the whole query for the millions of rows of result, and store the result somewhere that isn't constrained by InnoDB transaction isolation.
MyISAM table
Message queue
Plain file on disk
Cache like Memcached or Redis
PHP memory (but you said you aren't comfortable with this because of the size)
what timestamps are used for calculating the query_time parameter in the mysql slow query log? Can't find any definition of that parameter.
Only thing i found is
The time to acquire the initial locks is not counted as execution time.
from: http://dev.mysql.com/doc/refman/5.0/en/slow-query-log.html
I'm asking this question because i want to know if the time, given by the 'request_time' in slow log, includes waiting times in queues (if there are some).
If there are some queues, is there a possibility to log the current waiting queue length when a new query is going to be executed (preferably in the slow log).
If there are no queues, how does mysql handle incoming queries if all threads are currently executing some query?
Update: Remain those queries in the TCP-Buffer till they can be executed by a thread?
Links to further reading are welcome.
Regards
Korbinian
There are no queues like you describe.
When the query starts, the connection is dedicated to running it. It may be blocked by any of a large number of "mutexes" because of various things shared between connections. As the query runs, it may have to wait for I/O to read blocks from disk.
The query time in the slowlog is essentially the clock-on-the-wall time for the query. A query that normally takes 1 second may take 3 seconds if there are other connections hanging onto mutexes or doing lots of I/O.
Caching will let a query run faster. The first time you run a query (after restarting mysql), it will run slow because of all the uncached I/O.
If you are using Engine MyISAM, the "lock_time" will often be significant because of MyISAM's table locking. InnoDB rarely shows more than a fraction of a millisecond in lock_time.
In older version of MySQL, the timer was similar to TIMESTAMP with a 1-second resolution. In newer versions, it has much higher resolution.
The time at the start of a slowlog entry is the timestamp of when the query started. You may notice that the slowlog entries are not always in order according to that. This is simply because a slowlog entry is not written to the file until the query finishes.
OK, there is a queue -- but it is almost never needed. And it is at the time of establishing the connection. If there are already max_connection connections, then see back_log. I repeat, that is not a queue for executing queries.
I have an application that has been running fine for quite awhile, but recently a couple of items have started popping up in the slow query log.
All the queries are complex and ugly multi join select statements that could use refactoring. I believe all of them have blobs, meaning they get written to disk. The part that gets me curious is why some of them have a lock time associated with them. None of the queries have any specific locking protocols set by the application. As far as I know, by default you can read against locks unless explicitly specified.
so my question: What scenarios would cause a select statement to have to wait for a lock (and thereby be reported in the slow query log)? Assume both INNODB and MYISAM environments.
Could the disk interaction be listed as some sort of lock time? If yes, is there documentation around that says this?
thanks in advance.
MyISAM will give you concurrency problems, an entire table is completely locked when an insert is in progress.
InnoDB should have no problems with reads, even while a write/transaction is in progress due to it's MVCC.
However, just because a query is showing up in the slow-query log doesn't mean the query is slow - how many seconds, how many records are being examined?
Put "EXPLAIN" in front of the query to get a breakdown of the examinations going on for the query.
here's a good resource for learning about EXPLAIN (outside of the excellent MySQL documentation about it)
I'm not certain about MySql, but I know that in SQL Server select statements do NOT read against locks. Doing so will allow you to read uncommitted data, and potentially see duplicate records or miss a record entirely. The reason for this is because if another process is writing to the table, the database engine may decide it's time to reorganize some data and shifts it around on disk. So it moves a record you already read to the end and you see it again, or it moves one from the end up higher where you've already past.
There's a guy on the net somewhere who actually wrote a couple of scripts to prove that this happens and I tried them once and it only took a few seconds before a duplicate showed up. Of course, he designed the scripts in a fashion that would make it more likely to happen, but it proves that it definitely can happen.
This is okay behaviour if your data doesn't need to be accurate and can certainly help prevent deadlocks. However, if you're working on an application dealing with something like people's money then that's very bad.
In SQL Server you can use the WITH NOLOCK hint to tell your select statement to ignore locks. I'm not sure what the equivalent in MySql would be but maybe someone else here will say.
I have a service that sits on top of a MySQL 5.5 database (INNODB). The service has a background job that is supposed to run every week or so. On a high level the background job does the following:
Do some initial DB read and write in one transaction
Execute UMQ (described below) with a set of parameters in one transaction.
If no records are returned we are done!
Process the result from UMQ (this is a bit heavy so it is done outside of any DB
transaction)
Write the outcome of the previous step to DB in one transaction (this
writes to tables queried by UMQ and ensures that the same records are not found again by UMQ).
Goto step 2.
UMQ - Ugly Monster Query: This is a nasty database query that joins a bunch of tables, has conditions on columns in several of these tables and includes a NOT EXISTS subquery with some more joins and conditions. UMQ includes ORDER BY also has LIMIT 1000. Even though the query is bad I have done what I can here - there are indexes on all columns filtered on and the joins are all over foreign key relations.
I do expect UMQ to be heavy and take some time, which is why it's executed in a background job. However, what I'm seeing is rapidly degrading performance until it eventually causes a timeout in my service (maybe 50 times slower after 10 iterations).
First I thought that it was because the data queried by UMQ changes (see step 4 above) but that wasn't it because if I took the last query (the one that caused the timeout) from the slow query log and executed it myself directly I got the same behavior only until I restated the MySQL service. After restart the exact query on the exact same data that took >30 seconds before restart now took <0.5 seconds. I can reproduce this behavior every time by restoring the database to it's initial state and restarting the process.
Also, using the trick described in this question I could see that the query scans around 60K rows after restart as opposed to 18M rows before. EXPLAIN tells me that around 10K rows should be scanned and the result of EXPLAIN is always the same. No other processes are accessing the database at the same time and the lock_time in the slow query log is always 0. SHOW ENGINE INNODB STATUS before and after restart gives me no hints.
So finally the question: Does anybody have any clue of why I'm seeing this behavior? And how can I analyze this further?
I have the feeling that I need to configure MySQL differently in some way but I have searched and tested like crazy without coming up with anything that makes a difference.
Turns out that the behavior I saw was the result of how the MySQL optimizer uses InnoDB statistics to decide on an execution plan. This article put me on the right track (even though it does not exactly discuss my problem). The most important thing I learned from this is that MySQL calculates statistics on startup and then once in a while. This statistics is then used to optimize queries.
The way I had set up the test data the table T where most writes are done in step 4 started out as empty. After each iteration T would contain more and more records but the InnoDB statistics had not yet been updated to reflect this. Because of this the MySQL optimizer always chose an execution plan for UMQ (which includes a JOIN with T) that worked well when T was empty but worse and worse the more records T contained.
To verify this I added an ANALYZE TABLE T; before every execution of UMQ and the rapid degradation disappeared. No lightning performance but acceptable. I also saw that leaving the database for half an hour or so (maybe a bit shorter but at least more than a couple of minutes) would allow the InnoDB statistics to refresh automatically.
In a real scenario the relative difference in index cardinality for the tables involved in UMQ will look quite different and will not change as rapidly so I have decided that I don't really need to do anything about it.
thank you very much for the analysis and answer. I've been searching this issue for several days during ci on mariadb 10.1 and bacula server 9.4 (debian buster).
The situation was that after fresh server installation during a CI cycle, the first two tests (backup and restore) runs smoothly on unrestarted mariadb server and only the third test showed that one particular UMQ took about 20 minutes (building directory tree during restore process from the table with about 30k rows).
Unless the mardiadb server was restarted or table has been analyzed the problem would not go away. ANALYZE TABLE or the restart changed the cardinality of the fields and internal query processing exactly as stated in the linked article.