My question baliscally is: if the sytem crashes right when a mysql transaction is executed, will the transaction rollback after system restart?
How are the transactions executed by mysql?
Will MySQL check for unfinished transactions after restart?
I'm asking this because I made a transaction system in php, but I'm not storing the final results anywhere for a future rollback in case of a system crash...
There are different kind of crashes. The MySQL server can crash (like if you kill it) or the whole Operating system can crash (like if you unplug the machine).
Where you should start reading is about the Binary Log and how it works and about the Recovery process for InnoDB engine
Yes definitely you can roll back. Refer the DOC for more details
Related
I have an AWS RDS MySQL 5.7 database with MyISAM tables that I would like to migrate to another RDS in a custom VPC, and once migrated, convert those MyISAM tables to InnoDB.
If I undertood correctly, the only way to create a correct automatic backup is using the following procedure explained here: "Automated Backups with Unsupported MySQL Storage Engines"
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_WorkingWithAutomatedBackups.html#Overview.BackupDeviceRestrictions
Stop all activity to your MyISAM tables (that is, close all sessions).
You can close all sessions by calling the mysql.rds_kill command for each process that is returned from the SHOW FULL PROCESSLIST command.
Lock and flush each of your MyISAM tables
Create a snapshot of your DB instance. When the snapshot has completed, release the locks and resume activity on the MyISAM tables
Has someone done this procedure before?
How is that the snapshots are being created successfully every night from the current RDS DBInstance, even though it contains MyISAM tables?
Thanks!
The problem isn't with snapshot creation. It's what can go wrong when you actually try to use one of the snapshots.
RDS snapshots work by capturing a snapshot if your RDS instance's underlying EBS volume (you can't see this volume, but it's there -- RDS runs on EC2, with "hidden" instances and volumes).
EBS snapshots capture the entire contents of the hard drive exactly as they happened to exist at the moment in time when the snapshot process starts.
What ends up on the snapshot is essentially the same thing that you would have on a MySQL Server if you executed sudo killall -9 mysqld -- it is as if the server had halted everything, immediately, without doing any of the things it normally does to clean up for a graceful shutdown. With RDS, things are not quite that dramatic, because RDS does take some precautions, but fundamentally, this is the nature of what is happening.
When you create an RDS instance from a snapshot, the first thing that happens when the instance starts up is the same thing your hypothetical server would do when you restarted the killed MySQL Server daemon: InnoDB Crash Recovery.
InnoDB Crash Recovery
To recover from a MySQL server crash, the only requirement is to restart the MySQL server. InnoDB automatically checks the logs and performs a roll-forward of the database to the present. InnoDB automatically rolls back uncommitted transactions that were present at the time of the crash.
https://dev.mysql.com/doc/refman/5.7/en/innodb-recovery.html#innodb-crash-recovery
Crash recovery is InnoDB's mechanism for bringing everything back into harmony in it internal data structures and ensure that all data is intact, exactly as your application left it. It's possible because InnoDB is a transactional storage engine. That means a lot of different things, but what it specifically means in this case is that InnoDB doesn't just change table data when you change a table. It goes through a process that can be simplified something like this:
store the proposed changes to disk¹
actually make the changes
mark the changes as complete
What this means is that until the changes are finalized, InnoDB can be interrupted and will subsequently be able to pick up where it left off, without corrupting or losing data.
MyISAM has no such mechanisms. It just writes to the data files, directly. Even if a MyISAM table isn't actively being used, it may still need to be repaired when the server comes up, to clean up its structures. In some circumstances, repairing the table can be impossible, and all or part of the data in the table will be lost.
If your MyISAM tables are flushed and locked when the snapshot occurs, they are in a quiescent state on the disk, as though the server had actually been gracefully shut down before the snapshot had occurred, so they will be stable on the snapshot.
But the snapshot process will always appear to succeed, because the snapshot is just duplicating whatever is on the disk, as it appears at the moment in time when the snapshot gets underway.
The problem is that what the snapshot captured may not be usable, and you have no way of knowing whether the snapshot will be fully viable.
¹ Note that the first step, "store the proposed changes to disk" is related to the system variable innodb_flush_log_at_trx_commit which makes the system slower if set to 1 but also is the safest setting, because your query doesn't actually succeed until that first step is done. A setting of 2 is still reasonably safe, because it still writes the changes but continues without requiring that the operating system confirm that they have actually been written to the hard drive before your query returns success... but in a crash, a transaction your application thinks was committed may or may not have survived.
I'm new to MySQL database and got some issue with table lock/deadlock. We are running a system with a heavy transactions run everyday and sometime deadlock happened. I would like to know what happened to the transactions if they exceeded wait timeout. Are they canceled (roll-back) ? Do we need to manually run the transaction again or did application auto retry the transaction after deadlock is resolved?
I'm using MySQL 5.7 with Innodb engine.
Thanks
it dosen't matter what db you are using ,if you are using a transaction it will only be committed on success ie if u look closely there is a commit transaction command at the end of try which u write , unless that line is invoked No changes to the DB will be made hence you can be assured that it will be roll backed at the situation of timeout error
I was going through the documentation for sync_binlog parameter and found a discrepancy in sync_binlog parameter documentation.
The documentation here http://dev.mysql.com/doc/refman/5.6/en/replication-options-binary-log.html#sysvar_sync_binlog says:
A value of 1 is the safest choice because in the event of a crash you lose at most one commit group from the binary log.
Which essentially means that the data will be updated but might not be there in the binlogs.
However, in the binary log documentation here http://dev.mysql.com/doc/refman/5.6/en/binary-log.html says:
For example, if you are using InnoDB tables and the MySQL server processes a COMMIT statement, it writes many prepared transactions to the binary log in sequence, synchronizes the binary log, and then commits this transaction into InnoDB. If the server crashes between those two operations, the transaction is rolled back by InnoDB at restart but still exists in the binary log.
Which essentially means that the transaction is first written in binlog and then committed to the InnoDB, so there is a chance that in case of crash the row is there in the binlog but doesn't exist in the Database.
I have asked this question in MySQL forum and waiting for the response, but would really appreciate if someone who has used this parameter can give the details on which of the following two behaviour is correct?
Syncing in the server itself is different than syncing with Slaves in a Replication setup.
The binlog (hence, not sync_binlog and "group commit") is not used in the integrity of InnoDB tables on the Master. However, as you point out, it is involved.
The crash you mention will happen after the logs are written to the local machine, hence the local machine (if it comes back up) can recover. But the "group commit" may not make it to the slaves. Even if it gets to the binlog, that does not assure that any Slave (much less, all Slaves) has gotten the value yet.
See also "semi-sync" replication, wherein at least one Slave must acknowledge before the COMMIT is ack'd.
You quoted something that implies that a transaction could get to a Slave, yet be rolled back on the Master. I think there is an open bug on that case.
GTIDs also confuse the issue. It could be that some of the quotes you made predate GTIDs and need updating.
Consider reporting this at http://bugs.mysql.com .
I wanted to set up a new replication slave for MySQL 5.6. Just after a CHANGE MASTER and starting the slave, I saw this line in the error log:
[Warning] Slave SQL: If a crash happens this configuration does not guarantee that the relay log info will be consistent, Error_code: 0
If it matters, these settings are included in my.ini:
skip-name-resolve
skip-host-cache
server-id = 111
report-host = myPC
relay-log-recovery
sync_master_info=1
sync_relay_log=1
sync_relay_log_info=1
replicate-do-db = myDB
skip-slave-start
Replication seems working, but this warning is scary. Any idea what makes MySQL issue this warning?
The warning is about crash-safe replication, a feature added to MySQL 5.6. It does not mean anything scary is going on (basically, replication works as in earlier releases) but it is notifying you that you could do better with the new feature set in MySQL 5.6.
A common request is to have replication crash-safe in the sense that the replication progress information always is in sync with what has actually been applied to the database, even in the event of a crash. Although transactions are not lost if the server crashes, it could require some tweaking to bring the slaves up again.
[...]
Crash-safe masters
If the master crashed when a binary log was rotated, it was possible that some orphan binlog files ended up in the binary log index file. This was fixed in 5.1 but is also a piece in the puzzle of having crash-safe replication.
Writing to the binary log is not an atomic operation, and if a crash occured while writing to the binary log, there were a possibility of a partial event at the end of the binary log.
Crash-safe slaves
If the replication information and data is stored in the same storage engine, it will allow both the data and the replication position to be updated as a single transaction, which means that it is crash-safe.
If the replication information and data is stored in different storage engines, but both support XA, they can still be committed as a single transaction.
The replication information is flushed to disk together with the transaction data. Hence writing the replication information directly to the InnoDB redo log does not offer a speed advantage, but does not prevent the user from reading the replication progress information easily.
The tables can be read from a normal session using SQL commands, which also means that it can be incorporated into such things as stored procedures and stored functions.
See the referenced blog post and the MySQL documentation for details of operation and set up instructions.
When exactly an event on the master(update/insert/alter) is logged on the binary log ?
When it started on the master server, it will be written to the bin log
When it started and completed on the master, then will be logged
In specific, I would like to know the behaviour in case of an ALTER
If answer is 1, there won't be lag between master and slave in normal conditions only due to ALTERs started on the master.
Non-transactional statements are written to the binary log immediately. For MySQL 5+, this means any statement against a storage engine that doesn't support transactions are written immediately.
You shouldn't be executing statements against a non-transactional table inside a transaction though.
Transactional statements are written to the binary log when the transaction is committed. For MySQL 5+, this means any statement against a transactional table.
Currently, ALTER TABLE forces an implicit commit, so this will immediately be written to the binary log. According to MySQL documentation:
Binary logging is done immediately after a statement completes but before any locks are released or any commit is done. This ensures that the log is logged in execution order.