I need to perform some scripted actions, which may take a while (like maybe a minute). At the beginning of these actions, I take some measures from the MySQL DB and it's important that they do not change until the actions are done. The DB has dozens of tables since it belongs to a quite old fashioned but huge CMS, and the CMS users have a dozen options to modify it.
I do not even want to change anything in the time my scripts runs in the DB myself, it just shall be frozen. It's not a Dump or Update. But tables should be kept open for reading for everyone, to prevent visitors of the connected homepage from getting errors.
If the database altering actions, which may be performed by other CMS users in the meantime would be triggered after the DB is unlocked again, it would be perfect, but if they fail, I would not mind.
So I thought at the beginning of the script I lock the tables down with
lock first_table write;
lock second_table write;
...
And after I do
unlock tables
I think that should do exactly what I want. But can I archive this for all tables of the db without naming them explicitly, to make this more futureproof?
This does not work for sure:
lock tables (select TABLE_NAME from information_schema.tables
where table_schema='whatever') write;
Another question would be, if someone can answer this on the fly, if I would have to perfom the lock/unlock with another MYSQL user than the one used by the CMS. If I understood this right, then yes.
Below is the statement to lock all tables (actually it creates a single global lock):
FLUSH TABLES WITH READ LOCK;
Then release it with:
UNLOCK TABLES;
Mysqldump does this, for example, unless you are backing up only transactional tables and use the --single-transaction option.
Read http://dev.mysql.com/doc/refman/5.7/en/flush.html for more details about FLUSH TABLES.
Re your comment:
Yes, this takes a global READ LOCK on all tables. Even your own session cannot write. My apologies for overlooking this requirement of yours.
There is no equivalent global statement to give you a write lock. You'll have to lock tables by name explicitly.
There's no syntax for wildcard table names, nor is there syntax for putting a subquery in the LOCK TABLES statement.
You'll have to get a list of table names and build a dynamic SQL query.
Related
SO, we are trying to run a Report going to screen, which will not change any stored data.
However, it is complex, so needs to go through a couple of (TEMPORARY*) tables.
It pulls data from live tables, which are replicated.
The nasty bit when it comes to take the "eligible" records from
temp_PreCalc
and populate them from the live data to create the next (TEMPORARY*) table output
resulting in effectively:
INSERT INTO temp_PostCalc (...)
SELECT ...
FROM temp_PreCalc
JOIN live_Tab1 ON ...
JOIN live_Tab2 ON ...
JOIN live_Tab3 ON ...
The report is not a "definitive" answer, expectation is that is merely a "snapshot" report and will be out-of-date as soon as it appears on screen.
There is no order or reproducibility issue.
So Ideally, I would turn my TRANSACTION ISOLATION LEVEL down to READ COMMITTED...
However, I can't because live_Tab1,2,3 are replicated with BIN_LOG STATEMENT type...
The statement is lovely and quick - it takes hardly any time to run, so the resource load is now less than it used to be (which did separate selects and inserts) but it waits (as I understand it) because of the SELECT that waits for a repeatable/syncable lock on the live_Tab's so that any result could be replicated safely.
In fact it now takes more time because of that wait.
I'd like to SEE that performance benefit in response time!
Except the data is written to (TEMPORARY*) tables and then thrown away.
There are no live_ table destinations - only sources...
these tables are actually not TEMPORARY TABLES but dynamically created and thrown away InnoDB Tables, as the report Calculation requires Self-join and delete... but they are temporary
I now seem to be going around in circles finding an answer.
I don't have SUPER privilege and don't want it...
So can't SET BIN_LOG=0 for this connection session (Why is this a requirement?)
So...
If I have a scratch Database or table wildcard, which excludes all my temp_ "Temporary" tables from replication...
(I am awaiting for this change to go through at my host centre)
Will MySQL allow me to
SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED;
INSERT INTO temp_PostCalc (...)
SELECT ...
FROM temp_PreCalc
JOIN live_Tab1 ON ...
JOIN live_Tab2 ON ...
JOIN live_Tab3 ON ...
;
Or will I still get my
"Cannot Execute statement: impossible to write to binary log since
BINLOG_FORMAT = STATEMENT and at least one table uses a storage engine
limited to row-based logging..."
Even though its not technically true?
I am expecting it to, as I presume that the replication will kick in simply because it sees the "INSERT" statement, and will do a simple check on any of the tables involved being replication eligible, even though none of the destinations are actually replication eligible....
or will it pleasantly surprise me?
I really can't face using an unpleasant solution like
SELECT TO OUTFILE
LOAD DATA INFILE
In fact I dont think I could even use that - how would I get unique filenames? How would I clean them up?
The reports are run on-demand directly by end users, and I only have MySQL interface access to the server.
or streaming it through the PHP client, just to separate the INSERT from the SELECT so that MySQL doesnt get upset about which tables are replication eligible....
So, it looks like the only way appears to be:
We create a second Schema "ScratchTemp"...
Set the dreaded replication --replicate-ignore-db=ScratchTemp
My "local" query code opens a new mysql connection, and performs a USE ScratchTemp;
Because I have selected the default database of the "ignore"d one - none of my queries will be replicated.
So I need to take huge care not to perform ANY real queries here
Reference my scratch_ tables and actual data tables by prefixing them all on my queries with the schema qualified name...
e.g.
INSERT INTO LiveSchema.temp_PostCalc (...) SELECT ... FROM LiveSchema.temp_PreCalc JOIN LiveSchema.live_Tab1 etc etc as above.
And then close this connection just as soon as I can, as it is frankly dangerous to have a non-replicated connection open....
Sigh...?
I'm looking to do some conversions of database tables to utf-8, as well as add some indexes to some large tables. I'm hoping to lock the tables for all other users, but my current user when performing these operations. Is there an easy way to do this with MySQL 4.x?
I've done some googling but haven't had luck finding a lock that allows me to still alter in the current session.
From what I read in the following: http://dev.mysql.com/doc/refman/5.0/en/lock-tables.html, the LOCK TABLES statement is only good for the current session. How can I permanently lock specific tables to make them read-only, for all connection sessions, until I explicitly unlock them?
I dont think you can simply lock any table like that. The best way you can do so is to revoke all update, insert and delete privileges
Somthing like this:
REVOKE DROP, INSERT, TRUNCATE ON database.table FOR 'user'#'host'
For large, time-consuming operations like that, one good option can be to copy the data to a new location, to do your manipulations. This essentially makes a snapshot of the data, leaving the database unhindered (and possibly still accepting reads and writes) while you perform your operation.
Stop MySQL
Copy data files to new location.
Restart MySqL
Perform manipulations on the data copy.
Delete copy (by way of DROP TABLE)
Can I call update/insert into table(tb1) after calling: LOCK TABLES tb1 READ?
I've read the MySQL manual which says about read lock:
The session that holds the lock can read the table (but not write it).
I'm looking for a lock which allows me to READ/WRITE a table but allowing others only READ access.
How to do it?
Mine is MyISAM database.
You can't do it in MySQL. The best way is to use WRITE lock but then all other sessions will have to wait for ability to read from table.
You probably want to read these first :
http://dev.mysql.com/doc/refman/5.0/en/locking-issues.html
http://en.wikipedia.org/wiki/Isolation_(database_systems)
Read also about lock escalations.
I had issues with earlier versions(4.x) of mysql where I couldn't update the same
table where I was reading from.
So after reading Performance in PDO / PHP / MySQL: transaction versus direct execution in regards to performance issues I was thinking about I did some research on locking tables in MySQL.
On http://dev.mysql.com/doc/refman/5.0/en/table-locking.html
Table locking enables many sessions to
read from a table at the same time,
but if a session wants to write to a
table, it must first get exclusive
access. During the update, all other
sessions that want to access this
particular table must wait until the
update is done.
This part struck me particularly because most of our queries will be updates rather than inserts. I was wondering if one created a table called foo on which all updates/inserts were carried out and then a view called foo_view (A copy of foo, or perhaps foo and a linkage of several other tables plus foo) on which all selects occurred, would this locking issue still occur?
That is, would SELECT queries on foo_view still have to wait for an update to finish on foo?
Another brief question my colleague asked. Does this affect caching? I.e. if the SELECT is cached will it hit the cache and return results, or will it wait for the lock to finish first?
Your view will experience the same locking as the underlying tables.
From the MySQL Reference page on locking:
MySQL grants table write locks as
follows:
If there are no locks on the table, put a write lock on it.
Otherwise, put the lock request in the write lock queue.
MySQL grants table read locks as
follows:
If there are no write locks on the table, put a read lock on it.
Otherwise, put the lock request in the read lock queue.
It's worth mentioning that this depends on the database engine you are using. MyISAM will follow the steps above and lock the entire table (even if it is split into multiple partitions) where an engine like InnoDB will do row level locking instead.
If you're not reaching the necessary performance benchmarks with MyISAM and you have shown your bottleneck is waiting on table locks via updates, I would suggest changing the storage engine of your table to InnoDB.