I need to encrypt some data on MySQL slave and using AES_Encrypt for the same. Is it possible to encrypt it while replicating it from the Master database?
Using MySQL 5.6
In MySQL replication, when using binlog_format=STATEMENT, triggers fire on the master and again on the slave, but there is no specific requirement that the triggers be defined identically on the master and slave, or that triggers be provisioned on both.
I say there is no specific requirement, because it is not enforced. If you want the data to be identical on master and slave -- as is almost always the case in a replicated environment -- then yes, the triggers have to be defined identically on both servers... but it is possible to customise the slave triggers -- and one way of doing it would be to define BEFORE INSERT and BEFORE UPDATE triggers on the slave that encrypted the columns before inserts and updates.
This does require STATEMENT replication, which I don't use because it is more delicate and some statements cannot be deterministically replicated in STATEMENT mode.
This would require extra care to ensure that master and slave are always consistent, and would require custom tools since standard table comparison tools would consider master and slave tables to be different.
With binlog_format=ROW, any triggers defined on the slave are ignored for replicated events, because they would be redundant -- the changes done by the triggers on the master will already be replicated to the slave as row events, and executing the triggers on the slave would not have the correct result. Row-based replication uses raw row images, so is always deterministic, unlike statement based replication.
The MariaDB team recognizes that there may be cases where identical data sets are not the objective, such as where you might want to build denormalized query tables on the slave, so in MariaDB 10.1, they introduced a feature to allow slaves to fire triggers for row-based events. This might also be a viable solution, if your master is not using statement-based replication. It does not appear to require the master to run MariaDB, only the slave.
https://mariadb.com/kb/en/mariadb/running-triggers-on-the-slave-for-row-based-events/
It is also possible to create accounts that cannot access all columns, since the MySQL permissions model allows grants to be made at the server, database, table, and column level. Without permission to access a column, you won't see it, in spite of it being there.
Arguably, though, if the data is sensitive, the most correct solution would be to encrypt it on the master.
Related
I have one master and two slaves.
Is it possible to restrict a particular read-only user to query only against the second slave (disallowing him from running any queries on the master and the first slave)?
I see that one can do the following to make un-replicated changes to the master, but what I think I need is to make changes to one slave and not the other.
SET sql_log_bin = {OFF|ON}
And the GRANT syntax allows one to limit what host users come from, but -- as far as I understand -- not which DB server(s) the person can use.
I didn't find much in a web search -- perhaps that's a hint that there's a better way to solve this problem. Basically I'm asking if this can be enforced by the database since the restriction I want applies to just this one user.
For context: a slave is basically just a server that copies every action that happened on the master. Depending on your configuration, the slaves will either just run the same queries that have been executed on the master, or apply a list of changes to individual rows to the slave.
To add a user just for a specific slave, you can do this directly on the slave. Anything you do here will only affect this slave. If your user currently exists on the master (and slaves), you would first have to drop him/remove his permissions, wait until this change has been replicated to all the slaves (which might also depend on your configuration), then add/modify this user directly on the slave.
You may need to temporarily disable a read_only or super_read_only setting (on the slave), which exists to prevent accidently executing something on the slave - but that is what you want to do.
Since your slave now deviates (slightly) from the master, if you would now run a query that alters that user on your master (e.g. drop it again), it might have a different effect on the master and the slave. This will depend on your configuration, but keep it in mind.
I'm currently thinking about the following problem:
A customer has set up a simple master/slave replication between two mariaDB systems. For unknown reasons they have set the flag "Replicate_Wild_Ignore_Table" to skip "logdb.%". Obviously, they decided to skip the skipping of that database and want the logdb to be included in the replication again.
I'm curious now, is it possible to somehow remove that flag and have the database in question be replicated as the rest or is there no way to circumvent the "stop slave, dump master, import dump, recreate replication based on current logpos, start slave" procedure?
You can't assume that the master still has all relevant binlogs that once contained updates to the logdb.% tables. That is, even if you could re-apply those updates, do you have enough history to account for all changes to the tables?
Another risk is if you use statement-based replication, if there were ever statements that referenced both a table in logdb.% and a table in another database, the replication filter has skipped that statement. So for example:
INSERT INTO mydb.mytable SELECT * FROM logdb.othertable;
Therefore even the tables that are not in logdb.% might be compromised. The point is you don't know for sure.
The bottom line is that you should definitely reinitialize the replica now by taking a current backup of the master, and avoid using replication filters in the future.
If you use InnoDB tables, you might consider using Percona XtraBackup to make the process easier. See https://www.percona.com/doc/percona-xtrabackup/2.3/howtos/setting_up_replication.html
I have a couple of servers and I want to prepare a Master - Slave Mysql replication for one database. The both servers have many databases and I don't want to altere the secuence of ID generation. For example, after I have prepared the configuration I don't want to have in the tables just even numbers for the all the IDs in one of servers.
The replicated database (slave server) will be not accesed for write.
Is posible to configure that scenario?
Many thanks in advance.
Update: The server_id has nothing to do with id generation. It just needs to be a unique integer greater than 0 on each server in your replica-set.
Below is my original answer, which was my guess about what you were asking about, because it's the only feature I could think of that has to do with both replication and auto-increment id generation.
You don't need to change id generation for simple replication.
The scenario where you might use auto_increment_increment=2 is the master-master replication, where two servers replicate from each other, and you want to minimize the risk of split-brain if an insert occurs on both servers. But this is not the scenario you describe.
If you have one master, and it's the only server you write changes on directly, and the replica(s) that replicate from that master are all read-only, then you don't need to change the auto_increment_increment.
I wonder if there is any easy way to keep the scheme consistent in two different MySQL clusters. Apart from classic replication, I would like to have a special "replication" which would reproduce all DDL queries (CREATE, ALTER, DROP, ...) on another cluster (namely the master of that cluster).
I don't need the actual data to be replicated.
Has anyone ever done or tried anything like this?
You can filter replication in MySQL based upon the database in which a query was executed. That doesn't prevent you making changes in other databases, however! So you can do;
USE ddl_repl_db;
ALTER TABLE other_db.foo ADD COLUMN <etc>
This relies on you configuring your servers correctly. I haven't set up MySQL replication for a while, but IIRC you can both filter what you send out from the master for replication and what you accept on the slave.
Old but still high in search.
So, on you DDL replica set all tables engine to BLACKHOLE
Lets say I have a datbase of Cars. I have Makes and Models (FK to Makes). I plan on having users track their cars. each Car has a FK to Model. Now, I have a lot of users, and I want to split up my database to distribute load. The Makes and Models tables don't change so much, but they need to be shared across shards. My thought is to use MySQL replication from a master DB of makes and models to each slave database. My question is: Can I safely write to the slave databases assuming I don't write to those tables on the master?
And while on the subject, is there anyway to guarantee one slave database has the latest data? For example, someone just added the 'Taurus' make, and then wants to add their car. Can I ensure that the slave database they are using has the latest master data?
Yes, in general you can safely write to a table on the slaves that is not being written on the master. If you do things like insert auto_increment rows on the slaves and on the master, independently, you will of course have problems. You should configure that table to be excluded from replication entirely, really.
For checking whether you have the latest data, SHOW SLAVE STATUS includes a field Seconds_Behind_Master that tells you whether the slave is up to date. Obviously you want it to be zero. To be certain that inserted and replicated data is present, of course, you need to wait a second and then see that Seconds_Behind_Master is zero.
This was a good solution I gleaned while searching
I included the main point as avilable here:
http://erlycoder.com/43/mysql-master-slave-and-master-master-replication-step-by-step-configuration-instructions-
MySQL master-master replication and autoincrement indexes
If you are using master-slave replication, than most likely you will design your application the way to write to master and read from slave or several slaves. But when you are using master-master replication you are going to read and write to any of master servers. So, in this case the problem with autoincremental indexes will raise. When both servers will have to add a record (different one each server simultaneously) to the same table. Each one will assign them the same index and will try to replicate to the salve, this will create a collision. Simple trick will allow to avoid such collisions on MySQL server.
On the Master 1/Slave 2 add to /etc/my.cnf:
auto_increment_increment= 2
auto_increment_offset = 1
On the Master 2/Slave 1 add to /etc/my.cnf:
auto_increment_increment= 2
auto_increment_offset = 2