I am trying to use Binlog feature of mysql to capture changes and feed them to DWH. (my requirement is not for replication).
By looking at official documentation (https://dev.mysql.com/doc/refman/8.0/en/replication-binlog-encryption.html) - "From MySQL 8.0.14, binary log files and relay log files can be encrypted.."
Does it mean it's not there for versions 5.x ?
I am trying to enable Binlog for mysql 5.x instances, and wanted to make sure of security.
can I assume it is impossible to have encrypted bin logs for mysql 5.x ?
As i went through the 5.7 documentation , it (encryption) is no where mentioned (https://dev.mysql.com/doc/refman/5.7/en/binary-log.html)
I may try to keep the Binary logs for 2 days probably as I am using Nifi to fetch these logs using CaptureChangeMySQL processor and load them to Data warehouse (DWH).. once the DWH is populated, i dont need the logs anymore.
Please help me with your suggestions or understanding..
The version of mysql 5.7 is also possible, and SSL is recommended. Of course, the encryption of statements like DML/DDL is also related to the binlog_fromat parameter.
Related
I am looking for a tool to sync data online from one MySQL server to another MySQL server, the binlog format of source MySQL server is MIXED.
The tools as I know, such as tidb-syncer, pt-table-sync, tidb-syncer need the source binlog format be ROW, pt-table-sync need the source binlog format be STATEMENT.
tidb-syncer is used to sync data from MySQL to TiDB, if you want to sync data from one MySQL to another MySQL why not just use mysql replication itself.
And you can change the binlog format if need.
Today morning I surprisingly found that some data is deleted form my MYSQL database, I am unable to find that data in any table.
Please suggest how to recover data from MySQL database, DOES MYSQL keeps any Log if yes where it is located?
Please suggest any query to get all records.
I am using MYSQL Workbench 5.
If you have binary logs active on your server then you can use mysqlbinlog
You can use the following:
mysqlbinlog binary_log_file > query_log.sql
If you don't have this, then you have probably lost it.
You can look here for more information on how to convert the binary logs to sql.
You can check if binary logging is enabled or not by running the following command:
SHOW VARIABLES LIKE 'log_bin';
For InnoDB tables, if binary logs are not enabled, there is a recovery tool provided by Percona which may help.
Created a script to automate the steps from the Percona tool which recovers the deleted rows (if exist) and provides SQL queries to load the data into database.
Please note:
The time between the deletion of rows and the database stop is
crucial. If pages are reused you can’t recover the data.
I'm trying to test the performance of using memcached on a MySQL server to improve performance.
I want to be able to use the normal MySQL command line, but I can't seem to get it to connect to memcached, even when I specify the right port.
I'm running the MySQL command on the same machine as both the memcached process and the MySQL server.
I've looked around online, but I can't seem to find anything about using memcached other than with program APIs. Any ideas?
Memcached has its own protocol. The MySQL client cannot connect directly to a memcached server.
You may be thinking of the MySQL 5.6 feature that allows MySQL server to respond to connections using a memcached-compatible protocol, and read and write directly to InnoDB tables. See http://dev.mysql.com/doc/refman/5.6/en/innodb-memcached.html
But this does not allow MySQL clients to connect to memcached -- it's the opposite, allowing memcached clients to connect to mysqld.
Re your comment:
The InnoDB memcached interface is not really a caching solution per se, it's a solution for using a familiar key/value API for persistent data in InnoDB tables. InnoDB does do transparent caching of data pages in its buffer pool, but this is no different from conventional data reads with SQL. InnoDB also commits all changes to its transaction log synchronously on commit.
Here's a blog from my colleague at Percona. He tested whether the MySQL 5.6 memcached API could be used as a caching layer, and found that actually using memcached is still superior.
http://www.mysqlperformanceblog.com/2013/03/29/mysql-5-6-innodb-memcached-plugin-as-a-caching-layer/
Here's one conclusion from that blog:
As expected, there is a slowdown for write operations when using the InnoDB version. But there is also a slight increase in the average fetch time.
Since I'm making a full backup of my entire debian system, I was thinking if having a copy of /var/lib/mysql directory is a viable alternative to dumping tables with mysqldump.
are all informations needed contained in that directory?
can single tables be imported in another mysql?
can there be problems while restoring those files on a (probably slightly) different mysql server version?
Yes
Yes if the table is using the MyISAM (default) engine. Not if it's using InnoDB.
Probably not, and if there is, you just need to execute mysql_upgrade to fix them
To avoid getting databases in a inconsistent state, you can either shutdown MySQL or use LOCK TABLES and then FLUSH TABLES before the backup. The second solution is a little better because the MySQL server will remain available during the backup (albeit read only).
This approach is only going to work safely if you shut the database down first. Otherwise you could well end up in an inconsistent state afterwards. Use the /etc/init.d/mysql stop command first. You can then restart it after the backup is taken.
It's perfectly OK as long as you shut down the MySQL sever first and use exactly the same version to retrieve the "backup". Otherwise it isn't.
For a complete discussion of the 2 strategies, you need to read this: https://dev.mysql.com/doc/refman/5.5/en/backup-types.html
The currently best free and open-source solution seems to be Percona's: http://www.percona.com/software/percona-xtrabackup
I'll go with a strong NO.
From my experience, backing up/restoring raw mysql data files can be used only on the same os/server version. It does not work cross platform (eg. ubuntu/macos) with same server versions nor if mysql server versions are different on same platform.
Percona XtraBackup (innobackupex) from Percona MySQL distro will let you do live & differential mysql backup and serve you the backup files that can be restored by copying to /var/lib/mysql/. You need to be running Percona Server for MySQL to use all of this.
I want to create Database replication i try but some problem is there so give me full steps of How To Set Up Database Replication In MySQL
Never done it personally at work but we do use it.
Seems that you want to setup a master/slave setup, the slave beeing the replicated server.
http://dev.mysql.com/doc/refman/5.0/en/replication-howto.html (check the documentation is up to date with your mysql version.)
Here is a step by step instruction how to get MySQL master-slave replication automatically installed and later scaled http://blog.jelastic.com/2017/03/14/scalable-mysql-master-slave-replication/ (if you are ready to run this cluster in the specific cloud)