Can not run "Percona XtraDB Cluster 5.7" on Centos 7 - mysql

I want used "Percona XtraDB Cluster 5.7".
So I installed "Percona XtraDB Cluster" by official guide.
https://www.percona.com/doc/percona-xtradb-cluster/5.7/install/yum.html#yum
But when I add nodes to cluster, my mysql coudn't run.
$ sudo service mysql start
Job for mysql.service failed. See 'systemctl status mysql.service' and 'journalctl -xn' for details.
$ sudo cat /var/log/messages
...
Jan 13 04:19:25 localhost mysqld_safe: 2017-01-12T19:19:25.588436Z mysqld_safe Skipping wsrep-recover for empty datadir: /var/lib/mysql
Jan 13 04:19:25 localhost mysqld_safe: 2017-01-12T19:19:25.590385Z mysqld_safe Assigning 00000000-0000-0000-0000-000000000000:-1 to wsrep_start_position
Jan 13 04:19:27 localhost mysql-systemd: State transfer in progress, setting sleep higher
Jan 13 04:19:40 localhost mysqld_safe: 2017-01-12T19:19:40.723030Z mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
Jan 13 04:19:47 localhost mysql-systemd: /usr/bin/mysql-systemd: 137 行: kill: (19791) - そのようなプロセスはありません
Jan 13 04:19:47 localhost mysql-systemd: ERROR! mysqld_safe with PID 19791 has already exited: FAILURE
Jan 13 04:19:47 localhost systemd: mysql.service: control process exited, code=exited status=1
Jan 13 04:19:47 localhost mysql-systemd: WARNING: mysql pid file /var/run/mysqld/mysqld.pid empty or not readable
Jan 13 04:19:47 localhost mysql-systemd: ERROR! mysql already dead
Jan 13 04:19:47 localhost systemd: mysql.service: control process exited, code=exited status=2
...
I used centos7 on 3 virtual machine.
And my /etc/my.cnf is default except following code.
wsrep_provider=/usr/lib64/galera3/libgalera_smm.so
wsrep_cluster_name=pxc-cluster
wsrep_cluster_address=gcomm://192.168.70.61,192.168.70.62,192.168.70.63
wsrep_node_name=pxc1
wsrep_node_address=192.168.70.61
wsrep_sst_method=xtrabackup-v2
wsrep_sst_auth=sstuser:passw0rd
pxc_strict_mode=ENFORCING
binlog_format=ROW
default_storage_engine=InnoDB
innodb_autoinc_lock_mode=2
What should I check?

For the first node, you should bootstrap it, instead of just starting it. On CentOS7 you can do so with:
systemctl start mysql#bootstrap.service
Then, you can go ahead and start nodes 02 and 03 normally:
systemctl start mysql

Related

Moving MySQL datadir on development machine

Operating System: Ubuntu 18.04
MySQL version: Ver 8.0.19 for Linux on x86_64 (MySQL Community Server - GPL)
My system is set-up so that Ubuntu is running on a 128gb SSD, with a separate 1TB SSD for storage. This system is dual-boot with Windows 10 Pro and Ubuntu 18.04. The storage drive is NTFS formatted.
I would like to move the MySQL datadir over to the storage drive, because my OS drive isn't big enough.
I've been following the tutorial on this URL: https://www.digitalocean.com/community/tutorials/how-to-move-a-mysql-data-directory-to-a-new-location-on-ubuntu-16-04
I've gotten to "Step 4 — Restarting MySQL" in the tutorial. Up to this point, I've followed everything to a T, no missed commas, forward slashes, etc. All done exactly as described (I've checked to make sure this is true numerous times)
When I enter the terminal command
sudo systemctl start mysql
I get the following output
Job for mysql.service failed because the control process exited with error code.
See "systemctl status mysql.service" and "journalctl -xe" for details.
Following this output, I enter the following command into the terminal
systemctl status mysql.service
and receive the following output
● mysql.service - MySQL Community Server
Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Tue 2020-03-31 22:36:46 CDT; 38s ago
Docs: man:mysqld(8)
http://dev.mysql.com/doc/refman/en/using-systemd.html
Process: 7368 ExecStart=/usr/sbin/mysqld (code=exited, status=1/FAILURE)
Process: 7316 ExecStartPre=/usr/share/mysql-8.0/mysql-systemd-start pre (code=exited, status=0/SUCCESS)
Main PID: 7368 (code=exited, status=1/FAILURE)
Status: "Server startup in progress"
Error: 13 (Permission denied)
Mar 31 22:36:46 user-desktop mysqld[7368]: 2020-04-01T03:36:46.436713Z 0 [Warning] [MY-010091] [Server] Can't create test file /media/user/storage/mysql_datadir/mysql/mysqld_tmp_file_case_insensitive_test.lower-test
Mar 31 22:36:46 user-desktop mysqld[7368]: 2020-04-01T03:36:46.436796Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.19) starting as process 7368
Mar 31 22:36:46 user-desktop mysqld[7368]: 2020-04-01T03:36:46.439912Z 0 [Warning] [MY-010091] [Server] Can't create test file /media/user/storage/mysql_datadir/mysql/mysqld_tmp_file_case_insensitive_test.lower-test
Mar 31 22:36:46 user-desktop mysqld[7368]: 2020-04-01T03:36:46.439923Z 0 [Warning] [MY-010159] [Server] Setting lower_case_table_names=2 because file system for /media/user/storage/mysql_datadir/mysql/ is case insensitive
Mar 31 22:36:46 user-desktop mysqld[7368]: 2020-04-01T03:36:46.439961Z 0 [ERROR] [MY-013276] [Server] Failed to set datadir to '/media/user/storage/mysql_datadir/mysql/' (OS errno: 13 - Permission denied)
Mar 31 22:36:46 user-desktop mysqld[7368]: 2020-04-01T03:36:46.440049Z 0 [ERROR] [MY-010119] [Server] Aborting
Mar 31 22:36:46 user-desktop mysqld[7368]: 2020-04-01T03:36:46.440146Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.19) MySQL Community Server - GPL.
Mar 31 22:36:46 user-desktop systemd[1]: mysql.service: Main process exited, code=exited, status=1/FAILURE
Mar 31 22:36:46 user-desktop systemd[1]: mysql.service: Failed with result 'exit-code'.
Mar 31 22:36:46 user-desktop systemd[1]: Failed to start MySQL Community Server.
Seeing that this was a permissions error (which should have been avoided using the -av tags when copying the original directory to the new drive) I entered in the following command to give open permissions to the entire drive.
sudo chmod a+rwx /media/user/storage/
to check the permissions I enter
ls -ls /media/user/storage/mysql_datadir
with output
total 12
12 drwxrwxrwx 1 root root 12288 Mar 31 02:45 mysql
I've also made sure the full file path to the new datadir is lowercase to avoid conflict between the two file systems.
I still cannot get MySQL to start.
I'd appreciate any help. Thank you in advance.
Edit:
namei -mo /media/user/storage/mysql_datadir
f: /media/user/storage/mysql_datadir
drwxr-xr-x root root /
drwxr-xr-x root root media
drwxr-x--- root root user
drwxrwxrwx root root storage
drwxrwxrwx root root mysql_datadir
Based on your post seems that you are having permission issue along with owner of the "data directory" must always be mysql.
STOP your mysql service if it's running.
Consider below example:
my.cnf looks:
#
# Original path change to avoid space issue on server
#
datadir=/var/lib/mysql
Now you change this datadir in your my.cnf file like:
#
# Original path change to avoid space issue on server
#
#datadir=/var/lib/mysql
datadir=/media/user/storage/mysql_datadir
After this check the permission:
cd /media/user/storage/mysql_datadir
ls -al
output:
drwxr-x--x 40 mysql mysql 4096 Mar 27 00:01 mysql
If this is not output then execute below command:
chmod -Rf 0751 /media/user/storage/mysql_datadir
chown -Rf mysql:mysql /media/user/storage/mysql_datadir
Finally
START your mysql service now.
I had the same problem/solution as Pratik, after following the instructions to define a new datadir in
/etc/mysql/mysql.conf.d/mysqld.conf
and
/etc/apparmor.d/usr.sbin.mysqld
I modified my fstab to mount to
/mnt
instead of
/media/[username]
and mysql start up without any errors. no additional permissions to set other than the mysql directory. My issue was on Ubuntu 18.04 must be something special about the /media mount point

Unable to restart MySQL server

I restarted MySQL server in one of the nodes in Percona Cluster.
Since the restart took much time, I interrupted the process. I tried restarting the MySQL server again. I got the following error:
Stale sst_in_progress file in datadir
I followed this link, https://www.percona.com/forums/questions-discussions/percona-xtradb-cluster/46846-sql-cluster-issue-need-help-please, and deleted the sst_in_progress file as mentioned in it.
Now, when I try restarting the MySQL server, I am getting this:
● mysql.service - LSB: Start and stop the mysql (Percona XtraDB Cluster) daemon
Loaded: loaded (/etc/init.d/mysql; bad; vendor preset: enabled)
Active: failed (Result: exit-code) since Wed 2018-03-14 11:04:07 IST; 16min ago
Docs: man:systemd-sysv-generator(8)
Process: 23568 ExecStart=/etc/init.d/mysql start (code=exited, status=1/FAILURE)
Mar 14 11:04:00 systemd[1]: Starting LSB: Start and stop the mysql (Percona XtraDB Cluster) daemon...
Mar 14 11:04:00 mysql[23568]: * Starting MySQL (Percona XtraDB Cluster) database server mysqld
Mar 14 11:04:00 /etc/init.d/mysql[23614]: MySQL PID not found, pid_file detected/guessed: /var/run/mysqld/mysqld.pid
Mar 14 11:04:07 mysql[23568]: * The server quit without updating PID file (/var/run/mysqld/mysqld.pid).
Mar 14 11:04:07 mysql[23568]: ...fail!
Mar 14 11:04:07 systemd[1]: mysql.service: Control process exited, code=exited status=1
Mar 14 11:04:07 systemd[1]: Failed to start LSB: Start and stop the mysql (Percona XtraDB Cluster) daemon.
Mar 14 11:04:07 systemd[1]: mysql.service: Unit entered failed state.
Mar 14 11:04:07 systemd[1]: mysql.service: Failed with result 'exit-code'.
One more thing is that no log is getting written to mysql-error.log file during restart because of which I am not able to continue debugging.
The best solution here, without being able to see more information, is to simply rm -rf $datadir and start the node back up. It will indeed SST which, depending on the dataset, will take a while. Estimate 1 hour for every 100GB of data over gigE.
If joiner node taking too much time so you can increase gcache.size upto 1 GB then restart joiner node, so if data is already there on new node so it will choose IST instead of SST.
set in my.cnf
wsrep_provider_options="gcache.size=1G"

MySQL Galera cluster can't initiate first node

I'm using Ubuntu 16.04 and installed MySQL version 5.7 via apt-get in AWS.
After that, I've managed to upgrade the MySQL to Galera Cluster by following the Documentation from the official website using apt-get install galera-3 galera-arbitrator-3 mysql-wsrep-5.6
After that I've configured the file /etc/mysql/conf.d/galera.cnf to be:
[mysqld]
binlog_format=ROW
default-storage-engine=innodb
innodb_autoinc_lock_mode=2
bind-address=0.0.0.0
# Galera Provider Configuration
wsrep_on=ON
wsrep_provider=/usr/lib/galera/libgalera_smm.so
# Galera Cluster Configuration
wsrep_cluster_name="test_cluster"
wsrep_cluster_address="gcomm://this_node_ip,second_ip,third_ip"
# Galera Synchronization Configuration
wsrep_sst_method=rsync
# Galera Node Configuration
wsrep_node_address="this_node_ip"
wsrep_node_name="this_node_name"
The step above is repeated for the other 2 nodes (with it's own IP address). And stop the mysql with the following command sudo service mysql stop for each of the node.
And in the first node, I'm trying to start with the following command:
sudo service mysql start --wsrep-new-cluster
But it returns: Job for mysql.service failed because the control process exited with error code. See "systemctl status mysql.service" and "journalctl -xe" for details.
In the systemctl status mysql.service, it says:
● mysql.service - LSB: start and stop MySQL
Loaded: loaded (/etc/init.d/mysql; bad; vendor preset: enabled)
Active: failed (Result: exit-code) since Fri 2018-02-09 15:53:16 +08; 33s ago
Docs: man:systemd-sysv-generator(8)
Process: 6264 ExecStart=/etc/init.d/mysql start (code=exited, status=1/FAILURE)
Feb 09 15:53:15 systemd[1]: Starting LSB: start and stop MySQL...
Feb 09 15:53:15 mysql[6264]: Starting MySQL
Feb 09 15:53:15 mysql[6264]: .180209 15:53:15 mysqld_safe Can't log to error log and syslog at the same time. Remove all --log-error configuration options for --syslog to take effect.
Feb 09 15:53:15 mysql[6264]: 180209 15:53:15 mysqld_safe Directory '/var/run/mysqld' for UNIX socket file don't exists.
Feb 09 15:53:16 mysql[6264]: * The server quit without updating PID file (/var/run/mysqld/mysqld.pid).
Feb 09 15:53:16 systemd[1]: mysql.service: Control process exited, code=exited status=1
Feb 09 15:53:16 systemd[1]: Failed to start LSB: start and stop MySQL.
Feb 09 15:53:16 systemd[1]: mysql.service: Unit entered failed state.
Feb 09 15:53:16 systemd[1]: mysql.service: Failed with result 'exit-code'.
Hope these information helps.

Percona MySQL Server working but filling the messages log with errors

I have Percona MySQL server 5.7 running under CentOS 7 and although mysql is running without any noticeable errors, it is filling my /var/log/messages with the following every ten seconds:
Nov 15 10:07:27 server systemd: mysqld.service holdoff time over, scheduling restart.
Nov 15 10:07:27 server systemd: Starting MySQL Percona Server...
Nov 15 10:07:27 server mysqld_safe: 171115 10:07:27 mysqld_safe Adding '/usr/lib64/libjemalloc.so.1' to LD_PRELOAD for mysqld
Nov 15 10:07:27 server mysqld_safe: 171115 10:07:27 mysqld_safe Logging to '/var/lib/mysql/server.local.err'.
Nov 15 10:07:27 server mysqld_safe: 171115 10:07:27 mysqld_safe A mysqld process already exists
Nov 15 10:07:27 server systemd: mysqld.service: main process exited, code=exited, status=1/FAILURE
Nov 15 10:07:28 server systemd: Failed to start MySQL Percona Server.
Nov 15 10:07:28 server systemd: Unit mysqld.service entered failed state.
Nov 15 10:07:28 server systemd: Triggering OnFailure= dependencies of mysqld.service.
Nov 15 10:07:28 server systemd: mysqld.service failed.
Nov 15 10:07:28 server systemd: Started Service Status Monitor.
Nov 15 10:07:28 server systemd: Starting Service Status Monitor...
Even though it's stating in there that it failed to start the Percona server, it appears to be working as my website is still doing mysql queries. I know very little about mysql admin and was hoping a mysql guru could shed some light on what is happening.
The clue is here: "A mysqld process already exists". It can't start mysqld because another mysqld process is already running, and using the same port. You need to kill that process before the one you tried to start can start.
Re your comment:
Since this is CentOS 7, I assume mysql.service is being called by systemd.
In my experience, if you start mysqld "ad hoc" without using systemd, then systemd has no idea that it's running, and tries to start mysqld on its own. Systemd also cannot shut down an instance of mysqld unless it started that instance.
the mysqld process is active,ps -ef |grep mysqld ,kill -9 {}

Mariadb doesn't starts. Control process Exited

I am trying to start mariadb with systemctl start mariadb also tried using sudo along.
This is what it says
Job for mariadb.service failed because the control process exited with
error code. See "systemctl status mariadb.service" and "journalctl
-xe" for details.
I saw this at many places but nothing helped.
It fails, I removed, installed again, nothing happened.
This is a part of /var/log/messages
Jun 17 18:03:09 controller systemd: Starting MariaDB 10.1 database
server...
Jun 17 18:03:09 controller mysqld: 2016-06-17 18:03:09 140477178509440
[Note] /usr/libexec/mysqld (mysqld 10.1.12-MariaDB) starting as process 120408 ...
Jun 17 18:03:12 controller systemd: mariadb.service: main process exited, code=exited, status=1/FAILURE
Jun 17 18:04:13 controller systemd: mariadb.service: control process exited, code=exited status=3
Jun 17 18:04:13 controller systemd: Failed to start MariaDB 10.1 database server.
Jun 17 18:04:13 controller systemd: Unit mariadb.service entered failed state.
Jun 17 18:04:13 controller systemd: mariadb.service failed.
I am really new to CentOS and mariadb, so I don't know how to proceed.
EDIT
The mariadb.log says
2016-06-17 19:24:39 140652810123392 [Note] Server socket created on IP: '::'.
2016-06-17 19:24:39 140652810123392 [ERROR] Can't start server: Bind on TCP/IP port. Got error: 98: Address already in use
2016-06-17 19:24:39 140652810123392 [ERROR] Do you already have another mysqld server running on port: 3306 ? 2016-06-17 19:24:39 140652810123392 [ERROR] Aborting
This is my my.cnf
#
# This group is read both both by the client and the server
# use it for options that affect everything
#
[client-server]
#
# This group is read by the server
#
[mysqld]
port=5555
bind-address = 10.23.77.68
default-storage-engine = innodb
innodb_file_per_table
collation-server = utf8_general_ci
init-connect = 'SET NAMES utf8'
character-set-server = utf8
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
#
# include all files from the config directory
#
!includedir /etc/my.cnf.d
I tried adding a line in my.cnf
port=5555
Didn't help, mariadb.log says the same thing as I have mentioned earlier.
I solved as follows:
After installing
Run: > mysql_install_db --user=mysql --basedir=/usr --datadir=/var/lib/
Then: >mysql_secure_installation
And then: systemctl start mariadb
With this this, I can resolved.
Just remove following files at location
/var/lib/mysql
ib_logfile0
ib_logfile1
fuser -k 3306/tcp will solve the probem.
-k is to Kill processes accessing the file.