ERROR [InnoDB] mmap(137428992 bytes) failed; errno 12 - mysql

The web application does not connect to the database on VDS (0.5 RAM, 1 core CPU). I can't understand the reason. I do not know what to do please tell me.
This is mysql configuration my.cnf. I included there innodb_buffer_pool_size = 250M
# For explanations see
# http://dev.mysql.com/doc/mysql/en/server-system-variables.html
# * IMPORTANT: Additional settings that can override those from this file!
# The files must end with '.cnf', otherwise they'll be ignored.
#
!includedir /etc/mysql/conf.d/
!includedir /etc/mysql/mysql.conf.d/
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
symbolic-links=0
max_connections=25
max_user_connections=20
wait_timeout=10
interactive_timeout=50
long_query_time=5
#log-queries-not-using-indexes
#log-slow-queries=/var/log/mysql/log-slow-queries.log
bind-address=185.231.154.57
#key_buffer = 16M
myisam_sort_buffer_size = 32M
join_buffer_size=1M
read_buffer_size=1M
sort_buffer_size=2M
#table_cache=1024
thread_cache_size=286
interactive_timeout=25
connect_timeout=5
max_allowed_packet=1M
max_connect_errors=1000
#query_cache_limit=1M
#query_cache_size=8M
#query_cache_type=1
tmp_table_size=16M
#innodb_use_native_aio = 0
innodb_buffer_pool_size = 250M
innodb_file_per_table
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
logs:
root#v101877:/home/servers/apache-tomee-plume-8.0.0-M1/binsystemctl status mysql
* mysql.service - MySQL Community Server
Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Sun 2019-01-13 10:04:17 MSK; 10min ago
Docs: man:mysqld(8)
http://dev.mysql.com/doc/refman/en/using-systemd.html
Process: 681 ExecStart=/usr/sbin/mysqld (code=exited, status=1/FAILURE)
Process: 642 ExecStartPre=/usr/share/mysql-8.0/mysql-systemd-start pre (code=exited, status=0/SUCCESS)
Main PID: 681 (code=exited, status=1/FAILURE)
Status: "SERVER_BOOTING"
Jan 13 10:04:17 v101877.hosted-by-vdsina.ru mysqld[681]: 2019-01-13T07:04:17.608676Z 0 [ERROR] [MY-012681] [InnoDB] mmap(137428992 bytes) failed; errno 12
Jan 13 10:04:17 v101877.hosted-by-vdsina.ru mysqld[681]: 2019-01-13T07:04:17.610517Z 1 [ERROR] [MY-012956] [InnoDB] Cannot allocate memory for the buffer pool
Jan 13 10:04:17 v101877.hosted-by-vdsina.ru mysqld[681]: 2019-01-13T07:04:17.610553Z 1 [ERROR] [MY-012930] [InnoDB] Plugin initialization aborted with error Generic error.
Jan 13 10:04:17 v101877.hosted-by-vdsina.ru mysqld[681]: 2019-01-13T07:04:17.611113Z 1 [ERROR] [MY-010334] [Server] Failed to initialize DD Storage Engine
Jan 13 10:04:17 v101877.hosted-by-vdsina.ru mysqld[681]: 2019-01-13T07:04:17.612755Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.
Jan 13 10:04:17 v101877.hosted-by-vdsina.ru mysqld[681]: 2019-01-13T07:04:17.613398Z 0 [ERROR] [MY-010119] [Server] Aborting
Jan 13 10:04:17 v101877.hosted-by-vdsina.ru mysqld[681]: 2019-01-13T07:04:17.626404Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.13) MySQL Community Server - GPL.
Jan 13 10:04:17 v101877.hosted-by-vdsina.ru systemd[1]: mysql.service: Main process exited, code=exited, status=1/FAILURE
Jan 13 10:04:17 v101877.hosted-by-vdsina.ru systemd[1]: mysql.service: Failed with result 'exit-code'.
Jan 13 10:04:17 v101877.hosted-by-vdsina.ru systemd[1]: Failed to start MySQL Community Server.

Suggestions to consider for your my.cnf [mysqld] section
# join_buffer_size=1M
# read_buffer_size=1M
# sort_buffer_size=2M
# table_cache=1024
# thread_cache_size=286
innodb_buffer_pool_size=10M
until you can get started. Yes, DISABLE all 5 of these lines so the DEFAULTS will allow you to get running.
If you have .5 G of RAM, expect FRUSTRATION. Very few people use less than 4 G of RAM at this point in time.

I'm using Vagrant with libvirt and I stumbled upon this error.
After hours of struggling, the problem has turned out to be the Vagrant configuration, instead of the mysql/mysqld one.
In the Vagrantfile I uncommented the option to customize the virtual machine's virtual RAM. The default was 512MB, with the 1024 value is working fine now:
config.vm.provider "libvirt" do |vb|
# # Display the VirtualBox GUI when booting the machine
# vb.gui = true
#
# Customize the amount of memory on the VM:
vb.memory = "1024"
end
I hope it helps someone, I never had this error using Virtualbox, maybe it comes with a different default memory value out of the box.

Related

MySQL Restart Failed in MySQL 8.0.30

While Restarting the MySQL server version 8.0.30 it is getting failed and couldn't able to fix the issue till now.
ESCOD
● mysqld.service - MySQL Server
Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Sat 2022-12-31 08:37:07 GMT; 1h 7min ago
Docs: man:mysqld(8)
http://dev.mysql.com/doc/refman/en/using-systemd.html
Process: 631823 ExecStart=/usr/sbin/mysqld $MYSQLD_OPTS (code=exited, status=1/FAILURE)
Process: 631795 ExecStartPre=/usr/bin/mysqld_pre_systemd (code=exited, status=0/SUCCESS)
Main PID: 631823 (code=exited, status=1/FAILURE)
Status: "Server shutdown complete"
Dec 31 08:37:06 fcg-db-mysql-oracle-lx-129.142 systemd[1]: Starting MySQL Server...
Dec 31 08:37:07 fcg-db-mysql-oracle-lx-129.142 mysqld[631823]: 2022-12-31T08:37:07.614931Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.30) starting as process 631823
Dec 31 08:37:07 fcg-db-mysql-oracle-lx-129.142 mysqld[631823]: 2022-12-31T08:37:07.614964Z 0 [ERROR] [MY-010338] [Server] Can't find error-message file '/usr/share/mysql-8.0/errmsg.sys'. Check error-message file location and 'lc-messages-dir' configuration directive.
Dec 31 08:37:07 fcg-db-mysql-oracle-lx-129.142 mysqld[631823]: 2022-12-31T08:37:07.620958Z 1 [ERROR] [MY-011011] [Server] Failed to find valid data directory.
Dec 31 08:37:07 fcg-db-mysql-oracle-lx-129.142 mysqld[631823]: 2022-12-31T08:37:07.621191Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.
Dec 31 08:37:07 fcg-db-mysql-oracle-lx-129.142 mysqld[631823]: 2022-12-31T08:37:07.621215Z 0 [ERROR] [MY-010119] [Server] Aborting
Dec 31 08:37:07 fcg-db-mysql-oracle-lx-129.142 mysqld[631823]: 2022-12-31T08:37:07.621569Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.30) MySQL Community Server - GPL.
Dec 31 08:37:07 fcg-db-mysql-oracle-lx-129.142 systemd[1]: mysqld.service: Main process exited, code=exited, status=1/FAILURE
Dec 31 08:37:07 fcg-db-mysql-oracle-lx-129.142 systemd[1]: mysqld.service: Failed with result 'exit-code'.
Dec 31 08:37:07 fcg-db-mysql-oracle-lx-129.142 systemd[1]: Failed to start MySQL Server.
~
while from suggestions from some blogs i have tried mysqld --initialize --console, but it is also throwing error like below
mysqld: Can't create directory '/var/lib/mysql/' (OS errno 17 - File exists)
2022-12-31T18:12:49.257034Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.30) initializing of server in progress as process 3353
2022-12-31T18:12:49.257062Z 0 [Warning] [MY-010339] [Server] Using pre 5.5 semantics to load error messages from /usr/share/mysql-8.0/. If this is not intended, refer to the documentation for valid usage of --lc-messages-dir and --language parameters.
2022-12-31T18:12:49.257067Z 0 [ERROR] [MY-010342] [Server] Can't read from messagefile '/usr/share/mysql-8.0/errmsg.sys'
2022-12-31T18:12:49.257809Z 0 [ERROR] [MY-013236] [Server] The designated data directory /var/lib/mysql/ is unusable. You can remove all files that the server added to it.
2022-12-31T18:12:49.257841Z 0 [ERROR] [MY-010119] [Server] Aborting
2022-12-31T18:12:49.257943Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.30) MySQL Community Server - GPL.
[fcgredim#fcg-db-mysql-oracle-lx-129 ~]$
Can anyone help on this?
1 . Run the below command to see if there are any errors in the file configuration.
mysqld --validate-config
Can't create directory '/var/lib/mysql/' (OS errno 17 - File exists)
2 . The above error indicates that something wrong with /var/lib/mysql/. Check if this path exists, and it has this permission drwx------ mysql mysql. (Also, if SElinx installed, check if it has the right label)

MySQL service won't start anymore because of OS error 105

Complete MySQL error Log: https://pastebin.com/20QX1ZSc
Overnight my MySQL install suddenly stopped working...
I am using ubuntu 20.04
systemctl:
● mysql.service - MySQL Community Server
Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Thu 2021-05-13 14:50:18 CEST; 17min ago
Process: 4562 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=0/SUCCESS)
Process: 4570 ExecStart=/usr/sbin/mysqld (code=exited, status=3)
Main PID: 4570 (code=exited, status=3)
Status: "Server startup in progress"
May 13 14:50:18 vault systemd[1]: mysql.service: Scheduled restart job, restart counter is at 5.
May 13 14:50:18 vault systemd[1]: Stopped MySQL Community Server.
May 13 14:50:18 vault systemd[1]: mysql.service: Start request repeated too quickly.
May 13 14:50:18 vault systemd[1]: mysql.service: Failed with result 'exit-code'.
May 13 14:50:18 vault systemd[1]: Failed to start MySQL Community Server.
/var/log/mysql:
2021-05-13T12:50:18.017977Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.25-0ubuntu0.20.04.1) starting as process 4570
2021-05-13T12:50:18.032753Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2021-05-13T12:50:18.194467Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2021-05-13T12:50:18.428966Z 1 [Warning] [MY-012637] [InnoDB] 16384 bytes should have been read. Only 12288 bytes read. Retrying for the remaining bytes.
2021-05-13T12:50:18.604489Z 1 [Warning] [MY-012638] [InnoDB] Retry attempts for reading partial data failed.
2021-05-13T12:50:18.604910Z 1 [ERROR] [MY-012642] [InnoDB] Tried to read 16384 bytes at offset 16498688, but was only able to read 12288
2021-05-13T12:50:18.605273Z 1 [ERROR] [MY-012592] [InnoDB] Operating system error number 5 in a file operation.
2021-05-13T12:50:18.605645Z 1 [ERROR] [MY-012596] [InnoDB] Error number 5 means 'Input/output error'
2021-05-13T12:50:18.606374Z 1 [ERROR] [MY-012646] [InnoDB] File mysql.ibd: 'read' returned OS error 105. Cannot continue operation
2021-05-13T12:50:18.606762Z 1 [ERROR] [MY-012981] [InnoDB] Cannot continue operation.
looks like a data/ file corruption.try these steps:
remove the syslog entry under --> /etc/mysql/conf.d/mysqld_safe_syslog.cnf
backup & remove both ib_logfile0 & ib_logfile1 files under --> /var/lib/mysql
remove entry innodb_force_recovery = 1 from --> etc/mysql/my.cnf
run command --> sudo service mysql start
reference Forcing InnoDB Recovery

why Change the status of Mysql to Failed every few hours

Mysql status fails every few hours, I also checked the buffer size, no problem with them and
The following message is displayed with the below command:
systemctl status mysqld
● mysqld.service - MySQL Server
Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
Active: failed (Result: start-limit) since Wed 2020-12-02 17:05:42 EST; 8h ago
Docs: man:mysqld(8)
http://dev.mysql.com/doc/refman/en/using-systemd.html
Process: 25565 ExecStart=/usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid $MYSQLD_OPTS (code=exited, status=1/FAILURE)
Process: 25543 ExecStartPre=/usr/bin/mysqld_pre_systemd (code=exited, status=0/SUCCESS)
Main PID: 17582 (code=killed, signal=KILL)
Dec 02 17:05:42 test.local systemd[1]: Failed to start MySQL Server.
Dec 02 17:05:42 test.local systemd[1]: Unit mysqld.service entered failed state.
Dec 02 17:05:42 test.local systemd[1]: mysqld.service failed.
Dec 02 17:05:42 test.local systemd[1]: mysqld.service holdoff time over, scheduling restart.
Dec 02 17:05:42 test.local systemd[1]: Stopped MySQL Server.
Dec 02 17:05:42 test.local systemd[1]: start request repeated too quickly for mysqld.service
Dec 02 17:05:42 test.local systemd[1]: Failed to start MySQL Server.
Dec 02 17:05:42 test.local systemd[1]: Unit mysqld.service entered failed state.
Dec 02 17:05:42 test.local systemd[1]: mysqld.service failed.
I also checked the log file and the following event occurred:
2020-12-02T22:05:42.179677Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation $2020-12-02T22:05:42.184072Z 0 [Note] /usr/sbin/mysqld (mysqld 5.7.31) starting as process 25570 ...
2020-12-02T22:05:42.204497Z 0 [Note] InnoDB: PUNCH HOLE support available
2020-12-02T22:05:42.204557Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2020-12-02T22:05:42.204565Z 0 [Note] InnoDB: Uses event mutexes
2020-12-02T22:05:42.204574Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2020-12-02T22:05:42.204579Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2020-12-02T22:05:42.204584Z 0 [Note] InnoDB: Using Linux native AIO
2020-12-02T22:05:42.204998Z 0 [Note] InnoDB: Number of pools: 1
2020-12-02T22:05:42.205184Z 0 [Note] InnoDB: Using CPU crc32 instructions
2020-12-02T22:05:42.208098Z 0 [Note] InnoDB: Initializing buffer pool, total size = 6G, instances = 8, chunk size = 128M
2020-12-02T22:05:42.443914Z 0 [ERROR] InnoDB: mmap(137428992 bytes) failed; errno 12
2020-12-02T22:05:42.510301Z 0 [ERROR] InnoDB: Cannot allocate memory for the buffer pool
2020-12-02T22:05:42.510359Z 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
2020-12-02T22:05:42.510376Z 0 [ERROR] Plugin 'InnoDB' init function returned error.
2020-12-02T22:05:42.510384Z 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2020-12-02T22:05:42.510391Z 0 [ERROR] Failed to initialize builtin plugins.
2020-12-02T22:05:42.510399Z 0 [ERROR] Aborting
2020-12-02T22:05:42.510438Z 0 [Note] Binlog end
2020-12-02T22:05:42.510525Z 0 [Note] Shutting down plugin 'CSV'
2020-12-02T22:05:42.511922Z 0 [Note] /usr/sbin/mysqld: Shutdown complete
also My.cf file:
[mysqld]
datadir = /var/lib/mysql
socket = /var/lib/mysql/mysql.sock
symbolic-links=0
innodb_file_per_table = 1
myisam_sort_buffer_size = 64M
read_rnd_buffer_size = 1024K
net_buffer_length = 8K
read_buffer_size = 1024K
sort_buffer_size = 1024K
table_open_cache = 64
max_allowed_packet = 2M
key_buffer_size = 64M
innodb_buffer_pool_size = 6024M
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
I tested this project on a server with 2 GB of RAM, but it failed on a server with 8 GB of RAM.
Consider these changes to your my.cnf [mysqld] section
innodb_buffer_pool_size=4G
and REMOVE to allow defaults to work for improved performance
read_rnd_buffer_size
read_buffer_size
sort_buffer_size
to reduce RAM footprint.
For additional suggestions, view my profile, Network profile for contact info and free downloadable Utility Scripts to assist with performance tuning.

Mysql serivce can't start it, with the control process exited with error code problem

my os is Redhat8.
currtenly I install Mysql, version is "mysql Ver 8.0.13 for Linux on x86_64 (Source distribution)"
I use command "sudo yum install mysql, mysql-server"
But i get error when i want to start mysql services. I keep finding soluction, but still I don't know how to solved it.
Can anyone help me to look at it?
#--------------------------------------------------------------------------------------------------------
[yliu#rhlab1 ~]$ sudo systemctl start mysqld
Job for mysqld.service failed because the control process exited with error code.
See "systemctl status mysqld.service" and "journalctl -xe" for details.
# --------------------------------------------------------------------------------------------
[yliu#rhlab1 ~]$ sudo systemctl status mysqld
● mysqld.service - MySQL 8.0 database server
Loaded: loaded (/usr/lib/systemd/system/mysqld.service; disabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Wed 2020-02-19 12:12:15 AST; 15s ago
Process: 13080 ExecStopPost=/usr/libexec/mysql-wait-stop (code=exited, status=0/SUCCESS)
Process: 13052 ExecStart=/usr/libexec/mysqld --basedir=/usr (code=exited, status=1/FAILURE)
Process: 13015 ExecStartPre=/usr/libexec/mysql-prepare-db-dir mysqld.service (code=exited, status=0/SUCCESS)
Process: 12991 ExecStartPre=/usr/libexec/mysql-check-socket (code=exited, status=0/SUCCESS)
Main PID: 13052 (code=exited, status=1/FAILURE)
Status: "SERVER_BOOTING"
Error: 2 (No such file or directory)
Feb 19 12:12:12 systemd[1]: Starting MySQL 8.0 database server...
Feb 19 12:12:15 systemd[1]: mysqld.service: Main process exited, code=exited, status=1/FAILURE
Feb 19 12:12:15 systemd[1]: mysqld.service: Failed with result 'exit-code'.
Feb 19 12:12:15 systemd[1]: Failed to start MySQL 8.0 database server.
# ----------------------------------------------------------------------------------------
[yliu#rhlab1 ~]$ sudo cat /var/log/mysql/mysqld.log | nl | tail -n 10
38 2020-02-19T15:48:25.492390Z 0 [System] [MY-010910] [Server] /usr/libexec/mysqld: Shutdown complete (mysqld 8.0.13) Source distribution.
39 2020-02-19T15:48:25.492468Z 0 [ERROR] [MY-010065] [Server] Failed to shutdown components infrastructure.
40 2020-02-19T16:12:13.332724Z 0 [System] [MY-010116] [Server] /usr/libexec/mysqld (mysqld 8.0.13) starting as process 13052
41 2020-02-19T16:12:13.585641Z 1 [ERROR] [MY-010781] [Server] Found ./mysql/index_stats.frm file in mysql schema. DD will create .ibd file with same name. Please rename table and start upgrade process again.
42 2020-02-19T16:12:13.585666Z 1 [ERROR] [MY-010336] [Server] Found .frm file with same name as one of the Dictionary Tables.
43 2020-02-19T16:12:13.585833Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.
44 2020-02-19T16:12:13.585871Z 0 [ERROR] [MY-013236] [Server] Newly created data directory /var/lib/mysql/ is unusable. You can safely remove it.
45 2020-02-19T16:12:13.585881Z 0 [ERROR] [MY-010119] [Server] Aborting
46 2020-02-19T16:12:15.000469Z 0 [System] [MY-010910] [Server] /usr/libexec/mysqld: Shutdown complete (mysqld 8.0.13) Source distribution.
47 2020-02-19T16:12:15.000550Z 0 [ERROR] [MY-010065] [Server] Failed to shutdown components infrastructure.
My /etc/my.cnf.d/mysql-server.cnf information is:
[yliu#rhlab1 ~]$ sudo cat /etc/my.cnf.d/mysql-server.cnf
#
# This group are read by MySQL server.
# Use it for options that only the server (but not clients) should see
#
# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/en/server-configuration-defaults.html
# Settings user and group are ignored when systemd is used.
# If you need to run mysqld under a different user or group,
# customize your systemd unit file for mysqld according to the
# instructions in http://fedoraproject.org/wiki/Systemd
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
log-error=/var/log/mysql/mysqld.log
pid-file=/run/mysqld/mysqld.pid
I solved the problem, hope it will helpful for everyone who meet the same problem.
Base on the log:
41 2020-02-19T16:12:13.585641Z 1 [ERROR] [MY-010781] [Server] Found ./mysql/index_stats.frm file in mysql schema. DD will create .ibd file with same name. Please rename table and start upgrade process again.
42 2020-02-19T16:12:13.585666Z 1 [ERROR] [MY-010336] [Server] Found .frm file with same name as one of the Dictionary Tables.
43 2020-02-19T16:12:13.585833Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.
44 2020-02-19T16:12:13.585871Z 0 [ERROR] [MY-013236] [Server] Newly created data directory /var/lib/mysql/ is unusable. You can safely remove it.
so we can remove all the files under /var/lib/mysql/
and then make sure the mysql file owner and group are mysql.
(Django) [yliu#rhlab1 ops]$ sudo ls -al /var/lib/ | grep mysql
drwxr-xr-x. 7 mysql mysql 4096 Feb 24 10:16 mysql
and then you can start mysqld again
sudo systemctl start mysqld

MYSQL 5.7 Percona XtraDBCluster - Cant start MYSQL - Digital Ocean Box

I rebooted a digital ocean box and now I can't start mysql. When I run start command i get :
Redirecting to /bin/systemctl restart mysql.service
Job for mysql.service failed because the control process exited with error code. See "systemctl status mysql.service" and "journalctl -xe" for details.
Result of systemctl status mysql.service
● mysql.service - Percona XtraDB Cluster
Loaded: loaded (/usr/lib/systemd/system/mysql.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Mon 2017-12-11 17:08:18 UTC; 5s ago
Process: 26300 ExecStopPost=/usr/bin/mysql-systemd stop-post (code=exited, status=0/SUCCESS)
Process: 26270 ExecStop=/usr/bin/mysql-systemd stop (code=exited, status=2)
Process: 25674 ExecStartPost=/usr/bin/mysql-systemd start-post $MAINPID (code=exited, status=1/FAILURE)
Process: 25673 ExecStart=/usr/bin/mysqld_safe --basedir=/usr (code=exited, status=0/SUCCESS)
Process: 25632 ExecStartPre=/usr/bin/mysql-systemd start-pre (code=exited, status=0/SUCCESS)
Main PID: 25673 (code=exited, status=0/SUCCESS)
Dec 11 17:08:18 server-name-hidden mysql-systemd[25674]: ERROR! mysqld_safe with PID 25673 has already exited: FAILURE
Dec 11 17:08:18 server-name-hidden systemd[1]: mysql.service: control process exited, code=exited status=1
Dec 11 17:08:18 server-name-hidden mysql-systemd[26270]: WARNING: mysql pid file /var/run/mysqld/mysqld.pid empty or not readable
Dec 11 17:08:18 server-name-hidden mysql-systemd[26270]: ERROR! mysql already dead
Dec 11 17:08:18 server-name-hidden systemd[1]: mysql.service: control process exited, code=exited status=2
Dec 11 17:08:18 server-name-hidden mysql-systemd[26300]: WARNING: mysql pid file /var/run/mysqld/mysqld.pid empty or not readable
Dec 11 17:08:18 server-name-hidden mysql-systemd[26300]: WARNING: mysql may be already dead
Dec 11 17:08:18 server-name-hidden systemd[1]: Failed to start Percona XtraDB Cluster.
Dec 11 17:08:18 server-name-hidden systemd[1]: Unit mysql.service entered failed state.
Dec 11 17:08:18 server-name-hidden systemd[1]: mysql.service failed.
/var/run/mysqld/ is owned by mysql user - but it's empty. If i add a mysqld.pid file it gets removed when i run mysql start.
Does anyone know why a reboot would cause this or give me any next steps. I have reviewed the mysqld.log file and can't see anything of use. here are the last 30 lines
017-12-11T15:14:42.473164Z 0 [Note] WSREP: Received shutdown signal. Will sleep for 10 secs before initiating shutdown. pxc_maint_mode switched to SHUTDOWN
2017-12-11T15:14:52.474584Z 0 [Note] WSREP: Stop replication
2017-12-11T15:14:52.474691Z 0 [Note] WSREP: Closing send monitor...
2017-12-11T15:14:52.475160Z 0 [Note] WSREP: Closed send monitor.
2017-12-11T15:14:52.475268Z 0 [Note] WSREP: gcomm: terminating thread
2017-12-11T15:14:52.475305Z 0 [Note] WSREP: gcomm: joining thread
2017-12-11T15:14:52.475886Z 0 [Note] WSREP: gcomm: closing backend
2017-12-11T15:14:52.476079Z 0 [Note] WSREP: Current view of cluster as seen by this node
2017-12-11T15:14:52.476491Z 0 [Note] WSREP: gcomm: closed
2017-12-11T15:14:52.476532Z 0 [Note] WSREP: Received self-leave message.
2017-12-11T15:14:52.476559Z 0 [Note] WSREP: Flow-control interval: [0, 0]
2017-12-11T15:14:52.476595Z 0 [Note] WSREP: Trying to continue unpaused monitor
2017-12-11T15:14:52.476602Z 0 [Note] WSREP: Received SELF-LEAVE. Closing connection.
2017-12-11T15:14:52.476608Z 0 [Note] WSREP: Shifting SYNCED -> CLOSED (TO: 13842228)
2017-12-11T15:14:52.476632Z 0 [Note] WSREP: RECV thread exiting 0: Success
2017-12-11T15:14:52.477098Z 0 [Note] WSREP: recv_thread() joined.
2017-12-11T15:14:52.477110Z 0 [Note] WSREP: Closing replication queue.
2017-12-11T15:14:52.477116Z 0 [Note] WSREP: Closing slave action queue.
2017-12-11T15:14:52.477123Z 0 [Note] Giving 63 client threads a chance to die gracefully
2017-12-11T15:14:52.478905Z 0 [Warning] /usr/sbin/mysqld: Forcing close of thread 21 user: ‘hidden’
2017-12-11T15:14:52.478957Z 0 [Warning] /usr/sbin/mysqld: Forcing close of thread 22 user: 'hidden'
2017-12-11T15:14:52.478994Z 0 [Warning] /usr/sbin/mysqld: Forcing close of thread 6387 user: 'hidden'
2017-12-11T15:14:52.479034Z 0 [Warning] /usr/sbin/mysqld: Forcing close of thread 6367 user: 'hidden'
2017-12-11T15:14:52.479084Z 0 [Warning] /usr/sbin/mysqld: Forcing close of thread 6373 user: 'hidden'
2017-12-11T15:14:52.479130Z 0 [Warning] /usr/sbin/mysqld: Forcing close of thread 6368 user: 'hidden'
2017-12-11T15:14:54.479289Z 0 [Note] WSREP: Waiting for active wsrep applier to exit
I've also been told to try : systemctl start mysql#bootstrap however this fails with the same error. here is the result from journalctl -xe
Dec 11 17:24:58 dropletname systemd[1]: mysql#bootstrap.service: control process exited, code=exited status=1
Dec 11 17:24:58 dropletname mysql-systemd[28663]: WARNING: mysql pid file /var/run/mysqld/mysqld.pid empty or not readable
Dec 11 17:24:58 dropletname mysql-systemd[28663]: ERROR! mysql already dead
Dec 11 17:24:58 dropletname systemd[1]: mysql#bootstrap.service: control process exited, code=exited status=2
Dec 11 17:24:58 dropletname mysql-systemd[28694]: WARNING: mysql pid file /var/run/mysqld/mysqld.pid empty or not readable
Dec 11 17:24:58 dropletname mysql-systemd[28694]: WARNING: mysql may be already dead
Dec 11 17:24:58 dropletname systemd[1]: Failed to start Percona XtraDB Cluster with config /etc/sysconfig/mysql.bootstrap.
Dec 11 17:24:58 dropletname systemd[1]: Unit mysql#bootstrap.service entered failed state.
Dec 11 17:24:58 dropletname systemd[1]: mysql#bootstrap.service failed.
Dec 11 17:24:58 dropletname polkitd[510]: Unregistered Authentication Agent for unix-process:28009:649814 (system bus name :1.159, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_GB.UTF-8) (disconnected from bus)
Dec 11 17:25:02 dropletname sendmail[27970]: unable to qualify my own domain name (dropletname) -- using short name
Dec 11 17:25:02 dropletname sendmail[27970]: vBBHP2cx027970: from=hidden, size=1655, class=-60, nrcpts=1, msgid=<201712111725.vBBHP2cx027970#dropletname>, relay=hidden#localhost
Dec 11 17:25:02 dropletname sendmail[28728]: vBBHP26v028728: from=<hidden#dropletname>, size=1935, class=-60, nrcpts=1, msgid=<201712111725.vBBHP2cx027970#dropletname>, proto=ESMTP, daemon=MTA, relay=dropletname [127.0.0.1]
Dec 11 17:25:02 dropletname sendmail[27970]: vBBHP2cx027970: to=hidden, ctladdr=hidden (1000/1000), delay=00:00:00, xdelay=00:00:00, mailer=relay, pri=139655, relay=[127.0.0.1] [127.0.0.1], dsn=2.0.0, stat=Sent (vBBHP26v028728 Message accepted for del
Dec 11 17:25:02 dropletname sendmail[28729]: vBBHP26v028728: to=<hidden#dropletname>, ctladdr=<hidden#dropletname> (1000/1000), delay=00:00:00, xdelay=00:00:00, mailer=local, pri=140148, dsn=2.0.0, stat=Sent
Dec 11 17:25:02 dropletname systemd[1]: Removed slice User Slice of hidden
ps aux|grep mysql
root 11590 0.0 0.0 107924 608 pts/0 T 15:43 0:00 cat /var/log/mysqld.log
root 24666 0.0 0.0 107924 612 pts/0 T 16:52 0:00 cat /var/log/mysqld.log
root 32182 0.0 0.0 112664 972 pts/2 S+ 17:59 0:00 grep --color=auto mysq
I managed to resolve this in the end however i'm not sure what 100% did it. I changed /etc/my.conf to look like this :
!includedir /etc/my.cnf.d/
!includedir /etc/percona-xtradb-cluster.conf.d/
[client]
socket=/var/lib/mysql/mysql.sock
[mysqld]
server-id=1
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
log-bin
log_slave_updates
expire_logs_days=7
innodb_strict_mode=OFF
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
I created a mysqld.pid and mysqld.lock file in /var/run/mysqld/ ensuring the owner and group was mysql:mysql
touch /var/run/mysqld/mysqld.pid;
touch /var/run/mysqld/mysqld.lock;
chown -R mysql:mysql /var/run/mysqld/;
I then ran the following two commands which seems to fail but may have helped
service mysql start --wsrep-cluster-address="gcomm://";
systemctl start mysql --wsrep-new-cluster;
Finally I ran
systemctl start mysql#bootstrap.service
service mysql start
This had previously failed every time I tried. Subsequently I then found that the /var/lib/mysql/mysql.sock had been created which previously did not exist.