Issue Details:
I was running MySQL v8 on one of my Raspberry Pi 4 (8 GB) for a few months, however the I am unable to start mysql using sudo service mysql start since morning and receiving below error message:
Job for mysql.service failed because the control process exited with error code.
See "systemctl status mysql.service" and "journalctl -xe" for details.
Troubleshooting & Analysis
Re-installed MySQL 8
Tried to start the MySQL service using the above command, the service started without any issues.
However, when I updated the data directory from /etc/mysql/mysql.conf.d/mysqld.cnf file, the issue crept up again.
Does this tell me that there could be an issue on the the data directory ? Any solution to recover from this scenario ?
Execution Result: systemctl status mysql.service
mysql.service - MySQL Community Server
Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Tue 2021-02-02 18:08:10 IST; 1min 16s ago
Process: 3285 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=0/SUCCESS)
Process: 3294 ExecStart=/usr/sbin/mysqld (code=exited, status=2)
Main PID: 3294 (code=exited, status=2)
Status: "Server startup in progress"
error.log
2021-02-02T12:38:03.012089Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.23-0ubuntu0.20.10.1) starting as process 3185
2021-02-02T12:38:03.059589Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2021-02-02T12:38:03.596555Z 1 [ERROR] [MY-013183] [InnoDB] Assertion failure: fil0fil.cc:10815:initial_fsize == (file->size * phy_page_size) thread 281473363472416
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/8.0/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
12:38:03 UTC - mysqld got signal 6 ;
Most likely, you have hit a bug, but this error can also be caused by malfunctioning hardware.
Thread pointer: 0xaaaadfc38ef0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = ffff9fd7e690 thread_stack 0x46000
/usr/sbin/mysqld(my_print_stacktrace(unsigned char const*, unsigned long)+0x44) [0xaaaaceb08ae4]
/usr/sbin/mysqld(handle_fatal_signal+0x294) [0xaaaacdd1e484]
linux-vdso.so.1(__kernel_rt_sigreturn+0) [0xffffacbaf5d8]
/lib/aarch64-linux-gnu/libc.so.6(gsignal+0xd8) [0xffffac24ef58]
/lib/aarch64-linux-gnu/libc.so.6(abort+0xf4) [0xffffac23b50c]
/usr/sbin/mysqld(ut_dbg_assertion_failed(char const*, char const*, unsigned long)+0x26c) [0xaaaacedb993c]
/usr/sbin/mysqld(fil_tablespace_redo_extend(unsigned char*, unsigned char const*, page_id_t const&, unsigned long, bool)+0x4a8) [0xaaaaceeedd68]
/usr/sbin/mysqld(+0x1e79ba0) [0xaaaacec9aba0]
/usr/sbin/mysqld(+0x1e7aa44) [0xaaaacec9ba44]
/usr/sbin/mysqld(+0x1e7f670) [0xaaaaceca0670]
/usr/sbin/mysqld(recv_recovery_from_checkpoint_start(log_t&, unsigned long)+0x580) [0xaaaaceca1570]
/usr/sbin/mysqld(srv_start(bool)+0x1e68) [0xaaaaced6f7c8]
/usr/sbin/mysqld(+0x1dc8154) [0xaaaacebe9154]
/usr/sbin/mysqld(dd::bootstrap::DDSE_dict_init(THD*, dict_init_mode_t, unsigned int)+0x6c) [0xaaaace9205fc]
/usr/sbin/mysqld(dd::upgrade_57::do_pre_checks_and_initialize_dd(THD*)+0x164) [0xaaaacead8224]
/usr/sbin/mysqld(+0xf9d874) [0xaaaacddbe874]
/usr/sbin/mysqld(+0x21b9d84) [0xaaaacefdad84]
/lib/aarch64-linux-gnu/libpthread.so.0(+0x7f74) [0xffffac64ff74]
/lib/aarch64-linux-gnu/libc.so.6(+0xd73dc) [0xffffac2ee3dc]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0): is an invalid pointer
Connection ID (thread ID): 1
Status: NOT_KILLED
Execution Result: sudo service mysql status
systemd[1]: mysql.service: Scheduled restart job, restart counter is at 5.
systemd[1]: Stopped MySQL Community Server.
systemd[1]: mysql.service: Start request repeated too quickly.
systemd[1]: mysql.service: Failed with result 'exit-code'.
systemd[1]: Failed to start MySQL Community Server.
It's a directory permission issue, make sure your directory is owned by mysql, and permission set to drwxr-x--- , you can do this by
chown mysql:mysql "data-dir"
chmod 760 "data-dir"
Related
I would like to install MySQL5.7 to my amazon linux in some way.
However I do not know that It is possible to install mysql5.7 in Amazon-linux.
Originally in my EC2 , Maria DB was set as default.
I did these steps.
But it did not start.
$ sudo yum install -y
$ yum list installed | grep mysql
$ sudo yum-config-manager --disable mysql80-community
$ sudo yum-config-manager --enable mysql57-community
$ yum repolist all | grep -E "mysql[0-9]+-community/x86"
$ sudo yum install -y mysql-community-server
Loaded :extras_suggestions, langpacks, priorities, update-motd
amzn2-core | 3.7 kB 00:00:00
385 packages excluded due to repository priority protections
Package mysql-community-server-5.7.36-1.el7.x86_64 is the latest version.
$ sudo systemctl start mysqld
Job for mysqld.service failed because the control process exited with error code. See "systemctl status mysqld.service" and "journalctl -xe" for details.
$ systemctl status mysqld.service
● mysqld.service - MySQL Server
Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
Active: failed (Result: start-limit) since Fri 2021-12-10 20:52:34 JST; 26s ago
Docs: man:mysqld(8)
http://dev.mysql.com/doc/refman/en/using-systemd.html
Process: 32505 ExecStart=/usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid $MYSQLD_OPTS (code=exited, status=1/FAILURE)
Process: 32484 ExecStartPre=/usr/bin/mysqld_pre_systemd (code=exited, status=0/SUCCESS)
12- 10 20:52:34 ip-172-91-96-191.ap-northeast-1.compute.internal systemd[1]: mysqld.service: control process exited, code=exited status=1
12- 10 20:52:34 ip-172-91-96-191.ap-northeast-1.compute.internal systemd[1]: Failed to start MySQL Server.
12- 10 20:52:34 ip-172-91-96-191.ap-northeast-1.compute.internal systemd[1]: Unit mysqld.service entered failed state.
12- 10 20:52:34 ip-172-91-96-191.ap-northeast-1.compute.internal systemd[1]: mysqld.service failed.
12- 10 20:52:34 ip-172-91-96-191.ap-northeast-1.compute.internal systemd[1]: mysqld.service holdoff time over, scheduling restart.
12- 10 20:52:34 ip-172-91-96-191.ap-northeast-1.compute.internal systemd[1]: start request repeated too quickly for mysqld.service
12- 10 20:52:34 ip-172-91-96-191.ap-northeast-1.compute.internal systemd[1]: Failed to start MySQL Server.
12- 10 20:52:34 ip-172-91-96-191.ap-northeast-1.compute.internal systemd[1]: Unit mysqld.service entered failed state.
12- 10 20:52:34 ip-172-91-96-191.ap-northeast-1.compute.internal systemd[1]: mysqld.service failed.
I also did these steps.
I tried look up the way as far I can.
https://aws.amazon.com/amazon-linux-2/
$ /etc/init.d/mysql stop
-bash: /etc/init.d/mysql: No such file or directory
$ service mysql stop
Redirecting to /bin/systemctl stop mysql.service
Failed to stop mysql.service: The name org.freedesktop.PolicyKit1 was not provided by any .service files
See system logs and 'systemctl status mysql.service' for details.
$ killall -KILL mysql mysqld_safe mysqld
mysql: no process found
mysqld_safe: no process found
mysqld: no process found
$ /etc/init.d/mysql start
-bash: /etc/init.d/mysql: No such file or directory
$ sudo systemctl stop mysqld
$ sudo systemctl start mysqld
Job for mysqld.service failed because the control process exited with error code. See "systemctl status mysqld.service" and "journalctl -xe" for details.
$ sudo cat /var/log/mysqld.log
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x40000
/usr/sbin/mysqld(my_print_stacktrace+0x3b)[0xf525bb]
/usr/sbin/mysqld(handle_fatal_signal+0x486)[0x7e3f06]
/lib64/libpthread.so.0(+0x117e0)[0x7f0dc34517e0]
/lib64/libc.so.6(gsignal+0x110)[0x7f0dc1da0c20]
/lib64/libc.so.6(abort+0x148)[0x7f0dc1da20c8]
/usr/sbin/mysqld[0x7b3eb0]
/usr/sbin/mysqld(_ZN2ib5fatalD1Ev+0xfd)[0x12882cd]
/usr/sbin/mysqld[0x133fce2]
/usr/sbin/mysqld[0x134022b]
/usr/sbin/mysqld(_Z6fil_ioRK9IORequestbRK9page_id_tRK11page_size_tmmPvS8_+0x208)[0x134baf8]
/usr/sbin/mysqld(_Z13buf_read_pageRK9page_id_tRK11page_size_t+0xe4)[0x12f9d74]
/usr/sbin/mysqld(_Z16buf_page_get_genRK9page_id_tRK11page_size_tmP11buf_block_tmPKcmP5mtr_tb+0x42d)[0x12c916d]
/usr/sbin/mysqld(_Z31trx_rseg_get_n_undo_tablespacesPm+0x15d)[0x126742d]
/usr/sbin/mysqld[0x122f258]
/usr/sbin/mysqld(_Z34innobase_start_or_create_for_mysqlv+0x3996)[0x12339a6]
/usr/sbin/mysqld[0x10f8a2f]
/usr/sbin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x51)[0x836df1]
/usr/sbin/mysqld[0xd35d35]
/usr/sbin/mysqld(_Z40plugin_register_builtin_and_init_core_sePiPPc+0x2f0)[0xd3c060]
/usr/sbin/mysqld[0x7dc3e4]
/usr/sbin/mysqld(_Z11mysqld_mainiPPc+0xa85)[0x7dfc05]
/lib64/libc.so.6(__libc_start_main+0xea)[0x7f0dc1d8e0ba]
/usr/sbin/mysqld[0x7d3864]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
2021-12-10T12:20:30.164676Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2021-12-10T12:20:30.171606Z 0 [Note] /usr/sbin/mysqld (mysqld 5.7.36) starting as process 1117 ...
2021-12-10T12:20:30.192632Z 0 [Note] InnoDB: PUNCH HOLE support available
2021-12-10T12:20:30.192667Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2021-12-10T12:20:30.192671Z 0 [Note] InnoDB: Uses event mutexes
2021-12-10T12:20:30.192675Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2021-12-10T12:20:30.192678Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2021-12-10T12:20:30.192681Z 0 [Note] InnoDB: Using Linux native AIO
2021-12-10T12:20:30.192972Z 0 [Note] InnoDB: Number of pools: 1
2021-12-10T12:20:30.193097Z 0 [Note] InnoDB: Using CPU crc32 instructions
2021-12-10T12:20:30.203486Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2021-12-10T12:20:30.218799Z 0 [Note] InnoDB: Completed initialization of buffer pool
2021-12-10T12:20:30.224633Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2021-12-10T12:20:30.252634Z 0 [ERROR] [FATAL] InnoDB: Table flags are 0 in the data dictionary but the flags in file ./ibdata1 are 0x4000!
2021-12-10 21:20:30 0x7f9a7c57d740 InnoDB: Assertion failure in thread 140301487822656 in file ut0ut.cc line 921
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.7/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
12:20:30 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
Attempting to collect some information that could help diagnose the problem.
As this is a crash and something is definitely wrong, the information
collection process might fail.
key_buffer_size=8388608
read_buffer_size=131072
max_used_connections=0
max_threads=151
thread_count=0
connection_count=0
It is possible that mysqld could use up to
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
Complete MySQL error Log: https://pastebin.com/20QX1ZSc
Overnight my MySQL install suddenly stopped working...
I am using ubuntu 20.04
systemctl:
● mysql.service - MySQL Community Server
Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Thu 2021-05-13 14:50:18 CEST; 17min ago
Process: 4562 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=0/SUCCESS)
Process: 4570 ExecStart=/usr/sbin/mysqld (code=exited, status=3)
Main PID: 4570 (code=exited, status=3)
Status: "Server startup in progress"
May 13 14:50:18 vault systemd[1]: mysql.service: Scheduled restart job, restart counter is at 5.
May 13 14:50:18 vault systemd[1]: Stopped MySQL Community Server.
May 13 14:50:18 vault systemd[1]: mysql.service: Start request repeated too quickly.
May 13 14:50:18 vault systemd[1]: mysql.service: Failed with result 'exit-code'.
May 13 14:50:18 vault systemd[1]: Failed to start MySQL Community Server.
/var/log/mysql:
2021-05-13T12:50:18.017977Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.25-0ubuntu0.20.04.1) starting as process 4570
2021-05-13T12:50:18.032753Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2021-05-13T12:50:18.194467Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2021-05-13T12:50:18.428966Z 1 [Warning] [MY-012637] [InnoDB] 16384 bytes should have been read. Only 12288 bytes read. Retrying for the remaining bytes.
2021-05-13T12:50:18.604489Z 1 [Warning] [MY-012638] [InnoDB] Retry attempts for reading partial data failed.
2021-05-13T12:50:18.604910Z 1 [ERROR] [MY-012642] [InnoDB] Tried to read 16384 bytes at offset 16498688, but was only able to read 12288
2021-05-13T12:50:18.605273Z 1 [ERROR] [MY-012592] [InnoDB] Operating system error number 5 in a file operation.
2021-05-13T12:50:18.605645Z 1 [ERROR] [MY-012596] [InnoDB] Error number 5 means 'Input/output error'
2021-05-13T12:50:18.606374Z 1 [ERROR] [MY-012646] [InnoDB] File mysql.ibd: 'read' returned OS error 105. Cannot continue operation
2021-05-13T12:50:18.606762Z 1 [ERROR] [MY-012981] [InnoDB] Cannot continue operation.
looks like a data/ file corruption.try these steps:
remove the syslog entry under --> /etc/mysql/conf.d/mysqld_safe_syslog.cnf
backup & remove both ib_logfile0 & ib_logfile1 files under --> /var/lib/mysql
remove entry innodb_force_recovery = 1 from --> etc/mysql/my.cnf
run command --> sudo service mysql start
reference Forcing InnoDB Recovery
I am running a AWS Lightsail instance with Ubuntu 18.04 and latest MySQL. Since a few days, the MySQL service stops at random times and then forces me to perform a manual restart after which everything goes back to normal for 24-48 hours until I face another stop.
Terminal echoes this error when running sudo service mysql status:
● mysql.service - MySQL Community Server
Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Sun 2020-04-26 17:54:44 UTC; 10h ago
Process: 30117 ExecStart=/usr/sbin/mysqld --daemonize --pid-file=/run/mysqld/mysqld.pid (code=exited, status=1/FAILURE)
Process: 30095 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=0/SUCCESS)
Main PID: 14182 (code=killed, signal=KILL)
Apr 26 17:54:44 ip-172-26-4-241 systemd[1]: mysql.service: Failed with result 'exit-code'.
Apr 26 17:54:44 ip-172-26-4-241 systemd[1]: Failed to start MySQL Community Server.
Apr 26 17:54:44 ip-172-26-4-241 systemd[1]: mysql.service: Service hold-off time over, scheduling restart.
Apr 26 17:54:44 ip-172-26-4-241 systemd[1]: mysql.service: Scheduled restart job, restart counter is at 8.
Apr 26 17:54:44 ip-172-26-4-241 systemd[1]: Stopped MySQL Community Server.
Apr 26 17:54:44 ip-172-26-4-241 systemd[1]: mysql.service: Start request repeated too quickly.
Apr 26 17:54:44 ip-172-26-4-241 systemd[1]: mysql.service: Failed with result 'exit-code'.
Apr 26 17:54:44 ip-172-26-4-241 systemd[1]: Failed to start MySQL Community Server.
After digging through the /var/log/mysql/error.log file, I found dozens of lines like this:
2020-04-26T10:10:46.710146Z 2 [ERROR] InnoDB: innodb_force_recovery is on. We do not allow database modifications by the user. Shut down mysqld and edit my.cnf to$
Since the error message is truncated at exactly the wrong spot, I did some more digging through other Warning messages and found this one:
2020-04-26T17:54:44.312663Z 0 [ERROR] InnoDB: mmap(137428992 bytes) failed; errno 12
2020-04-26T17:54:44.312671Z 0 [ERROR] InnoDB: Cannot allocate memory for the buffer pool
2020-04-26T17:54:44.312675Z 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
2020-04-26T17:54:44.312680Z 0 [ERROR] Plugin 'InnoDB' init function returned error.
2020-04-26T17:54:44.312684Z 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2020-04-26T17:54:44.312688Z 0 [ERROR] Failed to initialize builtin plugins.
2020-04-26T17:54:44.312691Z 0 [ERROR] Aborting
I am by no means a MySQL expert, but I interpret that mmamp error as the system running out of space, which is a little weird since the instance uses around 10% space and has at most 2 users at the same time (hence little to no CPU usage). I run a single site Wordpress installation (just to divert users) and phpmyadmin plus some minor testing scripts.
Appreciate any help with this, thank you.
The most likely cause is that your configuration is broken in a way that makes mysqld try to allocate more memory that your server has.
my os is Redhat8.
currtenly I install Mysql, version is "mysql Ver 8.0.13 for Linux on x86_64 (Source distribution)"
I use command "sudo yum install mysql, mysql-server"
But i get error when i want to start mysql services. I keep finding soluction, but still I don't know how to solved it.
Can anyone help me to look at it?
#--------------------------------------------------------------------------------------------------------
[yliu#rhlab1 ~]$ sudo systemctl start mysqld
Job for mysqld.service failed because the control process exited with error code.
See "systemctl status mysqld.service" and "journalctl -xe" for details.
# --------------------------------------------------------------------------------------------
[yliu#rhlab1 ~]$ sudo systemctl status mysqld
● mysqld.service - MySQL 8.0 database server
Loaded: loaded (/usr/lib/systemd/system/mysqld.service; disabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Wed 2020-02-19 12:12:15 AST; 15s ago
Process: 13080 ExecStopPost=/usr/libexec/mysql-wait-stop (code=exited, status=0/SUCCESS)
Process: 13052 ExecStart=/usr/libexec/mysqld --basedir=/usr (code=exited, status=1/FAILURE)
Process: 13015 ExecStartPre=/usr/libexec/mysql-prepare-db-dir mysqld.service (code=exited, status=0/SUCCESS)
Process: 12991 ExecStartPre=/usr/libexec/mysql-check-socket (code=exited, status=0/SUCCESS)
Main PID: 13052 (code=exited, status=1/FAILURE)
Status: "SERVER_BOOTING"
Error: 2 (No such file or directory)
Feb 19 12:12:12 systemd[1]: Starting MySQL 8.0 database server...
Feb 19 12:12:15 systemd[1]: mysqld.service: Main process exited, code=exited, status=1/FAILURE
Feb 19 12:12:15 systemd[1]: mysqld.service: Failed with result 'exit-code'.
Feb 19 12:12:15 systemd[1]: Failed to start MySQL 8.0 database server.
# ----------------------------------------------------------------------------------------
[yliu#rhlab1 ~]$ sudo cat /var/log/mysql/mysqld.log | nl | tail -n 10
38 2020-02-19T15:48:25.492390Z 0 [System] [MY-010910] [Server] /usr/libexec/mysqld: Shutdown complete (mysqld 8.0.13) Source distribution.
39 2020-02-19T15:48:25.492468Z 0 [ERROR] [MY-010065] [Server] Failed to shutdown components infrastructure.
40 2020-02-19T16:12:13.332724Z 0 [System] [MY-010116] [Server] /usr/libexec/mysqld (mysqld 8.0.13) starting as process 13052
41 2020-02-19T16:12:13.585641Z 1 [ERROR] [MY-010781] [Server] Found ./mysql/index_stats.frm file in mysql schema. DD will create .ibd file with same name. Please rename table and start upgrade process again.
42 2020-02-19T16:12:13.585666Z 1 [ERROR] [MY-010336] [Server] Found .frm file with same name as one of the Dictionary Tables.
43 2020-02-19T16:12:13.585833Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.
44 2020-02-19T16:12:13.585871Z 0 [ERROR] [MY-013236] [Server] Newly created data directory /var/lib/mysql/ is unusable. You can safely remove it.
45 2020-02-19T16:12:13.585881Z 0 [ERROR] [MY-010119] [Server] Aborting
46 2020-02-19T16:12:15.000469Z 0 [System] [MY-010910] [Server] /usr/libexec/mysqld: Shutdown complete (mysqld 8.0.13) Source distribution.
47 2020-02-19T16:12:15.000550Z 0 [ERROR] [MY-010065] [Server] Failed to shutdown components infrastructure.
My /etc/my.cnf.d/mysql-server.cnf information is:
[yliu#rhlab1 ~]$ sudo cat /etc/my.cnf.d/mysql-server.cnf
#
# This group are read by MySQL server.
# Use it for options that only the server (but not clients) should see
#
# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/en/server-configuration-defaults.html
# Settings user and group are ignored when systemd is used.
# If you need to run mysqld under a different user or group,
# customize your systemd unit file for mysqld according to the
# instructions in http://fedoraproject.org/wiki/Systemd
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
log-error=/var/log/mysql/mysqld.log
pid-file=/run/mysqld/mysqld.pid
I solved the problem, hope it will helpful for everyone who meet the same problem.
Base on the log:
41 2020-02-19T16:12:13.585641Z 1 [ERROR] [MY-010781] [Server] Found ./mysql/index_stats.frm file in mysql schema. DD will create .ibd file with same name. Please rename table and start upgrade process again.
42 2020-02-19T16:12:13.585666Z 1 [ERROR] [MY-010336] [Server] Found .frm file with same name as one of the Dictionary Tables.
43 2020-02-19T16:12:13.585833Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.
44 2020-02-19T16:12:13.585871Z 0 [ERROR] [MY-013236] [Server] Newly created data directory /var/lib/mysql/ is unusable. You can safely remove it.
so we can remove all the files under /var/lib/mysql/
and then make sure the mysql file owner and group are mysql.
(Django) [yliu#rhlab1 ops]$ sudo ls -al /var/lib/ | grep mysql
drwxr-xr-x. 7 mysql mysql 4096 Feb 24 10:16 mysql
and then you can start mysqld again
sudo systemctl start mysqld
I have an EC2 instance in which Mysql server is working fine. I created an AMI from this and launched a new instance from this AMI. Now Mysql server is not able to start on this new AMI at all.
Below is the o/p:
ubuntu#ip-172-31-66-160:~$ sudo service mysql status
● mysql.service - MySQL Community Server
Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
Active: activating (start-post) (Result: exit-code) since Sun 2019-07-21 20:59:25 IST; 28s ago
Process: 1870 ExecStart=/usr/sbin/mysqld (code=exited, status=1/FAILURE)
Process: 1862 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=0/SUCCESS)
Main PID: 1870 (code=exited, status=1/FAILURE); : 1871 (mysql-systemd-s)
Tasks: 2
Memory: 3.2M
CPU: 247ms
CGroup: /system.slice/mysql.service
└─control
├─1871 /bin/bash /usr/share/mysql/mysql-systemd-start post
└─2529 sleep 1
Jul 21 20:59:25 ip-172-31-66-160 systemd[1]: mysql.service: Service hold-off time over, scheduling restart.
Jul 21 20:59:25 ip-172-31-66-160 systemd[1]: Stopped MySQL Community Server.
Jul 21 20:59:25 ip-172-31-66-160 systemd[1]: Starting MySQL Community Server...
Jul 21 20:59:26 ip-172-31-66-160 systemd[1]: mysql.service: Main process exited, code=exited, status=1/FAILURE
ubuntu#ip-172-31-66-160:~$ sudo service mysql restart
Job for mysql.service failed because the control process exited with error code. See "systemctl status mysql.service" and "journalctl -xe" for details.
I tried everything but not able to get this started.
Closest threads which I came across and followed without any luck:
https://askubuntu.com/questions/916009/mysql-wont-start-because-of-apparmor
https://support.plesk.com/hc/en-us/articles/360004185293-Unable-to-start-MySQL-on-Ubuntu-AVC-apparmor-DENIED-operation-open-
Kindly help, I am just clueless about this.
Finally, I was able to make the server run on my new instance. What I was missing is that mysql also has many config options which gets changed over due course of time and they also depend on the EC2 instance config.
In my case, my new instance has very limited RAM and the mysql.cnf file was trying to allocate more RAM than the machine had hence it was failing. Somehow,
journalctl -xe
is giving a different error which is forcing me to think in wrong direction. I should have looked in to the mysql log file which quickly pointed me to the right issue.
I found below in the error log file which helped me to isolate / fix the issue:
2019-07-21T16:00:55.658985Z 0 [ERROR] InnoDB: mmap(137428992 bytes) failed; errno 12 2019-07-21T16:00:55.658993Z 0 [ERROR] InnoDB: Cannot allocate memory for the buffer pool
2019-07-21T16:00:55.658998Z 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
2019-07-21T16:00:55.659003Z 0 [ERROR] Plugin 'InnoDB' init function returned error.
2019-07-21T16:00:55.659620Z 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2019-07-21T16:00:55.659628Z 0 [ERROR] Failed to initialize builtin plugins.
2019-07-21T16:00:55.659632Z 0 [ERROR] Aborting
So mysql wasn't able to start since requested amount of memory isn't present in the system. I commented all the memory allocation lines in my.cnf file (relying on the default settings) and then it worked.
innodb_buffer_pool_size = 16G
key_buffer_size = 2G
max_allowed_packet = 128M
group_concat_max_len = 50000
query_cache_size = 2147483648
query_cache_limit =67108864
Hope someone would find this information useful.