MySQL docker image server stopped working after I added new table to the db. It was quite major change, because new table is from now on major one.
Long story short, before I had some entities related to user (many to one), now I added another relation to the entities which basically kind of groups them.
Before it was like this:
User -OneToMany- Processes
Process -OneToOne- CompanySearches
Main view was the User panel with list of Processes
and now it is like:
User -OneToMany- Processes
Process -OneToOne- CompanySearch
Company(new) -OneToMany- Processes (which includes CompanySearch)
Main view is User panel with list of Companies
Company view has list of processes related to this company
App is built with NestJS + Sequelize + MySQL + React
If I delete database and build everything from scratch everything works like a charm, but if I create some records on the previous version and try to run new version of app this error is displayed in the console
2022-07-27T13:26:05.182073Z 0 [System] [MY-010229] [Server] Starting XA crash recovery...
2022-07-27T13:26:05.193465Z 0 [System] [MY-010232] [Server] XA crash recovery finished.
2022-07-27T13:26:05.455066Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
2022-07-27T13:26:05.455892Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
2022-07-27T13:26:05.471899Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
2022-07-27T13:26:05.697069Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '::' port: 33060, socket: /var/run/mysqld/mysqlx.sock
2022-07-27T13:26:05.697720Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.28' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server - GPL.
2022-07-27T13:26:06.380296Z 0 [ERROR] [MY-013183] [InnoDB] Assertion failure: dict0dict.cc:3378:for_table || ref_table thread 278221408000
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/8.0/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
13:26:06 UTC - mysqld got signal 6 ;
Most likely, you have hit a bug, but this error can also be caused by malfunctioning hardware.
Thread pointer: 0x40cc000b20
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 40c749bc20 thread_stack 0x100000
/usr/sbin/mysqld(my_print_stacktrace(unsigned char const*, unsigned long)+0x3d) [0x400212557d]
/usr/sbin/mysqld(print_fatal_signal(int)+0x303) [0x40011a2453]
/usr/sbin/mysqld(my_server_abort()+0x6e) [0x40011a256e]
/usr/sbin/mysqld(my_abort()+0xa) [0x400211f70a]
/usr/sbin/mysqld(ut_dbg_assertion_failed(char const*, char const*, unsigned long)+0x326) [0x40023f6ce6]
/usr/sbin/mysqld(dict_foreign_add_to_cache(dict_foreign_t*, char const**, bool, bool, dict_err_ignore_t)+0x3aa) [0x40024dcb1a]
/usr/sbin/mysqld(dd_table_load_fk_from_dd(dict_table_t*, dd::Table const*, char const**, dict_err_ignore_t, bool)+0x4fc) [0x40024f52bc]
/usr/sbin/mysqld(dd_table_load_fk(dd::cache::Dictionary_client*, char const*, char const**, dict_table_t*, dd::Table const*, THD*, bool, bool, std::deque<char const*, ut::allocator<char const*, ut::detail::allocator_base_pfs<char const*> > >*)+0x68) [0x40024f7698]
/usr/sbin/mysqld(dict_table_t* dd_open_table_one<dd::Table>(dd::cache::Dictionary_client*, TABLE const*, char const*, dd::Table const*, THD*, std::deque<char const*, ut::allocator<char const*, ut::detail::allocator_base_pfs<char const*> > >&)+0x1bd0) [0x40024f9340]
/usr/sbin/mysqld(dict_table_t* dd_open_table<dd::Table>(dd::cache::Dictionary_client*, TABLE const*, char const*, dd::Table const*, THD*)+0xc1) [0x40024fcdd1]
/usr/sbin/mysqld(dd_table_open_on_dd_obj(THD*, dd::cache::Dictionary_client*, dd::Table const&, dd::Partition const*, char const*, dict_table_t*&, TABLE const*)+0xd98) [0x40024fdbd8]
/usr/sbin/mysqld(+0x24fe019) [0x40024fe019]
/usr/sbin/mysqld(dd_table_open_on_id(unsigned long, THD*, MDL_ticket**, bool, bool)+0x89d) [0x40024fed3d]
/usr/sbin/mysqld(row_purge_step(que_thr_t*)+0x5ba) [0x40023562da]
/usr/sbin/mysqld(que_run_threads(que_thr_t*)+0xd38) [0x400230d718]
/usr/sbin/mysqld(trx_purge(unsigned long, unsigned long, bool)+0x1df) [0x40023bb50f]
/usr/sbin/mysqld(srv_purge_coordinator_thread()+0xbca) [0x400238fefa] /usr/sbin/mysqld(std::thread::_State_impl<std::thread::_Invoker<std::tuple<Detached_thread, void (*)()> > >::_M_run()+0xbc) [0x40022ae56c]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xbbb2f) [0x4006115b2f]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7fa3) [0x4005c0dfa3]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x4006473eff]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0): Connection ID (thread ID): 0
Status: NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
exited with code 2
2022-07-27 13:26:10+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2022-07-27 13:26:11+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.28-1debian10 started.
2022-07-27T13:26:14.491966Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.28) starting as process 1
2022-07-27T13:26:14.544524Z 0 [Warning] [MY-010159] [Server] Setting lower_case_table_names=2 because file system for /var/lib/mysql/ is case insensitive
2022-07-27T13:26:14.758211Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2022-07-27T13:26:15.185006Z 1 [ERROR] [MY-012585] [InnoDB] Linux Native AIO interface is not supported on this platform. Please check your OS documentation and install appropriate binary of InnoDB.
2022-07-27T13:26:15.186329Z 1 [Warning] [MY-012654] [InnoDB] Linux Native AIO disabled.
2022-07-27T13:26:17.842927Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
I can't even run migration when db server crashes
Related
I restarted a mysql database instance that had crashed through the service manager on a windows 10 2016 server but it produced an error regarding windows aio. At that point I tried to stop the service but it appears to be hung with a stopPending status and is no longer listening for incoming connections. Below is the error log:
InnoDB: ###### Diagnostic info printed to the standard error stream
2023-02-02T14:48:41.755096Z 0 [ERROR] [MY-012872] [InnoDB] [FATAL] Semaphore wait has lasted > 600 seconds. We intentionally crash the server because it appears to be hung.
2023-02-02T14:48:41.775212Z 0 [ERROR] [MY-013183] [InnoDB] Assertion failure: ut0ut.cc:536 thread 4360
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/8.0/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
14:48:41 UTC - mysqld got exception 0x80000003 ;
Most likely, you have hit a bug, but this error can also be caused by malfunctioning hardware.
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
7ff7baad60d2 mysqld.exe!my_sigabrt_handler()[my_thr_init.cc:372]
7ffb145fcfa7 ucrtbase.dll!raise()
7ffb145fde31 ucrtbase.dll!abort()
7ff7bacba8c0 mysqld.exe!ut_dbg_assertion_failed()[ut0dbg.cc:98]
7ff7bac93391 mysqld.exe!ib::fatal::~fatal()[ut0ut.cc:536]
7ff7baba61b3 mysqld.exe!srv_error_monitor_thread()[srv0srv.cc:1822]
7ff7babf0ea5 mysqld.exe!Runnable::operator()<void (__cdecl*)(void)>()[os0thread-create.h:103]
7ff7babf1225 mysqld.exe!std::thread::_Invoke<std::tuple<Runnable,void (__cdecl*)(void)>,0,1>()[thread:40]
7ffb145afb80 ucrtbase.dll!_o__realloc_base()
7ffb16c184d4 KERNEL32.DLL!BaseThreadInitThunk()
7ffb17aee871 ntdll.dll!RtlUserThreadStart()
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
2023-02-02T16:36:06.118930Z 0 [System] [MY-010116] [Server] C:\Program Files\MySQL\MySQL Server 8.0\bin\mysqld (mysqld 8.0.19) starting as process 9616
2023-02-02T16:36:06.529268Z 0 [Warning] [MY-012363] [InnoDB] innodb-page-size has been changed from the default value 16384 to 65536.
2023-02-02T16:36:37.661274Z 0 [System] [MY-010931] [Server] C:\Program Files\MySQL\MySQL Server 8.0\bin\mysqld: ready for connections. Version: '8.0.19' socket: '' port: 3306 MySQL Community Server - GPL.
2023-02-02T16:36:37.713658Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '::' port: 33060
2023-02-02T16:37:09.441481Z 0 [ERROR] [MY-012611] [InnoDB] Operating system error number 1 in a file operation.
2023-02-02T16:37:09.442489Z 0 [ERROR] [MY-012646] [InnoDB] File .\local\data.ibd: 'Windows aio' returned OS error 101. Cannot continue operation
2023-02-02T16:37:09.443660Z 0 [ERROR] [MY-012981] [InnoDB] Cannot continue operation.
2023-02-02T17:11:43.686693Z 0 [System] [MY-013105] [Server] C:\Program Files\MySQL\MySQL Server 8.0\bin\mysqld: Normal shutdown.
2023-02-02T17:11:45.698697Z 0 [Warning] [MY-010909] [Server] C:\Program Files\MySQL\MySQL Server 8.0\bin\mysqld: Forcing close of thread 8 user: 'service_account'.
2023-02-02T17:11:45.700138Z 0 [Warning] [MY-010909] [Server] C:\Program Files\MySQL\MySQL Server 8.0\bin\mysqld: Forcing close of thread 9 user: 'service_account'.
2023-02-02T17:11:45.701433Z 0 [Warning] [MY-010909] [Server] C:\Program Files\MySQL\MySQL Server 8.0\bin\mysqld: Forcing close of thread 10 user: 'service_account'.
I tried using mysqladmin.exe to force a shutdown but the service is no longer listening for connections. I tried restarting the service but I get a timeout error. I am reluctant to kill the process for obvious reasons.
everyone. I have noticed strange behavior during MySQL docker container startup. Basically I'm trying to run the following command
docker run --name mysql -e MYSQL_ROOT_PASSWORD=root -d mysql:8.0.29
but after that I see from the logs that container after successful startup suddenly receives SHUTDOWN command, and reinitialize database again
2022-07-25T19:33:26.423537Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.29) MySQL Community Server - GPL.
Here is all the logs:
2022-07-25 19:33:19+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.29-1.el8 started.
2022-07-25 19:33:19+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2022-07-25 19:33:19+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.29-1.el8 started.
2022-07-25 19:33:19+00:00 [Note] [Entrypoint]: Initializing database files
2022-07-25T19:33:19.770157Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.29) initializing of server in progress as process 42
2022-07-25T19:33:19.775554Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2022-07-25T19:33:20.037982Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2022-07-25T19:33:21.277581Z 6 [Warning] [MY-010453] [Server] root#localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.
2022-07-25 19:33:23+00:00 [Note] [Entrypoint]: Database files initialized
2022-07-25 19:33:23+00:00 [Note] [Entrypoint]: Starting temporary server
2022-07-25T19:33:23.494658Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.29) starting as process 91
2022-07-25T19:33:23.505604Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2022-07-25T19:33:23.590481Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2022-07-25T19:33:23.778598Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
2022-07-25T19:33:23.778621Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
2022-07-25T19:33:23.779801Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
2022-07-25T19:33:23.790924Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Socket: /var/run/mysqld/mysqlx.sock
2022-07-25T19:33:23.790961Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.29' socket: '/var/run/mysqld/mysqld.sock' port: 0 MySQL Community Server - GPL.
2022-07-25 19:33:23+00:00 [Note] [Entrypoint]: Temporary server started.
'/var/lib/mysql/mysql.sock' -> '/var/run/mysqld/mysqld.sock'
Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/leapseconds' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/tzdata.zi' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.
2022-07-25 19:33:24+00:00 [Note] [Entrypoint]: Stopping temporary server
2022-07-25T19:33:24.911894Z 10 [System] [MY-013172] [Server] Received SHUTDOWN from user root. Shutting down mysqld (Version: 8.0.29).
2022-07-25T19:33:26.423537Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.29) MySQL Community Server - GPL.
2022-07-25 19:33:26+00:00 [Note] [Entrypoint]: Temporary server stopped
2022-07-25 19:33:26+00:00 [Note] [Entrypoint]: MySQL init process done. Ready for start up.
2022-07-25T19:33:27.122487Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.29) starting as process 1
2022-07-25T19:33:27.128121Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2022-07-25T19:33:27.217345Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2022-07-25T19:33:27.410605Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
2022-07-25T19:33:27.410659Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
2022-07-25T19:33:27.412287Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
2022-07-25T19:33:27.431360Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '::' port: 33060, socket: /var/run/mysqld/mysqlx.sock
2022-07-25T19:33:27.431373Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.29' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server - GPL.
And after second attempt container is up and running normally. So my question is - how to identify what causes to the SHUTDOWN command?
That's a normal consequence of the way the mysql image runs its first-time initialization scripts. Note the Initializing database files message and the Temporary server started...Temporary server stopped around the first Received SHUTDOWN message.
You can see this logic in the entrypoint script itself. That script runs (using shell functions defined previously in the script, trimmed down somewhat)
# there's no database, so it needs to be initialized
if [ -z "$DATABASE_ALREADY_EXISTS" ]; then
docker_init_database_dir "$#"
mysql_note "Starting temporary server"
docker_temp_server_start "$#"
mysql_note "Temporary server started"
docker_setup_db
docker_process_init_files /docker-entrypoint-initdb.d/*
mysql_note "Stopping temporary server"
docker_temp_server_stop
mysql_note "Temporary server stopped"
fi
The mysql_note lines are visible in the container output you quote.
More specifically, the temporary server is accessible over a Unix socket within the container filesystem, but it is not accessible over the network. This gives a point where the database hasn't been set up yet, but it's still possible to run local mysql commands, for example from the initdb scripts. Once this phase has been complete, then the database restarts with its normal options.
If you're probing to see if the database is ready yet, maybe using a mysql client in a shell loop or a TCP client like the wait-for script, this means that the database won't appear "ready" until all of the initdb scripts have run.
I want to create a custom MySQL docker image in https://labs.play-with-docker.com/ using existing mysql file. MySQL file looks like below.
abc.sql
create schema my-schema1;
use my-schema1;
create table table1(
id bigint primary key,
name varchar(50)
);
insert into table1 values(7009009001,'Carry');
select * from table1;
create schema my-schema2;
use my-schema2;
create table table2(
id bigint primary key,
name varchar(50)
);
insert into table1 values(4009009001,'Marry');
select * from table2;
Dockerfile
FROM mysql
COPY ./abc.sql /docker-entrypoint-initdb.d/abc.sql
I am running below commands :
docker pull mysql:latest
docker build -f Dockerfile -t custom-mysql-img .
docker run --name=mysql-con --volume custom-mysql-volume:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=root -d custom-mysql-img
docker exec -it mysql-con mysql -u root -p
Container gets started and asks for password. But on executing show databases; command it only displays default schemas. I need multiple schemas and tables in my .sql file, hence don't want to put schema name in Dockerfile.
Please suggest where I am wrong.
Below is the output of docker logs mysql-con :
$ docker logs mysql-con
2021-04-18 13:16:39+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.23-1debian10 started.
2021-04-18 13:16:39+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2021-04-18 13:16:40+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.23-1debian10 started.
2021-04-18 13:16:40+00:00 [Note] [Entrypoint]: Initializing database files
2021-04-18T13:16:40.236069Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.23) initializing of server in progress as process 42
2021-04-18T13:16:40.253242Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2021-04-18T13:16:40.923755Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2021-04-18T13:16:43.583388Z 6 [Warning] [MY-010453] [Server] root#localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.
2021-04-18 13:16:47+00:00 [Note] [Entrypoint]: Database files initialized
2021-04-18 13:16:47+00:00 [Note] [Entrypoint]: Starting temporary server
2021-04-18T13:16:47.959622Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.23) starting as process 87
2021-04-18T13:16:48.100393Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2021-04-18T13:16:48.506573Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2021-04-18T13:16:48.753607Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Socket: /var/run/mysqld/mysqlx.sock
2021-04-18T13:16:49.015712Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
2021-04-18T13:16:49.018919Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
2021-04-18T13:16:49.023237Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
2021-04-18T13:16:49.082436Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.23' socket: '/var/run/mysqld/mysqld.sock' port: 0 MySQL Community Server - GPL.
2021-04-18 13:16:49+00:00 [Note] [Entrypoint]: Temporary server started.
Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.
2021-04-18 13:16:55+00:00 [Note] [Entrypoint]: /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/abc.sql
/usr/local/bin/docker-entrypoint.sh: line 75: /docker-entrypoint-initdb.d/abc.sql: Permission denied
2021-04-18 13:17:43+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.23-1debian10 started.
2021-04-18 13:17:43+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2021-04-18 13:17:43+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.23-1debian10 started.
2021-04-18T13:17:44.067090Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.23) starting as process 1
2021-04-18T13:17:44.082414Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2021-04-18T13:17:45.999169Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2021-04-18T13:17:46.234843Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '::' port: 33060, socket: /var/run/mysqld/mysqlx.sock
2021-04-18T13:17:46.322310Z 0 [System] [MY-010229] [Server] Starting XA crash recovery...
2021-04-18T13:17:46.378281Z 0 [System] [MY-010232] [Server] XA crash recovery finished.
2021-04-18T13:17:46.454642Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
2021-04-18T13:17:46.455015Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
2021-04-18T13:17:46.459673Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
2021-04-18T13:17:46.491885Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.23' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server - GPL.
Good day!
I am getting the followin error and everytime i get this the sql service gets restarted automatically.
We tried changing the max_open_files and Limitnofile but still unlucky.
The error below:#
2021-02-24T05:48:23.615394Z 135 [Warning] [MY-010057] [Server] IP address '144.217.89.37' has been resolved to the host name '37.ip-144-217-89.net', which resembles IPv4-address itself.
05:55:13 UTC - mysqld got signal 11 ;
Most likely, you have hit a bug, but this error can also be caused by malfunctioning hardware.
Thread pointer: 0x7fba5418e130
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7fba64a5ed10 thread_stack 0xc0000
/usr/sbin/`mysqld`(my_print_stacktrace(unsigned char const*, unsigned long)+0x3d) [0x55b2aaf1049d]
/usr/sbin/mysqld(handle_fatal_signal+0x2fb) [0x55b2a9eba22b]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x12980) [0x7fbac2056980]
/usr/sbin/mysqld(+0xf293ce) [0x55b2a9d5d3ce]
/usr/sbin/mysqld(+0xf295f2) [0x55b2a9d5d5f2]
/usr/sbin/mysqld(+0xf29623) [0x55b2a9d5d623]
/usr/sbin/mysqld(JOIN::finalize_table_conditions()+0xeb) [0x55b2a9d6387b]
/usr/sbin/mysqld(JOIN::optimize()+0x2e16) [0x55b2a9d77866]
/usr/sbin/mysqld(SELECT_LEX::optimize(THD*)+0xb6) [0x55b2a9dd3c46]
/usr/sbin/mysqld(SELECT_LEX_UNIT::optimize(THD*, TABLE*, bool)+0x7b) [0x55b2a9e423eb]
/usr/sbin/mysqld(SELECT_LEX::optimize(THD*)+0x115) [0x55b2a9dd3ca5]
/usr/sbin/mysqld(SELECT_LEX_UNIT::optimize(THD*, TABLE*, bool)+0x7b) [0x55b2a9e423eb]
/usr/sbin/mysqld(Sql_cmd_dml::execute_inner(THD*)+0x3d) [0x55b2a9dd265d]
/usr/sbin/mysqld(Sql_cmd_dml::execute(THD*)+0x598) [0x55b2a9ddc6e8]
/usr/sbin/mysqld(mysql_execute_command(THD*, bool)+0xaae) [0x55b2a9d7f46e]
/usr/sbin/mysqld(sp_instr_stmt::exec_core(THD*, unsigned int*)+0x51) [0x55b2a9cfbe71]
/usr/sbin/mysqld(sp_lex_instr::reset_lex_and_exec_core(THD*, unsigned int*, bool)+0x17b) [0x55b2a9cfdd4b]
/usr/sbin/mysqld(sp_lex_instr::validate_lex_and_execute_core(THD*, unsigned int*, bool)+0xde) [0x55b2a9cfecfe]
/usr/sbin/mysqld(sp_instr_stmt::execute(THD*, unsigned int*)+0xc3) [0x55b2a9d00543]
/usr/sbin/mysqld(sp_head::execute(THD*, bool)+0x3be) [0x55b2a9cf50de]
/usr/sbin/mysqld(sp_head::execute_procedure(THD*, mem_root_deque<Item*>*)+0x7e3) [0x55b2a9cf8423]
/usr/sbin/mysqld(Sql_cmd_call::execute_inner(THD*)+0x136) [0x55b2aa20b106]
/usr/sbin/mysqld(Sql_cmd_dml::execute(THD*)+0x598) [0x55b2a9ddc6e8]
/usr/sbin/mysqld(mysql_execute_command(THD*, bool)+0xaae) [0x55b2a9d7f46e]
/usr/sbin/mysqld(dispatch_sql_command(THD*, Parser_state*)+0x3f9) [0x55b2a9d82d29]
/usr/sbin/mysqld(dispatch_command(THD*, COM_DATA const*, enum_server_command)+0x15ed) [0x55b2a9d8480d]
/usr/sbin/mysqld(do_command(THD*)+0x1c4) [0x55b2a9d85fe4]
/usr/sbin/mysqld(+0x1077020) [0x55b2a9eab020]
/usr/sbin/mysqld(+0x2640091) [0x55b2ab474091]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fbac204b6db]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fbac023d71f]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7fba549617f8): is an invalid pointer
Connection ID (thread ID): 28
Status: NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
2021-02-24T05:55:13.760686Z 0 [Warning] [MY-010140] [Server] Could not increase number of max_open_files to more than 10000 (request: 25000)2021-02-24T05:55:13.760686Z 0 [Warning] [MY-010140] [Server] Could not increase number of max_open_files to more than 10000 (request: 25000)
2021-02-24T05:55:14.042554Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.23) starting as process 31229
2021-02-24T05:55:14.054670Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2021-02-24T05:55:14.941930Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2021-02-24T05:55:15.232295Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '::' port: 33060, socket: /var/run/mysqld/mysqlx.sock
2021-02-24T05:55:15.260938Z 0 [System] [MY-010229] [Server] Starting XA crash recovery...
2021-02-24T05:55:15.273573Z 0 [System] [MY-010232] [Server] XA crash recovery finished.
2021-02-24T05:55:15.316036Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
2021-02-24T05:55:15.316219Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
2021-02-24T05:55:15.344388Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.23' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server - GPL.
2021-02-24T05:55:15.551426Z 8 [Warning] [MY-010057] [Server] IP address '167.114.185.61' has been resolved to the host name '61.ip-167-114-185.net', which resembles IPv4-address itself.
Anyone has anyidea. The current mysql version is 8.0.23 community version.
I am trying to add an auto increment column to a mysql table as:
mysql> ALTER TABLE scraped_comments ADD row_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY;
There are about 15 million records in the table. The query runs for about one and a half hour and then ends with the error:
ERROR 2013 (HY000): Lost connection to MySQL server during query
I tried setting higher timeout values by editing /etc/mysql/mysql.conf.d/mysqld.cnf
wait_timeout = 31536000
interactive_timeout = 31536000
and then running the query again. However the error persisted. While the query was running, I thought the issue may be caused by less memory so I had htop open in another terminal but less than half of memory was being used.
I checked /var/log/mysql/error.log file and found following in that file:
...
InnoDB: ###### Diagnostic info printed to the standard error stream
2019-06-14T06:21:49.965496Z 0 [ERROR] [MY-012872] [InnoDB] Semaphore wait has lasted > 38632704 seconds. We intentionally crash the server because it appears to be hung.[FATAL] Semaphore wait has lasted > 600 seconds. We intentionally crash the server because it appears to be hung.
2019-06-14T06:21:49.965563Z 0 [ERROR] [MY-013183] [InnoDB] Assertion failure: ut0ut.cc:629 thread 140385843095296
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/8.0/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
06:21:49 UTC - mysqld got signal 6 ;
Most likely, you have hit a bug, but this error can also be caused by malfunctioning hardware.
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x46000
/usr/sbin/mysqld(my_print_stacktrace(unsigned char*, unsigned long)+0x3d) [0x1dc2f5d]
/usr/sbin/mysqld(handle_fatal_signal+0x351) [0xeee441]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x11390) [0x7fae50d4f390]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38) [0x7fae4f241428]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x16a) [0x7fae4f24302a]
/usr/sbin/mysqld(ut_dbg_assertion_failed(char const*, char const*, unsigned long)+0x2db) [0x20736cb]
/usr/sbin/mysqld(ib::fatal::~fatal()+0x18e) [0x2075afe]
/usr/sbin/mysqld(srv_error_monitor_thread()+0xc07) [0x200ccf7]
/usr/sbin/mysqld(std::thread::_Impl<std::_Bind_simple<Runnable (void (*)())> >::_M_run()+0x63) [0x1e6e543]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb8c80) [0x7fae4fbadc80]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba) [0x7fae50d456ba]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fae4f31341d]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
2019-06-14T06:21:53.867586Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.16) starting as process 6199
2019-06-14T06:21:58.097227Z 0 [System] [MY-010229] [Server] Starting crash recovery...
2019-06-14T06:21:58.114311Z 0 [System] [MY-010232] [Server] Crash recovery finished.
2019-06-14T06:21:59.096291Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
2019-06-14T06:21:59.225066Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.16' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server - GPL.
2019-06-14T06:21:59.436191Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Socket: '/var/run/mysqld/mysqlx.sock' bind-address: '::' port: 33060
...
Thanks for the help
Came across
MySQL: InnoDb: Semaphore wait has lasted > 600 seconds. We intentionally crash the server
and increased thread_cache_size,innodb_buffer_pool_size and table_open_cache. I guess the issue was due to innodb_buffer_size as I first tried with innodb_buffer_size of 2 GB which didn't work. I then increased it to 3 GB and the query completed successfully