MariaDB: ERROR 2013 (HY000): Lost connection persists only for one database - mysql

I have several databases:
- curr
- curr_add
and many others on the same MariaDB instance. When I submit an sql to curr_add I get connection and the results. When I send the query to curr I get the error:
MariaDB: ERROR 2013 (HY000): Lost connection to MySQL server during query
and the service stops. When I restart the service each time I want to access the database curr the serivice stops. This is not the case for all other databases of the same instance. The instance and databases exists since one year and I had never such problems. Is there a way to find out what is the problem and how to solve it?
Working environment:
Windows 7
MariaDB 10.2.6
I entered as root: mysql -uroot -h localhost -p
Update (1):
I have access to database information_schema. For example, I can count the rows of table system_variables: SELECT COUNT(*) FROM system_variables. But if I try the same with table columns: SELECT COUNT(*) FROM columns, the connection will be lost (see figure).
I submit:
SELECT COUNT(*) FROM tables
or
SELECT table_schema, table_name FROM tables
I get results.
But if I submit
SELECT * FROM tables
or
SELECT table_schema, table_name, engine, table_rows FROM tables LIMIT 10
the connection will be lost.
Summary: some database disconnect the mysql-server and also some columns of tables.
Update (2): error informations from file .err
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
2017-11-28 19:42:43 7820 [Note] InnoDB: Mutexes and rw_locks use Windows interlocked functions
2017-11-28 19:42:43 7820 [Note] InnoDB: Uses event mutexes
2017-11-28 19:42:43 7820 [Note] InnoDB: Compressed tables use zlib 1.2.3
2017-11-28 19:42:43 7820 [Note] InnoDB: Number of pools: 1
2017-11-28 19:42:43 7820 [Note] InnoDB: Using generic crc32 instructions
2017-11-28 19:42:43 7820 [Note] InnoDB: Initializing buffer pool, total size = 14G, instances = 8, chunk size = 128M
2017-11-28 19:42:44 7820 [Note] InnoDB: Completed initialization of buffer pool
2017-11-28 19:42:44 7820 [Note] InnoDB: Highest supported file format is Barracuda.
2017-11-28 19:42:44 7820 [Note] InnoDB: Starting crash recovery from checkpoint LSN=556718604758
2017-11-28 19:42:52 7820 [Note] InnoDB: 128 out of 128 rollback segments are active.
2017-11-28 19:42:52 7820 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2017-11-28 19:42:52 7820 [Note] InnoDB: Creating shared tablespace for temporary tables
2017-11-28 19:42:52 7820 [Note] InnoDB: Setting file '.\ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2017-11-28 19:42:52 7820 [Note] InnoDB: File '.\ibtmp1' size is now 12 MB.
2017-11-28 19:42:52 7820 [Note] InnoDB: 5.7.14 started; log sequence number 556718604767
2017-11-28 19:42:52 4868 [Note] InnoDB: page_cleaner: 1000ms intended loop took 7784ms. The settings might not be optimal. (flushed=0 and evicted=0, during the time.)
2017-11-28 19:42:52 7380 [Note] InnoDB: Loading buffer pool(s) from C:\Program Files\MariaDB 10.2\data\ib_buffer_pool
2017-11-28 19:42:52 7820 [Note] Server socket created on IP: '::'.
2017-11-28 19:42:52 7820 [Note] Reading of all Master_info entries succeded
2017-11-28 19:42:52 7820 [Note] Added new Master_info '' to hash table
2017-11-28 19:42:52 7820 [Note] C:\Program Files\MariaDB 10.2\bin\mysqld.exe: ready for connections.
Version: '10.2.6-MariaDB' socket: '' port: 3306 mariadb.org binary distribution
2017-11-28 19:44:08 7380 [Note] InnoDB: Buffer pool(s) load completed at 171128 19:44:08
2017-11-28 19:44:20 9820 [Warning] InnoDB: Retry attempts for reading partial data failed.
2017-11-28 19:44:20 9820 [ERROR] InnoDB: Tried to read 16384 bytes at offset 4947968, but was only able to read 0
2017-11-28 19:44:20 9820 [ERROR] InnoDB: File (unknown): 'read' returned OS error 0. Cannot continue operation
171128 19:44:20 [ERROR] mysqld got exception 0x80000003 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
Server version: 10.2.6-MariaDB
key_buffer_size=134217728
read_buffer_size=131072
max_used_connections=1
max_threads=65537
thread_count=7
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 136026 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x7d1b9fd8
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
mysqld.exe!my_parameter_handler()[my_init.c:259]
mysqld.exe!raise()[signal.cpp:516]
mysqld.exe!abort()[abort.cpp:71]
mysqld.exe!os_file_handle_error_cond_exit()[os0file.cc:5209]
mysqld.exe!os_file_read_page()[os0file.cc:5091]
mysqld.exe!os_file_read_func()[os0file.cc:5433]
mysqld.exe!fil_io()[fil0fil.cc:5436]
mysqld.exe!buf_read_page_low()[buf0rea.cc:179]
mysqld.exe!buf_read_page()[buf0rea.cc:436]
mysqld.exe!buf_page_get_gen()[buf0buf.cc:4267]
mysqld.exe!btr_cur_search_to_nth_level()[btr0cur.cc:1115]
mysqld.exe!btr_pcur_open_low()[btr0pcur.ic:457]
mysqld.exe!btr_pcur_open_on_user_rec_func()[btr0pcur.cc:597]
mysqld.exe!dict_load_foreign()[dict0load.cc:3334]
mysqld.exe!dict_load_foreigns()[dict0load.cc:3587]
mysqld.exe!dict_load_table_one()[dict0load.cc:2958]
mysqld.exe!dict_load_table()[dict0load.cc:2670]
mysqld.exe!dict_table_open_on_name()[dict0dict.cc:1174]
mysqld.exe!ha_innobase::open_dict_table()[ha_innodb.cc:6976]
mysqld.exe!ha_innobase::open()[ha_innodb.cc:6618]
mysqld.exe!handler::ha_open()[handler.cc:2507]
mysqld.exe!open_table_from_share()[table.cc:3278]
mysqld.exe!open_table()[sql_base.cc:1874]
mysqld.exe!open_and_process_table()[sql_base.cc:3409]
mysqld.exe!open_tables()[sql_base.cc:3926]
mysqld.exe!open_and_lock_tables()[sql_base.cc:4682]
mysqld.exe!execute_sqlcom_select()[sql_parse.cc:6352]
mysqld.exe!mysql_execute_command()[sql_parse.cc:3448]
mysqld.exe!mysql_parse()[sql_parse.cc:7879]
mysqld.exe!dispatch_command()[sql_parse.cc:1814]
mysqld.exe!do_command()[sql_parse.cc:1361]
mysqld.exe!threadpool_process_request()[threadpool_common.cc:346]
mysqld.exe!tp_callback()[threadpool_common.cc:192]
ntdll.dll!TpPostWork()
ntdll.dll!RtlRealSuccessor()
kernel32.dll!BaseThreadInitThunk()
ntdll.dll!RtlUserThreadStart()
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x78b38ff0): SELECT COUNT(*) FROM curr.patient
Connection ID (thread ID): 9
Status: NOT_KILLED
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.

Based on the stack trace, it seems to be the InnoDB system tablespace that is shorter than expected. When the function dict_load_foreigns() is accessing the InnoDB system table SYS_FOREIGN or SYS_FOREIGN_COLS, it is requesting a page that is not in the buffer pool. The page read request causes InnoDB to commit suicide, because the file is too short.
InnoDB notoriously does not report the problematic file name. We should refactor the I/O code in MariaDB at some point. In this case, we do know that the problem is in the InnoDB system tablespace, because the InnoDB internal SYS_ tables are located there.
There already exist some related bugs in the MariaDB tracker. I think that this scenario is already covered by these:
MDEV-13542 Crashing on a corrupted page is unhelpful (Yes, I copied the title of the ancient MySQL Bug#10132)
MDEV-11633 Make the InnoDB system tablespace optional (This is not going to happen soon, and the design is not finalized)
It would be interesting to know how the corruption occurred in the first place. Before MDEV-11556, InnoDB data file extension in MariaDB was not fully crash-safe. (MySQL does not contain this fix at all.)
Could it be that the files were copied at some point? A bug in the copy procedure? Or could the system tablespace have originally consisted of multiple files, but the server was started up with the wrong innodb_data_file_path so that the last file(s) were ignored? Everything would appear fine until a page in the ‘missing’ files is being accessed.
You might ask: How to work around this error? Unfortunately, I don’t think that there currently is any way to skip the read of the foreign key metadata. So, if the metadata tables are corrupted, in the worst case you will be unable to access any InnoDB tables. For this, I would welcome a MariaDB bug report.

I wanted to reinstall MariaDB. When I started deinstall/change of MaridaDB through the application wizard of Window 7 it asked if I wanted to Change/Repair/Remove. I decided to repair. After that MariaDB worked as usual. That is I could submit queries without losing connection.
Upgrade from MariaDB from 10.2.6 to 10.2.11 didn't help. Repair allowed me to be successful.
Lesson learned: Before asking SO do repair MariaDB.

Related

How to fix corupted MySQL installation

Due to some issues with the server, a number of InnoDB database folders with ibd, frm and TRN files were deleted (only affects databases not really needed anymore). Currently MySQL service starts only with the following setting:
innodb_force_recovery = 5
Then it crashes every few seconds, here's an extract from the log file
2022-10-11 17:38:54 27975 [Note] InnoDB: 1.2.10 started; log sequence number 313646609430
2022-10-11 17:38:54 27975 [Note] InnoDB: !!! innodb_force_recovery is set to 5 !!!
2022-10-11 17:38:54 27975 [Note] Server hostname (bind-address): '*'; port: 3306
2022-10-11 17:38:54 27975 [Note] IPv6 is available.
2022-10-11 17:38:54 27975 [Note] - '::' resolves to '::';
2022-10-11 17:38:54 27975 [Note] Server socket created on IP: '::'.
2022-10-11 17:38:54 27975 [Note] Event Scheduler: Loaded 0 events
2022-10-11 17:38:54 27975 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.6.10' socket: '/var/lib/mysql/mysql.sock' port: 3306 MySQL Community Server (GPL)
2022-10-11 17:38:55 7f1d2c06a700 InnoDB: Error: page 4087 log sequence number 315137164614
InnoDB: is in the future! Current system log sequence number 313646609430.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: for more information.
2022-10-11 17:38:55 27975 [ERROR] InnoDB: Failed to find tablespace for table '"wp_website"."wp_options"' in the cache. Attempting to load the tablespace with space id 34249.
2022-10-11 17:38:55 27975 [Warning] InnoDB: Allocated tablespace 34249, old maximum was 0
2022-10-11 17:38:55 7f1d2c06a700 InnoDB: Error: page 252899 log sequence number 471118864984
InnoDB: is in the future! Current system log sequence number 313646609430.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: for more information.
2022-10-11 17:38:55 7f1d2c06a700 InnoDB: Error: page 85552 log sequence number 471118895771
InnoDB: is in the future! Current system log sequence number 313646609430.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: for more information.
2022-10-11 17:38:55 7f1d2c06a700 InnoDB: Error: page 147803 log sequence number 471118895778
InnoDB: is in the future! Current system log sequence number 313646609430.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: for more information.
2022-10-11 17:38:55 7f1d2c06a700 InnoDB: Error: page 4 log sequence number 470722597051
InnoDB: is in the future! Current system log sequence number 313646609440.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: for more information.
2022-10-11 17:38:55 7f1d2c06a700 InnoDB: Error: page 13 log sequence number 315130852444
InnoDB: is in the future! Current system log sequence number 313646609440.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: for more information.
2022-10-11 17:38:55 7f1d2c06a700 InnoDB: Error: page 3 log sequence number 315130852367
InnoDB: is in the future! Current system log sequence number 313646609440.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: for more information.
2022-10-11 17:38:55 7f1d2c06a700 InnoDB: Error: page 22 log sequence number 315133587106
InnoDB: is in the future! Current system log sequence number 313646609440.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: for more information.
InnoDB: A new raw disk partition was initialized or
InnoDB: innodb_force_recovery is on: we do not allow
InnoDB: database modifications by the user. Shut down
InnoDB: mysqld and edit my.cnf so that newraw is replaced
InnoDB: with raw, and innodb_force_... is removed.
14:38:55 UTC - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
key_buffer_size=8388608
read_buffer_size=131072
max_used_connections=1
max_threads=151
thread_count=1
connection_count=1
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 68216 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x1d46900
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7f1d2c069e18 thread_stack 0x40000
/usr/sbin/mysqld(my_print_stacktrace+0x35)[0x8bfd45]
/usr/sbin/mysqld(handle_fatal_signal+0x4a4)[0x65b074]
/lib64/libpthread.so.0[0x34c700f7e0]
/usr/sbin/mysqld[0x9a45b2]
/usr/sbin/mysqld[0x97e382]
/usr/sbin/mysqld[0x943807]
/usr/sbin/mysqld[0x9a236d]
/usr/sbin/mysqld[0x9a2bfd]
/usr/sbin/mysqld[0x8e1729]
/usr/sbin/mysqld(_Z15ha_rollback_lowP3THDb+0x87)[0x5a27a7]
/usr/sbin/mysqld(_Z17ha_rollback_transP3THDb+0x4c)[0x5a256c]
/usr/sbin/mysqld(_Z19trans_rollback_stmtP3THD+0x29)[0x764a19]
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x23d)[0x6d2fdd]
/usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x32f)[0x6d7a5f]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0xe28)[0x6d8978]
/usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0xcf)[0x6a682f]
/usr/sbin/mysqld(handle_one_connection+0x47)[0x6a6957]
/usr/sbin/mysqld(pfs_spawn_thread+0x139)[0xadc7d9]
/lib64/libpthread.so.0[0x34c7007aa1]
/lib64/libc.so.6(clone+0x6d)[0x34c6ce8c4d]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7f1d04004f30): is an invalid pointer
Connection ID (thread ID): 1
Status: NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
221011 17:38:55 mysqld_safe Number of processes running now: 0
221011 17:38:55 mysqld_safe mysqld restarted
How can I fix the problem? Or at least create the dumps of a few databases I need?

mysql large tables (>4Gb) corrupt after local server restart

I hope someone can help me with this problem I'm having (I'm a newbie to Mysql and Stackoverflow, so it's probably something silly I'm missing)...
Whenever I build a large table in MySQL and then restart xampp's local MySQL server, the table then becomes unreadable under code runs that consider the 'whole' table - e.g. I can't drop it, run mySQLCheck on it or run any aggregation functions like sum / count.
This only happens after a server restart. It is fine when I build the table and then, without restarting, I run the 'whole table' codes on it I mentioned above.
Also, even after a server restart I can still do things like see the indexes in place, or sample the first x rows using Limit.
The error thrown is 'mysqld.exe has stopped working' - 'a problem caused the program to stop working correctly. Windows will close the program and notify you if a solution is available'
I stop and restart my server using the xampp start / stop control panel buttons
This only seems to happen when the table is more than 4gb. I'm using innodb
Thank you in advance!
Paul
===============================
An example of the error log result is below. This table (wikivisitstable10) was created by adding an index to the prior table. There are no crash issues with the prior table, just this one - and any similar tables >4gb:
`2016-08-25 0:22:57 6804 [Warning] option
'innodb-max-dirty-pages-pct': value 0 adjusted to 0.001
2016-08-25 0:22:57 6804 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2016-08-25 0:22:57 6804 [Note] InnoDB: The InnoDB memory heap is disabled
2016-08-25 0:22:57 6804 [Note] InnoDB: Mutexes and rw_locks use Windows interlocked functions 2016-08-25 0:22:57 6804
[Note] InnoDB: Memory barrier is not used 2016-08-25 0:22:57 6804
[Note] InnoDB: Compressed tables use zlib 1.2.3
2016-08-25 0:22:57 6804 [Note] InnoDB: Using generic crc32 instructions
2016-08-25 0:22:57 6804 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2016-08-25 0:22:57 6804 [Note] InnoDB: Completed initialization of
buffer pool
2016-08-25 0:22:57 6804 [Note] InnoDB: Highest supported
file format is Barracuda.
2016-08-25 0:22:57 6804 [Note] InnoDB: The log sequence numbers 400931068139 and 400931068139 in ibdata files do not match the log sequence number 400931068159 in the ib_logfiles!
2016-08-25 0:22:57 6804 [Note] InnoDB: Database was not shutdown
normally!
2016-08-25 0:22:57 6804 [Note] InnoDB: Starting crash
recovery.
2016-08-25 0:22:57 6804 [Note] InnoDB: Reading tablespace
information from the .ibd files...
2016-08-25 0:22:58 6804 [Note] InnoDB: Restoring possible half-written data pages
2016-08-25 0:22:58 6804 [Note] InnoDB: from the doublewrite buffer...
2016-08-25 00:22:59 1a94 InnoDB: Error: table 'revil2/page'
InnoDB: in InnoDB data dictionary has tablespace id 180, InnoDB: but a tablespace with that id does not exist. There is InnoDB: a tablespace of name revil2/page and id 181, though. Have InnoDB: you deleted or moved .ibd files?
InnoDB: Please refer to InnoDB: DATADICT LINK (REMOVED)
InnoDB: for how to resolve the issue.
2016-08-25 0:22:59 6804 [ERROR] InnoDB: Table revil2/wikipages7 in the InnoDB data dictionary has tablespace id 175, but tablespace with that id or name does not exist. Have you deleted or moved .ibd files? This may also be a table created with CREATE TEMPORARY TABLE whose .ibd and .frm files MySQL automatically removed, but the table still exists in the InnoDB internal data dictionary.
InnoDB: Please refer to InnoDB: DATA DICT LINK (REMOVED)
InnoDB: for how to resolve the issue.
2016-08-25 0:22:59 6804 [Note] InnoDB: 128 rollback segment(s) are active.
2016-08-25 0:22:59 6804 [Note] InnoDB: Waiting for purge to start
2016-08-25 0:22:59 6804 [Note] InnoDB: Percona XtraDB (PERCONA
WEBSITE (REMOVED)) 5.6.28-76.1 started; log sequence number
400931068159
2016-08-25 0:22:59 5820 [Note] InnoDB: Dumping buffer pool(s) not yet started
2016-08-25 00:22:59 16bc InnoDB: Loading buffer pool(s) from .\\ib_buffer_pool
2016-08-25 0:22:59 6804 [Note] Plugin 'FEEDBACK' is disabled.
2016-08-25 0:22:59 6804 [Note] Server socket created on IP: '::'.
2016-08-25 00:22:59 16bc InnoDB: Buffer pool(s) load completed at 160825 0:22:59
2016-08-25 0:23:00 6804 [Note] c:\xampp\mysql\bin\mysqld.exe: ready for connections. Version: '10.1.13-MariaDB' socket: '' port: 3306 mariadb.org binary distribution
InnoDB: Error: trying to access page number 1794 in space 299, InnoDB: space name revil2/wikivisitstable10, InnoDB: which is outside the tablespace bounds.
InnoDB: Byte offset 0, len 16384, i/o type 10.
InnoDB: If you get this error at mysqld startup, please check that InnoDB: your my.cnf matches the ibdata files that you have in the InnoDB: MySQL server.
2016-08-25 00:23:46 1388 InnoDB: Assertion failure in thread 5000 in file fil0fil.cc line 5866
InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed
bug report to HTTP BUGS. MYSQL. COM InnoDB: If you get repeated
assertion failures or crashes, even InnoDB: immediately after the
mysqld startup, there may be InnoDB: corruption in the InnoDB
tablespace.
Please refer to InnoDB: FORCING INNODB RECOVERY LINK (REMOVED)
InnoDB: about forcing recovery. 160825 0:23:46
[ERROR] mysqld got exception 0x80000003 ; This could be because you hit a
bug. It is also possible that this binary or one of the libraries it
was linked against is corrupt, improperly built, or misconfigured.
This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, something
is definitely wrong and this may fail.
Server version: 10.1.13-MariaDB
key_buffer_size=268435456
read_buffer_size=268435456
max_used_connections=1
max_threads=1001
thread_count=1
It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 788380 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x0x1c172e20 Attempting backtrace. You can use the
following information to find out where mysqld died. If you see no
messages after this, something went terribly wrong...
mysqld.exe!my_parameter_handler() mysqld.exe!my_mb_ctype_mb()
mysqld.exe!?get_ctx#MDL_ticket##QBEPAVMDL_context##XZ()
mysqld.exe!?get_ctx#MDL_ticket##QBEPAVMDL_context##XZ()
mysqld.exe!?functype#Item_func_dyncol_create##UBE?AW4Functype#Item_func##XZ()
mysqld.exe!?get_ctx#MDL_ticket##QBEPAVMDL_context##XZ()
mysqld.exe!?get_ctx#MDL_ticket##QBEPAVMDL_context##XZ()
mysqld.exe!?get_trg_event_map#Update_rows_log_event##UAEEXZ()
mysqld.exe!?ha_check#handler##QAEHPAVTHD##PAUst_ha_check_opt###Z()
mysqld.exe!??_9handler##$BBAE#AE()
mysqld.exe!?execute#Sql_cmd_check_table##UAE_NPAVTHD###Z()
mysqld.exe!?mysql_execute_command##YAHPAVTHD###Z()
mysqld.exe!?mysql_parse##YAXPAVTHD##PADIPAVParser_state###Z()
mysqld.exe!?dispatch_command##YA_NW4enum_server_command##PAVTHD##PADI#Z()
mysqld.exe!?do_command##YA_NPAVTHD###Z()
mysqld.exe!?threadpool_process_request##YAHPAVTHD###Z()
mysqld.exe!?tp_end##YAXXZ() KERNEL32.DLL!SetUserGeoID()
ntdll.dll!TpSimpleTryPost() ntdll.dll!EtwNotificationRegister()
KERNEL32.DLL!BaseThreadInitThunk()
ntdll.dll!RtlUnicodeStringToInteger()
ntdll.dll!RtlUnicodeStringToInteger()
Trying to get some variables. Some pointers may be invalid and cause
the dump to abort. Query (0x1c232750): CHECK TABLE
`wikivisitstable10` Connection ID (thread ID): 2 Status: NOT_KILLED`
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on

Mysql server crash on start

Mysql server crash on start on my Ubuntu 12.04.2 server. Mysql just not starting. Not after reboot, not after service mysql start(it return start: Job failed to start).
Previously it worked fine, but at some point the server crashed and I was not able to run it.
/var/log/mysql/error.log:
151109 10:02:14 [Note] Plugin 'FEDERATED' is disabled.
151109 10:02:14 InnoDB: The InnoDB memory heap is disabled
151109 10:02:14 InnoDB: Mutexes and rw_locks use GCC atomic builtins
151109 10:02:14 InnoDB: Compressed tables use zlib 1.2.3.4
151109 10:02:14 InnoDB: Initializing buffer pool, size = 128.0M
151109 10:02:14 InnoDB: Completed initialization of buffer pool
InnoDB: Database page corruption on disk or a failed
InnoDB: file read of page 5.
InnoDB: You may have to recover from a backup.
151109 10:02:15 InnoDB: Page dump in ascii and hex (16384 bytes):
len 16384; hex 76607...(I removed the part)...3c3f438045ca5727; asc v` E W1 9 - . / 0 1 2 3 4 5 6 7 8 9 : ; < = > ? ! " # $ % & ' ( ) * + , _ # _ # jPs <?C E W';
InnoDB: End of page dump
151109 10:02:15 InnoDB: Page checksum 1986035693, prior-to-4.0.14-form checksum 3548252131
InnoDB: stored checksum 1986035693, prior-to-4.0.14-form stored checksum 1010779008
InnoDB: Page lsn 0 1170888497, low 4 bytes of lsn at page end 1170888487
InnoDB: Page number (if stored to page already) 5,
InnoDB: space id (if created with >= MySQL-4.1.1 and stored already) 0
InnoDB: Page may be a transaction system page
InnoDB: Database page corruption on disk or a failed
InnoDB: file read of page 5.
InnoDB: You may have to recover from a backup.
InnoDB: It is also possible that your operating
InnoDB: system has corrupted its own file cache
InnoDB: and rebooting your computer removes the
InnoDB: error.
InnoDB: If the corrupt page is an index page
InnoDB: you can also try to fix the corruption
InnoDB: by dumping, dropping, and reimporting
InnoDB: the corrupt table. You can use CHECK
InnoDB: TABLE to scan your table for corruption.
InnoDB: See also http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
InnoDB: Ending processing because of a corrupt database page.
151109 10:02:15 InnoDB: Assertion failure in thread 3065161472 in file buf0buf.c line 3629
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
10:02:15 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
key_buffer_size=16777216
read_buffer_size=131072
max_used_connections=0
max_threads=151
thread_count=0
connection_count=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 346064 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x30000
/usr/sbin/mysqld(my_print_stacktrace+0x33)[0xb733d003]
/usr/sbin/mysqld(handle_fatal_signal+0x484)[0xb71e9f24]
[0xb6ea0400]
/usr/sbin/mysqld(+0x590e78)[0xb7453e78]
/usr/sbin/mysqld(+0x591852)[0xb7454852]
/usr/sbin/mysqld(+0x58109a)[0xb744409a]
/usr/sbin/mysqld(+0x54f75d)[0xb741275d]
/usr/sbin/mysqld(+0x551d09)[0xb7414d09]
/usr/sbin/mysqld(+0x53d5e0)[0xb74005e0]
/usr/sbin/mysqld(+0x501a22)[0xb73c4a22]
/usr/sbin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x4e)[0xb71ec74e]
/usr/sbin/mysqld(+0x1fbd82)[0xb70bed82]
/usr/sbin/mysqld(_Z11plugin_initPiPPci+0xbc3)[0xb70c2723]
/usr/sbin/mysqld(+0x16691a)[0xb702991a]
/usr/sbin/mysqld(_Z11mysqld_mainiPPc+0x66a)[0xb702d61a]
/usr/sbin/mysqld(main+0x27)[0xb7022d27]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xb6b5f4d3]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.

"ERROR 2013 (HY000): Lost connection to MySQL server during query" while dropping database

I have an empty database that I cannot drop. Initially, it contained a table that I couldn't SELECT from. So I DROP'ed all tables from that database and tried to DROP the database without success :
mysql> drop database my_database;
ERROR 2013 (HY000): Lost connection to MySQL server during query
Now I have an empty database that I cannot remove in my server.
I check the mysql error.log, here is the output :
130812 10:02:45 InnoDB: Assertion failure in thread 140409656780544 in file row0mysql.c line 3682
InnoDB: Failing assertion: table
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
14:02:45 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
key_buffer_size=16777216
read_buffer_size=131072
max_used_connections=3
max_threads=151
thread_count=3
connection_count=3
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 346685 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x7fb3ad55d030
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7fb3abb71e60 thread_stack 0x30000
/usr/sbin/mysqld(my_print_stacktrace+0x29)[0x7fb3ac1516b9]
/usr/sbin/mysqld(handle_fatal_signal+0x3d8)[0x7fb3ac039318]
/lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0)[0x7fb3aab96cb0]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35)[0x7fb3aa1ff425]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x17b)[0x7fb3aa202b8b]
/usr/sbin/mysqld(+0x561acd)[0x7fb3ac192acd]
/usr/sbin/mysqld(+0x542c63)[0x7fb3ac173c63]
/usr/sbin/mysqld(+0x40837f)[0x7fb3ac03937f]
/usr/sbin/mysqld(_Z24plugin_foreach_with_maskP3THDPFcS0_P13st_plugin_intPvEijS3_+0x165)[0x7fb3abf4daa5]
/usr/sbin/mysqld(_Z11mysql_rm_dbP3THDPcbb+0x300)[0x7fb3abf23580]
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x203c)[0x7fb3abf418ac]
/usr/sbin/mysqld(+0x31301e)[0x7fb3abf4401e]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x19e7)[0x7fb3abf46247]
/usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x105)[0x7fb3abfe0405]
/usr/sbin/mysqld(handle_one_connection+0x50)[0x7fb3abfe0520]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a)[0x7fb3aab8ee9a]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fb3aa2bcccd]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7fb388004b90): is an invalid pointer
Connection ID (thread ID): 59
Status: NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
130812 10:02:45 [Note] Plugin 'FEDERATED' is disabled.
130812 10:02:45 InnoDB: The InnoDB memory heap is disabled
130812 10:02:45 InnoDB: Mutexes and rw_locks use GCC atomic builtins
130812 10:02:45 InnoDB: Compressed tables use zlib 1.2.7
130812 10:02:45 InnoDB: Using Linux native AIO
130812 10:02:45 InnoDB: Initializing buffer pool, size = 128.0M
130812 10:02:45 InnoDB: Completed initialization of buffer pool
130812 10:02:45 InnoDB: highest supported file format is Barracuda.
130812 10:02:45 InnoDB: Waiting for the background threads to start
130812 10:02:46 InnoDB: 5.5.32 started; log sequence number 25852489043
130812 10:02:46 InnoDB: !!! innodb_force_recovery is set to 4 !!!
130812 10:02:46 [Note] Server hostname (bind-address): '127.0.0.1'; port: 3306
130812 10:02:46 [Note] - '127.0.0.1' resolves to '127.0.0.1';
130812 10:02:46 [Note] Server socket created on IP: '127.0.0.1'.
130812 10:02:46 [Note] Event Scheduler: Loaded 0 events
130812 10:02:46 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.5.32-0ubuntu0.12.10.1' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (Ubuntu)
I tried to put the configuration innodb_force_recovery = 4 and restart MySQL but it had no effect at all.
Reinstalling the server is not an option here. It happened on many machines (2 Ubuntu and 2 CentOS), but I cannot reproduce the problem right now.
I believe it has to do with InnoDB table corruption. I too had a similar issue and was able to pinpoint it to a single table. I'm still in the process of correcting the issue through duplicating the table and removing the single record that is bad in my case.
This link helped me significantly:
Recovering InnoDB Table Corruption
Sorry for incomplete answer, but this helped me quite a bit once I found it.

Cannot Start MySql

Our server has been running mysql just fine for over a year. I ran a set of sql script to build a rather large database and in the middle of those scripts, I started getting errors that I had lost connection. Nobody did anything else happened as far as we know. When I tried to log in to mysql, I got:
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
When I try to restart mysql, I get:
# sudo service mysql restart
stop: Unknown instance:
start: Job failed to start
The error.log shows:
130212 9:37:51 [Note] Plugin 'FEDERATED' is disabled.
130212 9:37:51 InnoDB: The InnoDB memory heap is disabled
130212 9:37:51 InnoDB: Mutexes and rw_locks use GCC atomic builtins
130212 9:37:51 InnoDB: Compressed tables use zlib 1.2.7
130212 9:37:51 InnoDB: Using Linux native AIO
130212 9:37:51 InnoDB: Initializing buffer pool, size = 10.0G
130212 9:37:51 InnoDB: Completed initialization of buffer pool
130212 9:37:51 InnoDB: highest supported file format is Barracuda.
InnoDB: Log scan progressed past the checkpoint lsn 2186809272046
130212 9:37:51 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Doing recovery: scanned up to log sequence number 2186814514688
InnoDB: Doing recovery: scanned up to log sequence number 2186816162838
InnoDB: 1 transaction(s) which must be rolled back or cleaned up
InnoDB: in total 27964 row operations to undo
InnoDB: Trx id counter is 18834200
130212 9:37:51 InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 0 1 2 3 4 5 6 7 8 9 10 11 InnoDB: Database page corruption on disk or a failed
InnoDB: file read of page 4424818.
InnoDB: You may have to recover from a backup.
130212 9:37:51 InnoDB: Page dump in ascii and hex (16384 bytes):
len 16384; hex 058425e20043847200356a3f003e3720000001fd2807769645bf00000000000000000000000000ef31a083ca0000000031980005000003c8000$
InnoDB: End of page dump
18 130212 9:37:51 InnoDB: Page checksum 1501194131, prior-to-4.0.14-form checksum 441953139
InnoDB: stored checksum 92546530, prior-to-4.0.14-form stored checksum 1240647222
InnoDB: Page lsn 509 671577750, low 4 bytes of lsn at page end 441447404
InnoDB: Page number (if stored to page already) 4424818,
InnoDB: space id (if created with >= MySQL-4.1.1 and stored already) 0
InnoDB: Page may be an index page where index id is 1096815
InnoDB: Database page corruption on disk or a failed
InnoDB: file read of page 4424818.
InnoDB: You may have to recover from a backup.
InnoDB: It is also possible that your operating
InnoDB: system has corrupted its own file cache
InnoDB: and rebooting your computer removes the
InnoDB: error.
InnoDB: If the corrupt page is an index page
InnoDB: you can also try to fix the corruption
InnoDB: by dumping, dropping, and reimporting
InnoDB: the corrupt table. You can use CHECK
InnoDB: TABLE to scan your table for corruption.
InnoDB: See also http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
InnoDB: Ending processing because of a corrupt database page.
130212 9:37:51 InnoDB: Assertion failure in thread 140114781574912 in file buf0buf.c line 3603
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
16:37:51 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
key_buffer_size=16777216
read_buffer_size=131072
max_used_connections=0
max_threads=151
thread_count=0
connection_count=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 346681 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x30000
/usr/sbin/mysqld(my_print_stacktrace+0x29)[0x7f72aa2435b9]
/usr/sbin/mysqld(handle_fatal_signal+0x3d8)[0x7f72aa12c548]
/lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0)[0x7f72a8c8dcb0]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35)[0x7f72a82f6425]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x17b)[0x7f72a82f9b8b]
/usr/sbin/mysqld(+0x605429)[0x7f72aa32d429]
/usr/sbin/mysqld(+0x631b69)[0x7f72aa359b69]
/usr/sbin/mysqld(+0x5c20a8)[0x7f72aa2ea0a8]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a)[0x7f72a8c85e9a]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f72a83b3cbd]
I cannot find anything running for mysql including any sockets.
I am taking over for a previous SysAdmin and am fairly new to linux and MySql. We've got to get this system back online soon. Please help.
Have you tried adding the following line to your /etc/mysql/my.cnf and then restarting the server?
[mysqld]
innodb_force_recovery = 4
To add to #elico3000 you will now need to dump your corrupt table(s) and Data to repair the innodb fs. There are a number of ways to do this. You can read through the logs to determine the point of failure and possible tablenames, then dump and recreate those specific tables. Or you can dump the entire MySQL DB and all schemas using a single command, but that will take some time depending on how big your DB. Either way once you have addressed the corrupt table(s) you can set the innodb_force option to 0 and restart mysqld_safe.
Here is a good tutorial on recovery options for both MyISAM and InnoDB MySQL instances and covers a few options. It is far easier to point you here, than regurgitate the commands and concepts again in this answer.
Good luck and come back to ask more pointed questions once you have tried one of the options. There are probably more tutorials out there, but I have used this in Development to rebuild my Dev DB and it has plenty of information.
Look here