MySQL databases irrecoverable after system reboot - mysql

We had a power cycle in our lab, and our mysql server restarted.
It seems that the system created a tar ball of my mysql data folder, then un-tarred it. Or in some other way, tried to 'recover' and really messed things up.
Now I cant access any of my tables "table __ does not exist".
All the necessary files (frm, idb etc) seem to be there and of the correct size (quite large).
Unfortunately I am no DB admin and do not have backups.
From searching around on the webz, it seems there is nothing to do but start from scratch, but I find this quite hard to believe, that mysql could mess up 100+ tables in 1 single operation.
Any help is much appreciated.
All the permissions are correct (I think), and mysqlcheck has not helped
Here are some log exerpts from when I believe the error occured (timestamp is screwey i believe cause of the power outage)
070102 2:13:07 [Warning] The syntax '--log' is deprecated and will be removed in a future release. Please use '--general-log'/'--general-log-file' instead.
070102 2:13:07 InnoDB: The InnoDB memory heap is disabled
070102 2:13:07 InnoDB: Mutexes and rw_locks use GCC atomic builtins
070102 2:13:07 InnoDB: Compressed tables use zlib 1.2.7
070102 2:13:07 InnoDB: Initializing buffer pool, size = 2.0G
070102 2:13:07 InnoDB: Completed initialization of buffer pool
InnoDB: The first specified data file /test_results/mysql/innodata/ibdata1 did not exist:
InnoDB: a new database to be created!
070102 2:13:08 InnoDB: Setting file /test_results/mysql/innodata/ibdata1 size to 10 MB
InnoDB: Database physically writes the file full: wait...
070102 2:13:08 InnoDB: Log file /test_results/mysql/innolog/ib_logfile0 did not exist: new to be created
InnoDB: Setting log file /test_results/mysql/innolog/ib_logfile0 size to 64 MB
InnoDB: Database physically writes the file full: wait...
070102 2:13:08 InnoDB: Log file /test_results/mysql/innolog/ib_logfile1 did not exist: new to be created
InnoDB: Setting log file /test_results/mysql/innolog/ib_logfile1 size to 64 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Doublewrite buffer not found: creating new
InnoDB: Doublewrite buffer created
InnoDB: 127 rollback segment(s) active.
InnoDB: Creating foreign key constraint system tables
InnoDB: Foreign key constraint system tables created
070102 2:13:09 InnoDB: Waiting for the background threads to start
070102 2:13:10 InnoDB: 5.5.34 started; log sequence number 0
070102 2:13:10 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
070102 2:13:10 [Note] - '0.0.0.0' resolves to '0.0.0.0';
070102 2:13:10 [Note] Server socket created on IP: '0.0.0.0'.
070102 2:13:11 [Note] Event Scheduler: Loaded 0 events
070102 2:13:11 [Note] /usr/local/libexec/mysqld: ready for connections.

Here is the first tip, it sounds odd, but fire up a copy of MySQLWorkbench. Open an instance of the administrator and shutdown MySQL. Once it's shutdown, start it back up again.
Now try to access your database.

Related

mysql large tables (>4Gb) corrupt after local server restart

I hope someone can help me with this problem I'm having (I'm a newbie to Mysql and Stackoverflow, so it's probably something silly I'm missing)...
Whenever I build a large table in MySQL and then restart xampp's local MySQL server, the table then becomes unreadable under code runs that consider the 'whole' table - e.g. I can't drop it, run mySQLCheck on it or run any aggregation functions like sum / count.
This only happens after a server restart. It is fine when I build the table and then, without restarting, I run the 'whole table' codes on it I mentioned above.
Also, even after a server restart I can still do things like see the indexes in place, or sample the first x rows using Limit.
The error thrown is 'mysqld.exe has stopped working' - 'a problem caused the program to stop working correctly. Windows will close the program and notify you if a solution is available'
I stop and restart my server using the xampp start / stop control panel buttons
This only seems to happen when the table is more than 4gb. I'm using innodb
Thank you in advance!
Paul
===============================
An example of the error log result is below. This table (wikivisitstable10) was created by adding an index to the prior table. There are no crash issues with the prior table, just this one - and any similar tables >4gb:
`2016-08-25 0:22:57 6804 [Warning] option
'innodb-max-dirty-pages-pct': value 0 adjusted to 0.001
2016-08-25 0:22:57 6804 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2016-08-25 0:22:57 6804 [Note] InnoDB: The InnoDB memory heap is disabled
2016-08-25 0:22:57 6804 [Note] InnoDB: Mutexes and rw_locks use Windows interlocked functions 2016-08-25 0:22:57 6804
[Note] InnoDB: Memory barrier is not used 2016-08-25 0:22:57 6804
[Note] InnoDB: Compressed tables use zlib 1.2.3
2016-08-25 0:22:57 6804 [Note] InnoDB: Using generic crc32 instructions
2016-08-25 0:22:57 6804 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2016-08-25 0:22:57 6804 [Note] InnoDB: Completed initialization of
buffer pool
2016-08-25 0:22:57 6804 [Note] InnoDB: Highest supported
file format is Barracuda.
2016-08-25 0:22:57 6804 [Note] InnoDB: The log sequence numbers 400931068139 and 400931068139 in ibdata files do not match the log sequence number 400931068159 in the ib_logfiles!
2016-08-25 0:22:57 6804 [Note] InnoDB: Database was not shutdown
normally!
2016-08-25 0:22:57 6804 [Note] InnoDB: Starting crash
recovery.
2016-08-25 0:22:57 6804 [Note] InnoDB: Reading tablespace
information from the .ibd files...
2016-08-25 0:22:58 6804 [Note] InnoDB: Restoring possible half-written data pages
2016-08-25 0:22:58 6804 [Note] InnoDB: from the doublewrite buffer...
2016-08-25 00:22:59 1a94 InnoDB: Error: table 'revil2/page'
InnoDB: in InnoDB data dictionary has tablespace id 180, InnoDB: but a tablespace with that id does not exist. There is InnoDB: a tablespace of name revil2/page and id 181, though. Have InnoDB: you deleted or moved .ibd files?
InnoDB: Please refer to InnoDB: DATADICT LINK (REMOVED)
InnoDB: for how to resolve the issue.
2016-08-25 0:22:59 6804 [ERROR] InnoDB: Table revil2/wikipages7 in the InnoDB data dictionary has tablespace id 175, but tablespace with that id or name does not exist. Have you deleted or moved .ibd files? This may also be a table created with CREATE TEMPORARY TABLE whose .ibd and .frm files MySQL automatically removed, but the table still exists in the InnoDB internal data dictionary.
InnoDB: Please refer to InnoDB: DATA DICT LINK (REMOVED)
InnoDB: for how to resolve the issue.
2016-08-25 0:22:59 6804 [Note] InnoDB: 128 rollback segment(s) are active.
2016-08-25 0:22:59 6804 [Note] InnoDB: Waiting for purge to start
2016-08-25 0:22:59 6804 [Note] InnoDB: Percona XtraDB (PERCONA
WEBSITE (REMOVED)) 5.6.28-76.1 started; log sequence number
400931068159
2016-08-25 0:22:59 5820 [Note] InnoDB: Dumping buffer pool(s) not yet started
2016-08-25 00:22:59 16bc InnoDB: Loading buffer pool(s) from .\\ib_buffer_pool
2016-08-25 0:22:59 6804 [Note] Plugin 'FEEDBACK' is disabled.
2016-08-25 0:22:59 6804 [Note] Server socket created on IP: '::'.
2016-08-25 00:22:59 16bc InnoDB: Buffer pool(s) load completed at 160825 0:22:59
2016-08-25 0:23:00 6804 [Note] c:\xampp\mysql\bin\mysqld.exe: ready for connections. Version: '10.1.13-MariaDB' socket: '' port: 3306 mariadb.org binary distribution
InnoDB: Error: trying to access page number 1794 in space 299, InnoDB: space name revil2/wikivisitstable10, InnoDB: which is outside the tablespace bounds.
InnoDB: Byte offset 0, len 16384, i/o type 10.
InnoDB: If you get this error at mysqld startup, please check that InnoDB: your my.cnf matches the ibdata files that you have in the InnoDB: MySQL server.
2016-08-25 00:23:46 1388 InnoDB: Assertion failure in thread 5000 in file fil0fil.cc line 5866
InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed
bug report to HTTP BUGS. MYSQL. COM InnoDB: If you get repeated
assertion failures or crashes, even InnoDB: immediately after the
mysqld startup, there may be InnoDB: corruption in the InnoDB
tablespace.
Please refer to InnoDB: FORCING INNODB RECOVERY LINK (REMOVED)
InnoDB: about forcing recovery. 160825 0:23:46
[ERROR] mysqld got exception 0x80000003 ; This could be because you hit a
bug. It is also possible that this binary or one of the libraries it
was linked against is corrupt, improperly built, or misconfigured.
This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, something
is definitely wrong and this may fail.
Server version: 10.1.13-MariaDB
key_buffer_size=268435456
read_buffer_size=268435456
max_used_connections=1
max_threads=1001
thread_count=1
It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 788380 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x0x1c172e20 Attempting backtrace. You can use the
following information to find out where mysqld died. If you see no
messages after this, something went terribly wrong...
mysqld.exe!my_parameter_handler() mysqld.exe!my_mb_ctype_mb()
mysqld.exe!?get_ctx#MDL_ticket##QBEPAVMDL_context##XZ()
mysqld.exe!?get_ctx#MDL_ticket##QBEPAVMDL_context##XZ()
mysqld.exe!?functype#Item_func_dyncol_create##UBE?AW4Functype#Item_func##XZ()
mysqld.exe!?get_ctx#MDL_ticket##QBEPAVMDL_context##XZ()
mysqld.exe!?get_ctx#MDL_ticket##QBEPAVMDL_context##XZ()
mysqld.exe!?get_trg_event_map#Update_rows_log_event##UAEEXZ()
mysqld.exe!?ha_check#handler##QAEHPAVTHD##PAUst_ha_check_opt###Z()
mysqld.exe!??_9handler##$BBAE#AE()
mysqld.exe!?execute#Sql_cmd_check_table##UAE_NPAVTHD###Z()
mysqld.exe!?mysql_execute_command##YAHPAVTHD###Z()
mysqld.exe!?mysql_parse##YAXPAVTHD##PADIPAVParser_state###Z()
mysqld.exe!?dispatch_command##YA_NW4enum_server_command##PAVTHD##PADI#Z()
mysqld.exe!?do_command##YA_NPAVTHD###Z()
mysqld.exe!?threadpool_process_request##YAHPAVTHD###Z()
mysqld.exe!?tp_end##YAXXZ() KERNEL32.DLL!SetUserGeoID()
ntdll.dll!TpSimpleTryPost() ntdll.dll!EtwNotificationRegister()
KERNEL32.DLL!BaseThreadInitThunk()
ntdll.dll!RtlUnicodeStringToInteger()
ntdll.dll!RtlUnicodeStringToInteger()
Trying to get some variables. Some pointers may be invalid and cause
the dump to abort. Query (0x1c232750): CHECK TABLE
`wikivisitstable10` Connection ID (thread ID): 2 Status: NOT_KILLED`
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on

Error Starting MySQL after Crash

I was merrily installing wordpress with MAMP pro when my Mac crashed. Now I cannot start MySql. Most stuff I read online tells me to throw some terminal commands at it, trying to kill the process.
killall -9 mysqld
But terminal report "No matching processes belonging to you were found"
So I have had a look in the log and can see this from the time of the first restart attempt after the crash
150506 21:11:33 mysqld_safe Starting mysqld daemon with databases from /Library/Application Support/appsolute/MAMP PRO/db/mysql
150506 21:11:33 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
150506 21:11:33 [Warning] Setting lower_case_table_names=2 because file system for /Library/Application Support/appsolute/MAMP PRO/db/mysql/ is case insensitive
150506 21:11:33 [Note] Plugin 'FEDERATED' is disabled.
150506 21:11:33 InnoDB: The InnoDB memory heap is disabled
150506 21:11:33 InnoDB: Mutexes and rw_locks use GCC atomic builtins
150506 21:11:33 InnoDB: Compressed tables use zlib 1.2.3
150506 21:11:33 InnoDB: Initializing buffer pool, size = 128.0M
150506 21:11:33 InnoDB: Completed initialization of buffer pool
InnoDB: The first specified data file ./ibdata1 did not exist:
InnoDB: a new database to be created!
150506 21:11:33 InnoDB: Setting file ./ibdata1 size to 10 MB
InnoDB: Database physically writes the file full: wait...
150506 21:11:33 InnoDB: Log file ./ib_logfile0 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile0 size to 5 MB
InnoDB: Database physically writes the file full: wait...
150506 21:11:33 InnoDB: Log file ./ib_logfile1 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile1 size to 5 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Doublewrite buffer not found: creating new
InnoDB: Doublewrite buffer created
InnoDB: 127 rollback segment(s) active.
InnoDB: Creating foreign key constraint system tables
InnoDB: Foreign key constraint system tables created
150506 21:11:34 InnoDB: Waiting for the background threads to start
150506 21:11:35 InnoDB: 5.5.42 started; log sequence number 0
150506 21:11:35 [Note] Event Scheduler: Loaded 0 events
150506 21:11:35 [Note] /Applications/MAMP/Library/bin/mysqld: ready for connections.
Version: '5.5.42' socket: '/Applications/MAMP/tmp/mysql/mysql.sock' port: 0 Source distribution
150506 21:16:44 [Note] /Applications/MAMP/Library/bin/mysqld: Normal shutdown
So Table Space size error.. I am very much a front end guy and have no idea where to begin with this...??
Help please, I really can't afford to loose the last week of work.
Thanks you
As you pointed out InnoDB tablespace is corrupt, so InnoDB fails to start.
InnoDB can tolerate some errors if you start it with option innodb_force_recovery. Try values from 1 to 6 until MySQL starts.
If you're lucky and MySQL starts with innodb_force_recovery then dump all tables into an sql file with mysqldump and re-create MySQL database from scratch. I.e. move existing files to some safe place, start mysql_install_db, change file permissions and start MySQL. Then load the database back.
If you're less lucky and MySQL doesn't start with innodb_force_recovery=6 then check how to Recover Corrupt MySQL Database .
And to avoid this experience in future use XtraBackup for Mac OS to take backups.
Please see her:
MAMP PRO crashes; MySQL will not start on reboot
Thanks for the heads up with Xtrabackup for MacOS, one to keep an eye on that.

XAMPP MySQL suddenly doesnt work anymore

It was working fine for month. But today i wanna start working and cant connect to MySQL anymore.
Im using XAMPP 1.7.7 on Windows 7
XAMPP Control Panel says MySQL is "Running" and shows "Mysql started [Port 3306]"
But when i go to localhost and click status, it shows "MySQL database Deactivated"
When i go to phpmyadmin it shows: "2002 - The server is not responding (or the local MySQL server's socket is not correctly configured)"
Also mysql_error.log file at C:\xampp\mysql\data shows no errors?
140312 15:02:24 [Note] Plugin 'FEDERATED' is disabled.
140312 15:02:24 InnoDB: The InnoDB memory heap is disabled
140312 15:02:24 InnoDB: Mutexes and rw_locks use Windows interlocked functions
140312 15:02:24 InnoDB: Compressed tables use zlib 1.2.3
140312 15:02:24 InnoDB: Initializing buffer pool, size = 16.0M
140312 15:02:24 InnoDB: Completed initialization of buffer pool
140312 15:02:24 InnoDB: highest supported file format is Barracuda.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
140312 15:02:24 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
140312 15:02:24 InnoDB: Waiting for the background threads to start
140312 15:02:25 InnoDB: 1.1.8 started; log sequence number 3612444192
140312 15:02:25 [Note] Event Scheduler: Loaded 0 events
140312 15:02:25 [Note] mysql\bin\mysqld.exe: ready for connections.
Version: '5.5.16' socket: '' port: 3306 MySQL Community Server (GPL)
Any ideas? :(
The question is old, but I have just had the same issue so I will write my solution.
It could be a problem with Xampp.
Try to uninstall and install again.
But before, you can try also something that worked for me, namely run the file:
C:\xampp\mysql\resetroot.bat

MySQL service hung in "Starting" mode

I am trying to install multiple instances of MySQL on a Windows Server 2003 using this link.
First instance which is on port 6699 get configured and working successfully. But when I try to install another instance on default port (3306) MySQL service gets hung on "Starting" status. This is the log
140210 17:13:20 [Note] Plugin 'FEDERATED' is disabled.
140210 17:13:20 InnoDB: The InnoDB memory heap is disabled
140210 17:13:20 InnoDB: Mutexes and rw_locks use Windows interlocked functions
140210 17:13:20 InnoDB: Compressed tables use zlib 1.2.3
140210 17:13:20 InnoDB: Initializing buffer pool, size = 128.0M
140210 17:13:20 InnoDB: Completed initialization of buffer pool
InnoDB: The first specified data file .\ibdata1 did not exist:
InnoDB: a new database to be created!
140210 17:13:21 InnoDB: Setting file .\ibdata1 size to 10 MB
InnoDB: Database physically writes the file full: wait...
140210 17:13:21 InnoDB: Log file .\ib_logfile0 did not exist: new to be created
InnoDB: Setting log file .\ib_logfile0 size to 5 MB
InnoDB: Database physically writes the file full: wait...
140210 17:13:23 InnoDB: Log file .\ib_logfile1 did not exist: new to be created
InnoDB: Setting log file .\ib_logfile1 size to 5 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Doublewrite buffer not found: creating new
InnoDB: Doublewrite buffer created
InnoDB: 127 rollback segment(s) active.
InnoDB: Creating foreign key constraint system tables
InnoDB: Foreign key constraint system tables created
140210 17:13:26 InnoDB: Waiting for the background threads to start
140210 17:13:27 InnoDB: 1.1.8 started; log sequence number 0
140210 17:13:27 [Note] Server hostname (bind-address): '0.0.0.0'; port: 6699
140210 17:13:27 [Note] - '0.0.0.0' resolves to '0.0.0.0';
140210 17:13:27 [Note] Server socket created on IP: '0.0.0.0'.
140210 17:13:27 [Note] Event Scheduler: Loaded 0 events
140210 17:13:27 [Note] C:\Program Files\MySQL\MySQL Server 5.5\bin\mysqld.exe: ready for connections.
Version: '5.5.25' socket: '' port: 6699 MySQL Community Server (GPL)
140210 17:42:18 [Note] Plugin 'FEDERATED' is disabled.
140210 17:42:18 InnoDB: The InnoDB memory heap is disabled
140210 17:42:18 InnoDB: Mutexes and rw_locks use Windows interlocked functions
140210 17:42:18 InnoDB: Compressed tables use zlib 1.2.3
140210 17:42:19 InnoDB: Initializing buffer pool, size = 128.0M
140210 17:42:19 InnoDB: Completed initialization of buffer pool
140210 17:42:19 InnoDB: highest supported file format is Barracuda.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
Observe line 140210 17:42:19 InnoDB: Database was not shut down normally!
InnoDB: The log sequence number in ibdata files does not match
Double check your configuration!
Make a binay backup or entire /var/lib/mysql.
If you don't need most recent data, delete the log files and try to start.
If you need your data, try the followling:
Try innodb_force_recovery = 1 to 6 all six options in an increasing order.
Assume the worst -- innodb disk image corrupted.
If you are lucky you can salvage most of your data, perhaps mysql starts in one of recovery modes, run checksum on each table, see what tables are still ok.
If mysql refuses to start up to certain recovery mode N and crashes on startup with recovery mode N, then go looking for those daily/weekly backups I hope you system was configured to take.

MySQL ...Starting crash recovery

I have searched high and low for a solution to this problem. My MySql gives me the following readout:
121231 20:41:05 [Note] Plugin 'FEDERATED' is disabled.
121231 20:41:05 InnoDB: The InnoDB memory heap is disabled
121231 20:41:05 InnoDB: Mutexes and rw_locks use Windows interlocked functions
121231 20:41:05 InnoDB: Compressed tables use zlib 1.2.3
121231 20:41:05 InnoDB: Initializing buffer pool, size = 512.0M
121231 20:41:05 InnoDB: Completed initialization of buffer pool
121231 20:41:05 InnoDB: highest supported file format is Barracuda.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
121231 20:41:05 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
121231 20:41:06 InnoDB: Waiting for the background threads to start
121231 20:41:07 InnoDB: 1.1.8 started; log sequence number 124716458
121231 20:41:07 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
121231 20:41:07 [Note] - '0.0.0.0' resolves to '0.0.0.0';
121231 20:41:07 [Note] Server socket created on IP: '0.0.0.0'.
121231 20:41:09 [Note] Event Scheduler: Loaded 0 events
121231 20:41:09 [Note] c:\xampp\mysql\bin\mysqld.exe: ready for connections.
Version: '5.5.27' socket: '' port: 3306 MySQL Community Server (GPL)
Everytime it starts. I've tried deleting files, dropping the database, running Magento's Repair Database Tool, starting all over with a new database but no matter what I get this error.
It makes me uncomfortable because I'm not sure something might get corrupted or IS corrupted. I'm using Windows Vista, and Xampp but I also was using nginX with the same thing showing.
I just recreated a brand new database and now after shutdown (which I have to do sometimes) the error is showing up again! Is this a normal thing or is there somehing wrong?
Thanks
As you can see from the log InnoDB starts crash recovery:
InnoDB: Starting crash recovery.
The reason for that is MySQL shut down wasn't clean. Why? May be MySQL has been exiting too long and OS kills the process (if you reboot the server). Or MySQL crashes due to a bug.