I am using MariaDB with MyRocks engine. After short time, I try to insert bunch of data into some tables (tables could be InnoDB and MyRocks engine), my disk grows very fast. I check MariaDB data directory and I see this:
drwxr-x--x 2 mysql mysql 12K Jun 26 12:07 #rocksdb
drwxr-xr-x 7 mysql mysql 4.0K Jun 26 12:07 .
drwxr-xr-x 6 root root 4.0K Jun 21 10:22 ..
-rw-rw---- 1 mysql mysql 1.1G Jun 25 21:39 1.000001
-rw-rw---- 1 mysql mysql 1.1G Jun 25 21:40 1.000002
-rw-rw---- 1 mysql mysql 1.6G Jun 25 21:44 1.000003
-rw-rw---- 1 mysql mysql 1.1G Jun 25 21:48 1.000004
-rw-rw---- 1 mysql mysql 1.1G Jun 25 21:50 1.000005
-rw-rw---- 1 mysql mysql 218M Jun 26 06:53 1.000006
-rw-rw---- 1 mysql mysql 357 Jun 26 06:56 1.000007
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:06 1.000008
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:09 1.000009
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:17 1.000010
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:19 1.000011
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:22 1.000012
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:24 1.000013
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:27 1.000014
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:29 1.000015
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:32 1.000016
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:34 1.000017
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:37 1.000018
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:40 1.000019
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:42 1.000020
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:45 1.000021
-rw-rw---- 1 mysql mysql 1.1G Jun 26 07:51 1.000022
-rw-rw---- 1 mysql mysql 1.1G Jun 26 08:00 1.000023
-rw-rw---- 1 mysql mysql 1.1G Jun 26 08:07 1.000024
-rw-rw---- 1 mysql mysql 1.1G Jun 26 08:15 1.000025
...
I see many files whose name contains only numbers. I don't know those files were created by MyRocks or MySQL, and can I delete them?
A guess: You configured the following, assuming it turned on binlogging:
log_bin = 1
and those are binlogs starting with the '1'.
A new file every few minutes implies a huge amount of writes (such as the inserts you mention).
If you don't need binlogging and change to this and restart, then the files should stop being created:
log_bin = OFF
The files (1.*) can be removed.
If they don't stop, then my guess is wrong.
If you do need binlogging (for replication and/or point-in-time backups, then we need to investigate the huge rapidity of them.
I'll bet you did a restart about Jun 26 06:53.
Rocksdb is probably not relevant to the question. (But I could be wrong.)
Related
When i try to start mysql, i get the following error:
170228 15:14:38 mysqld started
InnoDB: Error: log file ./ib_logfile2 is of different size 0 0 bytes
InnoDB: than specified in the .cnf file 0 5242880 bytes!
170228 15:14:38 Can't init databases
170228 15:14:38 Aborting
170228 15:14:38 InnoDB: Warning: shutting down a not properly started
InnoDB: or created database!
170228 15:14:38 /usr/local/mysql/bin/mysqld: Shutdown Complete
170228 15:14:38 mysqld ended
When i delete this log file, as i understand i should do in order to fix it, i get the follofwing error:
170228 15:21:39 mysqld started
170228 15:21:39 InnoDB: Log file ./ib_logfile2 did not exist: new to be created
170228 15:21:39 Can't init databases
170228 15:21:39 Aborting
170228 15:21:39 InnoDB: Warning: shutting down a not properly started
InnoDB: or created database!
170228 15:21:39 /usr/local/mysql/bin/mysqld: Shutdown Complete
170228 15:21:39 mysqld ended
After that i actually return the original situation, where "ib_logfile2 is of different size 0 0 bytes".
How can i fix this issue?
Edit:
Here is my DB data folder:
drwxr-xr-x 2 mysql mysql 132 Feb 27 15:38 forum_test
-rw-r--r-- 1 mysql mysql 10737418240 Feb 27 15:41 ibdata1
-rw-r--r-- 1 mysql mysql 10737418240 Feb 27 15:33 ibdata2
-rw-r--r-- 1 mysql mysql 10737418240 Feb 27 15:31 ibdata3
-rw-r--r-- 1 mysql mysql 10737418240 Feb 27 15:37 ibdata4
-rw-r--r-- 1 mysql mysql 10737418240 Feb 27 15:35 ibdata5
-rw-rw---- 1 mysql mysql 5242880 Feb 27 19:03 ib_logfile0
-rw-rw---- 1 mysql mysql 5242880 Feb 27 16:20 ib_logfile1
-rw-rw---- 1 mysql mysql 0 Feb 28 15:21 ib_logfile2
-rw-r--r-- 1 mysql mysql 21076 Feb 27 18:58 my.cnf_abc
drwxr-xr-x 2 mysql mysql 4096 Feb 27 15:37 mysql
drwxr-xr-x 2 mysql mysql 102 Feb 27 15:39 test
drwxr-xr-x 2 mysql mysql 16384 Feb 27 15:39 abc
drwxr-xr-x 2 mysql mysql 4096 Feb 27 15:37 abcAdv
drwxr-xr-x 2 mysql mysql 36864 Feb 27 15:39 abcExcel
drwxr-xr-x 2 mysql mysql 81 Feb 27 15:37 abcHul
drwxr-xr-x 2 mysql mysql 4096 Feb 27 15:39 abcMesh
drwxr-xr-x 2 mysql mysql 4096 Feb 27 15:39 abcMilt
-rw-rw---- 1 mysql mysql 4603 Feb 28 15:21 abc-restore180.log
drwxr-xr-x 2 mysql mysql 4096 Feb 27 15:39 abcSold
drwxr-xr-x 2 mysql mysql 137 Feb 27 15:39 abcVod
I don't know if this will help you cause I use Windows and not Ubuntu like almost every other user here. So anyway, go to the C:/xampp/mysql/data/ directory and change "ibdata1" to "ibdata1---" or rather add "---" to "ibdata1" and restart xampp and mysql
It will create a new ibdata1 file when you run mysql and start working normally.
Hope this helps :)
For some reason, I can't start MySql.
$ sudo /etc/init.d/mysql start
[sudo] password for abigail:
Starting MySQL.............................................[FAILED]................................................................................
What caused this?
This is what in my /var/lib/mysql
$ ll -h /var/lib/mysql
total 1005M
-rw-rw----. 1 mysql mysql 16K Mar 28 11:08 aria_log.00000001
-rw-rw----. 1 mysql mysql 52 Mar 28 11:08 aria_log_control
-rw-rw----. 1 mysql mysql 908M Mar 31 20:10 ibdata1
-rw-rw----. 1 mysql mysql 48M Mar 31 20:09 ib_logfile0
-rw-rw----. 1 mysql mysql 48M Mar 31 20:10 ib_logfile1
-rw-rw----. 1 mysql mysql 5 Mar 28 11:09 localhost.pid
-rw-rw----. 1 mysql mysql 0 Oct 29 23:04 multi-master.info
drwx--x--x. 2 mysql mysql 4.0K Oct 29 23:01 mysql
drwx------. 2 mysql mysql 19 Oct 29 23:01 performance_schema
-rw-rw----. 1 mysql mysql 24K Mar 28 11:09 tc.log
drwx------. 2 mysql mysql 4.0K Nov 3 22:10 test
I think I might have to reinstall mysql.
my.cnf is supposed to be in /etc/my.cnf.
Check the start log for the error message, like in /var/log/mysqld.log.
I'm pretty lost right now.
The RAID controller auf my server broke and all attached hard drives were corrupted. Luckily I copied the plain files off the server right before the crash (phew).
I set up the new server with Ubuntu 12.04 and a fresh mysql installation. I copied the backup off a server directly to the new one, without saving them to a windows machine (to prevent privileges errors).
My back up folder looks like this:
drwx------ 15 jensma jensma 4.0K Dec 12 18:09 .
drwxr-xr-x 76 jensma jensma 4.0K Nov 22 12:00 ..
drwx------ 2 mysql mysql 4.0K Nov 21 02:50 gamefilesenter code here
drwx------ 2 mysql mysql 4.0K Aug 5 00:17 circuitboard
-rw-r--r-- 1 mysql mysql 0 Oct 25 07:49 debian-5.5.flag
-rw-rw---- 1 mysql mysql 26M Dec 12 02:22 ibdata1
-rw-rw---- 1 mysql mysql 5.0M Dec 12 18:09 ib_logfile0
-rw-rw---- 1 mysql mysql 5.0M Dec 12 02:22 ib_logfile1
drwx------ 2 mysql mysql 4.0K Oct 25 07:49 mysql
-rw-rw---- 1 mysql mysql 6 Oct 25 07:49 mysql_upgrade_info
drwx------ 2 mysql mysql 4.0K Oct 25 07:49 performance_schema
drwx------ 2 mysql mysql 4.0K May 25 2013 phpmyadmin
drwx------ 2 mysql mysql 4.0K Jul 4 18:29 test
drwx------ 2 mysql mysql 4.0K May 19 2013 usr_web10_5
A folder contains data like this:
drwx------ 2 mysql mysql 4096 Aug 5 00:17 .
drwx------ 15 jensma jensma 4096 Dec 12 18:09 ..
-rw-rw---- 1 mysql mysql 65 Jul 10 15:04 db.opt
-rw-rw---- 1 mysql mysql 8620 Aug 1 23:38 stat.frm
-rw-rw---- 1 mysql mysql 8700 Aug 5 00:17 user.frm
I just copied the whole content to /var/lib/mysql and ran
sudo /usr/sbin/mysqld –innodb_log_file_size=5242880 –innodb_force_recovery=6
Well, the command kinda works - the shell shows up after half a minute. But the database is still empty :/
I'm pretty now to this. I do have full admin access to the server.
Any help is appreciated, thanks!
I see today that
show databases
displays an unknown database:
(msl_ampcs_v4_0_0_2g)> show databases;
+----------------------+
| Database |
+----------------------+
| information_schema |
| #mysql50#.ssh |
| global_lad
Normally when I see something like that it is due to a bad file in /var/lib/mysql:
-rw-r--r-- 1 mysql mysql 1663 Mar 17 2011 /var/lib/mysql/ca-cert.pem
drwx------ 2 mysql mysql 4096 Jun 18 15:00 /var/lib/mysql/global_lad
drwx------ 2 mysql mysql 4096 Jun 18 15:00 /var/lib/mysql/global_lad_unit_test
-rw-rw---- 1 mysql mysql 3231711232 Aug 22 07:20 /var/lib/mysql/ibdata1
-rw-rw---- 1 mysql mysql 5242880 Aug 22 07:20 /var/lib/mysql/ib_logfile0
-rw-rw---- 1 mysql mysql 5242880 Aug 21 16:25 /var/lib/mysql/ib_logfile1
-rw-rw---- 1 mysql mysql 5 Feb 21 2013 /var/lib/mysql/istsdb2.jpl.nasa.gov.pid
-rw-rw---- 1 mysql mysql 4596466 Oct 15 2010 /var/lib/mysql/istsdb2-slow.log
drwx------ 2 mysql mysql 4096 Jul 13 2011 /var/lib/mysql/msl_ampcs_v2_0_0
drwx------ 2 mysql mysql 4096 Jul 21 2012 /var/lib/mysql/msl_ampcs_v3_0_0
drwx------ 2 mysql mysql 4096 Jun 20 02:55 /var/lib/mysql/msl_ampcs_v4_0_0
drwx------ 2 mysql mysql 4096 Sep 5 19:45 /var/lib/mysql/msl_ampcs_v4_0_0_2g
drwx------ 2 mysql mysql 4096 Sep 30 2010 /var/lib/mysql/msl_gdsdb_v5_0
drwx------ 2 mysql mysql 4096 Jan 19 2011 /var/lib/mysql/msl_gdsdb_v5_5
drwx------ 2 mysql mysql 4096 Nov 21 2011 /var/lib/mysql/msl_gdsdb_v5_6
drwx------ 2 mysql root 4096 Mar 23 2010 /var/lib/mysql/mysql
srwxrwxrwx 1 mysql mysql 0 Feb 21 2013 /var/lib/mysql/mysql.sock
-rw------- 1 root root 841 Jun 10 2010 /var/lib/mysql/nohup.out
drwx------ 2 mysql mysql 4096 Feb 5 2013 /var/lib/mysql/pdpp_automation
-rw-r--r-- 1 mysql mysql 1310 Mar 17 2011 /var/lib/mysql/server-cert.pem
-rw-r--r-- 1 mysql mysql 1679 Mar 17 2011 /var/lib/mysql/server-key.pem
But nothing like that is present. The nohup shouldn't be there. But
#mysql50#.ssh
does not appear.
I have drupal website, the database is about 10GB. I was checking the datadir of my drupal database and found that some files are .gz!! why is this? and does it affect performance??
the website is working fine and i can use those tables normally although there is no corresponding file for.gz (ie. I have content_type_company.MYD.gz in the data directory but there is no content_type_company.MYD)
the list is something like this:
ls -lhS *.gz
-rw-rw---- 1 mysql mysql 3.4M 2011-04-18 06:52 content_field_headline_image.MYI.gz
-rw-rw---- 1 mysql mysql 379K 2011-04-14 15:50 print_node_conf.MYD.gz
-rw-rw---- 1 mysql mysql 105K 2011-04-18 06:56 content_type_company.MYD.gz
-rw-rw---- 1 mysql mysql 22K 2011-04-14 15:50 nodewords.MYD.gz
-rw-rw---- 1 mysql mysql 18K 2011-04-18 06:59 content_field_company_fax.MYD.gz
-rw-rw---- 1 mysql mysql 18K 2011-04-18 06:59 content_field_company_email.MYI.gz
-rw-rw---- 1 mysql mysql 4.7K 2011-04-17 21:55 content_type_editor_choice.MYI.gz
-rw-rw---- 1 mysql mysql 4.2K 2011-02-01 09:15 content_node_field_instance.MYD.gz
-rw-rw---- 1 mysql mysql 2.7K 2011-01-31 14:03 content_node_field.MYD.gz
-rw-rw---- 1 mysql mysql 1.9K 2011-04-18 01:31 content_type_press_review.MYI.gz
-rw-rw---- 1 mysql mysql 1.4K 2011-04-18 06:59 users.MYI.gz
-rw-rw---- 1 mysql mysql 1.1K 2011-02-02 12:25 batch.MYD.gz
-rw-rw---- 1 mysql mysql 1.1K 2011-01-19 12:35 node_access.MYI.gz
-rw-rw---- 1 mysql mysql 801 2010-09-06 09:49 menu_custom.MYD.gz
-rw-rw---- 1 mysql mysql 679 2011-04-17 12:12 scheduler.MYI.gz
-rw-rw---- 1 mysql mysql 671 2011-01-12 11:24 users.frm.gz
-rw-rw---- 1 mysql mysql 659 2011-04-14 22:53 video_rendering.MYI.gz
-rw-rw---- 1 mysql mysql 570 2011-04-17 14:39 content_type_slideshow.MYD.gz
-rw-rw---- 1 mysql mysql 533 2011-02-02 09:46 permission.MYI.gz
-rw-rw---- 1 mysql mysql 471 2011-01-30 12:53 panels_display.MYD.gz
-rw-rw---- 1 mysql mysql 412 2010-09-07 14:41 content_field_slideshow_image.frm.gz
-rw-rw---- 1 mysql mysql 372 2011-01-12 11:24 date_formats.MYD.gz
-rw-rw---- 1 mysql mysql 355 2010-07-26 15:51 contact.frm.gz
-rw-rw---- 1 mysql mysql 351 2010-07-26 16:15 sessions.frm.gz
-rw-rw---- 1 mysql mysql 349 2011-03-28 18:02 cache_update.frm.gz
-rw-rw---- 1 mysql mysql 345 2011-03-28 18:03 ctools_object_cache.frm.gz
-rw-rw---- 1 mysql mysql 326 2010-07-26 15:54 print_mail_page_counter.frm.gz
-rw-rw---- 1 mysql mysql 317 2010-07-26 15:54 search_index.frm.gz
-rw-rw---- 1 mysql mysql 310 2010-07-26 15:51 i18n_strings.frm.gz
-rw-rw---- 1 mysql mysql 292 2010-07-26 15:49 authmap.frm.gz
-rw-rw---- 1 mysql mysql 291 2011-01-12 11:24 scheduler.frm.gz
-rw-rw---- 1 mysql mysql 286 2010-07-26 15:54 permission.frm.gz
-rw-rw---- 1 mysql mysql 275 2010-07-26 15:49 block_class.frm.gz
-rw-rw---- 1 mysql mysql 272 2010-07-26 15:51 captcha_points.frm.gz
-rw-rw---- 1 mysql mysql 269 2010-07-26 15:54 page_manager_weights.frm.gz
-rw-rw---- 1 mysql mysql 251 2011-03-28 18:03 imagecache_preset.frm.gz
-rw-rw---- 1 mysql mysql 249 2010-11-24 13:19 content_type_press_review.frm.gz
-rw-rw---- 1 mysql mysql 248 2011-01-26 09:21 aggregator_category_feed.frm.gz
-rw-rw---- 1 mysql mysql 243 2010-07-26 15:54 role.frm.gz
-rw-rw---- 1 mysql mysql 240 2010-07-26 16:15 variable.frm.gz
-rw-rw---- 1 mysql mysql 202 2010-08-09 13:20 stylizer.MYI.gz
-rw-rw---- 1 mysql mysql 192 2011-04-05 07:50 vocabulary_node_types.MYD.gz
-rw-rw---- 1 mysql mysql 152 2010-07-26 16:15 taxonomy_manager_merge.MYI.gz
-rw-rw---- 1 mysql mysql 78 2011-04-14 03:06 video_rendering.MYD.gz
-rw-rw---- 1 mysql mysql 58 2011-02-01 17:40 content_type_video_gallery.MYD.gz
The files are compressed to save space, of course, and gzip works like that - when a file called 'x' is gzipped, it's replaced by 'x.gz' the original file ''becomes'' the compressed file.
If the database works as expected, I wouldn't worry about it.
The size of gzipped files is much smaller than Your database size. Nothing to worry.