Got this from some mysql queries, puzzled since error 122 is usually a 'out of space' error but there's plenty of space left on the server... any ideas?
The answer: for some reason Mysql had its tmp tables on the /tmp partition which was limited to 100M, and was filled up by eaccelerator cache to 100M even though eaccel is limited to 16M of usage. Very weird, but I just moved eaccel cache elsewhere and problem solved.
Error 122 often indicates a "Disk over quota" error. Is it possible disk quotas exist on the server?
Try to turn off the disk quota using the quotaoff command.
Using the -a flag will turn off all file system quotas.
quotaoff -a
are you using innodb tables? if so, you might not have auto-grow turned on and inno can't expand the table space any more.
if these are myisam tables and it only happens on specific tables, i would suspect corruption. do a REPAIR on the tables in question.
I resolve this issue by increasing my disk size.
try df -h to check whether there are enough disk space on your server.
Related
I have setup with Linux, Debian Jessie with Mysql 5.7.13 installed.
I have set following settings in
my.cnf: default_storage_engine= innodb, innodb_buffer_pool_size= 44G
When I start MySQL I manually set max_connections with SET GLOBAL max_connections = 1000;
Then I trigger my loadtest that sends a lot of traffic to the DB server which mostly consists of slow/bad queries.
The result I expected was that I would reach close to 1000 connections but somehow MySQL limits it to 462 connections and I can not find the setting that is responsible for this limit. We are not even close to maxing out the CPU or Memory.
If you have any idea or could point me in a direction where you think the error might be it would be really helpful.
What loadtest did you use? Are you sure that it can utilize about thousands of connections?
You may maxing out your server resources in the disk IO area, especially if you're talking about lot of slow/bad queries. Did you check for disk utilization on your server?
Even if your InnoDB pool size is large your DB still need to read your DB to the cache first, and if your entire DB is large it will not help you.
I can recommend you to perform such a test once more time and track your disk performance during loadtest using iostat or iotop utility.
Look here for more examples of the server performance troubleshooting.
I found the issue, it was du to limitation of Apache server, there is a "hidden" setting inside /etc/apache2/mods-enabled/mpm_prefork.conf which will overwrite setting inside /etc/apache2/apache2.conf
Thank you!
i will really appreciate if someone help me with this.
I have spend like 8hours googling around and found no solution to problem.
I have MySQL server version 5.7.7 on Windows server 2008 R2
Table engine is innodb
innodb_file_per_table = 1
I get error "Table is full" when table reaches 4Gb.
MySQL documentation sais that there is actualy only one limit on table size, filesystem.
(http://dev.mysql.com/doc/refman/5.7/en/table-size-limit.html)
HDD where are data stored uses NTFS, just to be sure i created 5Gb file without problems. And sure there is more than 10Gb of free space.
I understand setting "innodb_data_file_path" is irrelevant if "innodb_file_per_table" is enabled, but i tried to set it. No differences.
I have tried to do clean install of mysql. Same result.
EDIT
Guy that installed MySQL server before me accidentally installed 32bit version. Migration to 64bit mysql solved that problem
About the only way for 4GB to be a file limit is if you have a 32-bit version of MySQL. Also check for 32-bit Operating system. (Moved from comment, where it was verified.)
i am also not sure but read this it may help you.
http://jeremy.zawodny.com/blog/archives/000796.html
one more thing one guy had same problem.he had made changes to
NNODB settings for the innodb_log_file_size and innodb_log_buffer_size!changes were :
1) shutdown mysql
2) cd /var/lib/mysql
3) mkdir oldIblog
4) mv ib_logfile* oldIblog
5) edit /etc/my.cnf find the line innodb_log_file_size= and increase it to an appropriate value (he went to 1000MB as he was dealing with a very large dataset... 250million rows in one table). If you are not sure I suggest doubling the number every time you get a table is full error. he set innodb_log_buffer_size to 1/4 of the size of his log file and the problems went away.
I didnt find solution to this, i have no idea why mysql is unable to create more than 4Gb table.
As a workaround i moved only this table back to ibdata by setting "innodb_file_per_table" back to 0 and recreated that table.
Interesting is that even ibdata1 reported table is full when it reached 4Gb, even without setting max and enabled autoexpand.
So i created ibdata2 and let it autoexpand, now i am able to write new data to that table.
I have a problem with my unix server. This started a week ago. One day after a backup (I used to keep 3 backup files) I visited a website on the server but it wouldn't work. I restarted the server and it seemed to be working fine except the mysql service. My attempts to restart it failed. Then I figured that was because the server was full, so I deleted one of the backups, cleaned up some space and the mysql service restarted successfully. Than I figured tables in one of the databases (MYIsam tables) were corrupt. So I repaired them through myisamchk command via ssh and all worked fine. However, the very next day I woke up they were corrupt again (despite mysql was working fine), and this time there was no disk space problem on the server. I repaired them again. The next day the same thing happenned; and this time innodb tables that were part of another database were corrupt as well. I've fixed them too, so now all is working well but I guess the same thing will happen after tonight's backup.
I can't identify the problem and I don't know what logs to look into to understand the problem. Can anyone please help me out? Thanks very much in advance.
No easy answer here. My immediate thought is that the dbase is still busy when the backups commence, possibly corrupting indexes, interferring with caches, etc. Turn on full logging and check for problems when the backup starts happens. Maybe you will find something.
Look for the my.cnf file. On my CentOs it is located in /etc/my.cnf. It will have a config setting for the location of the error log.
My strongest suspect is OOM kill by the kernel or some other issue that results from running the system out of memory. Try this:
Start top on the server and press M to sort by memory so the biggest memory user is at the top.
note the pid of mysqld
manually perform the backup as you observe the value of the RES column in the top output (resident memory size)
once the backup is over see if the pid of mysqld has changed
If the pid has changed (meaning restart took place), and you saw the memory footprint of mysqld take up something comparable to the total amount of system memory, then my suspicion is correct, and we need to lower some settings in my.cnf to make it use less memory, e.g key_buffer_size and innodb_buffer_pool_size.
EDIT - From the log you posted, there are additional issues although it is not clear how they could be contributing to the table corruption. Your server appears to be running with --skip-innodb and your backup script is not able to deal with the absence of InnoDB storage engine printing exception error messages, but nevertheless continuing. It is also attempting to do a repair, which is failing due to the lack of system privileges (error 1 is Operation not permitted). It is possible that encountering those errors triggers some faulty logic in your backup script that leaves the tables corrupted.
At this point I would recommend disabling MySQL backup using the cPanel tool, and using mysqldump or some other solution (e.g. Xtrabackup (https://www.percona.com/doc/percona-xtrabackup/2.3/index.html)) from a cron job instead.
EDIT2 - from the test results. The manual backup does not run the system out of memory and does not crash the server. The jury is still out on the automatic one.
Don't kill mysqld; shut it down gracefully.
Switch from MyISAM to InnoDB; the latter does not suffer from that 'error'.
I'm trying to add a Fulltext index to a table. When I run the query,
ALTER TABLE thisTable ADD FULLTEXT(thisText);
I get the message
SQL Error (2013): Lost connection to MySQL server during query
and the mysql service will indeed be stopped. If I restart the service and try to add the index again I get another error.
SQL Error (1813): Tablespace for table 'thisTable/#sql-ib21134' exists. Please DISCARD the tablespace before IMPORT.
The engine is InnoDb and I run MySQL 5.6.12, so Fulltext index should be supported. The column is a TEXT column.
I'd be very grateful if someone could point me in the right direction where the error is coming from.
The problem is related to sort buffer size. It is known bug of mysql/mariadb/percona.
Even after many months I have reported this bug it was not fixed (we are using latest Mariadb)
The second error happens because the table (or the fulltext index table) was partially modified (or created) when the server crashed. Drop and re-create your table from scratch.
Now, why did the server crash? Hard to tell for sure, but it is likely that some buffer reached capacity. The usual suspect is innodb_buffer_pool_size. Try to increase it progressively.
On Ubuntu server 20.04,
despite reasonable logic to increase innodb_buffer_pool_size I have idenified that OOM Killer killed mysql when that buffer is huge.
OOM Killer is a function of the Linux kernel that is intended to kill rogue processes that are requesting more memory that the OS can allocate, so that the system can survive.
Excerpt from syslog:
oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/system.slice/mysql.service,task=mysqld,pid=98524,uid=114
Jan 10 06:55:40 vps-5520a8d5 kernel: [66118.230690] Out of memory:
Killed process 98524 (mysqld) total-vm:9221052kB, anon-rss:5461436kB,
file-rss:0kB, shmem-rss:0kB, UID:114 pgtables:11232kB oom_score_adj:0
So this means, due tu huge innodb_buffer_pool_size setting, it is trying to allocate too big chunks of memory, too much agressiveley.
So using that fact, logic dictates to reduce size or set default values for innodb_buffer_pool_size, I have completley removed it from /etc/mysql/mysql.conf.d/mysqld.conf then restarted server with
systemctl restart mysql
and now adding fulltext on my 61M table is not crashing.
Bad luck pal...
InnoDB tables do not support FULLTEXT indexes.
Source - http://dev.mysql.com/doc/refman/5.5/en/innodb-restrictions.html
I am working on a project where i need to create a database with 300 tables for each user who wants to see the demo application. it was working fine but today when i was testing with a new user to see a demo it showed me this error message
1030 Got error 28 from storage engine
After spending some time googling i found it is an error that is related to space of database or temporary files. I tried to fix it but i failed. now i am not even able to start mysql. How can i fix this and i would also like to increase the size to maximum so that i won't face the same issue again and again.
Mysql error "28 from storage engine" - means "not enough disk space".
To show disc space use command below.
myServer# df -h
Results must be like this.
Filesystem Size Used Avail Capacity Mounted on
/dev/vdisk 13G 13G 46M 100% /
devfs 1.0k 1.0k 0B 100% /dev
To expand on this (even though it is an older question); It is not not about the MySQL space itself probably, but about space in general, assuming for tmp files or something like that.
My mysql data dir was not full, the / (root) partition was
I had the same issue in AWS RDS. It was due to the Freeable Space (Hard Drive Storage Space) was Full. You need to increase your space, or remove some data.
My /tmp was %100. After removing all files and restarting mysql everything worked fine.
My /var/log/apache2 folder was 35g and some logs in /var/log totaled to be the other 5g of my 40g hard drive. I cleared out all the *.gz logs and after making sure the other logs werent going to do bad things if I messed with them, i just cleared them too.
echo "clear" > access.log
etc.
Check your /backup to see if you can delete an older not needed backup.
I had a similar issue, because of my replication binary logs.
If this is the case, just create a cronjob to run this query every day:
PURGE BINARY LOGS BEFORE DATE_SUB( NOW(), INTERVAL 2 DAY );
This will remove all binary logs older than 2 days.
I found this solution here.
A simple:
$sth->finish();
Would probably save you from worrying about this. Mysql uses the system's tmp space instead of it's own space.
sudo su
cd /var/log/mysql
and lastly type: > mysql-slow.log
This worked for me
Drop the problem database, then reboot mysql service (sudo service mysql restart, for example).
If you want to use the tokudb plugin
This can happen if you have less than 5% (by default) of free space.
see the option: tokudb_fs_reserve_percent