Problem:
MySQL is taking over 350MB on idle, as shown in docker stats
Tried:
Tweaking the configuration file as suggested in posts like this one.
Added these lines to the file: /etc/mysql/my.cnf
innodb_buffer_pool_size=64M
innodb_log_buffer_size=256K
max_connections=5
key_buffer_size=8
thread_cache_size=0
host_cache_size=0
innodb_ft_cache_size=1600000
innodb_ft_total_cache_size=32000000
thread_stack=131072
sort_buffer_size=32K
read_buffer_size=8200
read_rnd_buffer_size=8200
max_heap_table_size=16K
tmp_table_size=1K
bulk_insert_buffer_size=0
join_buffer_size=128
net_buffer_length=1K
innodb_sort_buffer_size=64K
Dockerfile
FROM mysql
COPY my.cnf /etc/mysql/
Actual:
I made sure my.cnf changes were in the container, however its still taking over 350MB idle, is it possible to get it below that or am I trying something just not possible?
Prepared repository:
https://github.com/alexanderkoller/low-memory-mysql
Before using this repository's configurations memory is about 458Mib/s
After: 31.12Mib/s
Tested on mysql 5.6
You can try and appropriately reduce footage.
https://github.com/major/MySQLTuner-perl
Related
My Ubuntu 22.04 server is suddenly telling me that "The redo log file "./#innodb_redo/#ib_redo0 size 23289856 is not a multiple of innodb_page_size." My innodb_page_size is 16K, so the error is correct, but I can't seem to find any advice on how to fix it. I tried moving ib_redo0 out of the way but that didn't help. Any ideas?
I also encountered this issue. It appeared to be specific to using ZFS on Ubuntu, in my case it was during an upgrade to MYSQL 8.0.30-0ubuntu0.20.04.2.
Following details in this Ubuntu issue report and this MySQL issue report I was able to come up with a solution that worked in my environment.
There are 3 commands below to be ran as root or with sudo. You should replace 8192 in the first with the result of <broken_file_size> % <default_page_size>. The default page size is usually 16384 unless modified.
You may need to replace the #ib_redo0 part of the second command with the broken file reported in the error message.
These commands are intended to pad out the reportedly invalid file with zeros.
Perform a backup before running!
# Gather required zeros to append
# Will create a "zeros" file in the current directory
# This has been calculated based upon 23289856 % 16384 = 8192 or <broken_file_size> % <default_page_size>
dd if=/dev/zero bs=1 count=8192 of=./zeros
# Append zeroes to invalid file
cat zeros >> /var/lib/mysql/#innodb_redo/#ib_redo0
# Restart MySQL
systemctl restart mysql.service
I'd be wary of remaining on ZFS, even if the above fixes things, for the sake of potentially hitting the same issue again.
I had the same Problem in a LXD Container running on ZFS. I had to move it to a different type of storage-pool, e.g. Directory or BTRFS.
After that the solution of #DanBrown worked for me too.
Thank you.
I've been trying to setup multiple instances of mysqld on an almalinux box.
I can't get the systemctl start mysqld#replica03 statements to recognize the associated options setup in the /etc/my.cnf file.
If I change the [mysqld#replica03] or [mysqld#replica04] to [mysqld] it works and uses the appropriate datadir and port. As soon as I use [mysqld#replica03] and [mysqld#replica04] in the /etc/my.cnf file it uses the default /var/lib/msyql datadir and options instead of the ones I specified
I have run
systemctl enable mysqld#replica03
systemctl enable mysqld#replica04
Any ideas would be appreciated.
/etc/my.cnf file:
[client-server]
[mysqld#replica03]
server_id=3
port=3306
datadir=/var/lib/mysql_replica03
socket=/var/lib/mysql_replica03/mysql.sock
log-error=/var/log/mysql_replica03/mysqld.log
pid-file=/var/run/mysqld_replica03/mysqld.pid
[mysqld#replica04]
server_id=4
port=3307
datadir=/var/lib/mysql_replica04
socket=/var/lib/mysql_replica04/mysql.sock
log-error=/var/log/mysql_replica04/mysqld.log
pid-file=/var/run/mysqld_replica04/mysqld.pid
When you need multiple instances, and you're using systemd to achieve it, the server option groups configuration is the way to go.
The comment you made on your question shows that the suffix used is prefixed with a dot.
ExecStart=/usr/libexec/mysqld --defaults-group-suffix=.%I
So the server groups should be rewritten to
[mysqld.replica03]
server_id=3
port=3306
datadir=/var/lib/mysql_replica03
socket=/var/lib/mysql_replica03/mysql.sock
log-error=/var/log/mysql_replica03/mysqld.log
pid-file=/var/run/mysqld_replica03/mysqld.pid
[mysqld.replica04]
server_id=4
port=3307
datadir=/var/lib/mysql_replica04
socket=/var/lib/mysql_replica04/mysql.sock
log-error=/var/log/mysql_replica04/mysqld.log
pid-file=/var/run/mysqld_replica04/mysqld.pid
Another important observation is that the order in configuration files matters. Theses replica servers definitions need to be parsed after main [mysql] section.
my_print_defaults will help you with it:
$ my_print_defaults --defaults-group-suffix=.another mysqld
--datadir=/var/lib/mysql
--socket=/var/lib/mysql/mysql.sock
--log-error=/var/log/mariadb/mariadb.log
--pid-file=/run/mariadb/mariadb.pid
--datadir=/db/another/mysql
--socket=/db/another/mysql/mysql.sock
--log-error=/db/another/log/mariadb/mariadb.log
--pid-file=/run/mariadb/mariadb-another.pid
$
If your OS has a /etc/my.cnf.d/ directory, as mine does, you will need to sort the filenames in order to accommodate this requirement.
As I'm using mariadb, and I have a /etc/my.cnf.d/mariadb-server.cnf, my another server group is on a file named zz-another.cnf, making sure it will be parsed after mariadb-server.cnf.
I don't have a tech background, but decided to create a Google Compute Engine instance to be able to create a 1.6GB database. I use PostgreSQL (Pgadmin3) to access the database on GCE. It has been working fine so far, but now I am trying to import a new file and am getting the error message
"Panic: could not write to file "pg_xlog/xlogtemp.1293": No space left on the device"
I looked at the documentation in Google for resizing the disk. I did a df -h in SSH and got
/dev/sda1 9.8G 9.2G 16M 100% /
I also did
sudo lsblk --output NAME,size,state
NAME SIZE STATE
sda 20G running
└─sda1 10G
So then I did
sudo resize2fs /dev/disk/by-id/google-myddisk-part1
resize2fs 1.42.12 (29-Aug-2014)
The filesystem is already 2620928 (4k) blocks long. Nothing to do!
Anyone have any ideas?
As this is the root disk you have to reboot the instance. It should then automatically resize the disk during boot. Please check details here: https://cloud.google.com/compute/docs/disks/create-root-persistent-disks#repartitionrootpd
So I have the following in my monitrc file:
check process apache with pidfile /usr/local/apache/logs/httpd.pid
group apache
start program = "/etc/init.d/httpd start"
stop program = "/etc/init.d/httpd stop"
if failed host XXX port 80 protocol http
and request "/monit/token" then restart
if cpu is greater than 60% for 2 cycles then alert
if cpu 80% for 5 cycles then restart
if totalmem 500 MB for 5 cycles then restart
if children 250 then restart
if loadavg(5min) greater than 10 for 8 cycles then stop
if 3 restarts within 5 cycles then timeout
but I keep getting the error that:
Error: service name conflict, apache already defined '/usr/local/apache/logs/httpd.pid'
If the hostname of the server is 'apache' then the conflict is with the default rule for monitoring the system load.
Monit seems to have the implicit rule of 'check system hostname', where the hostname is the output of hostname command.
You can overwrite that by adding just a line like:
check system newhostname
For example:
check system localhost
I saw this error when I forgot to comment out the line:
include /etc/monit/conf.d/*
in a custom /etc/monit/conf.d/myprogram.conf file, so it was recursively including that file.
By any chance do you have an entry with a host name apache beneath this entry or in a separate monit config file?
You have the same service defined more than once. Check all your monit config files for that service. This includes your monitrc and all files listed under the "Includes" section (like include /etc/monit/conf.d/*).
If you redefine "Includes" within a file in one of your "Includes" directories, you will run into recursive reference problems.
Very very important thing : you need monit 5.5
For example in ubuntu 12.04 available in repo only 5.3
So you need to download and install from other repo.
Solution for me , for example :
wget http://mirrors.kernel.org/ubuntu/pool/universe/m/monit/monit_5.5.1-1_amd64.deb && sudo dpkg -i monit_5.5.1-1_amd64.deb
For my case, I simply had to restart monit to get rid of the service name error:
sudo service monit restart
Check if you have had any conflicts for Apache defined in any of the monit conf files under /etc/monit.d/ directory, I accidentally did added nginx for my puma.conf and ran into the same error before.
How can I view mysql slow_query_log to see which query is taking too much time?
First, you need to check if it's enabled in your MySQL configuration (mysql.ini or mysql.cnf, depending on your system):
# enable slow log:
slow_query_log = 1
# log queries longer than n seconds:
long_query_time = 5
# where to log:
slow_query_log_file = /path/to/your/logs/mysql-slow.log
Restart your MySQL server, then watch the logfile using whatever program you like - tail is the simplest:
tail -f /path/to/your/logs/mysql-slow.log
You may need to play a bit with the long_query_time setting to find the limit where the volume of logging isn't too low or too high, but just right.
Check the location of this log in my.ini file, and then open it in any text editor.
if you ask google for "slow_query_log", this is the first hit - explaining all you need to know. you have to enable it, set a filename you like (if it's already set, you can find the configuration in you my.ini), start your queries and look ito that file...
If you're running mysqld < 5.2, your my.cnf may look like
log-slow-queries=/var/log/mysql/mysql-slow.log //where to store log
long_query_time=3 //quickest query to log