I have ISP Config 3 Install on amazon and i just expanded the volume from 8gig to 30gig and when i ssh into the main ispconfig instance or any subdomain instances i get whats below. But when i try an add files or edit any of my sub domains i get "file system is full". How can i edit the subdomain settings so the subdomain knows it has enough space?
/dev/xvda1 30G 6.2G 22G 22% /
udev 821M 4.0K 821M 1% /dev
tmpfs 332M 240K 332M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 829M 0 829M 0% /run/shm
/dev/xvdb 147G 188M 140G 1% /mnt
/dev/mapper/vgpool-data
3.0G 2.9G 28K 100% /mnt/data
In case anyone wanted to know. I needed to expand my LVM /dev/mapper/vgpool-data
I checked lvdisplay and vgdisplay to make sure i had enough room to expand and ran resize2fs -p on it and it worked. I got this info from here
Related
I got a nodeJS application with a MySQL DB on a Linux (Ubunut 16.04 LTS). I'm running on localhost. I'm using sequelize (3.23.0) to do DB manipulation (create, update, etc.)
I got an error message when the following code gets executed (file is called "initSequelize.js"):
db
.sequelize
.sync({alter:true})
.finally(function(err){
if (err) {
console.log("init sequelize error");
throw err[0];
} else {
console.log("foo sequelize");
}
})
.catch(function(error){
console.log("uh oh in Sequelize: ", error);
});
The error message:
foo sequelize
uh oh in Sequelize: {SequelizeDatabaseError: ER_GET_ERRNO: Got error 168 from storage engine
//...
code: 'ER_GET_ERRNO',
errno: 1030,
sqlState: 'HY000',
index: 0,
As you can see, it goes into the else clause and then throws that error. But in the else clause is nothing that could cause an error. Also, I got the permission to write in the mysql folder.
At first I thought I'm running short of storage. Therefore I deleted not needed files. I got over 100 GB of free storage place. But the error still shows.
Further below I would see this error message:
From previous event:
at Promise.then (/home/path/to/project/node_modules/sequelize/lib/promise.js:21:17)
at Sequelize.sync (/home/path/to/project/node_modules/sequelize/lib/sequelize.js:995:6)
at database (/home/path/to/project/init/initSequelize.js:11:6)
Line 11 is referring to ".sync" in the code above
I also deleted my whole project and cloned the project again, deleted the whole MySQL database and re-created an empty one (basically start from scratch). Yet the error still persists (and it used to work before).
EDIT:
I ran df -h
Filesystem Size Used Avail Use% Mounted on
udev 3,9G 0 3,9G 0% /dev
tmpfs 794M 20M 775M 3% /run
/dev/sda1 222G 113G 98G 54% /
tmpfs 3,9G 236M 3,7G 6% /dev/shm
tmpfs 5,0M 4,0K 5,0M 1% /run/lock
tmpfs 3,9G 0 3,9G 0% /sys/fs/cgroup
cgmfs 100K 0 100K 0% /run/cgmanager/fs
tmpfs 794M 68K 794M 1% /run/user/1000
/dev/sdc1 7,3G 2,6G 4,8G 36% /media/fooBarUser/2256-3EDA
The mysql data directory is /var/lib/mysql as stated in /etc/mysql/mysql.conf.d
I'm running a VPS, with specs:
Ubuntu 12.04.5 LTS (GNU/Linux 3.13.0-32-generic x86_64)
512mb RAM
1 CPU
20gb SSD
If you're wondering it's a DigitalOcean droplet. It's running TS3, LAMP (with wordpress), OpenVPN, BYOBU, and OwnCloud.
Now my problem is with mySQL dying on me after like 30m to 1hour. Usually after a reboot, the memory usage is 54% and mySQL doesn't have a problem, but as the memory usage goes towards 80-89% I start to get issues.
System load: 0.01 Users logged in: 0
Usage of /: 22.1% of 19.56GB IP address for eth0: *****
Memory usage: 90% IP address for as0t0: *****
Swap usage: 0% IP address for as0t1: *****
Processes: 93
As you can see, the memory usage is VERY high, and I've noticed the trend that mySQL process dies as the memory usage gets higher. However the swap usage is 0%.
Is there a way to make mySQL and the other processes to use the swap?
Would letting mySQL make use of the swap stop letting it die after my memory usage gets so high?
After the high memory usage, the process dies and I get this error:
[2002] SQLSTATE[HY000] [2002] Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111)
The processor load never goes above 25% in most cases. The server also runs a fast SSD, so it wouldn't be a problem to use a swap, and I don't have that much traffic.
Fixed it, by making a swap file of size 256mb. mySQL doesn't stop now after having no available memory to work in.
After following this tutorial by Etel Sverdlov:
https://www.digitalocean.com/community/tutorials/how-to-add-swap-on-ubuntu-12-04
I was able to make a swap file. I'll copy the tutorial for the sake it gets deleted.
How To Add Swap on Ubuntu 12.04
About Linux Swapping
Linux RAM is composed of chunks of memory called pages. To free up pages of RAM, a “linux swap” can occur and a page of memory is copied from the RAM to preconfigured space on the hard disk. Linux swaps allow a system to harness more memory than was originally physically available.
However, swapping does have disadvantages. Because hard disks have a much slower memory than RAM, virtual private server performance may slow down considerably. Additionally, swap thrashing can begin to take place if the system gets swamped from too many files being swapped in and out.
Check for Swap Space
Before we proceed to set up a swap file, we need to check if any swap files have been enabled on the VPS by looking at the summary of swap usage.
sudo swapon -s
An empty list will confirm that you have no swap files enabled:
Filename Type Size Used Priority
Check the File System
After we know that we do not have a swap file enabled on the virtual server, we can check how much space we have on the server with the df command. The swap file will take 256MB— since we are only using up about 8% of the /dev/sda, we can proceed.
df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda 20907056 1437188 18421292 8% /
udev 121588 4 121584 1% /dev
tmpfs 49752 208 49544 1% /run
none 5120 0 5120 0% /run/lock
none 124372 0 124372 0% /run/shm
Create and Enable the Swap File
Now it’s time to create the swap file itself using the dd command :
sudo dd if=/dev/zero of=/swapfile bs=1024 count=256k
“of=/swapfile” designates the file’s name. In this case the name is swapfile.
Subsequently we are going to prepare the swap file by creating a linux swap area:
sudo mkswap /swapfile
The results display:
Setting up swapspace version 1, size = 262140 KiB
no label, UUID=103c4545-5fc5-47f3-a8b3-dfbdb64fd7eb
Finish up by activating the swap file:
sudo swapon /swapfile
You will then be able to see the new swap file when you view the swap summary.
swapon -s
Filename Type Size Used Priority
/swapfile file 262140 0 -1
This file will last on the virtual private server until the machine reboots. You can ensure that the swap is permanent by adding it to the fstab file.
Open up the file:
sudo nano /etc/fstab
Paste in the following line:
/swapfile none swap sw 0 0
Swappiness in the file should be set to 10. Skipping this step may cause both poor performance, whereas setting it to 10 will cause swap to act as an emergency buffer, preventing out-of-memory crashes.
You can do this with the following commands:
echo 10 | sudo tee /proc/sys/vm/swappiness
echo vm.swappiness = 10 | sudo tee -a /etc/sysctl.conf
To prevent the file from being world-readable, you should set up the correct permissions on the swap file:
sudo chown root:root /swapfile
sudo chmod 0600 /swapfile
All credit to: Etel Sverdlov at: https://www.digitalocean.com/community/tutorials/how-to-add-swap-on-ubuntu-12-04
i am working on a linux CentOS server.
Some day ago i discovered that mysql gave error 1030 (no space avaible on the disk).
Yesterday the admin of the server restarted mysql, now it won't restart because mysql can't find space to create a "pid" file.
My question is:
Since i can't access to mysql server, but i can access to the machine, what is the folder where i can find the cache? I think that i have to delete it because some day ago someone attacked us wih a sql injection, in the log i saw that the hacker executed a lot of "select" queries, so i think the problem is the cache
It may not be the mysql process which has filled up the disk. You can scan the disk and find out where the space is being used with (as the root user):
du -sh /*
This will return a list of directories and their space usage, then you can drill down using:
du -sh /var/*
Or whatever directory is using the most space.
First you should try to find which partition is full by running df command. Ex:
/dev/sdb1 105601116 26127676 74086128 27% /
dev 2022860 0 2022860 0% /dev
run 2025640 508 2025132 1% /run
tmpfs 2025640 28704 1996936 2% /dev/shm
tmpfs 2025640 0 2025640 0% /sys/fs/cgroup
tmpfs 2025640 85100 1940540 5% /tmp
/dev/sda1 307534224 92590832 199298504 32% /data
tmpfs 405128 8 405120 1% /run/user/1000
Then for MySQL it's probably your / partition that is full. Try to run du -sh /var/ (/var is the directory mysql stores files). Ex:
6.4G /var/cache
16K /var/db
4.0K /var/empty
8.0K /var/games
0 /var/run
585M/var/log
It is probably your
/var/log directory which is full of log. You can delete some files (e.g., apache logs) to gain space. Better is to detect where the most space is taken in your disk by running du -sh mydir, starting from / directory and digging to find what to delete.
I have the following error with one of our web applications -
Query3 failed: Error writing file '/tmp/MY1fnqpm' (Errcode: 28) ... INSERT MailList... (removed the rest of the query for security reasons)
Any ideas - is this some hard disk space issue on my server?
Use the perror command:
$ perror 28
OS error code 28: No space left on device
Unless error codes are different on your system, your file system is full.
We have experienced similar issue, and the problem was MySQL used /tmp directory for its needs (it's default configuration). And /tmp was located on its own partition, that had too few space for big MySQL requests.
For more details take a look for this answer:
https://stackoverflow.com/a/3716778/994302
I had same problem but disk space was okay (only 40% full).
Problem were inodes, I had too many small files and my inodes were full.
You can check inode status with df -i
The error means that you dont have enough space to create temp files needed by MySQL.
The first thing you can try is to expand the size of your /tmp/ partition. If you are under LVM, check the lvextend command.
If you are not able to increase the size of your partition /tmp/ you can work in the MySQL configuration, edit the my.cnf (typically on /etc/mysql/my.cnf) file and look for this line:
tmpdir = /tmp/
Change it for whatever you want (example /var/tmp/). Just be sure to have space and assign write permission for the mysql user in the new directory.
Hope this helps!
Run the following code:
du -sh /var/log/mysql
Perhaps mysql binary logs filled the memory, If so, follow the removal of old logs and restart the server. Also add in my.cnf:
expire_logs_days = 3
This error occurs when you don't have enough space in the partition. Usually MYSQL uses /tmp on linux servers. This may happen with some queries because the lookup was either returning a lot of data, or possibly even just sifting through a lot of data creating big temp files.
Edit your /etc/mysql/my.cnf
tmpdir = /your/new/dir
e.g
tmpdir = /var/tmp
Should be allocated with more space than /tmp that is usually in it's own partition.
I had this same error and the problem was simply not enough space on my virtual machine. I deleted some unnecessary files and it started working again.
my
memory/disk space allocation looked something like this
df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/centos-root 37G 37G 127M 100% /
...
For those like me who don't have a full disk or full inodes and that are using MySQL 8, you should take a look at your /var/lib/mysql directory to check if there is some large binlog.XXXXXX files.
It was the case for me and it prevented a query to be executed until the end because it seemed like this query was generating a large binlog file.
I just had to turn off the logbin generation by adding this parameter in /etc/mysql/mysql.conf.d/mysqld.cnf :
disable_log_bin
Then restart mysql service :
service mysql restart
After that i didn't have the "No space left on device" error anymore!
You can also try using this line if the other doesn't work:
du -sh /var/lib/mysql/database_Name
You may also want to check with your host and see how big they allow your databases to be.
For xampp users: on my experience, the problem was caused by a file, named '0' and located in the 'mysql' folder. The size was tooooo huge (mine exploded to about 256 Gb). Its removal fixed the problem.
Today. I have same problem... my solution:
1) check inode: df -i
I saw:
root#vm22433:/etc/mysql# df -i
Filesystem Inodes IUsed IFree IUse% Mounted on
udev 124696 304 124392 1% /dev
tmpfs 127514 452 127062 1% /run
/dev/vda1 1969920 1969920 0 100% /
tmpfs 127514 1 127513 1% /dev/shm
tmpfs 127514 3 127511 1% /run/lock
tmpfs 127514 15 127499 1% /sys/fs/cgroup
tmpfs 127514 12 127502 1% /run/user/1002
2) I began to look what folders use the maximum number of inods:
for i in /*; do echo $i; find $i |wc -l; done
soon I found in /home/tomnolane/tmp folder, which contained a huge number of files.
3) I removed /home/tomnolane/tmp folder
PROFIT.
4) checked:
Filesystem Inodes IUsed IFree IUse% Mounted on
udev 124696 304 124392 1% /dev
tmpfs 127514 454 127060 1% /run
/dev/vda1 1969920 450857 1519063 23% /
tmpfs 127514 1 127513 1% /dev/shm
tmpfs 127514 3 127511 1% /run/lock
tmpfs 127514 15 127499 1% /sys/fs/cgroup
tmpfs 127514 12 127502 1% /run/user/1002
it's ok.
5) restart mysql service - it's ok!!!!
I had the same problem and after little research found that the snap directory was occupying most of the space. So executed following command to get rid of it:
sudo apt autoremove --purge snapd
After that ran following command to get rid of useless/dev/loop mounts:
`sudo apt purge snapd ubuntu-core-launcher squashfs-tools`
After that ran following command to restart mysql:
sudo service mysql restart
It worked for me!
I am getting this error from mysql
ERROR 20 (HY000) at line 1: Disk is full writing './homo_sapiens_core_60_37e/dna_align_feature.MYD' (Errcode: 28). Waiting for someone to free space... (Expect up to 60 secs delay for server to continue after freeing disk space)
I think this could be because i only have space on /local (df - h gives)
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/components-root
19G 18G 0 100% /
none 1.4G 240K 1.4G 1% /dev
none 1.4G 100K 1.4G 1% /dev/shm
none 1.4G 80K 1.4G 1% /var/run
none 1.4G 0 1.4G 0% /var/lock
none 1.4G 0 1.4G 0% /lib/init/rw
none 19G 18G 0 100% /var/lib/ureadahead/debugfs
/dev/sdb1 241G 43G 186G 19% /local
/dev/sda1 228M 51M 166M 24% /boot
How do you specify where mysql writes its databases to? Perhaps there is no space where it is writing the database?
How do you configure where mysql writes its databases? perhaps there is no space where it is trying to write teh data?
I am a unix novice and don't know much mysql either so please be gentle in your answers!
thanks
In your /etc/my.cnf, under the heading [mysqld], your the data directory is identified by datadir.
Also perhaos of interest, in case they were changed from their default, you should also check
the location of your temporary directory tmpdir
the location (log-bin) and history length (expire-log-days) of binary logs you're saving (if any)
the innodb directories (innodb_data_file_path and innodb_data_home_dir)
the location of your slow logs with slow-query-log-file (the size of these logs will depend on how low your slow threshold is).
You can check all of these by doing a SHOW VARIABLES from the MySQL command line. Also note that MySQL only reads the values in the /etc/my.cnf upon startup, so changing the file while MySQL is running won't do anything.
Lastly, you might get more help/ideas at serverfault.
If you have /etc/my.cnf then it may be helpful. If I understand correctly,, The data directory is specified either in startup options or it is defaulted from that file. You would want to edit my.cnf and choose a directory in /local and then manually transfer your existing data from wherever it was to your new place in /local. Make sure MySQL is not running during the transfer.