Please let me know what I am doing wrong. As website goes down at every 300 concurrent users.
First step to consider is in your my.cnf [mysqld] section
thread_cache_size=100 # CAP suggested by V8.0 for avoid OOM.
This should be a Dynamic Global Variable that can be set with
SET GLOBAL thread_cache_size=100;
to avoid shutdown/restart.
top looks like ~2000 threads were trying to do something with ~50 running which may drive context switching through the roof.
Also please post your error log for any abnormal shutdown, there are likely clues of the leading cause.
Related
After a few weeks running, mysqld (mariadb) is reserving more and more memory as you can see on the screenshot below
htop screenshot
Some answers says that disabling query caching will help, so i did add to /etc/mysql/my.cnf
[mysqld]
query_cache_size = 0
query_cache_type = 0
and tried to restart mysqld but the reserved memory is still the same after the restart.
I didn't touch anything else on the my.conf and using both myisam and innodb tables
Also RESET QUERY CACHE on selected db does nothing at all
This is normal, and 209MB of RAM is very small for a typical MySQL deployment. Did you change any settings away from defaults? That memory consumption looks about right if you haven't. How big is your data?
Some things that are allocated in RAM grow up to some limit, then stick with that setting. The buffer_pool (cf innodb_buffer_pool_size) is the main memory allocation. When you start mysqld, only a little space is allocated for it. As you process data, tables are opened, etc, and the buffer_pool grows. This is probably what you are seeing.
The Query cache is generally useless and should be left off. (There are rare exceptions.) Even if you use it, it would probably not explain what you are seeing. It quickly fills up, fragments and remains (I think) stuck at query_cache_size.
If you need the memory for other apps, then do decrease innodb_buffer_pool_size. With only 170MB of data (including indexes?), a setting of 250M is probably more than adequate. Possibly 100M would work OK, but I would not go that low without adequate need for the RAM elsewhere.
The buffer_pool is used for a variety of things, but it is a "cache". That is, to handle all your data, it needs to be more then 170M, but being a cache, it is find to set it less than that. (But understand that may lead to more I/O.)
The Htop shows that not all of your 8GB of RAM is in use.
Also, lower max_connections; this will decrease the per-connection memory allocations. For most users 20 is sufficient. (Then watch for error messages something like 'cannot get a connection'.)
I am using Windows 7 and My Computer Name is 'COREI5' and have a 1tb Hard Drive.
My Hard Drive is showing as Full but i was not able to Locate which File is so huge to Block the Drive Space.Now Seems i Figured out the File Source.
C:\ProgramData\MySQL\MySQL Server 5.6\data\COREI5-PC-slow
So it seems this 'COREI5-PC-slow' is the culprit file as it is showing a size of aprox 640GB.Note that this filw is shown as a txt file.
My queries are:
1) Will deleting this file harm my computer ? (I am getting error "You Need permission from Computer administrator to make changes")
2) I am not able to delete this file (even after i logged in as administrator)
3) Also Tried to give special permissions but now working
Any Solution ?
Note: I am not much savvy with Such Programs and commands to request you to give details or keep it simple.
I suspect the file is the "slow query" log in the MySQL data directory.
To confirm, connect to MySQL database, and run a query:
SHOW VARIABLES LIKE 'slow%'
Variable_name Value
------------------- --------------------------------------------------------------
slow_launch_time 2
slow_query_log OFF
slow_query_log_file C:\ProgramData\MySQL\MySQL Server\MyLaptop-slow.log
I suspect that in your case, slow_query_log is set to ON. If the filename shown for slow_query_log_file matches the file on your system, you can safely turn off the slow_query_log and then delete the file.
To turn off the slow query log:
SET GLOBAL slow_query_log = 0
Re-run the SHOW VARIABLES LIKE 'slow%' to confirm it's off.
And then you can delete the file from the filesystem. (If you are doing it from the GUI, don't just Delete the file and put it the Recycle Bin. Hold down the shift key when you click Delete, and it will prompt you if you want to "permanently" delete the file.
I'd be concerned that MySQL has logged 640GB worth of slow queries.
That slow_query_launch_time determines the amount of time a query executes before it's considered slow. There also may be a setting that sends all queries that don't use an index into the slow query log, even if it runs faster than slow_query_launch_time.
While you're at it, check that the general log is turned off as well.
SHOW VARIABLES LIKE 'general%'
This question might better be asked on dba.stackexchange.com
For hunting down huge space consumers, I recommend TreeSize Free from JAM Software. An easy to use old-style windows explorer interface, that gives you the total size of directories and files.
My final objective was to delete that file stated above and i was able to achieved the same with help of SHIFT+DELETE and then restart of PC.
It worked - thank you once again.
I have to change max_allowed_packet size in MySQL using phpmyadmin, but I don't know how to do it. When I try set global max_allowed_packet=10M in phpmyadmin it give this error
#1227 - Access denied; you need the SUPER privilege for this operation
I can't get SUPER privilege, because server is not in my control.
So, How can I change it?
You will have to set this in MySQL as well .. Generally found here:
/etc/mysql/my.cnf
Example:
max_allowed_packet = 16M
If the server is not in your control, you are going to have to ask for access to said file.
You cannot.
To change it dynamically, as with the SET you tried, you need the SUPER privilege, there is no way around it. And this is a good thing, because 1. the setting is global, which means it affects all connections, and 2. it might jeopardize the server (it makes it easier to DoS a server, for example).
To set it permanently, you need access to the MySQL configuration file and be able to restart the service, as Zak advises.
The real question is, however, why do you need such a high limit. Unless you are trying to import a large dump, having a need for such a limit almost always suggests something was wrongly designed in the first place. If you are importing a dump, try to import smaller bits at a time.
You can change variables from the "Server variables and settings" page, which is accessible via "Variables" at the top or at [server]/phpmyadmin/server_variables.php
Look up "max_allowed_packet", and hit Edit - default is 4194304 (4MB, in bytes).
When I restart mysql, website becomes too slow for about 10 minutes.I think it is because innodb_buffer pool and query cache.
innodb buffer pool =8G and query cache = 2G.
I am changing my .cnf for optimizing but i have 200 online users and slowing down server for 10 minutes makes users angy.
Is there anyway I can change mysql variables without mysql restart?
Those aren't the variables making you slow after restart, those are the variables making you faster after a few minutes when everything is in memory again.
Some global variables may be changed with SET global varname=..., some can't. So, it all depends. Just a reminder: if you're satisfied with them... don't forget to add them in the config for when you DO need to restart. You wouldn't be the first to accidentally use a vital piece of configuration that way.
Having multiple mysql servers & taking one offline for a reload (the other one gets all the queries) can be blessing, but may be pricey of course.
I am getting the following error:
mysql_connect(): Too many connections
It has completely shut down my site, which has been running seamlessly for several years.
Note: I have shared hosting with GoDaddy.
How do I fix this?
ALSO: is there a way to close all connections and restart when on a shared hosting plan?
This is a Technical Response
You will get this "too many connections" error upon connecting to MySQL when the MySQL server has reached its software configurable artificial limit of maximum concurrent client connections.
So, the proper way to fix this is:
Directly connect to the MySQL server, and perform the query: SET GLOBAL max_connections = 1024; to change the connection limit at runtime (no downtime).
Make your change permanent, and edit the /etc/my.cnf (or similar) and add the line max_connections = 1024 line within the [mysqld] section; then restart if you couldn't do the live change.
The chosen limit of 1024 was pure subjective; use whatever limit you want to. You can also inspect your current limit via query SHOW VARIABLES LIKE "max_connections";. Keep in mind, that these limits are there for good use, in order to prevent unnecessary overload of your backend database. So always choose wise limits.
However, for those steps you are required to have direct access to your database MySQL server.
As you said, you are using GoDaddy (I do not know them that much), you are left out with the option to contact your service provider (i.e. GoDaddy). However, they will see this in their logs anyway, also.
Possible Root Causes
This of course means, that too many clients are attempting to connect to the MySQL server at the same time - as of the per configuration specified artificial software limit.
Most probably, you have been a subject of a DDoS attack.
People on this forum complain on exactly same thing with exactly same provider.
The answer is this:
VB told me it was a DOS attack - here is their message:
This is not an 'exploit'. This is a DoS attack (Denial of Service). Unfortunately there is nothing we can do about this. DoS attacks can only be fought at the server or router level, and this is the responsibility of your host. Instead of doing this they have decided to take the easy way out and suspend your account.
If you cannot get them to take this seriously, then you should look for another host. Sorry for the bad news.
A possible workaround can be this: if your connection fails with mysql_connect(): Too many connections, you don't quit, but instead sleep() for half a second and try to connect again, and exit only when 10 attempts fail.
It's not a solution, it's a workaround.
This of course will delay your page loading, but it's better than an ugly too many connections message.
You also can come with a some kind of a method which tells bots and browsers apart.
Like, set a salted SHA1 cookie, redirect to the same page and then check that cookie and connect to MySQL only if the user agent had passed the test.
Another thing that can cause this error is if the Database has run out of space. I recently had this occur, and the issue wasn't connections, it was Disk space. Hope this helps someone else!
Do you close your connection when you're done with them? Are you using some type of connection pooling? Sounds like you're opening connections and not closing them.
EDIT: Already answered by Quassnoi. In the case it is a DDoS, and you're using shared hosting, you may be left with just contacting your host and working it out with them. Unfortunately this is a risk when you don't have control of your whole system.
Consider using mysql_pconnect(). Your host may have adding some sort of throttling for connections. Like a maximum of 100 per 20 minutes or something weird.
First, check your database connections:
show variables like 'max_connections';
to check all variables
show variables;
Connect your MySQL server and update the connection size:
SET GLOBAL max_connections = 1001;