we're using MySql on CloudSql for quite some time now.
Obviously, we started with Mysql 5 but after a long wait and the final release of Mysql8 we decided to upgrade our database server.
As the title promotes, we now see a strange behavior of our memory utilization.
As you can see here it constantly fills up until server max resources are reached and then restarts and start filling up again.
I mean there could be an issue with one of our services but before the upgrade our memory consumption looked like this:
So you can see, memory consumption was more or less constant.
Furthermore, we increased resources when we upgraded to mysql8 and switched from db-n1-standard-1 to db-n1-standard-2, to have more available resources when data grows up.
Does anyone knows this behavior? Is there a change in Mysql5 to 8? I didn't find any information about it. Just found some notes that it's normal that Mysql takes as much memory as it can get. But I'm still wondering why it didn't on Mysql5.
Some more details on the configuration:
We're using read replica for HA
Binarylogs activated
Slow Query log enabled with FILE output.
Everything else is default CloudSql Configuration.
Any help is much appreciated.
Best regards,
Chris
Indeed, it seems that MySQL 8 is consuming more memory than MySQL 5. As shown in some tests performed by the author of the article MySQL 8 and MySQL 5.7 Memory Consumption on Small Devices
, the memory used by the version 8 in same VM settings is considerably higher than on versions 5, including both resident and virtual memories - even though these are tests in small VMs, it's a good indication that this occurs in bigger configurations as well.
So, yes, it seems that, as you mentioned, it's normal that Mysql takes as much memory as it can get, but that indeed, MySQL 8 is consuming more memory than the 5 one.
Related
I have a java / spring / hibernate app running with connections to a MySQL db.
We recently upgraded from 5.6 to 5.7 (on a Windows server) and the app has gone from taking 3 hours to 3 days to complete. It essentially uses hibernate connections to retrieve read only data from the db before processing it and dumping the result elsewhere.
However as a first step, partly to check it was the upgraded version that is causing the problem, I installed 5.7.21 on my dev machine. I then noticed that even doing a db restore took several hours rather than what used to take about 10 minutes on 5.6. This has lead me to believe it may be more of a config issue than 'drivers' being out of date (I did think my first step was going to be upgrading app dependencies). I didn't install the server but I installed my dev machine with a default 'developer' install. Both the server and the dev machine are 64 bit Windows.
I've had a scoot around for obvious gotchas and not found anything yet. I just wondered if anyone could point me in the right direction before I start seriously thrashing about ? I have a good basic understanding of out of the box MySQL but I haven't done much config so even pointers to likely suspects in my.ini and best ways to investigate would be helpful.
When upgrading, pay attention to innodb_buffer_pool_size variable value.
It controls how much memory MySQL uses to make I/O operations faster. Usually, this is the one that makes it fly or that makes it crawl like a snail. There's a lot to be written about this particular variable, there's a plethora of excellent blog posts about it so I'll avoid explaining it into detail.
To see the current value, type in MySQL terminal:
SHOW VARIABLES LIKE '%innodb_buffer_pool%';
Change the value in config file and restart MySQL.
For the value, don't go overboard, don't exceed your entire RAM. You want it as high as possible, especially for servers with a lot of data.
We have a 2GB Digital Ocean server, and it is dedicated for a MySQL server of other two PHP servers. we are using Percona MySQL Server 5.6 on this server. We configured MySQL replication and these configuration is working fine
Our issue is sometime our site monitoring tools reporting that some of the URL hosted with this server is down (May be this is happening once in a week or two). When I am checking, I could see that Mysql Master server load is too much high (May be 35 - 40), so the MySQL server was not responded. # that I usually do a MySQl service restart, this restart cause to server load become normal and the sites started working after service restart.
This is a back-end MySQL database server of 20-25 PHP applications (WordPress, Drupal and some custom applications server).
Here are my questions,
Why this server load automatically goes down, after a spikes happens?
Is there any way in which database is causing issues? So that I can identify the application too.
How can I identify the root cause of this issues
Depending upon your working dataset, a 2GB server providing access for 20-25 PHP applications (WordPress, Drupal and some custom applications server) could be the issue.
For example, if you have a 1.4GB buffer pool (assuming all tables are InnnoDB) and 10GB of data, then your various applications could end up competing for resources, such as I/O, buffer pool pages, Adaptive Hash Index, query cache. They could also, assuming caching is used, be invalidating theit caches within a similar timeframe, thus sending expensive queries to the database.
Whilst a load of 50 is something that you would normally want to avoid, the load average is not something that you should concern yourself with if showing in isolation.
The use of the uninterruptible state has since grown in the Linux
kernel, and nowadays includes uninterruptible lock primitives. If the
load average is a measure of demand in terms of running and waiting
threads (and not strictly threads wanting hardware resources), then
they are still working the way we want them to.
http://www.brendangregg.com/blog/2017-08-08/linux-load-averages.html
If the issue is happening once per week then it is starting to sound like a batch process, or cache expiration issue - too much happening at once for the resources available.
The best thing to do is to monitor and look for the cause. Since you are already using Percona Server, using PMM should give you the perfect insight to find the cause, although it works with Oracle MySQL, MariaDB, Aurora, etc. You can try a demo to see the insights that you can gain:
https://pmmdemo.percona.com. The software is Open Source and free to use.
You can look in QAN to find the most expensive queries, whilst looking at Prometheus data to give an insight into the host itself. There are some recommendations to get the most from PMM, depending upon your flavour of MySQL.
I have installed a SpringMVC Web application with JPA and a Mysql Database.
The application is displaying statistics from the database (with a lot of selects)
It works quite fast on Linux(mysql 5.5.54), but it is very slow on Windows 10 (mysql 5.6.38).
Do you know what could cause such a behaviour on Windows?
Or could you give me hints or tell me where to search?
[UPDATE]
Linux : Intel® Core™ i7-4510U CPU # 2.00GHz × 4 / 8GoRAM
Windows : Intel Xeon CPU E31220 3.1Ghz 4GoRAM
I know that the windows machine is not as "powerful" than the linux one. I wonder if, by increasing the memory, that could be enough. Or does Mysql needs a lot of CPU too.
My list would be:
Check configs are identical - not just the settings in my.ini - values not set here are set at compile time and the 2 instances have definitely been compiled seperately! You'll need to capture and compare the output of SHOW VARIABLES
Check file deployment is similar - whether innodb is configured to use one file per table, whether the files are distributed across multiple disks
Check adequate memory available for caching on MSWindows
disable anti-virus
Make sure MSWindows is configured as a server (prioritize background tasks)
Windows sucks, deal with it :)
I have a Mac Pro with i7 processor, 16GB RAM, and sufficient storage running Win 8.1 via Parallel on top of OS X Yosemite. I have a 23GB MySQL data and I am wondering if I am able to have such a big data loaded into MySQL in my PC. I started to import data but it stops after an hour throwing error
Error 1114 (HY000) at line 223. The table X is full.
I googled the error and found the same error discussed in Stackoverflow (but not this much of data). I tried to resolve using the given solutions but failed. MySQL imports about 3G of data and then throws the error.
Now, here are my 3 main questions.
Is my data much more bigger than a MySQL data engine can have on a PC?
If this is not the case and I am good to go with that much data, do I have any configuration required to enable running a 23GB data on my PC?
Final concluding question is how big is big that one cannot run on its machine? Is it only matter to be able to store data locally or it needs some other things?
Of course MySQL on Windows can handle 23GB of data. That's not even close to its limit.
Keep in mind that a database takes lots of disk space for indexes and other things. 23GB of raw data probably will need 100GB of disk space to load, to index, and to get running. If you are loading it into an InnoDB table you will also need transaction rollback space for the load.
It seems likely that your Windows 8.1 virtual machine running on Parallels is running out of disk space. You can allocate more of your Mac's disk for use by Parallels. Read this. http://kb.parallels.com/en/113972
Your answers can be found within the MySQL reference
The effective maximum table size for MySQL databases is usually
determined by operating system constraints on file sizes, not by MySQL
internal limits. The following table lists some examples of operating
system file-size limits. This is only a rough guide and is not
intended to be definitive. For the most up-to-date information, be
sure to check the documentation specific to your operating system.
Recently we changed app server of our rails website from mongrel to passenger [with REE and Rails 2.3.8]. The production setup has 6 machines pointing to a single mysql server and a memcache server. Before each machine had 5 mongrel instance. Now we have 45 passenger instance as the RAM in each machine is 16GB with 2, 4 core cpu. Once we deployed this passenger set up in production. the Website became so slow. and all the request starting to queue up. And eventually we had to roll back.
Now we suspect that the cause should be the increased load to the Mysql server. As before there where only 30 mysql connection and now we have 275 connection. The mysql server has the similar set up as our website machine. bUt all the configs were left to the defaul limit. The buffer_pool_size is only 8 mb though we have 16GB ram. and number of Concurrent threads is 8.
Will this increased simultaneous connection to mysql would have caused mysql to respond slowly than when we had only 30 connections? If so, how can we make mysql perform better with 275 simultaneous connection in place.
Any advice greatly appreciated.
UPDATE:
More information on the mysql server:
RAM : 16GB CPU: two processors each having 4 cores
Tables are innoDB. with only default innodb config values.
Thanks
An idle MySQL connection uses up a stack and a network buffer on the server. That is worth about 200 KB of memory and zero CPU.
In a database using InnoDB only, you should edit /etc/sysctl.conf to include vm.swappiness = 0 to delay swapping out processes as long as possible. You should then increase innodb_buffer_pool_size to about 80% of the systems memory assuming a dedicated database server machine. Make sure the box does not swap, that is, VSIZE should not exceed system RAM.
innodb_thread_concurrency can be set to 0 (unlimited) or 32 to 64, if you are a bit paranoid, assuming MySQL 5.5. The limit is lower in 5.1, and around 4-8 in MySQL 5.0. It is not recommended to use such outdated versions of MySQL in a machine with 8 or 16 cores, there are huge improvements wrt to concurrency in MySQL 5.5 with InnoDB 1.1.
The variable thread_concurrency has no meaning inside a current Linux. It is used to call pthread_setconcurrency() in Linux, which does nothing. It used to have a function in older Solaris/SunOS.
Without further information, the cause for your performance problems cannot be determined with any security, but the above general advice may help. More general advice geared at my limited experience with Ruby can be found in http://mysqldump.azundris.com/archives/72-Rubyisms.html That article is the summary of a consulting job I once did for an early version of a very popular Facebook application.
UPDATE:
According to http://pastebin.com/pT3r6A9q , you are running 5.0.45-community-log, which is awfully old and does not perform well under concurrent load. Use a current 5.5 build, it should perform way better than what you have there.
Also, fix the innodb_buffer_pool_size. You are going nowhere with only 8M of pool here.
While you are at it, innodb_file_per_table should be ON.
Do not switch on innodb_flush_log_at_trx_commit = 2 without understanding what that means, but it may help you temporarily, depending on your persistence requirements. It is not a permanent solution to your problems in any way, though.
If you have any substantial kind of writes going on, you need to review the innodb_log_file_size and innodb_log_buffer_size as well.
If that installation is earning money, you dearly need professional help. I am no longer doing this as a profession, but I can recommend people. Contact me outside of Stack Overflow if you want.
UPDATE:
According to your processlist, you have very many queries in state Sending data. MySQL is in this state when a query is being executed, that is, the main interior Join Loop/Query Execution loop is busy. SHOW ENGINE INNODB STATUS\G will show you something like
...
--------------
ROW OPERATIONS
--------------
3 queries inside InnoDB, 0 queries in queue
...
If that number is larger than say 4-8 (inside InnoDB), 5.0.x is going to have trouble. 5.5.x will perform a lot better here.
Regarding the my.cnf: See my previous comments on your InnoDB. See also my comments on thread_concurrency (without innodb_ prefix):
# On Linux, this does exactly nothing.
thread_concurrency = 8
You are missing all innodb configuration at all. Assuming that you ARE using innodb tables, you are not performing well, no matter what you do.
As far as I know, it's unlikely that merely maintaining/opening the connections would be the problem. Are you seeing this issue even when the site is idle?
I'd try http://www.quest.com/spotlight-on-mysql/ or similar to see if it's really your database that's the bottleneck here.
In the past, I've seen basic networking craziness lead to behaviour similar to what you describe - someone had set up the new machines with an incorrect submask.
Have you looked at any of the machine statistics on the database server? Memory/CPU/disk IO stats? Is the database server struggling?