I have a java / spring / hibernate app running with connections to a MySQL db.
We recently upgraded from 5.6 to 5.7 (on a Windows server) and the app has gone from taking 3 hours to 3 days to complete. It essentially uses hibernate connections to retrieve read only data from the db before processing it and dumping the result elsewhere.
However as a first step, partly to check it was the upgraded version that is causing the problem, I installed 5.7.21 on my dev machine. I then noticed that even doing a db restore took several hours rather than what used to take about 10 minutes on 5.6. This has lead me to believe it may be more of a config issue than 'drivers' being out of date (I did think my first step was going to be upgrading app dependencies). I didn't install the server but I installed my dev machine with a default 'developer' install. Both the server and the dev machine are 64 bit Windows.
I've had a scoot around for obvious gotchas and not found anything yet. I just wondered if anyone could point me in the right direction before I start seriously thrashing about ? I have a good basic understanding of out of the box MySQL but I haven't done much config so even pointers to likely suspects in my.ini and best ways to investigate would be helpful.
When upgrading, pay attention to innodb_buffer_pool_size variable value.
It controls how much memory MySQL uses to make I/O operations faster. Usually, this is the one that makes it fly or that makes it crawl like a snail. There's a lot to be written about this particular variable, there's a plethora of excellent blog posts about it so I'll avoid explaining it into detail.
To see the current value, type in MySQL terminal:
SHOW VARIABLES LIKE '%innodb_buffer_pool%';
Change the value in config file and restart MySQL.
For the value, don't go overboard, don't exceed your entire RAM. You want it as high as possible, especially for servers with a lot of data.
Related
we're using MySql on CloudSql for quite some time now.
Obviously, we started with Mysql 5 but after a long wait and the final release of Mysql8 we decided to upgrade our database server.
As the title promotes, we now see a strange behavior of our memory utilization.
As you can see here it constantly fills up until server max resources are reached and then restarts and start filling up again.
I mean there could be an issue with one of our services but before the upgrade our memory consumption looked like this:
So you can see, memory consumption was more or less constant.
Furthermore, we increased resources when we upgraded to mysql8 and switched from db-n1-standard-1 to db-n1-standard-2, to have more available resources when data grows up.
Does anyone knows this behavior? Is there a change in Mysql5 to 8? I didn't find any information about it. Just found some notes that it's normal that Mysql takes as much memory as it can get. But I'm still wondering why it didn't on Mysql5.
Some more details on the configuration:
We're using read replica for HA
Binarylogs activated
Slow Query log enabled with FILE output.
Everything else is default CloudSql Configuration.
Any help is much appreciated.
Best regards,
Chris
Indeed, it seems that MySQL 8 is consuming more memory than MySQL 5. As shown in some tests performed by the author of the article MySQL 8 and MySQL 5.7 Memory Consumption on Small Devices
, the memory used by the version 8 in same VM settings is considerably higher than on versions 5, including both resident and virtual memories - even though these are tests in small VMs, it's a good indication that this occurs in bigger configurations as well.
So, yes, it seems that, as you mentioned, it's normal that Mysql takes as much memory as it can get, but that indeed, MySQL 8 is consuming more memory than the 5 one.
We've just moved to a new server, both run ubuntu 14.04LTS, the only difference is basically that the old server ran mysql5.5, the new one has mysql5.6. Both servers are cloud machines hosted by digitalocean. Both operate on default my.cnf settings, not much has been tweaked.
An other important difference is, that the new server has double the RAM, and CPU power.
Still - while old server ran with an avg of 0.6 second response time for an api call we monitor server health with, the new one is 1.6-1.8 slower. Yes, they contain heavy joins, but that's not my point - the codebase is exactly the same, and the machine itself is supposed to be stronger. New server also shows peaks of CPU usage few times every hour, which never happened with mysql 5.5.
Does this make any sense? For me, not so much, but I'm no MySQL guru.
Ran MySQL Tuner, but unsure if there's anything relevant within:
mysqltuner output for OLD server:
http://pastebin.com/cqSSssW0
mysqltuner output for NEW server:
http://pastebin.com/uk3g1KZa
The only thing that has been tweaked in my.cnf is that it should log slow queries.
Any idea, why this could happen? MySQL5.6 clearly runs faster on benchmarks I saw online. Any help is very much appreciated.
Following a recent upgrade to Windows 10 my XAMPP didn't seem to want to work (neither Apache or MySQL would start). So I upgraded that too to XAMPP for Windows 5.6.12. There were a few port issues initially (due to new? services in Windows 10) but once those were fixed I have both Apache and MySQL running.
However, now the php pages that I am working on, which do a great deal of reading and writing back to a MySQL database, run unbelievably slowly. A page that used to take a minute or two before any upgrade now takes about 30 minutes. I can see that writing to the database is very slow and the hard disc is always sitting at around 90 to 100%. I have tried many suggested changes: stopping various services, changing the page size etc but it still runs very slowly. I have checked the event log but there is nothing that stands out as an issue.
I am not sure if it the upgrade to Windows 10 or XAMPP that has done it, and I have run out of ideas. I realise this may sound a bit vague and I am happy to post logs etc but I am not sure whether there is a simple reason for this, or something simple for me to check.
As most of you know, Sun acquired MySQL (and later Oracle acquired Sun), and during these acquisitions, there were a lot of FUD in MySQL community which resulted in creation of various forks.
Today we have MySQL from MySQL, Percona (XtraDB) MySQL, OurDelta MySQL, MariaDB, Drizzle to name a few. Which brings us to the source of the problem.
We are in the process of upgrading our databases (hardware/software) and I would like to know which one of the forks should I go with. Each has their own set of pros/cons. We are currently using MySQL 5.0.x from MySQL/Linux on an 8-core machine. Our new hardware is a monster with 32 cores and 32GB of memory connecting to a fast NetApp Storage via FC.
I would like to stick with MySQL from MySQL but I have heard horror stories on how badly MySQL 5.1 performs on many cores. I have also heard that MySQL 5.4 performs better on multi-core machines but that's still not production ready. In addition, I have also heard a lot of good things about Percona builds.
This is what I know so far:
MySQL 5.1 from MySQL: Reliable choice, but doesn't scale well on a big machine
Percona: Scales well, good backing company. I don't have much experience with it
MariaDB: Don't know much about it besides that it was founded by Original MySQL developers (including Monty)
OurDelta: Don't know much
Drizzle: Mostly optimized for cloud computing
I would like to know what's the general notion about this problem. Which build/version should I go with? How are you guys picking your builds/versions?
Thanks!
All of the 'forks' you mentioned (except Drizzle) re-base off newer official releases of MySQL. I think that the word fork can lead you down the wrong path - since the intention is just to provide after market modifications. I wrote about this here:
http://mtocker.livejournal.com/50931.html
Since they re-base of MySQL, and MySQL 5.0 is in "extended maintenance", only major vulnerabilities are to be fixed. This means you want to be moving to 5.1. If we work with this assumption, then it cuts OurDelta out from your decisions - since they are just the 5.1 MariaDB build/packaging partners.
I think you can also rule out Drizzle, since it is currently pre-beta. They're planning a beta by the end of the summer - but it's still much earlier than what you'll want. I really don't think you use a database not ready for production, so this rules out MySQL 5.5 as well.
So this leaves Percona Server, Official MySQL 5.1 and MariaDB. It's true that 5.1 has some poor scalability with many cores, but if you enable the InnoDB plugin it is better.
The rest of my response from here on in is biased - I work for Percona:
Percona XtraDB has additional CPU scalability fixes than MySQL 5.1+InnoDB plugin. The ones that are going to matter are covered here:
http://www.percona.com/docs/wiki/percona-xtradb:features:start
One I can comment on in particular that will matter is this one:
http://www.percona.com/docs/wiki/percona-xtradb:patch:innodb_split_buf_pool_mutex
Personally, I think that everyone who needs to set up a new MySQL installation today should be using MariaDB. MariaDB is based on the latest MySQL 5.1, actively maintained (including fixes for bugs reported against MySQL) and packages Percona XtraDB / InnoDB plugin.
I don't even know if "fork" accurately describes Maria, since they are committed to keeping up with current releases of MySQL.
In short: MariaDB 5.1 = MySQL 5.1 + Percona XtraDB + additional useful patches + active development and maintenance
Update late 2011- May 2012: I switched from MariaDB to Percona Server to get to MySQL 5.5 but I intend on switching back once the Maria team has a stable 5.5 based release. I think that everyone who needs to set up a new MySQL installation today should be running Percona Server. If you are using SSDs, you *must* run Percona Server.
2013-2018: Happily ran Percona server 5.6 this whole time.
2018: upgraded to Percona server 5.7 and then almost immediately to MySQL 8.0. [ At this point, I am not knowledgable enough to talk about the differences between MySQL 8, Percona Server 5.7 and MariaDB 10 and I can't say why you might want to choose one over the others ]
Use MariaDB. They've just released a version lately. Maria engine is also much better than MyISAM. With opensource project, you go where the founder go.
Just an update on this. We chose to go with Percona and we can't be happier. Percona guys definitely know their stuff and keep up with the new technologies. Just checkout http://www.mysqlperformanceblog.com/ Top notch software plus top notch people and great support, you can't go wrong.
I think it's still too early to write off MySQL entirely. I'm not enthusiastic about Sun/Oracle gaining the reins, but I don't have a clear idea what direction they'll take with it. Regardless, I'd wait another year before seriously thinking of switching to a fork. Time will tell which project has the resources and motivation to survive, or if a fork is even necessary.
Google has a memory library patch out that supposedly improves performance on multi-thread / core mysql installations. Supposedly, the performance bottleneck was in the memory allocation routines, and would manifest itself starting at 4 cores. I'm currently running a dual proc box with 4 cores each, and am having no problems with 5.1.x without the memory library patch.
My biggest suggestion to you would be to install 5.4 and run a benchmark suite against it. If it does the job, go with it. The fragmentation of the various forks does suck, but maybe it will be for the best. At least the founders have money now to fund their own direction... I wouldn't take the advice to follow them blindly though... It took a lot of time and efffort to set up the mysql infrastructure initially, and while it wouldn't hurt to support the team that did it the first time, there is no guarantee they will succeed the second time. So... support them, but don't rely on them until you know you can rely on them.
If you're upgrading your databases anyway, you should consider other FOSS DBMSes, or data management software systems as well, not just MySQL.
For example, if your DB workload is analytics more than transactions, and data is mostly added in batches, you should consider using MonetDB, a columnar DBMSes which is about 1.5-2 orders of magnitude faster than MySQL when doing analytics-only.
Another option is PostgreSQL. It's more transaction-oriented - like MySQL; in fact, they're similar enough to be proper competitors. There are arguments for preferring PostgreSQL and arguments for preferring a MySQL variant.
On DB-engines.com you can find other potentially-relevant DBMSes, FOSS and closed-source/commercial.
I'm working on debugging a slowness issue I've got with running ExpressionEngine (a PHP application) on IIS 7.
I don't think this is actually an issue with ExpressionEngine, but rather an issue with my PHP/MySQL setup.
The problem shows itself thusly:
Go to webite address
IE "spins" for 10-15 seconds, waiting to load. During this time:
processor usage is minimal on the server, and PHP's process is inactive
I see a connection for the site user in MySQL, but the thread is in "sleeping" more.
There is plenty of free memory on the server
pretty much, the server is doing nothing
After 10-15 seconds, I see the connection MySQL run some really quick queries (very fast) and the site loads in under a second.
This is a fairly complex site, but it doesn't make any sense that the whole system is just sitting there waiting for 10 seconds - not processing anything. I'm using FastCGI on IIS7, which seems to be working fine, and to me this seems like some sort of a timeout issue where FastCGI, PHP, or maybe even MySQL is waiting for something, not getting it, and after the timeout occurs, continuing to process.
Anyone had similar experiences?
Thanks!
P.S. - I should also add that the database (MySQL) and PHP are running on the same server.
mysql might be trying to do a reverse DNS on the connection from the web server. if you don't need to filter mysql connections by DNS name, then add skip-name-resolve to your mysql configuration file
Twitter plugins cause lots of issues in CMS systems. usually there is a function that checks if twitter is there/alive. When twitter gets busy, this slows down the system (thats why it can appear as intermittent) Find the twitter plugin, and the routine that checks if twitter is there. Comment out this code and return true (ie. don't ask twitter if it is there, just assume it is)
I have no idea why, but the solution to this was to install PHP 5.3. I had PHP 5.2.10 running, and I guess 5.3 added some extensive optimizations for Windows. Or fixed some other weird problem - who knows.
Actually, after some further digging, it appears the problem was with the Twitter plugin. It waits 25 seconds to come back from Twitter sometime with an error. Maybe this is related to the DNS?