I can't find the answer anywhere and I've looked all over the internet.
I am about to launch an application (mobile game) that connects to a web-server. I am expecting about 1,000 concurrent users to be on the game at peak hours. EVERY request to the game requires a database request to either get or add new data.
I am currently using a t2.small database from AWS RDS. Apparently the max_connections for this type of database is 150. My questions are:
1) Does this mean there can only be a maximum of 150 concurrent users accessing my application/game? This seems like an EXTREMELY low number for a $30/month database server.
2) How long does each database connection last? If I have 500 people putting in a database request and the limit is 150 connections then it wouldn't be so bad if each connection is only open for say 250ms. If each connection is open for 1 second then people waiting approximately 3 seconds per request is too long.
3) Do I need to reserve a database connection for the admin to be able to login every time?
Any help would be great, thank you.
I think something have gone terribly wrong.
Users shouldn't access MySQL server directly, and you needn't create a new MySQL connection for every individual user. What you should do is to setup a server to handle all requests from users. The MySQL connections only exist between Server and MySQL server, it could be one or several depending on your architecture.
The framework just looks like this:
And this is the simplest one usually.
So, in this case, the limit of 150 connections is just more than enough. Actually 150 connections is adequate for most applications.
Related
What is meaning of DB Connections(Count) report on AWS RDS?
I have gone through their documentation but didn't find my answer there.
I am quite confused with DB Connection report on my AWS. I can see their only 1 connection available but I am sure that there are always 100-150 concurrent users on my website on different pages which users database operation.
As the user concurrency is 100-150 then why it shows me only 1 connection on report.
Note : My website is working well on good performance.
The database connection count is exactly what it sounds like: "The number of database connections in use."
It's a count, so it shouldn't be summed. Maximum or averaging are recommended. It may be registering low because you have a very efficient database pool, have server-level caching, or are looking at the wrong database in your statistics.
I have been researching this for a while but got no convinced answer.
From mysql tutorial, the default connections number is less than two hundred, and it says max_connection_num can be set to 2000 in Linux box as long as you have enough resource. I think this number is far from enough in real world deployment as there might be millions people visit your website at the same time.
There are couple of articles talking about how to optimize to reduce time cost by each query. But none of them tells me how this issue is root caused. I think there must be some mechanism like queue to prevent massive connections from happening simultaneously. otherwise you will finally get "too connection" exception.
anyone has some expertise in this area? thank you.
There are several options.
Connection pooling
As you mentionned: queuing. If too many clients connect at the same time, then the application layer should handle this exception, put the request to sleep for a short period of time and try again. Requests lasting more than a couple of seconds should usually be banned in such a high traffic environment.
Load balancing through replication and/or clustering
Normally, your application is supposed to reuse connections already established. However, the language you chose to implement your application introduces limitations. If you use Java or .Net you can have pool of connections. For PHP it is not the case, you can check this discussion
If you exceed the max_connection_num, you do get a too many connections error. But if you really have 1 million users at your web server at the exact same time, you can't handle that with one server anyway, 1 million concurrent connections really requires a very big farm to handle.
However, the clients to your database is a webapp, that webapp usually connects to the database through abstractions called a connection pool, which does limit the number of connections to the database on the client side as long as all the database connections goes through that same pool.
I'm running a small server with MySQL installed that will have 10 to 20 clients remotely connected to it (direct connection) flooding it with data. The data could be coming in at 200-300 writes per second. Possibly more/less.
I'm wondering how well MySQL can handle this. The 'server' is an off the shelf computer. Nothing special/enterprise.
I don't know whether I should write a small async server that will sit between MySQL and the clients to buffer the data. Every 1000 rows or so, it can dump the data into MySQL.
Is this necessary? Or can MySQL handle this load?
Facebook makes extensive use of MySQL at commit rates higher than yours, so it's certainly possible. I don't know the exact commit rate at which it stops being easy.
Sorry for the newb factor, but I was reading about "Too many connections" to mysql.
http://dev.mysql.com/doc/refman/5.5/en/too-many-connections.html
How are "simultaneous client connections" quantified in mysql?
For example if 20 million people are on gmail (let's say they use mysql with only 1 table to store everything just for sake of example) and all those people simultaneously all click on an email to open up, does that mean there are 20 million simultaneous connections or just one connection since all the users are connecting to the same table?
EDIT: I'm trying to understand what the term 'client' means. Is a 'client' someone who is using the application, or is a 'client' the part of the application (ex. php script) that is connecting to the database?
When a visitor goes to your website and the server-side script connects to the database it is 1 connection - you can make as many queries as necessary during that connection to any number of tables/databases - and on termination of the script the connection ends. If 31 people request a page (and hence a db connection) and your limit is 30, then the 31st person will get an error.
You can upgrade server hardware so MySQL can efficiently handle loads of connections or spread the load across multiple database servers. It is possible to have your server-side scripting environment maintain a persistent connection to MySQL in which case all scripts make queries through that single connection. This will probably have adverse effects on the correct queuing of queries and their order to maintain usable speeds under high load, and ultimately doesn't solve the CPU/memory/disk bottlenecks with handling large numbers of queries.
In the case of a webmail application, the query to check for new messages runs so fast (in the milliseconds) that hitting server limits isn't likely unless it's on a large scale.
Google's applications scale on a level previously unheard of. Check out the docs on MapReduce, GoogleFS, etc. It's awesome.
In answer to your edit - anything that connects directly to MySQL is considered a client in this case. Each PHP script that connects to MySQL is a client, as is the MySQL console on the command line, or anything else.
Hope that helps
The connections mentioned are server connection. Every client has one or more. For example if your php script connects mysql, there may be more web requests at a time and thus more connections to db.
Sometimes you can ran out of them, because they are not closed properly after they become useless.
And I thing Gmail is stored different way than in one large mysql db :]
I have read every possible answer to this question and searched via Google in order to find the correct answer to the following question, but I am rather a novice and don't seem to get a clear understanding.
A lot I've read has to do with web servers, but I don't have a web server, but an intranet database.
I have a MySQL dsatabase in a Windows server at work.
I will have many users accessing this database constantly to perform simple queries and writting back to it new records.
The read/write will not be that heavy (chances are 50-100 users will do so exactly at the same time, even if 1000's could be connected).
The GUI will be either via Excel forms and/or Access.
What I need to know is the maximum number of active connections I can have at any given time to the database.
I know I can change the number on Mysql Admin however I really need to know what will really work...
I don't want to put 1000 users if the system will really handle 100 correctly (after that, although connected, the performance will be too slow, for example)
Any ideas or own experiences will be appreciated
This depends mainly on your server hardware (RAM, cpu, networking) and server load for other processes if not dedicated to the database. I think you won't have an absolute answer and the best way is testing.
I think something like 1000 should work ok, as long as you use 64 bit MySQL server. With 32 bit, too many connections may create virtual memory pressure - a connection has an own thread, and every thread needs a stack, so the stack memory will reduce possible size of the buffer pool and other buffers.
MySQL generally does not slow down if you have many idle connections, however special commands e.g "show processlist" or "kill", that enumerate every connection will be somewhat slower.
If idle connection stays idle for too long (idle time exceeds wait_timeout parameter), it is dropped by the server. If this is the case in your possible scenario, you might want to increase wait_timeout (its default value is 8 hours)