We are building a POS app which is cloud based
We decided to 4GB VPS for MySQL
And 2GB VPS for nodejs
Our client count will be atleast 5000 and atleast 1500 users will update the tables at same time
I checked how much memory does MySQL takes for 1500 concurrent connections
Now my doubt is , even 1500 connections at a time require more than 4GB RAM .
But I have seen lot and lot of DB servers serving lot of request just 4 GB RAM
Please explain me whether max user connections mean number of end users at front end OR something else?
(Not a duplicate question. I searched many Stack overflow questions but still confusing)
As MySQL documentation on max_connections setting says:
The maximum permitted number of simultaneous client connections.
So, this is the maximum number of concurrent mysql connections that you can have. Nothing to do with the number of front end users, this is purely a mysql level configuration.
atleast 1500 users will update the tables at same time
No, they won't. Depending on how badly your application is designed, you may have 5000 concurrent connections - updates will not all happen at the same time.
We decided to 4GB VPS
How did you decide?
Even if you were ask the question properly, it is not something which can be sensibly answered here.
Currently you are not in any position to evaluate any answers you get here - you don't understand the application you are writing nor capacity planning. Your question will be downvoted and closed. Read the linked discussion then go do some modelling and testing. Then go read up on MySQL tuning.
Small sample
Python app needs from 1 to 8MB (stack_size) for one thread
(if will be more connection server throw error stackoverflow (memory error))
1500 x 8MB => 12 GB for python app
Mysql need 1.5 GB per 500 conn +-
1500 conn => 4.5GB (mysql)
Read:
https://www.percona.com/blog/2019/02/25/mysql-challenge-100k-connections/
https://www.percona.com/blog/2014/02/04/16000-active-connections-percona-server-continues-work-others-die/
Related
I can't find the answer anywhere and I've looked all over the internet.
I am about to launch an application (mobile game) that connects to a web-server. I am expecting about 1,000 concurrent users to be on the game at peak hours. EVERY request to the game requires a database request to either get or add new data.
I am currently using a t2.small database from AWS RDS. Apparently the max_connections for this type of database is 150. My questions are:
1) Does this mean there can only be a maximum of 150 concurrent users accessing my application/game? This seems like an EXTREMELY low number for a $30/month database server.
2) How long does each database connection last? If I have 500 people putting in a database request and the limit is 150 connections then it wouldn't be so bad if each connection is only open for say 250ms. If each connection is open for 1 second then people waiting approximately 3 seconds per request is too long.
3) Do I need to reserve a database connection for the admin to be able to login every time?
Any help would be great, thank you.
I think something have gone terribly wrong.
Users shouldn't access MySQL server directly, and you needn't create a new MySQL connection for every individual user. What you should do is to setup a server to handle all requests from users. The MySQL connections only exist between Server and MySQL server, it could be one or several depending on your architecture.
The framework just looks like this:
And this is the simplest one usually.
So, in this case, the limit of 150 connections is just more than enough. Actually 150 connections is adequate for most applications.
in ejabberd.yml we have following line :
##
## Number of connections to open to the database for each virtual host
##
## odbc_pool_size: 10
we are running mysql enabled ejabberd server. MySql server connection limit is 300.
After doing research online (on very limited documentation available) , it seems like increase odbc_pool_size from default 10 mainly affects (decreases) the connecting time of client to server. we have an average of ~1500 users online at one given time instance.
My question : what exact purpose does odbc_pool_size variable serve. How will increasing the pool size affect server connect time / latency ?
UPDATE
Ejabberd Server stats :
8 gb RAM
Dual core
~2000 users (peak hours)
average cpu utilaztion 13.5%
MySql Server stats:
max supported simultaneous connection: 300
write IOPS (QPS) 23.1/sec
read IOPS 1/sec
Memory usage : 2.5/15gb
According to you what will be a good odbc_pool_size for above configuration? (I was thinking of something around 50?)
Like any pool, its size decide of the number of request that can be processed in parallel. If your pool size in 10, only 10 requests can be process in parallel, the other are queued. It means if you have 100 users that tried to connect at the same time, the last one to be process will have to wait for 10 batches of queries to have been processed, thus increasing the latency.
Increasing the pool size can help with latency, up to a point where database cannot cope with more parallelism and global performance will decrease. Good value depends on your database sizing, your use case and your overall architecture.
You need to perform benchmarks and experiment to adapt the sizing to your own case as it really depend on actual traffic patterns.
My simple question:
How can I increase the possible number of connections of my Amazon RDS Database?
I used a parameter group where I set
max_connections = 30000
which seems to work on the first hand, as
SHOW VARIABLES LIKE 'max_connections';
returns the expected.
But when I run a stress test the monitoring metrics always show a maximum number of 1200 connections.
So obviously there have to be other limiting factors, I just don't know.
Any help would be highly appreciated.
My test setup:
1 Load Balancer
8 fat EC2 instances (m4.4xlarge) (which is a bit overdimensioned, but I'm still testing)
1 DB: r3.4xlarge with 140 GB memory, 1 TB storage and 10.000 provisioned IOPS
Test: 30.000 virtual users in 10 minutes making 4 requests each (2 reading the DB, 1 writing it, 1 not using the DB).
Fails after about two minutes because of too many errors (caused by DB timeouts).
Concerning the hardware this setup should be able to handle the test requests, shouldn't it?
So I hope I'm just missing the obvious and there's a parameter which has to be adapted to make everything working.
I would strongly suggest that the first problem is not with the configuration of the server, but with your test methodology and interpretation of what you are seeing.
Hitting max_connections does not initially cause "db timeouts." It causes connection errors, because the server actively rejects excessive connection attempts, with a refusal to negotiate further. This is not the same thing as a timeout.
At what point, during what operation, are the timeouts occurring? Initial connection phase? That's not going to be related to max_connections, at least not directly.
The maximum connections you observe seems like a suspiciously round number and potentially is even derivable from your test parameters... You mentioned 30000 users and 10 minutes and 4 requests... and 30000 × 4 ÷ 10 ÷ 10 = 1200. Yes, I threw in the "10" twice for no particular reason other than 1200 just seems very suspicious. I wonder whether, if you used 15000 users, the number would drop from 1200 to 600. That would be worth investigating.
Importantly, to serve 30000 concurrent users, your application does not need 30000 database connections. If it does, it's written very, very badly. I don't know how you're testing this, but only a naive implementation given the stated parameters would assume 30000 connections should be established.
Equally important, 30000 connections to a single MySQL server regardless of size seems completely detached from reality, except maybe with thread pooling, which isn't available in the version of MySQL used in RDS. If you were to successfully create that many connections, on a cold server or one without a massive thread cache already warmed up, it would likely take several minutes just for the OS to allow MySQL to create that many new threads. You would indeed see timeouts here, because the OS would not let the server keep up with the incoming demand, but it would be unrelated to max_connections.
It would seem like your most likely path at this point would not be to assume that max_connections isn't actually set to the value that it claims, and to scale down your test parameters, see how the behavior changes and go from there in an effort to understand what is actually happening. Your test parameters also need to be meaningful related to the actual workload you're trying to test against.
Thanks to Michael and the hints of a colleague I was finally able to solve this problem:
As Michael already supposed it wasn't caused by the DB.
The answer was hidden in the Apache configuration which I took under examination after DB problems seem to be out of question (finally).
All my eight EC2 instances were limited by MaxRequestWorkers=150 (-> 8*150=1200).
What is obvious for every holiday admin took me day.
At least everything's working now.
One database connection is equal to one web request (in case, of course, your client reads the database on each request). By using a connection pool these connections are pre-created, but they are still used one-per-request.
Now to some numbers - if you google for "Tomcat concurrent connections" or "Apache concurrent connections", you'll see that they support without any problem 16000-20000 concurrent connections.
On the other hand, the MySQL administrator best practices say that the maximum number of concurrent database connections is 4096.
On a quick search, I could not find any information about PostgreSQL.
Q1: is there a software limit to concurrent connections in PostgreSQL, and is the one of MySQL indeed 4096
Q2. Do I miss something, or MySQL (or any db imposing a max concurrent connections limit) will appear as a bottleneck, provided the hardware and the OS allow a large number of concurrent connections?
Update: Q3 how exactly a higher connection count is negative to performance?
Q2: You can have far more users on your web site than connections to your database because each user doesn't hold a connection open. Users only require a connection every so often and then only for a short time. Your web app connection pool will generally have far fewer than the 4096 limit.
Think of a restaurant analogy. A restaurant may have 100 customers (users) but only 5 waiters (connections). It works because customers only require a waiter for a short time every so often.
The time when it goes wrong is when all 100 customers put their hand up and say 'check please', or when all 16,000 users hit the 'submit order' button at the same time.
Q1: you set a configuration paramter called max_connections. It can be set well above 4096, but you are definitely advised to keep it much lower than that for performance reasons.
Q2: you usually don't need that many connections, and things will be much faster if you limit the number of concurrent queries on your database. You can use something like pgbouncer in transaction mode to interleave many transactions over fewer connections.
The Wikipedia Study Case
30 000 HTTP requests/s during peak-time
3 Gbit/s of data traffic
3 data centers: Tampa, Amsterdam, Seoul
350 servers, ranging between 1x P4 to 2x Xeon Quad-
Core, 0.5 - 16 GB of memory
...managed by ~ 6 people
This is a little bit off-topic of your questions. But I think you could find this useful. you don't always kick the DB for each request. a correct caching strategy is almost always the best performance improvement you can apply to your web app. lot of static content could remain in cache until it explicitly change. this is how Wikipedia does it.
From the link you provided to "MySQL administrator best practices"
"Note: connections take memory and your OS might not be able to handle a lot of connections. MySQL binaries for Linux/x86 allow you to have up to 4096 concurrent connections, but self compiled binaries often have less of a limit."
So 4096 seems like the current maximum. Bear in mind that the limit is per server and you can have multiple slave servers that can be used to serve queries.
http://dev.mysql.com/doc/refman/5.0/en/replication-solutions-scaleout.html
I have read every possible answer to this question and searched via Google in order to find the correct answer to the following question, but I am rather a novice and don't seem to get a clear understanding.
A lot I've read has to do with web servers, but I don't have a web server, but an intranet database.
I have a MySQL dsatabase in a Windows server at work.
I will have many users accessing this database constantly to perform simple queries and writting back to it new records.
The read/write will not be that heavy (chances are 50-100 users will do so exactly at the same time, even if 1000's could be connected).
The GUI will be either via Excel forms and/or Access.
What I need to know is the maximum number of active connections I can have at any given time to the database.
I know I can change the number on Mysql Admin however I really need to know what will really work...
I don't want to put 1000 users if the system will really handle 100 correctly (after that, although connected, the performance will be too slow, for example)
Any ideas or own experiences will be appreciated
This depends mainly on your server hardware (RAM, cpu, networking) and server load for other processes if not dedicated to the database. I think you won't have an absolute answer and the best way is testing.
I think something like 1000 should work ok, as long as you use 64 bit MySQL server. With 32 bit, too many connections may create virtual memory pressure - a connection has an own thread, and every thread needs a stack, so the stack memory will reduce possible size of the buffer pool and other buffers.
MySQL generally does not slow down if you have many idle connections, however special commands e.g "show processlist" or "kill", that enumerate every connection will be somewhat slower.
If idle connection stays idle for too long (idle time exceeds wait_timeout parameter), it is dropped by the server. If this is the case in your possible scenario, you might want to increase wait_timeout (its default value is 8 hours)