I am trying to build a desktop application and ideally connect to AWS RDS (MySQL) database. My use case is that at least 5 users will be using this app at the same time and be writing into the database at the same time likely. My question is does RDS handle concurrency issue? Or do I need to write some script to handle this in the desktop app?
Thank you!
RDS is a managed service, you don't need to worry about concurrency or other configurations, unless and until you need any custom behavior.
The maximum number of simultaneous database connections varies by the DB engine type and the memory allocation for the DB instance class. The maximum number of connections is set in the parameter group associated with the DB instance, except for Microsoft SQL Server, where it is set in the server properties for the DB instance in SQL Server Management Studio (SSMS).
You can read here more.
Related
I have an RDS instance hosting a mySQL database. Instance size is db.t2.micro
I also have an ExpressJS backend connecting to the mySQL RDS instance via a connection pool:
Additionally i have a mobile app, the client, feeding off the ExpressJS API.
The issue i'm facing is, either via the mobile app or via Postman, there are times where i get a 'Too many connections' error and therefore several requests fail:
On the RDS instance. On current activity i sometimes get 65 connections, showing it's reaching the limit. What i need clarity on is:
When 200 mobile app instances connect to the API, to the RDS instance, does it register as 200 connections or 1 connection from ExpressJS?
Is it normal to be reaching the RDS instance 65 connection limit?
Is this just a matter of me using db.t2.micro instance size which is not recommended for prod? Will upgrading the instance size resolve this issue?
Is there something i'm doing wrong with my requests?
Thank you and your feedback is appreciated.
If your app creates a connection pool of 100, that's the number of database connections it will try to open. It must be lower than your MySQL connection limit.
Typically connection pools open all the connections for the pool, so they are ready when a client calls the http API. The connections might normally be running no SQL queries, if there are not many clients using the API at a given moment. The database connections are nevertheless connected.
Sort of like when you ssh to a remote linux server but you just sit there at a shell prompt for a while before running any command. You're still connected.
You asked if a db.t2.micro instance was not recommended for production. Yes, I would agree with that. It's tempting to use the smallest instance possible to save money, but a db.t2.micro is too small for anything but light testing, in my opinion.
In fact, I would not use any t2 instance for production, regardless of size. The t2 type uses "burstable" performance. This means it can provide only brief periods of good performance. Once the instance depletes its performance credits, they recharge slowly, and while they recharge, the performance of that instance is very low. This is okay for testing, but not for production, if you expect to provide consistent performance at any time.
I want to test Node and Deno and try to redirect users via proxy to one MySQL DB.
How will it impact the database?
Can some timestamp conflicts via CRUD operations arise or does MySQL have some mechanism to cope with connections from multiple servers?
What about performance or memory footprint of the database in RAM? Will it be occupying the same amount of space as if there was only one server requesting the database to CRUD something?
What would happen if I added another server that will connect to the DB, for example, java or Go server?
It will virtually have no impact on the database other than having any other concurrent processes connecting to it.
This is not a deno issue but rather a database issue.
The exact same problems can happen even with your current single Node.js instance, because the nature of all systems these days is concurrent/parallel.
You might as well replace the Deno app with another Node.js instance, Java, etc. Or even your current Node.js app.
Data in a database can change once you loaded it to the client, and it is up to you to implement the code that will handle such scenarios.
The fact that MySQL is not "ACID" is neither negative nor relevant in and of itself because it is doesn't have context.
If you need complete absolute integrity on a registry make sure you lock it when you select it, but there will be a trade off.
I have a Rails 4.x application running on server A and MySQL on server B.
Using ab to do a load test of my API calls I notice that the MySQL server is showing CPU activity. So I go back to the code and check, but no SQL statements are triggered, to be sure I also deactivate all before filters, but still the MySQL server shows CPU load.
I went to MySQL and run
show processlist;
but that also shows no active SQL statements
Why would there be load on the DB server?
A Rails application initializes connection pools to the configured database on app load and also loads basic schema data for each ActiveModel defined to populate runtime mappings from the DB to instances of that model.
These connections/queries will happen as soon as you have loaded the application and running traffic.
If this is not what is responsible for the activity on your database server, you will need to use other tools to see what is responsible. For example, NewRelic's system monitoring tools are great for snapshotting CPU/memory usage over time correlated to what processes were running. This will help you rule out MySQL itself using resources vs. other things running on the DB server.
According to this article, storage engines like Innodb may have their own per thread and/or global memory allocations which is probably accounting for the CPU overhead. If this is a stock (non-tuned) MySql install, you're probably just seeing baseline CPU activity. The article mentions a number of places to look that might indicate areas that can be tuned to reduce this footprint.
I know it is very basic. But want to clear some concept of mysql connections. I have following scenario.
Db server and web servers are on different locations.
Web server is running an article based web site.
Articles data is stored in db server.
Web server is delivering 100 articles/pages per second.
My questions are as follows:
Single connection between web server and db server can handle it ?
How many connections would be created by default ?
If I suppose connections as pipes, what is i/o capacity of each connection ?
Is there any relationship between db server's RAM, processor, OS and number of connections ?
Thanks in advance
Single connection between web server and db server can handle it?
A single connection can pass several requests, but not at the same time, and only for a single client process. So the best scaling approach might be one connection pool per web server process, where threads can obtain an established connection to perform their request, and return the connection to the pool once they are done.
How many connections would be created by default ?
Depends on the client language, the MySQL connector implementation, the frameworks you use, the web server configuration, and probably a bunch of other details like this. So your best bet is to simply look at the list of network connections to the MySQL service, e.g. using lsof or netstat.
If I suppose connections as pipes, what is i/o capacity of each connection ?
The limiting factor will probably be shared resources, like the network bandwidth or processing capabilities at either end. So the number of connections shouldn't have a large impact on the data transfer rate. The overhead to establish a connection is significant, though, which is why I suggested reducing the number of connections using pooling.
Is there any relationship between db server's RAM, processor, OS and number of connections ?
Might be, if some application makes choices based on these parameters, but in general I'd consider this rather unlikely.
One of the ERP applications I worked with was configured in such a way that there was only 1 user (for example USER A) who connected to the database. Any user of the application (workforce was in the thousands) who logged on to the system and tried to do anything was in effect calling USER A to connect to the database and execute queries for him. The database was Oracle.
I was wondering how to achieve a similar thing with mySQL. I have a web application built with php and mySQL database. I expect different people to query the database via the web. Currently when a user opens up the web page, a connection to the database is made via a single db user. At the end of the query, I close the connection. However the database has a maximum user connection of 10 which in my understanding means one user can only establish a max of 10 connections. I do not want to have to create several users for all the people who try to use my application (I do not even know the number of people who will use the application and I do not believe this will be a scalable solution)
You should look for a db connection caching mechanism as a component for either your web server or your programming language. Such a mechanism will reuse connections transparently for you.
If the database connection is refused return HTTP error 502. If connections are closed at the end of each pageload they should only last ~100ms, so concurrent connections will be low for most situations.
Should you need to adjust it, edit my.cnf to increase concurrent connections:
max_connections = 150
max_user_connections = 150
If traffic is very high you can enable persistent MySQL connections in PHP, or cache your content so not to hammer the database.
Hope that helps!