I have running one big web application and manipulate around 2 GP of data with 1500 users in a node system.i didn't go with server. while all users are in live the server goes down.
so want to know the below the things
How many connections MySQL(opensource) will accept at a time.
You can go to /etc/my.cnf file and check the maximum connection which is allowed.
The default setting for max_connections is 100. However you can update that.
You can change the setting to e.g. 200 by issuing the following command without having to restart the MySQL server
set global max_connections = 200;
If you get a Too many connections error when you try to connect to the mysqld server, this means that all available connections are in use by other clients.
The number of connections permitted is controlled by the max_connections system variable. The default value is 151 to improve performance when MySQL is used with the Apache Web server. (Previously, the default was 100.) If you need to support more connections, you should set a larger value for this variable.
mysqld actually permits max_connections+1 clients to connect. The extra connection is reserved for use by accounts that have the SUPER privilege. By granting the SUPER privilege to administrators and not to normal users (who should not need it), an administrator can connect to the server and use SHOW PROCESSLIST to diagnose problems even if the maximum number of unprivileged clients are connected.
The maximum number of connections MySQL supports depends on the quality of the thread library on a given platform, the amount of RAM available, how much RAM is used for each connection, the workload from each connection, and the desired response time. Linux or Solaris should be able to support at least 500 to 1000 simultaneous connections routinely and as many as 10,000 connections if you have many gigabytes of RAM available and the workload from each is low or the response time target undemanding. Windows is limited to (open tables × 2 + open connections) < 2048 due to the Posix compatibility layer used on that platform.
Related
i have a database that thousands of users need to connect to (via ODBC) for very brief periods (it's a subscription licensing database for a win32 desktop app). They connect, get their approval to run and disconnect).
max_connections is set to 1000 but am not seeing the re-use i would expect server side. i.e. server currently has about 800 processes/connections sleeping (and another 200 connected to real data in other databases on the same server) .... yet a new attempt by a client app was rejected 'too many connections'.
What am i missing?
have increased the max_connections for now to 1500 but if that just means another 500 sleeping connections it's not a long term solution. pretty sure clients are disconnecting properly but am adding some diagnostics to the win32 app just in case.
MariaDB 10.3.11
with MySQL ODBC 5.3 ANSI Driver
It's normal to see a lot of sessions "Sleeping". That means the client is connected, but not executing a query at this moment. The client is likely doing other tasks, before or after running an SQL query. Just like if you are logged into a server with ssh, most of the time you're just sitting at the shell prompt not running any program.
It's up to you to design your clients to wait to connect until they need data, then disconnect promptly after getting their data. It's pretty common in apps that they connect to the database at startup, and remain connected. It's also pretty common in some frameworks to make multiple connections at startup, and treat them as a pool that can be used by multiple threads of the client app. It's your app, so you should configure this as needed.
Another thing to try is to enable the thread pool in the MariaDB server. See https://mariadb.com/kb/en/thread-pool-in-mariadb/
This is different from a client-side connection pool. The thread pool allows many thousands of clients to think they're connected, without allocating a full-blown thread in the MariaDB server for every single connection. When a client has something to query, at that time it is given one of the threads. When that client is done, it may continue to maintain a connection, but the thread in the MariaDB server is reallocated to a different client's request.
This is good for "bursty" workloads by many clients, and it sounds like your case might be a good candidate.
On MySQL, is there any reason not to set max_connections to max value (100000)?
What would be the side effects of such approach?
From the docs:- Too many connections:
The number of connections permitted is controlled by the
max_connections system variable. The default value is 151 to improve
performance when MySQL is used with the Apache Web server.
(Previously, the default was 100.) If you need to support more
connections, you should set a larger value for this variable.
mysqld actually permits max_connections+1 clients to connect. The
extra connection is reserved for use by accounts that have the SUPER
privilege. By granting the SUPER privilege to administrators and not
to normal users (who should not need it), an administrator can connect
to the server and use SHOW PROCESSLIST to diagnose problems even if
the maximum number of unprivileged clients are connected. See Section
12.7.5.30, “SHOW PROCESSLIST Syntax”.
The maximum number of connections MySQL can support depends on the
quality of the thread library on a given platform, the amount of RAM
available, how much RAM is used for each connection, the workload from
each connection, and the desired response time. Linux or Solaris
should be able to support at 500 to 1000 simultaneous connections
routinely and as many as 10,000 connections if you have many gigabytes
of RAM available and the workload from each is low or the response
time target undemanding. Windows is limited to (open tables × 2 + open
connections) < 2048 due to the Posix compatibility layer used on that
platform.
Increasing open-files-limit may be necessary. Also see Section 2.5,
“Installing MySQL on Linux”, for how to raise the operating system
limit on how many handles can be used by MySQL.
If you try to make more connection then what your system can support then eventually all those connections will be closed by default as your system configuration will not support it.
Is possible to have multiple websites (800 total) hosted in a dedicated server to connect and use the tables of a unique mysql database?
It should work normally:
Maximum concurrent connections to MySQL
You'd probably run out of memory, file handles, and network sockets,
on your server long before you got anywhere close to that limit.
But keep in mind that you are using one MySQL Server which can have lower limits in the settings.
So you should be using caching solutions to lower the amound of requests.
We are using SQL Server 2008 express. At the moment in the connection string we have the default pool size of 100. At peak times we are receiving time-out error when processing transactions into the DB.
The situation has improved since we have set the pool-size to 200. My question is what is the maximum pool-size you can have in express edition? This edition is run with a GB of RAM.
Also we have the full-blown version of SQL. Can we specify a larger pool-size with the full blown version and if we do increase the pool size, would we have to increase such things as RAM etc to see the benefit?
Thanks in advance,
Matt.
Whenever you have such questions you should go to Maximum Capacity Specifications for SQL Server. Strictly speaking Connection pool size is a client setting but the engine needs to keep open a connection so you can hit server side limits. The Connections per client values is Maximum value of configured connections which implies is configurable through a Server Configuration Option and ideed it is, see user connections:
The user connections option specifies the maximum number of
simultaneous user connections that are allowed on an instance of SQL
Server. The actual number of user connections allowed also depends on
the version of SQL Server that you are using, and also the limits of
your application or applications and hardware. SQL Server allows a
maximum of 32,767 user connections. Because user connections is a
dynamic (self-configuring) option, SQL Server adjusts the maximum
number of user connections automatically as needed, up to the maximum
value allowable.
One of the ERP applications I worked with was configured in such a way that there was only 1 user (for example USER A) who connected to the database. Any user of the application (workforce was in the thousands) who logged on to the system and tried to do anything was in effect calling USER A to connect to the database and execute queries for him. The database was Oracle.
I was wondering how to achieve a similar thing with mySQL. I have a web application built with php and mySQL database. I expect different people to query the database via the web. Currently when a user opens up the web page, a connection to the database is made via a single db user. At the end of the query, I close the connection. However the database has a maximum user connection of 10 which in my understanding means one user can only establish a max of 10 connections. I do not want to have to create several users for all the people who try to use my application (I do not even know the number of people who will use the application and I do not believe this will be a scalable solution)
You should look for a db connection caching mechanism as a component for either your web server or your programming language. Such a mechanism will reuse connections transparently for you.
If the database connection is refused return HTTP error 502. If connections are closed at the end of each pageload they should only last ~100ms, so concurrent connections will be low for most situations.
Should you need to adjust it, edit my.cnf to increase concurrent connections:
max_connections = 150
max_user_connections = 150
If traffic is very high you can enable persistent MySQL connections in PHP, or cache your content so not to hammer the database.
Hope that helps!