Newbie question....sorry
I have a simple mysql database running in our Intranet (windows server) which >20 people connect to for searching/inserting records, etc
This is done with a simple Excel GUI.
Process is:
Search Strings are typed in excel cells
VBA opens connection to Mysql and query is run
Results retrieved are put on excel Connection to
mysql closed with VBA
The above process takes in general 0-2 seconds. Records retrieved <100.
Everthing runs fine so far.
In order to be able to connect more people in future, I would like to have some feedback on whether it is ok to continously connect and disconnect from mysql in the way I am doing.
Can it cause some type of crash/memory leaks, etc ??
Is there some better way to do this?
I am hoping to get <2000 users, but I understand the more users connected, worse it is.
By disconnecting after each search/insert, I am hoping to keep the number of live connections as low as possible.
thanks for your input
This constant connecting and disconnecting is an expensive process.
A better way would be to use server-side scripting to manage your connections. This way you would have a single persistent connection to each server, and the users will execute their queries through the single connection. You will also need to implement some sort of job queue for execution.
Related
I am using this
var mysql = require('mysql');
in my node.js app. I am interested to make my app perform the fastest. I have many functions that connect to SQL. There is 2 approaches I am familiar with
For every request, I make a new connection and then execute the query and the close the connection.
Open the connection and make it a global variable, and then never close it. Then for every request that comes in, it just uses the opened connection saved globally.
Which is generally better to use? Also for number 2, if the server closes unexpectedly, then the sql connection doesn't close. Is that bad?
Thanks
Approach 2 is faster, but to avoid the potential problem of connections dropping without unexpectedly, you'll have to implement testing mechanism for every segment that queries the database (ex: count the number of returned rows).
To take this approach further, you can define connections bank or pool. Where you can deal with connection testing and distributions. The basic idea is to have many connections to the database and just inject healthy connections to consumers (functions, or objects that query the database). As Andrew mentions in the comments You can check this question: node.js + mysql connection pooling
Since the database is an essential asset to a project, if this is not a homework or learning project, it might not be a bad idea to explore 3rd party libraries, where a lot of the connections and security details is covered and automated.
What is the effect of multiple tadoconnection?
Here is what I did :
I put a TADOConnection to almost every form in my application.
Those TADOConnection will connect to the database(MySQL) everytime I create an instance of a form.
In an average use of the application, about 15 forms will be used(15 tadoconnections connected to the database). So far my application is running smooth. But yesterday, a user complained of an error "MySQL has gone away".
I've encountered that error in the past and it was because the data is too large, or hardware problem. But today, the data is not big and the hardware is in excellent condition. By the way, the connection is local. Does the multiple tadoconnection produced the error?
The effect of multiple ADOConnections is that you, open multiple independent Session in the Database. I wouldnt recommend your solution, in consideration of Transactionmanagement and table locking
Server has gone away: http://dev.mysql.com/doc/refman/5.1/en/gone-away.html
Sorry for the newb factor, but I was reading about "Too many connections" to mysql.
http://dev.mysql.com/doc/refman/5.5/en/too-many-connections.html
How are "simultaneous client connections" quantified in mysql?
For example if 20 million people are on gmail (let's say they use mysql with only 1 table to store everything just for sake of example) and all those people simultaneously all click on an email to open up, does that mean there are 20 million simultaneous connections or just one connection since all the users are connecting to the same table?
EDIT: I'm trying to understand what the term 'client' means. Is a 'client' someone who is using the application, or is a 'client' the part of the application (ex. php script) that is connecting to the database?
When a visitor goes to your website and the server-side script connects to the database it is 1 connection - you can make as many queries as necessary during that connection to any number of tables/databases - and on termination of the script the connection ends. If 31 people request a page (and hence a db connection) and your limit is 30, then the 31st person will get an error.
You can upgrade server hardware so MySQL can efficiently handle loads of connections or spread the load across multiple database servers. It is possible to have your server-side scripting environment maintain a persistent connection to MySQL in which case all scripts make queries through that single connection. This will probably have adverse effects on the correct queuing of queries and their order to maintain usable speeds under high load, and ultimately doesn't solve the CPU/memory/disk bottlenecks with handling large numbers of queries.
In the case of a webmail application, the query to check for new messages runs so fast (in the milliseconds) that hitting server limits isn't likely unless it's on a large scale.
Google's applications scale on a level previously unheard of. Check out the docs on MapReduce, GoogleFS, etc. It's awesome.
In answer to your edit - anything that connects directly to MySQL is considered a client in this case. Each PHP script that connects to MySQL is a client, as is the MySQL console on the command line, or anything else.
Hope that helps
The connections mentioned are server connection. Every client has one or more. For example if your php script connects mysql, there may be more web requests at a time and thus more connections to db.
Sometimes you can ran out of them, because they are not closed properly after they become useless.
And I thing Gmail is stored different way than in one large mysql db :]
I have developed a windows service using Delphi 2007. It connects to a remote MySql database via the internet using TAdoConnection and TAdoQuery. I have retained the default value of 30 seconds for CommandTimeout property. I also create the connection/query objects on each new query and free them when done (i.e. I don't create the database connection at startup and keep it open).
Every once in a while the service stops and the event viewer shows "Lost connection to MySQL server during query". I have everything wrapped in exceptions. My suspicion is that there is a drop in the network while the query is executing.
Anyone have any resolution/ideas?
What triggers windows to shutdown the service?
Also, I have the service "Recovery" set to restart the service but this never happens.
My next step will be to start logging when each query starts and compare this to the date/time of the shutdown. Because as of now I don't know how log this is.
This is may be not a direct answer, but I had same problem few days ago, and I have the mysql on local server, and I connect using Mydac components.
After many tries, I found the problem came from one table that has BLOB fields, I tried to query on the table like
select * from table
And I got this problem after the query fetch around 1600 rows, after few inspection I found the problem came from few records and seems they corrupted. so when I do query like
Select * from my table where id not
between 1599 and 1650
I had this problem, if I removed Not from they query which fetch only 51 records I got the error, which means there's some records are corrupted, also I did mysqlcheck but it didn't fix the problem, and I tried some other check tools and I got same result, I didn't tried to delete these records, because I need why this problem happen, but I got busy with other things so I left the server for a while.
BTW, I used MySql Query browser for trying to do the queries, because other tools will give me the errors without showing how many records fetched before mysql instance terminated unexpectedly.
I'm using R to read some data from a MySQL database using the RODBC package. The data is then processed and some results are sent back to the database. The problem is that the server closes the connection after about a minute due to inactivity, which is the time needed to process the data locally. It's a shared server, so the host won't bump up the timeout time.
I think there are two possibilities to get around this:
Open a connection before every database transaction and close it immediately after
Send some small 'ping' command to the server every 30 seconds or so to let the server know that I'm still there.
I can implement the first fairly easily, but it seems pretty slow to constantly open and close connections. Does anyone know an efficient command for the second? Or is a better way altogether?
The first solution is the one I prefer. It's really hard to do the latter with a single threaded program like R. If R is busy running analysis there's no way for it to handle the ping. Unless you are doing hundreds of reads/writes the method of opening and closing the connection should not introduce an extreme amount of overhead.