I have attempted to find the answer here and via Google on how to control connections for a linked server ODBC connection.
Overview
I have a linked server from SQL Server 2014 to MySQL for the purposes of extracting data for our data warehouse. I've queried the database quite a few times without issue. Then yesterday, suddenly the query to read from the table is slow, and then I get reports that the application using this MySQL database are getting a "too many connections" error.
Details
The following query selects the data from MySQL and inserts to the SQL Server table.
INSERT INTO tmpCustomers
(fieldlist)
SELECT
myc.contact_id,
myl.franchise_id,
myl.lead_source,
LEFT(RTRIM(myc.first_name) + ' ' + RTRIM(myc.last_name),100) AS Name,
myc.first_name,
myc.last_name,
myc.company,
myc.Email,
myc.primary_phone,
myc.home_phone,
myc.mobile_phone,
myc.work_phone,
myc.fax,
myc.address1,
myc.Address2,
myc.City,
myc.[state],
myc.zip_code,
myc.created_date,
myc.updated_date
FROM [MYSQLDB]...[franchise] myf
INNER JOIN [MYSQLDB]...[leads] myl
ON myl.franchise_id = myf.franchise_id
INNER JOIN [MYSQLDBE]...[contact] myc
ON myc.contact_id = myl.contact_id
This query returns about 200K rows of data, and will grow. The MySQL database is used by our customer base, and this is a back-end process to pull data into our data warehouse.
The query has been working without issue over the past week of testing, until yesterday, where it caused our MySQL support to restart the MySQL server twice.
The ODBC setup was done using the "mysql-connector-odbc-5.3.6-win64.msi" version. I don't find any settings there to limit the number of connections. ODBC does show "Allow multiple statements", which this is not. It also has "Enable automatic reconnect", which I can't imagine why for a single query would be needed.
Summary
I can't afford to stop customers from connecting, and need to disable the process from using too many connections when doing the import.
Any input on this would be greatly appreciated.
Thanks
KDS
Update: 2016-Oct-05
AWS server - M3.xlarge
4 CPU
15 GiB
2 40 GiB SSD drives
It's better to optimize the MySQL server if you can't afford to stop customers from connecting.
With this much information, it hard to optimize or suggest something for MySQL optimization.
https://dev.mysql.com/doc/refman/5.5/en/too-many-connections.html
Better to update your configuration file. Or max_connections limit and InnoDB variable if you are using innoDB. And RAM also.
Can you update the above information in question section.
I'm going to mark this as answered as it's been about a year and no real solution to it. The issue was locks on the MySQL server as the SQL Server linked server was reading the data. SQL Server arguments like NOLOCK had no impact on resolving this.
So, what was done was to take a backup of the MySQL database nightly and restore it to a separate database that we linked to for SQL Server, and process the data from there. The reads are usually done in a matter of a minute or two. SQL Server was still putting a lock on the MySQL table, and users then started to stack multiple connections until all the connections to MySQL were used up.
So, since I only needed the data for reporting purposes daily, this separate database copy worked, but I don't know of any other fix to this.
Thanks
KD
Mysql Query Results
some times a get max_user_connections exceeds and ispite of this i can make a connection and other thing is how many currently connections being used . how do i know that please help.
The ‘max_used_connections’ variable defines the total number of connections that have been in use simultaneously since the server was started.
This variable is different when compared to the variable ‘max_connections’.
The system variable ‘max_connections’ defines the maximum permitted number of simultaneous connections by a client.
You can find the maximum connections after which the connections will be refused by using the formula (Max_used_connections/max_connections). You can refer various monitoring services like MONyog which defines these terms well.
I'm running SQL Server 2012 and have setup a Linked Server connection to a Linux MySQL instance via the latest MySQL ODBC drivers. I'm a bit disappointed by the time taken to return results of a fairly straightforward 'Select' query.
select * from OPENQUERY(LinkedServer, 'select * from mysqltable')
The table has approximately 150,000 rows and takes about 70 seconds to return the results to SSMS. Conversely if I query the table via a MySQL Client App (In this case Navicat Essentials, on the same machine as SQL Server) the query executes in about 1 second.
I know that Linked Servers and ODBC will be slower but I'm surprised by this performance hit, particularly with such a straight forward query.
I've tried both the Unicode and ANSI drivers and the performance is similar. The MySQL DB is UTF-8 CharSET and Coalition and the table is InnoDB. I've tried explicitly selecting columns rather than * also. Again no difference.
Has anyone experienced this before and got any tips for speeding up the performance or is this fairly normal?
Thanks in advance.
In linked server
I do not think there is a possibility to improve significantly
But you can try through SSIS
And use bulk insert.
Is there any way, and if, how to set arbitrary maximum number of:
query execution time
open connection number
simulatanous data readers
for specific database user in SQL Server 2008?
As for the user he'll be only operating on views with readonly access.
Is there anyway to prioritize or throttle a query at all in MySQL?
I'm a DBA on a server that sees a lot of unoptimized queries come into the server and they just destroy the CPU. I'm looking at throttling certain users that hit on the database in poor fashion.
Clarification:
I realize that MySQL has facilities built in to limit number of queries, connections, etc. But those aren't really the issue, it's that once in a blue moon a user will send an unoptimized query, and I'd need to time it out or something similar.
http://dev.mysql.com/doc/refman/4.1/en/user-resources.html
Starting from MySQL 4.0.2, you can limit access to the following server resources for individual accounts:
The number of queries that an account can issue per hour
The number of updates that an account can issue per hour
The number of times an account can connect to the server per hour
Additional Info -
MySQL does not have a query timeout value, or any other built-in way to throttle an individual query. However, it is pretty trivial to write a script that will kill long-running queries. You could even capture the user and query so that you can beat the offending user/programmer later.
Here is an example of one using PERL.
I know that phpMyAdmin has an area for Max # of Queries per hour, Max # of Updates per hour, Max # of Connections per hour, and Max # of User Connections per user. There are probably other ways to set these values other than phpMyAdmin