I have been experiencing an intermittent SQL connection problem that I can't resolve. First of all I will explain the setup:
Windows 2008 R2 Standard SP1 (and all patches delivered through windows update):
This server runs IIS 7.5
The ASP.NET MVC 4 Web App runs within it's own App Pool
The App Pool identity is used to set permission on the folder containing the code
The Web App passed a username and password in the connection string - server xxx; database=xxx; Persist Security Info=True; User ID=xxx; Password=xxx; Pooling=False; Network=DBMSSOCN;
elmah used to log errors
Windows 2008 R2 Standard SP1 (and all patches delivered through windows update):
This server runs SQL Server 2008 R2 Web Edition SP2
TCP/IP is the only enabled Client Protocol
These are VM's hosted by our cloud provider. The cloud provider has investigated, from an infrastructure point of view, but cannot find any problems.
The error messages generated are as follows:
System.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: SQL Network Interfaces, error: 9 - Associating port with I/O completion mechanism failed)
I've been unable to find any information on error code 9.
System.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: TCP Provider, error: 0 - An operation was attempted on something that is not a socket.) ---> System.ComponentModel.Win32Exception (0x80004005): An operation was attempted on something that is not a socket
While these errors are generated, IIS will show the 'friendly error' page. It usually takes around 15 minutes for the issueto resolve itself or an IISReset. When this issue first occurred I downloaded a SQL Load Generator from codeplex and ran test up to 300 simultaneous connections with multiple query's and everything was ok. I also ran another test while the web app was generating the SQL exceptions - all connections from the load test succeeded. This really confused me!?!
No Event Logs are created during the outage.
To me the error message indicates a problem connecting to the SQL box, however the load test run during the most recent outage contradicts this - yet IIS is still serving content (configured error message).
I realize that this is a difficult question, however I would be very grateful for any advice regarding techniques to find the cause of the error. Any information on what error 9 relates to would also be helpful.
Related
I have a report that is connecting to a new server.
When I run the report in Visual Studio, it runs fine but when I deploy the report, the new data source is getting an error:
A network-related or instance-specific error occurred while
establishing a connection to SQL Server. The server was not found or
was not accessible. Verify that the instance name is correct and that
SQL Server is configured to allow remote connections. (provider: Named
Pipes Provider, error: 40 - Could not open a connection to SQL Server)
The Type is MS SQL Server and the Connection String is basic (Data Source=;Initial Catalog=). When I hit Test, it takes about a minute to come up with the error. I also get the error whether using stored credentials or Windows Integrated.
I have recreated the data source by hand without any difference.
I have went through the Microsoft troubleshooting without success. I have numerous other similar data sources but have never had this issue.
MS Docs: Troubleshoot connecting to the SQL Server Database Engine
Don't know what else to try. Any suggestions to fix this?
SSRS 2015 - SQL Server 13.0.5820.21
Here is my latest effort at revising this question. But this time, I am trying to follow the good counsel given by Oded in his article Getting good answers on StackOverflow.
I need to find out how I can determine the root cause for the following error:
Communication link failure
TCP Provider: The specified network name is no longer available
From time to time, I am seeing this error when running a set of SSIS packages. This error can occur when one to many packages are run from:
A SQL Server Agent Job
A batch file
In debug mode from BIDS
The full error message I see is as follows:
SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005.
An OLE DB record is available. Source: "Microsoft SQL Server Native Client 10.0" Hresult: 0x80004005 Description: "Communication link failure".
An OLE DB record is available. Source: "Microsoft SQL Server Native Client 10.0" Hresult: 0x80004005 Description: "TCP Provider: The specified network name is no longer available.
".
SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005.
An OLE DB record is available. Source: "Microsoft SQL Server Native Client 10.0" Hresult: 0x80004005 Description: "Protocol error in TDS stream".
An OLE DB record is available. Source: "Microsoft SQL Server Native Client 10.0" Hresult: 0x80004005 Description: "Communication link failure".
An OLE DB record is available. Source: "Microsoft SQL Server Native Client 10.0" Hresult: 0x80004005 Description: "TCP Provider: An existing connection was forcibly closed by the remote host."
This is an overview of how I have designed the ETL process:
Two servers
Both are virtual machines
The SSIS packages run on an application server
The SQL Server database lives on a database server
I use an OLE DB connection manager to connect from the SSIS package on the application server to the SQL Server database on the database server.
The packages run as a file system deployment on the application server and not as a database deployment on the database server.
The main reason for this is that the ETL is integrated with a set of tools no found on and drives not accessible to the database server. These tools include Apex Data Loader for Salesforce and pgAdmin III.
So far I cannot consistently reproduce this error. However, this is what I have observed:
Failure occurs more frequently during regular business hours
Failure occurs less frequently during off hours
For about a two hour period on a Friday morning I was able to successfully reproduce the error on a specific package.
The error occurred during a large data flow if a child package call that precedes the large data flow was enabled.
The error did not occur during the same large data flow if the child package call that precedes the large data flow was disabled.
The child package in question calls back to the database to retrieve a tiny amount of information for use in an email body and then sends the email.
It feels like maybe a resource limit is being exceeded?
Maybe a connection limit?
I am wondering what tools I should be using to try and determine the root cause of the error.
Technical details about the two servers involved are listed below:
SQL Server and Database Server info:
Microsoft SQL Server 2008 R2 (SP1) - 10.50.2500.0 (X64) Jun 17 2011 00:54:03 Copyright (c) Microsoft Corporation Enterprise Edition (64-bit) on Windows NT 6.1 (Build 7601: Service Pack 1) (Hypervisor)
SSIS info:
Microsoft Visual Studio 2008 Version 9.0.30729.1 SP
Microsoft .NET Framework Version 3.5 SP1
Application Server info:
OS name: Microsoft Windows Server 2008 R2 Standard
Version: 6.1.7601 Service Pack 1 Build 7601
I have researched the error message online and found these, but would really like to get an expert's insight before proceeding:
How to Disable TCP Chimney, TCPIP Offload Engine (TOE) or TCP Segmentation Offload (TSO).
Using Netsh Commands to Enable or Disable TCP Chimney Offload
Any help is appreciated.
Thanks
UPDATE:
Further testing shows that this is not "an SSIS thing" as the same error is seen at the same rate when using SQL Server Management Studio. The complexity of the query does not make the error more or less likely. In an attempt to resolve, we have tried one fix (below):
#1 How to Disable TCP Chimney, TCPIP Offload Engine (TOE) or TCP Segmentation Offload (TSO).
This was our first attempt. TCP Chimney is now disabled on the Application Server and the Database Server. Testing shows that the same error occurs at the same rate.
So where to go from here? Honestly I am not sure. One seemingly good option remains:
Application Server and Database Server SQL Server installations do not exactly match
Application Server = SQL Server 2008 (SP1) - 10.0.2531.0 (X64)
Database Server = SQL Server 2008 R2 (SP1) - 10.50.2500.0 (X64)
The plan is to upgrade the SQL Server installation on the Application server. Its kind of a hit and hope, but at this point this seems like the best option. Something in my brain tells me that this might be solved by fixing a hardware issue (by that I mean a repair or a replace) and that there might not be anything that hardware and software configuration can do about it.
However, I am still not sure how to go about determining a root cause. I am still left wondering what tools I should be using to diagnose the root cause.
Do you have AV software on Application Server side? If yes, try to disable AV - sometime AV blocks TCP/IP traffic. Issue with "The specified network name is no longer available" was solved by disabling AV here:
https://community.spiceworks.com/topic/239423-the-specified-network-name-is-no-longer-available-while-writing-to-shared-dir
The error message indicates that the connection was forcibly closed. You have also mentioned this occur when you run a lot of jobs. Chances are, the network firewall is to blame. You should reach out to the firewall administrator to research the logs in order to see if the firewall closed the connection. If this is the case then two potential solutions exist:
Add an exception to whatever firewall rule is being triggered and causing the connection to be closed.
Stop running so many jobs concurrently. You should consider running them in sequence. This also adheres to the idea of being a good network citizen.
First off did you tried to remove the large send offload setting on the nic ?
Second point, can you run a wireshark to capture the packets if you can reproduce the error ?
Third point, did you tried to change the vnic from the VM ? some model can cause issue. (If you use vmxnet3, try e1000, etc..)
Last point, does you have a vswitch between them, they are on same host, a physical switch between, etc... A badly configured switch can drop traffic, if inside the host the same host and same vswitch it's the best test, as the traffic never leave the server.
It seems there are multiple causes for this issue. For me it was the SQL file being too big at about 150MiB. Splitting it into smaller files worked for me.
One possible root cause could be the active directory membership count as we experiencer recently. That will among other influence the size of Kerberos token that can influence the SQL server login steps as well.
Details:
The login packet is structurally invalid
Kerberos Token Size
Note: We observed error log entries on the SQL Server like:
The login packet used to open the connection is structurally invalid; the connection has been closed. Please contact the vendor of the client library.'
On the client we observed the following error message:
A connection was successfully established with the server, but then an error occurred during the login process. (provider: TCP Provider, error: 0 - The specified network name is no longer available.) (.Net SqlClient Data Provider)
This issue was only affecting some AD users. There was no general service interruption.
Using the following command we determined the active directory membership:
PS> $groups = (az ad user get-member-groups --id {user identifier} |ConvertFrom-Json -AsHashtable)
PS> $groups.Count
294
Or we can also use the following command:
PS> $token = [System.Security.Principal.WindowsIdentity]::GetCurrent()
PS> $token.Groups.count
771
Connecting to the server with different providers yield different result:
Microsoft ODBC Driver for SQL Server Version 17.07.0002:
Data Source Name: SOME_NAME
Data Source Description:
Server: SQLSERVER.DOMAIN.COM
Database: CORE_DATABASE
Language: (Default)
Data Encryption: Yes
Trust Server Certificate: Yes
Multiple Active Result Sets(MARS): No
Mirror Server:
Translate Character Data: Yes
Log Long Running Queries: No
Log Driver Statistics: No
Use Regional Settings: No
Use ANSI Quoted Identifiers: Yes
Use ANSI Null, Paddings and Warnings: Yes
Authentication: ActiveDirectoryIntegrated
-----------
Running connectivity tests...
Attempting connection
Connection established
Verifying option settings
INFO: Connection was encrypted without server certificate validation.
Disconnecting from server
TESTS COMPLETED SUCCESSFULLY!
Microsoft SQL Server Native Client Version 11.00.7462:
Data Source Name: SOME_NAME
Data Source Description:
Server: SQLSERVER.DOMAIN.COM
Use Integrated Security: Yes
Database: (Default)
Language: (Default)
Data Encryption: No
Trust Server Certificate: No
Multiple Active Result Sets(MARS): No
Translate Character Data: Yes
Log Long Running Queries: No
Log Driver Statistics: No
Use Regional Settings: No
Use ANSI Quoted Identifiers: Yes
Use ANSI Null, Paddings and Warnings: Yes
-----------
Running connectivity tests...
Attempting connection
[Microsoft][SQL Server Native Client 11.0]TCP Provider: The specified network name is no longer available.
[Microsoft][SQL Server Native Client 11.0]Communication link failure
TESTS FAILED!
SQL Server - was not able to configure:
Try using ODBC instead of OLE DB for your connection to the database.
I'm connected to analysis server:localhost\sqlr2.
I'm using windows authentication,so user name and password are all grayed out.
I'm trying to deploy a project from Visual Studio BI to SQL Server 2008R2 analysis server and get the following errors:
Error 1 Internal error: The operation terminated unsuccessfully. 0 0
Error 2 OLE DB error: OLE DB or ODBC error: Login timeout expired; HYT00; A network-related or instance-specific error has occurred while establishing a connection to SQL Server. Server is not found or not accessible. Check if instance name is correct and if SQL Server is configured to allow remote connections. For more information see SQL Server Books Online.; 08001; Named Pipes Provider: Could not open a connection to SQL Server [53]. ; 08001. 0 0
Error 3 Errors in the high-level relational engine. A connection could not be made to the data source with the DataSourceID of 'Max Min Manufacturing DM', Name of 'Max Min Manufacturing DM'. 0 0
It looks like your datasource is set up wrong. When you process, what server is the cube pointed at? What account does SSAS run under? How have you set up the connection string in the datasource? Are you seeing failed login attempts in the SQL Server logs?
Error 2 OLE DB error: OLE DB or ODBC error: Login timeout expired; HYT00;
A network-related or instance-specific error has occurred while establishing a
connection to SQL Server. Server is not found or not accessible.
Check if instance name is correct and if SQL Server is configured
to allow remote connections.
I'd focus on this. I assume you must be able to connect to SQL Server in Visual Studio to build your cube, so I guess Analysis Services doesn't have permissions. Check your connection strings.
You say you're using Windows Authentication, so does the service accounts that Analysis Services runs under have access to the required database?
I would like to share my experience with this error and how I discovered the solution.
Hope it helps!
http://afsawaf.blogspot.com/2013/08/ole-db-error-ole-db-or-odbc-error.html
---Thanks #kleopatra for the advice.
In my case I was connecting with a wrong provider, I was trying to connect using SQL Server Native Client 11.0 to MS SQL Server 2005 DB which caused my similar problem.
When I reverted to version 10.0, worked perfectly!
When in try to start azure storage emulator i get the following error.
Caught exception while probing for SQL endpoint. A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: SQL Network Interfaces, error: 26 - Error Locating Server/Instance Specified) Number of Sql Errors Reported: 1 Sql Error: System.Data.SqlClient.SqlError: A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: SQL Network Interfaces, error: 26 - Error Locating Server/Instance Specified)
All the Components have been correctly installed .
Kindly Help
Thanx
I managed to fix it by deleting and creating my LocalDb instance. In my case it happened because I installed Azure SDK 2.2 over version 2.1.
The step-by-step to fix it:
Run sqllocaldb i to get your instance name:
C:\Users\myUser>sqllocaldb i
v11.0
Stop LocalDB instance:
C:\Users\myUser>sqllocaldb p v11.0
LocalDB instance "v11.0" stopped.
Delete your LocalDB instance:
C:\Users\myUser>sqllocaldb d v11.0
LocalDB instance "v11.0" deleted.
Create your LocalDB instance with the same name:
C:\Users\myUser>sqllocaldb c v11.0
LocalDB instance "v11.0" created with version 11.0.
Delete database files to allow emulator to recreate db files
C:\Users\myUser\DevelopmentStorageDb22.mdf
C:\Users\myUser\DevelopmentStorageDb22_log.ldf
That should fix your issues.
It looks like your windows account under which you logged in is not sysadmin in sqlexpress instance. Normally emulator uses database within sqlexpress with windows authentication.
The first time you run the Windows Azure storage emulator, an initialization process runs to configure the environment. By default, the initialization process creates a database in SQL Express LocalDB. So, please make sure that your SQLExpress service is running and your login have sufficient rights to use the database within SQLExpress.
If you want to make use of a local instance of an SQL Server to be used by Storage Emulator follow steps listed at http://msdn.microsoft.com/en-us/library/windowsazure/gg433134.aspx.
HTH
Sameer
What i am suspecting is that your LocalDB configuration with emulator is somehow corrupted or misconfigured.
Please follow the blog below to see if you can verify LocalDB instance is ready/working and it is used by Emulator:
http://blogs.msdn.com/b/avkashchauhan/archive/2012/07/10/windows-azure-sdk-1-7-storage-emulator-and-localdb.aspx
If you still have problem the easiest method would be to uninstall SDK 1.7 completely and re-install which will fix any issue with your configuration rather the finding the problem and try to fix which could take longer.
I was getting a similar message:
Probing SQL Instance: 'localhost\SQLExpress'.
Caught exception while probing for SQL endpoint. A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: SQL Network Interfaces, error: 26 - Error Locating Server/Instance Specified)
Number of SqlErrors Reported: 1
SqlError: System.Data.SqlClient.SqlError: A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: SQL Network Interfaces, error: 26 - Error Locating Server/Instance Specified)
No available SQL Instance was found.
One or more initialization actions have failed. Resolve these errors before attempting to run the storage emulator again.
Error: No available SQL Instance was found.
For me the problem was due to the fact that I was not running with administrator privileges.
Might be a little late to the party here, but I've seen this happen if you clear out or remove items in your "C:\Users[USER_NAME]\AppData\Local\Temp" folder.
At the time of writing this, Azurite is the replacement for "azure storage emulator". In my case, I was able to simply follow the steps here https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azurite?tabs=npm#install-azurite to get it installed, and you should be good to go after that.
I get the error message below when accessing Reporting Services through a browser
URL used: http://<name of server>/ReportServer
Reporting Services Error
The report server cannot open a connection to the report server
database. A connection to the database is required for all requests
and processing. (rsReportServerDatabaseUnavailable) Get Online Help A
network-related or instance-specific error occurred while establishing
a connection to SQL Server. The server was not found or was not
accessible. Verify that the instance name is correct and that SQL
Server is configured to allow remote connections. (provider: Named
Pipes Provider, error: 40 - Could not open a connection to SQL Server)
Have you verified that the configuration is correct? Use "Reporting Service Configuration Manager" which is installed with SSRS. Step through the screens looking for problems; try applying the configuration and make sure databases exist and have correct permissions.
Try running RSConfigTool.exe or Report Services Configuration Manager from Windows and check if the server is accessible