Making MySQL client program thread safe? - mysql

I'm running into an interesting threadding problem while running a D programming that uses the MySQL C API. I am getting error 2013 "Lost connection to MySQL server during query." The problem appears to occurs when enough threads flood the network interface buffer, but the server still has more to transfer. This is my best guess based on some research and running the program on two different computers. One computer has a 100Mb connection to the server and the other has a 1Gb connection. The computer with the 100Mb connection throws the error, while the 1Gb computer does not. I am wondering if I am running into what is described in the first paragraph of How to Write a Threaded Client in the MySQL documentation. If I am, what do I need to do with SIGPIPE and how do I do it?
For those who are interested, I am calling mysql_library_init before any library call and I am creating a new MYSQL* for each thread with mysql_init and mysql_real_connect. Also of note, the queries that I am executing are small SELECTs, only a few thousand records returned from each query and all queries are executed from the same table.

Please try this before mysql_real_connect:
my_bool myb = 1;
mysql_options(conn, mysql_option.MYSQL_OPT_RECONNECT, &myb);
Also please check this mysql troubleshooting page:
http://dev.mysql.com/doc/refman/5.5/en/gone-away.html

Related

Azure database for MySQL DB 5.7 Transient handling in .net core

I am creating .net core 2.1 MVC application and using Azure database for MySQL DB 5.7.
I have read below links but seems they are applicable for MS SQL DB.
https://learn.microsoft.com/en-us/azure/mysql/concepts-high-availability
https://learn.microsoft.com/en-us/azure/architecture/best-practices/retry-service-specific
Transient handling for MySQL not possible? Help me link to MYSQL related similar pages.
A transient error, also known as a transient fault, is an error that will resolve itself. Most typically these errors manifest as a connection to the database server being dropped. Also new connections to a server can't be opened. Transient errors can occur for example when hardware or network failure happens.
Transient errors should be handled using retry logic. Situations that must be considered:
An error occurs when you try to open a connection
An idle connection is dropped on the server side. When you try to issue a command it can't be executed
An active connection that currently is executing a command is dropped.
The first and second case are fairly straight forward to handle. Try to open the connection again. When you succeed, the transient error has been mitigated by the system. You can use your Azure Database for MySQL again. We recommend having waits before retrying the connection. Back off if the initial retries fail. This way the system can use all resources available to overcome the error situation. A good pattern to follow is:
Wait for 5 seconds before your first retry.
For each following retry, the increase the wait exponentially, up to 60 seconds.
Set a max number of retries at which point your application considers the operation failed.
Read more here.
And you can read more on how to troubleshoot connection issues to Troubleshoot connection issues to Azure Database for MySQL here.

MySQL Query running even after losing connection

I've a MySQL 5.1.41 Server installed on a Ubuntu machine. I get connected to it through Workbench from my Windows machine over TCP/IP. I run a bigger query, after 900 seconds I got the below message, (there is no wait_timeout defined in the server's configuration file my.cnf)
Error Code: 2013. Lost connection to MySQL server during query
But when I look into the process list by using show processlist; command, I can still see my query running.
I got this link http://dev.mysql.com/doc/refman/5.0/en/gone-away.html where I found the below lines,
The problem on Windows is that in some cases MySQL does not get an
error from the OS when writing to the TCP/IP connection to the server,
but instead gets the error when trying to read the answer from the
connection.
I'm not sure whether this is the reason for my observation.
Please clarify me on this.
Thanks in advance!!
Closing connection is not a reason to stop a query. A query might be update, or kind of transaction, or select with output to remote (server) file.
Closed connection is just is just means, that you will not receive any data from DBMS after executing query (data, timings - nothing).
The reason of closing connection could be different, as SO-User posted. Try increasing
on server side:
wait_timeout
max_allowed_packet
on client side:
any kinds of timeout you find in your client (i.e. that SO-User suggests)
Do not forget to reload DBMS config and restart client (for sure)
In MySQL WorkBench we have an option to change timeout.
Find it under
Edit → Preferences → SQL Editor → DBMS connection read time out (in seconds): 600
Changed the value to 6000 or something higher.
Update
Lost connection to MySQL server
There are three likely causes for this error message.
Usually it indicates network connectivity trouble and you should check
the condition of your network if this error occurs frequently. If the
error message includes “during query,” this is probably the case you
are experiencing.
Sometimes the “during query” form happens when millions of rows are
being sent as part of one or more queries. If you know that this is
happening, you should try increasing net_read_timeout from its default
of 30 seconds to 60 seconds or longer, sufficient for the data
transfer to complete.
More rarely, it can happen when the client is attempting the initial
connection to the server. In this case, if your connect_timeout value
is set to only a few seconds, you may be able to resolve the problem
by increasing it to ten seconds, perhaps more if you have a very long
distance or slow connection. You can determine whether you are
experiencing this more uncommon cause by using SHOW GLOBAL STATUS LIKE
'Aborted_connects'. It will increase by one for each initial
connection attempt that the server aborts. You may see “reading
authorization packet” as part of the error message; if so, that also
suggests that this is the solution that you need.
If the cause is none of those just described, you may be experiencing
a problem with BLOB values that are larger than max_allowed_packet,
which can cause this error with some clients. Sometime you may see an
ER_NET_PACKET_TOO_LARGE error, and that confirms that you need to
increase max_allowed_packet.
Doc link: Error lost connection
and also check here

A transport-level error has occurred when receiving results from the server [closed]

Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed 6 years ago.
Improve this question
I'm getting a SQL Server error:
A transport-level error has occurred
when receiving results from the
server. (provider: Shared Memory
Provider, error: 0 - The handle is
invalid.)
I'm running Sql Server 2008 SP1, Windows 2008 Standard 64 bit.
It's a .Net 4.0 web application. It happens when a request is made to the server. It's intermittent. Any idea how I can resolve it?
The database connection is closed by the database server. The connection remains valid in the connection pool of your app; as a result, when you pickup the shared connection string and try to execute it's not able to reach the database. If you are developing Visual Studio, simply close the temporary web server on your task bar.
If it happens in production, resetting your application pool for your web site should recycle the connection pool.
Try the following command on the command prompt:
netsh interface tcp set global autotuning=disabled
This turns off the auto scaling abilities of the network stack
I had the same problem. I restarted Visual Studio and that fixed the problem
Transport level errors are often linked to the connection to sql server being broken ... usually network.
Timeout Expired is usually thrown when a sql query takes too long to run.
So few options can be :
Check for the connection in VPN (if used) or any other tool
Restart IIS
Restart machine
Optimize sql queries.
For those not using IIS, I had this issue when debugging with Visual Studio 2010. I ended all of the debugger processes: WebDev.WebServer40.EXE which solved the issue.
All you need is to Stop the ASP.NET Development Server and run the project again
If you are connected to your database via Microsoft SQL Server Management, close all your connections and retry.
Had this error when connected to another Azure Database, and worked for me when closed it.
Still don't know why ..
Look at the MSDN blog which details out this error:
Removing Connections
The connection pooler removes a connection from the pool after it has
been idle for a long time, or if the pooler detects that the
connection with the server has been severed.
Note that a severed connection can be detected only after attempting
to communicate with the server. If a connection is found that is no
longer connected to the server, it is marked as invalid.
Invalid connections are removed from the connection pool only when
they are closed or reclaimed.
If a connection exists to a server that has disappeared, this
connection can be drawn from the pool even if the connection pooler
has not detected the severed connection and marked it as invalid.
This is the case because the overhead of checking that the connection
is still valid would eliminate the benefits of having a pooler by
causing another round trip to the server to occur.
When this occurs, the first attempt to use the connection will detect
that the connection has been severed, and an exception is thrown.
Basically what you are seeing is that exception in the last sentence.
A connection is taken from the connection pool, the application does
not know that the physical connection is gone, an attempt to use it is
done under the assumption that the physical connection is still there.
And you get your exception.
There are a few common reasons for this.
The server has been restarted, this will close the existing connections.
In this case, have a look at the SQL Server log, usually found at:
C:\Program Files\Microsoft SQL Server\\MSSQL\LOG
If the timestamp for startup is very recent, then we can suspect that
this is what caused the error. Try to correlate this timestamp with
the time of exception.
2009-04-16 11:32:15.62 Server Logging SQL Server messages in file
‘C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\LOG\ERRORLOG’.
Someone or something has killed the SPID that is being used.
Again, take a look in the SQL Server log. If you find a kill, try to
correlate this timestamp with the time of exception.
2009-04-16 11:34:09.57 spidXX Process ID XX was killed by
hostname xxxxx, host process ID XXXX.
There is a failover (in a mirror setup for example) again, take a look in the SQL Server log.
If there is a failover, try to correlate this timestamp with the time
of exception.
2009-04-16 11:35:12.93 spidXX The mirrored database “” is changing roles from “PRINCIPAL” to “MIRROR” due to
Failover.
Was getting this, always after about 5 minutes of operation. Investigated and found that a warning from e1iexpress always occurred before the failure. This apparently is an error having to do with certain TCP/IP adapters. But changing from WiFi to hardwired didn't affect it.
So tried Plan B and restarted Visual Studio. Then it worked fine.
On closer study I noticed that, when working correctly, the message The Thread '<No Name>' has exited with code 0 occurred at almost exactly the time the run crashed in previous attempts. Some Googling reveals that that message comes up when (among other things) the server is trimming the thread pool.
Presumably there was a bogus thread in the thread pool and every time the server attempted to "trim" it it took the app down.
You get this message when your script make SQL Service stopped for some reasons. so if you start SQL Service again perhaps your problem will be resolved.
I know this may not help everyone (who knows, maybe yes), but I had the same problem and after some time, we realized that the cause was something out of the code itself.
The computer trying to reach the server, was in another network, the connection could be established but then dropped.
The way we used to fix it, was to add a static route to the computer, allowing direct access to the server without passing thru the firewall.
route add –p YourServerNetwork mask NetworkMask Router
Sample:
route add –p 172.16.12.0 mask 255.255.255.0 192.168.11.2
I hope it helps someone, it's better to have this, at least as a clue, so if you face it, you know how to solve it.
I got the same error in Visual Studion 2012 development environment, stopped the IIS Express and rerun the application, it started working.
I had the same issue. I solved it, truncating the SQL Server LOG.
Check doing that, and then tell us, if this solution helped you.
For me the solution was totally different.
In my case I had an objectsource which required a datetimestamp parameter. Even though that ODS parameter ConvertEmptyStringToNull was true 1/1/0001 was being passed to SelectMethod. That in turn caused a sql datetime overflow exception when that datetime was passed to the sql server.
Added an additional check for datetime.year != 0001 and that solved it for me.
Weird that it would throw a transport level error and not a datetime overflow error.
Anyways..
In my case the "SQL Server" Server service stopped. When I restarted the service that enabled me to run the query and eliminate the error.
Its also a good idea to examine your query to find out why the query made this service stop
For me the answer is to upgrade the OS from 2008R2 to 2012R2, the solution of iisreset or restart apppool didn't work for me.
I also tried to turn of TCP Chimney Offload setting, but I didn't restart the server because it is a production server, which didn't work either.
We encountered this error recently between our business server and our database server.
The solution for us was to disable "IP Offloading" on the network interfaces.
Then the error went away.
One of the reason I found for this error is 'Packet Size=xxxxx' in connection string. if the value of xxxx is too large, we will see this error. Either remove this value and let SQL server handle it or keep it low, depending on the network capabilities.
It happened to me when I was trying to restore a SQL database and checked following Check Box in Options tab,
As it's a stand alone database server just closing down SSMS and reopening it solved the issue for me.
This occurs when the database is dropped and re-created some shared resources is still considering the database still exists, so when you re-run execute query to create tables in the database after it was re-created the error will not show again and Command(s) completed successfully. message will show instead of the error message Msg 233, Level 20, State 0, Line 0 A transport-level error has occurred when sending the request to the server. (provider: Shared Memory Provider, error: 0 - No process is on the other end of the pipe.).
Simply ignore this error when you are dropping and recreating databases and re-execute your DDL queries with no worries.
I faced the same issue recently, but i was not able to get answer in google.
So thought of sharing it here, so that it can help someone in future.
Error:
While executing query the query will provide few output then it will throw below error.
"Transport level error has occurred when receiving output from
server(TCP:provider,error:0- specified network name is no longer
available"
Solution:
Check the provider of that linked server
In that provider properties ,Enable "Allow inprocess" option for that particular provider to fix the issue.

Separate dbx connection to a thread

I made a small app that connects to a mysql db using dbx. It works ok with my local mysql server, but it's supposed to work with a remote server.
Connecting to the remote server takes a few seconds, which freezes the app.
So my question is, how can I put the connection code in a different thread?
I'll have to pass that connection to the main thread somehow, so that the dbgrid I have on the main form works.
I read that db stuff working in a different thread should have their own connections. So I'm not sure how to do what I want.
Any ideas? Anything to read about working with remote servers?
Thanks.
Edit: The components I'm using on the form are: TSQLConnection -> TSimpleDataSet > TDataSource > TDBGrid.
You only need a connection per thread if your threads are going to do simultaneous database access. Basically what you want, is for a thread to connect, and come back to you when the connection has been established. You can do this in a thread, and when the thread is ready (i.e. connection established), it can send a message back to the main thread to let it know that the dbx connection is now available. See this tutorial for ideas on how to set up the thread, and communicate between the thread and the main VCL thread.
Threading Tutorial
this has really helped me for doing multi thread Apps in rad studio
Writing multi-threaded applications Index
if their is any thing else post and ill try to help

A transport-level error has occurred... (Existing connection closed by the server) Sql Server 2008

I have a web application which is currently running on Windows XP operating system with SQL Server 2005 database and IIS 6.0.
Now, we are trying to upgrade it to IIS 7.0 on Windows Server 2008 and SQL server 2008 database.
When I run the application then it is throwing "A transport-level error has occurred when sending the request to the server. (provider: TCP Provider, error: 0 - An existing connection was forcibly closed by the remote host.)" randomly at different database calls.
I have verified using "sp_who" that only one connection which I am using is opened.
Can anyone tell me, what could be the cause of this issue?
This blog post by Michael Aspengren explains the error message "A transport-level error has occurred when sending the request to the server."
Maybe the database is set to auto-close? This used to be the default for databases created on the older MSDE/Express edition.
ALTER DATABASE YourDatabaseName SET AUTO_CLOSE OFF;
More:
The SQL Server 2005 Express Edition Overview states:
Features like Auto-Close and the ability to copy databases as files are enabled by default in SQL Server Express ...
I can't remember which service pack changed the default, but the New Database UI in the SP3 version of Management Studio Express defaults the Auto Close setting to false.
This is an error that occurrs when connecting to database due to a lower layer network-related error that you cannot handle from Sql Server.
It would be great that the API used to conenct handled this error but AFAIK it is not. The only way to solve this is to retry connecting to the database when this error occurrs.
You should try to reconnect at least a couple of times more to get sure that the error is not persistent and if it is the case then throw the exception. My experience tells me that trying to reconnect, the 99.9% of times you will succed.
Often, it's the database being offline/closed or the SQL Instance being restarted/offline
In my case. We have DELL R720, NICs are on board and PCI Broadcom NetXtreme Gigabit Ethernet cards. We run into two issues.
Network speed is very slow for HyperV VMs. The solution is to disable the "Virtual Machine queues" of the HyperV host Broadcom nic property windows. It is in the Advanced tab.
SQL client having this transport level issue. It only happened once in every 10-15 min. At the same time the RDP service kick out client from time to time also. The solution is to disable the "Large Send Offload V2" of the HyperV host Broadcom nic property windows. It is in the Advanced tab.
Check if you have MultipleActiveResultSets=true; set in your connection string.
Transport level errors are often linked to the connection to sql server being broken ... usually network.
Timeout Expired is usually thrown when a sql query takes too long to run. Is this the scenario in your case? Like stored proc taking too much time to execute or are there are batch jobs that are executed?
Use clearpool(urSQLconnection), along with dispose. This will clear the dirty connection. and you can open new connection.
Got this trying to execute commands from multiple threads on one connection object in .net, that was a stupid mistake. Locking it or using a separate connection per thread fixes that.
In my case it was McAffee that was pre-installed. Can tell if it was conflict with other solutions or not. The connection was on VPN, on WiFI. Maybe this will help someone in future.