I am using WinInet in C / C++ application to connect to a ASP.NET Web Service.
I want to increase my SESSION TIMEOUT time.
Currently somehow SESSION Time out is 20 minutes and I want to increase it to 50 minutes.
Which option do it use for the option INTERNET_OPTION_XXXXX in
InternetSetOption(hInstance, INTERNET_OPTION_XXXXX,(LPVOID) &timeout, sizeof(timeout));
Unlike WinHTTP that has WinHttpSetTimeouts there is no equivalent function available at WinINet.
I realise this is an old question however it seems there is no info on how to do this on SO. So I am posting this in case someone wants to know how to set timeouts with WinINet.
Normally you would use INTERNET_OPTION_CONNECT_TIMEOUT,INTERNET_OPTION_RECEIVE_TIMEOUT, or INTERNET_OPTION_SEND_TIMEOUT with InternetSetOption. See here for details on the option flags: https://learn.microsoft.com/en-us/windows/win32/wininet/option-flags
However, there is a bug which it seems MS has not fixed in about 20 years. The above timeout flags simply don't work.
So the way to work around this is to create second worker thread to watch for the connection request. The second request will kill the main connection request if it doesn't receive response from server in the timeout set. See this MS KB article for details and example:
https://mskb.pkisolutions.com/kb/224318
Related
I have an application deployed on GAE having endpoints. Each endpoint make a connection with database , get data and close connection and return data. Normally everything works fine but when there is hike in requests it starts taking more than 60 sec and requests get aborted. Due to this it does not close database connection and there mysql got 1000+ connections and then each requests starts aborting and it shows deadline exceeded error. Is there any solution for this ?
You could wrap the "get data" portion with a try... finally... statement and move the "close connection" portion in the finally section. Then start an "about to exceed deadline" timer before "get data" (something like say 45 seconds) and raise an exception if the timer expires, allowing you to close the connection in the finally portion, which should take care of the orphan open connections (but would not prevent errors in those requests).
If your application tolerates it you could also look into using task queues which have a 10 min deadline, which could help reducing/eliminating the errors in the requests as well.
You can also find some general advice for addressing deadline exceeded errors here: https://cloud.google.com/appengine/articles/deadlineexceedederrors, donno if applicable to your app.
EDIT: actually the suggestion in the first paragraph above doesn't work on GAE as the Python sandbox doesn't allow installing a custom signal handler:
signal.signal(signal.SIGALRM, timer_expired)
AttributeError: 'module' object has no attribute 'signal'
After seeing your code a somehow equivalent solution would be to replace your cursor.fetchall() with a loop of cursor.fetchone() or cursor.fetchmany() to split your operation in smaller pieces:
http://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-fetchone.html. You'd get a start timestamp (with time.time() for example) when entering your request handler. Then inside the loop you'd get another timestamp to measure the time elapsed so far since the start timestamp and you'd break out of the loop and close the DB connection when deadline expiration nears. Again, this won't help with actually replying successfully to the requests if it takes so much time to prepare the replies.
You can use this solution to close connections when deadlines are exceeded:
Dealing with DeadlineExceededErrors
This way you won't have any open connections hanging there forever.
Think about the design of your application -
1.Use the deadline exception handling - Design smells
Because there will be situation(s) where db operation takes more than 60 seconds , If its a simple query then its well and good , but reconsider the design of the application . User Expierence is going to be hurt.
2.Lets change the design to use the endpoints-
https://cloud.google.com/appengine/docs/java/endpoints/
The way to go ,future proof.
3.Using Back-end or Task-queues as descibed in this post
Max Time for computation on Google App Engine
You can set the Timeouts
interactive_timeout
and / or
wait_timeout
based on connection Type they use one of them
I'm running PHP commandline scripts as rabbitmq consumers which need to connect to a MySQL database. Those scripts run as Symfony2 commands using Doctrine2 ORM, meaning opening and closing the database connection is handled behind the scenes.
The connection is normally closed automatically when the cli command exits - which is by definition not happening for a long time in a background consumer.
This is a problem when the consumer is idle (no incoming messages) longer then the wait_timeout setting in the MySQL server configuration. If no message is consumed longer than that period, the database server will close the connection and the next message will fail with a MySQL server has gone away exception.
I've thought about 2 solutions for the problem:
Open the connection before each message and close the connection manually after handling the message.
Implementing a ping message which runs a dummy SQL query like SELECT 1 FROM table each n minutes and call it using a cronjob.
The problem with the first approach is: If the traffic on that queue is high, there might be a significant overhead for the consumer in opening/closing connections. The second approach just sounds like an ugly hack to deal with the issue, but at least i can use a single connection during high load times.
Are there any better solutions for handling doctrine connections in background scripts?
Here is another Solution. Try to avoid long running Symfony 2 Workers. They will always cause problems due to their long execution time. The kernel isn't made for that.
The solution here is to build a proxy in front of the real Symfony command. So every message will trigger a fresh Symfony kernel. Sound's like a good solution for me.
http://blog.vandenbrand.org/2015/01/09/symfony2-and-rabbitmq-lessons-learned/
My approach is a little bit different. My workers only process one message, then die. I have supervisor configured to create a new worker every time. So, a worker will:
Ask for a new message.
If there are no messages, sleep for 20 seconds. If not, supervisor will think there's something wrong and stop creating the worker.
If there is a message, process it.
Maybe, if processing a message is super fast, sleep for the same reason than 2.
After processing the message, just finish.
This has worked very well using AWS SQS.
Comments are welcomed.
This is a big problem when running PHP-Scripts for too long. For me, the best solution is to restart the script some times. You can see how to do this in this Topic: How to restart PHP script every 1 hour?
You should also run multiple instances of your consumer. Add a counter to any one and terminate them after some runs. Now you need a tool to ensure a consistent amount of worker processes. Something like this: http://kamisama.me/2012/10/12/background-jobs-with-php-and-resque-part-4-managing-worker/
I'm facing an load issue on MySql Server database.
In our application we have an requirement of inserting ,fetching and updating data for each call/click.
We are doing the load testing on this application, we wrote an program to call this application do it functionality. we running this test program for more than 400 times (I mean 400 instances) in MySql more than 100 connections are created and test program was throwing "The underlying provider failed on Open. -Too many connections"
I have set the max_connection variable to 400 and did the same testing but this time it was throwing "Thread stack overrun: 137376 bytes used of a 262144 byte stack, and 128000 bytes needed. Use 'mysqld --thread_stack=#' to specify a bigger stack."
Can anyone help me please.
I have resolved this issue by doing small modification in web.config and mysql variable setting
1) <httpRuntime executionTimeout = "number(in seconds)" />
this is used, to remove the request when it take's more time in process the request
2) By increasing the max_connections variable size in mysql
3) there was issue in the load testing application itself , it was sending the request without any time gap, so set the time gap between each request.
Now I can run up to 600 users instance at a same time , but load will be on CUP and MySql.
Thank you Kay Nelson for your quick reply.
Regards,
Sampath Kumar J
One of the more interesting "features" in Coldfusion is how it handles external requests. The basic gist of it is that when a query is made to an external source through <cfquery> or or any other external request like that it passes the external request on to a specific driver and at that point CF itself is unable to suspend it. Even if a timeout is specified on the query or in the cfsetting it is flatly ignored for all external requests.
http://www.coldfusionmuse.com/index.cfm/2009/6/9/killing.threads
So with that in mind the issue we've run into is that somehow the communication between our CF server and our mySQL server sometimes goes awry and leaves behind hung threads. They have the following characteristics.
The hung thread shows up in CF and cannot be killed from FusionReactor.
There is no hung thread visible in mySQL, and no active running query (just the usual sleeps).
The database is responding to other calls and appears to be operating correctly.
Max connections have not been reached for the DB nor the user.
It seems to me the only likely candidate is that somehow CF is making a request, mySQL is responding to that request but with an answer which CF ignores and continues to keep the thread open waiting for a response from mySQL. That would explain why the database seems to show no signs of problems, but CF keeps a thread open waiting for the mysterious answer.
Usually these hung threads appear randomly on otherwise working scripts (such as posting a comment on a news article). Even while one thread is hung for that script, other requests for that script will go through, which would imply that the script isn't neccessarily at fault, but rather the condition faced when the script was executed.
We ran some test to determine that it was not a mysql generated max_connections error... we created a user, gave it 1 max connections, tied that connection with a sleep(1000) query and executed another query. Unfortunately, it correctly errored out without generating a hung thread.
So, I'm left at this point with absolutely no clue what is going wrong. Is there some other connection limit or timeout which could be causing the communication between the servers to go awry?
One of the things you should start to look at is the hardware between the two servers. It is possible that you have a router or bridge or NIC that is dropping occasional packets. This can result in the mySQL box thinking it has completed the task while the CF server sits there and waits for a complete response indefinitely, creating a hung thread.
3com has some details on testing for packet loss here: http://support.3com.com/infodeli/tools/netmgt/tncsunix/product/091500/c11ploss.htm#22128
We had a similar problem with a MS SQL server. There, the root cause was a known issue in which, for some reason, the server thinks it's shutting down, and the thread hangs (even though the server is, obviously, not shutting down).
We weren't able to eliminate the problem, but were able to reduce it by turning off pooled DB connections and fiddling with the connection refresh rate. (I think I got that label right -- no access to administrator at my new employment.) Both are in the connection properties in Administrator.
Just a note: The problem isn't entirely with CF. The problem, apparently, affects all Java apps. Which does not, in any way, reduce how annoyed I get by this.
Long story short, but I believe the caused was due to Coldfusion's CF8 image processing. It was just buggy and now in CF9 I have never seen that problem again.
I am getting the following error:
mysql_connect(): Too many connections
It has completely shut down my site, which has been running seamlessly for several years.
Note: I have shared hosting with GoDaddy.
How do I fix this?
ALSO: is there a way to close all connections and restart when on a shared hosting plan?
This is a Technical Response
You will get this "too many connections" error upon connecting to MySQL when the MySQL server has reached its software configurable artificial limit of maximum concurrent client connections.
So, the proper way to fix this is:
Directly connect to the MySQL server, and perform the query: SET GLOBAL max_connections = 1024; to change the connection limit at runtime (no downtime).
Make your change permanent, and edit the /etc/my.cnf (or similar) and add the line max_connections = 1024 line within the [mysqld] section; then restart if you couldn't do the live change.
The chosen limit of 1024 was pure subjective; use whatever limit you want to. You can also inspect your current limit via query SHOW VARIABLES LIKE "max_connections";. Keep in mind, that these limits are there for good use, in order to prevent unnecessary overload of your backend database. So always choose wise limits.
However, for those steps you are required to have direct access to your database MySQL server.
As you said, you are using GoDaddy (I do not know them that much), you are left out with the option to contact your service provider (i.e. GoDaddy). However, they will see this in their logs anyway, also.
Possible Root Causes
This of course means, that too many clients are attempting to connect to the MySQL server at the same time - as of the per configuration specified artificial software limit.
Most probably, you have been a subject of a DDoS attack.
People on this forum complain on exactly same thing with exactly same provider.
The answer is this:
VB told me it was a DOS attack - here is their message:
This is not an 'exploit'. This is a DoS attack (Denial of Service). Unfortunately there is nothing we can do about this. DoS attacks can only be fought at the server or router level, and this is the responsibility of your host. Instead of doing this they have decided to take the easy way out and suspend your account.
If you cannot get them to take this seriously, then you should look for another host. Sorry for the bad news.
A possible workaround can be this: if your connection fails with mysql_connect(): Too many connections, you don't quit, but instead sleep() for half a second and try to connect again, and exit only when 10 attempts fail.
It's not a solution, it's a workaround.
This of course will delay your page loading, but it's better than an ugly too many connections message.
You also can come with a some kind of a method which tells bots and browsers apart.
Like, set a salted SHA1 cookie, redirect to the same page and then check that cookie and connect to MySQL only if the user agent had passed the test.
Another thing that can cause this error is if the Database has run out of space. I recently had this occur, and the issue wasn't connections, it was Disk space. Hope this helps someone else!
Do you close your connection when you're done with them? Are you using some type of connection pooling? Sounds like you're opening connections and not closing them.
EDIT: Already answered by Quassnoi. In the case it is a DDoS, and you're using shared hosting, you may be left with just contacting your host and working it out with them. Unfortunately this is a risk when you don't have control of your whole system.
Consider using mysql_pconnect(). Your host may have adding some sort of throttling for connections. Like a maximum of 100 per 20 minutes or something weird.
First, check your database connections:
show variables like 'max_connections';
to check all variables
show variables;
Connect your MySQL server and update the connection size:
SET GLOBAL max_connections = 1001;