I have a containerized a console app which run as a scheduler app and performe some logic at specific time. I tried to run it as an Azure App service but getting below error
2022-03-30T19:10:29.209Z INFO - Waiting for response to warmup request for container one-site-scheduler-app_0_d429ab9d. Elapsed time = 209.8994308 sec
2022-03-30T19:10:45.377Z INFO - Waiting for response to warmup request for container one-site-scheduler-app_0_d429ab9d. Elapsed time = 226.067164 sec
2022-03-30T19:10:49.746Z ERROR - Container scheduler-app_0_d for site one-site-scheduler-app did not start within expected time limit. Elapsed time = 230.4276536 sec
2022-03-30T19:10:49.901Z ERROR - Container scheduler-app_0_d didn't respond to HTTP pings on port: 80, failing site start. See container logs for debugging.
2022-03-30T19:10:49.977Z INFO - Stopping site scheduler-app because it failed during startup.
Although before it stops the container I see the logs from the scheduler app.
I am not sure what am I missing here.
I tried following https://ameshram57.medium.com/azure-app-service-container-container-didnt-respond-to-http-pings-e2e653d867fe#:\~:text=if%20you%20are%20getting%20%E2%80%9C%20ERROR,we%20map%20port%20in%20docker.
this article but no luck.
I learn that Azure app service pings on port 80 by default to check if it gets any response, since my container just ran a console application it never responded to the ping from the app service. To fix that I converted my console app to a single point web app using webhostbuilder and followed the steps from the below,
https://nitishkaushik.com/what-is-webhostbuilder-in-asp-net-core/
This trick worked.
Related
I have a single-instance (NO load balancer) Docker container (NO proxy server) that times out at exactly sixty seconds no matter what I do.
Yes, I'm aware of the many seemingly "duplicate" questions. I've been trying to solve this problem for 40+ hours. I've seen them all.
Every single answer to these questions informs the user that they must change the settings of NGINX or the load balancer.
However, I have NEITHER NGINX or a load balancer for the environment, yet it still times out. I am mostly convinced that this is an AWS bug.
I have an endpoint titled time_test for the mini server I created. When I make a POST request to the endpoint, I get a timeout at exactly 60 seconds (the request throws an exception on my end).
Here's the Python code to make the request.
import requests
url = f"http://...us-east-1.elasticbeanstalk.com/"
time_to_sleep = 65
url += f"time_test?time_to_sleep={time_to_sleep}"
response = requests.post(url=url, timeout=10000)
This throws an HTTPSException error, indicating that the server terminated the response, always at exactly 60 seconds.
However, the logs show a successful response.
My logs (specifically, "eb-docker/containers/eb-current-app/eb-blahblah-stdouterr.log) shows
[01/Jun/2022 22:05:49] "POST /time_test?time_to_sleep=65 HTTP/1.1" 200 -
Note the 200 successful status code.
I'm going to continue to find an answer to this problem, which seemingly has none, and will report back if so. Any help with how to change the environment to accept >60 second requests would be greatly appreciated. Please don't reply, "You should have shorter request times." Not helpful or applicable.
(Platform = Docker running on 64bit Amazon Linux 2/3.4.10)
Related:
How to increase FastAPI timeout in Docker to be deployed on AWS EB?
Elastic Beanstalk WebSocket Connection Dropped
PHP beanstalk application giving 504 errors
Blazor Server Side - Frequent 504 errors in AWS environment
504 error on aws elastic beanstalk
Deploying ebextensions on Elastic beanstalk and EC2
AWS bug. It magically started working after I reported this issue to support. No changes. Considering it magically stopped working, that's the conclusion I've come to.
I am trying to hit an https client api which is working fine on postman(gives response in 800ms) and in local mule flow but it is not working fine on cloudhub . I am getting Connect Timeout error. It tries connecting for 30 secs(as per logs) and then gives HTTP:CONNECTIVITY error.
failed: Connect timeout.
errorType=HTTP:CONNECTIVITY
cause=org.mule.extension.http.api.error.HttpRequestFailedException
Response Timeout that I have set is 5 mins.
The flow was working fine when deployed on cloudhub before.It stopped working a few days ago though I didn't make any changes to my code.I am unable to debug this issue as it is not reproducible on my local env(it works perfectly). Any help would be appreciated.
There are 4 different types of general timeouts mule HTTP calls offer. Each has its own differences.
Connection Idle Timeout
Response Timeout
Max Idle Timeout
Query or Transactions Timeout ( Applies for DB Connectors)
Since you are getting
HTTP:CONNECTIVITY ERROR.
Applying a 5 min Response Timeout doesn't help.
Response Timeout (means taking longer time to respond) should be worried only after Establishing a Connection Handshake.
Your problem is with the Connection itself.
The only possible way you could try fixing this is by Applying a Connection Idle Timeout and a Reconnection Strategy with some frequency gaps.
Since you are so sure about tests in local. I suggest you the below two steps:
1. Try using the same HTTP connector configuration in a separate new mule APP. Try with a simple listener and the failing requestor. Also add one more freely available online REST services into your code in other extra flow. Now try to test both. See which one is working and which is failing.
This would tell if it's a real HTTP CONNECTIVITY problem or anything else related to some mule bug.
2. Check your configurations once again and make sure if your hitting the same endpoint in the cloudhub version.
Finally, I hope you did not accidentally put any proxy conf in the local version.
If it was working, probably there was a networking change in the other side that prevents access from the CloudHub application. You didn't share the URL so it is not clear if it is an internal host or a public host. We also don't know if there is some kind of whitelisting on the server side.
You can test connectivity to the HTTP host and port using the Network Tools application, to see if it accessible from your CloudHub environment.
I am trying to intergrate our mattermost with zabbix to receive notifications on alerts. I've followed up the instructions on this link. We are using Zabbix 4.4 with MM 5.19.
After enabling the integration, No alerts are being posted on Mattermost. I tried testing the Media type on Administration > Media Types > Mattermost > Test.
I've added the following as the parameters, but it throws the error : Connection timeout of 3 seconds exceeded when connecting to Zabbix server "localhost".
bot_token : {Token generated for the Bot in Mattemost}
mattermost_url : {https://mattermost.our-company.com}
send_mode : alarm
Tried changing {ZABBIX_URL} to both http://127.0.0.1 and http://zabbix.our-company.com (The DNS is resolved only internally, but our mattermost is available on public network) but none of them work.
I checked the logs inside /var/log/zabbix but no error or anything. I even tried putting the zabbix logs to Debug mode but no luck in any case, the only Debug log I've got is the following :
2063:20200216:090224.146 trapper got '{"request":"alert.send","sid":"74095b240dd6783618571516f029187a","data":{"parameters":{"zabbix_url":"{$ZABBIX.URL}","send_mode":"alarm","send_to":"{ALERT.SENDTO}","event_tags":"{EVENT.TAGS}","event_name":"{EVENT.NAME}","event_nseverity":"{EVENT.NSEVERITY}","event_ack_status":"{EVENT.ACK.STATUS}","event_value":"{EVENT.VALUE}","event_update_status":"{EVENT.UPDATE.STATUS}","event_date":"{EVENT.DATE}","event_time":"{EVENT.TIME}","event_severity":"{EVENT.SEVERITY}","event_opdata":"{EVENT.OPDATA}","event_id":"{EVENT.ID}","event_update_message":"{EVENT.UPDATE.MESSAGE}","trigger_id":"{TRIGGER.ID}","trigger_description":"{TRIGGER.DESCRIPTION}","host_name":"{HOST.NAME}","host_ip":"{HOST.IP}","event_update_date":"{EVENT.UPDATE.DATE}","event_update_time":"{EVENT.UPDATE.TIME}","event_recovery_date":"{EVENT.RECOVERY.DATE}","event_recovery_time":"{EVENT.RECOVERY.TIME}","bot_token":"qs3rkqdappy6i8gs3a8871phxc","mattermost_url":"https:\/\/mattermost.our-company.com"},"mediatypeid":"7"}}'
What can be the issue? Is there a way to "debug" and find the root cause of this error? Any help is appreciated! Note that right now we have integrated Slack with Zabbix and it's working fine, but we are moving to Mattermost and therefore, we need to migrate the integrations as well.
We found out the issue with our Network Admin. The problem was that our Zabbix server was trying to resolve Mattermost name from local network route (i.e. 192.168.x.x) and it kept failing, therefore, no SSL connection could be initiated.
It seems that Zabbix integration tests' error messages are quite generic and sometimes, misleading. Thorough investigation is needed for finding out the root cause.
I've recently been having problems with my server which become unresponsive after certain period of time.
Basically after a certain amount of usage & time my node.js app stops responding to requests. I don't even see routes being fired on my console and the HTTP calls from my client (Android app) don't reach the server anymore. But after restart my node.js app server everything starts working again, until things inevitable stop again. The app never crashes, it just stops responding to requests.
I'm not getting any errors, and I've made sure to handle and log all DB connection errors so I'm not sure where to start.
Any clue as to what might be happening and how I can solve this problem?
Here's my stack:
Node.js on Digital Ocean server with Ubutnu 14.04 and Nginx (using Express 4.15.2 + PM2 2.4.6)
Database running MySQL (using node-mysql)
This Monday 24th, I had a problem with a container and Secure Gateway Client in Bluemix. The container was stopped and SecureGatewayClient was inhibited (it answered error 500 but it showed Started)
Is it possible to send an alert for a Container of Bluemix, for example, the alert will send an email or call an API if the container will stop?
In the case of SecureGatewayClient, I think to monitor the services through the SecureGateway, every 5 minutes I will test the services, but I can accept more ideas...
I can't really speak to potential container issues, but I can provide some details on how the Secure Gateway Client works. The Secure Gateway Client runs as a clustered process where the actual connective pieces are worker processes beneath a single management process. Because of this, if the worker process goes down, the container is essentially none the wiser as long as the management process is still running, as the management process is the entry point for the container.
The Secure Gateway Client supports a --service option that will cause the management process to monitor the worker count. Should the worker count reach 0, the manager will create new workers with the credentials passed on startup.
For example, starting with:
ibmcom/secure-gateway-client myGatewayID -t myGatewaySecurityToken --service
would spawn a worker that will attempt to connect to myGatewayID. Should that worker process terminate for some reason, the management process would create a new worker within 60s as a replacement.