I've a long request to handle at post_worker_init hook to download metadata from remote server, which will block the worker for several seconds. Wondering what's the behavior of the worker if a request comes in during this time. Will the arbiter assign the request to the worker, or the worker won't be assigned as it's still calling the callback at post_worker_init hook?
Related
In Apache Airflow, I can specify the worker queue upon starting the worker.
I have a use case where I would like to change the queue the worker is using live so that the existing worker will pull new jobs from that queue.
Is this possible?
As far as I can tell, no. You can launch a worker with airflow worker -q <queue>, but there doesn't seem to be a way to kill that worker, or interact with it at all once it's spawned.
Every time when a client sends any request (say HTTP), the request is received by a load balancer (if set up) and it will redirect the request to one of the instances. Now a connection is established between Client->LB->Server. This will persist as long as the client is constantly sending requests.
But if client stops sending request to the server for a period of time(more then idle time), the load balancer will stop the communication between client and that particular server. Now if the client tries sending the request once again after some period of time then the load balancer should forward that request to some other instance.
What is idle time?
It is a period of time when client is not sending any request to the Load balancer. It generally ranges between 60 to 3600 seconds depending upon the cloud service provider.
Finally my doubt.
Ideally after the idle timeout the load balancer should terminate the existing communication, but this is not the case with GCP's Internal load balancer(i have a PoC in this context too). GCP's load balancer doesnt terminate the communication even after idle time out and maintains it for infinite time. Is there any way one can re configure the load balancer to avoid such infinite time connection?
This Monday 24th, I had a problem with a container and Secure Gateway Client in Bluemix. The container was stopped and SecureGatewayClient was inhibited (it answered error 500 but it showed Started)
Is it possible to send an alert for a Container of Bluemix, for example, the alert will send an email or call an API if the container will stop?
In the case of SecureGatewayClient, I think to monitor the services through the SecureGateway, every 5 minutes I will test the services, but I can accept more ideas...
I can't really speak to potential container issues, but I can provide some details on how the Secure Gateway Client works. The Secure Gateway Client runs as a clustered process where the actual connective pieces are worker processes beneath a single management process. Because of this, if the worker process goes down, the container is essentially none the wiser as long as the management process is still running, as the management process is the entry point for the container.
The Secure Gateway Client supports a --service option that will cause the management process to monitor the worker count. Should the worker count reach 0, the manager will create new workers with the credentials passed on startup.
For example, starting with:
ibmcom/secure-gateway-client myGatewayID -t myGatewaySecurityToken --service
would spawn a worker that will attempt to connect to myGatewayID. Should that worker process terminate for some reason, the management process would create a new worker within 60s as a replacement.
I am using APE (Ajax Push Engine) at my application. The connection between the application and the ape-server is inline push. And I am using APE only to broadcast messages to the clients. I.e. the clients do not send messages to the application or to each other.
My question is: Does APE-Server processes the push requests synchronously or asynchronously.
I.e. if I send a message to push over APE-Server,
does APE-Server responses immediately to the application, and after that it pushes the messages to the clients? (asynchronously) OR
does APE-Server responses to the application after it has pushed the messages to the clients (synchronously)
I am asking this, because sometimes responses from APE-Server to the application last to long (sometimes over 1 minute).
Thanks in advance.
The connection type is asychronous as you embed the server logic in webserver and ape communicates using http with the webserver (client <-> APE <-> web server)
I am building a chat component(where all discussions are saved) for an application and I planned to have a long-polling server as an interface between the client machines and the rabbitmq server, that will parse format and enqueue messages in a job queue(on rabbitmq) to be handled by celery, which will then check if posting is allowed via checking the db for some information, etc, save the messages to a db, and enqueue them in rabbitmq again, this time to be consumed by the long-polling server to be pushed out to the client machines.
I planned to have one connection from the rabbit server to the long-polling server, containing many channels, that will each correspond to a nio socket channel, where a client machine is listening in on the other end.
The long polling server will be written using the Java Nio libraries. My question is, what would be an efficient way of sending the messages from the AMQP channel to the socketchannel to be pushed out to the clients?
From my understanding of NIO, the only time a selectionkey should be registered for OP_WRITE, is when a previously attempted write() call returned 0. In this case though, I am interested in writing to a socketchannel when information is present on the corresponding AMQP channel. How can I alert the socket channel once data comes in on its corresponding amqp channel so that I may send data through the socketchannel to the client?
Just call write()! No need to 'tell' the SocketChannel anything. Then if write() returns zero, do the OP_WRITE stuff.