In Apache Airflow, I can specify the worker queue upon starting the worker.
I have a use case where I would like to change the queue the worker is using live so that the existing worker will pull new jobs from that queue.
Is this possible?
As far as I can tell, no. You can launch a worker with airflow worker -q <queue>, but there doesn't seem to be a way to kill that worker, or interact with it at all once it's spawned.
Related
I met a problem about the runit service bootstrap. The service will setup and curl an external service endpoint to get its data. It will restart over and over again and keep sending requests until the data is ready for the external service.
So I thought there is remediation to reduce the requests which delay the runit service to run the script for the first time. But I could not find any way to delay the runit service. Does runit support delay its service to the first start? Or the solution has any improvement?
BTW, the service will setup with system boot.
You might try changing the runlevel of runit so that it doesn't start too soon, but that depends on the dependent process running first. A better solution, described in the documentation, is to use the fact that runit will try to start a service again if it dies, so you can do the following in your run script:
sv start dependent_service || exit 1
# my service code
This will ensure that dependent_service is started first.
A service start by Upstart or systemd and I want hot update it. After update, the new service process is running and then kill the old service process that start by upstart or systemd. A at last, the new process can be monitored by the upstart or systemd as the old process.
You didn't say what your service does. I am answering for the easy case, a network service with short-lived connections, such as an HTTP server.
Have systemd own the socket. Search for "systemd socket activation" (stackoverflow search). I describe how to do it in Go here: https://www.darkcoding.net/software/systemd-socket-activation-in-go/
While the service is running, replace the binary on disk with the new one.
systemctl restart <myservice>
In practice there will often also be some state in your service you will need to persist on shutdown, and load on startup.
The service shutting down might need to wait a brief amount of time until all it's requests complete.
For the more difficult case with many long-lived TCP connections (such as an XMPP server), it's no longer about systemd, you have to have your old and new processes co-ordinate to pass the sockets from one to another. I describe it in Online upgrades in Go, but it's a lot more work.
I am learning laravel 5.4 "queues" chapter. I have a problem about queue:restart command. Because when I test it on my windows 10 platform, I found this command seems just kill queue worker, but not restart worker. So I wonder whether this command does not work on windows or this command is just kill worker but not restart worker? Thanks.
The queue:restart command never actually restarts a worker, it just tells it to shutdown. It is supposed to be combined with a process manager like supervisor that will restart the process when it quits. This also happens when queue:work hits the configured memory limits.
To keep the queue:work process running permanently in the background, you should use a process monitor such as Supervisor to ensure that the queue worker does not stop running.
Source: https://laravel.com/docs/5.4/queues#running-the-queue-worker
This Monday 24th, I had a problem with a container and Secure Gateway Client in Bluemix. The container was stopped and SecureGatewayClient was inhibited (it answered error 500 but it showed Started)
Is it possible to send an alert for a Container of Bluemix, for example, the alert will send an email or call an API if the container will stop?
In the case of SecureGatewayClient, I think to monitor the services through the SecureGateway, every 5 minutes I will test the services, but I can accept more ideas...
I can't really speak to potential container issues, but I can provide some details on how the Secure Gateway Client works. The Secure Gateway Client runs as a clustered process where the actual connective pieces are worker processes beneath a single management process. Because of this, if the worker process goes down, the container is essentially none the wiser as long as the management process is still running, as the management process is the entry point for the container.
The Secure Gateway Client supports a --service option that will cause the management process to monitor the worker count. Should the worker count reach 0, the manager will create new workers with the credentials passed on startup.
For example, starting with:
ibmcom/secure-gateway-client myGatewayID -t myGatewaySecurityToken --service
would spawn a worker that will attempt to connect to myGatewayID. Should that worker process terminate for some reason, the management process would create a new worker within 60s as a replacement.
Can I set a new virtual host using the server object in the nginx.conf configuration file, without restarting the server (and without closing the active connections)?
Thanks
Yes you can.
nginx -s reload
Or you can send SIGHUP to the nginx process.
sudo kill -HUP [nginx's pid]
You can Load New Configuration Using Signals. It will
Read and test a new configuration. If the configuration invalid then do nothing.
When valid, start new processing workers with new configuration. Attach new workers to port listening, log, etc.
Detach old workers from listeners.
Gracefully shutdown old workers.
You can even Upgrade To a New Binary On The Fly. See the same doc.
To perform the reload, send the reload signal:
nginx -s reload