I have a worker that uses RabbitMq as the message queue
I want to configure the worker to scale depending on the number of messages in the queue
Am using cloudamqp how do I add a custom scaling trigger in elastic beanstalk
You can poll the RabbitMQ HTTP API: https://raw.githack.com/rabbitmq/rabbitmq-management/rabbitmq_v3_5_6/priv/www/api/index.html
eg. curl https://user:pass#srv1.rmq.cloudamqp.com/api/queues/myvhost?columns=name,messages
Related
One of AWS's published message handling patterns suggests using a message ReceiptHandle in order to extend it's visibility timeout, but I've discovered that when consuming messages in an Elastic Beanstalk worker (via aws-sqsd), the ReceiptHandle is not passed as one of the header parameters.
Does anyone have any suggestions on how I can get the ReceiptHandle for a message?
Thanks.
Sadly you can't do this. EB worker does not provide that for you by default. The retrieval and deletion of msgs from SQS is managed by EB sqs daemon, not your application.
This simplifies deployments as you don't have to worry about interacting with SQS, but then limits what you can do.
I am new to AWS/Database.
Since i am completely beginner to this, any suggestions will be appreciated.
Currently in the project it has been planned like data from AWS database will be pushed using SNS HTTP fanout to external MySql Database.
NOTE :
1.The data will be pushed by the Client using AWS SNS
2. We have no access to the AWS account nor we are planning to have a AWS account.
3. External MySql database is a private database running on Linux Server
I have gone through the Official documentation of AWS SNS, and also some websites. This is all i found :
Use external applications like Zapier to map the data.
Develop some application to map the data.
Is it like using a Servlet application in the receiver side to update the table, or is there any other methods?
AWS DB -----> SNS -----> _________ -----> External MySql DB
Thanks
If you cannot have an AWS Account, you can have your own web server consume the SNS Messages. SNS can deliver messages to an HTTP/HTTPS endpoint in a predefined structure. Read more details here. You can enable such an endpoint on your own server and share your server URL with the AWS Account owner. They can create a subscription from their SNS topic to your endpoint.
For setting up this endpoint, there are many options. ExpressJS is one such popular framework to quickly implement HTTP APIs.
Probably, option two would be more suited, or at least first to be considered. For that option you would have have to develop a lambda function which would receive data from SNS, re-format if needed and upload it to MySQL. So your architecture would look like:
Data--->SNS--->Lambda function---> MySQL
Depending on the amount of incoming data to the SNS, you may add SQS queue as well to the mix, to buffer the records and enable fun-out architecture. For example:
/---> SQS queue 1---> Lambda function 1---> MySQL
Data -->SNS --/
\
\--- SQS queue 2 ---> Lambda function 2, EC2 instance, Container ---> Other destination
Other solutions are possible. But I would first consider the above, before looking into other ways.
I added a SNS topic notification for my autoscaling group
Basically this just posts to SNS Topic and a lambda function uses a webhook to post to a Slack channel. All works fine.
However since adding this notification (almost a week ago)
My ElasticBeanstalk instance health has been showing as 'Degraded':
and the 'Causes' just shows
Auto Scaling group (awseb-e-rf8zhmbjwm-stack-XXXXXXXXX) notifications have been deleted or modified.
My ec2 instances seem healthy, and my app is working fine afaik.
Why does it keep this 'Degraded' state? How can I fix it?
I had wrestled with the same issue for a few days, this is how I fixed it:
Beanstalk does not like it when you change any piece of your environment outside of the Beanstalk configuration (i.e. adding an SNS notification to your autoscaling group) after the environment was created.
What you can do is make all changes to your infrastructure that you want in place - including the SNS notification. Your Beanstalk environment will show as "Degraded" as you know. Then, clone your environment into a new one under your application and then delete the old Degraded environment. Beanstalk will see the cloned environment as unchanged, and will then set the Health status to "OK".
I am using an external monitoring service (not stackdriver)
I wish to monitor the number of unhealthy hosts on my load balancer.
It seems like the google cloud api doesn't expose this metrics
therefore I implemented a custom script that gets the instance groups of the load balancer, get the instances' data (dns) and performs the health check
pretty cumbersome. is there a simple way to do it?
You can use the command 'gcloud compute backend-services get-health' to get the status of each instance in your backend service. This command will provide the current status of each instance, HEALTHY or UNHEALTHY, that is part of your backend service.
Is it possible to get the number of requests sent to a load balancer in AWS?
I am trying to monitor the number of requests that our load balancers are receiving. Both ELB and Application Load Balance (alb).
Is there a way to do this from the cli? or the Javascript sdk?
Amazon CloudWatch has a RequestCount metric that measures "The number of requests received by the load balancer".
The Load Balancer can also generate Access Logs that provide detailed information about each request.
See:
CloudWatch Metrics for Your Classic Load Balancer
CloudWatch Metrics for Your Application Load Balancer