I have a
SQS Queue from which messages are read by multiple hosts. I want to run some job (business logic) after all the messages in the queue have been processed.
How can I check that the queue is empty?
Yes, I can check for
ApproximateNumberOfMessagesNotVisible queue attributes but these are approximate numbers. I want to stop my hosts polling for messages in the queue when there are no messages left and then run the required job.
Any ideas? Thanks
You could simply note empty receives from the API response while you’re polling. Concerning CloudWatch, there is another metric that would be a better fit for this. From the documentation:
The number of ReceiveMessage API calls that did not return a message.
Valid Statistics: Average, Minimum, Maximum, Sum, Data Samples
(displays as Sample Count in the Amazon SQS console)
Some additional info:
- This metric only gets populated every 5 minutes. If you set up an alarm based on this metric, this means your minimum period should be 5 minutes.
- Sum is the most sensible statistic for your use case. If NumberOfEmptyReceives > 0, your polling job checked the queue and received no messages.
I personally used this metric to set up a cloudwatch alarm that will scale down an autoscaling group that hosts my polling job after the sum of NumberOfEmptyReceives > 0 for several consecutive periods. I like doing consecutive periods because it makes it more evident that the queue was not only empty, but has stayed empty.