Question:
I have a SQS Queue
from which messages are read by multiple hosts. I want to run some job (business logic) after all the messages in the queue have been processed.
How can I check that the queue is empty?
Yes, I can check for ApproximateNumberOfMessages
and ApproximateNumberOfMessagesNotVisible
queue attributes but these are approximate numbers. I want to stop my hosts polling for messages in the queue when there are no messages left and then run the required job.
Any ideas? Thanks
Answer:
You could simply note empty receives from the API response while you’re polling. Concerning CloudWatch, there is another metric that would be a better fit for this. From the documentation:
NumberOfEmptyReceives
The number of ReceiveMessage API calls that did not return a message.
Units: Count
Valid Statistics: Average, Minimum, Maximum, Sum, Data Samples
(displays as Sample Count in the Amazon SQS console)
Some additional info:
- This metric only gets populated every 5 minutes. If you set up an alarm based on this metric, this means your minimum period should be 5 minutes.
- Sum is the most sensible statistic for your use case. If NumberOfEmptyReceives > 0, your polling job checked the queue and received no messages.
I personally used this metric to set up a cloudwatch alarm that will scale down an autoscaling group that hosts my polling job after the sum of NumberOfEmptyReceives > 0 for several consecutive periods. I like doing consecutive periods because it makes it more evident that the queue was not only empty, but has stayed empty.