How to configure delay queue in SQS
Hello Everyone
Welcome to CloudAffaire and this is Debjeet.
In the last blog post, we have discussed how to enable long polling in SQS.
https://cloudaffaire.com/how-to-enable-long-polling-in-sqs-queue/
In this blog post, we will discuss how to configure delay queue in SQS.
How to configure delay queue in SQS:
AWS SQS provides delivery delay option to postpone the delivery of new messages to a queue. If delivery delay is defined for a queue, any new message will not be visible to the consumer for the duration of delay. The default (minimum) delay for a queue is 0 seconds. The maximum is 15 minutes.
Delay queues are similar to visibility timeouts because both features make messages unavailable to consumers for a specific period of time. The difference between the two is that, for delay queues, a message is hidden when it is first added to queue, whereas for visibility timeouts a message is hidden only after it is consumed from the queue. To set delay seconds on individual messages, rather than on an entire queue, use message timers to allow Amazon SQS to use the message timer’s DelaySeconds value instead of the delay queue’s DelaySeconds value.
Note: For standard queues, the per-queue delay setting is not retroactive; changing the setting doesn’t affect the delay of messages already in the queue. But for FIFO queues, the per-queue delay setting is retroactive; changing the setting affects the delay of messages already in the queue.
Next, we are going to configure delay queue in SQS through AWS CLI
Prerequisite for this demo:
- AWS SQS FIFO queue.
- AWS CLI configured with required access.
Step 1: Configure AWS CLI.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
## Create an EC2 instance with AWS Linux 2 AMI ## Create a role with admin access and attach to your instance ## Install aws cli and jq packages if not installed already ## Create a script to configure AWS cli vi assume_role.sh ----------------------- ## Replace #!/bin/bash curl http://169.254.169.254/latest/meta-data/iam/security-credentials/ export AWS_ACCESS_KEY_ID=$(cat cred.json| jq .AccessKeyId | xargs) export AWS_SECRET_ACCESS_KEY=$(cat cred.json| jq .SecretAccessKey| xargs) export AWS_SESSION_TOKEN=$(cat cred.json| jq .Token| xargs) export AWS_EXPIRATION=$(cat cred.json| jq .Credentials.Expiration| xargs) rm -f cred.json ------------------------ :wq ## Execute the script chmod +x assume_role.sh sh assume_role.sh ## Check if aws cli working aws sts get-caller-identity |
Step 2: Query the list of queues available and copy the queue URL.
|
1 |
aws sqs list-queues --region ap-south-1 |
Step 3: Get the current queue attributes.
|
1 2 3 4 5 |
## Get current queue attributes for your FIFO queue aws sqs get-queue-attributes \ --queue-url --region ap-south-1 \ --attribute-names All |
Note: Currently no delay queue has been configured (DelaySeconds = 0).
Step 4: Enable delay queue of 10 minutes.
|
1 2 3 4 5 |
## Configure delay queue aws sqs set-queue-attributes \ --queue-url --region ap-south-1 \ --attributes '{"DelaySeconds": "600"}' |
Step 5: Send a message to the queue.
|
1 2 3 4 5 6 7 |
## Send a message to your queue aws sqs send-message \ --queue-url --message-body "message one" \ --message-group-id "mymsggrpid1" \ --message-deduplication-id "mymsgddupid1" \ --region ap-south-1 |
Step 6: Try to consume the message within 10 mins from executing the above command.
|
1 2 3 4 |
## Try to receive message aws sqs receive-message \ --queue-url --region ap-south-1 |
Observation: We have enabled delay of 10 minutes and since this is a FIFO queue, the delay will be applied to all messages including the existing ones. If we send a message and try to consume it within 10 minutes, we will receive an empty response even in case of long polling.
Hope you have enjoyed this article. In the next blog post, we will discuss the dead letter queue in SQS.
To get more details on AWS SQS, please refer below AWS documentation
https://docs.aws.amazon.com/sqs/index.html