How to enable long polling in SQS queue
Hello Everyone
Welcome to CloudAffaire and this is Debjeet.
In the last blog post, we have discussed SQS queue default visibility timeout
https://cloudaffaire.com/sqs-queue-default-visibility-timeout/
In this blog post, we will discuss how to configure long polling in SQS queue.
How to enable long polling in SQS queue:
AWS SQS is a poll based service, meaning SQS will not push messages to your application but rather your application needs to poll messages from SQS queue. Further, AWS charges you on per request (SendMessage, ReceiveMessage and DeleteMessage) basis so if you poll your SQS queue and its returns an empty response, you will still be charged. The default polling for SQS is short polling, meaning the response is sent without any wait time even if the queue is empty or updated with new messages. Long polling helps reduce the cost of using AWS SQS by eliminating the number of empty responses (when there are no messages available for a ReceiveMessage request) and false empty responses (when messages are available but aren’t included in a response).
SQS is distributed by nature and messages are stored across different servers. In case of short polling, SQS only query a subset of these servers and returns the response. But in case of long polling, SQS tries to query the entire servers within 20 seconds and returns as many messages as possible. The polling behavior of a queue is controlled by ‘Receive Message Wait Time’ attributes and by default is set to 0 representing short polling.
Next, we are going to configure long polling for an SQS queue through AWS CLI
Prerequisite for this demo:
- AWS SQS FIFO queue.
- AWS CLI configured with required access.
Step 1: Configure AWS CLI.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
## Create an EC2 instance with AWS Linux 2 AMI ## Create a role with admin access and attach to your instance ## Install aws cli and jq packages if not installed already ## Create a script to configure AWS cli vi assume_role.sh ----------------------- ## Replace #!/bin/bash curl http://169.254.169.254/latest/meta-data/iam/security-credentials/ export AWS_ACCESS_KEY_ID=$(cat cred.json| jq .AccessKeyId | xargs) export AWS_SECRET_ACCESS_KEY=$(cat cred.json| jq .SecretAccessKey| xargs) export AWS_SESSION_TOKEN=$(cat cred.json| jq .Token| xargs) export AWS_EXPIRATION=$(cat cred.json| jq .Credentials.Expiration| xargs) rm -f cred.json ------------------------ :wq ## Execute the script to get access chmod +x assume_role.sh sh assume_role.sh ## Check if aws cli working aws sts get-caller-identity |
Step 2: Query the list of queues available and copy the queue URL.
|
1 |
aws sqs list-queues --region ap-south-1 |
Step 3: Configure the visibility timeout to 10 seconds for your queue.
|
1 2 3 4 |
## Change visibility timeout to 10 seconds for entire queue aws sqs set-queue-attributes \ --queue-url --attributes '{"VisibilityTimeout": "10"}' |
Step 4: Get the current attributes for your queue.
|
1 2 3 4 5 |
## Get current queue attributes for your FIFO queue aws sqs get-queue-attributes \ --queue-url --region ap-south-1 \ --attribute-names All |
Note: Currently the queue is set for short polling (ReceiveMessageWaitTimeSeconds = 0).
Step 5: Send a message to your queue.
|
1 2 3 4 5 6 7 |
## Send a message to your queue aws sqs send-message \ --queue-url --message-body "message one" \ --message-group-id "mymsggrpid1" \ --message-deduplication-id "mymsgddupid1" \ --region ap-south-1 |
Step 6: Try to receive message couple of times within 10 seconds.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
## Try to receive a message from the queue couple of times aws sqs receive-message \ --queue-url --region ap-south-1 aws sqs receive-message \ --queue-url --region ap-south-1 aws sqs receive-message \ --queue-url --region ap-south-1 |
Observation: Since the queue is set for short polling and visibility timeout is set to 10 seconds, once the message is received the second recevicemessage request returns an empty response. Next, we will enable long polling by defining Receive message wait time seconds value to 20.
Step 7: Configure long polling for the queue.
|
1 2 3 4 5 |
## Configure long polling for the queue aws sqs set-queue-attributes \ --queue-url --region ap-south-1 \ --attributes '{"ReceiveMessageWaitTimeSeconds": "20"}' |
Step 8: Try to receive message a couple of times.
|
1 2 3 4 5 6 7 8 |
## Try to receive message from the queue couple of times aws sqs receive-message \ --queue-url --region ap-south-1 aws sqs receive-message \ --queue-url --region ap-south-1 |
Observation: You will receive the message every time you send receive message request to the queue. Since the visibility timeout is set to 10 seconds and your receive message request wait for maximum 20 seconds the message is again visible to you within the polling time.
Hope you have enjoyed this article. In the next blog post, we will discuss the delay queue in SQS.
To get more details on AWS SQS, please refer below AWS documentation
https://docs.aws.amazon.com/sqs/index.html