Question:
Is there a way to run a Lambda on every DynamoDb table record?
I have a Dynamo table with name, last name, email and a Lambda that takes name, last name, email as parameters. I am trying to configure the environment such that, every day, the Lambda runs automatically for every value it finds within Dynamo; can’t do all the records in one Lambda as it won’t scale (will timeout once more users are added).
I currently have a CloudWatch rule set up that triggers the lambda on schedule but I had to manually add the parameters to the trigger from Dynamo – It’s not automatic and not dynamic/not connected to dynamo.
—
Another option would be to run a lambda every time a DynamoDb record is updated… I could update all the records weekly and then upon updating them the Lambda would be triggered but I don’t know if that’s possible either.
Some more insight on either one of these approaches would be appreciated!
Answer:
Is there a way to run a Lambda on every DynamoDb table record?
For your specific case where all you want to do is process each row of a DynamoDB table in a scalable fashion, I’d try going with a Lambda -> SQS -> Lambdas fanout like this:
- Set up a CloudWatch Events Rule that triggers on a schedule. Have this trigger a
dispatch
Lambda function. - The
dispatch
Lambda function’s job is to read all of the entries in your DynamoDB table and write messages to ajobs
SQS queue, one per DynamoDB item. - Create a
worker
Lambda function that does whatever you want it to do with any given item from your DynamoDB table. - Connect the
worker
Lambda to thejobs
SQS queue so that an instance of it will dispatch whenever something is put on the queue.