Running a script on an AWS server


I have a script that I need to run once a day that requires a lot of memory. I would like to run it on a dedicated amazon box.

Is there some automated way to build a box, download all required software (like ruby) and then run my script. After the script is ran, I would like to shutdown the box.

The two options I can think of are:

  1. I am thinking about hacking EMR to do this. (My script is a mapper against an empty directory)
  2. Chef – This seemed like too much for one simple script.


You can accomplish setting up a new EC2 instance on startup using the official Ubuntu AMIs, the official Amazon Linux AMIs, and any other AMI that supports the concept of a user-data script.

Create a script (bash, Perl, Python,

  1. whatever) that starts with #!
  2. Pass this script as the user-data when running the EC2 instance.
  3. The script will automatically be run as root on the first boot.

Here’s the article where I introduced the concept of a user-data script:

Automate EC2 Instance Setup with user-data Scripts

Your user-data script can install the required software, configure it, install your work script, and set up a cron job that runs the work script once a day.


If the installation script don’t take a long time to run (e.g., under an hour or few) then you don’t even have to run a single dedicated instance 24 hours a day. You can instead use an approach that lets AWS start an instance for you on a regular schedule.

Here’s an article I wrote that provides details on this approach with sample commands:

Running EC2 Instances on a Recurring Schedule with Auto Scaling

The general approach is to use Auto Scaling to start an instance with your user-data script on a regular schedule. Your job will terminate the instance when it has completed. They key is to suspend Auto Scaling’s normal desire to re-start instances that terminate so that you don’t pay for a running instance until the next time your job starts.

Leave a Reply