Scheduling with iron.io

In a previous post I mentioned Minimum Viable Product (MVP), this is a software release comprising only the core features that allow a product to be deployed, and no more. At travel.cloud we try to stick to MVP principles, building in short iterations and only implementing what needs to be done now; we have more detailed stories and future iterations but we try not to build for the future where possible unless it really does make sense. Our take on MVP is that it isn’t just about only producing lean software, it’s also about leveraging 3rd party solutions that already exist.

One instance where we have used a 3rd party product was with scheduling. It is common for software to perform tasks based on a schedule. Examples of tasks that get scheduled are: downloading a new set of reference data, applying time-based rules to bodies of data, kicking off maintenance tasks etc. When we needed to use a scheduler we knew that scheduled script execution was on the AWS Lambda product roadmap however we didn’t know when it would be available. So knowing our preferred solution would be available at some point in the future and not wanting to create our own scheduler, we looked at the scheduling solutions that already exist.

3rd Party solutions

Some of the common scheduling products available are:

• Cron - has existed in unix since the late 70s and has spawned a number of variants over the years. Tasks are scheduled using a cron expression which allows complex scheduling rules to be given in a reasonably easy to understand way. Cron requires a machine to be on 24x7 as it runs from within the OS so this requires a reserved AWS instance. There isn’t inbuilt support for clustering in cron.
• Quartz - created in Java, this is an enterprise scheduling package that supports clustering. Quartz using expressions for scheduling tasks in the same way cron does although the schedule strings are slightly different. As with cron, quartz requires a machine to be running 24x7 and also a database to persist scheduling information.
• Chronos - airbnb product built to run on mesos. Like quartz, this is a distributed, fault-tolerant scheduler which also requires a machine running mesos to be running 24x7.
• Iron Worker - web based scheduler that runs an uploaded application. Schedules are defined using an GUI, and Iron Worker worker supports clustering and monitoring.

Our chosen solution was to use Iron Worker - this gives us a robust scheduler for minimum work and no ongoing system maintenance or cost.

How we used Iron Worker

Design

The design of how we use a scheduler is very simple. travel.cloud is based on an event driven architecture, so we use this to our advantage in our design. When a scheduled task is triggered, we put an event on the relevant queue and let the interested application pick up the event and deal with it. It’s that simple.

Here is an example of how we use the scheduler:

1. Iron Worker polls the tasks internally
2. When a task run time criteria matches, Iron Worker will run the associated task code
3. The task code we created simply creates an event
4. Event posted to the SQS queue

This design has a number of benefits:

• The actual code that gets executed when the task is triggered is minimal and runs for seconds.
• We only need to learn a minimal part of the schedulers API as the work is done in the application.
• We can easily change scheduler in future, more on this later.

Implementation

The job which Iron Worker runs is a simple java application with a main method. The schedule on which a message will be sent to an SQS queue is largely determined by the settings in Iron Worker.

Our application reads the contents of the payload defined in the job configuration screen and uses the contents to create a connection to SQS.

Here is an example of a payload:

  {
    "tasks": [
      {
        "accessKey": "value",
        "secretKey": "value",
        "region": "value",
        "targetQueue": "value",
        "eventText": "value"
      }
    ]
  }

• accessKey is set to the access key of the AWS user running this job
• secretKey is set to the secret key of the AWS user running this job
• region is set to the AWS region in which the target queue resides
• targetQueue is set to the name of the target queue
• eventText is set to stringified JSON which contains the event name and some message content.

A single task and payload can also post messages to multiple queues, this is useful when using the schedule to kick of a number of tasks at regular intervals, for instance:

 {
    "tasks": [
      {
        "accessKey": "value",
        "secretKey": "value",
        "region": "value",
        "targetQueue": "value",
        "eventText": "value"
      },
      {
        "accessKey": "value",
        "secretKey": "value",
        "region": "value",
        "targetQueue": "value",
        "eventText": "value"
      }
    ]
  }

We have also added one piece of logic which can be optionally used, this simply checks if the date is the last day of the month. If this check is used the task will trigger on the specified day, this enables us to perform financial functions which only happen on the last day of the month, for example:

{
    "tasks": [
      {
        "accessKey": "value",
        "secretKey": "value",
        "region": "value",
        "targetQueue": "value",
        "eventText": "value",
        "eventFrequency": "LAST_DAY_OF_MONTH"
      }
    ]
  }

Future

At AWS re:Invent 2015, scheduled Lambda functions were announced, and our new scheduling functionality is being implemented using Lambda. We will gradually move our existing functionality to Lambda which should be straightforward as the worker for Iron Worker.io doesn’t do very much work at all. Because of this, we have chosen to open source the worker that posts a message to the queue. To view the code and use it for yourself, see our GitHub project.