Scheduler with Cloud Foundry

Once your application runs on the cloud, you have several ways to schedule jobs.

1. If you have multiple instances of your app, all instances will run the job that you may want to run only once. This is probably not what you want especially if you want to manipulate data.
2. You could handle this manually to run the job only from one instance. It’s requiring some work and it won’t be scalable except if you add more code to handle it.
3. In Java, you could use Quartz. It’s a nice library that I’ve used a lot in the past, but i find it overkill to run a simple job and not cloud friendly.
4. You can use Scheduler for PCF (Cloud Foundry) which I will go more in detail in a second.

5. Use a provider such as IronWorker.

Spring @ Scheduled annotation

Spring framework provides a very handy annotation to run a method with a Cron expression. While it’s very convenient, it will run the code as many times that you have instances on the cloud.

You can find the source code in my GitHub repo.

Logs example

2018-02-28T10:58:00.00-0500 [APP/PROC/WEB/0] OUT IP Address: 515d00e6-def4-4d08-6745-7bf9/10.255.22.200, Thread: pool-1-thread-1, Time: 15:58:00.001
2018-02-28T10:58:00.00-0500 [APP/PROC/WEB/1] OUT IP Address: 50c8060c-bb62-474a-69e8-7f43/10.251.58.164, Thread: pool-1-thread-1, Time: 15:58:00.001
2018-02-28T10:59:00.00-0500 [APP/PROC/WEB/0] OUT IP Address: 515d00e6-def4-4d08-6745-7bf9/10.255.22.200, Thread: pool-1-thread-1, Time: 15:59:00.001
2018-02-28T10:59:00.00-0500 [APP/PROC/WEB/1] OUT IP Address: 50c8060c-bb62-474a-69e8-7f43/10.251.58.164, Thread: pool-1-thread-1, Time: 15:59:00
2018-02-28T11:00:00.00-0500 [APP/PROC/WEB/0] OUT IP Address: 515d00e6-def4-4d08-6745-7bf9/10.255.22.200, Thread: pool-1-thread-1, Time: 16:00:00.001
2018-02-28T11:00:00.00-0500 [APP/PROC/WEB/1] OUT IP Address: 50c8060c-bb62-474a-69e8-7f43/10.251.58.164, Thread: pool-1-thread-1, Time: 16:00:00.001

The above sample has 2 running instances and you can notice in the logs that both of them are running the job every minute.

PCF Scheduler

Add PCF Scheduler service

Before deploying the app, you have to create a new Scheduler for PCF service available from the Marketplace or from the CF cli:

cf create-service scheduler-for-pcf standard pcf-scheduler

Once the pcf-scheduler app is deployed and bound to the pcf-scheduler service (Great names right? ;)), you can add a cron job to run the code every minute. You will need to install the CF cli plugin. The plugin is not available from the CF cli plugin page. Only from the Scheduler for PCF product page.

cf create-job pcf-scheduler my-demo ".java-buildpack/open_jdk_jre/bin/java -cp ./BOOT-INF/classes ninja.spring.pcfscheduler.Scheduler"
cf schedule-job my-demo "0/1 * ? * *"

The first command will create a job with the command to execute. Let me know if you find a more elegant way :) Btw you could run anything (Shell script, etc)

The second command will associate a cron schedule to the previously created job. Notice the syntax? Somehow it is NOT a regular cron expression and there is no documentation about it (custom Cron).

Logs example

2018-03-01T11:58:00.72-0500 [CELL/0] OUT Creating container
2018-03-01T11:58:01.48-0500 [CELL/0] OUT Successfully created container
2018-03-01T11:58:06.24-0500 [APP/TASK/08e8c101-f36c-4d21-99e5-9976e7fc6855-|-051838d5-9010-44d5-8a79-8196276211e6/0] OUT IP Address: 91c6f6a4-0ba0-4940-9f86-a849e734328c/10.255.210.170, Thread: main, Time: 16:58:06.226
2018-03-01T11:58:06.25-0500 [APP/TASK/08e8c101-f36c-4d21-99e5-9976e7fc6855-|-051838d5-9010-44d5-8a79-8196276211e6/0] OUT Exit status 0
2018-03-01T11:58:06.27-0500 [CELL/0] OUT Stopping instance 91c6f6a4-0ba0-4940-9f86-a849e734328c
2018-03-01T11:58:06.27-0500 [CELL/0] OUT Destroying container
2018-03-01T11:58:07.03-0500 [CELL/0] OUT Successfully destroyed container
2018-03-01T11:59:00.76-0500 [CELL/0] OUT Creating container
2018-03-01T11:59:01.32-0500 [CELL/0] OUT Successfully created container
2018-03-01T11:59:06.80-0500 [APP/TASK/08e8c101-f36c-4d21-99e5-9976e7fc6855-|-051838d5-9010-44d5-8a79-8196276211e6/0] OUT IP Address: 2033bbff-65c4-426e-a07a-ceaef072a8b5/10.249.187.45, Thread: main, Time: 16:59:06.793
2018-03-01T11:59:06.81-0500 [APP/TASK/08e8c101-f36c-4d21-99e5-9976e7fc6855-|-051838d5-9010-44d5-8a79-8196276211e6/0] OUT Exit status 0
2018-03-01T11:59:06.82-0500 [CELL/0] OUT Stopping instance 2033bbff-65c4-426e-a07a-ceaef072a8b5
2018-03-01T11:59:06.82-0500 [CELL/0] OUT Destroying container
2018-03-01T11:59:07.33-0500 [CELL/0] OUT Successfully destroyed container

Notice the creation of a new container that will execute the code only once every minute. It takes about 5 seconds to create a container and to start the job. Pretty cool huh?

Demo project

The demo project source code is available on GitHub https://github.com/christophe-f/pcf-scheduler-demo