Frequently Asked Questions

Some frequently asked questions about JobRunr

BackgroundJobServer FAQ

Does JobRunr need open ports for distributing jobs?

No, JobRunr does not require an open port for distributing the workload - this is orchestrated via the StorageProvider.

How is the coordination between different nodes done?

Each BackgroundJobServer registers itself on startup in the StorageProvider. For an RDBMS, this is a plain old table called jobrunr_backgroundjobservers. The master is the server which is the longest running (so, the one that was registered as first node).
Then, every 15 seconds, each BackgroundJobServer updates a lastHeartBeat timestamp. If a node crashes for some reason (this can also be the master node), the lastHeartBeat timestamp is not updated anymore. All other server participating in processing jobs see that the master node is not active anymore and it is removed from the StorageProvider.
Next, the master reelection process starts which is again nothing more than the longest running BackgroundJobServer.

Pro tip: if you are running in a Kubernetes environment, it is best to always keep your first BackgroundJobServer running and scale other pods up and down. This will result in less Master reelection processes and thus less database queries.

What is the role of the master?

The master is a BackgroundJobServer like all other nodes processing but it does some extra tasks:

  • it checks for recurring jobs and schedules them when they are about to run
  • it checks for scheduled jobs and enqueues them when they need to run
  • it checks for orphaned jobs and reschedules them
  • it does some zookeeping like deleting all the succeeded jobs

Job FAQ

What if I don’t want to have 10 retries when a job fails?

You can configure the amount of retries for all your jobs or per job.

  • To change the default for all jobs, just register a RetryFilter with the amount of retries you want using the withJobFilter method in the Fluent API or in case of the Spring configuration, just pass the filter to the setJobFilters method of the BackgroundJobServer class.
  • To change the amount of retries on a single Job, just use the @Job annotation:
@Job(name = "Doing some work", retries = 2)
public void doWork() {
    ...
}

I’m encountering a java.lang.IllegalThreadStateException

While developing, you may encounter the following error:

java.lang.IllegalThreadStateException: Job was too long in PROCESSING state without being updated.
at org.jobrunr.server.JobZooKeeper.lambda$checkForOrphanedJobs$2(JobZooKeeper.java:134)

This is because you stopped a running JVM instance where a BackgroundJobServer was processing a job. When a job is being processed, it is regularly updated with a timestamp so that in case of a node failure, the job can be retried automatically on a different server. The error message you see here, is an example of such a case.