JobRunr Pro

Dynamic Queues

Dynamic Queues - also knows as load-balancing or multi-tenancy support - guarantees some fair-use!

Are you running a multi-tenant application? Or do you have diverse types of jobs where certain types of jobs could potentially trigger peak workloads? Use JobRunr’s built-in dynamic queues to make sure that there is some fair-use and all jobs get a fair amount of resources!

▶ See it in the live demo

Walk through Step 11 — Fair Play in our hosted neobank demo. No install needed.

Open this step in the demo →
Note

JobRunr Pro supports unlimited dynamic queues and they can be used together with the priority queues.

Example Use Cases

  • In a multi-tenant application where each tenant can initiate their own jobs, JobRunr ensures fair-use processing. Regardless of whether one tenant generates millions of jobs, the system guarantees that jobs from other tenants are also duly processed.
  • In a diverse workload environment, certain jobs, such as mass mailings, could potentially trigger surges by generating millions of subsidiary child jobs that require processing. However, it’s crucial that other types of jobs remain unaffected by these spikes and continue to be processed smoothly.

Load-balancing types

JobRunr supports four different types of load-balancing:

  • Round Robin Dynamic Queues: here, each dynamic queue receives the same amount of resource usage
  • Weighted Round Robin Dynamic Queues: here, certain dynamic queues can be configured with an optional weight. A queue with a weight of 2 will be checked twice as often as a queue with a weight of 1.
  • Fixed Worker Size Dynamic Queues: here, a certain number of workers are reserved, so only jobs from the assigned queue can run on those reserved workers. Could be of use when you want some of your jobs to run as soon as they are enqueued, without waiting for other jobs enqueued earlier to finish processing.
  • Lenient Fixed Worker Size Dynamic Queues: here, a certain number of workers are reserved, so only jobs from the assigned queue can run on those reserved workers. If there are more jobs than reserved workers, workers from the unreserved pool will run the extra jobs when possible.

Dashboard

If you’re using this feature, you can also enable an extra Dynamic Queues view in the dashboard. This view gives an immediate overview of the amount of jobs per dynamic queue.

An overview of all the different dynamic queues

Configuration

Round Robin Dynamic Queues

With round robin, each dynamic queue receives the same amount of resource usage. This means that on average, JobRunr processes the same amount of jobs from all dynamic queues.

Configuring Round Robin Dynamic Queues by means of the Fluent API

You can configure your round robin dynamic queues easily by means of the Fluent API:

JobRunrPro.configure()
    .useStorageProvider(SqlStorageProviderFactory.using(dataSource))
    .useBackgroundJobServer(usingStandardBackgroundJobServerConfiguration()
            .andWorkerCount(100)
            .andDynamicQueuePolicy(new RoundRobinDynamicQueuePolicy("tenant:")))
    .useDashboard(usingStandardDashboardConfiguration()
            .andDynamicQueueConfiguration("Tenants", "tenant:"))
    .initialize();
Notice the RoundRobinDynamicQueuePolicy where we add the label prefix 'tenant:'.
We also enable the extra Dynamic Queue view in the dashboard and name it 'Tenants'

Configuring Round Robin Dynamic Queues by means of Spring Boot Properties

You can also enable the round robin dynamic queues easily via Spring properties:

jobrunr.jobs.dynamic-queue.round-robin.label-prefix=tenant: 
jobrunr.jobs.dynamic-queue.round-robin.title=Tenants

Weighted Round Robin Dynamic Queues

With weighted round robin, certain dynamic queues can be configured with an optional weight. A queue A with a weight of 2 will be checked twice as often as a queue B with a weight of 1. This means that on average, JobRunr processes twice as many jobs for queue A compared to queue B.

Configuring Weighted Round Robin Dynamic Queues by means of the Fluent API

You can again easily configure your weighted round robin dynamic queues by means of the Fluent API:

JobRunrPro.configure()
    .useStorageProvider(SqlStorageProviderFactory.using(dataSource))
    .useBackgroundJobServer(usingStandardBackgroundJobServerConfiguration()
            .andWorkerCount(100)
            .andDynamicQueuePolicy(new WeightedRoundRobinDynamicQueuePolicy("tenant:", Map.of("Tenant-A", 5))))
    .useDashboard(usingStandardDashboardConfiguration()
            .andDynamicQueueConfiguration("Tenants", "tenant:"))
    .initialize();
Notice the WeightedRoundRobinDynamicQueuePolicy where we add the label prefix 'tenant:'.
'Tenant-A' is configured with a weight of 5 meaning that it will get 5 times more resources than other tenants.

Configuring Weighted Round Robin Dynamic Queues by means of Spring Boot Properties

You can also enable the weighted round robin dynamic queues easily via properties:

jobrunr.jobs.dynamic-queue.weighted-round-robin.label-prefix=tenant:
jobrunr.jobs.dynamic-queue.weighted-round-robin.title=Tenants
jobrunr.jobs.dynamic-queue.weighted-round-robin.queues.tenantB=5

You can also create the configuration programatically by creating a dynamicQueuePolicy bean yourself in the same vein as the one passed in in the above Fluent API example.

Fixed amount of reserved workers

With fixed amount of reserved workers, only jobs from the assigned queue can run on those reserved workers. Could be of use when you want some of your jobs to run as soon as they are enqueued, without waiting for other jobs enqueued earlier to finish processing.

Note

When you reserve workers for certain jobs, those jobs are restricted to their dedicated workers and won’t run on unreserved ones. This effectively caps how many of those jobs a server can process at the same time. If you want them to also use unreserved workers when capacity is available, use the lenient variant. You can reserve a fix amount of workers for different queues using the fluent API or properties as follows:

Fluent API 

JobRunrPro.configure()
    .useStorageProvider(SqlStorageProviderFactory.using(dataSource))
    .useBackgroundJobServer(usingStandardBackgroundJobServerConfiguration()
            .andWorkerCount(20)
            .andDynamicQueuePolicy(new FixedSizeWorkerPoolDynamicQueuePolicy("tenant:", Map.of("Tenant-A", 6, "Tenant-B", 3)))
    .useDashboard(usingStandardDashboardConfiguration()
            .andDynamicQueueConfiguration("Tenants", "tenant:"))
    .initialize();
Notice the FixedSizeWorkerPoolDynamicQueuePolicy where we add the label prefix tenant. We’ve reserved 9 workers out of 20. We also enable the extra Dynamic Queue view in the dashboard and name it ‘Tenants’

Properties (Spring Boot) 

jobrunr.jobs.dynamic-queue.fixed-worker-pool-size.queues.Tenant-A=6
jobrunr.jobs.dynamic-queue.fixed-worker-pool-size.queues.Tenant-B=3
jobrunr.jobs.dynamic-queue.fixed-worker-pool-size.label-prefix=tenant:
jobrunr.jobs.dynamic-queue.fixed-worker-pool-size.title=Tenants

jobrunr.jobs.dynamic-queue.weighted-round-robin.label-prefix=tenant:
jobrunr.jobs.dynamic-queue.weighted-round-robin.title=Tenants
jobrunr.jobs.dynamic-queue.weighted-round-robin.queues.tenantB=5

You can also create the configuration programatically by creating a dynamicQueuePolicy bean yourself in the same vein as the one passed in in the above Fluent API example.

Lenient fixed amount of reserved workers

With a lenient fixed number of reserved workers, job processing behaves the same as the strict variant, except that jobs with dedicated workers can also run on unreserved workers when capacity is available.

You can reserve a lenient fix amount of workers for different queues using the fluent API or properties as follows:

Fluent API 

JobRunrPro.configure()
    .useStorageProvider(SqlStorageProviderFactory.using(dataSource))
    .useBackgroundJobServer(usingStandardBackgroundJobServerConfiguration()
            .andWorkerCount(20)
            .andDynamicQueuePolicy(new LenientFixedSizeWorkerPoolDynamicQueuePolicy("tenant:", Map.of("Tenant-A", 6, "Tenant-B", 3)))
    .useDashboard(usingStandardDashboardConfiguration()
            .andDynamicQueueConfiguration("Tenants", "tenant:"))
    .initialize();
Notice the LenientFixedSizeWorkerPoolDynamicQueuePolicy where we add the label prefix tenant. We’ve reserved 9 workers out of 20. We also enable the extra Dynamic Queue view in the dashboard and name it ‘Tenants’

Properties (Spring Boot) 

jobrunr.jobs.dynamic-queue.lenient-fixed-worker-pool-size.queues.Tenant-A=6
jobrunr.jobs.dynamic-queue.lenient-fixed-worker-pool-size.queues.Tenant-B=3
jobrunr.jobs.dynamic-queue.lenient-fixed-worker-pool-size.label-prefix=tenant:
jobrunr.jobs.dynamic-queue.lenient-fixed-worker-pool-size.title=Tenants

Usage

Using dynamic queues could not have been easier thanks to Job Labels:

Using the @Job annotation:

@Job(name = "Job %1 for %0", labels = {"tenant:%0", "slow-job"})
public void runBackgroundWork(String tenant, int index) {
    // business code here
}
We can use the @Job annotation and note how we re-use the previously configured label prefix 'tenant:'.
This can be done both hardcoded or dynamic by means of a placeholder in the label, like in this example.

Using the JobBuilder pattern:

If you prefer the JobBuilder pattern, this is also really easy:

jobScheduler.create(aJob()
    .withName("Job " + i + " for tenant " + tenant)
    .withLabels("tenant:" + tenant)
    .withJobLambda(() -> myService.runBackgroundWork(input)))
We can use the JobBuilder to create a Job and assign it a label.
Note how we re-use the previously configured label prefix `tenant:`.