Carbon Aware Processing

Use Carbon Aware Job Processing to optimize the carbon footprint when scheduling (recurring) jobs.

Carbon Aware Job Processing is a unique JobRunr feature that enables the scheduling of (recurring) jobs at the optimal carbon time; when the lowest amount of CO2 is being generated. This is made possible by integrating with external energy data providers such as the ENTSO-E services for the European Union that provide actual energy data for the coming day(s).

By adding a margin to the schedule of your jobs, JobRunr will execute them when the lowest amount of CO2 is being generated. A few examples:

// Schedule a daily job between 14pm and 18pm
jobScheduler.scheduleRecurrently("id-1", CarbonAware.dailyBetween(14, 18), x -> x.doWork())
// Schedule a daily job between 13pm and 19pm
jobScheduler.scheduleRecurrently("id-2", "0 16 * * * [PT3H/PT3H]", x->doWork())

These recurring jobs will create jobs in the AWAITING state (see Pending Jobs in the Dashboard), ready to be SCHEDULED at the optimal time. JobRunr guarantees that all jobs will be executed within the margin; even if there is no data available or the margin is too small.

More examples and details on how to schedule carbon aware jobs can be found in the Background methods: Carbon aware jobs section.

Architectural overview

For jobs to be scheduled carbon aware, JobRunr needs to fetch carbon information from the JobRunr Carbon Intensity API that acts as a buffer between JobRunr and ENTSO-E or any other future data provider. The API is hosted at api.jobrunr.io/carbon-intensity and is configurable in JobRunr Pro.

Important

Please make sure that the firewall allows the JobRunr server to reach api.jobrunr.io. If not, JobRunr will fall back to regular scheduling. See carbon aware jobs: important remarks.

Once it has carbon intensity data, you can add slack to a certain job schedule by providing a margin. JobRunr will then optimize that job within the specified margin based on the carbon intensity data.

Below is a schematic overview of how this works:

The Carbon Aware API Context Diagram.

Available Areas

The Carbon Intensity API provides carbon intensity forecasts through various data providers, each covering different geographical regions.

Data Providers

Below are the data providers integrated with the Carbon Intensity API:

Areas

The following table lists all geographical areas currently supported by the Carbon Intensity API:

Configuration

To enable Carbon Aware Job Processing, configure a CarbonAwareJobProcessingConfiguration with your area code so that the correct energy data is taken into account. If you’re unsure which region to select, browse the supported areas above.

Tip

Selecting an area, by clicking on a row, in the above list of supported areas will automatically updated the below configuration examples to your chosen area.

JobRunr
    .configure()
    // ...
    .useBackgroundJobServer(usingStandardBackgroundJobServerConfiguration()
            .andCarbonAwaitingJobsRequestSize(1000)
            .andCarbonAwareJobProcessingConfiguration(
                usingStandardCarbonAwareJobProcessingConfiguration()
                    .andAreaCode("BE")
                    .andDataProvider("ENTSO-E")
            ))
    // ...
Important

If you do not specify any carbon aware processing config, thus not enabling the carbon aware feature, but do schedule jobs with carbon aware margins, the jobs will still be scheduled at their usual time without taking the margins into account.

The awaiting jobs request size allows to set the maximum number of carbon aware jobs to update from awaiting to scheduled state per database round-trip. If not set, it will default to 1000.

On the carbon aware job processing configuration class, the following parameters can be configured:

  • enabled—Enables the Carbon Aware feature. The usingStandardCarbonAwareJobProcessingConfiguration() Fluent API enables this by default. Without it, pending jobs will still be scheduled at their preferred time, without taking the margin into consideration.
  • areaCode—Allows to set the ISO 3166-2 areaCode of your datacenter (the area where your application is hosted; e.g. “BE”, “US-CA”, “IT-NO”) in order to have more accurate carbon emissions forecasts. Unless specified, the forecast may be from any dataProvider that supports the areaCode. If you do not specify an area code, the Carbon Intensity API will try to determine the area of the JobRunr cluster callee based on IP. See above for the list of supported areas.
  • dataProvider—Allows to set your preferred carbon intensity forecast dataProvider (e.g. “ENTSO-E”, “Azure”, …). If you do not specify a data provider, the first region matching the area code will be returned. See above for the list of supported carbon intensity providers.
  • externalCode—Allows to set the code of an area as defined by your specified dataProvider in order to have more accurate carbon emissions forecasts (e.g. “IT-North”).
  • externalIdentifier—Allows to set the identifier of an area as defined by your specified dataProvider in order to have more accurate carbon emissions forecasts (e.g. “10Y1001A1001A73I”).
  • apiClientConnectTimeout—Allows to set the connect timeout for the API client (defaults to 3 seconds).
  • apiClientReadTimeout—Allows to set the read timeout for the API client (defaults to 3 seconds).
  • apiClientRetriesOnException—Configures the API client amount of retries when the call throws an exception (defaults to 3).
  • JobRunr Pro andCarbonIntensityApiUrl—Allows to set a custom Carbon Intensity API URL to create your own implementation. The area code, data provider, external code, and external provider settings will be passed in as a request parameter.
  • pollIntervalInMinutes—Allows to configure how often Carbon Aware Awaiting jobs will be picked up and processed (defaults to 5 minutes).
Note

You can only set either areaCode, externalCode, or externalIdentifier as region keys. A dataProvider is required in conjunction with the externalCode or externalIdentifier.

Once you have Carbon Aware Job Processing configured, it is time to take a look at how to enhance your jobs with the carbon aware margin: see Carbon aware jobs in the docs.