Jobs That Do Not Bring Down Your Database
Background jobs and online traffic on one database, without one slowing down the other.
Next session
Next date to be announced
Online, private to registrants. Recording the next day.
RegisterWho it is for
Developers running JobRunr in production, or about to. Architects who have to answer for the batch window and the database it shares with the online channel. DBAs who want to know what that jobs table is doing.
What we assume
You run JobRunr in at least one application, or you attended Foundations. This one is about protecting the database. Getting more jobs through is the Scaling Under Load session in January.
The hour, minute by minute
- 0:00
What a job costs your database
The queries JobRunr runs for one job from enqueue to success, the poll interval, the indexes on the jobs table, and how to read the numbers your database already gives you. Where the load really comes from: the job scheduler, or the code inside the job.
- 0:15
Keeping the jobs table small
Delete policies for succeeded and failed jobs, what to keep for auditing, and what a jobs table of millions of rows does to the dashboard and to your backups. Small job arguments: pass the ID and load the entity inside the job, instead of serializing the whole object into every row.
- 0:25
Fewer, larger queries
What the poll interval really changes (how fast a job is picked up, not how many get through) and what it costs when every server polls. What changes when you run many servers against one database. In JobRunr Pro, the
JobPrefetchQueuefetches jobs in chunks before workers ask for them. - 0:35
Worker count: fixed versus self-learning
Foundations covers how to set the worker count. This part is about your database: what happens when the workers outnumber the connection pool, and why a fixed count is always a guess. Then the
SelfLearningWorkerCapacityPolicyin JobRunr Pro 9, live: we starve the database on purpose and watch JobRunr lower the worker count until connections are fast again. - 0:45
Where JobRunr Pro picks up
Rate limiters and mutexes when the bottleneck is downstream of you. Postgres
LISTEN/NOTIFYfor instant processing without polling. The Analytics dashboard for finding the slow job in a week of data. Labelled as Pro, shown briefly. - 0:50
Your questions
Ten minutes for the questions from the room and from the registration forms: your database, your jobs table, and the job you suspect.
Before the session
- JobRunr running in an application you can look at during the session, ideally one that does real work.
-
Which database you run on, and roughly how many rows sit in your jobs table today. A
SELECT COUNT(*)onjobrunr_jobsis enough. - One job you suspect of being expensive. You will know more about it by the end.
What you leave with
- ✓ A tuning checklist for delete policies, job arguments, poll interval and worker count, per database
- ✓ The demo application, including the script that starves the database
- ✓ The recording, the next day
- ✓ A free Architecture Review where we look at your own job design and your database together
Your trainers
Ronald Dehuysser
Creator of JobRunr, codes the whole hour
Nicholas D'hondt
Host, asks the questions you are thinking
Register for this training
Free. The link arrives by email the day before the session.