Retention & History Purge

How long jobs and transitions are kept, the purge sweeps that bound history, and how it interacts with the Job History Policy.

A terminal job does not live forever. Once a job reaches Succeeded, Cancelled, Dead-Lettered, or Quarantined, a keep window starts running, and when it elapses a background sweep deletes the job and everything hanging off it. That keeps the store, the dashboard, and the Monitor bounded without an external cleanup job. Three independent levers shape this lifecycle, and the most common mistake is to conflate them: the Job History Policy decides whether transition history is recorded at all, the per-job transition cap bounds how many entries a single job keeps over its life, and the retention window decides how long a whole terminal job survives before it is purged.

Three levers, kept separate#

Each lever acts at a different point in a job's life. Recording happens at write time, the per-job cap applies as a job churns, and retention deletes the whole job after it goes terminal.

The recording policy decides whether a row is ever written; the transition cap decides how many survive within one job; retention decides when the whole job and its rows are deleted. They are orthogonal gates on the same data.

Retention windows#

Retention is expressed as a RetentionPolicy (in the BackWave.Core namespace). It carries two keep windows, one per terminal class, and the retention clock for a job always starts at the instant the job reached its terminal state, never when it was enqueued.

The property names mention only one state from each pair, so it is easy to assume Cancelled and Quarantined are unmanaged. They are not. KeepSucceeded governs both the clean-success class (Succeeded and Cancelled), and KeepDeadLettered governs both the failure class (Dead-Lettered and Quarantined). Failures get a longer leash than successes by default, fourteen days versus one, so that someone can still inspect a dead-letter or a quarantine well after it happened.

RetentionPolicy.Default is the instance carrying those two defaults. Non-terminal jobs are never eligible for purge; only a job that has reached one of the four terminal states is ever swept.

Configuring retention on a worker group#

Retention is configured per worker group, on the Retention property of WorkerGroupOptions (in BackWave.Hosting). It is on by default, set to RetentionPolicy.Default. To use your own windows, assign a RetentionPolicy with object-initializer syntax:

bw.AddWorkerGroup(new WorkerGroupOptions
{
    Name = "emails",
    Policy = new DispatchPolicy.Strict("emails"),
    Retention = new RetentionPolicy
    {
        KeepSucceeded = TimeSpan.FromHours(6),
        KeepDeadLettered = TimeSpan.FromDays(30),
    },
});

To turn the purge sweep off entirely on a group, set Retention to null:

bw.AddWorkerGroup(new WorkerGroupOptions
{
    Name = "audit",
    Policy = new DispatchPolicy.Strict("audit"),
    Retention = null, // no sweeping: terminal jobs and logs accumulate forever
});

Disabling retention means unbounded growth on that store: terminal jobs and their transition logs accumulate with nothing to remove them. Only do this when an external process prunes the tables, or for short-lived and test setups.

Retention lives on the worker group, not on a global switch, so different groups can carry different windows. Note that the sweep is not scoped to one group's queues: any group with retention enabled sweeps every eligible terminal job in the store by its own windows. If you run several groups with different windows, the shortest effective window wins for any given job. Setting retention consistently across groups avoids surprises.

The purge sweep#

The deletes run as part of each worker group's background maintenance, not on the claim path. The cadence is MaintenanceInterval on WorkerGroupOptions, which defaults to 5 seconds; the same tick that expires lapsed leases and mints recurring schedules also purges retained history. (For contrast, PollInterval, the claim poll, defaults to 1 second; keep maintenance slower so claim polls stay a cheap fast path.)

new WorkerGroupOptions
{
    // ...
    MaintenanceInterval = TimeSpan.FromSeconds(10), // how often the purge sweep runs
}

On each tick, retention runs two passes when it is enabled: one over the Succeeded-and-Cancelled class using KeepSucceeded, and one over the Dead-Lettered-and-Quarantined class using KeepDeadLettered. A job is eligible only when it is in the matching terminal class and its terminal instant is at or before now minus the keep window. Eligible jobs are deleted oldest-terminal-first.

Each pass is bounded: it deletes at most 500 jobs, then, if it filled that batch, schedules another pass, repeating until a pass deletes zero. Bounded batches repeated to drain mean a large backlog is cleared steadily over several ticks rather than in one long lock. A missed or delayed maintenance tick only postpones cleanup by one interval; it never affects correctness. A slower MaintenanceInterval simply runs the sweep less often.

What gets deleted, and what is kept#

When a job is purged, it is gone, not archived. The job row is deleted, and its entire transition log is deleted with it: the log lives exactly as long as the job. The job's output blob, which lives on the job row, goes with it too.

After a job is purged, reading it returns nothing, an unknown job rather than an empty-but-present record, and its timeline reads as an empty list. So a purged job disappears from the dashboard and from the Monitor entirely. Size your keep windows to your inspection needs: the 14-day default on the failure class exists precisely so dead-letters and quarantines stay inspectable long enough for a post-mortem. Reading that history while it is still retained is covered on The Monitor API.

Workflow jobs are kept as a unit#

A job that belongs to a Workflow (a Pro grouping over dependent jobs) is not purged on its own terminal instant. The whole Workflow is retained as a unit: a member becomes eligible only once every member has reached a terminal state, and then the keep window starts from that drain point, the latest member's terminal instant, rather than from each member's own terminal time. This keeps the Workflow graph coherent for its whole life. When the last member is purged, the now-orphaned Workflow identity is dropped with it. This behavior is fixed; there is no separate knob for it.

Recording: the Job History Policy#

Retention can only delete what was recorded in the first place, and what gets recorded is governed by the Job History Policy, a JobHistoryPolicy (in BackWave.Storage). It is a ladder; each rung adds to the one below.

The policy gates writes, never schema. The transition table always exists, so changing the policy is a configuration change and never a migration. It is an input to a run and does not affect determinism.

This is where recording and retention meet. With history Off, no transition rows are ever written, so there is nothing in the log for retention to purge; retention still deletes the terminal job rows on the same windows, there is simply no timeline attached to them.

Configuring the policy in two places#

The policy is set in two places, and they must agree. The store options are the source of truth that the store actually records by; each adapter exposes a HistoryPolicy property, all defaulting to TransitionsAndFailureDetail:

// Write side: the source of truth the store records by.
new PostgresStoreOptions
{
    ConnectionString = "...",
    HistoryPolicy = JobHistoryPolicy.Transitions, // record transitions, drop failure detail
}

The registration side carries the same value through UseHistoryPolicy on the BackWave builder:

bw.UseHistoryPolicy(JobHistoryPolicy.Transitions);

UseHistoryPolicy does not change what the store records, the store options do that. The Monitor reads the effective policy straight from the store, so the two cannot drift on the read side. UseHistoryPolicy now only feeds a startup guard: registering a Transition Observer while history is Off fails fast, because with no transitions recorded there is nothing for the observer to deliver. Pass it the same value you set on the store; most applications never need to call it.

Suppressing failure-detail capture without a redeploy#

There is an environment kill-switch for the top rung. Setting BACKWAVE_DISABLE_FAILURE_DETAIL to a truthy value downgrades an effective TransitionsAndFailureDetail to Transitions: transitions still record, but failure-detail capture is suppressed. This guards against stack traces carrying secrets or PII landing in the host database.

BACKWAVE_DISABLE_FAILURE_DETAIL=true

Truthy values are 1, true, yes, and on (case-insensitive, trimmed). The switch only downgrades the top rung; it never turns history fully off. It gates recording, not viewing, viewing captured detail in the dashboard is governed separately by the ViewSensitiveData permission.

The per-job transition cap#

The third lever bounds history within a single job's life, independent of retention. It lives on StoreBounds (in BackWave.Storage), set through the store options' Bounds property.

So a job that retries many times keeps only its newest 64 transitions, with the oldest aging out as it churns. The whole job and its surviving transitions are then deleted when retention's keep window elapses after it goes terminal.

Operational implications#

How the Monitor surfaces the effective policy, including the recording-disabled signal, is covered on The Monitor API.

Where to go next#

• The Monitor API: reading a job's transition timeline and failure detail while it is still retained.
• The Monitor API: how the Monitor reports the effective history policy and the recording-disabled state.
• The Dashboard: requeue, cancel, and the other state-machine transitions that move jobs toward (or away from) terminal.
• Workflows: the Pro grouping whose members are retained as a unit until the whole graph drains.
• Scheduling: the background maintenance tick that the purge sweep shares with recurring-schedule minting.