Storage Contract Reference

The adapter-author lookup for the IJobStore operation surface, its invariants, capability flags, schema versioning, and wake-up hints.

The Storage Contract is the semantic specification every storage implementation must guarantee. Its C# seam is the IJobStore interface. An adapter that persists BackWave state to a real database implements this interface; so does the deterministic in-memory store used in tests and development. Both are full implementations, and both are held to it by an executable conformance suite. This page is the lookup for authoring an adapter: the operations IJobStore exposes, the invariants each one must uphold, the capability flags a host uses to detect optional behavior, the bounds a store enforces, and the schema-versioning and wake-up surfaces that live alongside the interface.

Everything on this page is the public contract. If a guarantee is stated here, an adapter must provide it, and the conformance suite checks that it does.

The clock invariant#

One rule governs every other guarantee: an implementation must never read its own clock for a semantic decision. Every method that makes a time-dependent choice takes the current instant as an explicit now parameter, and the store's behavior is fully determined by its inputs. A claim decides what is due from the Now on the request, a lease expires against the now passed to the sweep, and retention compares against the terminalBefore the caller supplies. Nothing in a store calls DateTimeOffset.UtcNow on its own behalf.

This is what makes a store simulable and deterministic. Given the same sequence of calls with the same now values, every conforming store produces the same result, which is why the in-memory store can drive tests under virtual time and a real adapter can be certified against the same expectations.

Capability detection#

Optional behavior is discovered in exactly two shapes: one property flag on the interface, and two optional side-interfaces detected with an is check. A host inspects these once and adapts its behavior; nothing else about a store is conditional.

Transactional enqueue#

SupportsTransactionalEnqueue reports whether EnqueueAsync and EnqueueWorkflowAsync can enlist in a caller-supplied ADO.NET DbTransaction, so a job commits or rolls back atomically with the caller's own writes. A store that returns false must reject any non-null transaction passed to those methods loudly. It must never silently ignore a transaction it cannot honor, because a caller that believes its enqueue is transactional and finds it is not has lost the guarantee it was relying on.

History policy#

HistoryPolicy is the single source of truth for how much transition history a store records. It carries a default interface implementation returning JobHistoryPolicy.TransitionsAndFailureDetail, so a store that records everything need not override it. Because the effective policy is readable, a monitor can tell a genuinely empty timeline apart from one that is empty only because recording is turned off. The Job history policy section covers the ladder in full.

Transient-fault classification#

A store may implement IStoreFaultClassifier to tell the worker which store faults are worth degrading and retrying rather than treating as fatal.

public interface IStoreFaultClassifier
{
    bool IsTransientFault(Exception exception);
}

Return true only for a transient store fault the worker should degrade-and-retry on the next tick. Return false for everything else, including invariant violations and unknown faults, which leaves the host's default classification untouched. The method must be a pure, side-effect-free inspection, and it must never mistake a permanent fault for a transient one: a false positive turns a fatal error into an endless retry.

Most adapters do not need this. Networked providers already raise transient conditions with DbException.IsTransient set to true, and the host honors that. Implement the classifier only when your provider leaves IsTransient unset for a fault you know to be transient contention, such as a residual busy or locked condition that survives a busy-timeout.

The operation surface#

Every operation returns ValueTask or ValueTask<T> and takes a trailing CancellationToken cancellationToken = default, omitted from the tables below for brevity. The now parameter, where present, is always the caller's clock. The operations divide into three groups: write and lifecycle operations that change committed state, read operations that a monitor uses to observe it, and the observer-delivery operations that feed transition-log observers.

Write and lifecycle operations#

Read and monitor operations#

Read operations take no now; they return committed state only. Every one is a snapshot a monitor or a dependent job reads without changing anything.

Observer-delivery operations#

The five observer-delivery methods are mandatory members of IJobStore, not an optional side-interface. When HistoryPolicy is Off there is nothing to observe, so they no-op and report caught-up rather than erroring. They back the Transition Observer delivery pipeline, which delivers each transition-log row to a subscribed observer at least once.

Each delivery carries its own retry counter, distinct from the job's Attempt. The supporting records are ObserverClaimRequest, ObserverClaimedDelivery, ObserverClaim (with ObserverClaim.None(observerId)), ObserverDeliveryOutcome, ObserverDeliveryReport, ObserverLagRequest, ObserverLag, and ObserverDeadLetterRecord. A delivery is dispositioned as one of ObserverDeliveryDisposition.Delivered, Retry, or DeadLettered.

Result and value types#

Every write operation reports its outcome through a small result enum rather than an exception, so a caller branches on the return value. Bound violations, duplicates, and fenced-out attempts are all ordinary return values.

Any WorkflowEnqueueResult other than Ok inserts nothing. TerminalStateClass names the two retention classes, each purged on its own keep window. OperatorAction persists as its numeric value, so an adapter must append new actions rather than renumber existing ones.

The outcome a worker reports is a closed hierarchy, JobOutcome, whose cases carry the data the store persists:

The request and record types the operations exchange are listed below. Together they are the complete data surface an adapter reads from and writes to.

Over-cap success output is the one condition reported as an exception rather than a result enum:

public sealed class JobOutputTooLargeException : Exception
{
    public Guid JobId { get; }
    public int ActualBytes { get; }
    public int MaxOutputBytes { get; }
}

The job record and its states#

GetJobAsync, ClaimAsync, and ListJobsAsync all return JobRecord, the reader's snapshot of one job. Its required fields are JobId, WireName, Payload, Queue, State, and DueTime. The remaining fields are TraceContext?, Attempt (a claim increments it), LeaseOwner?, LeaseExpiry?, CancelRequested, TerminalAt?, TerminalCause?, ScheduleId?, ParentsRemaining (a countdown to zero), Sequence (store-assigned, strictly increasing with insertion order, used as the within-queue tiebreak and the paging cursor), Mode (default OnSuccess), WorkflowId? (immutable, at most one, never read by the core), Tags (default Empty), and Output? (present only when the job succeeded, fetched on demand rather than in listings).

State is a JobState, one of seven values. The full state machine, its transitions, and the recorded terminal causes live on the Job States and Transitions page.

The JobStates.IsTerminal() extension returns true for the four terminal states. A terminal job never transitions on its own; only an explicit operator action can move it.

Storage bounds#

A store enforces a small set of named size and batch limits, collected in the StoreBounds record. Each limit has a fixed default and a fixed enforcement rule. The governing principle is that with one exception, failure detail, every limit is enforced with a clear error rather than silent truncation. StoreBounds.Default is the singleton of the defaults below.

Output and failure detail are the two ends of the reject-versus-truncate rule. Output is the loud reject: over-cap success output throws rather than storing a partial blob. Failure detail is the sole truncation. To truncate it identically across every store, use the record's ClampFailureDetail(string?), which clamps to MaxFailureDetailBytes UTF-8 bytes, backing off continuation bytes so it never splits a code point and always round-trips as valid text. Null or short input passes through unchanged. The Limits and Defaults page presents these same bounds alongside the retry and lease defaults.

Job history policy#

JobHistoryPolicy is a ladder. Each rung records more than the one below it.

The one invariant an adapter must hold is that the policy gates writes, never schema. The transition table always exists, so changing the policy is a configuration change, never a migration. An Off store still answers GetJobHistoryAsync with an empty list and never errors.

An adapter resolves its effective policy through JobHistoryPolicyResolver.Resolve(configured), which downgrades the top rung to Transitions when the environment kill-switch is set. The switch is the variable BACKWAVE_DISABLE_FAILURE_DETAIL; it is truthy for 1, or for true, yes, or on case-insensitively. It guards against failure-detail text carrying stack traces, secrets, or PII into the host database. Resolving through this resolver is how a store honors the kill-switch identically to every other.

Deterministic minted ids#

Every store must derive a minted job's id the same way, because that is what makes scheduled minting exactly-once across a cluster. JobIds.ForMintedTick(string scheduleId, DateTimeOffset tick) is a pure function of its two arguments: the same schedule id and tick always produce the same Guid. Racing minters on different nodes compute the identical id and collide on the primary key, so only one instance inserts. An adapter must use this function rather than generate a fresh id, or the exactly-once guarantee is lost.

Guid id = JobIds.ForMintedTick(scheduleId, tick);

Wake-Up Hints#

A store may implement IWakeUpHintSource to notify workers that a queue has new work sooner than the next poll would find it. Implement it only if the store has a notification primitive to build on; a store without one simply does not implement the interface, and latency degrades to the configured poll interval.

public interface IWakeUpHintSource
{
    Task<IAsyncDisposable> SubscribeAsync(
        Action<string> onHint,
        CancellationToken cancellationToken = default);
}

onHint receives the queue name and may be called concurrently. Disposing the returned handle tears down the subscription. The subscription is self-maintaining: on channel loss it keeps reconnecting until disposed, and while it is down, latency degrades to the poll interval and nothing else changes.

A hint carries no delivery guarantee whatsoever. Hints may be dropped, duplicated, delayed, or reordered. A hint only ever makes a worker poll a queue sooner, and no correctness decision anywhere may depend on one. Polling is the sole source of truth, and the system must behave identically, apart from latency, if every hint is dropped. A callback that never fires, or fires for a queue with no work, is always acceptable.

Schema versioning and migration#

Schema versioning lives in each database adapter, not in the core storage surface. An adapter ships an ordered set of schema scripts and a single integer that names the schema version its build requires. The version is a plain equality gate: a store verifies the database's version on first use and refuses to run on any mismatch, because version skew must never be allowed to corrupt job state.

The schema scripts are the versioned artifact: each is numbered, applied in order, and records the new version as its final step, so a script filename maps one-to-one to a schema version. Because the scripts are additive and idempotent, an in-place upgrade with no drain is the supported upgrade path, and a mixed-version fleet is supported across a single version of skew. That single-version-skew guarantee is a design contract of the additive scripts, not a runtime check; the only mechanism the adapter enforces at runtime is the exact-version gate that refuses any mismatch.

A store's options are the adapter's public configuration surface. They carry the connection details, the Bounds (defaulting to StoreBounds.Default), the HistoryPolicy (defaulting to TransitionsAndFailureDetail, controlling writes only and never the schema), the auto-migrate flag (defaulting off), and the schema name.

The schema name#

An adapter can host its tables under a configurable schema name, defaulting to backwave. The default is a pure passthrough, so the emitted SQL is byte-identical to the authored scripts at no cost; a custom name substitutes on first sight of each distinct query. A schema name is validated at construction against a strict identifier pattern, ^[A-Za-z_][A-Za-z0-9_]*$, at most 63 bytes, unquoted, so an invalid name fails fast and never reaches the database. The wake-up hint channel is namespaced off the schema name, so two BackWave deployments sharing one database never cross-talk.

Certifying an adapter#

The Storage Contract has an executable form: a conformance suite that exercises every guarantee on this page against a real store. An adapter is certified by subclassing the suite in an xUnit test project and providing a fresh, empty store for each test.

public sealed class MyAdapterConformanceTests : ConformanceSuite
{
    protected override async ValueTask<IJobStore> CreateStoreAsync(JobHistoryPolicy historyPolicy)
    {
        // Return a clean-slate store honoring the given history policy.
    }
}

CreateStoreAsync is the one required override. It must return a fresh, empty store honoring the requested history policy, a clean slate per call. Every test is a public fact, so the runner discovers the whole suite by subclassing. The reference in-memory store passes it in full.

Two optional overrides tune what the suite exercises. An adapter that supports transactional enqueue overrides BeginTransaction(IJobStore store) to produce a transaction the suite can enlist; the default throws. An adapter whose batch outcome report is wrapped in one transaction sets BatchOutcomesAreAtomic to true to enable the whole-batch atomicity test. The suite also offers opt-in failpoint and interleaving hooks for the crash-mid-write and read-then-write anomaly tests; a store that cannot simulate a given fault leaves the hook returning null, and those tests skip.

The named invariants the suite asserts are the guarantees stated throughout this page, among them: each claimed job is leased to at most one caller; no AwaitingParent job survives its parent set going terminal, with the child gate resolving atomically with the parent's terminal transition; and a terminal state or a lease expiry frees a concurrency-limit slot, since slots are the count of live leases by construction. The suite additionally proves due-time ordering with an enqueue-order tiebreak, honoring the caller's ordered queue list without re-sorting, disjoint claims across concurrent pumps of one group, outcome fencing on the wrong worker or attempt or a lapsed lease, expiry counting the claim's attempt, exactly-once disposal under concurrent sweeps, cursor-fenced exactly-once minting, retention on the terminal clock, and that concurrent duplicate inserts converge idempotently rather than surfacing a raw duplicate-key error.

Where to go next#

• Job States and Transitions: the full state machine JobState drives, every transition, and the recorded terminal causes.
• Limits and Defaults: the storage bounds alongside the retry and lease defaults, in one place.
• Observers API: the Transition Observer surface the observer-delivery operations feed.
• Queues: claim order, concurrency limits, and pausing from the consumer's point of view.
• Scheduling: cron schedules, cursors, and the minting the store persists.
• Execution Model: how a claimed job is leased, heartbeated, and reported.

Found a problem on this page? Report an issue