The Determinism Boundary

The Storage Contract as the seam between the simulable Core and the imperative Shell, covering what lives above and below it, and why configuration is a fixed input rather than a runtime variable.

BackWave is built as a functional Core and an imperative Shell, and the line between them is the Storage Contract, the dialect-free semantic specification every storage implementation must honor. Everything above that seam is a deterministic function of state, event, and time, so it replays exactly under the Simulator on Virtual Time; everything below it is a real Storage Adapter over a database you already run, durable and concurrent and verified by the Conformance Suite rather than simulated. This page names where that boundary sits, catalogs what lives on each side, and explains why configuration is pinned as a run parameter instead of read live. That property is what lets "decades of simulated time" mean something precise about the Core's behavior, not an adapter's SQL.

Where the boundary sits#

The Determinism Boundary is not a separate artifact bolted onto the design; it is the Storage Contract, viewed as the line that separates the simulable from the non-simulable. The contract's own normative text opens by naming it that way: the In-Memory Store and every Storage Adapter are implementations of one document, and that document is the determinism boundary.

Placing the seam there is a deliberate consequence of what BackWave is. Unlike a system that owns its production storage, BackWave runs against a database you already operate. Building a fully simulable production engine was considered and rejected: it contradicts the "no new infrastructure" principle, it breaks multi-node without adding a consensus system, and it is a larger project than the rest of the product combined. So the boundary is drawn at the contract, and production storage stays yours.

In code, the seam is the public IJobStore interface, the single seam between BackWave's decision logic and a backing store. One rule is what turns it from an ordinary interface into a determinism boundary: an implementation must never read its own clock for a semantic decision. Time is always passed in as an explicit now parameter, so a store's behavior is fully determined by its inputs. The contract is dialect-free by design, naming no SQL, isolation level, locking primitive, or vendor. The moment an adapter leaks database-specific semantics through this seam, the Simulator is testing a fiction.

Above the boundary is the functional core: the Core's pure decision functions plus the Node Driver that carries a single node's whole logic. Below it is the imperative shell: a real Storage Adapter over Postgres or SQL Server, plus SQLite as the Embedded Adapter.

Above the boundary: the deterministic, simulable half#

The Node Driver is a sans-I/O state machine: it never awaits, times, or threads. It consumes NodeEvents as inputs and returns Commands as outputs, and every NodeEvent carries an explicit Now instant, so the Driver never finds the time out on its own. Its inputs include poll ticks, claim and execution completions, heartbeat ticks, reported outcomes, mint completions, and loaded schedules; its outputs are Commands like claim a batch, report an outcome batch, expire leases, mint due schedules, purge terminal jobs, heartbeat, execute a job, and request a poll. The Driver decides; only the Shell acts.

The pure functions the Driver invokes are the Core proper: the retry policy computing next-attempt-or-dead-letter, the mint planner and cron cache planning schedule mints, the dispatch policy (a smooth weighted round-robin) choosing a queue, and the retention policy computing purge windows. Each is a deterministic function of state, event, and time.

Scheduling judgment lives in the Driver, not in either caller. When an applied outcome releases a Dependency that may now be due, or a productive mint may have made new work claimable, the Driver emits a request-poll Command, so the production pump and the deterministic test pump stay identical by construction. That identity is the whole point of the sans-I/O split, treated in depth in The Sans-IO Node Driver.

Backpressure is enforced here, above the boundary. A claim pass caps its batch at free pool capacity, subtracting both in-flight executions and buffered-but-unreported outcomes, so the pool never overshoots its configured size. Coalesced reporting is a Driver decision too: terminal outcomes buffer and flush as one report batch on a trigger (the buffer fills, a drain-tail goes idle, or a poll or heartbeat tick arrives). A crash discards that buffer, the Leases lapse, and the jobs are reclaimed, which is At-Least-Once by design.

Below the boundary: the imperative shell#

The imperative shell is the real Storage Adapter: a production implementation of the Storage Contract against a database you already operate. It must be durable, and it is verified by the Conformance Suite rather than the Simulator. Every operation below the boundary is atomic under a crash at any instant, gives read-your-writes from any node (the database is the single authority), and is durable, so an acknowledged operation survives a process or machine restart.

The store is where concurrency actually resolves. Claiming is the single contended operation, and how contention is resolved is delegated entirely to the implementation; the contract specifies only the outcome: each due job goes to at most one claimer. A Postgres adapter and a SQL Server adapter can reach that outcome with completely different locking, and both are correct.

The Conformance Suite is the executable form of the contract: every MUST maps to at least one test, and it runs against the In-Memory Store (the reference implementation) before any production adapter. It is the adapters' substitute for simulation coverage. This is exactly the division of labor the boundary buys: a wrong SQL query is below the seam and out of the Simulator's reach, which is why proving it correct is the Conformance Suite's job and not the Simulator's.

The two pumps that drive the Driver are also part of the Shell, sitting just above the store call. The production pump runs a real clock, real threads, and a real adapter; the deterministic pump used in tests runs on explicit caller-supplied instants and the In-Memory Store. Both feed the identical Node Driver and execute its identical Commands through IJobStore.

The contract surface#

Every entry point below the boundary is a public method on IJobStore. The catalog below is the shape of the seam; each is a semantic operation, not a SQL statement.

The In-Memory Store: a real implementation on the simulable side#

The In-Memory Store is a first-class, publicly shipped implementation of the Storage Contract, the public InMemoryJobStore in the BackWave.Storage.InMemory namespace. It is a real implementation of the contract, not a stand-in: it is deterministic, runs on Virtual Time, and is the reference implementation the Conformance Suite runs against first. It is for tests and local development only and is never a supported production target, because it is neither durable nor multi-node.

Because it runs on Virtual Time, the durability requirement the contract places on production adapters does not apply, and for the same reason it is never a Benchmark Harness target, since a store on Virtual Time has no wall-clock throughput to measure. This is the store the Simulator drives: N virtual Node Drivers plus the In-Memory Store, run through compressed Virtual Time with seeded fault injection (reorderings, crashes, clock skew, lost hints). A single 64-bit seed fully determines a run, and any failure replays exactly from that seed. The mechanics of that setup are covered in How BackWave Is Tested.

Why configuration is a fixed input, not a runtime variable#

The Core stays a deterministic function of (state, event, time) only if nothing it depends on is sampled live during a run. Two rules enforce that: time is always supplied, never read, and configuration is captured up front and held constant.

Time first. Every operation that depends on time takes now from the caller, and an implementation must never consult its own clock for a semantic decision. That single rule is what makes the In-Memory Store runnable on Virtual Time at all. Advance the caller's clock by a simulated decade and the store's behavior follows, because it has no clock of its own to disagree.

Configuration second. A node's settings (its dispatch policy, pool size, lease duration, retry policy, maintenance interval, retention policy, and named bounds) are static per-run data. They parameterize the deterministic functions; they are not re-read mid-decision. A store even resolves its effective policy once, at construction, rather than sampling it repeatedly, which reinforces that a config value is frozen for the whole run.

What must actually cross the seam is reduced to pure data, so no executable code ever crosses the boundary. A retry policy is Core-side configuration, but the delegate that computes a backoff never travels to the store; the Core evaluates it into a RetryDisposition (a ceiling plus a precomputed delay per attempt) and ships only that data.

// Configuration captured up front, a fixed input to the run:
var policy = new RetryPolicy
{
    MaxAttempts = 5,
    Backoff = attempt => TimeSpan.FromSeconds(attempt), // Core-side delegate
};
 
// The Core evaluates the delays and ships only data across the seam;
// the Backoff delegate itself is never persisted.
RetryDisposition disposition = policy.ToDisposition();

Below the boundary, the store resolves the next attempt from that data alone, and it must treat the disposition as pure data, never as executable code. The same shape holds for a reported failure: the retry-versus-dead-letter choice is made above the store and delivered as data, where a present NextDueTime means retry and an absent one means the ceiling is exhausted. The Core decides; the store enforces only legality.

// A present NextDueTime means "retry"; absent means dead-letter. The choice is made
// above the store and delivered as data:
var outcome = new JobOutcome.Failure(NextDueTime: null, Error: "boom");

The following table shows every crossing of the seam as pure data.

Because config is pinned before the run and time is an input, configuration is never a source of non-determinism: same seed plus same config yields the same result. The Job History Policy makes this explicit: it is an input to a run, so the same seed and the same policy produce the same result. In practice the Simulator runs with the full history policy so its oracle keeps asserting on transition sequences, while the Conformance Suite verifies both the on and off states. Configuration and its effect on scheduling is covered in Execution Model.

What lives above versus below: the observability catalog#

Purely observational data never crosses into the Core, and the Simulator records none of it. Tags, failure detail, job output, and Workflows all sit outside the deterministic model. The Core never reads a Tag, so Tags never cross the boundary at all. Failure detail is captured by the Shell at the execution edge, keyed by (JobId, Attempt), and held there in the Shell (never on a NodeEvent or Command), so it never reaches the deterministic Core; both pumps use the identical Shell-side stash. Encryption at Rest lives entirely outside the deterministic model too: the Core and Simulator only ever move opaque bytes, so it adds no oracle surface and leaves the determinism battery unchanged.

Dependencies are the deliberate exception. They are the orchestration mechanism and are Core state, simulated and fenced like any other Core state. Workflows, built from those dependency edges, are not: the Core never reads a Workflow, and the Simulator records none. The distinction is that a Dependency changes what becomes due, so it must be inside the deterministic model, while a Workflow is a read-time grouping of jobs and belongs outside it. See React to Job Outcomes for how observers consume that outside layer.

Finally, the Benchmark Harness is the one body of testing deliberately outside the boundary (real clock, real I/O, real threads, non-deterministic by design), so a noisy run is never a bug. It measures performance and never correctness, and that line is what separates it from the Simulator, which proves Core correctness, and the Conformance Suite, which proves correctness against a real database. The three form a partition: nothing proves everything, and each proves exactly what its side of the boundary allows. Their numbers are treated in Performance & Benchmarks.

Where to go next#

• The Sans-IO Node Driver: the above-the-boundary decision unit and its two identical callers.
• How BackWave Is Tested: how the simulation drives virtual drivers and the In-Memory Store through Virtual Time.
• Storage overview: the Storage Contract from the adapter author's side of the seam.
• Execution Guarantee: the At-Least-Once and Effect-Once guarantees the boundary makes provable.