Storage

BackWave's Storage Contract and its adapters for Postgres, SQL Server, SQLite, and the deterministic In-Memory Store.

BackWave does not ship a server, a broker, or a datastore of its own. It stores its queue in a database you already run. Where a Hangfire-style system asks you to stand up its schema in a dedicated store, BackWave's state lives next to yours: the same Postgres instance, the same SQL Server, or the same SQLite file your application already uses. That single decision makes Transactional Enqueue possible, keeps the operational footprint small, and shapes everything on this page.

This is the lay of the land for that storage story: what the Storage Contract is and why it is the determinism boundary, the three shapes a store can take, what "durable" buys you, the handful of capabilities that are configurable rather than universal, and how BackWave proves every adapter behaves identically. Each subpage carries the specifics; this page routes you to them.

The Storage Contract is the determinism boundary#

BackWave is split into two halves. The Core is pure decision logic — scheduling, retries, due calculation, leasing and timeouts, state transitions. It is a set of deterministic functions of (state, event, time) and it performs no I/O. The Shell is the per-node loop that does the I/O: it fetches state, calls the Core, and executes the resulting commands against storage. The Shell owns all concurrency, the real clock, and every database round trip.

The Storage Contract is the seam between those two halves. It is the semantic specification of what any storage implementation must guarantee — how it behaves under concurrent Lease acquisition, what happens on a crash mid-write, which writes are atomic. And it is, precisely, the determinism boundary: everything inside it is simulable, everything beyond it is not. The Simulator can replay the Core against the In-Memory Store and reproduce any schedule of events exactly, because nothing inside the boundary reads a clock or a network. A real database sits outside it.

One rule makes that line hold. Every time-dependent operation takes the current instant as an explicit now argument supplied by the caller, and no implementation may read its own clock to make a semantic decision. The store is told what time it is; it never asks. That is why the same logic that runs against Postgres in production can run against the In-Memory Store on virtual time in a test and produce identical outcomes.

The Contract covers the full surface of what BackWave does with storage: enqueue and the job lifecycle, claiming work, reporting outcomes, leasing and lease expiry, scheduling, operator actions (cancel, requeue, pause and resume a queue, trigger a schedule now — each writing an atomic audit record), queue settings, the monitoring reads, retention, and transition-observer delivery. The method-by-method normative specification lives in the Storage Contract reference; you only implement it directly if you are writing a custom adapter.

What the Contract guarantees, always#

Three guarantees hold on every adapter, regardless of which database backs it:

• At-Least-Once Execution. A handler body may run more than once. A node that gets isolated mid-execution and then recovers can leave the same Attempt to be retried elsewhere. Idempotency is the handler author's responsibility — durability does not buy you exactly-once execution. See execution guarantee.
• Effect-Once. Despite at-least-once execution, the recorded outcome of an Attempt — and every transition that flows from it, such as the terminal state, a dependency latch decrement, or a concurrency-slot release — applies exactly once. The Contract enforces this at its boundary by fencing each outcome to the worker and attempt that produced it. A stale outcome from a recovered-but-superseded node mutates nothing.
• Atomic enqueue. A duplicate job id is rejected, never silently replaced. An enqueue that would exceed a bound is rejected rather than truncated. A workflow insert is all-or-nothing.

The adapter taxonomy#

A store can take one of three shapes. Two are durable, production-grade implementations of the Contract — these are Storage Adapters. The third is the In-Memory Store, which is emphatically not for production. The distinction between the two adapter topologies is about which deployment shapes they support, never which Contract clauses they honor. Every adapter honors all of them; the Embedded adapter is not a "lite" Contract.

Networked Adapters#

A Networked Adapter runs over a database server reachable across hosts. Your cluster of nodes can span many machines, all coordinating through the shared server. This is the default mental shape for a distributed worker fleet. BackWave ships two: Postgres and SQL Server.

The Embedded Adapter#

An Embedded Adapter runs over an in-process, single-host database engine rather than a network server. BackWave ships one: SQLite. It is durable and Conformance-verified like any adapter, and it is a fully supported production target — it simply trades cluster scale-out for zero operational overhead.

The part readers most often get wrong: embedded does not mean single-process. Any number of processes on the same host can share the database file, each acting as a node that coordinates through it. Concurrent claims, cluster-wide concurrency limits, and lease expiry all stay live across those processes. What it cannot do is span hosts — the engine's file locking is unsafe over a network filesystem, so the database is never shared across machines. "File-backed" tempts people to assume "easy multi-server," and it is the opposite.

SQLite has two deployments. In the co-resident deployment, BackWave's tables live in your application's own SQLite file, so a job and your business writes commit in one file, in one transaction — the tightest Transactional Enqueue there is. In the dedicated deployment, BackWave gets its own file while your business data lives elsewhere; this is fully supported and simply forgoes Transactional Enqueue, since no transaction can span two engines. Note that the Transactional-Enqueue capability is present in both deployments; only the co-resident one can exploit it. The SQLite page covers both.

The In-Memory Store#

The In-Memory Store is a first-class, publicly shipped implementation of the Contract — a real implementation that passes the same Conformance Suite, not a mock or a fake. It is deterministic, clock-free, and runs on virtual time, which is exactly what makes it the substrate for the Simulator and for fast tests. It implements the whole Contract, including transactional enqueue, workflows, observers, the transition log, and audit records.

It keeps all state in process memory and persists nothing. That makes it for tests and local development only — never a supported production target. Running it in production is unsupported, full stop: it is not durable, and a restart loses every in-flight job. You rarely construct it by hand; the testing harness builds one for you and exposes it. Because it lives in the core package, it needs no extra package reference.

Choosing an adapter is the decision page when you need to pick.

Durability#

Every Storage Adapter is durable; that is a requirement of being an adapter, not a feature some of them add. Postgres, SQL Server, and SQLite all persist your jobs through a process crash or a restart. The In-Memory Store does not — it persists nothing.

Durability is the reason the production/test split exists, and it pairs directly with the delivery model. Because storage is durable and because BackWave uses Leases that expire rather than locks that block, a crashed node's in-flight work does not vanish and does not stay stuck. The Lease times out, the job becomes reclaimable, and another node picks it up. That is the mechanical basis of at-least-once execution: durable state plus expiring leases means work is never lost and never permanently held.

Capabilities are configurable, not universal#

Beyond the universal guarantees, the Contract defines a small set of capabilities that vary by adapter or by configuration. Knowing which is which keeps you from assuming behavior an adapter doesn't offer. The full per-package matrix is on the packages and compatibility reference.

Transactional Enqueue#

Transactional Enqueue is the ability to enlist an enqueue in a database transaction you own, so the job commits or rolls back atomically with your own writes. BackWave never opens the business transaction; it rides along on the one you began. This is what replaces the outbox pattern — there is no relay process and no separate outbox table, because the job row and your rows commit as a unit.

Each store exposes whether it offers this through a readable capability flag, SupportsTransactionalEnqueue. An adapter that returns false must reject any transaction handed to it loudly, never ignore it silently. In v1 every shipping store reports true, including the In-Memory Store — but reporting the capability is not the same as benefiting from it. You only get atomic business-plus-job commits when BackWave's tables share the transaction scope: the same database via EF Core, or a co-resident SQLite file. A separate BackWave database or a dedicated SQLite file forgoes the benefit even though the flag is true. The Transactional Enqueue guide walks the whole pattern.

Wake-Up Hints#

A Wake-Up Hint is an optional, latency-only nudge from the store — "something was just enqueued, poll now" — that exists solely to cut the delay between an enqueue and a worker noticing it. It is never correctness-bearing. The system behaves identically, minus latency, if every hint is dropped, duplicated, delayed, or reordered; polling is the sole source of truth. There are no delivery guarantees of any kind.

Hints are an optional capability, so not every adapter has them. Postgres and SQLite both implement them. SQL Server does not — on SQL Server, a newly enqueued job is noticed on the next poll, so latency degrades to your configured poll interval rather than being near-instant. That is an honest trade, not a bug. Wake-up hints covers the mechanics.

Job History#

How much history a store records is configurable. The policy is a ladder, each rung adding to the one below: record nothing, record transition rows only, or record transitions plus full failure detail (the default). The store is the single source of truth for the effective policy.

Two things make this safe to change. First, it gates writes, never schema — the transition table always exists, so changing the policy is a configuration change, never a migration. Second, because it is an input to a run rather than a decision the Core makes, it never affects determinism. There is also a kill-switch for environments that must not persist diagnostic detail: setting the environment variable BACKWAVE_DISABLE_FAILURE_DETAIL to a truthy value downgrades an effective "transitions and failure detail" policy to transitions only, so stack traces — which can carry PII or secrets — never land in the host database. If you are ever debugging "why is my failure detail empty," that variable is the first thing to check. See configuration and job states.

Store fault classification#

Adapters may optionally classify provider-specific transient faults — the busy-or-locked conditions a generic transient flag misses — so the worker degrades and retries rather than fail-stopping. Most networked providers already surface transience through the standard flag, so most adapters need nothing here; SQLite implements classification for its residual busy conditions. This is an advanced internal detail; you do not configure it.

Bounds reject loudly#

Every store enforces named size and batch limits with sensible defaults — maximum payload size, wire-name length, claim-batch size, parents per job, monitor page size, and so on. What matters at this level is the failure mode: bounds are enforced with clear errors, never silent truncation, with one deliberate exception. Failure detail (a diagnostic, not data your code reads) is truncated rather than rejected. Everything else that exceeds a bound is rejected.

The clearest illustration is output. A successful outcome whose output blob exceeds the maximum throws JobOutputTooLargeException — it carries the job id, the actual size, and the limit, and its message tells you to store a reference (an id or a blob key) instead of the large value. Output is rejected, never truncated. The full table of bounds and defaults is on limits and defaults.

How it plugs in#

Storage is wired through the BackWave builder at startup. You call AddBackWave once and configure the store on the builder with UseStore; storage has no default, so registration throws if you skip it. Pass the store directly, or use the factory overload that resolves it from the service provider — UseStore(Func<IServiceProvider, IJobStore>) — when the store needs other services.

builder.Services.AddBackWave(bw => bw
    .UseStore(new PostgresJobStore(new PostgresStoreOptions
    {
        ConnectionString = connectionString,
    }))
    .UseJobs(BackWaveJobs.Module)
    .AddWorkerGroup(new WorkerGroupOptions
    {
        Name = "default",
        Policy = new DispatchPolicy.Strict("emails", "reports"),
    }));

Each adapter takes its own options record with a required ConnectionString. The exact constructor, connection-string shape, and adapter-specific options live on the adapter pages: Postgres, SQL Server, SQLite, and EF Core for the co-located transactional setup. Schema creation and upgrades are covered in schema migrations. The complete registration surface is on the configuration reference.

The Conformance Suite proves they match#

BackWave makes a strong claim — that every adapter, including the embedded one, honors identical semantics — and it has to back that claim with something other than trust. The Simulator can't do it: the Simulator stops at the Storage Contract boundary and runs the In-Memory Store, so it catches Core bugs and storage-fault handling, but a wrong Postgres query is invisible to it. That is the job of the Conformance Suite.

The Conformance Suite is the executable form of the Storage Contract: a single test suite, with each test named after the Contract clause it verifies, run against every implementation. The In-Memory Store is the reference implementation and passes it completely; any divergence on a real adapter is therefore a bug in exactly one of the two, and the suite tells you which. Every shipping store — In-Memory, Postgres, SQL Server, SQLite — runs the same suite against a real instance of its database. This is the mechanism behind "the Embedded adapter honors all the same clauses": it is verified, not asserted. See how BackWave is tested and the determinism boundary for how the two testing strategies divide the work.

Sharp edges worth knowing#

• The In-Memory Store is never production. It is not durable; a restart loses in-flight work. Use it for tests, the Simulator, and local development only.
• Embedded means single-host. Many processes on one host, fine. Across hosts or over a network filesystem, unsafe. Do not read "file-backed" as "easy multi-server."
• The Transactional Enqueue benefit is not the flag. The flag is true everywhere, but you only get atomic commits when BackWave's tables share the transaction scope — co-resident SQLite or the same database via EF Core. A dedicated file or a separate BackWave database forgoes it.
• Wake-up hints are latency-only and not universal. SQL Server has none; everywhere else they may be dropped or duplicated. Never treat them as reliable delivery — polling is the source of truth.
• Bounds reject loudly, except failure detail. Failure detail truncates; everything else, output included, is rejected outright.
• History toggles by config, not migration — and can be silenced. The transition table always exists, so changing the policy never requires a migration, and the BACKWAVE_DISABLE_FAILURE_DETAIL kill-switch downgrades failure detail without warning.
• At-least-once means idempotent handlers. Durable storage gives you at-least-once delivery, not exactly-once execution. Make the handler safe to run twice.

Where to go next#

• Choosing an adapter — pick the store that fits your deployment.
• Postgres, SQL Server, SQLite — setup and options per adapter.
• EF Core storage — co-locate BackWave with your business database.
• Schema migrations — creating and upgrading the schema.
• Wake-up hints — the latency-only nudge and which adapters offer it.
• Storage Contract reference — the method-by-method specification.
• Limits and defaults — every bound and its over-limit behavior.
• Execution guarantee — at-least-once and effect-once in depth.
• Testing overview — using the In-Memory Store and the harness.