Choosing an Adapter

A decision guide across the networked, embedded, and in-memory stores, with a capability matrix for durability, host scope, and transactional enqueue.

BackWave ships four implementations of one Storage Contract, and the right one for you falls out of two questions: how many machines run your Workers, and which database you already operate. Every implementation honors the entire Contract, so they behave identically where correctness is concerned. They differ in deployment shape, durability, the mechanism that wakes a Worker when new work lands, and how much you have to provision and run.

There is one rule to settle before anything else: the In-Memory Store is for tests and local development only. It is never a production target. Everything else on this page is a tradeoff; that one is a line.

The capability matrix#

How you register a store#

There is no AddPostgres()-style shortcut. You construct a store object and hand it to the builder's UseStore. That keeps every connection-string and tuning decision in one explicit place, and it is why the adapter choice is a code decision rather than a configuration toggle.

services.AddBackWave(b => b
    .UseStore(new PostgresJobStore(new PostgresStoreOptions
    {
        ConnectionString = "Host=localhost;Database=app;Username=app;Password=secret",
        AutoMigrate = true, // dev convenience
    }))
    .UseJobs(BackWaveJobs.Module));

If you call AddBackWave and never call UseStore, startup throws an InvalidOperationException telling you a store is required. There is no implicit default; you choose, every time.

Durability: the headline difference#

Postgres, SQL Server, and SQLite are all durable, Conformance-verified Storage Adapters. A Job you enqueue survives a process crash, a host restart, and a redeploy, and a Worker picks it back up. This is what makes BackWave's at-least-once execution guarantee mean anything: the guarantee rests on the store remembering the Job.

The In-Memory Store keeps all state in process memory and persists nothing. When the process exits, every Scheduled Job, every in-flight Lease, and every bit of history is gone. That is exactly what you want for a fast, deterministic test, and exactly what you must never ship. Running it in production is unsupported, full stop, because it is not durable. If your "simple deployment" can tolerate losing in-flight work on a restart, you still want SQLite, not the In-Memory Store: SQLite gives you durability for the cost of a file path.

Host scope and scale-out#

This is the axis that splits the durable three.

Postgres and SQL Server are Networked Adapters. Many Worker processes across many machines share one database safely. Claims, cluster-wide concurrency limits, and lease expiry all coordinate through the shared database, so you scale out horizontally by adding hosts that point at the same connection string.

SQLite is the Embedded Adapter, bounded to a single host. Any number of threads, pumps, and processes on that host share the file safely: concurrent claims, cluster-wide concurrency limits, and lease expiry all stay live across those processes. What you cannot do is point two separate machines at the same file and expect coordinated job processing. SQLite's file locking is unsafe over network filesystems, so putting the database on an NFS or SMB share that several machines mount is the one forbidden SQLite topology. One host, as many processes as you like; never two hosts.

The In-Memory Store is single process. Its state lives in in-process dictionaries, so it cannot coordinate across processes at all, even two on the same machine.

Wake-up and latency#

When a Job becomes due, a Worker has to notice. Polling is the source of truth for every adapter: each Worker group polls its queues on its configured interval and claims what is ready. A Wake-Up Hint is an optional, latency-only nudge on top of that. Hints are never correctness-bearing; the system behaves identically, minus some latency, if every hint is dropped. An adapter that has no notification primitive simply does not emit hints, and latency degrades gracefully to the poll interval.

This is where the two networked adapters diverge, and it is the single functional asymmetry between them.

• Postgres emits cross-host hints. It uses the database server's own notification channel, so a Job enqueued on one host can wake a Worker pump on another almost immediately. The hint is transactional: under transactional enqueue it fires on your commit, or never, so a pump is never woken to a row it cannot yet see. This is the lowest-latency option BackWave offers.
• SQL Server has no wake-up hints. It is polling only. There is no push acceleration, so the time to pick up newly enqueued work is bounded by the Worker group's poll interval. If you run SQL Server and care about sub-poll-interval pickup latency, tighten the poll interval; there is no hint to lean on.
• SQLite emits in-process hints, on by default. They nudge Worker pumps in the same process the enqueue happened in. A second process on the same host gets no hint and waits for its next poll. The hints are best-effort and never fire on a transactional enqueue you commit yourself.
• The In-Memory Store runs on Virtual Time. It is deterministic and clock-free, so there is no wall-clock latency to accelerate; the test advances the clock and the work happens.

Transactional enqueue#

Every shipped store reports the single capability flag, SupportsTransactionalEnqueue, as true. That flag governs whether EnqueueAsync may enlist in an ADO.NET transaction you already own, so the business write and the Job commit or roll back as a unit. (If you pass a transaction to a store whose flag is false — only possible with a custom store, since all four shipped stores return true — the client throws a NotSupportedException before touching the store. The transaction parameter is a DbTransaction?.)

The flag being true is necessary but not sufficient. The deployment has to make it reachable.

• Postgres and SQL Server support it fully. The Job and your business writes commit atomically, with no outbox table and no relay process, as long as BackWave's tables live in the same database your business writes target. Each adapter enlists in its provider's native transaction type and throws an ArgumentException if you hand it the wrong one.
• SQLite is the nuanced case, and the connection string decides it. Point ConnectionString at your application's own database file and you get the co-resident deployment: Jobs and business writes share one file and one transaction, the tightest transactional enqueue there is. Point it at a separate file and you get the dedicated deployment: still fully supported and durable, but transactional enqueue is unavailable in that shape, because no single transaction spans two database engines, even though the flag still reads true.

For the co-resident SQLite case there is a guard worth knowing about. On a transactional enqueue, BackWave verifies that the connection you pass is attached to the same on-disk file it is configured to use. A mismatch throws SqliteSameFileMismatchException (a public exception carrying ConfiguredPath and ConnectionPath) rather than committing the Job invisibly into the wrong file, which would be a silent split-brain. In-memory and temporary databases are skipped by the check.

The In-Memory Store offers a transactional-enqueue scope too, but it is a test affordance, not a real database transaction.

For the working pattern, see Transactional Enqueue with EF Core.

Operational overhead#

The networked adapters cost you a database to run and operate. SQLite costs you a file: it runs in your process, manages its own write serialization in WAL mode, and forces the connection pragmas it needs without you configuring them. The In-Memory Store costs nothing.

All three durable adapters share the same schema discipline. BackWave verifies its schema on first use, and a missing or mismatched schema stops the Worker before any job state can be corrupted, rather than running against a schema it does not understand. You enable AutoMigrate for development convenience, or run the public migrator from your deploy pipeline for production. See Schema migrations.

Options you can set#

Each adapter takes a public sealed record options object with a required ConnectionString and a shared set of knobs:

SQLite adds two more:

SQLite allows one writer at a time. Under heavy multi-process contention a too-short BusyTimeout can cause a worker to lose its lease and a Job to run twice, which at-least-once tolerates but you would rather avoid. The generous 5-second default keeps that rare; leave it alone unless you have measured a reason not to.

The StoreBounds defaults are the same on every adapter, so they do not factor into the choice. See Limits and defaults for the full set, and Configuration for HistoryPolicy.

Putting it together#

Choose Postgres as the default networked store. You get the lowest latency through cross-host transactional hints and full transactional enqueue. Reach for it whenever you run Postgres.

services.AddBackWave(b => b
    .UseStore(new PostgresJobStore(new PostgresStoreOptions
    {
        ConnectionString = connectionString,
        AutoMigrate = true,
    }))
    .UseJobs(BackWaveJobs.Module));

Choose SQL Server when SQL Server is the database you already run. You get full transactional enqueue with the same atomic semantics as Postgres. The one tradeoff: no wake-up hints, so pickup latency is bounded by the poll interval.

services.AddBackWave(backwave =>
{
    backwave.UseStore(new SqlServerJobStore(new SqlServerStoreOptions
    {
        ConnectionString = connectionString,
    }));
});

Choose SQLite (co-resident) for single-host applications, edge deployments, desktop apps, and small services where you want zero operational overhead and the tightest transactional enqueue there is: Jobs and business writes in one file, one transaction. Do not share the file across hosts.

var store = new SqliteJobStore(new SqliteStoreOptions
{
    ConnectionString = "Data Source=app.db",
    AutoMigrate = true,
});
builder.Services.AddBackWave(b => b.UseStore(store));

Choose SQLite (dedicated) — BackWave on its own file, single host — when you want durability and zero ops but your business data lives elsewhere. It is fully supported; it simply forgoes transactional enqueue, because no transaction spans two engines.

Choose the In-Memory Store for tests and local development, and nowhere else. It is deterministic, runs on Virtual Time, needs no database, and pairs with the testing harness. It is not durable, it is single process, and it is never a production target.

var store = new InMemoryJobStore();

Where to go next#

• Storage overview — the Storage Contract and the adapter family.
• Postgres, SQL Server, SQLite — per-adapter deep dives.
• Wake-up hints — the latency mechanism behind the Postgres-vs-SQL-Server difference.
• Schema migrations — AutoMigrate, the migrators, and the fail-stop on a bad schema.
• Transactional Enqueue with EF Core — the capability in practice.
• Execution guarantee — at-least-once, and why durability matters.
• Storage contract — the contract all four implement.
• Limits and defaults — the shared StoreBounds values.
• Testing overview and Virtual Time — the In-Memory Store's intended home.