PostgreSQL

The flagship networked adapter: registration, options, transactional enqueue, and LISTEN/NOTIFY wake-up hints for low latency.

Postgres is the adapter most BackWave clusters run in production. It is a networked store: any number of Worker processes across any number of hosts share one Postgres database, and the database itself arbitrates who gets which Job. Claim contention is handed entirely to Postgres through FOR UPDATE SKIP LOCKED, so there are no advisory locks, no lock table, and no coordination protocol of BackWave's own to tune or to break. The database clock is never consulted for semantics either; every time-dependent decision takes the current instant as a parameter, which is what keeps behavior reproducible. If you want durable jobs, horizontal scale-out, and transactional enqueue against a database you already operate, this is the adapter.

The package#

The adapter ships as BackWave.Postgres. It depends on Npgsql, the standard .NET Postgres driver, and it opens and owns its own connection pool from the connection string you give it. The store is IAsyncDisposable; the host that registered it owns disposal.

Registration#

There is no AddPostgres or UsePostgres extension method, and looking for one is the first thing that trips people up. BackWave has a single registration entry point, AddBackWave, and you select a store inside its builder with UseStore. The Postgres adapter is just an IJobStore you hand to UseStore, constructed with a PostgresStoreOptions.

builder.Services.AddBackWave(bw => bw
    .UseStore(new PostgresJobStore(new PostgresStoreOptions
    {
        ConnectionString = "Host=localhost;Database=app;Username=app;Password=secret",
        AutoMigrate = true, // create/upgrade the schema on first use (dev convenience)
    }))
    .UseJobs(BackWaveJobs.Module)
    .AddWorkerGroup(new WorkerGroupOptions
    {
        Name = "default",
        Policy = new DispatchPolicy.Strict("emails", "reports"),
    }));

PostgresJobStore has exactly one constructor, and it takes a PostgresStoreOptions. There is no overload that accepts a bare connection string. If you have seen new PostgresJobStore(connectionString) somewhere, it does not compile against this build; always wrap the connection string in the options object.

When the connection string comes from configuration or DI rather than a literal, use the factory overload of UseStore. It hands you the IServiceProvider, and you still build the options object from it.

bw.UseStore(sp => new PostgresJobStore(new PostgresStoreOptions
{
    ConnectionString = sp.GetRequiredService<IConfiguration>()
        .GetConnectionString("App")!,
}));

AddBackWave throws InvalidOperationException if you never call UseStore, if no job registry was configured, or if two worker groups share a name. These are startup-time misconfiguration errors, surfaced loudly rather than deferred.

The connection string#

The value is a standard Npgsql connection string and is passed straight through to the driver. The store opens its own pool from it, and the same string is reused for the dedicated LISTEN/NOTIFY connection described below. The role you connect as needs rights to read and write BackWave's tables, and, if you enable AutoMigrate, rights to create the schema.

Options#

PostgresStoreOptions is a record with five public properties.

ConnectionString is the only required property; most applications set it and leave the rest at their defaults.

Transactional enqueue#

This is the headline Postgres capability. The adapter reports SupportsTransactionalEnqueue as true, which means it can write a Job row on a transaction you already own rather than on a transaction of its own. The business write and the Job commit or roll back as a single atomic unit, which is what lets BackWave replace an outbox table and its relay process.

BackWave never opens the business transaction. You begin it, you commit it, you roll it back; the adapter only writes onto the open transaction you hand it. The transaction type it enlists in is the ADO.NET NpgsqlTransaction. If you pass any other kind of transaction, the store throws an ArgumentException whose message states that the Postgres adapter enlists in NpgsqlTransaction instances only. When you supply a transaction, BackWave runs its inserts on your connection and does not commit; your commit makes the Job durable and visible, and your rollback means the Job never existed. When you supply no transaction, the store opens its own connection and transaction, does the work, and commits.

Everything an enqueue writes rides that one transaction together: the Job row, its dependency edges to parent Jobs, any enqueue-time tags, and the first transition-log row. Under transactional enqueue they all commit or all roll back as a unit.

Most applications do not call the store's enqueue method directly. They go through the higher-level client, and the cleanest way to enlist in your own transaction is the EF Core integration. See Transactional enqueue with EF Core for the end-to-end pattern and EF Core storage for wiring a DbContext and the store against one database. The capability itself is described in Execution guarantee, and the contract clause it implements is in the storage contract.

Wake-up hints with LISTEN/NOTIFY#

Polling is always the source of truth for whether work is due. Wake-up hints are a latency optimization layered on top: a hint tells a Worker pump "poll this queue sooner than your next scheduled tick." Hints are never correctness-bearing. They may be dropped, duplicated, delayed, or reordered, and the system behaves identically, minus some latency, if every hint is lost. The Postgres adapter implements them over Postgres's own LISTEN / NOTIFY.

When an enqueue or a recurring-schedule tick creates work that is due now, the adapter publishes a notification on a single fixed channel, and the payload of that notification is the Queue name. The notification is issued inside the same transaction as the write that created the work. Because Postgres makes NOTIFY transactional, the hint fires on commit, or never. That has a clean consequence under transactional enqueue: the hint rides your commit, so a Job you roll back emits no spurious hint, and a Job that commits wakes the right queue. A hint is published on enqueue only when the Job is actually due at that moment; a future-dated Job emits no immediate hint, because there is nothing to poll for yet.

On the subscriber side, the adapter opens a dedicated connection, separate from the main pool, runs LISTEN on the channel, and forwards each notification's queue-name payload to the pump. The subscription is self-healing: if the channel is lost, it reconnects on its own, retrying every few seconds until the subscription is disposed. While it is down, nothing breaks; latency simply degrades to the configured poll interval, because polling was the source of truth all along. Adding more Worker pumps adds more of these connections, which is worth keeping in mind for connection-pool sizing.

For the cross-adapter picture of latency and graceful degradation, see Wake-up hints and latency.

Provisioning the schema#

BackWave's tables live in a dedicated backwave schema in your database. That name is configurable — set SchemaName to place the objects under a schema that fits your conventions, and pass the same value to the migrator's name-taking overload; the details and the rules are in Naming the schema. You provision it one of two ways.

Auto-migrate#

Set AutoMigrate = true and the store applies its versioned schema scripts on first use, before it stores any Job. This is convenient for development and for single-deployment applications where the app process is allowed to alter the schema. It is off by default.

Run the migrator yourself#

For a deliberate deploy step, use the public static PostgresMigrator. It works against an open NpgsqlDataSource and does not dispose it.

await using var dataSource = NpgsqlDataSource.Create(connectionString);
await PostgresMigrator.MigrateAsync(dataSource);

MigrateAsync applies every script in version order. It is idempotent: it is safe to run on every deploy and is a no-op against a database that is already current. The companion VerifySchemaVersionAsync checks that the deployed schema matches the version this build requires and throws InvalidOperationException if the schema is missing or the version differs. The build's required version is exposed as the constant PostgresMigrator.ExpectedSchemaVersion.

The version-skew guard#

Whichever path you choose, the store verifies the schema version once, lazily, on first use, and refuses to start on a mismatch. Deploying application code that expects a newer schema than the database has will fail fast rather than silently corrupt Job state. This is fail-stop by design, and it is worth knowing about because it is surprising the first time you meet it: a first run against an empty database with AutoMigrate off fails with a "schema not found" error directing you to apply the schema or opt in to auto-migrate. For a development machine, set AutoMigrate = true or run MigrateAsync first.

The depth on all of this, including auto-migrate versus the explicit migrator and the startup check, lives in Schema and migrations.

Bounds#

StoreBounds, the Bounds option, is the set of size and batch limits the adapter enforces. Most applications never touch it; StoreBounds.Default carries the values below. Limits split into three behaviors: a violation is either rejected, clamped down to the limit, or truncated.

Note the difference between the two 64 KiB limits. An over-size payload is rejected at enqueue, but as a result code, not an exception (see below). An over-size Job output is rejected loudly: reporting a successful outcome whose output exceeds MaxOutputBytes throws JobOutputTooLargeException, and the whole outcome write is aborted before it touches the store. When a result is large, store a reference to it, such as an id or a blob key, rather than the data itself. The full table is also in Limits and defaults.

History policy#

JobHistoryPolicy controls how much per-job timeline the store records. It is a ladder, each rung adding to the one below:

• Off records no transition rows.
• Transitions records the transition timeline but no failure detail.
• TransitionsAndFailureDetail is the default: the full timeline plus clamped failure detail on failing transitions.

This setting gates writes, never the schema, so changing it is a configuration change and not a migration. The authoritative place to set it is PostgresStoreOptions.HistoryPolicy. There is also a builder method, UseHistoryPolicy, but its only remaining job is a startup observer-deliverability guard; the Monitor reads the effective policy directly from the store, so the two cannot drift, and the store's option is what determines what gets recorded.

One operational override worth knowing about: the environment variable BACKWAVE_DISABLE_FAILURE_DETAIL (set to 1, true, yes, or on, case-insensitive) downgrades an effective TransitionsAndFailureDetail to Transitions, suppressing failure-detail capture. It is resolved once at construction. It exists so an operator can keep stack traces, which may carry secrets or PII, out of the database without touching code.

What can throw, and what does not#

Most enqueue limit violations are not exceptions. Enqueue returns an EnqueueResult value: Ok, Duplicate, UnknownParent, PayloadTooLarge, WireNameTooLong, or TooManyParents. Check the result rather than wrapping the call in a try/catch for those cases.

The exceptions a caller can actually hit are narrower:

• JobOutputTooLargeException when a successful outcome's output exceeds MaxOutputBytes. It carries JobId, ActualBytes, and MaxOutputBytes, and the write is rejected, not truncated.
• ArgumentException when a non-NpgsqlTransaction is passed as the enqueue transaction.
• InvalidOperationException when the schema is missing or its version does not match on startup, and from AddBackWave for the misconfigurations listed above.

Behavior in a cluster#

Because Postgres is a networked adapter, any number of nodes across any number of hosts share one database, which is the scale-out story this adapter exists for. Claiming is set-based: concurrent claimers grab disjoint rows with FOR UPDATE SKIP LOCKED and no extra coordination, and the rows a claimer gets back are ordered per queue by due time and then enqueue order. A queue's concurrency limit and paused state are read under a row lock so claimers serialize on the available slot count, while unlimited and unpaused queues take no lock at all; a paused queue simply yields nothing.

Every outcome write is fenced. A write only lands if the Job is still leased to the same Worker, on the same attempt, with an unexpired lease. A stale outcome from a node that was isolated and came back mutates nothing and reports that its lease was stale, so an effect is applied at most once even though delivery is at-least-once. Expired leases are reaped set-based and either rescheduled for retry or dead-lettered once they hit the attempt ceiling. The practical consequence is the one every at-least-once system shares: your handlers must be idempotent. The reasoning is in Execution guarantee.

For when Postgres is the right choice versus its siblings, the embedded single-host SQLite adapter and the SQL Server adapter, see Choosing an adapter.

Where to go next#

• Schema and migrations for auto-migrate versus the explicit migrator and the version-skew guard.
• Wake-up hints and latency for the LISTEN/NOTIFY latency story across adapters.
• Transactional enqueue with EF Core for enlisting a Job in your own transaction.
• EF Core storage for sharing one database between your context and the store.
• Choosing an adapter for where Postgres sits among the stores.
• Limits and defaults for the full bounds table.
• Storage contract for the guarantees these behaviors implement.