Scaling & Throughput Tuning

The dials you turn to scale a node and the cluster: pool size, pumps, the serial store-I/O ceiling, and in-process pumps versus adding nodes.

Throughput in BackWave comes from two distinct dials, and knowing which one to turn for a given symptom is most of the job. PoolSize controls how many jobs a group runs at once; Pumps controls how many independent claim loops a group runs in one process. They scale different bottlenecks, and they have different costs: idle pool slots are free, while every pump draws database connections whether it is busy or not. Both live on a single public record, WorkerGroupOptions, that you pass to AddWorkerGroup. When a node has no more room to grow, you add nodes to the cluster instead. For the Worker Group, Worker, Pump, and Concurrency Limit concepts themselves, see Worker Groups; this page is the tuning view.

The dials live on `WorkerGroupOptions`#

Every single-node throughput setting is a property on WorkerGroupOptions, configured once per group inside AddBackWave. The two headline dials are PoolSize and Pumps; the rest are secondary knobs or belong to other pages.

Option	Type	Default	What it controls
`Name`	`string` (required)	—	Group name, unique within one registration; appears in health, metrics, and logs.
`Policy`	`DispatchPolicy` (required)	—	Which queues the group serves and how effort is shared across them.
`PoolSize`	`int`	`20`	Max jobs the group runs concurrently per pump. Polling pauses while the pool is full.
`Pumps`	`int`	`1`	Number of independent claim loops the group runs in one process. Must be at least `1`.
`MaxClaimBatch`	`int`	`32`	Max jobs claimed in a single poll; capped at `PoolSize` in practice.
`MaxOutcomeBatch`	`int?`	`null` (uses `MaxClaimBatch`)	Max completed outcomes buffered before one batched store write.
`PollInterval`	`TimeSpan`	`1 second`	How often the group polls for work.
`MaintenanceInterval`	`TimeSpan`	`5 seconds`	Cadence of background maintenance, separate from claim polling.
`LeaseDuration`	`TimeSpan`	`60 seconds`	How long a claimed job's lease is held before it lapses.
`HeartbeatInterval`	`TimeSpan?`	`null` (uses `LeaseDuration / 3`)	Lease-renewal cadence.
`RetryPolicy`	`RetryPolicy`	`RetryPolicy.Default`	Backoff schedule and attempt ceiling.
`Retention`	`RetentionPolicy?`	`RetentionPolicy.Default` (`null` disables)	How terminal jobs are kept then purged.

A process may host several groups, and each group gets its own independent PoolSize, Pumps, polling, and retry settings. The sections below focus on the throughput dials; lease, heartbeat, retry, and retention are covered on their own pages.

`PoolSize`, execution concurrency per pump#

PoolSize is the maximum number of jobs a group runs concurrently. In domain terms it is the Worker count: a Worker is one execution slot in the pool, not a thread. This is execution concurrency, how many jobs run at the same time, which is a different thing from the fetch-loop parallelism that Pumps controls. The default is 20.

The default is a flat constant, not scaled to the machine's CPU count. The async claim loop has no thread-per-job to starve, and a machine-independent admission bound keeps the connection footprint predictable. You tune the value per deployment.

PoolSize is also the node-local flow control, or Backpressure. Polling pauses while the pool is full and resumes as jobs finish, so the group never claims more work than it has slots to run. One subtlety matters here: a job that has finished executing but whose outcome has not yet been flushed to the store still holds its slot until the batched write lands. So the effective free capacity for a claim pass is PoolSize minus executing jobs minus buffered-but-unreported outcomes, not just PoolSize minus executing jobs.

Idle pool slots cost nothing. Raising PoolSize is the cheap dial: it lets more jobs run at once with no added connection footprint. It is the right dial when handlers are slow or IO-bound and jobs are queuing up while CPU and the database sit idle. It is the wrong dial when the bottleneck is the rate at which one pump can claim and report through the store; see the next section.

PoolSize is a positive integer. There is no enforced lower bound, so set it deliberately.

The serial store-I/O ceiling#

A group's store I/O is serial within one pump: one round-trip is in flight at a time. A single pump's throughput is therefore bounded by store round-trip latency, not by the number of Workers or by CPU.

The reason is the shape of a pump. Each pump is a single loop that processes one event at a time and awaits each resulting store round-trip before moving to the next. Claiming, heartbeating, reporting outcomes, minting scheduled work, expiring leases, and purging retention all share that one loop and run one at a time on it. Outcome writes go further: completed outcomes are coalesced and flushed as a single batched store write, which keeps the writer single-threaded so throughput can rise without opening more database connections.

A single pump's ceiling is roughly 1 / (store round-trip latency) for its claim-then-report cycle, no matter how large PoolSize is or how much CPU the box has. Raising PoolSize lets more jobs execute at once, which helps when handlers are the bottleneck. It does not raise the rate at which one pump can claim and report. If a node is store-round-trip-bound, with the claim and report loop saturated rather than handler execution or CPU, then PoolSize will not help. You need more pumps or more nodes.

Overlapping a single pump's own round-trips to lift the single-pump ceiling without adding pumps is not a dial that exists today. To get more store-I/O parallelism, add pumps or nodes.

`Pumps`, fan-out within one process#

Pumps is how many independent claim loops the group runs in one process: its fetch-loop parallelism. Each pump is a self-contained claim, execute, and report loop with its own PoolSize pool and its own claim stream. Each claims under a distinct worker identity, so the database hands each pump a disjoint set of jobs with no extra coordination. The default is 1, which is exactly the single-loop behavior.

Raising Pumps is the lever for single-node throughput once one pump is bound by store round-trip latency rather than by concurrency or CPU. It is the direct answer to the serial ceiling above: more pumps multiply a group's store-I/O parallelism within one process.

Because each pump gets its own pool, the maximum concurrent executions for a group on a node is Pumps × PoolSize, not PoolSize alone. This is easy to misread. A group configured with PoolSize = 16 and Pumps = 4 runs up to 64 jobs at once on that node, across 4 parallel store-I/O loops.

The cost is connections. Unlike pool slots, every pump draws database connections whether it is busy or not. Budget roughly 2 to 3 connections per pump when sizing your client connection pool and the database's connection limit. A group with Pumps = 4 draws on the order of 8 to 12 connections at steady state. Leave Pumps at 1 unless a node is throughput-bound and has connection headroom to spare. This is sizing guidance, not an enforced limit.

Program.cs

builder.Services.AddBackWave(bw => bw
    .UseStore(new PostgresJobStore(connectionString))
    .UseJobs(BackWaveJobs.Module)
    .AddWorkerGroup(new WorkerGroupOptions
    {
        Name = "emails",
        Policy = new DispatchPolicy.Strict("emails"),
        PoolSize = 16,   // up to 16 jobs run concurrently per pump
        Pumps = 4,       // 4 independent claim loops -> ~8-12 DB connections
    }));

AddWorkerGroup validates two things. A duplicate group name throws InvalidOperationException with the message Worker Group '{name}' is configured twice., and a Pumps value below 1 throws InvalidOperationException with Worker Group '{name}' must run at least one Pump (Pumps = {n}).. PoolSize, the batch sizes, and the intervals are not validated; they fall back to their defaults.

In-process pumps versus adding nodes#

Once you know your bottleneck, the choice between scaling up in-process and scaling out the cluster follows from where your headroom is.

Add pumps, scaling up in one process, when a single node is store-round-trip-bound and you still have database connection headroom on that host. Pumps multiply store-I/O parallelism in one process at the cost of connections.

Add nodes, scaling out the cluster, when you have run out of connection headroom on a node, when you want host-level redundancy, or when a single host's CPU or network is saturated. A Node is a process coordinating through the shared store, and a cluster is many nodes; each node runs its own worker groups and pumps. Multi-host cluster scale-out is a Networked Adapter story: Postgres or SQL Server can span hosts.

One adapter boundary matters here. With the Embedded Adapter (SQLite), any number of processes on a single host may share the database, each as its own Node, but the database is never shared across hosts, because file locking is unsafe over network filesystems. An Embedded deployment scales out only to more processes on the same host. True multi-host scale-out requires a Networked Adapter.

Finally, capacity to claim is not the same as capacity to run. Adding pumps and nodes raises how fast work can be claimed, but a per-queue, cluster-wide Concurrency Limit caps how many jobs of a queue execute simultaneously across the entire cluster, regardless of how many pumps or nodes you add. Backpressure (node-local pool capacity) and the Concurrency Limit are independent gates. If a queue is Concurrency-Limit-saturated, more pumps or nodes will not raise its throughput; you raise the limit. The Concurrency Limit is a queue setting, not a WorkerGroupOptions dial. See Worker Groups for how it works.

Which dial for which symptom#

Symptom	Likely bound	Dial to turn	Why
Handlers slow or IO-bound; CPU and DB idle; jobs queue up	Execution concurrency	Raise `PoolSize`	More slots run more jobs at once, and idle slots cost nothing.
`PoolSize` already high but throughput is flat; the claim-and-report loop is the bottleneck	Serial store-I/O ceiling per pump	Raise `Pumps` (with connection headroom)	Each pump is an independent store-I/O loop, so more pumps multiply parallelism.
Throughput-bound but the DB connection limit is nearly exhausted	Connections on this host	Add nodes	Each pump costs roughly 2 to 3 connections; out of headroom means scale the cluster.
Need host redundancy, or a single host's CPU or network is saturated	Host capacity	Add nodes	A cluster of nodes coordinates through the shared store.
Latency from enqueue to pickup is too high	Poll cadence	Lower `PollInterval`	A shorter interval lowers latency at the cost of more store queries.
One queue is capped no matter what you add	Cluster-wide Concurrency Limit	Raise that queue's Concurrency Limit	A per-queue, cluster-wide cap is independent of pumps and nodes.

When a node halts, that is the fail-stop story rather than a throughput dial; pumps roll up into the group's health, and the group reads halted only once every pump is down. See Health & Fail-Stop.

Where to go next#

Worker Groups: Workers, Pumps, Dispatch Policy, Backpressure, and the Concurrency Limit as concepts.
Health & Fail-Stop: per-pump health rollup, draining, and the fail-stop behavior of a node.
The Dashboard: the operator surface for acting on jobs and queues.
Scheduling: the poll interval and maintenance interval that govern when due and recurring work becomes runnable.

Previous← Health, Fail-Stop & Draining NextRetention & History Purge →