A ReliabilityAdapter owns the write-ahead journal and the dead-letter queue. Destination adapters do not.
Implementations must preserve:
- Durable journal append before durable admission resolves.
- Destination success before journal acknowledgement.
- Durable DLQ write before acknowledgement on destination failure.
- An unacknowledged journal when both destination and DLQ fail.
A journal append failure in the batcher first tries a single-log writeDlq (so wal_full can still admit). That is an engine fallback, not a substitute for invariant 4 when destination insert fails.
DLQ states
type DlqState = 'pending' | 'processing' | 'dead';| State | Meaning |
|---|---|
pending |
Waiting for a scan |
processing |
Claimed; insert in flight |
dead |
Poisoned after retry.maxScanAttempts (default 10) |
Claims are atomic pending → processing. Success deletes the entry. Failure rewrites attempts and returns to pending. Poison moves processing → dead.
Orphaned processing entries return to pending at bootstrap and at every host DLQ scan (runMaintenance / retryDlqNow). A failed settle after claim is a recovery boundary, not only a process-start event.
id is opaque and stable. Filesystem paths may appear in metadata for diagnostics only. requeueDead and deleteDead accept the id, never a path. requeueDead preserves the id and resets attempts to 0.
listDlq defaults: pending is included unless includePending: false; processing and dead are omitted unless includeProcessing: true / includeDead: true.
Maintenance
There are no internal recurring DLQ or retention timers in 1.0. Hosts schedule:
await audit.flush(); // drain RAM queues; compact journal when durable
await audit.retryDlqNow(); // recover orphans + one DLQ scan
await audit.runMaintenance(); // flush + orphans + one DLQ scan + retention prunerunMaintenance() is single-flight. A caller that arrives during a pass sets a pending flag so one extra pass runs after, with no overlap. Each pass:
- Flushes queues.
- Recovers DLQ orphans.
- Scans pending DLQ entries once (four claims in flight).
- Runs configured retention pruning.
- Calls
requestMaintenance()(or deprecatedrearmMaintenance()) when the adapter exposes it.
One call is a bounded unit of work, not drain-until-idle. Remaining journal waves, leftover DLQ entries, or a prune_incomplete throw need a follow-up.
The call rejects on flush, DLQ scan, or prune failure (AggregateError with message multiple maintenance phases failed when more than one phase fails). Listen for those throws. Do not fire-and-forget maintenance.
After shutdown, query / bulkInsert / prune on built-in destination adapters throw. Open a new logger (or a fresh adapter) to read persisted rows.