Skip to content

Reliability

Journal, DLQ lifecycle, ordering invariants, and host-scheduled maintenance.

Updated View as Markdown

A ReliabilityAdapter owns the write-ahead journal and the dead-letter queue. Destination adapters do not.

Implementations must preserve:

  1. Durable journal append before durable admission resolves.
  2. Destination success before journal acknowledgement.
  3. Durable DLQ write before acknowledgement on destination failure.
  4. An unacknowledged journal when both destination and DLQ fail.

A journal append failure in the batcher first tries a single-log writeDlq (so wal_full can still admit). That is an engine fallback, not a substitute for invariant 4 when destination insert fails.

DLQ states

type DlqState = 'pending' | 'processing' | 'dead';
State Meaning
pending Waiting for a scan
processing Claimed; insert in flight
dead Poisoned after retry.maxScanAttempts (default 10)

Claims are atomic pending → processing. Success deletes the entry. Failure rewrites attempts and returns to pending. Poison moves processing → dead.

Orphaned processing entries return to pending at bootstrap and at every host DLQ scan (runMaintenance / retryDlqNow). A failed settle after claim is a recovery boundary, not only a process-start event.

id is opaque and stable. Filesystem paths may appear in metadata for diagnostics only. requeueDead and deleteDead accept the id, never a path. requeueDead preserves the id and resets attempts to 0.

listDlq defaults: pending is included unless includePending: false; processing and dead are omitted unless includeProcessing: true / includeDead: true.

Maintenance

There are no internal recurring DLQ or retention timers in 1.0. Hosts schedule:

await audit.flush();            // drain RAM queues; compact journal when durable
await audit.retryDlqNow();      // recover orphans + one DLQ scan
await audit.runMaintenance();   // flush + orphans + one DLQ scan + retention prune

runMaintenance() is single-flight. A caller that arrives during a pass sets a pending flag so one extra pass runs after, with no overlap. Each pass:

  1. Flushes queues.
  2. Recovers DLQ orphans.
  3. Scans pending DLQ entries once (four claims in flight).
  4. Runs configured retention pruning.
  5. Calls requestMaintenance() (or deprecated rearmMaintenance()) when the adapter exposes it.

One call is a bounded unit of work, not drain-until-idle. Remaining journal waves, leftover DLQ entries, or a prune_incomplete throw need a follow-up.

The call rejects on flush, DLQ scan, or prune failure (AggregateError with message multiple maintenance phases failed when more than one phase fails). Listen for those throws. Do not fire-and-forget maintenance.

After shutdown, query / bulkInsert / prune on built-in destination adapters throw. Open a new logger (or a fresh adapter) to read persisted rows.

Navigation

Type to search…

↑↓ navigate↵ selectEsc close