Pipeline | Logbun

What happens to an audit event from fire() until the destination acknowledges it.

fire / fireAsync
  → redact, cap, UUIDv7, optional integrity seal
  → durable: ReliabilityAdapter.appendJournal
      journal append failure → writeDlq (single log)
      journal + DLQ failure → admission fails (fireAsync rejects; fire drops)
  → RAM tenant queue / short batching timer
      new tenant key beyond maxActiveTenants:
        durable persistent → writeDlq (no journal)
        volatile → drop
      queue already at cap after journal append → fair-share dump or
        single-log DLQ + journal ack
  → destination bulkInsert
      success → acknowledgeJournal
      failure → writeDlq → acknowledgeJournal
      destination + DLQ failure → leave journal unacknowledged

The destination adapter must be idempotent on LogbunLog.id. Recovery can re-deliver a batch after a crash between a successful destination write and its journal ack.

Admission vs delivery

Admission is “the library accepted the event.” In durable mode, fireAsync resolves after the journal (or DLQ fallback) commit.
Delivery is “the destination bulkInsert succeeded.” That happens later, on a batching timer, flush(), or runMaintenance().

fire() never throws. Failures become onEvent (drop, wal_fail, …). On Workers, pass context.waitUntil so the isolate stays alive for the admission task.

fireAsync() waits for admission and may reject. It does not wait for destination insert unless you also flush().

Batching

RAM queues flush when they hit batching.maxSize (default 100) or batching.flushInterval (default 5_000 ms), whichever comes first. The interval is a short batching timer, not DLQ retry scheduling.

Queues snapshot before asynchronous insertion. Concurrent enqueues receive a fresh queue and cannot be erased by a DLQ spill.

Backpressure

Cap	Default	When exceeded
Per-tenant `batching.maxQueueSize`	1_000	Fair-share dump of the largest queues, or drop if `onQueueFull: 'drop'` (volatile only)
`maxActiveTenants`	10_000	New tenant key: durable → DLQ (`dlq` / `max_active_tenants`); volatile → drop
`maxTotalQueued`	50_000	Global sum of queue lengths + reservations
Journal / DLQ size	adapter-specific	`wal_full` / `dlq_full`

onQueueFull: 'drop' is rejected when mode: 'durable'.

Bootstrap order

Initialize reliability and acquire exclusive ownership when applicable (CloudflareReliabilityAdapter.init also recovers DLQ orphans and requests a pending-work alarm).
Initialize the destination (adapter.init).
Read a bounded wave of unacknowledged journal records into queues (maxRecoveryBatch, floored at 1).
Recover orphaned DLQ processing claims.
Construct the retry engine (no timers — hosts call runMaintenance()).

Logs accepted before ready sit in a volatile pre-ready buffer. Bootstrap drains that buffer through the real enqueue path (including journal) before ready resolves.