Skip to content

Production

Replica isolation, encryption, filesystem threat model, and ops alerts.

Updated View as Markdown

Choose a reliability backend

Deployment Reliability Notes
Long-lived process that can lose queued logs root memory default Volatile; monitor queue pressure
Node, Bun, or Deno replica FileReliabilityAdapter Unique namespace and local writable disk per replica
Cloudflare Workers CloudflareReliabilityAdapter in a DO Standard Worker calls the owning DO; DO alarm schedules maintenance

Durable mode rejects missing or non-persistent reliability synchronously.

Await audit.ready before fire / fireAsync in durable mode — the pre-ready buffer is volatile even when mode: 'durable'.

Filesystem checklist

  1. Store dataDir on durable local storage, not a shared multi-writer volume.
  2. Use a unique reliability namespace for each exclusive storage owner (FileReliabilityAdapter({ namespace })). LogbunConfig.namespace is a separate identifier validated at bootstrap; it does not choose the WAL/DLQ directory by itself. Typical replicas set both to the same replica id.
  3. Keep the default instance lock unless an external exclusivity mechanism is known to be correct. The lock prevents accidental multi-writer use; it is not a security boundary against a malicious process running as the same OS user.
  4. Set WAL/DLQ limits and alert on walApproxBytes, dlqPending, dlqProcessing, and dlqDead.
  5. Use encryptionKey when local journal/DLQ files require at-rest encryption.
  6. Use a destination with idempotent insert by LogbunLog.id. When integrityChain is enabled, persist prevHash / contentHash.

Layout for FileReliabilityAdapter({ dataDir, namespace }):

{dataDir}/{namespace}/
  .instance.lock
  .instance.lock.recovery
  wal/
    current.aof
    seg-000001.aof
    acked.ids
  dlq/
    {opaque-uuidv7}.batch
    {opaque-uuidv7}.batch.processing
    {opaque-uuidv7}.batch.dead

Sealed WAL segments are seg-NNNNNN.aof (six-digit sequence). Namespace and opaque IDs are validated; lookup is confined below the adapter-owned directory.

With filesystem fsync enabled, first-run WAL initialization publishes file entries and newly created wal / namespace / dataDir entries child-before-parent.

Encryption

encryptionKey on FileReliabilityAdapter: AES-256-GCM for WAL lines and DLQ files.

Accepted material (via normalizeEncryptionKey):

  • 32-byte Uint8Array
  • 64-character hex
  • standard base64 of exactly 32 bytes

Passphrases are rejected. A wrong or missing key fails closed on compact/read for a complete e1: line (ciphertext is not rewritten away as junk JSON). Complete plaintext lines while a key is configured also fail closed. Those compact errors reject flush() / runMaintenance(). error.name is WALEncryptedLineError, WALPlaintextLineError, or WALFailClosedError — these names are not root exports.

Torn EOF crash tails (no terminating newline) are skipped so compact/recovery can proceed.

Alerting

Observe via onEvent, getStats(), and getStatsDetailed().

getStats() is RAM-only: queued, tenants, degraded, recoveryBacklog, inflightFlushes. Before ready, queued is the pre-ready buffer length (tenants is 1 when that buffer is non-empty).

getStatsDetailed() adds walApproxBytes / dlqPending / dlqProcessing / dlqDead. When degraded or not ready those disk fields are 0. When reliability.getStats() throws they are omitted (not zeroed) and a stats / reliability_get_stats event is emitted.

Alert on bootstrap_fail, degraded, wal_fail, drop, poison, dlqDead, limit (unsafe_default_volatile, unsafe_default_require_tenant, wal_full, pre_ready_buffer_full, max_active_tenants), and sustained flush_fail. Listener throws never break the pipeline (safeEmit).

Filesystem security model

The filesystem adapter validates namespaces and opaque IDs, rejects observed symbolic-link path segments, revalidates storage directories and files around operations, and uses O_NOFOLLOW where the portable Node-compatible interface exposes it. These checks protect against traversal, accidental redirection, and filesystem substitutions that are present when validation runs.

They do not provide malicious same-user isolation. Node, Bun, and Deno do not expose portable openat / directory-handle-relative rename, link, and unlink, so another process with write access can rename an already validated ancestor in the interval before a path-based syscall. Put dataDir under OS permissions, a dedicated user/container, or another isolation boundary that excludes hostile writers.

The instance lock coordinates cooperative owners and catches accidental namespace sharing. A same-user attacker can replace or remove it, including racing the final inode recheck. Network filesystems may not provide the required exclusive-create or durability semantics.

Stale-lock recovery claims are published only after complete PID/process-start metadata has been synced. A valid claim is replaced only when its process is known dead or its process-start identity proves PID reuse; it never expires by elapsed time. Malformed legacy/crash remnants are eligible only after the configured safety age. Permission or other unknown liveness-probe failures remain potentially live and fail closed indefinitely.

Integration tests (this repository)

bun test is the in-process suite. It does not start Turso, ClickHouse, or Wrangler. This repo’s CI unit job uses Bun 1.4.0 (engines.node for consumers is >=18).

bun run test:integration

That command sets LOGBUN_INTEGRATION=1 and starts turso dev, ClickHouse (clickhouse/clickhouse-server:24.8 or a clickhouse binary), and bunx wrangler dev --local. It fails if those services cannot start. Override with TURSO_BIN, CLICKHOUSE_BIN, or LOGBUN_CLICKHOUSE_IMAGE. Optional: TURSO_URL + TURSO_AUTH_TOKEN also smoke Turso Cloud. Nothing is deployed to Cloudflare.

prepublishOnly is build && typecheck && test && assert:root-runtime — the fast suite, not integration.

Navigation

Type to search…

↑↓ navigate↵ selectEsc close