Expand description
LD_PRELOAD shim that faults writes aimed at configured path prefixes, used by integration tests to reproduce storage faults (a full disk, a failing or slow device) that are otherwise hard to provoke deterministically without root.
A test scopes injection by directory, filename, and time, letting it fault
one subsystem while the rest of the process keeps working. The directory
scope is a colon-separated list of absolute prefixes in
KUMO_FAULT_PATH_PREFIXES. The filename scope is an optional
colon-separated list of suffixes in KUMO_FAULT_PATH_SUFFIXES (e.g. .sst
to hit only flushed table files and leave the write-ahead log untouched).
The time scope is a sentinel file at KUMO_FAULT_ACTIVE_FILE: faults apply
only while that file exists. A test lets startup complete and creates the
sentinel to begin the fault, then removes it to restore service.
Two fault kinds are available. By default a faulted write fails with an
errno (ENOSPC, overridable via KUMO_FAULT_ERRNO), modelling a full or
failing disk. If KUMO_FAULT_DELAY_MS is non-zero a faulted write instead
sleeps that long and then succeeds, modelling slow storage – which, applied
to flush writes, stalls RocksDB into rejecting foreground writes with
Incomplete and exercises the backpressure paths.