Cache tiers

Blocks live in a hybrid cache built on foyer: a sharded RAM tier and an optional disk tier that share one key space. A lookup checks RAM, then disk. An insert goes to RAM and, when disk is configured, is written through to disk in the same step.

RAM

The RAM tier is bounded by bytes, not entries, and each block is weighted by its length plus its key. Eviction is S3-FIFO, which keeps one-hit blocks from a scan out of the main queue so a full-object read does not flush the working set of random readers behind it.

The tier is split into shards that each lock and evict independently. The default is two shards per core, capped so that every shard holds at least 32 MiB. A 256 MiB cache on a 16 core machine gets 8 shards rather than 32, since a shard smaller than a few dozen blocks evicts erratically. Set shards explicitly when the default is wrong for the workload.

Blocks are Bytes, so a hit hands the caller a reference-counted slice of the cached buffer. No copy is made on the read path.

Disk

Disk is optional. Without it, RAM is the only tier and a block that falls out of RAM is fetched again. With it, a block evicted from RAM is still a hit as long as it is on disk, which makes the working set the disk size rather than the RAM size.

Setting	Default	Meaning
path		Directory foyer manages. Existing content is recovered on start.
capacity		Bytes on disk. Filled before anything is reclaimed.
region_size	64 MiB	Append unit and reclaim unit. Larger regions mean fewer, larger sequential writes.
direct_io	true	Bypass the page cache on Linux so RAM stays available for the RAM tier.
compression	none	lz4 or zstd per block. Worth it for text, not for already compressed data.
recover	quiet	none starts empty, quiet recovers what it can, strict fails on any corruption.

The disk is written as a log of regions. A block is appended to the current region, regions are reclaimed FIFO when capacity is reached, and blocks within a reclaimed region are gone. There is no read-modify-write, so write amplification is one and a consumer-grade SSD lasts.

Inserts are written to disk at insertion time rather than on eviction from RAM. With this policy a block is durable on disk as soon as the origin has delivered it, so a restart shortly after warming does not lose the warm set, and RAM eviction never causes a burst of disk writes. Writes are buffered in a pool shared by the flushers, sized from capacity and clamped to [flushers * region_size, 256 MiB].

Disk I/O uses io_uring on Linux and falls back to psync where it is unavailable, such as under Docker's default seccomp profile.

Restart

Recovery scans the regions on disk and rebuilds the index, so an endpoint restarted with the same path serves hits from disk immediately. Only the metadata cache is lost, and it refills from the first read of each object as described in Consistency.

Removing the path, or starting with recover = "none", is a full cold start. Since Nestor holds no state the origin does not also hold, that is always safe.