Introduction

Nestor is a read-through block cache for S3-compatible object storage. Objects are split into fixed-size blocks held in RAM and on local disk. A read of any byte range resolves to the blocks covering it, hits are served locally and misses are fetched from the origin once, no matter how many readers wait on them.

The origin can be AWS S3 or anything speaking its API, such as MinIO, RustFS, Ceph or Cloudflare R2. Nestor only needs GET and HEAD on the objects it caches.

Three shapes

The same engine runs in three ways, chosen per deployment rather than per codebase.

• Library. The nestor crate embeds in a Rust service. Namespaces map to origins, reads return Bytes or a stream. nestor-store wraps it as an object_store::ObjectStore, so existing code keeps its API and gains the cache.
• S3 endpoint. The nestor binary listens as an S3 endpoint. GET and HEAD are served from cache, every other request is forwarded to the origin and re-signed. A client switches by changing AWS_ENDPOINT_URL_S3, on the same host or on a dedicated machine.
• Cluster. Several nestor binaries share one cache. Clients, or a nestor gateway in front of them, route each block to its owner with rendezvous hashing, so the cluster holds each block once and grows by adding nodes.

Features

• Block-level caching. Objects are cached as blocks, 1 MiB by default and configurable from 64 KiB to 16 MiB per namespace. A 4 KiB read of a 10 GiB object costs one block, a full scan streams block by block with a bounded window in flight.
• Hybrid RAM and disk. RAM is sharded and evicted with S3-FIFO. Disk is an optional second tier managed by foyer, with direct I/O and io_uring on Linux. Both tiers hold the same block keys.
• Origin protection. Concurrent misses on a block are coalesced into one fetch. Fetches are bounded by semaphores, retried with backoff and hedged against slow responses. A cold read costs one round trip, the first GET carries size and ETag.
• Strong consistency. Every block key carries a content tag derived from the object's ETag. A changed object never aliases the old blocks. Namespaces choose between ETag revalidation on a TTL and immutable objects that are never re-checked.
• Drop-in deployment. The S3 endpoint verifies SigV4 signatures and presigned URLs from clients, re-signs forwarded requests with its own credentials and invalidates or populates the cache on writes that pass through it.

Where it fits

Nestor suits read-heavy access to objects on S3-compatible storage where the same bytes are read more than once: segment files of a log or stream store, Parquet and Iceberg data files behind a query engine, model weights and datasets, build artifacts. Random small reads and large sequential reads both map to blocks, so one cache serves both.

It is a cache, not a store. Writes are forwarded to the origin unchanged, and durability is the origin's. A cache node can be stopped, wiped and restarted, the only cost is a cold start.