nestor-store

nestor-store connects Nestor to object_store in both directions. ObjectStoreOrigin turns any ObjectStore into an origin Nestor can read from. NestorStore turns a cached namespace back into an ObjectStore, so code written against the trait gains the cache without changing.

ObjectStoreOrigin

use std::sync::Arc;
use nestor::Namespace;
use nestor_store::ObjectStoreOrigin;
use object_store::aws::AmazonS3Builder;

let s3: Arc<dyn object_store::ObjectStore> = Arc::new(
    AmazonS3Builder::from_env().with_bucket_name("data").build()?,
);
let namespace = Namespace::new("data", Arc::new(ObjectStoreOrigin::new(s3)));

The origin maps GetOptions onto object_store::GetOptions, ranges become GetRange::Bounded, if_match and if_none_match pass through, and object_store errors are classified into OriginError variants so the fetcher retries and fails correctly. Anything object_store supports is an origin: S3 and compatible stores, GCS, Azure, HTTP, the local filesystem and InMemory for tests.

NestorStore

use nestor_store::NestorStore;

let cached: Arc<dyn ObjectStore> = Arc::new(NestorStore::new(nestor, ns, s3.clone()));

NestorStore takes the engine, a namespace id and the store to forward writes to. The wrapped store is normally the same one behind the namespace's origin, but it need not be: a write-only credential can back writes while the origin reads with another.

Operation	Behaviour
get, get_opts, get_range, get_ranges	Served through the cache. GetOptions preconditions and head are honoured, GetRange::Offset and Suffix map to ReadRange.
head	Metadata cache, origin HEAD on a miss.
put, put_opts	Forwarded. On success the object is invalidated and, by default, inserted into the cache with the ETag the store returned.
put_multipart	Forwarded. The object is invalidated when the upload completes.
delete, delete_stream	Forwarded, then invalidated.
copy, rename and their conditional forms	Forwarded, destinations and sources invalidated.
list, list_with_offset, list_with_delimiter	Forwarded untouched.

populate_on_write(false) keeps the invalidation and skips the insert, for write paths whose objects are rarely read back soon or are too large to hold. Populating requires the payload in memory, which PutPayload already is.

Errors map back to object_store::Error: NotFound, NotModified, Precondition and a generic error carrying the Nestor message otherwise.

Choosing between the two APIs

NestorStore is the right entry point for code that already speaks object_store, such as anything built on DataFusion, Parquet readers or Iceberg. Nothing changes but the constructor.

The nestor API is the right one for code that owns its read path. It exposes ReadStream::ready for early access to metadata, ReadRange::From and Suffix without allocating an object_store::GetOptions, insert for write paths that hold the bytes, and NamespaceId so one engine can serve many object classes with different block sizes and consistency modes.