`yggdrasil.aws.fs.s3_http`¶

s3_http ¶

Pure-HTTP S3 REST client — the boto3-free S3 data plane.

Speaks the S3 REST API directly over :class:yggdrasil.http_.session.HTTPSession (connection pooling, retries, streaming) with requests signed by :class:yggdrasil.aws.fs.sigv4.SigV4Signer. Covers exactly what :class:yggdrasil.aws.fs.path.S3Path needs:

HEAD / ranged GET / PUT / DELETE single objects,
ListObjectsV2 (paginated, CommonPrefixes + Contents),
multi-object DeleteObjects,
multipart upload (create / upload-part / complete / abort) so a multi-GB write streams to S3 in bounded-memory parts.

The wire is reached through an injectable transport — (method, url, headers, body) -> S3Response — defaulting to an :class:HTTPSession. Tests swap in an in-memory transport; production rides the pooled session. XML is parsed with the stdlib (S3 list/multipart payloads are small + flat).

S3Error ¶

S3Error(status: int, code: str, message: str, key: str = '')

Bases: OSError

Non-success S3 response (4xx/5xx that isn't a benign 404/416).

S3NotFound ¶

S3NotFound(status: int, code: str, message: str, key: str = '')

Bases: S3Error

404 / NoSuchKey.

S3Response `dataclass` ¶

S3Response(status: int, headers: Mapping[str, str], content: bytes)

Minimal wire response the transport hands back.

S3HttpClient ¶

S3HttpClient(
    *,
    bucket: str,
    endpoint: URL,
    signer: SigV4Signer,
    transport: Optional[Callable[..., S3Response]] = None,
    path_style: bool = False,
    region: Optional[str] = None,
    managed: bool = True,
    no_proxy: Optional[str] = None
)

Signed S3 REST client for one bucket.

endpoint is the scheme+host the bucket lives behind (virtual-hosted https://<bucket>.s3.<region>.amazonaws.com by default; a path-style override host is honored for S3-compatible stores). One instance per :class:yggdrasil.aws.fs.path.S3Bucket, so the pooled session and signer are shared by every key under it.

head ¶

head(key: str) -> 'S3Response | None'

HEAD — returns the response (Content-Length / Last-Modified / Content-Type in headers) or None when the object is absent.

get ¶

get(key: str, *, start: int = 0, length: int = -1) -> S3Response

Ranged GET. length < 0 from start reads to EOF; the whole-object fast path omits the Range header entirely.

put ¶

put(
    key: str,
    body: bytes,
    *,
    content_type: "str | None" = None,
    if_none_match: bool = False
) -> S3Response

PUT an object.

if_none_match=True sends If-None-Match: * so the PUT only succeeds if the key does not already exist — S3's atomic create-if-absent (GA since late 2024). A losing writer gets HTTP 412 PreconditionFailed, which we surface as :class:FileExistsError. This is what makes a Delta commit JSON race genuinely atomic on object storage (two writers contending for the same version: exactly one wins, the other rebases) — the local-FS path uses O_EXCL for the same guarantee.

delete_batch ¶

delete_batch(keys: 'list[str]') -> None

Multi-object delete (POST /?delete). Content-MD5 is required by S3 for this op; DeleteObjects errors are surfaced as one OSError.

put_streamed ¶

put_streamed(
    key: str, parts: "Iterator[bytes]", *, content_type: "str | None" = None
) -> int

Multipart-upload an iterator of byte chunks; returns total bytes.

Each chunk should be >= 5 MiB (the S3 part minimum) except the last. Aborts the upload on any failure so no orphaned parts linger.

yggdrasil.aws.fs.s3_http¶