yggdrasil.http_.stream

stream

Resumable HTTP response stream — :class:HTTPStream.

:class:HTTPStream wraps a live HTTP socket body with automatic retry-on-disconnect. When the source read raises a transient error (SSL EOF, connection reset, socket timeout), it reconnects to the origin with a Range: bytes=<received>- header and continues pulling from where it left off — transparent to the consumer.

Used by :meth:HTTPResponse.from_wire as the buffer for response bodies so large downloads survive mid-flight connection drops.

HTTPStream

HTTPStream(
    source: Any = None,
    *,
    request: "HTTPRequest | None" = None,
    session: Any = None,
    max_retries: int = 4,
    content_encoding: Optional[str] = None,
    **kwargs: Any
)

Bases: MemoryStream

MemoryStream with HTTP range-request retry on transient failures.

byte_size property

byte_size: int

Maximum bytes retained (memory + spill).

spill_threshold property

spill_threshold: int

In-memory window cap; bytes above spill to disk.

window_start property

window_start: int

Absolute offset of the first byte in the in-memory window.

window_end property

window_end: int

Absolute offset one past the last byte in the in-memory window (== :attr:size).

spill_start property

spill_start: int

Absolute offset of the first retained byte.

Equal to :attr:window_start when no spill is active. Otherwise it sits at the start of the spill region; bytes before this have been evicted and reads behind it raise.

has_spill property

has_spill: bool

True iff a spill tempfile is currently active.

eof property

eof: bool

True once the source has signalled EOF.

is_streaming property

is_streaming: bool

True while EOF hasn't been reached on the source feed.

:attr:size reflects only the bytes pulled so far, so cursor-anchored readers (:class:IO.read) need to bypass the standard cap at size clamp until the source signals EOF.

size_known property

size_known: bool

False while the source can still grow :attr:size.

The base :class:Holder reports True for in-memory holders because their size is a settled slot. A streaming holder is different: :attr:size reflects only the bytes pulled so far, so before the first read it is 0 even though the body is non-empty. Format leaves (parquet / arrow / csv / ndjson) guard their read with if self.size_known and self.size == 0: return as an empty-buffer short-circuit — with the base True that fired on an un-pulled stream and silently returned zero rows. Reporting the size as unknown until EOF makes the short-circuit defer to the actual read (which drives the pull), so a stream=True response body parses correctly.

read_mv

read_mv(size: int = -1, offset: int = 0, *, cursor: bool = False) -> memoryview

Read size bytes at absolute offset, pulling from source as needed. size < 0 reads to EOF.

cursor=True reads from the holder's internal cursor and advances it past the returned bytes. Reads are valid in [spill_start, size) — anything behind :attr:spill_start has been evicted (truly dropped from both memory and spill) and raises.

write_mv

write_mv(
    data: memoryview,
    offset: int = 0,
    *,
    size: int = -1,
    overwrite: bool = False,
    update_stat: bool = True,
    cursor: bool = False
) -> int

Splice bytes at offset. Appends past current end extend the stream and may slide the window; in-window writes overwrite.

cursor=True writes at the internal cursor and advances it. size>=0 slices the input buffer to at most size bytes before the splice. overwrite=True truncates the tail past offset + len(data) after the splice (same contract as :meth:Holder.write_mv). Writes behind :attr:window_start raise — the target bytes have already been evicted.

reserve

reserve(n: int) -> None

Pre-grow the underlying bytearray, capped at :attr:byte_size.

Capacity beyond :attr:byte_size would be evicted on the very next :meth:_slide_window call, so the cap is the honest ceiling here.

truncate

truncate(n: int) -> int

Set visible :attr:size to n. Shrinks drop the tail (in-memory and spill); extends zero-pad in memory.

Truncating below :attr:spill_start raises — those bytes are evicted and unrecoverable. A truncate that lands inside the spill region drops the trailing spill bytes and the whole in-memory window.

memoryview

memoryview() -> memoryview

View over the live window only (not the full stream).

Override of :meth:Holder.memoryview — that one would call self.read_mv(-1, 0), which raises once :attr:window_start

0 because position 0 is no longer in the window. The window view is the honest answer for a sliding-window holder.