Skip to content

yggdrasil.http_.send_config

send_config

CacheConfig 

CacheConfig(
    tabular: Optional[Holder] = None,
    mode: Mode = Mode.APPEND,
    anonymize: Literal["remove", "redact"] = "remove",
    received_from: Optional[datetime] = None,
    received_to: Optional[datetime] = None,
    cleanup_ttl: Optional[timedelta] = dt.timedelta(days=1),
    received_ttl: Optional[timedelta] = None,
)

refetch property 

refetch: bool

True when cached entries must be refetched instead of served.

UPSERT / MERGE / OVERWRITE / TRUNCATE are refresh dispositions: the caller wants the cached row replaced by a fresh wire response, so serving a hit would skip the refresh entirely. An explicit received_from / received_to window overrides this — the window decides freshness (in-window hits are still served) and the mode only shapes how the writeback lands.

local_cache_folder 

local_cache_folder(session: 'Session | None' = None) -> Path

Backend-agnostic root for the local cache.

Returns the bound :class:Folder's :attr:path when :attr:tabular is local (any :class:yggdrasil.io.path.Path subclass — LocalPath on disk, VolumePath on a Databricks Volume, S3Path on a bucket, …); otherwise builds the default LocalPath under ~/.cache/http/response, suffixed with the session's base_url host + path when one is available so different APIs sharing the same machine don't collide on disk:

  • base_url=https://api.example.com/v1/ → …/response/api.example.com/v1
  • base_url unset → …/response/default

Used as the per-config key for grouping cache hits in :class:yggdrasil.http_.response_batch.HTTPResponseBatch.

cache_tabular 

cache_tabular(session: 'Session | None' = None) -> 'Any'

Return the active cache backend as a :class:Tabular.

Single entry point both the local and remote pipelines call through:

  • :attr:tabular already set (constructor-supplied: live :class:Folder for local, Databricks Table or any third-party adapter for remote) — returned as-is.
  • Unset but the cache is otherwise enabled (received_* window) — the default ~/.yggdrasil/cache/response/... :class:Folder is materialised via :meth:local_cache_folder and memoised back on :attr:tabular so the next call short-circuits.

The on-disk layout is Hive-partitioned by whichever fields :meth:partition_columns reports (RESPONSE_SCHEMA's partition_by set — partition_key today), so the local Folder and the remote Table accept the same logical lookup primitive — the :class:Predicate built by :meth:make_lookup_predicate / :meth:make_batch_lookup_predicate — and the same :meth:Tabular.write_arrow_batches write call.

read_responses 

read_responses(
    requests: "Iterable[PreparedRequest]",
    *,
    spark_session: "Any" = None,
    session: "Session | None" = None
) -> "tuple[list[Response], list[PreparedRequest]]"

Read cache hits as :class:Response objects.

Returns (hits, misses) — matched by public_hash and filtered by received_from / received_to.

read_responses_tabular 

read_responses_tabular(
    requests: "Iterable[PreparedRequest]",
    *,
    spark_session: "Any" = None,
    session: "Session | None" = None
) -> "Tabular | None"

Read matching cache rows as a :class:Tabular.

Builds the batch lookup predicate from requests and reads from :attr:tabular (or the default local cache folder).

write_responses 

write_responses(
    responses: "list[Response]",
    *,
    mode: "Mode | None" = None,
    spark_session: "Any" = None,
    session: "Session | None" = None
) -> None

Write :class:Response objects to the cache backend.

write_responses_tabular 

write_responses_tabular(
    data: "Any",
    *,
    mode: "Mode | None" = None,
    spark_session: "Any" = None,
    session: "Session | None" = None
) -> None

Write Arrow / Spark response data to the cache backend.

make_lookup_predicate 

make_lookup_predicate(request: PreparedRequest | None = None) -> 'Any | None'

Single-request :class:Predicate for the cache read.

Shape: partition_key == <req.partition_key> AND the per-request match clause.

make_batch_lookup_predicate 

make_batch_lookup_predicate(
    requests: Iterable[PreparedRequest],
) -> "Any | None"

Batch :class:Predicate for the cache read.

Shape: partition_key IN (<distinct keys>) AND (req1_match) OR (req2_match) OR ….

SendConfig 

SendConfig(
    raise_error: bool = True,
    wait: "WaitingConfig | None" = None,
    remote_cache: "CacheConfig | None" = None,
    local_cache: "CacheConfig | None" = None,
    cache_only: bool = False,
    spark_session: "bool | None" = None,
    stream: bool = False,
)

split_requests 

split_requests(
    requests: list[PreparedRequest], *, session: "Any" = None
) -> "tuple[set[int], set[int], list[PreparedRequest]]"

Split requests into local hit hashes, remote hit hashes, and misses.

Performs lightweight hash-only lookups against local then remote cache. Returns (local_hashes, remote_hashes, misses) — full response rows are read on demand, not here.

read_hits 

read_hits(
    cache: "CacheConfig",
    requests: "list[PreparedRequest]",
    *,
    session: "Any" = None
) -> "Tabular | None"

Read full response rows for hit requests from a cache.

write_responses 

write_responses(responses: 'list[Response]', *, session: 'Any' = None) -> None

Write responses to both local and remote caches asynchronously.

write_responses_tabular 

write_responses_tabular(data: 'Any', *, session: 'Any' = None) -> None

Write Arrow/Spark response data to local and remote caches.

Both writes run concurrently via threads when both caches are configured.