Skip to content

yggdrasil.enums

enums

TimeUnit

Bases: str, Enum

Canonical time-unit token for temporal DataType instances.

Members carry the lowercase short form so the enum is a drop-in string replacement::

TimeType(unit=TimeUnit.MICROSECOND)
TimeType(unit="us")  # equivalent — both store ``"us"``

Use :meth:parse when accepting external input — it canonicalizes aliases ("microseconds", "micros", "µs") to a member and raises :class:ValueError for unknown tokens.

seconds property

seconds: float

Seconds per one of this unit (used for scalar epoch math).

Calendar-style interval units (year_month / day_time / month_day_nano) have no fixed second-count and return float('nan') so comparisons surface the mismatch instead of silently truncating to zero.

order property

order: int

Precision rank — higher = finer.

Used by TemporalType._merge_with_same_id to pick the wider of two units. Calendar interval units sit at rank -1 so they don't outrank fixed-precision ones in normal merges.

is_subsecond property

is_subsecond: bool

True for ms / us / ns.

is_calendar property

is_calendar: bool

True for the variable-length interval forms.

from_ classmethod

from_(value: Any, *, default: Any = ...) -> 'TimeUnit'

Coerce any Python value into a :class:TimeUnit.

Accepts:

  • :class:TimeUnit (returned as-is);
  • any string the alias table or canonical member values know — s / ms / us / ns / d / interval forms, plurals (microseconds), long forms (millisecond), mixed case, µs, hyphens / spaces;
  • objects exposing a time_unit / unit attribute (Polars Datetime / Duration, PyArrow TimestampType / DurationType / Time32Type / Time64Type); the attribute is re-funneled through from_;
  • None — returns default if supplied, else raises.

default swallows unknown / unparseable input. Without it, unknown tokens raise :class:ValueError and unsupported types raise :class:TypeError.

is_valid classmethod

is_valid(value: Any) -> bool

Return True when :meth:from_ would succeed for value.

Timezone dataclass

Timezone(iana: str)

An immutable wrapper around an IANA timezone identifier.

Instances are created via :meth:from_ (accepts strings, ZoneInfo, datetime.tzinfo, other Timezone objects, or None → UTC) or directly::

tz = Timezone("Europe/Paris")
tz = Timezone.from_("CET")        # → Timezone("Europe/Paris")
tz = Timezone.from_("+01:00")     # → Timezone("Etc/GMT-1")
tz = Timezone.from_(ZoneInfo("Europe/Paris"))

The naive case is represented by :attr:Timezone.NAIVE — a sentinel instance whose iana is the empty string and whose __bool__ is False. Use it instead of None so a tz: Timezone field type can stay non-optional.

key property

key: str

Alias for :attr:iana — matches ZoneInfo.key.

utc_seconds_offset property

utc_seconds_offset: int | None

Fixed UTC offset in seconds, or None for non-fixed zones.

Returns 0 for UTC and its equivalents (Etc/UTC, GMT, Etc/GMT+0, …), the parsed offset for Etc/GMT±N (note IANA's sign-flip — Etc/GMT-3 means UTC+3, so the property reports +10800 not -10800), and None for any zone whose offset depends on DST or for :attr:Timezone.NAIVE. Use :meth:utc_offset instead when you need a DST-aware offset for a specific instant.

Naming follows the "type-suffix unit" convention so utc_seconds_offset makes the unit explicit at the call site — utc_offset returns a :class:datetime.timedelta, utc_offset_hours returns fractional hours, and this property returns whole seconds (or None).

tzinfo property

tzinfo: tzinfo

Return as a stdlib tzinfo (same as to_zoneinfo()).

from_ classmethod

from_(obj: Any, *, default: Any = ...) -> 'Timezone'

Coerce any Python value into a :class:Timezone.

Accepts:

  • :class:Timezone (returned as-is — including Timezone.NAIVE);
  • :class:zoneinfo.ZoneInfo (extracts key);
  • a string — IANA name, alias (CET / EST / Z / GMT / Etc/UTC / +00:00 / …), or fixed offset ("+01:00", "UTC-05", "-0530");
  • datetime.tzinfo instance (ZoneInfo, datetime.timezone, third-party zones) — the key / zone attribute is preferred; otherwise falls back to the fixed UTC offset;
  • timezone-aware datetime / time (extracts tzinfo);
  • objects exposing a tz / time_zone / timezone / iana attribute (PyArrow TimestampType, Polars Datetime, foreign Timezone classes); the attribute is re-funneled through from_;
  • None — returns :attr:UTC for backward compatibility unless default is supplied.

default swallows unknown / unparseable input. Without it, bad input raises :class:ValueError (string parse failure) or :class:TypeError (unsupported value type).

to_zoneinfo cached

to_zoneinfo() -> ZoneInfo

Return the zoneinfo.ZoneInfo for this timezone.

Raises :class:ValueError for :attr:Timezone.NAIVE since there is no concrete zone to materialize.

utc_offset

utc_offset(at: datetime | None = None) -> _dt.timedelta

Return the UTC offset at instant at (default: now).

The result accounts for DST transitions.

Timezone("Europe/Paris").utc_offset(datetime(2024, 7, 1)) datetime.timedelta(seconds=7200) # +02:00 (CEST)

utc_offset_hours

utc_offset_hours(at: datetime | None = None) -> float

Return the UTC offset in fractional hours.

Timezone("Asia/Kolkata").utc_offset_hours() 5.5

is_naive

is_naive() -> bool

Return True for :attr:Timezone.NAIVE — the tz-less sentinel.

is_utc

is_utc() -> bool

Return True if this timezone is UTC (or an equivalent).

is_fixed_offset

is_fixed_offset() -> bool

Return True if this timezone has a fixed UTC offset (no DST).

is_dst

is_dst(at: datetime | None = None) -> bool

Return True if DST is active at instant at (default: now).

Timezone("Europe/Paris").is_dst(datetime(2024, 7, 1)) True Timezone("Europe/Paris").is_dst(datetime(2024, 1, 1)) False

dst_offset

dst_offset(at: datetime | None = None) -> _dt.timedelta

Return the DST adjustment at instant at.

Returns timedelta(0) when DST is not active or for fixed-offset timezones, and for :attr:Timezone.NAIVE.

abbreviation

abbreviation(at: datetime | None = None) -> str

Return the timezone abbreviation (e.g. "CET", "CEST") at at.

Timezone("Europe/Paris").abbreviation(datetime(2024, 7, 1)) 'CEST' Timezone("Europe/Paris").abbreviation(datetime(2024, 1, 1)) 'CET'

distance_to

distance_to(other: 'Timezone', at: datetime | None = None) -> _dt.timedelta

Return the offset difference between self and other at at.

A positive result means other is ahead of self.

Timezone.UTC.distance_to(Timezone.CET, datetime(2024, 7, 1)) datetime.timedelta(seconds=7200)

now

now() -> _dt.datetime

Return the current wall-clock time in this timezone.

The returned datetime is timezone-aware.

today

today() -> _dt.date

Return today's date in this timezone.

localize

localize(naive: datetime) -> _dt.datetime

Stamp a naive datetime with this timezone (no conversion).

Raises:

Type Description
ValueError

If naive is already timezone-aware, or if this is :attr:Timezone.NAIVE.

convert

convert(aware: datetime) -> _dt.datetime

Convert a timezone-aware datetime to this timezone.

Raises:

Type Description
ValueError

If aware is naive, or if this is :attr:Timezone.NAIVE.

midnight

midnight(date: date | None = None) -> _dt.datetime

Return midnight (00:00) in this timezone for date (default: today).

from_polars_type classmethod

from_polars_type(dtype: 'polars.DataType') -> 'Timezone | None'

Extract the timezone from a Polars Datetime dtype.

Returns None if the dtype is not Datetime or has no timezone.

Timezone.from_polars_type(pl.Datetime("us", "Europe/Paris")) Timezone('Europe/Paris')

polars_normalize classmethod

polars_normalize(
    col: "polars.Series | polars.Expr",
    *,
    lazy: bool = True,
    return_value: Literal["iana"] = "iana"
) -> "polars.Series | polars.Expr"

Normalize timezone strings in a Polars column using alias replacement only.

from_arrow_type classmethod

from_arrow_type(dtype: Any) -> 'Timezone | None'

Extract the timezone from a PyArrow TimestampType.

Returns None if the type has no tz attribute or it is empty.

import pyarrow as pa Timezone.from_arrow_type(pa.timestamp("us", tz="Europe/Paris")) Timezone('Europe/Paris')

arrow_timestamp_type

arrow_timestamp_type(unit: str = 'us') -> Any

Return a pa.timestamp(unit, tz=self.iana) type.

Timezone.CET.arrow_timestamp_type("ns") TimestampType(timestamp[ns, tz=Europe/Paris])

all_iana classmethod

all_iana() -> frozenset[str]

Return all IANA timezone identifiers available on this system.

Mode

Bases: IntEnum

is_read_only property

is_read_only: bool

True for read-only modes — i.e. no write disposition.

allows_write property

allows_write: bool

True when the mode admits any write disposition.

readable property

readable: bool

True when the mode admits reads.

Every :class:Mode canonically resolves to a + POSIX form (rb, rb+, wb+, ab+, xb+) — all of those admit reads. Only the strict :data:READ_ONLY rb and :data:IGNORE (which is no-op) deny writes; nothing here denies reads.

writable property

writable: bool

True when the mode admits writes — alias of :attr:allows_write.

appendable property

appendable: bool

True when writes append at EOF rather than at the cursor.

Only :data:APPEND carries POSIX O_APPEND semantics; every other write mode positions writes at the explicit cursor.

os_mode property

os_mode: str

Stdlib :func:open mode string for this :class:Mode.

  • :attr:READ_ONLY"rb"
  • :attr:OVERWRITE / :attr:TRUNCATE"wb+"
  • :attr:APPEND"ab+"
  • :attr:ERROR_IF_EXISTS"xb+"
  • everything else (AUTO, IGNORE, UPSERT, MERGE) → "rb+" (in-place edit; the disposition is enforced higher up).

from_ classmethod

from_(value: Optional[ModeLike], default: Optional[Mode] = None) -> Mode

Normalize value into a :class:Mode.

Accepts:

  • :class:Mode (returned as-is, idempotent).
  • Aliases like "overwrite", "OVERWRITE", "error-if-exists", "replace", "add" — see :data:STR_MAPPING.
  • POSIX-style mode strings — "rb", "wb", "ab+", "x", "r+b" — parsed structurally; any combination of one primary character (r/w/a/x) plus optional b/t/+ flags is accepted.
  • None → returns default if supplied, else :data:Mode.AUTO.

Falls back to :class:ValueError for unrecognized strings. Numeric / non-string non-Mode inputs raise :class:TypeError — the input grammar is "string or enum," not "anything stringifiable."

MimeType dataclass

MimeType(
    name: str,
    value: str,
    extensions: tuple[str, ...] = (),
    magics: tuple[MagicMatcher, ...] = (),
    is_codec: bool = False,
    is_tabular: bool = False,
    is_blob: bool = False,
)

Dataclass MIME descriptor + registries.

  • extensions: dotless, lower-case keys
  • magics: ordered matchers
  • is_codec: compression / wrapper formats
  • is_tabular: row/tabular-ish formats (read into a frame)
  • is_blob: opaque single-file payload — straight byte IO, no row structure (images, pdf, archives, pickle, …). Mutually exclusive with is_tabular; codecs and directory/connector mimes are neither.

get classmethod

get(value: object) -> 'MimeType | None'

Pure lookup — never raises. Returns None on miss.

Kept as the low-level "resolve by name/value" entry point: :meth:from_ / :meth:from_str / :meth:from_magic layer the default-handling contract on top.

parse_many classmethod

parse_many(obj: Any) -> list['MimeType']

Resolve obj to a flat, deduped list of :class:MimeType.

Accepts anything :meth:from_ accepts, plus:

  • iterables (list, tuple, set, generator) of any supported input
  • Accept-header strings: "application/json, text/csv;q=0.8"
  • composite format+codec strings: "application/csv+gzip", "parquet+zstd", "trades.parquet.zst"
  • wildcard strings: "*/*", "text/*", "image/*"
  • None[]

Order: first-seen wins, deduped by identity. For composites, the base format is emitted first, then the codec (codec wraps format).

Never raises on an unresolvable element — unknowns are dropped silently. For strict per-element resolution, call :meth:from_.

from_magic classmethod

from_magic(
    magic: Union[bytes, bytearray, memoryview, IO[bytes], str, Path],
    default: "MimeType | None" = _RAISE,
) -> "MimeType | None"

Resolve by sniffing magic bytes from magic.

Accepts raw bytes/memoryview, an IO, or anything the buffer class can wrap. Reads the first 64 bytes and walks the registered magic matchers in definition order.

:param default: see :class:MimeType class docstring for the shared default-handling contract. :raises ValueError: on a miss when default was not supplied.

from_str classmethod

from_str(value: str, default: 'MimeType | None' = _RAISE) -> 'MimeType | None'

Resolve a :class:str — path-like, bare extension, or mime value.

Tries, in order:

  • Direct lookup against the lower-cased input (covers "text/csv" / "json" / ".csv" without paying a :class:pathlib.Path allocation).
  • If the string looks path-like (contains / or \), take its suffix as an extension key.
  • Fall back to :meth:get (name / mime-value lookup with the application/ / text/ / … prefix stripping).
  • Structural sniff on leading { / [.

:param default: see :class:MimeType class docstring for the shared default-handling contract. :raises ValueError: on a miss when default was not supplied.

MimeTypes

Singleton MIME definitions.

MediaType dataclass

MediaType(mime_type: MimeType, codec: Codec | None = None)

from_many classmethod

from_many(
    mime_types: Iterable[MimeType], default: "MediaType" = ...
) -> "MediaType"

Compose a MediaType from an ordered mime list (e.g. URL extensions).

Two conventions land here:

  • trades.csv.zst["csv", "zst"]: the codec is the outer wrapper (you have to unzst before you can parse csv) → MediaType(CSV, codec=ZSTD).
  • part-xxx.zstd.parquet["zstd", "parquet"]: the format is the outer wrapper and the codec is the parquet page-codec hint baked into the file. Parquet handles the decompression internally → MediaType(PARQUET) with no outer codec; setting one would route the read through a decompressor that doesn't belong on this byte stream.

Order matters: the last mime decides. Last-is-codec promotes the codec to the wrapper slot; last-is-format keeps the format alone and drops earlier codec hints.

from_magic classmethod

from_magic(
    buf: Union[bytes, bytearray, memoryview, IO[bytes]],
    *,
    default: "MediaType" = ...
) -> "MediaType"

Sniff a :class:MediaType from raw bytes.

Two-stage: outer magic identifies the wrapper; if the wrapper is a codec, decompress a head-window and sniff the inner format. See module docstring for the full story.

Accepts bytes / bytearray / memoryview directly. For convenience, also accepts an IO[bytes] — in that case it delegates to :meth:from_io, which manages cursor save/restore.

:param default: Fallback when the outer sniff finds nothing. ... (default) raises :class:ValueError; any other value (including None) is returned as-is.

from_io classmethod

from_io(io_obj: IO[bytes], *, default: 'MediaType' = ...) -> 'MediaType'

Sniff a :class:MediaType from a file-like, seeking freely.

Captures the cursor on entry and restores it on exit, so the caller's stream position is unaffected. Inside the call we seek freely:

  1. seek(0) and read the first :data:_OUTER_PEEK bytes for outer-magic resolution.
  2. If the outer mime is a codec, seek(0) again and drive the codec's streaming decompressor directly to read the first :data:_INNER_PEEK bytes of decoded payload.
  3. Sniff the decoded head for the inner mime.

The codec resolution is one dict lookup (:meth:Codec.from_mime) — the outer magic loop already landed on the registered :class:MimeType singleton, so we reuse it as the key without re-resolving. For streaming codecs (gzip / zstd / lz4 / bz2 / xz / zlib / lzma) the decompressor is opened directly on the IO via :meth:Codec._open_decompress_reader — no IO wrap, no full-buffer materialization. Non-streaming codecs (snappy, brotli) fall back to reading the compressed body and calling :meth:Codec.decompress_bytes.

Decompression errors during the inner sniff are swallowed — the worst case is returning a less-specific MediaType (the outer codec is preserved). A separate caller that wants to validate decompressibility should call :meth:Codec.decompress directly.

:param io_obj: Any file-like with read + seek (stdlib io.BytesIO, an open file, our own IO, etc.). :param default: Fallback when the outer sniff finds nothing. ... (default) raises :class:ValueError; any other value (including None) is returned as-is.

Codec

Bases: ABC

Abstract compression codec.

is_streaming property

is_streaming: bool

True when both compress and decompress have streaming paths.

Callers with large (GiB-scale) inputs should check this before passing them to :meth:compress / :meth:decompress. When False, those methods fall back to materializing the full payload in memory through :meth:compress_bytes / :meth:decompress_bytes.

compress

compress(src: 'IO') -> 'IO'

Compress src into a new :class:IO.

Streams chunk-by-chunk when :meth:_open_compress_writer is available. Otherwise falls back to a full-in-memory bytes roundtrip — callers with multi-GiB inputs should inspect :attr:is_streaming first.

The source cursor is restored on exit.

decompress

decompress(src: 'IO', copy: bool = True) -> 'IO'

Decompress src into a new :class:IO.

Streams chunk-by-chunk when :meth:_open_decompress_reader is available. Otherwise falls back to a full-in-memory bytes roundtrip — callers with multi-GiB compressed inputs should inspect :attr:is_streaming first.

The source cursor is restored on exit.

read_start_end

read_start_end(
    src: "IO | bytes | bytearray | memoryview",
    *,
    n_start: int = 64,
    n_end: int = 64,
    chunk_size: int = _CHUNK
) -> tuple[bytes, bytes]

Return the first n_start and last n_end bytes of the decoded stream.

Streams the decompression and keeps only a bounded amount of state in memory (:math:n\_end + chunk\_size bytes) — safe for very large compressed inputs when the codec supports streaming decompression.

When the codec does NOT expose a streaming decoder, falls back to a full :meth:decompress_bytes call, which materializes the whole uncompressed payload in memory.

from_ classmethod

from_(obj: Any, default: 'Codec | None' = ...) -> 'Codec | None'

Parse an arbitrary input into a Codec instance.

Accepts: - :class:Codec instances (returned as-is). - Short names like "gzip", "zstd" (case-insensitive). - Anything :meth:MimeType.parse can resolve to a codec mime. - None → returns default.

Codecs

Singleton class for accessing all registered codecs.

ByteUnit

Bases: IntEnum

Canonical IEC binary byte-unit token + scalar value.

Each member's value IS the byte count for one unit, so 128 * ByteUnit.MIB reads naturally at the call site and slots into any int field. Use :meth:parse_size when accepting external config / API input — it canonicalizes "128 MB" / "1.5 GiB" / raw integers / ByteUnit members all to a plain integer byte count.

bytes property

bytes: int

Bytes per one of this unit (== member value).

short property

short: str

Short colloquial token: "B" / "KB" / "MB" / ….

iec property

iec: str

Strict IEC token: "B" / "KiB" / "MiB" / ….

from_ classmethod

from_(value: Any, *, default: Any = ...) -> 'ByteUnit'

Coerce any Python value into a :class:ByteUnit member.

Accepts:

  • :class:ByteUnit (returned as-is);
  • any string the alias table knows — B / KB / MiB / gigabyte / mixed case / trailing s;
  • an integer matching a member's byte value (1024 → :attr:KIB);
  • None — returns default if supplied, else raises.

default swallows unknown / unparseable input. Without it, unknown tokens raise :class:ValueError and unsupported types raise :class:TypeError.

is_valid classmethod

is_valid(value: Any) -> bool

True when :meth:from_ would succeed for value.

parse_size classmethod

parse_size(
    value: Union[int, str, "ByteUnit", None], *, default: Any = ...
) -> int

Coerce a size-like value to an integer byte count.

The single entry point for "config gave me a size, give me bytes." Accepts:

  • :class:int — passed through (must be non-negative);
  • :class:ByteUnit — its scalar value;
  • a quantity string "128 MB" / "1.5 GiB" / "512" — parsed with this enum's IEC conventions;
  • a bare unit string "MiB" — yields one unit (1024**2);
  • None — returns default if supplied, else raises.

Floating-point quantities round to the nearest byte ("1.5 KiB"1536). Negative values raise :class:ValueError.

format classmethod

format(n: int, *, iec: bool = True, precision: int = 1) -> str

Format an integer byte count as a human-readable string.

Picks the largest unit at which n divides cleanly into a scalar ≥ 1, defaulting to IEC tokens ("128 MiB"); pass iec=False for the colloquial short form ("128 MB"). precision controls fractional digits.

pretty classmethod

pretty(
    v: float,
    unit: "ByteUnit | str | None" = None,
    *,
    iec: bool = True,
    precision: int = 1
) -> str

Human-readable rendering of a quantity v expressed in unit.

v is a scalar count of unit (bytes by default); it's scaled to a byte count and handed to :meth:format. The companion to :meth:format for the common "I have N MiB, show it nicely" case::

ByteUnit.pretty(1536)             # "1.5 KiB"
ByteUnit.pretty(8, ByteUnit.MIB)  # "8.0 MiB"
ByteUnit.pretty(1.5, "gb")        # "1.5 GiB"

unit defaults to :attr:B (resolved at call time so the class default is usable inside the method body); pass iec=False for the colloquial short form.

Unit

Unit(symbol: str, factor: float, offset: float = 0.0)

Mixin providing :meth:from_ / :meth:convert / scalar helpers.

Subclasses combine this with :class:Enum and declare members as (symbol, factor_to_canonical, offset_to_canonical) tuples (or (symbol, factor) — offset defaults to 0.0). The canonical member of each family has factor=1.0 and offset=0.0; conversion goes through that canonical pivot.

Extra spellings ("megawatthours"MWH) live in a <Family>._ALIASES dict assigned outside the enum body — the enum metaclass would treat a class-body dict as a member.

from_ classmethod

from_(value: Any, *, default: Any = ...) -> 'Unit'

Coerce value to a member of this unit family.

Accepts: a member (passed through), a known symbol ("MWh" / "°C" / "psi"), the enum member name ("KWH"), or any alias the per-family _ALIASES dict knows ("megawatthours" / "celsius" / "pound"). Lookup is case-insensitive.

Pass default to swallow unknown / unparseable input; without it, unknown strings raise :class:ValueError with a hint listing the valid symbols, and non-string non-member input raises :class:TypeError.

is_valid classmethod

is_valid(value: Any) -> bool

True when :meth:from_ would resolve value.

convert classmethod

convert(value: float, source: Any, target: Any) -> float

Convert value from source unit to target unit (scalar).

to_canonical

to_canonical(value: float) -> float

Convert value in self's unit to the family's canonical unit.

from_canonical

from_canonical(value: float) -> float

Convert value in the canonical unit to self's unit.

canonical classmethod

canonical() -> 'Unit'

The family's canonical member (factor=1.0, offset=0.0).

factor_map classmethod

factor_map() -> dict[str, float]

{symbol: factor} for every member + alias, for per-row maps.

offset_map classmethod

offset_map() -> dict[str, float]

{symbol: offset} for every member + alias, for per-row maps.

EnergyUnit

EnergyUnit(symbol: str, factor: float, offset: float = 0.0)

Bases: Unit, Enum

Energy unit (canonical = joule).

Members cover SI scale (J / kJ / MJ / GJ / TJ), the watt-hour family that dominates electricity-market ingestion (Wh / kWh / MWh / GWh / TWh), and the imperial / domestic units (BTU / cal / kcal / therm).

from_ classmethod

from_(value: Any, *, default: Any = ...) -> 'Unit'

Coerce value to a member of this unit family.

Accepts: a member (passed through), a known symbol ("MWh" / "°C" / "psi"), the enum member name ("KWH"), or any alias the per-family _ALIASES dict knows ("megawatthours" / "celsius" / "pound"). Lookup is case-insensitive.

Pass default to swallow unknown / unparseable input; without it, unknown strings raise :class:ValueError with a hint listing the valid symbols, and non-string non-member input raises :class:TypeError.

is_valid classmethod

is_valid(value: Any) -> bool

True when :meth:from_ would resolve value.

convert classmethod

convert(value: float, source: Any, target: Any) -> float

Convert value from source unit to target unit (scalar).

to_canonical

to_canonical(value: float) -> float

Convert value in self's unit to the family's canonical unit.

from_canonical

from_canonical(value: float) -> float

Convert value in the canonical unit to self's unit.

canonical classmethod

canonical() -> 'Unit'

The family's canonical member (factor=1.0, offset=0.0).

factor_map classmethod

factor_map() -> dict[str, float]

{symbol: factor} for every member + alias, for per-row maps.

offset_map classmethod

offset_map() -> dict[str, float]

{symbol: offset} for every member + alias, for per-row maps.

PowerUnit

PowerUnit(symbol: str, factor: float, offset: float = 0.0)

Bases: Unit, Enum

Power unit (canonical = watt).

SI scale (W / kW / MW / GW / TW) plus mechanical horsepower. Energy-market feeds publish capacity in MW, transmission limits in GW, household-scale appliances in W — having one enum cover all of them is what keeps schema-per-source curated views from re-stringing the unit token at every read.

from_ classmethod

from_(value: Any, *, default: Any = ...) -> 'Unit'

Coerce value to a member of this unit family.

Accepts: a member (passed through), a known symbol ("MWh" / "°C" / "psi"), the enum member name ("KWH"), or any alias the per-family _ALIASES dict knows ("megawatthours" / "celsius" / "pound"). Lookup is case-insensitive.

Pass default to swallow unknown / unparseable input; without it, unknown strings raise :class:ValueError with a hint listing the valid symbols, and non-string non-member input raises :class:TypeError.

is_valid classmethod

is_valid(value: Any) -> bool

True when :meth:from_ would resolve value.

convert classmethod

convert(value: float, source: Any, target: Any) -> float

Convert value from source unit to target unit (scalar).

to_canonical

to_canonical(value: float) -> float

Convert value in self's unit to the family's canonical unit.

from_canonical

from_canonical(value: float) -> float

Convert value in the canonical unit to self's unit.

canonical classmethod

canonical() -> 'Unit'

The family's canonical member (factor=1.0, offset=0.0).

factor_map classmethod

factor_map() -> dict[str, float]

{symbol: factor} for every member + alias, for per-row maps.

offset_map classmethod

offset_map() -> dict[str, float]

{symbol: offset} for every member + alias, for per-row maps.

MassUnit

MassUnit(symbol: str, factor: float, offset: float = 0.0)

Bases: Unit, Enum

Mass unit (canonical = kilogram).

SI scale (mg / g / kg / t / Mt) plus the imperial pair (lb / oz) commonly used in commodity feeds.

from_ classmethod

from_(value: Any, *, default: Any = ...) -> 'Unit'

Coerce value to a member of this unit family.

Accepts: a member (passed through), a known symbol ("MWh" / "°C" / "psi"), the enum member name ("KWH"), or any alias the per-family _ALIASES dict knows ("megawatthours" / "celsius" / "pound"). Lookup is case-insensitive.

Pass default to swallow unknown / unparseable input; without it, unknown strings raise :class:ValueError with a hint listing the valid symbols, and non-string non-member input raises :class:TypeError.

is_valid classmethod

is_valid(value: Any) -> bool

True when :meth:from_ would resolve value.

convert classmethod

convert(value: float, source: Any, target: Any) -> float

Convert value from source unit to target unit (scalar).

to_canonical

to_canonical(value: float) -> float

Convert value in self's unit to the family's canonical unit.

from_canonical

from_canonical(value: float) -> float

Convert value in the canonical unit to self's unit.

canonical classmethod

canonical() -> 'Unit'

The family's canonical member (factor=1.0, offset=0.0).

factor_map classmethod

factor_map() -> dict[str, float]

{symbol: factor} for every member + alias, for per-row maps.

offset_map classmethod

offset_map() -> dict[str, float]

{symbol: offset} for every member + alias, for per-row maps.

LengthUnit

LengthUnit(symbol: str, factor: float, offset: float = 0.0)

Bases: Unit, Enum

Length unit (canonical = metre).

SI scale (mm / cm / m / km) plus the common imperial / aviation / maritime tokens (in / ft / yd / mi / nmi).

from_ classmethod

from_(value: Any, *, default: Any = ...) -> 'Unit'

Coerce value to a member of this unit family.

Accepts: a member (passed through), a known symbol ("MWh" / "°C" / "psi"), the enum member name ("KWH"), or any alias the per-family _ALIASES dict knows ("megawatthours" / "celsius" / "pound"). Lookup is case-insensitive.

Pass default to swallow unknown / unparseable input; without it, unknown strings raise :class:ValueError with a hint listing the valid symbols, and non-string non-member input raises :class:TypeError.

is_valid classmethod

is_valid(value: Any) -> bool

True when :meth:from_ would resolve value.

convert classmethod

convert(value: float, source: Any, target: Any) -> float

Convert value from source unit to target unit (scalar).

to_canonical

to_canonical(value: float) -> float

Convert value in self's unit to the family's canonical unit.

from_canonical

from_canonical(value: float) -> float

Convert value in the canonical unit to self's unit.

canonical classmethod

canonical() -> 'Unit'

The family's canonical member (factor=1.0, offset=0.0).

factor_map classmethod

factor_map() -> dict[str, float]

{symbol: factor} for every member + alias, for per-row maps.

offset_map classmethod

offset_map() -> dict[str, float]

{symbol: offset} for every member + alias, for per-row maps.

VolumeUnit

VolumeUnit(symbol: str, factor: float, offset: float = 0.0)

Bases: Unit, Enum

Volume unit (canonical = cubic metre).

SI scale (mL / L / m³) plus the gallon variants and the oil barrel — commodity feeds split between US gallons, UK gallons, and the 42-gallon oil barrel, and a curated view that doesn't keep them straight ships wrong totals.

from_ classmethod

from_(value: Any, *, default: Any = ...) -> 'Unit'

Coerce value to a member of this unit family.

Accepts: a member (passed through), a known symbol ("MWh" / "°C" / "psi"), the enum member name ("KWH"), or any alias the per-family _ALIASES dict knows ("megawatthours" / "celsius" / "pound"). Lookup is case-insensitive.

Pass default to swallow unknown / unparseable input; without it, unknown strings raise :class:ValueError with a hint listing the valid symbols, and non-string non-member input raises :class:TypeError.

is_valid classmethod

is_valid(value: Any) -> bool

True when :meth:from_ would resolve value.

convert classmethod

convert(value: float, source: Any, target: Any) -> float

Convert value from source unit to target unit (scalar).

to_canonical

to_canonical(value: float) -> float

Convert value in self's unit to the family's canonical unit.

from_canonical

from_canonical(value: float) -> float

Convert value in the canonical unit to self's unit.

canonical classmethod

canonical() -> 'Unit'

The family's canonical member (factor=1.0, offset=0.0).

factor_map classmethod

factor_map() -> dict[str, float]

{symbol: factor} for every member + alias, for per-row maps.

offset_map classmethod

offset_map() -> dict[str, float]

{symbol: offset} for every member + alias, for per-row maps.

TemperatureUnit

TemperatureUnit(symbol: str, factor: float, offset: float = 0.0)

Bases: Unit, Enum

Temperature unit (canonical = kelvin).

The one affine family — Celsius and Fahrenheit have non-zero offsets relative to kelvin, so the conversion is canonical = value * factor + offset (not pure scaling). Symbols use the standard degree glyphs (°C / °F) so the curated tables render legibly in BI tools; aliases cover the plain-ascii forms ("C" / "F" / "K").

from_ classmethod

from_(value: Any, *, default: Any = ...) -> 'Unit'

Coerce value to a member of this unit family.

Accepts: a member (passed through), a known symbol ("MWh" / "°C" / "psi"), the enum member name ("KWH"), or any alias the per-family _ALIASES dict knows ("megawatthours" / "celsius" / "pound"). Lookup is case-insensitive.

Pass default to swallow unknown / unparseable input; without it, unknown strings raise :class:ValueError with a hint listing the valid symbols, and non-string non-member input raises :class:TypeError.

is_valid classmethod

is_valid(value: Any) -> bool

True when :meth:from_ would resolve value.

convert classmethod

convert(value: float, source: Any, target: Any) -> float

Convert value from source unit to target unit (scalar).

to_canonical

to_canonical(value: float) -> float

Convert value in self's unit to the family's canonical unit.

from_canonical

from_canonical(value: float) -> float

Convert value in the canonical unit to self's unit.

canonical classmethod

canonical() -> 'Unit'

The family's canonical member (factor=1.0, offset=0.0).

factor_map classmethod

factor_map() -> dict[str, float]

{symbol: factor} for every member + alias, for per-row maps.

offset_map classmethod

offset_map() -> dict[str, float]

{symbol: offset} for every member + alias, for per-row maps.

PressureUnit

PressureUnit(symbol: str, factor: float, offset: float = 0.0)

Bases: Unit, Enum

Pressure unit (canonical = pascal).

SI scale (Pa / kPa / MPa / hPa) plus the bar family (bar / mbar), atmosphere, psi, and torr. Weather feeds publish in hPa or mbar, oil & gas in psi or bar, process control in kPa or MPa — one enum keeps them aligned.

from_ classmethod

from_(value: Any, *, default: Any = ...) -> 'Unit'

Coerce value to a member of this unit family.

Accepts: a member (passed through), a known symbol ("MWh" / "°C" / "psi"), the enum member name ("KWH"), or any alias the per-family _ALIASES dict knows ("megawatthours" / "celsius" / "pound"). Lookup is case-insensitive.

Pass default to swallow unknown / unparseable input; without it, unknown strings raise :class:ValueError with a hint listing the valid symbols, and non-string non-member input raises :class:TypeError.

is_valid classmethod

is_valid(value: Any) -> bool

True when :meth:from_ would resolve value.

convert classmethod

convert(value: float, source: Any, target: Any) -> float

Convert value from source unit to target unit (scalar).

to_canonical

to_canonical(value: float) -> float

Convert value in self's unit to the family's canonical unit.

from_canonical

from_canonical(value: float) -> float

Convert value in the canonical unit to self's unit.

canonical classmethod

canonical() -> 'Unit'

The family's canonical member (factor=1.0, offset=0.0).

factor_map classmethod

factor_map() -> dict[str, float]

{symbol: factor} for every member + alias, for per-row maps.

offset_map classmethod

offset_map() -> dict[str, float]

{symbol: offset} for every member + alias, for per-row maps.

JoinType

Bases: IntEnum

Canonical join kind for tabular join surfaces.

Use :meth:from_ when accepting external input — it canonicalizes aliases ("left", "LEFT JOIN", "anti", "outer", integer codes) to a member and raises :class:ValueError for unknown tokens.

Pass :attr:arrow / :attr:polars / :attr:sql to engine join APIs — pyarrow's :meth:pa.Table.join and polars' :meth:DataFrame.join accept different spellings of the same concept, and storing the integer code keeps the enum a clean discriminator without binding to either spelling.

arrow property

arrow: str

pyarrow's :meth:pa.Table.join join_type token.

polars property

polars: str

polars' :meth:DataFrame.join how token.

Polars has no built-in right semi / right anti form — those raise :class:NotImplementedError. Swap operands and use the left-side equivalent.

sql property

sql: str

SQL JOIN clause keyword (uppercase, with trailing JOIN).

is_inner property

is_inner: bool

True for :attr:INNER.

is_outer property

is_outer: bool

True for any of the three outer joins.

is_semi property

is_semi: bool

True for :attr:LEFT_SEMI / :attr:RIGHT_SEMI.

is_anti property

is_anti: bool

True for :attr:LEFT_ANTI / :attr:RIGHT_ANTI.

is_cross property

is_cross: bool

True for :attr:CROSS.

from_ classmethod

from_(value: Any, *, default: Any = ...) -> 'JoinType'

Coerce any Python value into a :class:JoinType.

Accepts:

  • :class:JoinType (returned as-is);
  • any string the alias table or canonical Arrow tokens know — "inner", "left", "left outer", "LEFT JOIN", "anti", "left_anti", "outer", "cross", …; mixed case, hyphens / underscores / spaces all normalize;
  • an integer code matching a member's value (round-trips with int(JoinType.X));
  • None — returns default if supplied, else raises.

default swallows unknown / unparseable input. Without it, unknown tokens raise :class:ValueError and unsupported types raise :class:TypeError.

is_valid classmethod

is_valid(value: Any) -> bool

Return True when :meth:from_ would succeed for value.

Scheme

Bases: str, Enum

Canonical URL-scheme token for a Yggdrasil :class:URLBased subclass.

Subclasses :class:str so a member is interchangeable with its scheme token everywhere a string is expected (url.scheme == Scheme.DBFS works, f"{Scheme.S3}://bucket/key" reads naturally). The lazy-import resolver lives on :meth:path_class.

from_ classmethod

from_(value: Any, *, default: Any = ...) -> 'Scheme'

Coerce value to a :class:Scheme member.

Accepts:

  • :class:Scheme (returned as-is);
  • a scheme string — case-insensitive, trailing :// and whitespace tolerated, common aliases ("s3a" → :attr:S3, "local" → :attr:FILE, "memory" → :attr:MEMORY);
  • None — returns default if supplied, else raises.

default swallows unknown / unparseable input. Without it, unknown tokens raise :class:ValueError and unsupported types raise :class:TypeError.

is_valid classmethod

is_valid(value: Any) -> bool

True when :meth:from_ would succeed for value.

path_class

path_class() -> 'type[URLBased]'

Return the concrete :class:URLBased subclass for this scheme.

Lazy: the backend module is imported on first use, which fires the :meth:URLBased.__init_subclass__ side-effect that registers the class. Subsequent calls hit the per-process cache.

Raises :class:ImportError when the backend's optional dependencies aren't installed (databricks-sdk for the Databricks schemes, boto3 for s3, …) — the message names the missing extra so the caller can install it.

resolve classmethod

resolve(value: Any, *, default: Any = ...) -> 'type[URLBased]'

Shortcut: cls.from_(value).path_class().

Useful where the caller has a raw scheme string (or URL) and just wants the concrete URLBased class. default is forwarded to :meth:from_ for forgiving lookup; it does not catch :class:ImportError from :meth:path_class.

NodeType

Bases: str, Enum

Canonical Databricks node_type_id values.

Member values are the exact strings the Databricks SDK expects. Use :attr:NodeType.DEFAULT for the codebase-wide default node type, the semantic-size aliases (:attr:SMALL / :attr:MEDIUM / :attr:LARGE / :attr:XLARGE) for intent-driven sizing, or the explicit cloud-specific members when you need a precise SKU.

The enum is intentionally not exhaustive — Databricks publishes hundreds of SKUs per cloud and most callers want a small, opinionated set. Coerce caller-provided strings through :meth:to_id / :meth:from_ so unknown SKUs round-trip as plain strings instead of raising.

from_ classmethod

from_(value: Any, *, default: Any = ...) -> 'NodeType'

Coerce value into a :class:NodeType member.

Accepts:

  • :class:NodeType (returned as-is);
  • any string in :data:_NODETYPE_ALIASES, case-insensitive;
  • a bare member name ("FLEET_XLARGE");
  • None — returns default if supplied, else :attr:NodeType.DEFAULT.

Raises :class:ValueError for unknown strings unless default is supplied. To accept arbitrary SKU strings (without rejecting valid cloud-specific identifiers), call :meth:to_id instead.

to_id classmethod

to_id(
    value: Union[str, "NodeType", None],
    *,
    default: Optional[Union[str, "NodeType"]] = None
) -> str

Coerce value to a Databricks node_type_id string.

Unlike :meth:from_, this is forgiving: arbitrary SKU strings round-trip unchanged so callers can pass cloud-specific identifiers the enum does not enumerate ("r5d.metal", custom marketplace images, …). Aliases are resolved when present.

Resolution order:

  1. NodeType member → its string value.
  2. Alias / member name / value match → the canonical string.
  3. Bare string → trimmed and returned as-is.
  4. Nonedefault if supplied, else :attr:NodeType.DEFAULT.

is_known classmethod

is_known(value: Any) -> bool

True when :meth:from_ would succeed for value.

NodeSpec dataclass

NodeSpec(
    cpu_cores: int,
    ram_gib: float,
    gpu_count: int = 0,
    local_disk_gib: float = 0.0,
    cloud: str = "",
)

Hardware specs for a :class:NodeType member.

The numbers reflect the vendor's published per-VM characteristics for a single worker (or driver) — not the cluster-wide totals. ram_gib is binary GiB (1024 ** 3 bytes), matching every cloud's own documentation convention.

Attributes

cpu_cores Virtual CPU count exposed to Spark. ram_gib Memory available to Spark, in IEC GiB. gpu_count Number of GPUs (0 for non-GPU SKUs). local_disk_gib Local SSD/NVMe storage attached to the instance. 0 when the SKU has no local disk (relies on remote/elastic storage). cloud Cloud family identifier ("fleet" / "aws" / "azure" / "gcp") used by :meth:NodeType.from_cpu_and_ram when prefer is set.

State

Bases: IntEnum

Unified execution state for async statement / job results.

Use :meth:from_ to normalize backend-specific tokens, and the is_* predicates to derive done / failed / started without re-implementing the membership sets per backend.

is_idle property

is_idle: bool

True for :attr:IDLE — built locally, not submitted yet.

is_queued property

is_queued: bool

True for :attr:QUEUED — prepared, sitting in a submission queue.

is_pending property

is_pending: bool

True for :attr:PENDING — handed to a backend, awaiting ack.

is_accepted property

is_accepted: bool

True for :attr:ACCEPTED — acknowledged but not yet running.

is_running property

is_running: bool

True for :attr:RUNNING — actively executing.

is_partial property

is_partial: bool

True for :attr:PARTIAL — partially filled, still active.

is_started property

is_started: bool

True for anything from :attr:RUNNING onward.

Mirrors :attr:StatementResult.started: once the backend has actually started executing, is_started flips and stays True through every terminal state. :attr:ACCEPTED is not started — the venue holds the order, no execution yet.

is_active property

is_active: bool

True for non-terminal states with backend awareness.

Covers :attr:QUEUED through :attr:PARTIAL — anything the caller can reasonably wait on. :attr:IDLE is excluded (nothing has been submitted yet) and every terminal state is excluded (no more transitions).

is_done property

is_done: bool

True for terminal states (:attr:SUCCEEDED, :attr:REJECTED, :attr:FAILED, :attr:CANCELED, :attr:EXPIRED) — no more transitions expected.

is_failed property

is_failed: bool

True for :attr:REJECTED / :attr:FAILED / :attr:CANCELED / :attr:EXPIRED.

Every non-success terminal state counts as failed because each one means "the caller asked for a result and didn't get one": cancellation, rejection, mid-run error, or TTL expiry all leave the operation incomplete from the caller's view, and the per-backend raise_for_status raises on each.

is_succeeded property

is_succeeded: bool

True for :attr:SUCCEEDED — terminal with a full result.

is_rejected property

is_rejected: bool

True for :attr:REJECTED — refused at submission / accept.

is_canceled property

is_canceled: bool

True for :attr:CANCELED.

is_expired property

is_expired: bool

True for :attr:EXPIRED — TTL / day-rollover terminal.

from_ classmethod

from_(value: Any, *, default: Any = ...) -> 'State'

Coerce any Python value into a :class:State.

See module docstring for the accepted shapes. default swallows unknown / unparseable input; without it, unknown tokens raise :class:ValueError and unsupported types raise :class:TypeError.

is_valid classmethod

is_valid(value: Any) -> bool

Return True when :meth:from_ would succeed for value.

IOKind

Bases: IntEnum

What a backend reports a path/holder entry is.

EngineType

Bases: IntEnum

Compute engine for a Databricks :class:~yggdrasil.databricks.table.table.Table read / write.

  • :attr:YGGDRASIL — yggdrasil's native DeltaFolder (direct _delta_log
  • parquet over UC-vended credentials), no warehouse.
  • :attr:DATABRICKS_SQL_WAREHOUSE — the Databricks SQL warehouse.
  • :attr:SPARK — a Spark session (Databricks Connect / cluster / notebook).

from_str classmethod

from_str(value: Any, *, default: Any = ...) -> 'Optional[EngineType]'

Coerce an alias string into an :class:EngineType.

Matches a forgiving alias table ("warehouse" / "sql"DATABRICKS_SQL_WAREHOUSE; "ygg" / "native"YGGDRASIL; "spark" / "connect"SPARK). On an unknown string: raise :class:ValueError when default is ..., else return default.

from_numeric classmethod

from_numeric(value: Any, *, default: Any = ...) -> 'Optional[EngineType]'

Coerce an integer code into an :class:EngineType.

On an out-of-range / non-integer code: raise when default is ..., else return default. bool is rejected (it's an int subclass but never a meaningful engine code).

from_ classmethod

from_(value: Any, *, default: Any = ...) -> 'Optional[EngineType]'

Coerce an :class:EngineType / alias string / int code / None.

None (or ...) is "unset" and returns None. A string routes to :meth:from_str, a number to :meth:from_numeric; both honour default (... → raise, else return the fallback).

AWSRegion

Bases: str, Enum

partition property

partition: str

The AWS partition this region belongs to (aws / aws-us-gov / aws-cn).

from_ classmethod

from_(value: Any, *, default: Any = ...) -> 'Optional[AWSRegion]'

Coerce value (an :class:AWSRegion, a region string, or None) to a member. Unknown / None returns default if given, else raises.

from_text classmethod

from_text(text: Any, *, default: Any = None) -> 'Optional[AWSRegion]'

Find an AWS region code embedded in text (e.g. a bucket name like acme-dls3-eu-central-1-p), or default if none is present.

Matches a region only when it stands as a whole, delimiter-bounded token (so eu-central-12 / ...-1foo don't false-positive), and prefers the longest code at a position (ap-southeast-1 over ap-south-1).

unit_family_for

unit_family_for(value: Any) -> type[Unit]

Find the unit family that recognises value (member or symbol).

Useful when a curated row carries a free-form unit token and the caller needs to dispatch to the right family without hard-coding a per-source switch. Returns the first family for which :meth:Unit.is_valid returns True; raises :class:ValueError when no family claims the token (the message lists every family that was consulted).