yggdrasil.enums¶
enums ¶
TimeUnit ¶
Bases: str, Enum
Canonical time-unit token for temporal DataType instances.
Members carry the lowercase short form so the enum is a drop-in string replacement::
TimeType(unit=TimeUnit.MICROSECOND)
TimeType(unit="us") # equivalent — both store ``"us"``
Use :meth:parse when accepting external input — it canonicalizes
aliases ("microseconds", "micros", "µs") to a member
and raises :class:ValueError for unknown tokens.
seconds
property
¶
Seconds per one of this unit (used for scalar epoch math).
Calendar-style interval units (year_month / day_time /
month_day_nano) have no fixed second-count and return
float('nan') so comparisons surface the mismatch instead of
silently truncating to zero.
order
property
¶
Precision rank — higher = finer.
Used by TemporalType._merge_with_same_id to pick the wider
of two units. Calendar interval units sit at rank -1 so
they don't outrank fixed-precision ones in normal merges.
from_
classmethod
¶
Coerce any Python value into a :class:TimeUnit.
Accepts:
- :class:
TimeUnit(returned as-is); - any string the alias table or canonical member values know —
s/ms/us/ns/d/ interval forms, plurals (microseconds), long forms (millisecond), mixed case,µs, hyphens / spaces; - objects exposing a
time_unit/unitattribute (PolarsDatetime/Duration, PyArrowTimestampType/DurationType/Time32Type/Time64Type); the attribute is re-funneled throughfrom_; None— returns default if supplied, else raises.
default swallows unknown / unparseable input. Without it,
unknown tokens raise :class:ValueError and unsupported types
raise :class:TypeError.
is_valid
classmethod
¶
Return True when :meth:from_ would succeed for value.
Timezone
dataclass
¶
An immutable wrapper around an IANA timezone identifier.
Instances are created via :meth:from_ (accepts strings,
ZoneInfo, datetime.tzinfo, other Timezone objects, or
None → UTC) or directly::
tz = Timezone("Europe/Paris")
tz = Timezone.from_("CET") # → Timezone("Europe/Paris")
tz = Timezone.from_("+01:00") # → Timezone("Etc/GMT-1")
tz = Timezone.from_(ZoneInfo("Europe/Paris"))
The naive case is represented by :attr:Timezone.NAIVE — a
sentinel instance whose iana is the empty string and whose
__bool__ is False. Use it instead of None so a
tz: Timezone field type can stay non-optional.
utc_seconds_offset
property
¶
Fixed UTC offset in seconds, or None for non-fixed zones.
Returns 0 for UTC and its equivalents (Etc/UTC,
GMT, Etc/GMT+0, …), the parsed offset for
Etc/GMT±N (note IANA's sign-flip — Etc/GMT-3 means
UTC+3, so the property reports +10800 not -10800),
and None for any zone whose offset depends on DST or for
:attr:Timezone.NAIVE. Use :meth:utc_offset instead when
you need a DST-aware offset for a specific instant.
Naming follows the "type-suffix unit" convention so
utc_seconds_offset makes the unit explicit at the call
site — utc_offset returns a :class:datetime.timedelta,
utc_offset_hours returns fractional hours, and this
property returns whole seconds (or None).
from_
classmethod
¶
Coerce any Python value into a :class:Timezone.
Accepts:
- :class:
Timezone(returned as-is — includingTimezone.NAIVE); - :class:
zoneinfo.ZoneInfo(extractskey); - a string — IANA name, alias (
CET/EST/Z/GMT/Etc/UTC/+00:00/ …), or fixed offset ("+01:00","UTC-05","-0530"); datetime.tzinfoinstance (ZoneInfo,datetime.timezone, third-party zones) — thekey/zoneattribute is preferred; otherwise falls back to the fixed UTC offset;- timezone-aware
datetime/time(extractstzinfo); - objects exposing a
tz/time_zone/timezone/ianaattribute (PyArrowTimestampType, PolarsDatetime, foreign Timezone classes); the attribute is re-funneled throughfrom_; None— returns :attr:UTCfor backward compatibility unlessdefaultis supplied.
default swallows unknown / unparseable input. Without it,
bad input raises :class:ValueError (string parse failure) or
:class:TypeError (unsupported value type).
to_zoneinfo
cached
¶
Return the zoneinfo.ZoneInfo for this timezone.
Raises :class:ValueError for :attr:Timezone.NAIVE since
there is no concrete zone to materialize.
utc_offset ¶
Return the UTC offset at instant at (default: now).
The result accounts for DST transitions.
Timezone("Europe/Paris").utc_offset(datetime(2024, 7, 1)) datetime.timedelta(seconds=7200) # +02:00 (CEST)
utc_offset_hours ¶
Return the UTC offset in fractional hours.
Timezone("Asia/Kolkata").utc_offset_hours() 5.5
is_fixed_offset ¶
Return True if this timezone has a fixed UTC offset (no DST).
is_dst ¶
Return True if DST is active at instant at (default: now).
Timezone("Europe/Paris").is_dst(datetime(2024, 7, 1)) True Timezone("Europe/Paris").is_dst(datetime(2024, 1, 1)) False
dst_offset ¶
Return the DST adjustment at instant at.
Returns timedelta(0) when DST is not active or for fixed-offset
timezones, and for :attr:Timezone.NAIVE.
abbreviation ¶
Return the timezone abbreviation (e.g. "CET", "CEST") at at.
Timezone("Europe/Paris").abbreviation(datetime(2024, 7, 1)) 'CEST' Timezone("Europe/Paris").abbreviation(datetime(2024, 1, 1)) 'CET'
distance_to ¶
Return the offset difference between self and other at at.
A positive result means other is ahead of self.
Timezone.UTC.distance_to(Timezone.CET, datetime(2024, 7, 1)) datetime.timedelta(seconds=7200)
now ¶
Return the current wall-clock time in this timezone.
The returned datetime is timezone-aware.
localize ¶
Stamp a naive datetime with this timezone (no conversion).
Raises:
| Type | Description |
|---|---|
ValueError
|
If naive is already timezone-aware, or if
this is :attr: |
convert ¶
Convert a timezone-aware datetime to this timezone.
Raises:
| Type | Description |
|---|---|
ValueError
|
If aware is naive, or if this is
:attr: |
midnight ¶
Return midnight (00:00) in this timezone for date (default: today).
from_polars_type
classmethod
¶
Extract the timezone from a Polars Datetime dtype.
Returns None if the dtype is not Datetime or has no timezone.
Timezone.from_polars_type(pl.Datetime("us", "Europe/Paris")) Timezone('Europe/Paris')
polars_normalize
classmethod
¶
polars_normalize(
col: "polars.Series | polars.Expr",
*,
lazy: bool = True,
return_value: Literal["iana"] = "iana"
) -> "polars.Series | polars.Expr"
Normalize timezone strings in a Polars column using alias replacement only.
from_arrow_type
classmethod
¶
Extract the timezone from a PyArrow TimestampType.
Returns None if the type has no tz attribute or it is empty.
import pyarrow as pa Timezone.from_arrow_type(pa.timestamp("us", tz="Europe/Paris")) Timezone('Europe/Paris')
arrow_timestamp_type ¶
Return a pa.timestamp(unit, tz=self.iana) type.
Timezone.CET.arrow_timestamp_type("ns") TimestampType(timestamp[ns, tz=Europe/Paris])
all_iana
classmethod
¶
Return all IANA timezone identifiers available on this system.
Mode ¶
Bases: IntEnum
readable
property
¶
True when the mode admits reads.
Every :class:Mode canonically resolves to a + POSIX form
(rb, rb+, wb+, ab+, xb+) — all of those
admit reads. Only the strict :data:READ_ONLY rb and
:data:IGNORE (which is no-op) deny writes; nothing here
denies reads.
appendable
property
¶
True when writes append at EOF rather than at the cursor.
Only :data:APPEND carries POSIX O_APPEND semantics; every
other write mode positions writes at the explicit cursor.
os_mode
property
¶
Stdlib :func:open mode string for this :class:Mode.
- :attr:
READ_ONLY→"rb" - :attr:
OVERWRITE/ :attr:TRUNCATE→"wb+" - :attr:
APPEND→"ab+" - :attr:
ERROR_IF_EXISTS→"xb+" - everything else (AUTO, IGNORE, UPSERT, MERGE) →
"rb+"(in-place edit; the disposition is enforced higher up).
from_
classmethod
¶
Normalize value into a :class:Mode.
Accepts:
- :class:
Mode(returned as-is, idempotent). - Aliases like
"overwrite","OVERWRITE","error-if-exists","replace","add"— see :data:STR_MAPPING. - POSIX-style mode strings —
"rb","wb","ab+","x","r+b"— parsed structurally; any combination of one primary character (r/w/a/x) plus optionalb/t/+flags is accepted. None→ returns default if supplied, else :data:Mode.AUTO.
Falls back to :class:ValueError for unrecognized strings.
Numeric / non-string non-Mode inputs raise
:class:TypeError — the input grammar is "string or enum,"
not "anything stringifiable."
MimeType
dataclass
¶
MimeType(
name: str,
value: str,
extensions: tuple[str, ...] = (),
magics: tuple[MagicMatcher, ...] = (),
is_codec: bool = False,
is_tabular: bool = False,
is_blob: bool = False,
)
Dataclass MIME descriptor + registries.
- extensions: dotless, lower-case keys
- magics: ordered matchers
- is_codec: compression / wrapper formats
- is_tabular: row/tabular-ish formats (read into a frame)
- is_blob: opaque single-file payload — straight byte IO, no row
structure (images, pdf, archives, pickle, …). Mutually exclusive
with
is_tabular; codecs and directory/connector mimes are neither.
get
classmethod
¶
Pure lookup — never raises. Returns None on miss.
Kept as the low-level "resolve by name/value" entry point:
:meth:from_ / :meth:from_str / :meth:from_magic layer
the default-handling contract on top.
parse_many
classmethod
¶
Resolve obj to a flat, deduped list of :class:MimeType.
Accepts anything :meth:from_ accepts, plus:
- iterables (list, tuple, set, generator) of any supported input
- Accept-header strings:
"application/json, text/csv;q=0.8" - composite
format+codecstrings:"application/csv+gzip","parquet+zstd","trades.parquet.zst" - wildcard strings:
"*/*","text/*","image/*" None→[]
Order: first-seen wins, deduped by identity. For composites, the base format is emitted first, then the codec (codec wraps format).
Never raises on an unresolvable element — unknowns are dropped
silently. For strict per-element resolution, call :meth:from_.
from_magic
classmethod
¶
from_magic(
magic: Union[bytes, bytearray, memoryview, IO[bytes], str, Path],
default: "MimeType | None" = _RAISE,
) -> "MimeType | None"
Resolve by sniffing magic bytes from magic.
Accepts raw bytes/memoryview, an IO, or anything the buffer class can wrap. Reads the first 64 bytes and walks the registered magic matchers in definition order.
:param default: see :class:MimeType class docstring for the
shared default-handling contract.
:raises ValueError: on a miss when default was not supplied.
from_str
classmethod
¶
Resolve a :class:str — path-like, bare extension, or mime value.
Tries, in order:
- Direct lookup against the lower-cased input (covers
"text/csv"/"json"/".csv"without paying a :class:pathlib.Pathallocation). - If the string looks path-like (contains
/or\), take its suffix as an extension key. - Fall back to :meth:
get(name / mime-value lookup with theapplication//text// … prefix stripping). - Structural sniff on leading
{/[.
:param default: see :class:MimeType class docstring for the
shared default-handling contract.
:raises ValueError: on a miss when default was not supplied.
MimeTypes ¶
Singleton MIME definitions.
MediaType
dataclass
¶
from_many
classmethod
¶
Compose a MediaType from an ordered mime list (e.g. URL extensions).
Two conventions land here:
trades.csv.zst→["csv", "zst"]: the codec is the outer wrapper (you have to unzst before you can parse csv) →MediaType(CSV, codec=ZSTD).part-xxx.zstd.parquet→["zstd", "parquet"]: the format is the outer wrapper and the codec is the parquet page-codec hint baked into the file. Parquet handles the decompression internally →MediaType(PARQUET)with no outer codec; setting one would route the read through a decompressor that doesn't belong on this byte stream.
Order matters: the last mime decides. Last-is-codec promotes the codec to the wrapper slot; last-is-format keeps the format alone and drops earlier codec hints.
from_magic
classmethod
¶
from_magic(
buf: Union[bytes, bytearray, memoryview, IO[bytes]],
*,
default: "MediaType" = ...
) -> "MediaType"
Sniff a :class:MediaType from raw bytes.
Two-stage: outer magic identifies the wrapper; if the wrapper is a codec, decompress a head-window and sniff the inner format. See module docstring for the full story.
Accepts bytes / bytearray / memoryview directly. For
convenience, also accepts an IO[bytes] — in that case it
delegates to :meth:from_io, which manages cursor save/restore.
:param default: Fallback when the outer sniff finds nothing.
... (default) raises :class:ValueError; any other
value (including None) is returned as-is.
from_io
classmethod
¶
Sniff a :class:MediaType from a file-like, seeking freely.
Captures the cursor on entry and restores it on exit, so the caller's stream position is unaffected. Inside the call we seek freely:
seek(0)and read the first :data:_OUTER_PEEKbytes for outer-magic resolution.- If the outer mime is a codec,
seek(0)again and drive the codec's streaming decompressor directly to read the first :data:_INNER_PEEKbytes of decoded payload. - Sniff the decoded head for the inner mime.
The codec resolution is one dict lookup
(:meth:Codec.from_mime) — the outer magic loop already
landed on the registered :class:MimeType singleton, so we
reuse it as the key without re-resolving. For streaming
codecs (gzip / zstd / lz4 / bz2 / xz / zlib / lzma) the
decompressor is opened directly on the IO via
:meth:Codec._open_decompress_reader — no IO wrap, no
full-buffer materialization. Non-streaming codecs (snappy,
brotli) fall back to reading the compressed body and calling
:meth:Codec.decompress_bytes.
Decompression errors during the inner sniff are swallowed —
the worst case is returning a less-specific MediaType (the
outer codec is preserved). A separate caller that wants to
validate decompressibility should call :meth:Codec.decompress
directly.
:param io_obj: Any file-like with read + seek (stdlib
io.BytesIO, an open file, our own IO, etc.).
:param default: Fallback when the outer sniff finds nothing.
... (default) raises :class:ValueError; any other
value (including None) is returned as-is.
Codec ¶
Bases: ABC
Abstract compression codec.
is_streaming
property
¶
True when both compress and decompress have streaming paths.
Callers with large (GiB-scale) inputs should check this before
passing them to :meth:compress / :meth:decompress. When
False, those methods fall back to materializing the full
payload in memory through :meth:compress_bytes /
:meth:decompress_bytes.
compress ¶
Compress src into a new :class:IO.
Streams chunk-by-chunk when :meth:_open_compress_writer is
available. Otherwise falls back to a full-in-memory bytes
roundtrip — callers with multi-GiB inputs should inspect
:attr:is_streaming first.
The source cursor is restored on exit.
decompress ¶
Decompress src into a new :class:IO.
Streams chunk-by-chunk when :meth:_open_decompress_reader is
available. Otherwise falls back to a full-in-memory bytes
roundtrip — callers with multi-GiB compressed inputs should
inspect :attr:is_streaming first.
The source cursor is restored on exit.
read_start_end ¶
read_start_end(
src: "IO | bytes | bytearray | memoryview",
*,
n_start: int = 64,
n_end: int = 64,
chunk_size: int = _CHUNK
) -> tuple[bytes, bytes]
Return the first n_start and last n_end bytes of the decoded stream.
Streams the decompression and keeps only a bounded amount of
state in memory (:math:n\_end + chunk\_size bytes) — safe
for very large compressed inputs when the codec supports
streaming decompression.
When the codec does NOT expose a streaming decoder, falls back
to a full :meth:decompress_bytes call, which materializes
the whole uncompressed payload in memory.
from_
classmethod
¶
Parse an arbitrary input into a Codec instance.
Accepts:
- :class:Codec instances (returned as-is).
- Short names like "gzip", "zstd" (case-insensitive).
- Anything :meth:MimeType.parse can resolve to a codec mime.
- None → returns default.
Codecs ¶
Singleton class for accessing all registered codecs.
ByteUnit ¶
Bases: IntEnum
Canonical IEC binary byte-unit token + scalar value.
Each member's value IS the byte count for one unit, so 128 *
ByteUnit.MIB reads naturally at the call site and slots into
any int field. Use :meth:parse_size when accepting external
config / API input — it canonicalizes "128 MB" /
"1.5 GiB" / raw integers / ByteUnit members all to a
plain integer byte count.
from_
classmethod
¶
Coerce any Python value into a :class:ByteUnit member.
Accepts:
- :class:
ByteUnit(returned as-is); - any string the alias table knows —
B/KB/MiB/gigabyte/ mixed case / trailings; - an integer matching a member's byte value (
1024→ :attr:KIB); None— returns default if supplied, else raises.
default swallows unknown / unparseable input. Without it,
unknown tokens raise :class:ValueError and unsupported types
raise :class:TypeError.
parse_size
classmethod
¶
Coerce a size-like value to an integer byte count.
The single entry point for "config gave me a size, give me bytes." Accepts:
- :class:
int— passed through (must be non-negative); - :class:
ByteUnit— its scalar value; - a quantity string
"128 MB"/"1.5 GiB"/"512"— parsed with this enum's IEC conventions; - a bare unit string
"MiB"— yields one unit (1024**2); None— returns default if supplied, else raises.
Floating-point quantities round to the nearest byte ("1.5 KiB"
→ 1536). Negative values raise :class:ValueError.
format
classmethod
¶
Format an integer byte count as a human-readable string.
Picks the largest unit at which n divides cleanly into a
scalar ≥ 1, defaulting to IEC tokens ("128 MiB"); pass
iec=False for the colloquial short form ("128 MB").
precision controls fractional digits.
pretty
classmethod
¶
pretty(
v: float,
unit: "ByteUnit | str | None" = None,
*,
iec: bool = True,
precision: int = 1
) -> str
Human-readable rendering of a quantity v expressed in unit.
v is a scalar count of unit (bytes by default); it's
scaled to a byte count and handed to :meth:format. The companion
to :meth:format for the common "I have N MiB, show it nicely"
case::
ByteUnit.pretty(1536) # "1.5 KiB"
ByteUnit.pretty(8, ByteUnit.MIB) # "8.0 MiB"
ByteUnit.pretty(1.5, "gb") # "1.5 GiB"
unit defaults to :attr:B (resolved at call time so the class
default is usable inside the method body); pass iec=False for
the colloquial short form.
Unit ¶
Mixin providing :meth:from_ / :meth:convert / scalar helpers.
Subclasses combine this with :class:Enum and declare members
as (symbol, factor_to_canonical, offset_to_canonical) tuples
(or (symbol, factor) — offset defaults to 0.0). The
canonical member of each family has factor=1.0 and
offset=0.0; conversion goes through that canonical pivot.
Extra spellings ("megawatthours" → MWH) live in a
<Family>._ALIASES dict assigned outside the enum body — the
enum metaclass would treat a class-body dict as a member.
from_
classmethod
¶
Coerce value to a member of this unit family.
Accepts: a member (passed through), a known symbol
("MWh" / "°C" / "psi"), the enum member name
("KWH"), or any alias the per-family _ALIASES dict
knows ("megawatthours" / "celsius" / "pound").
Lookup is case-insensitive.
Pass default to swallow unknown / unparseable input; without
it, unknown strings raise :class:ValueError with a hint
listing the valid symbols, and non-string non-member input
raises :class:TypeError.
convert
classmethod
¶
Convert value from source unit to target unit (scalar).
to_canonical ¶
Convert value in self's unit to the family's canonical unit.
from_canonical ¶
Convert value in the canonical unit to self's unit.
canonical
classmethod
¶
The family's canonical member (factor=1.0, offset=0.0).
factor_map
classmethod
¶
{symbol: factor} for every member + alias, for per-row maps.
offset_map
classmethod
¶
{symbol: offset} for every member + alias, for per-row maps.
EnergyUnit ¶
Bases: Unit, Enum
Energy unit (canonical = joule).
Members cover SI scale (J / kJ / MJ / GJ / TJ), the watt-hour family that dominates electricity-market ingestion (Wh / kWh / MWh / GWh / TWh), and the imperial / domestic units (BTU / cal / kcal / therm).
from_
classmethod
¶
Coerce value to a member of this unit family.
Accepts: a member (passed through), a known symbol
("MWh" / "°C" / "psi"), the enum member name
("KWH"), or any alias the per-family _ALIASES dict
knows ("megawatthours" / "celsius" / "pound").
Lookup is case-insensitive.
Pass default to swallow unknown / unparseable input; without
it, unknown strings raise :class:ValueError with a hint
listing the valid symbols, and non-string non-member input
raises :class:TypeError.
convert
classmethod
¶
Convert value from source unit to target unit (scalar).
to_canonical ¶
Convert value in self's unit to the family's canonical unit.
from_canonical ¶
Convert value in the canonical unit to self's unit.
canonical
classmethod
¶
The family's canonical member (factor=1.0, offset=0.0).
factor_map
classmethod
¶
{symbol: factor} for every member + alias, for per-row maps.
offset_map
classmethod
¶
{symbol: offset} for every member + alias, for per-row maps.
PowerUnit ¶
Bases: Unit, Enum
Power unit (canonical = watt).
SI scale (W / kW / MW / GW / TW) plus mechanical horsepower. Energy-market feeds publish capacity in MW, transmission limits in GW, household-scale appliances in W — having one enum cover all of them is what keeps schema-per-source curated views from re-stringing the unit token at every read.
from_
classmethod
¶
Coerce value to a member of this unit family.
Accepts: a member (passed through), a known symbol
("MWh" / "°C" / "psi"), the enum member name
("KWH"), or any alias the per-family _ALIASES dict
knows ("megawatthours" / "celsius" / "pound").
Lookup is case-insensitive.
Pass default to swallow unknown / unparseable input; without
it, unknown strings raise :class:ValueError with a hint
listing the valid symbols, and non-string non-member input
raises :class:TypeError.
convert
classmethod
¶
Convert value from source unit to target unit (scalar).
to_canonical ¶
Convert value in self's unit to the family's canonical unit.
from_canonical ¶
Convert value in the canonical unit to self's unit.
canonical
classmethod
¶
The family's canonical member (factor=1.0, offset=0.0).
factor_map
classmethod
¶
{symbol: factor} for every member + alias, for per-row maps.
offset_map
classmethod
¶
{symbol: offset} for every member + alias, for per-row maps.
MassUnit ¶
Bases: Unit, Enum
Mass unit (canonical = kilogram).
SI scale (mg / g / kg / t / Mt) plus the imperial pair (lb / oz) commonly used in commodity feeds.
from_
classmethod
¶
Coerce value to a member of this unit family.
Accepts: a member (passed through), a known symbol
("MWh" / "°C" / "psi"), the enum member name
("KWH"), or any alias the per-family _ALIASES dict
knows ("megawatthours" / "celsius" / "pound").
Lookup is case-insensitive.
Pass default to swallow unknown / unparseable input; without
it, unknown strings raise :class:ValueError with a hint
listing the valid symbols, and non-string non-member input
raises :class:TypeError.
convert
classmethod
¶
Convert value from source unit to target unit (scalar).
to_canonical ¶
Convert value in self's unit to the family's canonical unit.
from_canonical ¶
Convert value in the canonical unit to self's unit.
canonical
classmethod
¶
The family's canonical member (factor=1.0, offset=0.0).
factor_map
classmethod
¶
{symbol: factor} for every member + alias, for per-row maps.
offset_map
classmethod
¶
{symbol: offset} for every member + alias, for per-row maps.
LengthUnit ¶
Bases: Unit, Enum
Length unit (canonical = metre).
SI scale (mm / cm / m / km) plus the common imperial / aviation / maritime tokens (in / ft / yd / mi / nmi).
from_
classmethod
¶
Coerce value to a member of this unit family.
Accepts: a member (passed through), a known symbol
("MWh" / "°C" / "psi"), the enum member name
("KWH"), or any alias the per-family _ALIASES dict
knows ("megawatthours" / "celsius" / "pound").
Lookup is case-insensitive.
Pass default to swallow unknown / unparseable input; without
it, unknown strings raise :class:ValueError with a hint
listing the valid symbols, and non-string non-member input
raises :class:TypeError.
convert
classmethod
¶
Convert value from source unit to target unit (scalar).
to_canonical ¶
Convert value in self's unit to the family's canonical unit.
from_canonical ¶
Convert value in the canonical unit to self's unit.
canonical
classmethod
¶
The family's canonical member (factor=1.0, offset=0.0).
factor_map
classmethod
¶
{symbol: factor} for every member + alias, for per-row maps.
offset_map
classmethod
¶
{symbol: offset} for every member + alias, for per-row maps.
VolumeUnit ¶
Bases: Unit, Enum
Volume unit (canonical = cubic metre).
SI scale (mL / L / m³) plus the gallon variants and the oil barrel — commodity feeds split between US gallons, UK gallons, and the 42-gallon oil barrel, and a curated view that doesn't keep them straight ships wrong totals.
from_
classmethod
¶
Coerce value to a member of this unit family.
Accepts: a member (passed through), a known symbol
("MWh" / "°C" / "psi"), the enum member name
("KWH"), or any alias the per-family _ALIASES dict
knows ("megawatthours" / "celsius" / "pound").
Lookup is case-insensitive.
Pass default to swallow unknown / unparseable input; without
it, unknown strings raise :class:ValueError with a hint
listing the valid symbols, and non-string non-member input
raises :class:TypeError.
convert
classmethod
¶
Convert value from source unit to target unit (scalar).
to_canonical ¶
Convert value in self's unit to the family's canonical unit.
from_canonical ¶
Convert value in the canonical unit to self's unit.
canonical
classmethod
¶
The family's canonical member (factor=1.0, offset=0.0).
factor_map
classmethod
¶
{symbol: factor} for every member + alias, for per-row maps.
offset_map
classmethod
¶
{symbol: offset} for every member + alias, for per-row maps.
TemperatureUnit ¶
Bases: Unit, Enum
Temperature unit (canonical = kelvin).
The one affine family — Celsius and Fahrenheit have non-zero
offsets relative to kelvin, so the conversion is
canonical = value * factor + offset (not pure scaling).
Symbols use the standard degree glyphs (°C / °F) so the
curated tables render legibly in BI tools; aliases cover the
plain-ascii forms ("C" / "F" / "K").
from_
classmethod
¶
Coerce value to a member of this unit family.
Accepts: a member (passed through), a known symbol
("MWh" / "°C" / "psi"), the enum member name
("KWH"), or any alias the per-family _ALIASES dict
knows ("megawatthours" / "celsius" / "pound").
Lookup is case-insensitive.
Pass default to swallow unknown / unparseable input; without
it, unknown strings raise :class:ValueError with a hint
listing the valid symbols, and non-string non-member input
raises :class:TypeError.
convert
classmethod
¶
Convert value from source unit to target unit (scalar).
to_canonical ¶
Convert value in self's unit to the family's canonical unit.
from_canonical ¶
Convert value in the canonical unit to self's unit.
canonical
classmethod
¶
The family's canonical member (factor=1.0, offset=0.0).
factor_map
classmethod
¶
{symbol: factor} for every member + alias, for per-row maps.
offset_map
classmethod
¶
{symbol: offset} for every member + alias, for per-row maps.
PressureUnit ¶
Bases: Unit, Enum
Pressure unit (canonical = pascal).
SI scale (Pa / kPa / MPa / hPa) plus the bar family (bar / mbar), atmosphere, psi, and torr. Weather feeds publish in hPa or mbar, oil & gas in psi or bar, process control in kPa or MPa — one enum keeps them aligned.
from_
classmethod
¶
Coerce value to a member of this unit family.
Accepts: a member (passed through), a known symbol
("MWh" / "°C" / "psi"), the enum member name
("KWH"), or any alias the per-family _ALIASES dict
knows ("megawatthours" / "celsius" / "pound").
Lookup is case-insensitive.
Pass default to swallow unknown / unparseable input; without
it, unknown strings raise :class:ValueError with a hint
listing the valid symbols, and non-string non-member input
raises :class:TypeError.
convert
classmethod
¶
Convert value from source unit to target unit (scalar).
to_canonical ¶
Convert value in self's unit to the family's canonical unit.
from_canonical ¶
Convert value in the canonical unit to self's unit.
canonical
classmethod
¶
The family's canonical member (factor=1.0, offset=0.0).
factor_map
classmethod
¶
{symbol: factor} for every member + alias, for per-row maps.
offset_map
classmethod
¶
{symbol: offset} for every member + alias, for per-row maps.
JoinType ¶
Bases: IntEnum
Canonical join kind for tabular join surfaces.
Use :meth:from_ when accepting external input — it canonicalizes
aliases ("left", "LEFT JOIN", "anti", "outer",
integer codes) to a member and raises :class:ValueError for
unknown tokens.
Pass :attr:arrow / :attr:polars / :attr:sql to engine join
APIs — pyarrow's :meth:pa.Table.join and polars'
:meth:DataFrame.join accept different spellings of the same
concept, and storing the integer code keeps the enum a clean
discriminator without binding to either spelling.
polars
property
¶
polars' :meth:DataFrame.join how token.
Polars has no built-in right semi / right anti form —
those raise :class:NotImplementedError. Swap operands and use
the left-side equivalent.
from_
classmethod
¶
Coerce any Python value into a :class:JoinType.
Accepts:
- :class:
JoinType(returned as-is); - any string the alias table or canonical Arrow tokens know —
"inner","left","left outer","LEFT JOIN","anti","left_anti","outer","cross", …; mixed case, hyphens / underscores / spaces all normalize; - an integer code matching a member's value (round-trips with
int(JoinType.X)); None— returns default if supplied, else raises.
default swallows unknown / unparseable input. Without it,
unknown tokens raise :class:ValueError and unsupported types
raise :class:TypeError.
is_valid
classmethod
¶
Return True when :meth:from_ would succeed for value.
Scheme ¶
Bases: str, Enum
Canonical URL-scheme token for a Yggdrasil :class:URLBased
subclass.
Subclasses :class:str so a member is interchangeable with its
scheme token everywhere a string is expected (url.scheme ==
Scheme.DBFS works, f"{Scheme.S3}://bucket/key" reads
naturally). The lazy-import resolver lives on :meth:path_class.
from_
classmethod
¶
Coerce value to a :class:Scheme member.
Accepts:
- :class:
Scheme(returned as-is); - a scheme string — case-insensitive, trailing
://and whitespace tolerated, common aliases ("s3a"→ :attr:S3,"local"→ :attr:FILE,"memory"→ :attr:MEMORY); None— returns default if supplied, else raises.
default swallows unknown / unparseable input. Without it,
unknown tokens raise :class:ValueError and unsupported types
raise :class:TypeError.
path_class ¶
Return the concrete :class:URLBased subclass for this scheme.
Lazy: the backend module is imported on first use, which fires
the :meth:URLBased.__init_subclass__ side-effect that
registers the class. Subsequent calls hit the per-process cache.
Raises :class:ImportError when the backend's optional
dependencies aren't installed (databricks-sdk for the
Databricks schemes, boto3 for s3, …) — the message
names the missing extra so the caller can install it.
resolve
classmethod
¶
Shortcut: cls.from_(value).path_class().
Useful where the caller has a raw scheme string (or URL) and
just wants the concrete URLBased class. default is forwarded
to :meth:from_ for forgiving lookup; it does not catch
:class:ImportError from :meth:path_class.
NodeType ¶
Bases: str, Enum
Canonical Databricks node_type_id values.
Member values are the exact strings the Databricks SDK expects. Use
:attr:NodeType.DEFAULT for the codebase-wide default node type, the
semantic-size aliases (:attr:SMALL / :attr:MEDIUM / :attr:LARGE /
:attr:XLARGE) for intent-driven sizing, or the explicit cloud-specific
members when you need a precise SKU.
The enum is intentionally not exhaustive — Databricks publishes
hundreds of SKUs per cloud and most callers want a small, opinionated
set. Coerce caller-provided strings through :meth:to_id /
:meth:from_ so unknown SKUs round-trip as plain strings instead of
raising.
from_
classmethod
¶
Coerce value into a :class:NodeType member.
Accepts:
- :class:
NodeType(returned as-is); - any string in :data:
_NODETYPE_ALIASES, case-insensitive; - a bare member name (
"FLEET_XLARGE"); None— returns default if supplied, else :attr:NodeType.DEFAULT.
Raises :class:ValueError for unknown strings unless default is
supplied. To accept arbitrary SKU strings (without rejecting valid
cloud-specific identifiers), call :meth:to_id instead.
to_id
classmethod
¶
to_id(
value: Union[str, "NodeType", None],
*,
default: Optional[Union[str, "NodeType"]] = None
) -> str
Coerce value to a Databricks node_type_id string.
Unlike :meth:from_, this is forgiving: arbitrary SKU strings
round-trip unchanged so callers can pass cloud-specific identifiers
the enum does not enumerate ("r5d.metal", custom marketplace
images, …). Aliases are resolved when present.
Resolution order:
NodeTypemember → its string value.- Alias / member name / value match → the canonical string.
- Bare string → trimmed and returned as-is.
None→defaultif supplied, else :attr:NodeType.DEFAULT.
NodeSpec
dataclass
¶
NodeSpec(
cpu_cores: int,
ram_gib: float,
gpu_count: int = 0,
local_disk_gib: float = 0.0,
cloud: str = "",
)
Hardware specs for a :class:NodeType member.
The numbers reflect the vendor's published per-VM characteristics for
a single worker (or driver) — not the cluster-wide totals. ram_gib
is binary GiB (1024 ** 3 bytes), matching every cloud's own
documentation convention.
Attributes¶
cpu_cores
Virtual CPU count exposed to Spark.
ram_gib
Memory available to Spark, in IEC GiB.
gpu_count
Number of GPUs (0 for non-GPU SKUs).
local_disk_gib
Local SSD/NVMe storage attached to the instance. 0 when the
SKU has no local disk (relies on remote/elastic storage).
cloud
Cloud family identifier ("fleet" / "aws" / "azure" /
"gcp") used by :meth:NodeType.from_cpu_and_ram when
prefer is set.
State ¶
Bases: IntEnum
Unified execution state for async statement / job results.
Use :meth:from_ to normalize backend-specific tokens, and the
is_* predicates to derive done / failed / started
without re-implementing the membership sets per backend.
is_queued
property
¶
True for :attr:QUEUED — prepared, sitting in a submission queue.
is_accepted
property
¶
True for :attr:ACCEPTED — acknowledged but not yet running.
is_started
property
¶
True for anything from :attr:RUNNING onward.
Mirrors :attr:StatementResult.started: once the backend has
actually started executing, is_started flips and stays
True through every terminal state. :attr:ACCEPTED is
not started — the venue holds the order, no execution yet.
is_active
property
¶
True for non-terminal states with backend awareness.
Covers :attr:QUEUED through :attr:PARTIAL — anything the
caller can reasonably wait on. :attr:IDLE is excluded
(nothing has been submitted yet) and every terminal state is
excluded (no more transitions).
is_done
property
¶
True for terminal states (:attr:SUCCEEDED, :attr:REJECTED,
:attr:FAILED, :attr:CANCELED, :attr:EXPIRED) — no more
transitions expected.
is_failed
property
¶
True for :attr:REJECTED / :attr:FAILED / :attr:CANCELED /
:attr:EXPIRED.
Every non-success terminal state counts as failed because each
one means "the caller asked for a result and didn't get one":
cancellation, rejection, mid-run error, or TTL expiry all leave
the operation incomplete from the caller's view, and the
per-backend raise_for_status raises on each.
from_
classmethod
¶
Coerce any Python value into a :class:State.
See module docstring for the accepted shapes. default
swallows unknown / unparseable input; without it, unknown
tokens raise :class:ValueError and unsupported types raise
:class:TypeError.
is_valid
classmethod
¶
Return True when :meth:from_ would succeed for value.
IOKind ¶
Bases: IntEnum
What a backend reports a path/holder entry is.
EngineType ¶
Bases: IntEnum
Compute engine for a Databricks :class:~yggdrasil.databricks.table.table.Table
read / write.
- :attr:
YGGDRASIL— yggdrasil's native DeltaFolder (direct_delta_log - parquet over UC-vended credentials), no warehouse.
- :attr:
DATABRICKS_SQL_WAREHOUSE— the Databricks SQL warehouse. - :attr:
SPARK— a Spark session (Databricks Connect / cluster / notebook).
from_str
classmethod
¶
Coerce an alias string into an :class:EngineType.
Matches a forgiving alias table ("warehouse" / "sql" →
DATABRICKS_SQL_WAREHOUSE; "ygg" / "native" → YGGDRASIL;
"spark" / "connect" → SPARK). On an unknown string: raise
:class:ValueError when default is ..., else return default.
from_numeric
classmethod
¶
Coerce an integer code into an :class:EngineType.
On an out-of-range / non-integer code: raise when default is ...,
else return default. bool is rejected (it's an int subclass
but never a meaningful engine code).
from_
classmethod
¶
Coerce an :class:EngineType / alias string / int code / None.
None (or ...) is "unset" and returns None. A string routes
to :meth:from_str, a number to :meth:from_numeric; both honour
default (... → raise, else return the fallback).
AWSRegion ¶
Bases: str, Enum
partition
property
¶
The AWS partition this region belongs to (aws / aws-us-gov /
aws-cn).
from_
classmethod
¶
Coerce value (an :class:AWSRegion, a region string, or None)
to a member. Unknown / None returns default if given, else
raises.
from_text
classmethod
¶
Find an AWS region code embedded in text (e.g. a bucket name like
acme-dls3-eu-central-1-p), or default if none is present.
Matches a region only when it stands as a whole, delimiter-bounded
token (so eu-central-12 / ...-1foo don't false-positive), and
prefers the longest code at a position (ap-southeast-1 over
ap-south-1).
unit_family_for ¶
Find the unit family that recognises value (member or symbol).
Useful when a curated row carries a free-form unit token and the
caller needs to dispatch to the right family without hard-coding
a per-source switch. Returns the first family for which
:meth:Unit.is_valid returns True; raises :class:ValueError
when no family claims the token (the message lists every family
that was consulted).