`yggdrasil.data.types.primitive.temporal`¶

temporal ¶

Unified temporal module — types + framework-native cast helpers.

Nuclear-simplified version. Design rules:

Framework casts only. Each engine (Arrow, Polars, Spark, pandas) gets one cast dispatcher that delegates straight to pc.cast / series.cast / column.cast. No multi-format coalesce, no ISO-duration regex, no per-row Python fallback, no fractional-second re-attachment.
ISO-8601 only for strings. Strings parse via each framework's native ISO strptime (pc.strptime with one ISO format, pl.Series.str.to_datetime default, Spark to_timestamp default). Any non-ISO shape (dd/MM/yyyy, PT15M, HH:MM:SS clock durations, etc.) is caller's problem — pre-parse before handing the array here.
Wall-clock reinterpret for naive→aware. One rule, always: a naive timestamp cast to a tz-aware target keeps its wall-clock digits and stamps on the target zone. No unsafe_tz flag, no "assume UTC" mode, no DST null-on-nonexistent threading — we lean on whatever the framework does by default and accept the consequences.

TemporalType `dataclass` ¶

TemporalType(
    byte_size: int | None = None,
    unit: TimeUnit = TimeUnit.MICROSECOND,
    tz: str | None = None,
)

Bases: PrimitiveType, ABC

Base class for Date / Time / Timestamp / Duration.

Holds shared fields (unit / tz) and cross-engine dispatch logic. Subclasses implement type_id, the handles_* / from_* class methods, and to_arrow / to_polars / to_spark / to_spark_name.

select_fields ¶

select_fields(
    identifiers: "SelectType | Iterable[SelectType]" = (),
    *others: SelectType,
    raise_error: bool = True
) -> list["Field"]

Resolve one or more identifiers into the matching :class:Field objects.

Accepts strings (resolved by name), ints (resolved by index), and existing :class:Field instances (resolved by .name against this container — so callers can copy a field set between sibling schemas without first stringifying everything).

Calling shapes that all work the same way:

schema.select_fields("price") — single identifier.
schema.select_fields("price", "qty", 0) — multiple positionals.
schema.select_fields(["price", "qty"]) — single iterable.
schema.select_fields(other_schema.children) — copy a sibling's fields by name into this schema.
schema.select_fields("price", ["qty", "ts"], 0) — mixed; each positional is itself flattened so iterables and scalars can be interleaved.

:param identifiers: First identifier or iterable of identifiers. :param others: Additional identifiers. Each is flattened the same way as the first. :param raise_error: True (default) — missing identifiers raise via :meth:field_by with the same suggestion-rich error message used elsewhere. False — missing identifiers yield None in the returned list, preserving caller order.

:returns: A list of :class:Field (or Field | None when raise_error=False), one entry per resolved identifier in caller order. Duplicates in the input produce duplicates in the output — this is intentional, since select is the natural place to express a projection and projections sometimes repeat columns.

:raises KeyError: With suggestions, when raise_error is True and an identifier doesn't resolve. :raises TypeError: When an identifier is not a str / int / Field.

pretty_format `abstractmethod` ¶

pretty_format(indent: int = 2, level: int = 0) -> str

Pretty-print this dtype with one element per line.

indent is the per-level step in spaces. level is the current depth — the line is prefixed with indent * level spaces. Flat dtypes render as a single line; nested dtypes (struct / list / map) override this to lay each child out on its own line at level + 1.

short ¶

short(depth: int = 2) -> str

A compact, single-line type tag — recursive (bounded) for nested types.

Scalars render as i64 / f64 / str / bool / date / ts / dec / bin / json; nested types recurse through their own children — list<i64>, struct<x:i64, y:str>, map<str,f64>, even list<struct<id:i64>> — until depth runs out (then a bare list / struct / map). Built from this type's :attr:type_id + :attr:children, not any one engine. Struct fields are capped and the whole tag is elided past :data:_SHORT_TAG_MAX chars, so a deep schema can't widen a :meth:yggdrasil.io.tabular.Tabular.display header forever.

to_pyhint ¶

to_pyhint() -> Any

Return the Python type hint that maps back to this DataType.

Cached when :meth:from_pytype stamped an explicit hint on the instance (preserves user-defined dataclasses, enum classes, numpy.int64 and other narrow aliases the canonical reconstruction would collapse). Otherwise falls back to :meth:_default_pyhint, the subclass hook that builds a canonical hint from the dtype's own state.

expand_alias `classmethod` ¶

expand_alias(name: str) -> str

Expand a known short-alias prefix.

pa.Table → pyarrow.Table, pl.DataFrame → polars.DataFrame, etc. Returns name unchanged when no registered prefix matches. The table lives at :attr:PYHINT_ALIASES.

strip_annotated `staticmethod` ¶

strip_annotated(hint: Any) -> Any

Strip Annotated[T, ...] down to T (recursive).

unwrap_newtype `staticmethod` ¶

unwrap_newtype(hint: Any) -> Any

Unwrap a chain of NewType aliases to the base type.

normalize_hint `classmethod` ¶

normalize_hint(hint: Any) -> Any

strip_annotated then unwrap_newtype — common preamble.

Every from_pytype / unwrap_* flow needs the hint stripped of Annotated metadata and the NewType chain unwrapped before the real dispatch. Bundled into one classmethod so the ordering ("strip Annotated before NewType, recursive on both") lives in one place.

unwrap_optional `classmethod` ¶

unwrap_optional(hint: Any) -> tuple[bool, Any]

Return (is_optional, inner) for Optional[T] / T | None.

Only collapses a Union whose non-None arms reduce to exactly one type — Optional[int] → (True, int), int | None → (True, int). Multi-type unions return (False, hint) so callers can decide how to handle them (the cast registry generally falls through to StringType).

unwrap_nullable_hint `classmethod` ¶

unwrap_nullable_hint(hint: Any) -> tuple[Any, bool]

Field-flavoured Optional unwrap: (inner, has_null).

Differences from :meth:unwrap_optional:

String hints route through :class:ParsedDataType.from_ so the field name / nullability tag baked into the DSL form (e.g. "int64?") survives.
Multi-type unions stay intact — the caller wants (Union[A, B], True), not (Union[A, B], False) — so :class:yggdrasil.data.Field can stamp nullable=True without losing the multi-arm shape.

is_runtime_value `staticmethod` ¶

is_runtime_value(x: Any) -> bool

True for runtime values (42, [], MyClass()).

False for type hints — distinguishes convert(42, int) (runtime value with target hint) from convert(int, str) (two hints, no value). Used to gate dispatch decisions in downstream tooling.

resolve_str_annotation `classmethod` ¶

resolve_str_annotation(
    s: str, func_globals: Optional[dict[str, Any]] = None
) -> Any

Resolve a string annotation to a real type.

Tries, in order:

eval in func_globals + builtins — picks up local imports and aliases declared in the function's own module.
eval against typing for generic shapes (Optional[int], list[int]) when func_globals doesn't have them in scope.
Alias-prefix expansion via :meth:expand_alias + dotted importlib.import_module lookup so pa.Table / pl.DataFrame resolve without the function's globals having ever imported the alias.

Returns s verbatim when every path fails — callers treat that as "unresolved, skip coercion / leave as-is".

resolve_function_annotations `classmethod` ¶

resolve_function_annotations(func: 'Callable[..., Any]') -> dict[str, Any]

Resolve every annotation on func to a real type.

Two-pass best effort so a single unresolvable annotation doesn't blow up the rest:

:func:inspect.get_annotations with eval_str=True — evaluates every annotation in the function's globals + builtins in one shot.
Per-annotation :meth:resolve_str_annotation fallback for entries the fast path left as strings (it silently returns the literal when its eval misses).

Anything still a string after both passes is left untouched — callers treat it as "couldn't resolve, skip coercion" rather than raising.

as_polars ¶

as_polars() -> 'DataType'

Return a Polars-flavored :class:DataType for this type.

Same shape as :meth:as_spark — stays on the yggdrasil side of the boundary and returns a :class:DataType whose :meth:to_polars lands on a dtype Polars natively represents. Defaults to self; subclasses Polars can't store at their declared width / precision override:

Float8Type and Float16Type widen to Float32Type (Polars has no sub-32-bit floats);
TimestampType / DurationType with second-precision (unit="s") widen to unit="ms" (Polars supports ms / us / ns only);
nested types (ArrayType / MapType / StructType) recurse via as_polars on their child fields.

:class:Field and :class:Schema expose a matching as_polars that delegates to self.dtype.as_polars and re-wraps so callers chain through Field-shaped APIs without dropping back to a plain :class:DataType.

as_spark ¶

as_spark() -> 'DataType'

Return a Spark-flavored :class:DataType for this type.

as_spark lives on the yggdrasil side of the boundary: it returns a :class:DataType that maps cleanly to a Spark dtype (i.e. one self.to_spark() would round-trip without a widening-time surprise). For types Spark already represents natively (signed ints, Float32 / Float64, Date, String / Binary / Boolean, decimal, naive / UTC timestamps), the default is to return self unchanged.

Subclasses Spark cannot represent natively (IntegerType with signed=False, Float16Type, DurationType, TimeType, non-UTC TimestampType) override this to return the closest Spark-compatible yggdrasil dtype — usually a widened integer, a StringType, or a naive timestamp. Nested types (ArrayType / MapType / StructType) recurse via as_spark on their child fields so the whole tree comes back Spark-compatible in one call.

from_pytype `classmethod` ¶

from_pytype(hint: Any) -> 'DataType'

Parse a Python annotation into a :class:DataType.

Stamps the resulting instance's _pyhint_cache with the original hint when the canonical reconstruction (:meth:_default_pyhint) wouldn't round-trip — so from_pytype(MyDataclass).to_pyhint() returns MyDataclass itself rather than a generic struct hint, from_pytype(MyEnum).to_pyhint() returns the enum class, and from_pytype(np.int64).to_pyhint() returns np.int64 rather than the canonical int.

cast_arrow_batch_iterator ¶

cast_arrow_batch_iterator(
    batches: "Iterable[pa.RecordBatch]",
    options: "CastOptions | None" = None,
    **more_options
) -> "Iterator[pa.RecordBatch]"

Cast a stream of :class:pa.RecordBatch against this dtype.

Non-struct dtypes promote to a single-column struct via :meth:to_struct and reuse the struct's iterator helper.

cast_polars_expr ¶

cast_polars_expr(
    series: "polars.Series | polars.Expr",
    options: "CastOptions | None" = None,
    **more_options
) -> "polars.Series | polars.Expr"

Expr-shape passthrough to :meth:cast_polars_series.

:meth:cast_polars_series already dispatches by isinstance — this method exists so callers that know they hold an Expr can name it.

TimestampType `dataclass` ¶

TimestampType(
    byte_size: int | None = None,
    unit: TimeUnit = TimeUnit.MICROSECOND,
    tz: Timezone = Timezone.NAIVE,
)

Bases: TemporalType

tz_iana `property` ¶

tz_iana: str | None

IANA token for :attr:tz, or None when naive.

Bridge for engine APIs (pa.timestamp, pl.Datetime, Spark) that take a string. New code should prefer self.tz directly — it carries :class:Timezone helpers like is_utc() and utc_offset().

select_fields ¶

select_fields(
    identifiers: "SelectType | Iterable[SelectType]" = (),
    *others: SelectType,
    raise_error: bool = True
) -> list["Field"]

Resolve one or more identifiers into the matching :class:Field objects.

Accepts strings (resolved by name), ints (resolved by index), and existing :class:Field instances (resolved by .name against this container — so callers can copy a field set between sibling schemas without first stringifying everything).

Calling shapes that all work the same way:

schema.select_fields("price") — single identifier.
schema.select_fields("price", "qty", 0) — multiple positionals.
schema.select_fields(["price", "qty"]) — single iterable.
schema.select_fields(other_schema.children) — copy a sibling's fields by name into this schema.
schema.select_fields("price", ["qty", "ts"], 0) — mixed; each positional is itself flattened so iterables and scalars can be interleaved.

:param identifiers: First identifier or iterable of identifiers. :param others: Additional identifiers. Each is flattened the same way as the first. :param raise_error: True (default) — missing identifiers raise via :meth:field_by with the same suggestion-rich error message used elsewhere. False — missing identifiers yield None in the returned list, preserving caller order.

:returns: A list of :class:Field (or Field | None when raise_error=False), one entry per resolved identifier in caller order. Duplicates in the input produce duplicates in the output — this is intentional, since select is the natural place to express a projection and projections sometimes repeat columns.

:raises KeyError: With suggestions, when raise_error is True and an identifier doesn't resolve. :raises TypeError: When an identifier is not a str / int / Field.

short ¶

short(depth: int = 2) -> str

A compact, single-line type tag — recursive (bounded) for nested types.

Scalars render as i64 / f64 / str / bool / date / ts / dec / bin / json; nested types recurse through their own children — list<i64>, struct<x:i64, y:str>, map<str,f64>, even list<struct<id:i64>> — until depth runs out (then a bare list / struct / map). Built from this type's :attr:type_id + :attr:children, not any one engine. Struct fields are capped and the whole tag is elided past :data:_SHORT_TAG_MAX chars, so a deep schema can't widen a :meth:yggdrasil.io.tabular.Tabular.display header forever.

to_pyhint ¶

to_pyhint() -> Any

Return the Python type hint that maps back to this DataType.

Cached when :meth:from_pytype stamped an explicit hint on the instance (preserves user-defined dataclasses, enum classes, numpy.int64 and other narrow aliases the canonical reconstruction would collapse). Otherwise falls back to :meth:_default_pyhint, the subclass hook that builds a canonical hint from the dtype's own state.

expand_alias `classmethod` ¶

expand_alias(name: str) -> str

Expand a known short-alias prefix.

pa.Table → pyarrow.Table, pl.DataFrame → polars.DataFrame, etc. Returns name unchanged when no registered prefix matches. The table lives at :attr:PYHINT_ALIASES.

strip_annotated `staticmethod` ¶

strip_annotated(hint: Any) -> Any

Strip Annotated[T, ...] down to T (recursive).

unwrap_newtype `staticmethod` ¶

unwrap_newtype(hint: Any) -> Any

Unwrap a chain of NewType aliases to the base type.

normalize_hint `classmethod` ¶

normalize_hint(hint: Any) -> Any

strip_annotated then unwrap_newtype — common preamble.

Every from_pytype / unwrap_* flow needs the hint stripped of Annotated metadata and the NewType chain unwrapped before the real dispatch. Bundled into one classmethod so the ordering ("strip Annotated before NewType, recursive on both") lives in one place.

unwrap_optional `classmethod` ¶

unwrap_optional(hint: Any) -> tuple[bool, Any]

Return (is_optional, inner) for Optional[T] / T | None.

Only collapses a Union whose non-None arms reduce to exactly one type — Optional[int] → (True, int), int | None → (True, int). Multi-type unions return (False, hint) so callers can decide how to handle them (the cast registry generally falls through to StringType).

unwrap_nullable_hint `classmethod` ¶

unwrap_nullable_hint(hint: Any) -> tuple[Any, bool]

Field-flavoured Optional unwrap: (inner, has_null).

Differences from :meth:unwrap_optional:

String hints route through :class:ParsedDataType.from_ so the field name / nullability tag baked into the DSL form (e.g. "int64?") survives.
Multi-type unions stay intact — the caller wants (Union[A, B], True), not (Union[A, B], False) — so :class:yggdrasil.data.Field can stamp nullable=True without losing the multi-arm shape.

is_runtime_value `staticmethod` ¶

is_runtime_value(x: Any) -> bool

True for runtime values (42, [], MyClass()).

False for type hints — distinguishes convert(42, int) (runtime value with target hint) from convert(int, str) (two hints, no value). Used to gate dispatch decisions in downstream tooling.

resolve_str_annotation `classmethod` ¶

resolve_str_annotation(
    s: str, func_globals: Optional[dict[str, Any]] = None
) -> Any

Resolve a string annotation to a real type.

Tries, in order:

eval in func_globals + builtins — picks up local imports and aliases declared in the function's own module.
eval against typing for generic shapes (Optional[int], list[int]) when func_globals doesn't have them in scope.
Alias-prefix expansion via :meth:expand_alias + dotted importlib.import_module lookup so pa.Table / pl.DataFrame resolve without the function's globals having ever imported the alias.

Returns s verbatim when every path fails — callers treat that as "unresolved, skip coercion / leave as-is".

resolve_function_annotations `classmethod` ¶

resolve_function_annotations(func: 'Callable[..., Any]') -> dict[str, Any]

Resolve every annotation on func to a real type.

Two-pass best effort so a single unresolvable annotation doesn't blow up the rest:

:func:inspect.get_annotations with eval_str=True — evaluates every annotation in the function's globals + builtins in one shot.
Per-annotation :meth:resolve_str_annotation fallback for entries the fast path left as strings (it silently returns the literal when its eval misses).

Anything still a string after both passes is left untouched — callers treat it as "couldn't resolve, skip coercion" rather than raising.

from_pytype `classmethod` ¶

from_pytype(hint: Any) -> 'DataType'

Parse a Python annotation into a :class:DataType.

Stamps the resulting instance's _pyhint_cache with the original hint when the canonical reconstruction (:meth:_default_pyhint) wouldn't round-trip — so from_pytype(MyDataclass).to_pyhint() returns MyDataclass itself rather than a generic struct hint, from_pytype(MyEnum).to_pyhint() returns the enum class, and from_pytype(np.int64).to_pyhint() returns np.int64 rather than the canonical int.

cast_arrow_batch_iterator ¶

cast_arrow_batch_iterator(
    batches: "Iterable[pa.RecordBatch]",
    options: "CastOptions | None" = None,
    **more_options
) -> "Iterator[pa.RecordBatch]"

Cast a stream of :class:pa.RecordBatch against this dtype.

Non-struct dtypes promote to a single-column struct via :meth:to_struct and reuse the struct's iterator helper.

cast_polars_expr ¶

cast_polars_expr(
    series: "polars.Series | polars.Expr",
    options: "CastOptions | None" = None,
    **more_options
) -> "polars.Series | polars.Expr"

Expr-shape passthrough to :meth:cast_polars_series.

:meth:cast_polars_series already dispatches by isinstance — this method exists so callers that know they hold an Expr can name it.

arrow_cast ¶

arrow_cast(
    array: "pa.Array | pa.ChunkedArray",
    target: "pa.DataType",
    *,
    safe: bool = False
) -> "pa.Array | pa.ChunkedArray"

Cast an Arrow array to target by routing through polars.

Arrow array → polars Series → cast_polars_array_to_temporal → Arrow array. Polars' native chrono parser handles ISO-8601 strings, unit conversion, and wall-clock tz reinterpret in one pass.

Falls back to pc.cast when:

the target is non-temporal,
polars can't represent the target (second-precision Datetime / Duration), or
the cast is a same-family unit conversion where pyarrow's cast semantics already match polars (no tz reinterpret, no string parsing) — see :func:_pc_cast_equivalent_to_polars.

cast_polars_array_to_temporal ¶

cast_polars_array_to_temporal(
    array: Union["pl.Series", "pl.Expr"],
    source: Any,
    target: Any,
    safe: bool,
    source_tz: str | None = None,
    target_tz: str | None = None,
    to_expr: bool = False,
    parent_name: str | None = None,
) -> Union["pl.Series", "pl.Expr"]

Cast a polars Series / Expr to a temporal dtype via the native cast.

Strings parse through str.to_datetime / str.to_date / str.to_time with default ISO format. Naive→aware uses replace_time_zone (wall-clock reinterpret). Everything else is .cast(target).

spark_cast ¶

spark_cast(
    column: "ps.Column",
    target: Any,
    *,
    safe: bool = False,
    unit: str = "us",
    tz: str | None = None
) -> "ps.Column"

Cast a Spark column to target via native column.cast.

Strings cast straight — Spark's cast(TimestampType()) accepts ISO inputs natively, and rejects everything else (returns null in best-effort mode, raises in strict).

yggdrasil.data.types.primitive.temporal¶

temporal ¶

TemporalType dataclass ¶

select_fields ¶

pretty_format abstractmethod ¶

short ¶

to_pyhint ¶

expand_alias classmethod ¶

strip_annotated staticmethod ¶

unwrap_newtype staticmethod ¶

normalize_hint classmethod ¶

unwrap_optional classmethod ¶

unwrap_nullable_hint classmethod ¶

is_runtime_value staticmethod ¶

resolve_str_annotation classmethod ¶

resolve_function_annotations classmethod ¶

as_polars ¶

as_spark ¶

from_pytype classmethod ¶

cast_arrow_batch_iterator ¶

cast_polars_expr ¶

TimestampType dataclass ¶

tz_iana property ¶

select_fields ¶

short ¶

to_pyhint ¶

expand_alias classmethod ¶

strip_annotated staticmethod ¶

unwrap_newtype staticmethod ¶

normalize_hint classmethod ¶

unwrap_optional classmethod ¶

unwrap_nullable_hint classmethod ¶

is_runtime_value staticmethod ¶

resolve_str_annotation classmethod ¶

resolve_function_annotations classmethod ¶

from_pytype classmethod ¶

cast_arrow_batch_iterator ¶

cast_polars_expr ¶

arrow_cast ¶

cast_polars_array_to_temporal ¶

spark_cast ¶

`yggdrasil.data.types.primitive.temporal`¶

TemporalType `dataclass` ¶

pretty_format `abstractmethod` ¶

expand_alias `classmethod` ¶

strip_annotated `staticmethod` ¶

unwrap_newtype `staticmethod` ¶

normalize_hint `classmethod` ¶

unwrap_optional `classmethod` ¶

unwrap_nullable_hint `classmethod` ¶

is_runtime_value `staticmethod` ¶

resolve_str_annotation `classmethod` ¶

resolve_function_annotations `classmethod` ¶

from_pytype `classmethod` ¶

TimestampType `dataclass` ¶

tz_iana `property` ¶

expand_alias `classmethod` ¶

strip_annotated `staticmethod` ¶

unwrap_newtype `staticmethod` ¶

normalize_hint `classmethod` ¶

unwrap_optional `classmethod` ¶

unwrap_nullable_hint `classmethod` ¶

is_runtime_value `staticmethod` ¶

resolve_str_annotation `classmethod` ¶

resolve_function_annotations `classmethod` ¶

from_pytype `classmethod` ¶