yggdrasil.data.types.nested

nested

DataType dataclass

DataType()

Bases: BaseChildrenFields, ABC

select_fields

select_fields(
    identifiers: "SelectType | Iterable[SelectType]" = (),
    *others: SelectType,
    raise_error: bool = True
) -> list["Field"]

Resolve one or more identifiers into the matching :class:Field objects.

Accepts strings (resolved by name), ints (resolved by index), and existing :class:Field instances (resolved by .name against this container — so callers can copy a field set between sibling schemas without first stringifying everything).

Calling shapes that all work the same way:

• schema.select_fields("price") — single identifier.
• schema.select_fields("price", "qty", 0) — multiple positionals.
• schema.select_fields(["price", "qty"]) — single iterable.
• schema.select_fields(other_schema.children) — copy a sibling's fields by name into this schema.
• schema.select_fields("price", ["qty", "ts"], 0) — mixed; each positional is itself flattened so iterables and scalars can be interleaved.

:param identifiers: First identifier or iterable of identifiers. :param others: Additional identifiers. Each is flattened the same way as the first. :param raise_error: True (default) — missing identifiers raise via :meth:field_by with the same suggestion-rich error message used elsewhere. False — missing identifiers yield None in the returned list, preserving caller order.

:returns: A list of :class:Field (or Field | None when raise_error=False), one entry per resolved identifier in caller order. Duplicates in the input produce duplicates in the output — this is intentional, since select is the natural place to express a projection and projections sometimes repeat columns.

:raises KeyError: With suggestions, when raise_error is True and an identifier doesn't resolve. :raises TypeError: When an identifier is not a str / int / Field.

pretty_format abstractmethod

pretty_format(indent: int = 2, level: int = 0) -> str

Pretty-print this dtype with one element per line.

indent is the per-level step in spaces. level is the current depth — the line is prefixed with indent * level spaces. Flat dtypes render as a single line; nested dtypes (struct / list / map) override this to lay each child out on its own line at level + 1.

short

short(depth: int = 2) -> str

A compact, single-line type tag — recursive (bounded) for nested types.

Scalars render as i64 / f64 / str / bool / date / ts / dec / bin / json; nested types recurse through their own children — list<i64>, struct<x:i64, y:str>, map<str,f64>, even list<struct<id:i64>> — until depth runs out (then a bare list / struct / map). Built from this type's :attr:type_id + :attr:children, not any one engine. Struct fields are capped and the whole tag is elided past :data:_SHORT_TAG_MAX chars, so a deep schema can't widen a :meth:yggdrasil.io.tabular.Tabular.display header forever.

to_pyhint

to_pyhint() -> Any

Return the Python type hint that maps back to this DataType.

Cached when :meth:from_pytype stamped an explicit hint on the instance (preserves user-defined dataclasses, enum classes, numpy.int64 and other narrow aliases the canonical reconstruction would collapse). Otherwise falls back to :meth:_default_pyhint, the subclass hook that builds a canonical hint from the dtype's own state.

expand_alias classmethod

expand_alias(name: str) -> str

Expand a known short-alias prefix.

pa.Table → pyarrow.Table, pl.DataFrame → polars.DataFrame, etc. Returns name unchanged when no registered prefix matches. The table lives at :attr:PYHINT_ALIASES.

strip_annotated staticmethod

strip_annotated(hint: Any) -> Any

Strip Annotated[T, ...] down to T (recursive).

unwrap_newtype staticmethod

unwrap_newtype(hint: Any) -> Any

Unwrap a chain of NewType aliases to the base type.

normalize_hint classmethod

normalize_hint(hint: Any) -> Any

strip_annotated then unwrap_newtype — common preamble.

Every from_pytype / unwrap_* flow needs the hint stripped of Annotated metadata and the NewType chain unwrapped before the real dispatch. Bundled into one classmethod so the ordering ("strip Annotated before NewType, recursive on both") lives in one place.

unwrap_optional classmethod

unwrap_optional(hint: Any) -> tuple[bool, Any]

Return (is_optional, inner) for Optional[T] / T | None.

Only collapses a Union whose non-None arms reduce to exactly one type — Optional[int] → (True, int), int | None → (True, int). Multi-type unions return (False, hint) so callers can decide how to handle them (the cast registry generally falls through to StringType).

unwrap_nullable_hint classmethod

unwrap_nullable_hint(hint: Any) -> tuple[Any, bool]

Field-flavoured Optional unwrap: (inner, has_null).

Differences from :meth:unwrap_optional:

• String hints route through :class:ParsedDataType.from_ so the field name / nullability tag baked into the DSL form (e.g. "int64?") survives.
• Multi-type unions stay intact — the caller wants (Union[A, B], True), not (Union[A, B], False) — so :class:yggdrasil.data.Field can stamp nullable=True without losing the multi-arm shape.

is_runtime_value staticmethod

is_runtime_value(x: Any) -> bool

True for runtime values (42, [], MyClass()).

False for type hints — distinguishes convert(42, int) (runtime value with target hint) from convert(int, str) (two hints, no value). Used to gate dispatch decisions in downstream tooling.

resolve_str_annotation classmethod

resolve_str_annotation(
    s: str, func_globals: Optional[dict[str, Any]] = None
) -> Any

Resolve a string annotation to a real type.

Tries, in order:

1. eval in func_globals + builtins — picks up local imports and aliases declared in the function's own module.
2. eval against typing for generic shapes (Optional[int], list[int]) when func_globals doesn't have them in scope.
3. Alias-prefix expansion via :meth:expand_alias + dotted importlib.import_module lookup so pa.Table / pl.DataFrame resolve without the function's globals having ever imported the alias.

Returns s verbatim when every path fails — callers treat that as "unresolved, skip coercion / leave as-is".

resolve_function_annotations classmethod

resolve_function_annotations(func: 'Callable[..., Any]') -> dict[str, Any]

Resolve every annotation on func to a real type.

Two-pass best effort so a single unresolvable annotation doesn't blow up the rest:

1. :func:inspect.get_annotations with eval_str=True — evaluates every annotation in the function's globals + builtins in one shot.
2. Per-annotation :meth:resolve_str_annotation fallback for entries the fast path left as strings (it silently returns the literal when its eval misses).

Anything still a string after both passes is left untouched — callers treat it as "couldn't resolve, skip coercion" rather than raising.

as_polars

as_polars() -> 'DataType'

Return a Polars-flavored :class:DataType for this type.

Same shape as :meth:as_spark — stays on the yggdrasil side of the boundary and returns a :class:DataType whose :meth:to_polars lands on a dtype Polars natively represents. Defaults to self; subclasses Polars can't store at their declared width / precision override:

• Float8Type and Float16Type widen to Float32Type (Polars has no sub-32-bit floats);
• TimestampType / DurationType with second-precision (unit="s") widen to unit="ms" (Polars supports ms / us / ns only);
• nested types (ArrayType / MapType / StructType) recurse via as_polars on their child fields.

:class:Field and :class:Schema expose a matching as_polars that delegates to self.dtype.as_polars and re-wraps so callers chain through Field-shaped APIs without dropping back to a plain :class:DataType.

as_spark

as_spark() -> 'DataType'

Return a Spark-flavored :class:DataType for this type.

as_spark lives on the yggdrasil side of the boundary: it returns a :class:DataType that maps cleanly to a Spark dtype (i.e. one self.to_spark() would round-trip without a widening-time surprise). For types Spark already represents natively (signed ints, Float32 / Float64, Date, String / Binary / Boolean, decimal, naive / UTC timestamps), the default is to return self unchanged.

Subclasses Spark cannot represent natively (IntegerType with signed=False, Float16Type, DurationType, TimeType, non-UTC TimestampType) override this to return the closest Spark-compatible yggdrasil dtype — usually a widened integer, a StringType, or a naive timestamp. Nested types (ArrayType / MapType / StructType) recurse via as_spark on their child fields so the whole tree comes back Spark-compatible in one call.

autotag

autotag() -> dict[bytes, bytes]

Return a dict of Databricks-friendly tags derived from this type.

These are auto-tags: they describe shape, not intent. The base output is a single kind key — a lowercase form of type_id ("integer", "string", "timestamp", "array", ...) that tag-based Unity Catalog policies can match on. Subclasses extend this with dtype-specific detail (unit, tz, precision / scale, signed, srid, ...) — always via super().autotag() so the kind key stays present.

Keys are bare (no t: prefix) — prefixing is handled by BaseMetadata.update_tags when these land on a Field.

from_pytype classmethod

from_pytype(hint: Any) -> 'DataType'

Parse a Python annotation into a :class:DataType.

Stamps the resulting instance's _pyhint_cache with the original hint when the canonical reconstruction (:meth:_default_pyhint) wouldn't round-trip — so from_pytype(MyDataclass).to_pyhint() returns MyDataclass itself rather than a generic struct hint, from_pytype(MyEnum).to_pyhint() returns the enum class, and from_pytype(np.int64).to_pyhint() returns np.int64 rather than the canonical int.

cast_arrow_batch_iterator

cast_arrow_batch_iterator(
    batches: "Iterable[pa.RecordBatch]",
    options: "CastOptions | None" = None,
    **more_options
) -> "Iterator[pa.RecordBatch]"

Cast a stream of :class:pa.RecordBatch against this dtype.

Non-struct dtypes promote to a single-column struct via :meth:to_struct and reuse the struct's iterator helper.

cast_polars_expr

cast_polars_expr(
    series: "polars.Series | polars.Expr",
    options: "CastOptions | None" = None,
    **more_options
) -> "polars.Series | polars.Expr"

Expr-shape passthrough to :meth:cast_polars_series.

:meth:cast_polars_series already dispatches by isinstance — this method exists so callers that know they hold an Expr can name it.

NestedType dataclass

NestedType()

Bases: DataType, ABC

Base class for nested data types (struct, map, array).

Nested types expose their inner fields via children and therefore share a common equals implementation that compares their children pairwise.

select_fields

select_fields(
    identifiers: "SelectType | Iterable[SelectType]" = (),
    *others: SelectType,
    raise_error: bool = True
) -> list["Field"]

Resolve one or more identifiers into the matching :class:Field objects.

Accepts strings (resolved by name), ints (resolved by index), and existing :class:Field instances (resolved by .name against this container — so callers can copy a field set between sibling schemas without first stringifying everything).

Calling shapes that all work the same way:

• schema.select_fields("price") — single identifier.
• schema.select_fields("price", "qty", 0) — multiple positionals.
• schema.select_fields(["price", "qty"]) — single iterable.
• schema.select_fields(other_schema.children) — copy a sibling's fields by name into this schema.
• schema.select_fields("price", ["qty", "ts"], 0) — mixed; each positional is itself flattened so iterables and scalars can be interleaved.

:param identifiers: First identifier or iterable of identifiers. :param others: Additional identifiers. Each is flattened the same way as the first. :param raise_error: True (default) — missing identifiers raise via :meth:field_by with the same suggestion-rich error message used elsewhere. False — missing identifiers yield None in the returned list, preserving caller order.

:returns: A list of :class:Field (or Field | None when raise_error=False), one entry per resolved identifier in caller order. Duplicates in the input produce duplicates in the output — this is intentional, since select is the natural place to express a projection and projections sometimes repeat columns.

:raises KeyError: With suggestions, when raise_error is True and an identifier doesn't resolve. :raises TypeError: When an identifier is not a str / int / Field.

pretty_format abstractmethod

pretty_format(indent: int = 2, level: int = 0) -> str

Pretty-print this dtype with one element per line.

indent is the per-level step in spaces. level is the current depth — the line is prefixed with indent * level spaces. Flat dtypes render as a single line; nested dtypes (struct / list / map) override this to lay each child out on its own line at level + 1.

short

short(depth: int = 2) -> str

A compact, single-line type tag — recursive (bounded) for nested types.

Scalars render as i64 / f64 / str / bool / date / ts / dec / bin / json; nested types recurse through their own children — list<i64>, struct<x:i64, y:str>, map<str,f64>, even list<struct<id:i64>> — until depth runs out (then a bare list / struct / map). Built from this type's :attr:type_id + :attr:children, not any one engine. Struct fields are capped and the whole tag is elided past :data:_SHORT_TAG_MAX chars, so a deep schema can't widen a :meth:yggdrasil.io.tabular.Tabular.display header forever.

to_pyhint

to_pyhint() -> Any

Return the Python type hint that maps back to this DataType.

Cached when :meth:from_pytype stamped an explicit hint on the instance (preserves user-defined dataclasses, enum classes, numpy.int64 and other narrow aliases the canonical reconstruction would collapse). Otherwise falls back to :meth:_default_pyhint, the subclass hook that builds a canonical hint from the dtype's own state.

expand_alias classmethod

expand_alias(name: str) -> str

Expand a known short-alias prefix.

pa.Table → pyarrow.Table, pl.DataFrame → polars.DataFrame, etc. Returns name unchanged when no registered prefix matches. The table lives at :attr:PYHINT_ALIASES.

strip_annotated staticmethod

strip_annotated(hint: Any) -> Any

Strip Annotated[T, ...] down to T (recursive).

unwrap_newtype staticmethod

unwrap_newtype(hint: Any) -> Any

Unwrap a chain of NewType aliases to the base type.

normalize_hint classmethod

normalize_hint(hint: Any) -> Any

strip_annotated then unwrap_newtype — common preamble.

Every from_pytype / unwrap_* flow needs the hint stripped of Annotated metadata and the NewType chain unwrapped before the real dispatch. Bundled into one classmethod so the ordering ("strip Annotated before NewType, recursive on both") lives in one place.

unwrap_optional classmethod

unwrap_optional(hint: Any) -> tuple[bool, Any]

Return (is_optional, inner) for Optional[T] / T | None.

Only collapses a Union whose non-None arms reduce to exactly one type — Optional[int] → (True, int), int | None → (True, int). Multi-type unions return (False, hint) so callers can decide how to handle them (the cast registry generally falls through to StringType).

unwrap_nullable_hint classmethod

unwrap_nullable_hint(hint: Any) -> tuple[Any, bool]

Field-flavoured Optional unwrap: (inner, has_null).

Differences from :meth:unwrap_optional:

• String hints route through :class:ParsedDataType.from_ so the field name / nullability tag baked into the DSL form (e.g. "int64?") survives.
• Multi-type unions stay intact — the caller wants (Union[A, B], True), not (Union[A, B], False) — so :class:yggdrasil.data.Field can stamp nullable=True without losing the multi-arm shape.

is_runtime_value staticmethod

is_runtime_value(x: Any) -> bool

True for runtime values (42, [], MyClass()).

False for type hints — distinguishes convert(42, int) (runtime value with target hint) from convert(int, str) (two hints, no value). Used to gate dispatch decisions in downstream tooling.

resolve_str_annotation classmethod

resolve_str_annotation(
    s: str, func_globals: Optional[dict[str, Any]] = None
) -> Any

Resolve a string annotation to a real type.

Tries, in order:

1. eval in func_globals + builtins — picks up local imports and aliases declared in the function's own module.
2. eval against typing for generic shapes (Optional[int], list[int]) when func_globals doesn't have them in scope.
3. Alias-prefix expansion via :meth:expand_alias + dotted importlib.import_module lookup so pa.Table / pl.DataFrame resolve without the function's globals having ever imported the alias.

Returns s verbatim when every path fails — callers treat that as "unresolved, skip coercion / leave as-is".

resolve_function_annotations classmethod

resolve_function_annotations(func: 'Callable[..., Any]') -> dict[str, Any]

Resolve every annotation on func to a real type.

Two-pass best effort so a single unresolvable annotation doesn't blow up the rest:

1. :func:inspect.get_annotations with eval_str=True — evaluates every annotation in the function's globals + builtins in one shot.
2. Per-annotation :meth:resolve_str_annotation fallback for entries the fast path left as strings (it silently returns the literal when its eval misses).

Anything still a string after both passes is left untouched — callers treat it as "couldn't resolve, skip coercion" rather than raising.

as_polars

as_polars() -> 'DataType'

Return a Polars-flavored :class:DataType for this type.

Same shape as :meth:as_spark — stays on the yggdrasil side of the boundary and returns a :class:DataType whose :meth:to_polars lands on a dtype Polars natively represents. Defaults to self; subclasses Polars can't store at their declared width / precision override:

• Float8Type and Float16Type widen to Float32Type (Polars has no sub-32-bit floats);
• TimestampType / DurationType with second-precision (unit="s") widen to unit="ms" (Polars supports ms / us / ns only);
• nested types (ArrayType / MapType / StructType) recurse via as_polars on their child fields.

:class:Field and :class:Schema expose a matching as_polars that delegates to self.dtype.as_polars and re-wraps so callers chain through Field-shaped APIs without dropping back to a plain :class:DataType.

as_spark

as_spark() -> 'DataType'

Return a Spark-flavored :class:DataType for this type.

as_spark lives on the yggdrasil side of the boundary: it returns a :class:DataType that maps cleanly to a Spark dtype (i.e. one self.to_spark() would round-trip without a widening-time surprise). For types Spark already represents natively (signed ints, Float32 / Float64, Date, String / Binary / Boolean, decimal, naive / UTC timestamps), the default is to return self unchanged.

Subclasses Spark cannot represent natively (IntegerType with signed=False, Float16Type, DurationType, TimeType, non-UTC TimestampType) override this to return the closest Spark-compatible yggdrasil dtype — usually a widened integer, a StringType, or a naive timestamp. Nested types (ArrayType / MapType / StructType) recurse via as_spark on their child fields so the whole tree comes back Spark-compatible in one call.

autotag

autotag() -> dict[bytes, bytes]

Return a dict of Databricks-friendly tags derived from this type.

These are auto-tags: they describe shape, not intent. The base output is a single kind key — a lowercase form of type_id ("integer", "string", "timestamp", "array", ...) that tag-based Unity Catalog policies can match on. Subclasses extend this with dtype-specific detail (unit, tz, precision / scale, signed, srid, ...) — always via super().autotag() so the kind key stays present.

Keys are bare (no t: prefix) — prefixing is handled by BaseMetadata.update_tags when these land on a Field.

from_pytype classmethod

from_pytype(hint: Any) -> 'DataType'

Parse a Python annotation into a :class:DataType.

Stamps the resulting instance's _pyhint_cache with the original hint when the canonical reconstruction (:meth:_default_pyhint) wouldn't round-trip — so from_pytype(MyDataclass).to_pyhint() returns MyDataclass itself rather than a generic struct hint, from_pytype(MyEnum).to_pyhint() returns the enum class, and from_pytype(np.int64).to_pyhint() returns np.int64 rather than the canonical int.

cast_arrow_batch_iterator

cast_arrow_batch_iterator(
    batches: "Iterable[pa.RecordBatch]",
    options: "CastOptions | None" = None,
    **more_options
) -> "Iterator[pa.RecordBatch]"

Cast a stream of :class:pa.RecordBatch against this dtype.

Non-struct dtypes promote to a single-column struct via :meth:to_struct and reuse the struct's iterator helper.

cast_polars_expr

cast_polars_expr(
    series: "polars.Series | polars.Expr",
    options: "CastOptions | None" = None,
    **more_options
) -> "polars.Series | polars.Expr"

Expr-shape passthrough to :meth:cast_polars_series.

:meth:cast_polars_series already dispatches by isinstance — this method exists so callers that know they hold an Expr can name it.

StructType dataclass

StructType(fields: tuple['Field'] = tuple())

Bases: NestedType

select_fields

select_fields(
    identifiers: "SelectType | Iterable[SelectType]" = (),
    *others: SelectType,
    raise_error: bool = True
) -> list["Field"]

Resolve one or more identifiers into the matching :class:Field objects.

Accepts strings (resolved by name), ints (resolved by index), and existing :class:Field instances (resolved by .name against this container — so callers can copy a field set between sibling schemas without first stringifying everything).

Calling shapes that all work the same way:

• schema.select_fields("price") — single identifier.
• schema.select_fields("price", "qty", 0) — multiple positionals.
• schema.select_fields(["price", "qty"]) — single iterable.
• schema.select_fields(other_schema.children) — copy a sibling's fields by name into this schema.
• schema.select_fields("price", ["qty", "ts"], 0) — mixed; each positional is itself flattened so iterables and scalars can be interleaved.

:param identifiers: First identifier or iterable of identifiers. :param others: Additional identifiers. Each is flattened the same way as the first. :param raise_error: True (default) — missing identifiers raise via :meth:field_by with the same suggestion-rich error message used elsewhere. False — missing identifiers yield None in the returned list, preserving caller order.

:returns: A list of :class:Field (or Field | None when raise_error=False), one entry per resolved identifier in caller order. Duplicates in the input produce duplicates in the output — this is intentional, since select is the natural place to express a projection and projections sometimes repeat columns.

:raises KeyError: With suggestions, when raise_error is True and an identifier doesn't resolve. :raises TypeError: When an identifier is not a str / int / Field.

short

short(depth: int = 2) -> str

A compact, single-line type tag — recursive (bounded) for nested types.

Scalars render as i64 / f64 / str / bool / date / ts / dec / bin / json; nested types recurse through their own children — list<i64>, struct<x:i64, y:str>, map<str,f64>, even list<struct<id:i64>> — until depth runs out (then a bare list / struct / map). Built from this type's :attr:type_id + :attr:children, not any one engine. Struct fields are capped and the whole tag is elided past :data:_SHORT_TAG_MAX chars, so a deep schema can't widen a :meth:yggdrasil.io.tabular.Tabular.display header forever.

to_pyhint

to_pyhint() -> Any

Return the Python type hint that maps back to this DataType.

Cached when :meth:from_pytype stamped an explicit hint on the instance (preserves user-defined dataclasses, enum classes, numpy.int64 and other narrow aliases the canonical reconstruction would collapse). Otherwise falls back to :meth:_default_pyhint, the subclass hook that builds a canonical hint from the dtype's own state.

expand_alias classmethod

expand_alias(name: str) -> str

Expand a known short-alias prefix.

pa.Table → pyarrow.Table, pl.DataFrame → polars.DataFrame, etc. Returns name unchanged when no registered prefix matches. The table lives at :attr:PYHINT_ALIASES.

strip_annotated staticmethod

strip_annotated(hint: Any) -> Any

Strip Annotated[T, ...] down to T (recursive).

unwrap_newtype staticmethod

unwrap_newtype(hint: Any) -> Any

Unwrap a chain of NewType aliases to the base type.

normalize_hint classmethod

normalize_hint(hint: Any) -> Any

strip_annotated then unwrap_newtype — common preamble.

Every from_pytype / unwrap_* flow needs the hint stripped of Annotated metadata and the NewType chain unwrapped before the real dispatch. Bundled into one classmethod so the ordering ("strip Annotated before NewType, recursive on both") lives in one place.

unwrap_optional classmethod

unwrap_optional(hint: Any) -> tuple[bool, Any]

Return (is_optional, inner) for Optional[T] / T | None.

Only collapses a Union whose non-None arms reduce to exactly one type — Optional[int] → (True, int), int | None → (True, int). Multi-type unions return (False, hint) so callers can decide how to handle them (the cast registry generally falls through to StringType).

unwrap_nullable_hint classmethod

unwrap_nullable_hint(hint: Any) -> tuple[Any, bool]

Field-flavoured Optional unwrap: (inner, has_null).

Differences from :meth:unwrap_optional:

• String hints route through :class:ParsedDataType.from_ so the field name / nullability tag baked into the DSL form (e.g. "int64?") survives.
• Multi-type unions stay intact — the caller wants (Union[A, B], True), not (Union[A, B], False) — so :class:yggdrasil.data.Field can stamp nullable=True without losing the multi-arm shape.

is_runtime_value staticmethod

is_runtime_value(x: Any) -> bool

True for runtime values (42, [], MyClass()).

False for type hints — distinguishes convert(42, int) (runtime value with target hint) from convert(int, str) (two hints, no value). Used to gate dispatch decisions in downstream tooling.

resolve_str_annotation classmethod

resolve_str_annotation(
    s: str, func_globals: Optional[dict[str, Any]] = None
) -> Any

Resolve a string annotation to a real type.

Tries, in order:

1. eval in func_globals + builtins — picks up local imports and aliases declared in the function's own module.
2. eval against typing for generic shapes (Optional[int], list[int]) when func_globals doesn't have them in scope.
3. Alias-prefix expansion via :meth:expand_alias + dotted importlib.import_module lookup so pa.Table / pl.DataFrame resolve without the function's globals having ever imported the alias.

Returns s verbatim when every path fails — callers treat that as "unresolved, skip coercion / leave as-is".

resolve_function_annotations classmethod

resolve_function_annotations(func: 'Callable[..., Any]') -> dict[str, Any]

Resolve every annotation on func to a real type.

Two-pass best effort so a single unresolvable annotation doesn't blow up the rest:

1. :func:inspect.get_annotations with eval_str=True — evaluates every annotation in the function's globals + builtins in one shot.
2. Per-annotation :meth:resolve_str_annotation fallback for entries the fast path left as strings (it silently returns the literal when its eval misses).

Anything still a string after both passes is left untouched — callers treat it as "couldn't resolve, skip coercion" rather than raising.

autotag

autotag() -> dict[bytes, bytes]

Return a dict of Databricks-friendly tags derived from this type.

These are auto-tags: they describe shape, not intent. The base output is a single kind key — a lowercase form of type_id ("integer", "string", "timestamp", "array", ...) that tag-based Unity Catalog policies can match on. Subclasses extend this with dtype-specific detail (unit, tz, precision / scale, signed, srid, ...) — always via super().autotag() so the kind key stays present.

Keys are bare (no t: prefix) — prefixing is handled by BaseMetadata.update_tags when these land on a Field.

from_pytype classmethod

from_pytype(hint: Any) -> 'DataType'

Parse a Python annotation into a :class:DataType.

Stamps the resulting instance's _pyhint_cache with the original hint when the canonical reconstruction (:meth:_default_pyhint) wouldn't round-trip — so from_pytype(MyDataclass).to_pyhint() returns MyDataclass itself rather than a generic struct hint, from_pytype(MyEnum).to_pyhint() returns the enum class, and from_pytype(np.int64).to_pyhint() returns np.int64 rather than the canonical int.

cast_arrow_batch_iterator

cast_arrow_batch_iterator(
    batches: "Iterable[pa.RecordBatch]",
    options: "CastOptions | None" = None,
    **more_options
) -> "Iterator[pa.RecordBatch]"

Cast a stream of :class:pa.RecordBatch against this dtype.

Non-struct dtypes promote to a single-column struct via :meth:to_struct and reuse the struct's iterator helper.

cast_polars_expr

cast_polars_expr(
    series: "polars.Series | polars.Expr",
    options: "CastOptions | None" = None,
    **more_options
) -> "polars.Series | polars.Expr"

Expr-shape passthrough to :meth:cast_polars_series.

:meth:cast_polars_series already dispatches by isinstance — this method exists so callers that know they hold an Expr can name it.