Skip to content

yggdrasil.execution.expr.partition

partition

Partition-pruning extractor — over-approximate per-column accepted sets.

Engines that partition by a finite key set (Delta, Iceberg, Hive-style folder layouts) want a quick "which files can I skip" answer before any parquet open. The full :func:Expression.to_python / to_arrow evaluator filters rows; this extractor walks the predicate once and returns the set of partition-column values that could satisfy it. The returned dict is consumed by :meth:Snapshot.prune_files (and any other partition-aware reader) — the row-level predicate still runs on the surviving files, so the extractor is allowed to over-approximate.

extract_partition_filters 

extract_partition_filters(
    expr: Expression, columns: "Iterable[str]"
) -> "dict[str, frozenset]"

Over-approximate per-column accepted-value sets from a predicate.

Walks expr and returns, for each column in columns that the predicate constrains to a finite set, the :class:frozenset of values the column could take in any row the predicate accepts. Columns not in the returned dict are unconstrained — the predicate doesn't restrict their value to a finite, enumerable set.

The result is suitable for partition pruning: a file whose partition value for col isn't in the extracted set can be skipped. It is over-approximate — a file the constraints accept may still produce zero matching rows (the row-level filter catches the residual), but no row the predicate accepts can fall outside the constraints. That makes the extractor safe to use as a pre-filter before the row-level scan.

Supported shapes:

  • col == v: {col: {v}}.
  • col.is_in([v1, v2]): {col: {v1, v2}}. includes_null=True adds None to the set.
  • col.is_null(): {col: {None}}.
  • AND: per-column intersection of constraints. Columns constrained on only one side keep their original set.
  • OR: per-column union, but only for columns constrained on every operand (one unconstrained operand drops the column — the OR could accept any value via that branch).

Returns {} for NOT, ranges (< / <= / > / >= / BETWEEN), LIKE, !=, arithmetic on column references, column-vs-column comparisons, and col == NULL (always UNKNOWN in SQL — never accepts a row).

A returned {col: frozenset()} means the predicate is unsatisfiable on that column — the caller can skip every file whose partition value for col exists.