`yggdrasil.databricks.table.insert`¶

insert ¶

Databricks table insert — the one place insert DML is generated.

This module centralizes everything about loading rows into a Unity Catalog table for the synchronous warehouse / Spark / SQL paths:

:class:DatabricksTableInsert is the full description of one insert operation — target, mode, the staged source, plus the keyed-write surface (match_by, update_column_names, predicate, zorder_by, maintenance flags, safe_merge). It is :class:Awaitable and self-executing: :meth:~DatabricksTableInsert.execute renders its own INSERT / MERGE statement list and runs it via execute_many.
:func:make_sql_select renders the per-op SELECT over its staged source.
:func:make_sql_insert renders the full INSERT / MERGE / DELETE+INSERT / TRUNCATE / OPTIMIZE / VACUUM statement list.

Table.arrow_insert stages the rows as Parquet to a Volume, builds a :class:DatabricksTableInsert over that staged file, and calls :meth:~DatabricksTableInsert.execute.

DatabricksTableInsert `dataclass` ¶

DatabricksTableInsert(
    target: "Table | str",
    mode: Mode,
    data: "Tabular | Path | str",
    client: "DatabricksClient | None" = None,
    schema: "Field | None" = None,
    predicate: "Predicate | None" = None,
    match_by: "list[str] | None" = None,
    update_column_names: "list[str] | None" = None,
    schema_mode: "Mode | str | None" = None,
    zorder_by: "list[str] | None" = None,
    optimize_after_merge: bool = False,
    vacuum_hours: "int | None" = None,
    safe_merge: bool = False,
)

Bases: _InsertExecution

One insert operation — the full arrow_insert surface in one object.

Carries the target table, the save mode, the staged data location (a :class:Path / :class:VolumePath, or a uniform-URL string), and the keyed-write surface (schema, predicate, match_by, update_column_names, schema_mode, zorder_by, optimize_after_merge, vacuum_hours, safe_merge).

target may be a :class:Table or its full name; data is the staged Parquet source — a :class:Path / :class:VolumePath, or its uniform URL as a string (reconstructed through the bound client at execute time).

target_name `property` ¶

target_name: str

catalog.schema.table — the resolved target name.

result `property` ¶

result: Any

The inner :class:StatementBatch driving the load (None until :meth:start / :meth:execute).

data_path ¶

data_path(client: Any = None) -> 'Path'

Resolve the staged file data to a concrete :class:Path.

Already a :class:Path → returned as-is; a uniform-URL string → reconstructed through the bound (or supplied) client so the warehouse can read it wherever it landed.

staged_source ¶

staged_source(client: Any = None) -> Any

Rebuild the staged data into the concrete :class:Path the warehouse reads. A live :class:Path is returned unchanged; a serialized URL is rebuilt through the bound (or supplied) client.

select_sql ¶

select_sql(client: Any = None) -> str

Back-compat alias for :func:make_sql_select over this op.

progress ¶

progress() -> 'float | None'

A 0..1 completion fraction for a progress bar, or None if unknown.

A UI hook: a generic awaitable can't know its fraction, so the base returns None (drive a spinner, not a bar). Subclasses that do know — a batch's children done, a statement's rows fetched — override this. Consumed by :func:yggdrasil.cli.style.track.

watch ¶

watch(
    on_tick: "Any" = None, *, interval: float = 0.1, raise_error: bool = True
) -> "Awaitable"

Drive to completion, calling on_tick(self) each poll.

The hook a UI (spinner / progress bar) connects to without this trait importing any UI — keeping the layering clean. Starts the awaitable if it hasn't been, polls until done, then surfaces a failure (unless raise_error is False). Pairs with :func:yggdrasil.cli.style.track.

execute ¶

execute(
    *,
    target: Any = None,
    wait: WaitingConfigArg = True,
    raise_error: bool = True,
    engine: Any = None,
    retry: WaitingConfigArg | None = None
) -> "_InsertExecution"

Build and run this insert. target rebinds the destination :class:Table; engine forces the "api" / "spark" backend, retry a per-statement retry policy. With wait (default) blocks until the statements finish; wait=False fires them and returns immediately (poll via :meth:wait / await).

make_sql_select ¶

make_sql_select(
    op: "DatabricksTableInsert",
    *,
    client: Any = None,
    source: "str | None" = None
) -> str

The atomic per-op SELECT over the op's staged source.

Two source shapes:

default — render `SELECT * FROM parquet.`` over the op's staged Parquet (resolved from its :class:Path` / uniform URL).
explicit source — when the caller already has a source reference (the {__tmpsrc__} placeholder, which is substituted for the external-data VolumePath at prepare time), project the op's schema columns from it: SELECT <projection> FROM <source>.

make_sql_insert ¶

make_sql_insert(
    op: "DatabricksTableInsert",
    *,
    target_location: "str | None" = None,
    source_sql: "str | None" = None,
    columns: "list[str] | None" = None,
    client: Any = None
) -> list[str]

Render the full statement list for one insert.

Yields the INSERT / MERGE / DELETE+INSERT / TRUNCATE / OPTIMIZE / VACUUM statement list for the op.

target_location / source_sql / columns let the synchronous paths supply their own source reference (the {__tmpsrc__} placeholder, a Spark temp-view name, or a wrapped user query) and pre-resolved target location; when omitted they're derived from the op's target and staged data.

yggdrasil.databricks.table.insert¶