`yggdrasil.loki.engines.local`¶

local ¶

Shared base for local (on-workstation) engines.

A local model is free and private but bounded by the box, so — unlike a remote engine, which adapts its model to the prompt's cost tier — a local engine adapts its model to the machine's resources: the more CPU/RAM/GPU, the larger the default model it loads (:mod:yggdrasil.loki.resources). Concrete local engines (transformers, ollama) just declare a :attr:RESOURCE_MODELS ladder; sizing, labelling, and the bootstrap model all fall out of it here.

LocalEngine ¶

LocalEngine(*, model: Optional[str] = None, tier: Optional[str] = None)

Bases: TokenEngine

A :class:TokenEngine that runs on this workstation, sized to it.

bootstrap_model `property` ¶

bootstrap_model: str

The default model for this box — the resource-sized row, or the engine's :attr:default_model fallback.

resolve_model ¶

resolve_model(
    *,
    messages: Optional[list[dict[str, Any]]] = None,
    system: Optional[str] = None,
    tier: Optional[str] = None
) -> Optional[str]

An explicit pin wins; otherwise the model sized to this workstation.

Local models are resource-bound, so the remote fast/deep cost tier doesn't apply — the box, not the prompt, picks the size.

choose_tier ¶

choose_tier(
    messages: Optional[list[dict[str, Any]]] = None,
    system: Optional[str] = None,
) -> str

Adaptive tier for this request: "deep" or "fast".

Sizes on the message content (the actual work, not the fixed system boilerplate) and scans both message and system text for reasoning signals. Long or signalled requests get the deep tier; the rest stay fast. Override for a smarter policy.

usage ¶

usage() -> list[Any]

This engine's per-model usage rows from the global meter.

available `abstractmethod` ¶

available() -> bool

True when this engine has the credentials/config to run.

complete `abstractmethod` ¶

complete(
    messages: list[dict[str, Any]],
    *,
    system: Optional[str] = None,
    max_tokens: int = DEFAULT_MAX_TOKENS,
    tier: Optional[str] = None,
    **options: Any
) -> Completion

Run one chat completion and return a :class:Completion.

tier forces "fast" / "deep" model selection for this call; None (the default) lets the engine adapt.

generate ¶

generate(
    prompt: str,
    *,
    system: Optional[str] = None,
    tier: Optional[str] = None,
    **options: Any
) -> str

Convenience: complete a single user prompt → reply text.

stream ¶

stream(
    messages: list[dict[str, Any]],
    *,
    system: Optional[str] = None,
    max_tokens: int = DEFAULT_MAX_TOKENS,
    tier: Optional[str] = None,
    **options: Any
) -> "Iterator[str]"

Yield reply text incrementally as it is produced.

The default has no real streaming — it runs :meth:complete and yields the whole reply once. Engines whose SDK streams override this to yield token deltas live (and still record usage on completion).

generate_stream ¶

generate_stream(
    prompt: str,
    *,
    system: Optional[str] = None,
    tier: Optional[str] = None,
    **options: Any
) -> "Iterator[str]"

Convenience: stream a single user prompt → text chunks.

yggdrasil.loki.engines.local¶