yggdrasil.loki.resources¶
resources ¶
Workstation resource probe — CPU / RAM / accelerator → a model size tier.
A local model is bounded by the machine it runs on, so Loki sizes it to the
box: the more muscle, the larger the default local model it reaches for, and
the more confidently it keeps light work local instead of paying a remote API.
Probed live (hardware is cheap to read; torch stays cached in
sys.modules after the first import).
The accelerator probe spans the vendors a laptop actually ships today — NVIDIA
(cuda), Intel GPU (xpu — Arc / integrated Xe), Apple Silicon
(mps), plus a separate flag for an Intel NPU (AI Boost) — so a local
model lands on the accelerator instead of crawling on the CPU.
Resources
dataclass
¶
Resources(
cpu: int = 1,
ram_gb: float = 0.0,
gpu: bool = False,
accelerator: Optional[str] = None,
intel_gpu: bool = False,
npu: bool = False,
)
A workstation's compute, as typed fields instead of a loose dict.
gpu is the CUDA flag that drives the xlarge tier; accelerator is
the best torch-usable device (cuda/xpu/mps, or None); intel_gpu flags
an Intel GPU physically present even when torch can't target it; npu an
Intel NPU.
accelerator ¶
Best torch compute device for a local model on this box, or None.
Returns the device string the transformers pipeline accepts directly:
"cuda" (NVIDIA), "xpu" (Intel GPU — Arc or integrated Xe, via
the native XPU backend in recent torch / intel-extension-for-pytorch),
"mps" (Apple Silicon), or None for CPU-only. Drives the transformers
engine's device when YGG_LOKI_HF_DEVICE is unset. Intel NPUs are
reported separately by :func:has_npu — the HF pipeline can't target them.
intel_gpu_present ¶
Whether an Intel GPU (Arc / integrated Xe) is physically present — independent of whether torch can target it.
:func:accelerator only reports "xpu" when torch can actually drive the
GPU (an XPU build / IPEX). But a laptop's Intel iGPU is worth reporting
even on a stock CPU torch wheel — so this probes the OS directly: a DRM card
with Intel's PCI vendor id on Linux, the video-controller list on Windows.
Best-effort; False (e.g. on macOS, or when nothing matches).
has_npu ¶
True when an Intel NPU (AI Boost) is present on this box.
OpenVINO's device list is the authoritative signal when installed (and the
HF pipeline can't target the NPU directly — that path is OpenVINO /
optimum-intel). Without OpenVINO, fall back to OS-level signals so
the NPU is still detected on a bare box: the Linux intel_vpu accel
device, or the Windows "AI Boost" PnP entry. Best-effort; False when
nothing matches.
snapshot ¶
Current :class:Resources for this box (typed, not a loose dict).
accelerator is the best torch-usable device (see :func:accelerator);
gpu stays the CUDA flag that drives the xlarge tier (a discrete
NVIDIA GPU); intel_gpu flags an Intel GPU that's physically present
even when torch can't target it (see :func:intel_gpu_present); and npu
flags an Intel NPU (see :func:has_npu).
size_tier ¶
Model size tier for this box: small / medium / large / xlarge.
A CUDA GPU is xlarge; otherwise RAM decides (≥ 32 GB large, ≥ 16 GB
medium, else small). Local engines map this to a concrete model.
can_run_local ¶
True when this box can comfortably host a local model (any GPU
accelerator, or ≥ 4 cores + ≥ 8 GB RAM) — the gate for keeping light work
local instead of remote. An Intel GPU (xpu) counts as much as CUDA.