`yggdrasil.concurrent.pool`¶

pool ¶

Bounded thread-pool for large / infinite job streams.

JobPoolExecutor ¶

JobPoolExecutor(
    max_workers: int | None = None,
    max_in_flight: int | None = None,
    job_name_prefix: str = "",
)

Bases: ThreadPoolExecutor

ThreadPoolExecutor helper for large / infinite :class:Job streams.

Keeps at most max_in_flight futures submitted at any time so the caller is never overwhelmed when the job source is unbounded.

Parameters:

Name	Type	Description	Default
`max_workers`	`int \| None`	Thread-pool size (`None` → `os.cpu_count()`).	`None`
`max_in_flight`	`int \| None`	Max in-flight futures (`None` → `max_workers * 2`).	`None`
`job_name_prefix`	`str`	Thread-name prefix forwarded to `ThreadPoolExecutor`.	`''`

Yields (via :meth:as_completed): :class:JobResult in completion order (ordered=False) or submission order (ordered=True).

submit_job ¶

submit_job(job: Job) -> Future

Submit a single :class:Job and return its :class:Future.

as_completed ¶

as_completed(
    jobs: Iterable[Job],
    *,
    ordered: bool = False,
    max_in_flight: int | None = None,
    cancel_on_exit: bool = False,
    shutdown_on_exit: bool = False,
    shutdown_wait: bool = False,
    raise_error: bool = True
) -> Iterator[JobResult]

Consume a (possibly infinite) :class:Job iterable and yield :class:JobResult objects.