`yggdrasil.path.pypi`¶

pypi ¶

:class:PyPIPath — managed PEP 503 simple index at any :class:Path root.

A drop-in "package artifactory" that publishes Python distributions under a yggdrasil :class:~yggdrasil.io.path.path.Path root — local, workspace, S3, DBFS volumes, in-memory — and keeps a PEP 503-shaped index.html per project so the result is consumable by pip install --extra-index-url=<root>.

Two ingestion shapes:

:meth:PyPIPath.publish — build a real pip wheel for a local module (importable name or source path) and upload the versioned wheel under <root>/<normalized-project>/<wheel>.whl. This is the pip-installable path: the wheel filename carries the version, the index page lists every uploaded wheel, and re-publishing the same source short-circuits when the target already exists.
:meth:PyPIPath.publish_archive — zip a module via :meth:Path.upload_module and drop the archive under the same versioned layout. Used when the artefact ships as a raw zip (Spark addArtifacts(pyfile=True), sys.path extension) rather than a pip wheel.

Both shapes use the same versioning + subfolder convention; the caller picks based on the consumer.

:class:yggdrasil.databricks.jobs.workspace_pypi.WorkspacePyPI is a thin subclass that pins the root to /Workspace/Shared/.ygg/pypi/simple and binds the publisher to a workspace client.

PyPIPath ¶

PyPIPath(root: Union[str, 'Path', Path])

A PEP 503 simple index hosted at root (any yggdrasil :class:Path).

Parameters¶

root Where to publish. Anything :meth:Path.from_ accepts — a :class:Path instance, a URL string ("/Workspace/Shared/pypi/simple", "s3://my-bucket/pypi", "file:///tmp/pypi"), or a :class:pathlib.Path. Strings without a scheme resolve to :class:LocalPath.

Examples¶

Local index for testing::

pypi = PyPIPath("/tmp/my-index")
pypi.publish("my_local_pkg")
# → /tmp/my-index/my-local-pkg/my_local_pkg-0.1.0-py3-none-any.whl

Layered S3 index::

pypi = PyPIPath("s3://wheels.example.com/simple")
pypi.publish("internal_lib")

publish ¶

publish(
    module: Any,
    *,
    source_path: Optional[str] = None,
    version: Optional[str] = None,
    rebuild: bool = False
) -> "Path"

Build a wheel for module and upload it under :attr:root.

module is anything :func:resolve_local_lib_path accepts — an importable module name ("my_local_pkg"), a directory path, or a pre-built .whl. The wheel is built via python -m pip wheel --no-deps so we don't take a hard dependency on :mod:build. Pre-built wheels handed in via source_path are uploaded verbatim.

Re-publishing the same source is a no-op unless rebuild is set — the wheel filename carries the version, so an already-uploaded target short-circuits.

Returns the published :class:Path (full URL accessible via .full_path() or .url).

publish_archive ¶

publish_archive(
    module: Any,
    *,
    name: Optional[str] = None,
    version: Optional[str] = None,
    rebuild: bool = True
) -> "Path"

Zip module via :meth:Path.upload_module and place under :attr:root.

Mirrors :meth:publish but emits a raw .zip (suitable for SparkSession.addArtifacts(pyfile=True) / sys.path extension) instead of a pip wheel. The version is derived from installed metadata when available; pass version to override. The archive filename follows {name}-{version}.zip so the per-project subfolder still sorts by version.

rebuild=True (default) re-uploads each call — matches the cheap-and-idempotent contract of :meth:Path.upload_module.

import_module ¶

import_module(
    module_name: str,
    *,
    version: Optional[str] = None,
    install: bool = False,
    cache_dir: Any = None
) -> Any

Locate and import the published module_name from this index.

Locates the matching wheel / archive under <root>/<normalized-name>/, prefers the requested version when set (matched against the wheel filename), otherwise picks the lexicographically-highest filename (the typical PEP 440 ordering for the same project). The resolved :class:Path is then imported via :meth:Path.import_module.

install=False (default) downloads the artefact and prepends the local archive to sys.path. For .zip archives that's enough; for .whl wheels the import will fail with :class:ModuleNotFoundError unless the wheel is already installed locally — pass install=True to opt into the pip install fallback. The conservative default avoids mutating the active interpreter's site-packages on a casual lookup; callers that explicitly want the install step ask for it.

Raises :class:ModuleNotFoundError when no artefact is published under that name, with a hint pointing at :meth:publish / :meth:publish_archive.

normalize_pep503_name ¶

normalize_pep503_name(name: str) -> str

Return the PEP 503-normalized form of a project name.

parse_wheel_filename ¶

parse_wheel_filename(name: str) -> tuple[Optional[str], Optional[str]]

Return (distribution, version) from a wheel filename, or (None, None).

yggdrasil.path.pypi¶