Skip to content

yggdrasil.aws.client

client

AWS client / service / resource trio.

Mirrors the Databricks pattern: one client owns the configuration and the session, mints boto3 service clients on demand; service objects bind to a client; resource objects bind to a service. The split lets a single set of credentials cover an entire tree of objects without each one re-resolving auth.

Class summary

  • :class:AWSClient — the analog of :class:DatabricksClient. Holds every configuration knob directly (no separate AWSConfig class — :class:AWSClient is the config). Owns a lazily-built boto3 :class:Session, exposes per-service client factories (s3_client, sts_client), and per-service service objects (self.s3 returns :class:S3Service). Has a current() singleton + URL round-trip.

  • :class:AWSService — abstract base for service objects. Holds an :class:AWSClient, defers shared concerns (session, region) to it. Subclasses (:class:S3Service, future :class:DynamoService, …) layer their own client + behavior on top.

  • :class:AWSResource — abstract base for individual entities (an S3 object, a DynamoDB row). Holds a service; reaches the client via self.service.client.

Singleton & runtime defaults

:class:AWSClient is a :class:Singleton keyed on every identity-bearing init kwarg, so two callers building a client with the same auth share one boto :class:Session. Bare AWSClient() scrapes the managed-runtime context (AWS Lambda / Batch / ECS / EKS env vars: AWS_ACCESS_KEY_ID, AWS_REGION, AWS_PROFILE, AWS_ROLE_ARN, AWS_ROLE_SESSION_NAME, AWS_ENDPOINT_URL, AWS_S3_ADDRESSING_STYLE); anything still unset falls through to boto3's own credential chain at session-build time.

AWSClient.current() returns a process-global default. Service defaults flow from there: S3Service.current() builds against AWSClient.current() automatically. Pass explicit kwargs to any service / path constructor to escape the singleton (different account, different role, etc.).

AWSClient 

AWSClient(
    *,
    access_key_id: Any = ...,
    secret_access_key: Any = ...,
    session_token: Any = ...,
    region: Any = ...,
    profile: Any = ...,
    role_arn: Any = ...,
    role_session_name: Any = ...,
    external_id: Optional[str] = None,
    duration_seconds: int = 3600,
    endpoint_url: Any = ...,
    s3_addressing_style: Any = ...,
    sso_start_url: Any = ...,
    sso_region: Any = ...,
    sso_account_id: Any = ...,
    sso_role_name: Any = ...,
    refresher_key: Optional[str] = None,
    refresher: Optional[CredentialsRefresher] = None,
    singleton_ttl: Any = ...
)

Bases: Singleton

Merged AWS configuration + session + per-service client factory.

Holds every knob needed to mint a boto3 :class:Session directly on the instance — there is no separate AWSConfig class. Equality and hashing follow :meth:_singleton_key, which excludes :attr:refresher (callables aren't comparable) and lazy / transient session state. Use :attr:refresher_key as the discriminator when distinct refreshers must mint distinct clients.

Construction shapes:

  • Static credentials: pass access_key_id / secret_access_key / optional session_token.
  • Profile: pass profile (matches AWS_PROFILE); the session resolves through ~/.aws/credentials.
  • Assume-role: pass role_arn, optionally with role_session_name / external_id / duration_seconds. A refreshable credential provider drives STS AssumeRole on demand.
  • SSO (boto3-native): pass sso_start_url / sso_region / sso_account_id / sso_role_name for IAM Identity Center with external-browser device-code auth — boto3's SSOTokenProvider handles the device-code dance and token cache (typically primed by aws sso login).
  • Default chain: pass nothing. Runtime env vars (AWS_ACCESS_KEY_ID / AWS_REGION / AWS_PROFILE / AWS_ROLE_ARN / AWS_ROLE_SESSION_NAME / AWS_ENDPOINT_URL / AWS_S3_ADDRESSING_STYLE) are auto-detected so Lambda / Batch / ECS / EKS land with reasonable defaults; anything still empty falls through to boto3's own chain at session-build time.

session property 

session: 'boto3.Session'

Lazily-built boto3 :class:Session. Cached.

  • :meth:has_refresher → :class:RefreshableCredentials driven by the user-supplied callback.
  • :meth:has_assume_role → :class:RefreshableCredentials driven by STS AssumeRole.
  • :meth:has_sso → boto3-native SSO token provider via a transient profile.
  • Otherwise → static / profile / default-chain creds.

s3 property 

s3: 'S3Service'

The :class:S3Service bound to this client. Lazy + cached.

account_id property 

account_id: str

Resolve the account ID via STS. Cached on the instance.

effective_region property 

effective_region: Optional[str]

Configured region first, then boto session default.

explore_url property 

explore_url: URL

AWS Console home for this client's region — clickable from code.

account property 

account: 'AWSAccount'

The :class:AWSAccount resource for this client (STS-backed).

batch property 

batch: 'AWSBatch'

The :class:AWSBatch runtime resource for this client.

Pure os.environ read — no network, no credentials. .is_batch gates whether the job-id / queue / array-index fields are meaningful.

has_refresher 

has_refresher() -> bool

True iff a :attr:refresher callback is wired up.

Drives :meth:_build_session to mint a :class:RefreshableCredentials-backed session instead of a static one.

has_sso 

has_sso() -> bool

True iff this client is configured for IAM Identity Center.

Either :attr:sso_start_url alone (token-cache flow primed by aws sso login) or the full role triple (sso_account_id + sso_role_name + sso_region) is enough — boto3 picks up whichever set is present once the profile is materialised at session-build time.

refresh_metadata 

refresh_metadata() -> dict[str, Any]

Invoke :attr:refresher and return botocore-shaped metadata.

to_credentials 

to_credentials() -> AwsCredentials

Snapshot the static credentials into an :class:AwsCredentials.

Returns the configured static fields; does NOT materialize assumed-role tokens — exporting a live STS token would defeat the auto-refresh that's the whole point of using a role.

to_url 

to_url(scheme: Optional[str] = None) -> URL

Render this client as a URL.

Format: aws://[creds@]region/?profile=...&role_arn=...

  • Region goes in the host slot (a region is the closest AWS analog to a "host" — it parameterizes every endpoint).
  • Static creds go in user:password (when both set).
  • Everything else identity-bearing goes in the query string; the secret credential fields are emitted only via userinfo.

from_ classmethod 

from_(obj: Any) -> TC

Coerce obj (str / URL / dict / AWSClient) to a client.

from_parsed_url classmethod 

from_parsed_url(url: URL) -> TC

Parse an aws:// URL back into an :class:AWSClient.

from_credentials classmethod 

from_credentials(
    creds: AwsCredentials,
    *,
    region: Optional[str] = None,
    endpoint_url: Optional[str] = None,
    refresher: Optional[CredentialsRefresher] = None,
    **kwargs: Any
) -> TC

Construct from a static :class:AwsCredentials.

Pass refresher for self-renewing temporary credentials.

from_refresher classmethod 

from_refresher(
    refresher: CredentialsRefresher,
    *,
    region: Optional[str] = None,
    endpoint_url: Optional[str] = None,
    refresher_key: Optional[str] = None,
    **kwargs: Any
) -> TC

Build a self-refreshing :class:AWSClient from a credentials callback.

from_databricks_sql classmethod 

from_databricks_sql(
    query: str,
    *,
    client: Optional["DatabricksClient"] = None,
    region: Optional[str] = None,
    endpoint_url: Optional[str] = None,
    columns: Optional[dict[str, str]] = None,
    **kwargs: Any
) -> TC

Build an :class:AWSClient that vends credentials from a Databricks SQL row.

current classmethod 

current(*, reset: bool = False, **overrides: Any) -> TC

Process-global default :class:AWSClient.

reset=True rebuilds; overrides are passed to a fresh constructor. Mirrors :meth:DatabricksClient.current.

set_current classmethod 

set_current(client: Optional['AWSClient']) -> None

Replace the process-global current client. Pass None to clear.

connect 

connect(*, reset: bool = False) -> 'AWSClient'

Eagerly build the boto session. Idempotent.

close 

close() -> None

Drop the cached session, all per-service clients, all service objects.

client 

client(service: str, **overrides: Any) -> 'BaseClient'

Get a boto3 client for service. Cached per (service, overrides).

caller_identity 

caller_identity() -> dict[str, Any]

Wrap STS GetCallerIdentity. Network call; not cached.

to_singleton 

to_singleton(ttl: Any = ...) -> 'Singleton'

Promote this instance into the per-class _INSTANCES cache.

Hot listing paths (iterdir / _ls / glob) build children with singleton_ttl=False so the bounded cache doesn't fill up with thousands of short-lived entries. When a caller decides one of those children is worth keeping around (handing it to a long-running worker, returning it from an API), :meth:to_singleton registers self into the cache so the next constructor call with the same key collapses to the same instance.

ttl defaults to the subclass's _SINGLETON_TTL (... = no caching, None = process lifetime, or a seconds count). When a different instance is already cached under this key, that pre-existing one wins and is returned unchanged — the cache is the source of truth.

invalidate_singleton 

invalidate_singleton(remove_global: bool = True) -> None

Pop self from the per-class _INSTANCES cache.

Mutating ops on a Singleton-cached object (writes, deletes, schema invalidations on a Databricks table, put_object on an :class:S3Path) want to make sure the next caller asking for the same key gets a fresh build rather than collapsing onto this stale handle — that's what remove_global=True (the default) does. The pop is :meth:identity-guarded: only an entry that still points at self is removed, so a concurrent re-construction that already raced past this thread is left alone.

remove_global=False is a no-op. The keyword exists so subclass invalidators (invalidate_singleton, _invalidate_entity_tag_cache, …) can offer the same switch without branching at the call site.

AWSService 

AWSService(client: Optional[AWSClient] = None)

Bases: ABC

Abstract base for AWS service objects.

A service object binds an :class:AWSClient to a particular AWS service (S3, DynamoDB, …). Mirrors :class:DatabricksService: holds a client, delegates shared concerns (region, account_id, session) upstream, supports a current() singleton, and round-trips via URL.

Identity & singleton caching

Instances are cached per (class, client) in :attr:_INSTANCES. Pickling routes through :meth:__getnewargs__ so a service unpickled in the same process collapses to the live singleton. Subclasses add non-picklable handles to :attr:_TRANSIENT_STATE_ATTRS.

AWSResource 

AWSResource(service: Optional[AWSService] = None, *args, **kwargs)

Bases: ExploreUrlRepr, ABC

Abstract base for AWS-backed entities.

Concrete resources (:class:~yggdrasil.aws.account.AWSAccount, :class:~yggdrasil.aws.fs.path.S3Bucket, …) override :attr:explore_url to return a Console deep-link; the inherited :class:ExploreUrlRepr then gives a clickable repr / _repr_html_ for free.

explore_url property 

explore_url: 'Optional[URL]'

Web-UI deep-link for this resource, or None. Override in subclasses; the repr / HTML below key off it.