Migrate to 0.2.0

Before you upgrade

This guide is for an application that already uses Cuprum 0.1. Most of 0.2.0 is new, opt-in capability, but some changes need an edit before an existing application works again. Those come first, each with its fix, so an upgrade can stop as soon as the application works and come back for the rest later. Readers new to Cuprum need none of this: start with Run a command.

Cuprum 0.2.0 is published as a beta, and installers skip pre-releases unless asked. Name the release the application is tested against, and pin that exact beta, so that moving to a later pre-release, which may bring further breaking changes, is a deliberate choice:

Then work through the checklist. A row that does not match the application can be skipped.

Upgrading from 0.2.0-beta1

Beta testers already have most of 0.2.0. These changes arrived after beta1, and the first six can need an edit: execution options are no longer child flags; builder arguments are a typed contract, so bytes and other values outside ArgValue are rejected; sh.make resolves the scoped catalogue; observe hooks see new event phases, of which settled, group_teardown, and stdio_error are new since beta1; metrics count failures from the settled event; and the reserved builder keywords include env_mode and process_group, two new ExecutionContext fields.

The rest is additive: environment policies, executable bindings, byte-exact capture, the owned running-command handle, process-group ownership, the broken-pipe policy, control observation, the Windows raw-handle outcome, streaming stdin and standard-stream redirection, redaction for telemetry, and the py.typed marker.

Breaking changes

Each change below was verified against 0.1.0 and 0.2.0-beta2 before it was written down.

Capture and echo moved to RunOutputOptions

In 0.1, run() and run_sync() took capture= and echo= keywords. Those keywords are gone, and passing them raises TypeError. Output settings now travel together in one RunOutputOptions, passed as output=:

import io
import sys

from cuprum import ExecutionContext, Program, ProgramCatalogue, RunOutputOptions, sh

catalogue = ProgramCatalogue.from_programs(sys.executable, name="output-options")
command = sh.make(Program(sys.executable), catalogue=catalogue)("-c", "print('hi')")

# 0.1: command.run_sync(capture=False, echo=True)
terminal = io.StringIO()
result = command.run_sync(
    output=RunOutputOptions(capture=False, echo=True),
    context=ExecutionContext(stdout_sink=terminal),
)
assert result.stdout is None and terminal.getvalue() == "hi\n"

try:
    command.run_sync(capture=False)
except TypeError:
    pass
else:
    raise AssertionError("the 0.1 keyword should be rejected")

Pipeline.run() and run_sync() still accept the old keywords for now, but emit a DeprecationWarning; move them to output= as well. IOOptions is a deprecated alias of RunOutputOptions and also warns.

Scopes take a ScopeConfig

In 0.1, scoped and CuprumContext.narrow() took keyword arguments such as allowlist= and before_hooks=. Both now take a single ScopeConfig, which gathers the allowlist, hooks, a default timeout, environment settings, a catalogue, and executable bindings in one value. The old keywords raise TypeError.

import sys

from cuprum import Program, ScopeConfig, current_context, scoped

PYTHON = Program(sys.executable)
started = []

# 0.1: with scoped(allowlist=frozenset([PYTHON]), before_hooks=(hook,)):
with scoped(ScopeConfig(allowlist=frozenset([PYTHON]), before_hooks=(started.append,))):
    assert current_context().is_allowed(PYTHON)
    assert not current_context().is_allowed(Program("rm"))

When the scope only needs the programs of a catalogue, scoped(catalogue=...) is shorter; see sh.make resolves the scoped catalogue.

Disjoint nested scopes now deny everything

In 0.1, an empty allowlist meant "allow everything", including the empty intersection produced when a nested scope allowed only programs its parent did not, so a nested scope could widen its parent by accident. A nested scope now always narrows: when the intersection is empty, nothing may run.

from cuprum import Program, ScopeConfig, current_context, scoped

GIT, TAR = Program("git"), Program("tar")
with scoped(ScopeConfig(allowlist=frozenset([GIT]))):
    with scoped(ScopeConfig(allowlist=frozenset([TAR]))):
        # 0.1 allowed both here; 0.2.0 allows neither.
        assert not current_context().is_allowed(GIT)
        assert not current_context().is_allowed(TAR)

The default context, outside any scope, still permits every program. Code that genuinely needs to widen a scope must say so with allow; see Scopes, policy, and hooks.

Execution options are no longer child flags

Every builder keyword used to become a child --name=value argument, so python("-c", script, cwd=repo_dir) built --cwd=<path> rather than changing the child's working directory, and a tool that ignores unknown flags ran in the wrong place with no error at all. The builder now rejects every ExecutionContext field name and every run_sync parameter with a TypeError that names the right call:

import sys

from cuprum import Program, ProgramCatalogue, sh

catalogue = ProgramCatalogue.from_programs(sys.executable, name="reserved")
python = sh.make(Program(sys.executable), catalogue=catalogue)
try:
    python("-c", "print('unused')", cwd="/tmp")
except TypeError as exc:
    assert str(exc) == (
        "cwd is an execution option; pass ExecutionContext(cwd=...) to run_sync"
    )
else:
    raise AssertionError("cwd should not become a child flag")

Move the value to run_sync, passing context=ExecutionContext(cwd=...), timeout=..., or stdin=.... A command line that genuinely takes such a flag still gets it positionally: python("--cwd=<dir>"). Only the exact names are reserved, so working_dir=... and stdin_file=... continue to render as flags. Run a command lists every reserved name.

Builder arguments are a typed contract

Builders and build_argv now accept exactly str | int | float | bool | pathlib.Path | os.PathLike[str], published as ArgValue. sh.make returns a SafeCmdBuilder whose signature states that domain, so a call passing anything else is rejected by a static checker as well as at run time. Previously the builder stringified any object, so a list or a dict reached the command line as its repr, and b"release" was passed as the literal text b'release'. Values outside ArgValue now raise TypeError naming the offending type or value. Pass text, or a path, yourself:

from cuprum import Program, ProgramCatalogue, sh

catalogue = ProgramCatalogue.from_programs("echo", name="typed-arguments")
builder = sh.make(Program("echo"), catalogue=catalogue)

# A list used to become "['a', 'b']"; join it into the text you meant.
tags = ["a", "b"]
command = builder("--tags", ",".join(tags))
assert command.argv == ("--tags", "a,b")

Decode caller-held filesystem bytes with os.fsdecode() before building the command:

import os
import sys

from cuprum import Program, ProgramCatalogue, sh

catalogue = ProgramCatalogue.from_programs(sys.executable, name="bytes-arguments")
python = sh.make(Program(sys.executable), catalogue=catalogue)
argument = os.fsdecode(b"v\xc3\xbf")
command = python("-c", "print('two words')", argument)
assert command.argv == ("-c", "print('two words')", argument)

os.fsdecode() uses the configured filesystem encoding and error handler, which differ between POSIX and Windows. Custom path-like objects are accepted: a class whose __fspath__ returns text behaves exactly as a pathlib.Path does, and only a __fspath__ returning bytes is rejected.

sh.make resolves the scoped catalogue

scoped(catalogue=catalogue) derives the scope's allowlist from the catalogue's programs, and also activates the catalogue for sh.make, so call sites inside the block may drop the repeated catalogue= argument. Review call sites that build inside a catalogue scope for a different catalogue, or that rely on DEFAULT_CATALOGUE while a catalogue scope is active: sh.make() now resolves the scoped catalogue and raises UnknownProgramError at construction when it does not list the program. The ready-made builders in cuprum.builders resolve the same way, so inside a catalogue scope that catalogue must list the tool. Pass an explicit catalogue= where a different catalogue is the intent; the two checks are described under two enforcement points.

Construct ExecutionContext by keyword

ExecutionContext gained fields, and two of them, native_pump_cleanup_grace and timeout, sit straight after cancel_grace. A call that passed four or more positional arguments, such as ExecutionContext(env, cwd, 0.5, sink), now puts the fourth into a different field. Name every argument instead: ExecutionContext(env=env, cwd=cwd, cancel_grace=0.5, stdout_sink=sink).

Observe hooks see new event phases

An observe hook registered with observe receives every ExecEvent. In 0.1 the phases were plan, start, stdout, stderr, and exit. 0.2.0 adds stdin, stdin_error, stdio_error, timeout, teardown_error, capture_eof_grace_expired, pipeline_fail_fast, group_teardown, and settled. A hook that matches exhaustively on ExecEvent.phase and raises on an unknown value now fails the run, because Cuprum re-raises observe-hook failures. Give such hooks a default arm:

import sys

from cuprum import ExecEvent, Program, ProgramCatalogue, sh

catalogue = ProgramCatalogue.from_programs(sys.executable, name="phases")
python = sh.make(Program(sys.executable), catalogue=catalogue)
outcomes = []


def hook(event: ExecEvent) -> None:
    match event.phase:
        case "settled":
            outcomes.append(event.terminal_outcome)
        case _:
            pass  # Phases this hook does not track, including future ones.


with sh.observe(hook):
    python("-c", "pass").run_sync()
assert [outcome.value for outcome in outcomes] == ["exit_zero"]

settled is the one most hooks will want. It is emitted once for each execution whose plan event began observation, including spawn failure, timeout, cancellation, and other errors, and its TerminalOutcome is one of exit_zero, exit_nonzero, timeout, cancelled, or error. It reuses the plan's exec_id, carries a pid and exit_code only when those are known, and never includes exception text. Catalogue and allowlist validation still happen before observation and produce no lifecycle events; exit remains the event for actual child status and resource data. group_teardown reports a teardown outcome Cuprum could not resolve, such as a signal the kernel refused. stdio_error reports a failed standard-stream step, such as a stdin producer that raised, alongside the exception the caller receives.

Metrics count failures from the settled event

The bundled adapters now use settled rather than exit for terminal accounting. In MetricsHook, cuprum_failures_total and cuprum_duration_seconds are recorded from settled, so the failure counter now counts every outcome except exit_zero. That includes a timeout, a cancellation, and a spawn failure, where 0.1 counted only a non-zero exit. cuprum_failures_total and cuprum_executions_total also carry env_mode and capture_mode labels, and TracingHook closes its spans on settled. Review dashboards and alerts built on these series.

Import ExecHook from cuprum

The cuprum.context.ExecHook re-export has been removed. Import ExecHook from cuprum or from its definition site, cuprum.events. Only the import path changes, not the hook signature or registration behaviour.

Echo is bounded and no longer fails on encoding

Two echo behaviours changed; captured output is unaffected by either. Each echoed line is limited to 64 KiB by default, and longer lines are cut with a truncation marker: pass RunOutputOptions(max_echo_line_bytes=None) for the 0.1 unbounded echo. And a text-only sink that cannot encode the child's output, such as a Windows CP1252 console receiving UTF-8, no longer raises UnicodeEncodeError out of the run: echo stops for that stream, capture completes, a warning is logged, and the result records the fallback; see echo-fallback diagnostics.

Environment overlays resolve at spawn

Scoped env overlays resolve against the live os.environ when each child is spawned, not against a snapshot taken when the scope began. Code that relied on a snapshot should pass explicit values through the overlay or ExecutionContext.env instead.

Smaller changes

• ProgramCatalogue.visible_settings is now a property. The callable spelling still works during the migration period, but prefer attribute access.
• CommandResult now lives in cuprum.sh.results and Pipeline in cuprum.sh.pipeline. Both remain importable from cuprum and cuprum.sh, which are the supported paths.
• SafeCmdBuilder is now a protocol rather than a Callable[..., SafeCmd] alias.
• Cuprum now ships a py.typed marker, so type checkers read its annotations. A downstream type check may report errors it could not see before.

New in 0.2.0

Everything below is opt-in: an application that does not use it behaves as it did. Several additions are large enough that the guides teach them from the start: deadlines and input in Input, environment, and deadlines, run_concurrent() in Pipelines and concurrency, and the validated builders in Catalogues and builders. The sections that follow cover the remaining additions.

Shorter single-project catalogues

ProjectSettings.documentation_locations and noise_rules now default to empty tuples, and ProgramCatalogue.from_project() removes the ProgramCatalogue(projects=(settings,)) wrapper. Existing construction still works, and ProgramCatalogue.from_programs() builds a single-project catalogue straight from programs, as every example here does.

from cuprum import Program
from cuprum.catalogue import ProgramCatalogue, ProjectSettings

settings = ProjectSettings(name="rust-test-gates", programs=(Program("cargo"),))
catalogue = ProgramCatalogue.from_project(settings)
assert catalogue.lookup(Program("cargo")).project_name == "rust-test-gates"

Line-level output

SafeCmd.lines and RunOutputOptions.on_line observe decoded lines as they arrive. They are additive: callers of run() and pipelines need no change. For pipelines, line observation covers the final stage's stdout and every stage's stderr. See Control output.

An owned running-command handle

SafeCmd.start returns a RunningCommand rather than a CommandResult; run(), run_sync(), and lines() keep owning their child from start to finish. The handle owns the child's lifetime: leaving its async with block settles the run on every exit, and stops a still-running child down the same SIGTERM, cancel-grace, SIGKILL path a cancelled run() takes. Capture is always on for a handle, so a stopped child still reports the output it produced. It exposes wait(), signal(), terminate(), and kill(); signal() has two refusals, because the remedy differs: SignalAfterExitError means the child has already exited, and UnsupportedSignalError means the platform has no route to a caller-named signal, so use terminate() or kill(). The worked example is in Run a command.

Control observation

Control actions on a running child can be observed separately from the command lifecycle. Register a hook with observe_control to receive a ControlEvent for each one, recording whether the child complied, had to be killed, or the request was refused; nothing is observed unless a hook is registered, and control_metrics counts the events. See Observe production runs.

Byte-exact capture

SafeCmd.run_bytes and Pipeline.run_bytes(), with run_bytes_sync() counterparts, return the child's output undecoded through BytesCommandResult and BytesPipelineResult, whose output fields are bytes | None. A byte run is the same run in a different result mode: it takes the same output, timeout, and context arguments, keeps every measurement, still carries a timeout's partial output in the same field, and still re-raises an external cancellation rather than reporting a timeout. A caller-supplied on_line is rejected with ValueError before the child spawns, because a decoded-text callback would contradict a byte-exact capture; observation registered with sh.observe() still receives decoded lines. Callers who round-tripped output through a surrogate encoding can drop that and request the byte mode directly; see Control output.

Executable bindings

bind_executable separates a program's catalogue identity from the file that runs. Before, the only way to run a pinned path was to register the path as the program, which made that path the catalogue identity, the policy key, and the telemetry label. Now the binding is a scope and the identity is untouched; a caller that binds nothing keeps running each program under its own name, and the new resolved_path field is None on every event and result.

import sys

from cuprum import Program, ProgramCatalogue, ScopeConfig, bind_executable, scoped, sh

PYTHON = Program("python")
catalogue = ProgramCatalogue.from_programs(PYTHON, name="migration-bindings")
python = sh.make(PYTHON, catalogue=catalogue)

with (
    scoped(ScopeConfig(allowlist=frozenset([PYTHON]))),
    bind_executable(PYTHON, sys.executable),
):
    result = python("-c", "import sys; sys.stdout.write(sys.executable)").run_sync()
    assert result.program == PYTHON, "the catalogue identity is preserved"
    assert result.resolved_path == sys.executable, "the bound path ran instead"

A binding does not grant permission: a program outside the allowlist is refused with ForbiddenProgramError, and its resolver is never called. A resolver must return a str; at an execution boundary any resolver failure surfaces as ExecutableResolutionError, a RuntimeError naming the logical program with the original exception chained, so a FileNotFoundError raised inside a hand-rolled resolver no longer escapes as itself. A malformed path is rejected when the binding is made, with InvalidExecutableBindingError. Path validation is advisory, and resolved_path names the executable that started without attesting to the bytes at that path; see Scopes, policy, and hooks.

Environment policies

EnvMode, UNSET, and the env_mode fields on ScopeConfig, CuprumContext, and ExecutionContext are opt-in; import EnvMode and UNSET from cuprum.context. Existing callers keep the overlay behaviour. Use EnvMode.REPLACE only where a child must receive an explicitly supplied environment, and UNSET to remove an inherited variable from an overlay; see Input, environment, and deadlines.

Execution measurements

CommandResult now carries started_at, a wall-clock timestamp, and duration, in monotonic seconds. Both default to 0.0, so positional construction with the original six arguments still works. On Linux and macOS a direct command also reports its own user and system CPU time and maximum RSS. Those three fields are None on Windows, and for pipeline stages, whose usage cannot be attributed safely; keep None as unavailable when serializing them.

Echo-fallback diagnostics

CommandResult.relay_fallbacks records handled echo failures by stream and closed category, unicode_encode or broken_pipe, without output or arguments. The field is trailing and defaults to (), so existing construction is unaffected.

An opt-in broken-pipe policy for echoed output

A presentation sink whose destination closes under the run raises BrokenPipeError from the echo drain. RunOutputOptions.broken_pipe_policy defaults to BrokenPipePolicy.STRICT, which propagates the error and aborts the run, so an application that does not opt in behaves as before:

import sys

from cuprum import (
    BrokenPipePolicy,
    Program,
    ProgramCatalogue,
    RunOutputOptions,
    sh,
)

catalogue = ProgramCatalogue.from_programs(sys.executable, name="broken-pipe")
command = sh.make(Program(sys.executable), catalogue=catalogue)("-c", "print('hello')")
options = RunOutputOptions(broken_pipe_policy=BrokenPipePolicy.BEST_EFFORT)
assert options.broken_pipe_policy is BrokenPipePolicy.BEST_EFFORT
result = command.run_sync(output=options)
assert result.ok and result.stdout == "hello\n"

Under best-effort, a BrokenPipeError from a sink disables echo for the affected stream only, while capture, line observation, child reaping, and result diagnostics continue, and the result's relay_fallbacks names the broken_pipe category. Only BrokenPipeError is affected; any other sink OSError still propagates under both policies.

A heartbeat for quiet children

RunOutputOptions.idle_after defaults to None, so the heartbeat is off until set. When set, the built-in notice goes to the stderr sink and never into captured output.

import sys

from cuprum import Program, ProgramCatalogue, RunOutputOptions, sh

catalogue = ProgramCatalogue.from_programs(sys.executable, name="idle")
command = sh.make(Program(sys.executable), catalogue=catalogue)("-c", "print('done')")
result = command.run_sync(output=RunOutputOptions(idle_after=30.0))
assert result.ok and result.stdout == "done\n"

on_idle(elapsed_total, elapsed_idle) replaces the notice. It runs on the command's own event loop and must not block, and neither must the sink the built-in notice writes to. Wrap a slow sink so that writes hand off without waiting, for example through a queue that something off the loop drains:

import queue


class QueueSink:
    """Feed a queue that something off the run's loop drains."""

    def __init__(self, pending: queue.Queue[str]) -> None:
        self._pending = pending

    def write(self, text: str) -> None:
        self._pending.put_nowait(text)

    def flush(self) -> None:
        pass


pending: queue.Queue[str] = queue.Queue()
sink = QueueSink(pending)
sink.write("ready")
sink.flush()
assert pending.get_nowait() == "ready"

An opt-in presentation sink

GitHubActionsSink frames a run's echoed output as a collapsible group on GitHub Actions and turns a failure into one error annotation. Nothing is framed unless a sink is passed.

import sys

from cuprum import Program, ProgramCatalogue, RunOutputOptions, sh
from cuprum.sinks import GitHubActionsSink

catalogue = ProgramCatalogue.from_programs(sys.executable, name="actions")
command = sh.make(Program(sys.executable), catalogue=catalogue)("-c", "print('hello')")
result = command.run_sync(
    output=RunOutputOptions(echo=True, sink=GitHubActionsSink()),
)
assert result.ok and result.stdout == "hello\n"

Group and annotate flags

Two additive flags on RunOutputOptions, group and annotate_failure, reach the same framing without constructing a sink:

import sys

from cuprum import Program, ProgramCatalogue, RunOutputOptions, sh
from cuprum.sinks import GitHubActionsSink

catalogue = ProgramCatalogue.from_programs(sys.executable, name="migration-flags")
command = sh.make(Program(sys.executable), catalogue=catalogue)("-c", "print('hello')")
options = RunOutputOptions(echo=True, group=True, annotate_failure=True)
assert isinstance(options.sink, GitHubActionsSink)
result = command.run_sync(output=options)
assert result.ok and result.stdout == "hello\n"

Both default to False, so existing applications need no change and unflagged output is byte for byte what it was. The flags build a GitHubActionsSink, so they share its gate: they act only when GITHUB_ACTIONS is true, and an explicit sink= makes them no-ops. Anything but a bool raises ValueError.

Opt-in process-group ownership

Cuprum terminates the process it spawns, but only that process. When the child spawns descendants of its own, they are not signalled, and they can hold inherited pipes open after the child is reaped, which is the case a timeout most often creates. ExecutionContext(process_group=…) takes a ProcessGroupPolicy; it defaults to INHERIT, which tears down only the direct child, so a run that does not opt in behaves as before:

import sys

from cuprum import ExecutionContext, ProcessGroupPolicy, Program, ProgramCatalogue, sh

catalogue = ProgramCatalogue.from_programs(sys.executable, name="process-group")
command = sh.make(Program(sys.executable), catalogue=catalogue)(
    "-c", "import os, sys; print(os.getpgrp() == os.getpid())"
)
owned = ExecutionContext(process_group=ProcessGroupPolicy.OWN_GROUP)
assert owned.process_group is ProcessGroupPolicy.OWN_GROUP
assert command.run_sync(context=owned).stdout == "True\n"
# The default is unchanged: a run that says nothing keeps the caller's group.
assert ExecutionContext().process_group is ProcessGroupPolicy.INHERIT

Under OWN_GROUP the child leads a new session and process group, so teardown signals the group and reaches its descendants. Session leadership also detaches the child from the controlling terminal, so a terminal SIGINT or SIGHUP stops at the run, and a child that opens /dev/tty fails for want of one. The guarantee is bounded: a descendant that calls setsid() or setpgid() leaves the group and is out of reach, as are ancestors and siblings. OWN_GROUP raises ValueError at spawn on Windows, which has no POSIX process groups; INHERIT works everywhere. See Input, environment, and deadlines.

Stream-operation observation

A new opt-in channel reports each completed stream drain or pipeline transfer. Nothing is installed until a hook is registered.

import sys

from cuprum import Program, ProgramCatalogue, sh
from cuprum.adapters.metrics_adapter import InMemoryMetrics
from cuprum.adapters.stream_metrics import stream_operation_metrics_hook
from cuprum.stream_observation import observe_stream_operation

metrics = InMemoryMetrics()
catalogue = ProgramCatalogue.from_programs(sys.executable, name="metrics")
command = sh.make(Program(sys.executable), catalogue=catalogue)("-c", "print('hello')")
with observe_stream_operation(stream_operation_metrics_hook(metrics)):
    result = command.run_sync()
assert result.ok and result.stdout == "hello\n"

The Windows raw-handle hand-off outcome

RustPumpHandoffOutcome gains WINDOWS_RAW_HANDLE_REJECTED, recorded as windows_raw_handle_rejected in cuprum_rust_pump_handoff_total. It identifies the deliberate rejection when a direct native-shim caller supplies a raw Windows handle; normal pipeline hops still fall back to the Python pump as before. Update dashboards and exhaustive outcome handling to include the new value.

Streaming stdin and standard-stream redirection

Two additions cover input and output that should not pass through memory. Both are opt-in: an application that uses neither sees no change, and StdinInput works as before.

• Streaming input. stdin=StdinStream(chunks=...) feeds the child from an async iterable of str or bytes chunks, pulling the next chunk only after the previous write has drained. A producer failure terminates the child and raises StdinSourceError, with the original exception chained. See Stream input from a producer.
• Redirection. RunOutputOptions(stdout=..., stderr=...) takes a StdioTarget to send a stream to a file (path()), a descriptor the caller owns (fd()), or the parent's own stream (inherit()), instead of capturing it. Contradictory combinations, such as capture with a redirected stream or an inherited stdin alongside a stdin source, raise ValueError, and file or descriptor targets need a POSIX platform. See Send a stream to a file or the parent.

StdinStream, StdinSource, StdinSourceError, and StdioTarget are exported from the package root. Exhaustive observe hooks also see the new stdio_error phase; see Observe hooks see new event phases.

Opt-in redaction for telemetry projections

Structured logging and tracing export the argument vector, and events can carry environment values and output. To keep known secrets out of those exports, build a RedactionPolicy with the literal strings and regular expressions to mask, and pass it as redaction=policy to each exporting adapter: structured_logging_hook(), tracing_hook(), logging_hook, and GitHubActionsSink. Wrap any other observer with redacting_hook(observer, policy).

Nothing changes for an application that does not opt in. Redaction alters only what is exported, never the command, its environment, or the captured result, and a failing rule raises RedactionError rather than exporting raw text. See Keep secrets out of telemetry for an example and the full rules.