Job sheet 005 · Worked example Verified

Run jobs concurrently

Bounded concurrency with run_concurrent, fail-fast or collect-all, and inspecting every result.

Verified 2026-10-19 · Cuprum 0.2.0b2 · Python 3.12 · Linux

Concurrency

01 — Problem

The job

Three independent checks — lint, type checking, and a documentation build — have no reason to wait for each other. Run them side by side, but not all at once on a small machine, and afterwards know which failed and what each printed.

run_concurrent_sync runs commands concurrently and returns a ConcurrentResult; ConcurrentConfig.concurrency bounds how many children run at once. In async code, run_concurrent is the same call awaited.

02 — Before you start

Before you start

Level: Intermediate
Needs: Python 3.12 or later; nothing else, as the interpreter stands in for three checks
Install: Cuprum 0.2.0b2 from PyPI; see Getting started
Tested: 2026-10-19 against Cuprum 0.2.0b2 on Linux, Python 3.12; both runs repeated eight times with identical output
Uses: run_concurrent_sync, ConcurrentConfig (concurrency, fail_fast), ConcurrentResult

03 — Code

The code

checks.py Complete

import sys

from cuprum import (
    ConcurrentConfig,
    Program,
    ProgramCatalogue,
    run_concurrent_sync,
    scoped,
    sh,
)

PYTHON = Program(sys.executable)
CATALOGUE = ProgramCatalogue.from_programs(PYTHON, name="jobs")
python = sh.make(PYTHON, catalogue=CATALOGUE)


def job(name: str, seconds: float, code: int = 0):
    script = (
        f"import sys, time; time.sleep({seconds}); "
        f"print('{name}'); sys.exit({code})"
    )
    return python("-c", script)


jobs = [job("lint", 0.3), job("types", 0.1, code=2), job("docs", 0.2)]

with scoped(catalogue=CATALOGUE):
    result = run_concurrent_sync(*jobs, config=ConcurrentConfig(concurrency=2))

print(f"ok={result.ok}")
print(f"failures={result.failures}")
for index, item in zip(result.submission_indices, result.results):
    print(index, item.exit_code, item.stdout.strip())

Verified 2026-10-19 · Cuprum 0.2.0b2 · Python 3.12 · Linux

04 — Output

Expected output

Expected outputstdout

ok=False
failures=(1,)
0 0 lint
1 2 types
2 0 docs

Every check ran to completion, two at a time, and the results come back in submission order whatever order they finished in: the type check finished first but is listed second. failures holds the position of each result that exited non-zero, so the run as a whole is not ok.

05 — Failure path

When it fails

Collecting every result is the default. With fail_fast=True, the first non-zero exit cancels everything still running. Here the type check fails at once, and the two thirty-second checks are stopped.

checks_fail_fast.py Failure path

import sys

from cuprum import (
    ConcurrentConfig,
    Program,
    ProgramCatalogue,
    run_concurrent_sync,
    scoped,
    sh,
)

PYTHON = Program(sys.executable)
CATALOGUE = ProgramCatalogue.from_programs(PYTHON, name="jobs")
python = sh.make(PYTHON, catalogue=CATALOGUE)


def job(name: str, seconds: float, code: int = 0):
    script = (
        f"import sys, time; time.sleep({seconds}); "
        f"print('{name}'); sys.exit({code})"
    )
    return python("-c", script)


jobs = [job("types", 0.1, code=2), job("lint", 30), job("docs", 30)]

config = ConcurrentConfig(concurrency=3, fail_fast=True)
with scoped(catalogue=CATALOGUE):
    result = run_concurrent_sync(*jobs, config=config)

print(f"ok={result.ok}")
print(f"returned {len(result.results)} of {len(jobs)} results")
print(f"submission_indices={result.submission_indices}")
print(f"failure_submission_indices={result.failure_submission_indices}")

Verified 2026-10-19 · Cuprum 0.2.0b2 · Python 3.12 · Linux

Observed outputstdout

ok=False
returned 1 of 3 results
submission_indices=(0,)
failure_submission_indices=(0,)

The run returned in a fraction of a second, with one result: the cancelled commands have none. Because a result's position no longer matches its command's, map back with submission_indices and failure_submission_indices rather than trusting positions. A command that had already finished before the cancellation still appears.

06 — Why it is allowed

Why the policy permits it

The catalogue lists only this interpreter, and scoped(catalogue=...) makes it the allowlist for every command submitted inside the block. Each command is checked before it is spawned, so a forbidden one fails with ForbiddenProgramError rather than running.