Python bindings

The workspace now provides a ready-to-build Python wheel. pyproject.toml declares maturin as the build backend and targets tei-py/Cargo.toml, so the workflow looks like:

python -m pip install --upgrade pip maturin
maturin develop  # builds and installs tei_rapporteur into the active venv
python -c "import tei_rapporteur as tr; print(tr.Document('Wolf 359').title)"

Within Python, tei_rapporteur.Document constructs a validated TEI document by wrapping the Rust TeiDocument. The class exposes a .title property and an emit_title_markup() method that mirrors the Rust helper. The module also offers a top-level emit_title_markup(title: str) so scripting callers can work without instantiating a document. CI now builds the wheel on Ubuntu, installs it via pip, and imports the module to ensure the PyO3 glue remains healthy.

Python data classes now live in tei_rapporteur.structs. The submodule defines msgspec.Struct projections (Episode, TeiHeader, FileDesc, Paragraph, Utterance, DivBlock, ListBlock, Item, Label, StandOff, SpanGroup, Span, and the citation-declaration types) that mirror the Python-facing Rust projection. Inline nodes decode into plain Python objects, and TEI pointer-list attributes such as source, resp, corresp, and ana are exposed as list[str] instead of TEI's whitespace-separated attribute strings. MessagePack emitted by to_msgpack decodes directly into these classes, and encoding them feeds the payload straight back into from_msgpack.

Structural body content is exposed through tagged unions:

• BodyBlock = Paragraph | Utterance | DivBlock
• DivContent = Paragraph | Utterance | ListBlock | DivBlock
• Event = DocumentStart | HeaderEvent | ParagraphEvent | UtteranceEvent | DivEvent | DocumentEnd

DivBlock and streamed DivEvent values now expose div_type, optional subtype, optional head, optional xml_id, and recursive content, so chapter markers, guest-bio sections, and sponsor-read sections can be modelled without flattening the hierarchy into paragraphs.

Typed msgspec decoding lets callers inspect nested divisions without first round-tripping through XML or untyped dictionaries:

import msgspec
import tei_rapporteur as tei
from pathlib import Path
from tei_rapporteur.structs import DivBlock, Episode, Item, Label, ListBlock

tei_xml = Path("episode.tei.xml").read_text(encoding="utf-8")
document = tei.parse_xml(tei_xml)
payload = tei.to_msgpack(document)
episode = msgspec.msgpack.decode(payload, type=Episode)

body_block = episode.text.body.blocks[0]
if isinstance(body_block, DivBlock):
    print(body_block.div_type)
    for child in body_block.content:
        if isinstance(child, DivBlock):
            for nested in child.content:
                if isinstance(nested, ListBlock):
                    first_item = nested.items[0]
                    if isinstance(first_item, Item) and isinstance(first_item.label, Label):
                        print(first_item.label.content[0].value)

Citation metadata is split along TEI-native boundaries. Canonical citation declarations live under header.encoding_desc.refs_decl, utterance-local provenance stays on Utterance, and many-to-many overlays live in the optional root Episode.stand_off layer via SpanGroup and Span.

Binary interchange is now supported through tei_rapporteur.from_msgpack(payload: bytes). The helper accepts the bytes produced by msgspec.msgpack.encode (or any compatible encoder), decodes them via tei-serde (wrapping rmp-serde), and returns a Document. Invalid payloads raise ValueError, so Python callers receive a familiar exception instead of a Rust-specific error type. This allows workflows such as:

import msgspec
import tei_rapporteur as tei
from tei_rapporteur.structs import Episode, FileDesc, TeiBody, TeiHeader, TeiText

episode = Episode(
    header=TeiHeader(file_desc=FileDesc(title="Bridgewater")),
    text=TeiText(body=TeiBody()),
)
payload = msgspec.msgpack.encode(episode)
document = tei.from_msgpack(payload)
print(document.title)

The inverse helper, tei_rapporteur.to_msgpack(doc: Document), serializes the validated document into MessagePack bytes via tei_serde::msgpack. The function returns Python bytes, making it trivial to persist the payload or feed it straight into msgspec.msgpack.decode to hydrate a structured type. Non-Document inputs raise a TypeError, giving users immediate feedback when they miswire a call. A complete round trip therefore looks like:

doc = tei.Document("Bridgewater")
payload = tei.to_msgpack(doc)
from tei_rapporteur.structs import Episode
episode = msgspec.msgpack.decode(payload, type=Episode)

For JSON-style hand-offs, tei_rapporteur.from_dict(payload) and tei_rapporteur.to_dict(doc) use pyo3-serde to bridge Python built-ins and the Rust TeiDocument. The helpers accept any mapping/sequence tree that would be valid JSON, raising ValueError when required fields are missing or titles are blank and TypeError when a non-Document is passed. The output of to_dict matches what msgspec.to_builtins produces, so callers can stay with native Python objects:

doc = tei.Document("Bridgewater")
payload = tei.to_dict(doc)
assert payload["teiHeader"]["fileDesc"]["title"] == "Bridgewater"
round_tripped = tei.from_dict(payload)

When scripts already have TEI XML on disk, the new tei_rapporteur.parse_xml and tei_rapporteur.emit_xml functions avoid redundant conversions. parse_xml hands the string straight to the Rust parser, returning a Document that holds the validated TeiDocument. emit_xml performs the inverse operation and retains the forbidden-character guardrails enforced by tei-xml. A typical round trip combining XML and Python struct manipulation therefore looks like:

from pathlib import Path
import msgspec
import tei_rapporteur as tei
from tei_rapporteur.structs import Episode

doc = tei.parse_xml(Path("episode.tei.xml").read_text())
payload = tei.to_msgpack(doc)
episode = msgspec.msgpack.decode(payload, type=Episode)
episode.title = "Wolf 359 Reissue"
doc = tei.from_msgpack(msgspec.msgpack.encode(episode))
xml = tei.emit_xml(doc)

The BDD tests now cover successful decoding, encoding, XML parsing, emission, and the corresponding error paths, ensuring the entry points remain reliable as the API expands.

For spoken-runtime estimation, use tei_rapporteur.spoken_text_segments(xml) instead of traversing XML locally. The function accepts a complete TEI document string and returns tei_rapporteur.structs.SpokenTextSegment objects with text, locator, and xml_id fields. It includes performed text from <p>, <ab>, <l>, direct <u> content, and standalone <seg> in spoken context. It excludes speaker labels, stage directions, notes, lists, labels, headings, references, bibliography, show-note divisions (<div type="notes">), TEI header metadata, and stand-off metadata. Malformed XML and unsupported body markup raise ValueError; the API never falls back to raw-text counting.

import tei_rapporteur as tei

xml = """
<TEI>
  <teiHeader><fileDesc><title>Episode</title></fileDesc></teiHeader>
  <text><body>
    <sp>
      <speaker>Host</speaker>
      <p xml:id="line-1">Hello <seg>there</seg>.<note>cut?</note></p>
    </sp>
    <div type="notes"><p>Link dump.</p></div>
  </body></text>
</TEI>
"""

segments = tei.spoken_text_segments(xml)
assert segments[0].text == "Hello there."
assert segments[0].locator == "/TEI/text/body/sp[1]/p[1]"
assert segments[0].xml_id == "line-1"

The Document class exposes a validate() method that performs document-wide integrity checks. It verifies that all xml:id values are unique across the document (including annotation systems, stand-off span groups, stand-off spans, paragraphs, utterances, divisions, lists, and items), that utterance speaker references match the declared cast list when present, that refsDecl entries keep their required @match and @property values, and that internal #id pointers in utterance, item, and stand-off provenance attributes resolve against existing identifiers.

import tei_rapporteur as tei

doc = tei.from_dict(payload)
try:
    doc.validate()
    print("Document is valid")
except ValueError as e:
    print(f"Validation failed: {e}")

Validation raises ValueError with a descriptive message when:

• Duplicate xml:id values are detected across the document
• An utterance references a speaker not declared in the profile cast
• A speaker is referenced when the profile has an empty cast (an empty cast still counts as declared, so all speaker references fail until the cast is populated)
• A citeStructure or citeData declaration leaves a required attribute blank
• A Div leaves @type blank after trimming
• A stand-off spanGrp leaves @type blank after trimming
• A stand-off span omits both @target and @from, or uses @to without @from
• A #-prefixed pointer in source, resp, corresp, ana, target, from, or to does not resolve to a known xml:id

Documents without a profile cast allow speaker references without validation, enabling incremental validation of draft documents.

@corresp values follow TEI pointer semantics. A value beginning with # is an internal pointer and must resolve to an xml:id in the same TEI document. Use this form only when the referenced node is materialized in the document. Validation rejects unresolved internal pointers so callers do not accidentally ship dangling local references.

External identifiers such as urn:..., tag:..., or https://... may be used when the target lives outside the TEI document. Repository-owned objects, including Episodic reference-document revisions, should use an external identifier in @corresp unless that object is also represented in the same TEI document with an xml:id.

Guest biographies therefore link to their source reference revision as an external correspondence:

<item corresp="urn:episodic:reference-document-revision:019e1368">
  <label>Ada Lovelace</label>
  Mathematician and computing pioneer.
</item>

tei-rapporteur currently supports @corresp, @n, and xml:id on list items. @source on Item is not part of the public body model yet; it may be considered later if callers need stricter provenance semantics beyond the current correspondence link.