Running the clustering pipeline

Updated Sep 01, 2026

A Chutoro instance is constructed with ChutoroBuilder, followed by invocation of run with a DataSource implementation.

use chutoro_core::{ChutoroBuilder, DataSource, DataSourceError, ExecutionStrategy};

struct Dummy(Vec<f32>);

impl DataSource for Dummy {
    fn len(&self) -> usize { self.0.len() }
    fn name(&self) -> &str { "dummy" }
    fn distance(&self, i: usize, j: usize) -> Result<f32, DataSourceError> {
        let a = self.0.get(i).ok_or(DataSourceError::OutOfBounds { index: i })?;
        let b = self.0.get(j).ok_or(DataSourceError::OutOfBounds { index: j })?;
        Ok((a - b).abs())
    }
}

let chutoro = ChutoroBuilder::new()
    .with_min_cluster_size(8)
    .with_execution_strategy(ExecutionStrategy::CpuOnly)
    .build()?;
let result = chutoro.run(&Dummy(vec![1.0, 2.0, 4.0, 8.0]))?;
assert_eq!(result.cluster_count(), 1);
# Ok::<(), chutoro_core::ChutoroError>(())

On CPU builds, with_hnsw_params configures both one-shot batch runs and incremental sessions. For a batch run, max_connections and ef_construction are passed to CPU HNSW construction. The configured max_connections and the effective, dataset-bounded construction-search width derived from ef_construction are included when estimating the run's peak memory for a limit set with with_max_bytes:

use chutoro_core::{ChutoroBuilder, HnswParams};

let chutoro = ChutoroBuilder::new()
    .with_min_cluster_size(2)
    .with_hnsw_params(HnswParams::new(32, 128)?)
    .with_max_bytes(1_073_741_824)
    .build()?;
let result = chutoro.run(&Dummy(vec![1.0, 2.0, 4.0, 8.0]))?;
# Ok::<(), chutoro_core::ChutoroError>(())

When the estimate exceeds the configured limit, run returns ChutoroError::MemoryLimitExceeded before allocating the pipeline. Omitting with_max_bytes leaves this guard disabled.

When the metrics feature is enabled, one-shot runs emit bounded batch metrics. The chutoro.batch.runs_total counter has the labels backend, outcome, and error_code. backend is one of cpu, gpu, or unavailable; outcome is success or error; and error_code is none for successful runs or the stable ChutoroErrorCode string for failures. The resource histograms use only the bounded backend label:

  • chutoro.batch.max_connections (Count) records the configured HNSW connection width.
  • chutoro.batch.effective_ef_construction (Count) records the dataset- bounded HNSW construction-search width.
  • chutoro.batch.estimated_bytes (Bytes) records the estimated peak memory.
  • chutoro.batch.memory_limit_bytes (Bytes) records the configured memory limit when one is present.

The stable ChutoroErrorCode vocabulary includes CHUTORO_INVALID_MIN_CLUSTER_SIZE, CHUTORO_EMPTY_SOURCE, CHUTORO_INSUFFICIENT_ITEMS, CHUTORO_BACKEND_UNAVAILABLE, CHUTORO_DATA_SOURCE_FAILURE, CHUTORO_CPU_HNSW_FAILURE, CHUTORO_CPU_MST_FAILURE, CHUTORO_CPU_HIERARCHY_FAILURE, and CHUTORO_MEMORY_LIMIT_EXCEEDED. The CHUTORO_INVALID_MIN_CLUSTER_SIZE code is reported during builder validation; the remaining applicable codes can be reported as run outcomes. Neither these metrics nor the batch decision-point tracing includes source names, source paths, or source payload data.

The public run_cpu_pipeline entry point has been removed. Migrate callers to ChutoroBuilder::build()?.run(&source), which applies the same validated configuration and run preconditions as the supported batch API.

ExecutionStrategy::Auto runs the CPU backend. The gpu feature prepares the orchestration surface for a future accelerator backend; requesting ExecutionStrategy::GpuPreferred currently yields BackendUnavailable.