Evidence Bundles

An evidence bundle is a self-describing directory of statistical tables, accessible SVG figures, and a supplement of excluded or withheld rows, all derived from one completed campaign and filtered to one evidence tier. DispatchAtlas organizes exported evidence into these tiered bundles so that each bundle exposes only the data its evidence tier permits.

🪜 Bundle Tiers

The four tier ids come from a fixed public-safe vocabulary. Each tier is an executable disclosure policy, not an editorial label:

Each bundle is generated from the same tier-filtered dataset as its tables and figures, so a lower-tier export cannot surface higher-tier mechanisms, deployment claims, or withheld rows. The filter fails closed: rather than leaking, an export that would expose a later-tier mechanism under a lower tier raises. Rows a policy forbids are excluded and written to a supplementary exclusions manifest with the reason.

📁 Bundle Contents

A bundle contains:

• statistical tables (solver summaries, pairwise comparisons, rankings, benchmark coverage, infeasible rows),
• accessible SVG figures with a figure manifest recording each figure's role, provenance, and accessibility note,
• a supplement that records excluded or withheld rows and every method routed to limitations,
• bundle.json, the manifest that lists every generated file and hashes the canonical manifest payload.

See analysis exports for the full bundle layout and the statistical methods behind each table and figure.

⚙️ How Bundles Are Produced

One campaign exports to one bundle per tier through the export command:

uv run dispatchatlas export `
  --campaign-dir .\experiments\results\smoke-pilot `
  --target-dir .\experiments `
  --authorized-output-root .\experiments `
  --tier core

The bundle lands at evidence-bundles/{campaign_id}-{tier}/ under the target workspace — the curation layer of the experiments workspace. At scale, tools/build_campaign_evidence.py builds candidate benchmark catalogs, runs the configured campaigns, performs the analysis, and writes the bundles for every tier plus a portal dataset from one recorded configuration:

uv run python tools/build_campaign_evidence.py `
  --output-root .\experiments `
  --problem-count-per-profile 30 `
  --stochastic-seeds 10 `
  --campaign-suffix local

The builder emits phase progress to stderr for catalog materialization, campaign planning, execution, analysis, bundle export, and report writing, and intentionally writes generated evidence outside Git-tracked source trees.

✅ How Bundles Are Verified

• Manifest hashing. bundle.json lists every generated file and hashes the canonical manifest payload, so a bundle's contents are checkable against its own manifest.
• Determinism. Re-running the same export against the same inputs produces the same JSON, table, and figure payloads.
• Derivation, not authority. A bundle is always derived from the run-level records in the reproducibility layer — never an independent source of truth — and those records are themselves content-hashed and replay-verifiable through the campaign engine.
• Gated promotion. Bundles reach public surfaces only through the release and disclosure gates described in release readiness.

The evidence tiers map onto the ACM Artifact Review and Badging vocabulary, version 1.1 (2020), read alongside the stochastic-optimization guidance of López-Ibáñez, Branke and Paquete (2021). Any tier bundle with its content-hashed bundle.json and figure manifest is ready for the Artifacts Evaluated — Functional badge; a bundle published through the public portal is ready for Artifacts Available; a full campaign bundle whose reproducibility manifest pins the seeds, environment, run floor, and the budget and tuning parity is ready for Results Reproduced. Results Replicated calls for an independent implementation built from the written method descriptions alone, which no platform can grant about itself — stated here so the mapping stays honest.

🌐 What The Portal Consumes

The results portal consumes the portal dataset the same export path produces (portal-results.json plus a CSV index) — searchable tags, disclosure labels, objective values, run ids, and hash-linked campaign metadata — rendered into static site assets by tools/build_site_assets.py. The portal never reads the experiments workspace or runs campaigns; it presents committed, disclosure-filtered exports only.