Benchmark Model

dispatchatlas.bench defines benchmark evidence before solvers or campaigns consume it. A benchmark family declares its taxonomy, domain profile, profile class, assumptions, citation evidence, scale envelope, seed namespace, and output schema. Materialization validates each generated problem with dispatchatlas.core, characterizes the instance, wraps it in a provenance envelope, and records stable hashes.

The catalog covers generic combinatorial-optimization scheduling families and distributed-computing scheduling as co-equal peers, so the platform is not a distributed-computing-only tool.

Runnable example: examples/benchmark_continuum.py generates, characterizes, and catalogs a continuum benchmark atlas on the fly.

Scheduling Families

Each scheduling family is materialized as a first-class catalog peer with at least one generator profile. The catalog table below is generated from the benchmark generator registry and the citation matrix, so its family totals and source citations are countable from the rows themselves. A family-distribution chart above the table shows how the family profiles spread across the scheduling-family categories.

The component above carries the full generated inventory — the classical and distributed-computing core families below plus the continuum of Edge–Fog–Cloud families. Those foundational core families in document form:

The unrelated-machine and flexible job-shop families attach a CostModel execution-time matrix to each instance, exported as a materialized matrix alongside the problem JSON. The sequence-dependent setup flow-shop family instead attaches a CostModel setup matrix: a changeover between jobs of different families on a machine costs setup time, so the setup objective rewards grouping similar jobs. The named standard corpora are cited and linked only and are never redistributed inside the repository.

Several continuum families exercise multi-resource co-allocation: each task demands more than one resource at once and the constructor holds them together for its whole duration (see Domain Contracts). The accelerator-coscheduling family co-allocates a compute node and a scarce accelerator per job; the distributed-transaction family co-allocates a variable-cardinality lock set of data shards per transaction; and the fpga-partitioning family co-allocates a spatially contiguous run of reconfigurable fabric tiles per tenant kernel. Tasks whose resource sets intersect serialize while disjoint tasks run concurrently — the runnable examples/inspect_coallocation.py makes the lever explicit.

Beyond co-allocation, three continuum families exercise their own structural levers. The elastic-serverless-autoscale family exercises moldable execution: each function invocation declares more than one execution mode — a narrow home-only mode and a wide mode that borrows a worker from a small shared burst pool to finish sooner — so the schedule chooses one mode per task and the order decides which invocations claim the scarce wide-and-fast mode. The distributed-training-gang family exercises gang co-scheduling: the workers of a synchronous data-parallel training job share a gang and must co-start on distinct accelerators in an all-or-nothing launch — workers reuse the accelerator pool and jobs arrive over time, so a job cannot begin until enough accelerators free simultaneously, and the order decides which job acquires its full worker set first. The multi-tenant-fair-share family exercises dominant-resource fairness: several asymmetrically-sized tenants place placement-flexible tasks on a shared node pool, and the dominant-resource-share objective scores the spread between the most- and least-served tenant's dominant share — so the placement, which resources each tenant occupies, is the lever that balances or skews it. The runnable examples/serverless_autoscale_study.py, examples/distributed_training_gang_study.py, and examples/multi_tenant_fairshare_study.py make these three levers explicit.

Profile Classes

Evidence Labels

Smoke catalogs are never final evaluation evidence. They exist to prove that generators, validation, characterization, citation checks, and persistence work quickly. The portal catalog and downloads preview each continuum family at this small smoke scale (a three-resource pool); a co-allocation family cannot exhibit disjoint-resource parallelism on so small a pool, so the distinctive structure is a research-scale property. build_continuum_full_catalog() materializes every family at its declared research scale -- the larger resource pool and task count where the co-allocation, contention, and placement structure genuinely manifests -- for research-grade benchmark bundles.

Taxonomy

The taxonomy covers scheduling structures, environments, infrastructure realism, objective features, constraint features, uncertainty, and dynamism. Examples include DAG workflows, independent task batches, serverless functions, container and VM consolidation, edge and cloud environments, public traces, multi-objective optimization, deadlines, data locality, churn, and dynamic arrivals.

Citation Matrix

Benchmark claims are checked against CitationMatrix. Candidate and full-campaign evidence-grade claims fail validation unless they reference citation-backed sources. Unsupported material must stay exploratory until evidence is added.

The source set is declared in default_citation_matrix() with stable source identifiers, resolvable references, and a recorded license posture. It spans three tiers: the standard combinatorial-optimization corpora (cited and linked only, never bundled), production cluster traces, and a broad set of contemporary, real-world Edge–Fog–Cloud-continuum datasets — GPU and machine-learning cluster traces, microservice and serverless benchmark suites, scientific-workflow traces, supercomputer job traces, edge-placement and mobility datasets, IoT and cellular-demand datasets, grid carbon and energy signals, stream-processing workloads, federated-learning device-participation benchmarks, extended-reality systems testbeds, and low-earth-orbit satellite-network traces:

Canonical Reference Suites

The registry in default_reference_suites() records the canonical published instance suites each generic scheduling family anchors to: identity, family, instance count, retrieval pointer, and the best-known-solution trackers that publish bounds for the suite. Classical suites are cited and linked only — DispatchAtlas never bundles or redistributes third-party instance files.

Suites with a bundled parser (standard job-shop text, Taillard flow-shop matrices, .fjs flexible job-shop, WfCommons WfFormat JSON) are ingested with load_reference_suite(suite_id, instances_root=...) from files the operator downloads and places under a local resources/ tree. Ingestion runs entirely offline, reuses the same validation, characterization, hashing, and provenance envelope as the synthetic generators, and stamps every problem with its suite_id and upstream_instance_id. Registry-only suites are recorded with their citations and retrieval pointers without a bundled parser.

Best-Known-Solution Registries

Per-instance best-known values never ship with DispatchAtlas. The operator ingests them as JSON files under a private resources/benchmarks/bks/ directory, one file per suite, each carrying schema_version, the suite_id, the tracker source_id, the retrieval date, and the value entries (instance id, objective, value, optimum-or-upper-bound kind, optional lower bound). load_best_known_registry validates every file against the reference suites and the citation matrix and fails closed on unknown suites, unknown trackers, duplicate entries, or inconsistent bounds. Without an ingested registry, relative-deviation metrics are simply unavailable — they are never partially computed, and no best-known value appears on any public surface.

Calibration Divergences

The synthetic generic families are anchored to the canonical suites without claiming to reproduce their generation schemes. The known divergences are documented rather than hidden:

Comparisons against the published conventions route through the ingested canonical instances, not through the synthetic families.

Characterization

Each materialized problem receives normalized descriptors for:

• opportunity density
• compatibility sparsity
• contention and overload
• dependency depth
• communication pressure
• setup intensity
• load skew and heterogeneity
• objective conflict
• uncertainty and dynamism
• solver sensitivity

Distribution-Distance Bridge

Each full-campaign evidence-grade profile declares a distribution-distance bridge: its status (synthetic, calibrated-synthetic, trace-backed, or externally-sourced), calibration evidence, domain scenario, transfer and disruption coverage, and residual calibration risk. Promotion to the full-campaign evidence grade fails closed unless transfer and disruption coverage are declared, and a calibrated-synthetic profile must name the trace-backed reference it calibrates against. Calibrated-synthetic families name their trace reference explicitly; ingested canonical suites carry an externally-sourced bridge, and the WfCommons adapter is the first externally-parsed trace-backed instance source, giving distribution_distance_score a real trace-backed reference leg.

The named calibration metric reports the per-feature 1-Wasserstein (earth-mover) distance between a calibrated-synthetic profile's characterization feature distributions and those of its trace-backed reference instances. The per-feature distances are aggregated to a single distribution-distance score; a score above the maximum-distance threshold (default 0.25) means the profile has drifted too far from its reference and fails calibration. The metric is reproducible from the materialized instances and the named reference trace.

from dispatchatlas.bench import (
    build_smoke_catalog,
    metrics_from_instance_set,
    distribution_distance_score,
)
 
catalogs = {c.config.profile_id: c for c in build_smoke_catalog()}
synthetic = metrics_from_instance_set(catalogs["cloud-edge-capacity"])
reference = metrics_from_instance_set(catalogs["workflow-dag"])
score = distribution_distance_score(
    synthetic, reference, reference_trace_id="google-cluster-data"
)

Stratification And Subset Selection

difficulty_score aggregates the contention, overload, dependency-depth, and solver-sensitivity descriptors into a normalized difficulty score, and stratify_instances bins materialized instances into low, medium, and high difficulty strata. select_benchmark_subset picks a deterministic subset filtered by family, profile class, and difficulty stratum, ordered by problem identifier so the selection is reproducible.

Smoke Catalog

from dispatchatlas.bench import build_smoke_catalog, smoke_benchmark_provider
 
catalogs = build_smoke_catalog(root_seed=20260527)
provider = smoke_benchmark_provider(root_seed=20260527)
 
first_problem = provider.get_problem(provider.list_problem_ids()[0])

The bundled smoke catalog includes the two distributed-computing families (cloud/edge independent-task and workflow DAG) alongside the seventeen generic scheduling families (machine scheduling, job-shop, flexible job-shop, permutation flow-shop, sequence-dependent setup flow-shop, RCPSP, open-shop, hybrid flow-shop, distributed permutation flow-shop, no-wait flow-shop, blocking flow-shop, distributed assembly flow-shop, multi-objective permutation flow-shop, RCPSP/max, multi-mode RCPSP, multi-project RCPSP, and unrelated-parallel-machine sequence-dependent setup) as co-equal peers. Every instance is deterministic from the root seed and uses citation-backed generator metadata while remaining labeled as smoke development material.

Candidate Full Catalogs

Candidate full-campaign catalogs use the same materialization path with larger configured problem counts and stricter evidence labels:

from dispatchatlas.bench import build_full_catalog, full_benchmark_provider
 
catalogs = build_full_catalog(root_seed=2026052713, problem_count_per_profile=30)
provider = full_benchmark_provider(
    root_seed=2026052713,
    problem_count_per_profile=30,
)

Those full-campaign evidence-grade catalogs are citation-backed, characterized, hash-linked, and still marked as calibration evidence until campaign, disclosure, and publication gates promote specific claims.