ADR-007: Benchmark Never Changes Runtime
Status: Accepted
Date: 2026-07-05
Deciders: Platform Architecture, AI Platform
Last updated: 2026-07-05
Context
AI evaluation systems often mutate production configuration when tests fail — causing unreviewed behavior changes in live agents.
Decision
Benchmark and regression infrastructure (PL-006) is strictly read-only with respect to production runtime configuration. Benchmark runs produce reports; they never auto-promote prompts, models, or planner settings.
Promotion requires acceptance + human approval + version registry (PL-004).
Consequences
Positive
- Reproducible evaluation
- Clear audit trail from metric → human decision → registry
Negative
- Additional manual step for promotion (intentional)
Compliance
Architecture tests and separate code paths for benchmark runners.
Related
Breadcrumbs: Home → ADR → ADR-007