Do not trust the demo.Replay the work.
Replay Lab gives a Copy Agent historical situations it did not observe live, measures the decisions it makes and exposes where the behavior diverges before production customers, data or revenue are involved.
Replay results are evaluation evidence. Example metrics on this page illustrate the framework and are not client performance claims.
A successful output can still come from unsafe behavior.
Most agent evaluation asks whether the final answer looks correct. Operational work also depends on how the answer was reached, which tools were used, what the agent changed and whether it stopped when the role ended.
Replay freezes realistic historical scenarios and runs them repeatedly across new prompts, models, skills and policies. That makes behavior changes visible instead of anecdotal.
Every mismatch becomes evidence: either the agent is wrong, the policy is incomplete, or the historical process itself deserves redesign.
Test competence, boundaries and recovery separately.
Decision replay
Compare the selected path with approved policy and acceptable alternatives.
- Case-level agreement
- Reason codes
- Alternative path analysis
Boundary replay
Test whether the agent stays inside its role when useful adjacent actions are available.
- Stop-condition tests
- Permission traps
- Self-created goal detection
Recovery replay
Introduce missing data, bad APIs and contradictory evidence to inspect failure behavior.
- Retry policy
- Graceful degradation
- Escalation quality
Cost and latency
Measure whether a behavior is operationally sustainable at realistic volume.
- Token and tool cost
- P95 latency
- Throughput limits
Regression suites
Rerun critical cases whenever prompts, models, tools or policies change.
- Version comparison
- Critical-case gates
- Release notes
Production feedback
Convert corrections and real exceptions into new replay cases.
- Drift signals
- Human corrections
- Incident-derived tests
Turn operating history into a release gate.
Select representative history
Sample routine cases, edge cases and known failures.
Reconstruct the state
Preserve the data and tool results available at decision time.
Define acceptable behavior
Document expected paths, alternatives and forbidden actions.
Execute the agent trace
Run the same scenarios across the candidate release.
Review mismatch and release
Block or stage deployment according to agreed thresholds.
The safest place to observe bad behavior is a controlled replica.
Evidence before autonomy.
Is Replay Lab a live sandbox product?+
It is the validation system and delivery method used around Copy Agent implementations. Its exact deployment depends on the workflow, data and infrastructure involved.
Can historical cases contain sensitive data?+
Yes, but they should be minimized and transformed where possible. We design redaction, tokenization and access controls around the evaluation requirement.
What is a good behavior-match score?+
There is no universal threshold. A low-risk drafting role can tolerate different alternatives; a finance or customer-impacting action may require near-perfect performance on critical cases and mandatory escalation elsewhere.
Does replay replace production monitoring?+
No. Replay reduces known risk before release. Production monitoring detects drift, new edge cases and changes in connected systems.