TL;DR
AI agent verification should improve the next decision, not simply produce more tool calls. Select diagnostics that distinguish plausible explanations, enforce access and resource limits outside the model, and stop when further investigation is not justified. Keep evidence, approval, execution, and service recovery as separate states. A supported diagnosis can justify a change proposal, but it cannot grant permission to execute it. That permission must remain bound to the exact action and valid when execution occurs.
Introduction
An AI assistant investigating a checkout outage has identified several plausible explanations. Instead of choosing between them, it queries another dashboard, repeats a health check, searches previous incidents, and asks a second model to review its answer.
The investigation is now longer. The decision is not necessarily better.
Part 1, AI Confidence Is Not Evidence: Building an Evidence Contract, established how to qualify observations before using them. External checkout requests were timing out, node-health observations were normal, and a gateway configuration had recently changed. Those facts narrowed the investigation without establishing a root cause.
This article continues that illustrative scenario. The assistant has scoped diagnostic access, production changes require human approval, and ordinary incident processing does not update model weights. The workflow and policy below are proposed engineering patterns, not a validated implementation.
The next question is practical: What should the agent verify, when should it stop, and what must remain outside its authority even after the evidence improves?
Ask a Question That Could Change the Response
A useful diagnostic tests a distinction between explanations. It does not merely collect another presentation of information already available.
For the checkout incident, an approved internal application check could help distinguish a backend problem from a problem affecting the external request path. Comparing the deployed gateway route with its approved configuration could test a different hypothesis. Repeating the unchanged node-health summary may add little to either question.
| Candidate diagnostic | Question it tests | Interpretation boundary |
|---|---|---|
| Compare approved internal and external application checks | Does the failure affect both tested paths? | Different identities, timing, and request profiles can explain different results |
| Compare deployed and approved gateway configurations | Does the relevant route differ from the approved state? | A difference does not, by itself, establish causation |
| Inspect the registered probe’s collection status | Did the probe produce a valid observation? | A working collector does not prove that its request profile is correct |
| Repeat the existing node-health query | Has relevant node state changed? | Without a meaningful change or freshness need, this may repeat existing evidence |
Before a call is admitted, record the question, expected result classes, and how each result would affect the next response. Include an inconclusive result rather than forcing every tool response into success or failure.
Keep comparisons meaningful. The internal and external checks should exercise comparable application behavior, with differences documented. A successful internal liveness endpoint cannot establish that the externally exposed checkout operation works.
This turns the diagnostic plan into something an operator can review before the results arrive, rather than a justification reconstructed afterward.
Distinguish Information Gain from Decision Value
Poole and Mackworth’s Artificial Intelligence: Foundations of Computational Agents describes information value through its effect on decisions. Learning more is not automatically worth the cost of obtaining that information.
For a simplified, non-disruptive diagnostic, express the net value as:
Architecture & integration
Hybrid Platforms
Architectures that connect VCF, Azure, public cloud, Kubernetes, and edge.
Explore Hybrid Platforms →
Day-2 execution
Operations & Resilience
Security, recovery, lifecycle, observability, capacity, and FinOps.
Explore Operations →
