The final answer is not the whole execution
Google Cloud’s first lesson is deceptively simple: an evaluator should inspect what an agent did, not only what it said at the end. The team initially used a model judge to score final replies against a checklist. In an eleven-prompt suite, that grader marked three cases as failures even though the raw tool traces showed the required actions had happened. In one example, the agent checked command syntax before a Cloud Run cleanup but used its final response to emphasise the risk of permanent deletion rather than reciting the check.
This distinction matters because agents are processes, not conventional chatbots. A polished summary can conceal an unsafe command, while a terse warning can follow a correct sequence of checks. Evaluation therefore needs access to structured tool calls, arguments, return values, approvals and state changes. Outcome quality remains important, but it should be connected to evidence about the route taken.
Build graders around observable facts
Deterministic checks should carry the load when a fact can be measured directly. A test can verify whether a required API was called, whether a proposed command included a dry-run flag, whether a resource remained unchanged, or whether an approval occurred before a destructive step. These assertions are cheaper, more repeatable and easier to debug than asking another language model to interpret every trace.
Model-based graders still have a role where judgement is genuinely semantic, such as assessing whether a plan addressed an ambiguous user goal. Their rubrics should be narrow and supported by the relevant trace segments. Teams should record the grader model and prompt version because a silent change to either can move scores even when the agent stays constant. A small human-reviewed calibration set helps expose that drift.
Separate agent failure from test failure
An evaluation harness has its own dependencies, credentials, quotas and transient errors. A timeout from a cloud API is different from an agent choosing the wrong tool, yet a naïve score may label both as task failure. The harness should classify infrastructure faults separately, retain raw evidence and allow safe retries. Otherwise, teams risk optimising prompts to compensate for unreliable test plumbing.
This separation also improves incident ownership. Model and orchestration teams can investigate reasoning or routing defects, while platform teams handle expired credentials and unavailable services. A dashboard that combines the categories into one pass rate may be convenient, but it obscures whether a release became less capable or the laboratory simply became noisier.
Evaluation suites are production assets
Prompts, tools and cloud services change continuously, so an agent evaluation suite needs version control, code review and maintenance. Cases should represent real user intents, include expected safety boundaries and cover common as well as high-impact rare operations. When a production incident reveals a new failure mode, a regression case should be added before the fix is considered complete.
The test data also needs governance. Traces can contain source code, resource names, customer information and credentials returned by poorly designed tools. Access, retention and redaction policies should be defined before large-scale evaluation begins. Synthetic cases are useful for coverage, but they should not become a substitute for carefully sanitised examples of how people actually use the agent.
A practical release gate
A mature gate can combine deterministic invariants, semantic quality judgements and operational reliability. It might require every destructive action to obtain approval, set a minimum task-success rate on representative cases, and cap the rate of tool or infrastructure errors. Results should be sliced by task type because an aggregate score can improve while performance on a critical workflow falls.
Google’s lessons are valuable precisely because they come from a plugin that acts on cloud resources. The same discipline applies to coding agents, research systems and business-process automation. The durable principle is to evaluate the behaviour users depend on, preserve enough evidence to explain failures, and maintain the measurement system with the same care as the agent itself.
From test case to operational learning
Evaluation should not end when a release passes its gate. Production traces can reveal new intents, tool combinations and edge cases that the original suite never represented. A sampling process should select informative sessions without collecting more sensitive data than necessary, and reviewers should be able to turn a confirmed failure into a stable test. Trend reporting then needs to distinguish capability improvement from a changing user mix. If easier tasks suddenly dominate traffic, the overall pass rate may rise while the agent remains unchanged. Conversely, a lower aggregate score can accompany successful expansion into harder work. Maintaining fixed benchmark cohorts alongside live-distribution measures gives leaders both comparability and relevance. It also prevents teams from celebrating a metric shift that came from routing or sampling rather than a safer, more useful agent. Review quality needs its own audit.