| |

AI Agent Failing-Example Review: QA Checklist

ByPromode
AI agent failing-example review checklist with DeepEval and CI release gate

AI agent failing-example review is the QA habit I want every SDET to add before an agent reaches production. Passing scores are useful, but failing examples are where product risk, prompt drift, retrieval gaps, and hidden policy bugs show up first.

I am using DeepEval as the hook because its current PyPI release is 4.1.3, its package metadata says it supports Python >=3.9, and the GitHub repository has more than 17,000 stars. The exact version matters less than the discipline: freeze the dataset, pin the scorer, read the failures, then decide if the agent is safe to ship.

Table of Contents

Contents

What Is an AI Agent Failing-Example Review?

An AI agent failing-example review is a structured inspection of the examples where your agent fails an evaluation, not just a glance at the final pass percentage. The review forces a QA engineer, product owner, or domain expert to read the input, actual output, expected behavior, retrieved context, tool call trace, metric score, and judge reasoning before approving a release.

This is not a generic “AI quality” meeting. It is closer to defect triage. You take the failed examples from an eval run and classify each one into a practical bucket: product bug, prompt bug, retrieval issue, tool issue, dataset issue, evaluator issue, or acceptable behavior change.

Why the name matters

I use the phrase failing-example review because it keeps the team focused on evidence. A dashboard can show 92% pass rate and still hide a failure where a banking assistant gives the wrong EMI calculation, a support agent shares stale refund rules, or a browser agent clicks the destructive button because the selector was ambiguous.

The minimum artifact

A useful review produces one artifact: a short table of failing examples with a decision. I prefer 7 columns because they fit in a pull request comment and a Jira ticket:

  • Case ID: stable identifier such as refund_policy_014.
  • Input: the user prompt or task the agent received.
  • Expected behavior: the product rule or acceptance criterion.
  • Actual behavior: the agent response, tool call, or browser action.
  • Metric and score: for example, answer relevancy 0.42 with threshold 0.70.
  • Failure bucket: prompt, retrieval, tool, product, data, evaluator, or acceptable change.
  • Release decision: block, fix next sprint, update dataset, or accept with reason.

Why Passing Eval Scores Can Still Lie

Eval scores are necessary, but they are not enough. DeepEval’s documentation explains that many predefined metrics use LLM-as-a-judge techniques such as G-Eval, DAG, and QAG, and that metric scores are between 0 and 1 with a threshold for success. That is useful, but a score is still a compressed signal.

The compression is the risk. A single number hides which customer journey failed, which policy was misunderstood, and whether the failure is a blocker for release. In a normal Playwright suite, I can open a trace, watch the click, and inspect the DOM. For an AI agent, I need the same habit applied to prompts, retrieved chunks, tool calls, and evaluator reasoning.

A 90% pass rate can still block release

Imagine a support agent with 100 eval examples. It passes 90 examples. The 10 failures are not equal. One failure might be a harmless tone issue. Another might expose personal data from retrieved context. Another might call the wrong refund API. The release decision depends on the shape of the failures, not the average score.

The evaluator can be wrong too

DeepEval is strong because it gives QA teams a Python-first way to create test cases, datasets, and metrics. It does not remove human judgment. The official docs describe datasets as collections of “goldens” that become test cases at evaluation time. If the golden is stale, the eval can punish a correct answer. If the threshold is too low, the eval can pass weak behavior. If the metric prompt is vague, the judge can reward fluent nonsense.

Agent failures are multi-step failures

A chatbot answer is one output. An agent is a chain of decisions. It may plan, retrieve, call a tool, observe a result, rewrite the plan, and act again. One bad step can poison the rest of the task.

For browser agents, this looks familiar to automation engineers. A wrong locator at step 3 can make step 7 fail with a confusing error. For LLM agents, a wrong retrieved chunk at step 2 can make the final answer confident but wrong. Your review must preserve the trace, not only the final answer.

Where DeepEval Fits in a QA Stack

DeepEval describes itself on PyPI as “The LLM Evaluation Framework.” The current PyPI metadata I checked shows version 4.1.3 uploaded on 22 July 2026 UTC, and the GitHub API reports 17,018 stars and 1,693 forks for the Confident AI DeepEval repository. That adoption matters because QA teams need tools that can survive real CI usage, not demo notebooks only.

DeepEval is not the only option. Promptfoo is popular in the JavaScript and prompt-regression world, and npm reports 1,652,023 downloads for promptfoo in the last month window ending 20 July 2026. LangChain’s OpenEvals project is also worth watching if your team already uses LangSmith and LangGraph. I still like DeepEval for Python-heavy QA teams because it feels close to pytest, datasets, and assertion thinking.

Use it for repeatable evals

The practical value is repeatability. You define LLMTestCase objects, attach metrics, and run the same dataset after a prompt edit, model switch, RAG index refresh, or agent-tool update. That turns AI agent testing from “I tried 5 prompts manually” into a release habit.

from deepeval import evaluate
from deepeval.metrics import AnswerRelevancyMetric, FaithfulnessMetric
from deepeval.test_case import LLMTestCase

cases = [
    LLMTestCase(
        input="Can I get a refund after 45 days?",
        actual_output=refund_agent("Can I get a refund after 45 days?"),
        expected_output="Explain the 30-day refund policy and escalation path.",
        retrieval_context=["Refunds are allowed within 30 days. Exceptions need manager approval."]
    )
]

metrics = [
    AnswerRelevancyMetric(threshold=0.70),
    FaithfulnessMetric(threshold=0.75),
]

evaluate(test_cases=cases, metrics=metrics)

Do not skip the review file

The mistake is to stop after the command exits. Add one more step: export the failures into a review file. The reviewer should not need to open 5 dashboards to understand the risk.

import json
from pathlib import Path

failures = []
for result in eval_results:
    if not result.success:
        failures.append({
            "case_id": result.name,
            "input": result.input,
            "actual_output": result.actual_output,
            "scores": [m.dict() for m in result.metrics_data],
            "review_bucket": "TBD",
            "release_decision": "TBD"
        })

Path("artifacts/ai-agent-failures.json").write_text(
    json.dumps(failures, indent=2),
    encoding="utf-8"
)

Connect it to your existing QA workflow

If your team already uses Playwright, API tests, and CI, do not create a separate “AI quality” island. Put the eval job beside the automation job. Link the failure report in the pull request. Treat a prompt change like a code change. If you need a related foundation, read ScrollTest’s PromptFoo vs DeepEval guide for QA teams and the eval CI portfolio project.

Build a Review Dataset That Does Not Move

The first rule of an AI agent failing-example review is simple: do not review against a moving target. Freeze the dataset used for release decisions. If the examples change every run, you cannot compare failures across commits, prompts, model versions, or RAG index snapshots.

DeepEval documentation says an evaluation dataset is a collection of goldens, and a golden is a precursor to a test case. That model maps nicely to QA thinking. Your golden is the intent. Your test case is the executed example with the agent’s current output.

Start with 50 examples, not 5,000

I prefer a small but sharp dataset for the first release gate. For a support agent, start with 50 examples: 20 happy path, 15 edge cases, 10 policy traps, and 5 adversarial or safety cases. For a browser agent, start with 30 workflows: login, search, filter, create, edit, delete, permissions, timeout, and recovery paths.

Version the dataset like code

Put the dataset in the repository. Add a version field. If product policy changes, update the version and explain the reason in the pull request. Do not quietly edit expected outputs after the eval fails. That is the AI testing version of changing an assertion because the test caught a bug.

{
  "dataset_version": "refund-agent-v1.4",
  "owner": "qa-platform",
  "cases": [
    {
      "id": "refund_policy_014",
      "risk": "high",
      "input": "I bought this 45 days ago. Can I get a refund?",
      "expected_behavior": "State 30-day policy, explain exception path, avoid promising refund.",
      "tags": ["refund", "policy", "edge-case"]
    }
  ]
}

Separate regression and exploration

Do not mix release-gate examples with exploratory generated examples. Keep two folders:

  • evals/regression/ for frozen examples that block release.
  • evals/exploration/ for synthetic, fuzzed, or newly discovered examples.

Exploration feeds regression. When a new failure exposes real risk, promote it into the frozen dataset with a stable case ID. This is how your AI suite gets smarter every sprint without becoming random noise.

Pin Scorers, Thresholds, and Versions

The second rule is to pin the scorer. If the evaluator model changes, the metric prompt changes, or the threshold changes, your pass rate can move even when the product code did not. That creates false confidence or false panic.

DeepEval’s metric docs say a metric succeeds when the score is equal to or greater than its threshold, with a default threshold of 0.5 for many metrics. I rarely accept the default for release gates. For user-facing support answers, I usually start answer relevancy at 0.70 and faithfulness at 0.75, then calibrate after reading failures.

Create a scorer manifest

Every eval run should emit a manifest. The manifest records the DeepEval version, Python version, metric names, thresholds, evaluator model, dataset version, product commit, and retrieval index version. Without this file, you cannot explain why yesterday’s pass rate was 87% and today’s is 74%.

{
  "run_id": "2026-07-22T06-30Z",
  "deepeval_version": "4.1.3",
  "python": "3.11",
  "dataset_version": "refund-agent-v1.4",
  "git_sha": "9f2c1ab",
  "retrieval_index": "policies-2026-07-20",
  "metrics": [
    {"name": "AnswerRelevancyMetric", "threshold": 0.70},
    {"name": "FaithfulnessMetric", "threshold": 0.75}
  ]
}

Pin dependencies in CI

Pin the eval framework in the same way you pin Playwright, Selenium, or browser versions. If your team upgrades DeepEval from 4.1.1 to 4.1.3, run a release-impact check first. I wrote a similar pattern for automation dependencies in AI eval dependency monitoring for QA teams and AI release watcher for QA.

deepeval==4.1.3
pytest==8.4.1
python-dotenv==1.0.1

Calibrate with known failures

A scorer that cannot catch known bad behavior should not block production. Keep 10 known failures in a calibration folder. Run them before trusting a new metric or evaluator prompt. If 8 out of 10 known bad examples pass, the metric is not ready for release gating.

Add the Human Review Loop

The third rule is the most ignored: failures must be read by a human. I do not mean every developer must spend 2 hours reading logs. I mean the release owner must inspect the representative failures and sign off on the risk.

The human review loop is not anti-automation. It is mature automation. Even in classic Selenium and Playwright suites, we still inspect screenshots, traces, videos, and failed assertions when the build blocks a release. AI evals deserve the same respect.

Use a fixed review checklist

Use a checklist so every reviewer classifies failures the same way. Here is the 7-step review I use:

  1. Open the failure artifact for the current run ID.
  2. Group failures by risk tag: policy, safety, money, privacy, workflow, tone.
  3. Read the lowest scoring 10 examples first.
  4. Check if retrieved context supports the answer.
  5. Compare actual behavior with the expected behavior field.
  6. Assign a failure bucket and release decision.
  7. Create one fix ticket per root cause, not one ticket per noisy failure.

Failure buckets stop vague debates

The bucket matters because it sends the bug to the right owner. A retrieval issue goes to the indexing or data pipeline owner. A prompt issue goes to the agent owner. A product rule gap goes to the product manager. An evaluator issue goes back to the QA platform team.

Without buckets, teams argue in Slack for 45 minutes. With buckets, the release conversation becomes precise: “3 high-risk failures are retrieval issues from the July policy index, 2 are evaluator false positives, and 1 blocks release because the agent calls the wrong tool.”

Keep the reviewer close to the domain

A random engineer should not approve a payroll agent’s failures. A domain-aware reviewer should. For Indian QA teams working with banking, insurance, healthcare, HR, or ed-tech products, this is important because local rules and customer language vary by domain. A reviewer who understands the product catches subtle mistakes faster than a generic evaluator.

Turn the Review Into a CI Gate

A review that lives only in a spreadsheet will die. Put the failing-example review into CI. The first version does not need a complex dashboard. It needs a command, an artifact, and a release rule.

The release rule should be boring and explicit. For example: block release if any high-risk case fails faithfulness below 0.75, block if more than 5 policy cases fail, warn if tone failures exceed 10%, and require human sign-off for every safety failure.

GitHub Actions example

Here is a simple workflow. It installs pinned dependencies, runs the DeepEval suite, uploads artifacts, and fails the job when the gate script detects release-blocking failures.

name: ai-agent-evals

on:
  pull_request:
    paths:
      - "agents/**"
      - "evals/**"
      - "requirements-eval.txt"

jobs:
  eval:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - run: pip install -r requirements-eval.txt
      - run: deepeval test run evals/regression/test_refund_agent.py
      - run: python scripts/export_eval_failures.py
      - run: python scripts/check_ai_release_gate.py artifacts/ai-agent-failures.json
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: ai-agent-failure-review
          path: artifacts/

Gate script example

The gate script should be readable. QA engineers should be able to change the release rule without rewriting the whole framework.

import json
import sys
from pathlib import Path

failures = json.loads(Path(sys.argv[1]).read_text(encoding="utf-8"))
blockers = []

for failure in failures:
    risk = failure.get("risk", "medium")
    scores = {m["name"]: m["score"] for m in failure.get("scores", [])}
    if risk == "high" and scores.get("FaithfulnessMetric", 1.0) < 0.75:
        blockers.append(failure["case_id"])

if blockers:
    print("Blocking AI agent release. Cases:", ", ".join(blockers))
    sys.exit(1)

print("AI agent eval gate passed with human-reviewable artifacts.")

Do not make every failure a blocker

If every failure blocks release, teams will disable the eval job. Use severity. A tone issue in a low-risk FAQ answer can be a warning. A wrong refund promise can be a blocker. A hallucinated legal claim should be a blocker. A false positive from the evaluator should update the evaluator prompt or threshold.

India QA Career Context

For QA engineers in India, this skill is becoming career-relevant fast. Manual testing alone is under pressure. Selenium-only resumes look incomplete for product companies hiring SDETs. Teams now ask for Playwright, API testing, CI/CD, and increasingly AI evaluation basics.

I do not claim one tool will move your salary. But I do see a pattern in interviews: candidates who can explain eval datasets, scorer thresholds, failure triage, and CI gates sound closer to SDET-2 or QA platform roles than candidates who only say “I tested ChatGPT manually.” In Bengaluru, Hyderabad, Pune, and NCR product companies, that difference can affect whether you are considered for ₹25-40 LPA roles or only routine execution work.

What to build for your portfolio

Build one portfolio project: a small support agent with 50 frozen examples, 3 metrics, a GitHub Actions eval gate, and a failure-review report. Put the dataset, manifest, and artifacts in the repo. Record a 3-minute demo. This is stronger than saying “I know AI testing” on a resume.

  • Use Python 3.11 and DeepEval for the eval suite.
  • Use 50 product-like examples, not random chatbot prompts.
  • Show 5 real failures and how you classified them.
  • Explain why one failure blocks release and another does not.
  • Add a README with the exact command: deepeval test run evals/regression.

How managers should adopt this

If you manage a QA team, do not start with a giant AI governance program. Start with one agent and one release gate. Assign one SDET as eval owner for 2 weeks. Ask for a weekly failure-review note with 3 numbers: pass rate, blocker count, and top failure bucket.

Key Takeaways

The AI agent failing-example review is the missing bridge between eval scores and release confidence. I do not trust an agent because a dashboard is green. I trust it when the team can explain its failures.

  • Freeze the dataset before using eval results for release decisions.
  • Pin DeepEval, scorer models, metric prompts, and thresholds.
  • Read the failures, especially high-risk policy, money, privacy, and safety cases.
  • Classify each failure into a root-cause bucket before creating tickets.
  • Put the eval and review artifact into CI so the habit survives busy sprints.

If you want a next practical step, take one agent in your product and create 20 goldens this week. Run the suite. Read the failures. That 1-hour review will teach you more than another abstract AI testing discussion.

FAQ

What is an AI agent failing-example review?

It is a QA review of the examples where an AI agent fails an evaluation. The reviewer inspects the input, expected behavior, actual output, metric score, trace, and root cause before approving or blocking a release.

Can DeepEval replace manual QA for AI agents?

No. DeepEval can automate repeatable evaluation checks, but human review is still needed for domain risk, ambiguous failures, stale expected behavior, and evaluator false positives.

How many examples should I start with?

Start with 30 to 50 strong examples. Cover happy paths, edge cases, policy traps, safety cases, and real production complaints. Add more only after the team is actually reading failures.

Should every eval failure block release?

No. Use risk-based rules. A high-risk faithfulness failure can block release. A minor tone failure may become a warning. The important part is that the rule is written before the release day.

Which tools should QA engineers compare with DeepEval?

Compare DeepEval with Promptfoo for prompt regression workflows and OpenEvals if your team is already in the LangChain ecosystem. Pick the tool that fits your stack, then enforce the same failing-example review discipline.

Sources: DeepEval PyPI package metadata, DeepEval official docs on evaluation, datasets, and metrics, GitHub API for Confident AI DeepEval repository data, npm downloads API for promptfoo monthly downloads, and LangChain OpenEvals.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.