4 Quality Checks Every AI-Generated Test Must Pass Before It Exists

Every AI-generated test in my project now passes 4 quality checks before it exists. Not after. Before. This is how we stopped AI from creating technical debt faster than it solved problems.

🤖 Learning AI-powered testing? Go hands-on with LLM, RAG, and AI-agent testing in the AI-Powered Testing Mastery course at The Testing Academy.

Check 1: Assertion Quality Score

AI-generated tests love to assert that elements exist. That is the lowest-value assertion. We score assertions on a 1-5 scale:

• Level 1: Element exists (toBeVisible) — nearly useless alone
• Level 2: Element has correct text (toHaveText) — basic validation
• Level 3: State change verified (before/after comparison) — meaningful
• Level 4: Business logic validated (calculation, workflow completion) — high value
• Level 5: Cross-system verification (API + UI + DB consistency) — production-grade

Gate: Average assertion level must be 3.0 or higher. Tests with only Level 1 assertions are rejected automatically.

Check 2: Selector Reliability Analysis

AI loves generating CSS selectors and XPath. We run every selector through a reliability scorer:

• Green: getByRole, getByLabel, getByTestId — approved
• Yellow: getByText (fragile if text changes) — requires justification
• Red: CSS class selectors, XPath, nth-child — rejected unless no alternative

Check 3: Test Independence Verification

Each AI-generated test must be independently runnable. We verify by running each test in isolation and in random order. Any test that fails when run alone but passes in sequence is rejected.

Check 4: Naming and Documentation

Test names must describe the behavior being verified, not the implementation. We reject names like test1, testLogin, or should work correctly. Accepted pattern: should redirect to dashboard after successful login with valid credentials.

Implementing as a Pre-Commit Hook

#!/bin/bash
# .husky/pre-commit - AI test quality gate

STAGED_TESTS=$(git diff --cached --name-only | grep '.spec.ts$')

if [ -n "$STAGED_TESTS" ]; then
  echo "Running AI test quality checks..."
  npx ts-node scripts/check-assertion-quality.ts $STAGED_TESTS
  npx ts-node scripts/check-selector-reliability.ts $STAGED_TESTS
  npx ts-node scripts/check-test-independence.ts $STAGED_TESTS
  npx ts-node scripts/check-test-naming.ts $STAGED_TESTS
  echo "All 4 quality gates passed"
fi