4 Quality Checks Every AI-Generated Test Must Pass Before It Exists
Every AI-generated test in my project now passes 4 quality checks before it exists. Not after. Before. This is how we stopped AI from creating technical debt faster than it solved problems.
Contents
Check 1: Assertion Quality Score
AI-generated tests love to assert that elements exist. That is the lowest-value assertion. We score assertions on a 1-5 scale:
- Level 1: Element exists (toBeVisible) — nearly useless alone
- Level 2: Element has correct text (toHaveText) — basic validation
- Level 3: State change verified (before/after comparison) — meaningful
- Level 4: Business logic validated (calculation, workflow completion) — high value
- Level 5: Cross-system verification (API + UI + DB consistency) — production-grade
Gate: Average assertion level must be 3.0 or higher. Tests with only Level 1 assertions are rejected automatically.
Check 2: Selector Reliability Analysis
AI loves generating CSS selectors and XPath. We run every selector through a reliability scorer:
- Green: getByRole, getByLabel, getByTestId — approved
- Yellow: getByText (fragile if text changes) — requires justification
- Red: CSS class selectors, XPath, nth-child — rejected unless no alternative
Check 3: Test Independence Verification
Each AI-generated test must be independently runnable. We verify by running each test in isolation and in random order. Any test that fails when run alone but passes in sequence is rejected.
Check 4: Naming and Documentation
Test names must describe the behavior being verified, not the implementation. We reject names like test1, testLogin, or should work correctly. Accepted pattern: should redirect to dashboard after successful login with valid credentials.
Implementing as a Pre-Commit Hook
#!/bin/bash
# .husky/pre-commit - AI test quality gate
STAGED_TESTS=$(git diff --cached --name-only | grep '.spec.ts$')
if [ -n "$STAGED_TESTS" ]; then
echo "Running AI test quality checks..."
npx ts-node scripts/check-assertion-quality.ts $STAGED_TESTS
npx ts-node scripts/check-selector-reliability.ts $STAGED_TESTS
npx ts-node scripts/check-test-independence.ts $STAGED_TESTS
npx ts-node scripts/check-test-naming.ts $STAGED_TESTS
echo "All 4 quality gates passed"
fi
