LLM-Powered Test Generation: Build a Production Pipeline From Requirements to Playwright Tests

LLMs generate tests 50-70% faster than manual writing. But only if you give them the right context. Here is how to build a production-grade test generation pipeline.

🤖 Learning AI-powered testing? Go hands-on with LLM, RAG, and AI-agent testing in the AI-Powered Testing Mastery course at The Testing Academy.

The Test Generation Pipeline

Requirements/PRD
    |
    v
Context Enricher (adds codebase context, existing tests, POM)
    |
    v
LLM Generator (produces test code)
    |
    v
Quality Gate (assertion scoring, locator check, independence)
    |
    v
Human Review (business logic, edge cases)
    |
    v
Merge to Test Suite

Context-Aware Test Generation

class TestGenerator:
    def __init__(self, llm, codebase_context):
        self.llm = llm
        self.context = codebase_context

    def generate(self, requirement: str) -> str:
        prompt = (
            "Generate Playwright TypeScript tests for this requirement.\n"
            "Requirement: {requirement}\n"
            "Existing Page Objects: {self.context.list_page_objects()}\n"
            "Conventions: getByRole locators, POM in src/pages/\n"
            "Generate: positive, negative, and edge cases"
        )
        return self.llm.generate(prompt)

    def validate(self, generated_code: str) -> dict:
        checks = {
            "has_assertions": "expect(" in generated_code,
            "no_xpath": "xpath" not in generated_code.lower(),
            "uses_getByRole": "getByRole" in generated_code,
            "independent": "beforeAll" not in generated_code,
            "has_negative_tests": "error" in generated_code.lower(),
        }
        score = sum(checks.values()) / len(checks)
        return {"score": score, "checks": checks}

Quality Gate Automation

class TestQualityGate:
    MIN_SCORE = 0.8

    def evaluate(self, test_code: str) -> bool:
        scores = {
            "assertion_quality": self._score_assertions(test_code),
            "locator_resilience": self._score_locators(test_code),
            "test_independence": self._score_independence(test_code),
            "naming_quality": self._score_names(test_code),
            "coverage_breadth": self._score_coverage(test_code),
        }
        avg = sum(scores.values()) / len(scores)
        return avg >= self.MIN_SCORE, scores

Integration with Claude Code

# .claude/commands/generate-tests.md

Generate Playwright tests for: $ARGUMENTS

BEFORE generating:
1. Read existing tests in same directory for patterns
2. Read relevant Page Objects in src/pages/
3. Check CLAUDE.md for conventions

Generate tests that:
- Match existing patterns exactly
- Cover positive, negative, boundary, security cases
- Use getByRole() locators
- Include API setup via fixtures
- Look like a teammate wrote them