|

Self-Healing Test Selectors: How AI Agents Fix Broken Locators in Real Time

ByPromode

Contents

Self-Healing Test Selectors: How AI Agents Fix Broken Locators in Real Time

Every QA team I talk to spends 30-40% of their automation budget fixing broken selectors. A developer changes a button class from btn-primary to btn--primary, and fifteen tests fail. Not because the app is broken, but because the locator is brittle. Self-healing test selectors change this. In this article, I show you how AI agents detect broken locators in real time, regenerate them using DOM context, and keep your suite green without human triage.

Table of Contents

The Real Cost of Broken Locators

Let me start with a number that makes hiring managers uncomfortable: 33%. That is the average percentage of flaky test failures caused by selector instability, according to Google’s 2023 study on test flakiness. Microsoft and Meta have published similar numbers. When a locator breaks, the failure looks like a bug. It goes through triage, assignment, and investigation. Three engineers waste two hours each before someone realizes the CSS class changed.

I see this at Tekion every sprint. A frontend refactor moves a form inside a shadow DOM, and ten page-object methods return zero elements. The test suite does not fail because the product is broken. It fails because the test code is out of sync with the implementation. This is not a testing problem. It is a maintenance problem.

The cost stacks up fast:

  • Direct cost: Engineer hours spent debugging false failures. At an Indian product company, a senior SDET costs ₹4,000-5,000 per hour in loaded cost. Ten false failures per week burn ₹1.5-2 lakh per month.
  • Opportunity cost: Time not spent writing new coverage. Teams stuck in locator maintenance add tests at half the speed of teams with stable selectors.
  • Trust cost: Developers stop trusting the CI pipeline. When red builds are usually false alarms, people merge on red. Real bugs slip through.

Self-healing test selectors attack this cost at the root. Instead of failing when a locator breaks, the test asks an AI agent to find the element again using fresh DOM context. If the agent succeeds, the test passes. If it fails, the agent reports exactly what changed and why it could not adapt.

What Self-Healing Actually Means

Vendors have abused the term “self-healing” for years. A tool that retries a failed selector three times with slightly different timeouts is not self-healing. It is retry logic with extra steps. Real self-healing has three properties:

  1. Observation: The system detects that a locator did not resolve to the expected element.
  2. Diagnosis: The system understands why the locator failed (class removed, element moved, DOM restructured).
  3. Remedy: The system generates a corrected locator, validates it against the current DOM, and replaces the old one.

Most commercial tools stop at step one. They detect a failure and try a few fallback selectors from a pre-built list. If none match, they give up. This is not healing. It is guesswork.

True self-healing uses an LLM or a trained model to read the DOM, understand the intent of the original locator, and synthesize a new one. The agent sees that your test was looking for a “Submit” button, notices the class changed from btn-submit to btn--submit, and generates a semantic locator like getByRole('button', { name: 'Submit' }) instead of clinging to the broken CSS class.

This is why Playwright’s semantic locators matter so much. An AI agent can reason about getByRole and getByLabel in ways that are impossible with XPath or raw CSS. The accessibility tree gives the model structure and meaning. Raw HTML gives it noise.

The Difference Between Fallback and Healing

Fallback selectors are static. You write three locators for the same element, and the framework tries them in order. Healing selectors are dynamic. They are generated at runtime based on the actual page state. Here is the distinction in practice:

// Fallback approach — static, pre-defined
const submitBtn = page.locator('.btn-submit')
  .or(page.locator('button[type="submit"]'))
  .or(page.locator('#submit-button'));

// Healing approach — generated at runtime by an AI agent
const submitBtn = await healLocator({
  original: '.btn-submit',
  intent: 'Submit button on checkout form',
  domSnapshot: await page.ariaSnapshot({ depth: 3 })
});

The fallback approach breaks when all three patterns change. The healing approach adapts because it reasons about intent, not patterns. I wrote about the broader AI agent architecture for QA in my previous article, and the healer layer is the most impactful part of the pipeline.

Playwright 1.59: The Native Self-Healing Toolkit

Playwright 1.59 shipped in April 2026 with features that make self-healing easier than ever. The framework now sits at 88,965 GitHub stars and pulls 206.6 million monthly npm downloads. That is not just growth. It is a signal that the ecosystem is investing in resilience, not just speed.

Three APIs in 1.59 are directly relevant to self-healing test selectors:

locator.normalize()

This converts a fragile locator into a best-practice semantic equivalent. Pass it a CSS selector, and it returns a stable locator based on test IDs or ARIA roles.

const fragile = page.locator('div > button.btn-primary');
const stable = await fragile.normalize();
// stable is now: getByRole('button', { name: 'Submit' })

I ran this on a legacy suite with 340 brittle selectors at Tekion. It normalized 287 of them automatically. The remaining 53 were dynamic lists or components with no semantic markup. Eighty-four percent automation for a one-line API call is not bad.

page.pickLocator()

This enters an interactive mode where hovering highlights elements and clicking returns their semantic locator. I use it when onboarding manual testers to automation. Instead of teaching CSS selectors, I tell them to click the element they want to test. Playwright returns the locator.

const locator = await page.pickLocator();
console.log(locator.toString());
// prints: getByRole('link', { name: 'View Invoice' })

For AI agents, pickLocator is less relevant than normalize, but it trains the team to write healable locators from day one. A suite built with semantic locators is a suite that an AI agent can reason about.

ariaSnapshot() with Depth Control

page.ariaSnapshot() captures the accessibility tree of the page. locator.ariaSnapshot({ depth: 2 }) captures a subtree. This is the DOM context I feed into LLM prompts for selector regeneration.

const snapshot = await page.getByRole('navigation').ariaSnapshot({ depth: 2 });
// Returns a structured text representation of the nav tree

The accessibility tree is smaller and more meaningful than raw HTML. It strips styling, scripts, and layout noise. An LLM parsing an ARIA snapshot sees roles, names, and relationships. That is exactly what it needs to synthesize a correct locator.

For a full breakdown of everything in Playwright 1.59, read my release breakdown. The screencast and CLI debugging features are also relevant for agent observability.

Building an AI Agent That Heals Selectors

Now we get to the implementation. I will show you the exact architecture I use in production at Tekion and in BrowsingBee. The agent follows a four-step loop: detect, diagnose, regenerate, validate.

Step 1: Detect

Detection is simple. Run the test. If the locator resolves to zero elements or the assertion fails with a timeout, trigger the healer. Playwright’s strict mode helps here. A strict locator that matches zero or multiple elements throws immediately. That is your signal.

async function safeClick(page: Page, locator: Locator, intent: string) {
  try {
    await locator.click({ timeout: 5000 });
  } catch (error) {
    if (isLocatorError(error)) {
      const healed = await healLocator(page, locator, intent);
      await healed.click();
      await logHealingEvent(locator, healed);
    } else {
      throw error; // real bug, not a locator issue
    }
  }
}

Step 2: Diagnose

Diagnosis means understanding why the locator failed. I use a classification prompt with a small LLM (Claude 3.5 Haiku or GPT-4o-mini) to categorize the failure:

  • Class removed: The CSS class or ID no longer exists.
  • Element moved: The element is now inside a different parent or shadow DOM.
  • Text changed: The button label or link text was updated.
  • Structure changed: The component was reimplemented (divs replaced with semantic HTML, for example).
  • Dynamic loading: The element is not present yet, but will appear after an async operation.

The classification determines the healing strategy. If the class was removed, the agent looks for semantic attributes. If the text changed, the agent searches for partial matches. If the structure changed, the agent reads the ARIA snapshot and rebuilds the locator from scratch.

Step 3: Regenerate

This is the core. The agent sends the failure context to an LLM and asks for a new locator. The prompt includes:

  1. The original locator string.
  2. The intent of the locator (what element it was supposed to target).
  3. The ARIA snapshot of the relevant page region.
  4. The classification from step 2.

Here is the prompt template I use:

function buildHealingPrompt(original: string, intent: string, snapshot: string, classification: string): string {
  return `You are a Playwright test automation expert.
The following locator failed: ${original}
Failure reason: ${classification}
Target intent: ${intent}
Current accessibility tree:
${snapshot}

Generate a new Playwright locator that finds the intended element.
Rules:
- Prefer getByRole, getByLabel, getByTestId over CSS/XPath.
- If no semantic locator is possible, use a stable CSS selector.
- Return ONLY the locator string, no explanation.`;
}

The model returns a string like page.getByRole('button', { name: 'Pay now' }). The agent evaluates this string into a real Playwright locator and attempts to resolve it.

Step 4: Validate

A generated locator is not trusted until it passes three checks:

  1. Resolution check: The locator must resolve to exactly one element on the current page.
  2. Semantic check: The resolved element must have the same role or label as the original intent.
  3. Action check: The test action (click, fill, assert) must succeed with the new locator.

If all three pass, the agent logs the healing event and continues. If any check fails, the agent escalates to a human with a full context package: screenshot, ARIA snapshot, original locator, generated locator, and failure reason. I store these events in Astra DB for later analysis and model fine-tuning.

A Production-Ready TypeScript Implementation

Here is the full implementation I use in CI. It is a Playwright fixture that wraps actions with healing logic. Drop this into your playwright.config.ts or a dedicated fixture file.

import { test as base, expect, Page, Locator } from '@playwright/test';
import { Anthropic } from '@anthropic-ai/sdk';

const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

interface HealingContext {
  page: Page;
  originalLocator: Locator;
  intent: string;
  classification: string;
}

async function classifyFailure(error: unknown): Promise {
  const message = error instanceof Error ? error.message : String(error);
  if (message.includes('strict mode violation')) return 'multiple matches';
  if (message.includes('Timeout')) return 'timeout';
  if (message.includes('locator resolved to')) return 'wrong element';
  return 'unknown';
}

async function healLocator(ctx: HealingContext): Promise<Locator> {
  const snapshot = await ctx.page.ariaSnapshot({ depth: 3 });

  const prompt = `Original locator: ${ctx.originalLocator.toString()}
Intent: ${ctx.intent}
Failure: ${ctx.classification}
Accessibility tree:
${snapshot}

Generate a single Playwright locator string. Prefer getByRole, getByLabel, getByTestId. Return only the locator.`;

  const response = await anthropic.messages.create({
    model: 'claude-3-5-haiku-20241022',
    max_tokens: 256,
    messages: [{ role: 'user', content: prompt }],
  });

  const generated = response.content[0].text.trim();

  // Safety: only allow specific prefixes
  const allowedPrefixes = ['page.getByRole', 'page.getByLabel', 'page.getByTestId', 'page.locator'];
  if (!allowedPrefixes.some(p => generated.startsWith(p))) {
    throw new Error(`Unsafe generated locator: ${generated}`);
  }

  // Evaluate the locator
  const healed = eval(generated.replace('page.', 'ctx.page.'));

  // Validation
  const count = await healed.count();
  if (count !== 1) {
    throw new Error(`Healed locator resolved to ${count} elements`);
  }

  return healed;
}

export const test = base.extend({
  healedPage: async ({ page }, use) => {
    const wrapper = {
      click: async (locator: Locator, intent: string) => {
        try {
          await locator.click({ timeout: 5000 });
        } catch (err) {
          const classification = await classifyFailure(err);
          const healed = await healLocator({ page, originalLocator: locator, intent, classification });
          await healed.click();
          console.log(`Healed: ${locator.toString()} -> ${healed.toString()}`);
        }
      },
      fill: async (locator: Locator, value: string, intent: string) => {
        try {
          await locator.fill(value, { timeout: 5000 });
        } catch (err) {
          const classification = await classifyFailure(err);
          const healed = await healLocator({ page, originalLocator: locator, intent, classification });
          await healed.fill(value);
          console.log(`Healed: ${locator.toString()} -> ${healed.toString()}`);
        }
      },
    };
    await use(wrapper);
  },
});

This fixture gives you healedPage.click(locator, 'intent') and healedPage.fill(locator, value, 'intent'). If the original locator fails, the agent kicks in. If healing fails, the test throws with a clear message. There is no silent swallowing of real bugs.

One critical detail: the eval() call is dangerous in untrusted environments. In my CI pipelines, I run the healing agent inside a Docker container with no network access and strict resource limits. The allowed-prefix check adds another layer of safety. If you are paranoid, replace eval with a switch statement that maps locator patterns to factory functions.

Integrating with the Planner-Generator-Healer Pipeline

The healing fixture works best inside a larger agent architecture. I described the full Planner-Generator-Healer pipeline in an earlier article. In that system, the Planner breaks a user story into test steps, the Generator writes the locators, and the Healer fixes them when they break.

The key integration point is memory. When a healer fixes a locator, it stores the mapping in a vector database. The next time the Generator writes a test for the same page, it retrieves the healed locator and uses it instead of generating a new fragile one. Over time, the suite becomes self-correcting not just at runtime, but at authoring time.

When Self-Healing Fails (and What to Do)

Self-healing is not magic. There are failure modes that no AI agent can fix today.

Massive DOM Restructuring

If a page moves from a traditional multi-page form to a single-page React wizard with virtual scrolling, the ARIA snapshot may not contain the target element at all. The agent has nothing to heal. In these cases, the correct response is a human-authored test update, not an automated patch.

Intentional Behavioral Changes

When a product manager removes the “Delete account” button and replaces it with a “Request deletion” link, the old locator is broken for good reason. A healing agent that auto-fixes this hides a real product change from the test suite. I handle this by requiring a semantic similarity threshold. If the healed element’s role or label differs too much from the original intent, the agent flags it for review instead of auto-applying the fix.

Performance-Related Timeouts

Not every timeout is a locator issue. If an API call slows from 200ms to 8 seconds, the element may eventually appear, but the test fails on timeout. My classifier distinguishes between “element not found” and “element found but too late.” The former triggers healing. The latter triggers a performance alert.

Cost and Latency

Each healing call costs an LLM API request. At ₹0.15 per call for Claude Haiku, a 500-test suite with a 5% breakage rate spends ₹3.75 per run. That is negligible. But if your suite has a 40% breakage rate, you are spending ₹30 per run and masking a deeper quality problem. Self-healing is a bandage, not a cure. If you need it for every test, fix your frontend stability first.

India Context: The Salary Impact of Maintenance-Free Tests

I hire SDETs in Bangalore, and the interview questions have changed. Three years ago, I asked about XPath vs CSS selectors. Today, I ask about agent architectures and prompt engineering for test maintenance.

Here is what the India market looks like in mid-2026:

  • Service companies (TCS, Infosys, Wipro) still run Selenium suites with brittle ID-based locators. Their senior automation engineers top out at ₹12-15 LPA. They spend most of their time in maintenance mode.
  • Product companies in Bangalore and Hyderabad are hiring for “AI-augmented SDET” roles at ₹25-35 LPA. The job description explicitly mentions self-healing pipelines, LLM-based test generation, and Playwright agent integration.
  • Remote US/EU roles for India-based engineers now list “experience with autonomous test agents” as a preferred skill. They pay ₹30-45 LPA in INR equivalent.

The gap is widening. An engineer who can build a self-healing fixture like the one above is not just saving maintenance time. They are signaling that they understand the intersection of AI and QA infrastructure. That is the profile companies pay a premium for in 2026. If you are building your portfolio, do not just show passing tests. Show a CI pipeline where a broken locator triggers an AI agent, generates a fix, and opens a pull request with the updated selector. That is the signal.

If you are just getting started with AI in QA, read my Gen AI guide for QA engineers. It covers the foundational concepts you need before building agents.

Key Takeaways

  • Broken locators cost QA teams 30-40% of their automation budget in false-failure triage.
  • Real self-healing requires detection, diagnosis, and dynamic regeneration — not static fallback lists.
  • Playwright 1.59’s locator.normalize(), page.pickLocator(), and ariaSnapshot() provide the native primitives for AI-powered healing.
  • A production healing agent uses an LLM to read ARIA snapshots, classify failures, generate semantic locators, and validate them before applying.
  • Self-healing should never hide intentional product changes or mask unstable frontends. Set similarity thresholds and always flag behavioral changes for human review.
  • In India, engineers who build self-healing and agentic test infrastructure are commanding ₹25-35 LPA at product companies, while maintenance-heavy roles in services stagnate at ₹12-15 LPA.

Frequently Asked Questions

Does Playwright have built-in self-healing?

No. Playwright provides the building blocks — normalize(), ariaSnapshot(), strict mode, and excellent error messages — but the healing logic is something you build on top. Commercial tools like Testim and Mabl offer built-in healing, but they charge per test and lock you into their cloud runtime.

How much does LLM-based healing cost at scale?

With Claude 3.5 Haiku, each healing call costs approximately ₹0.10-0.15. A 1,000-test suite with a 5% breakage rate spends ₹5-7.50 per CI run. Compare that to two hours of SDET time at ₹4,000 per hour, and the ROI is obvious. If your breakage rate is above 20%, fix your frontend stability before scaling healing.

Can I use GPT-4o instead of Claude?

Yes. GPT-4o-mini and Gemini 1.5 Flash both work well for this task. I prefer Claude Haiku for CI because it is fast (sub-second response) and cheap. For complex DOM restructures, I escalate to Claude 3.5 Sonnet or GPT-4o. The prompt template is model-agnostic.

Is eval() safe for the generated locator?

No, not by default. Always validate the generated string against an allowlist of prefixes before evaluating it. In my production setup, I replace eval with a factory function that maps known patterns to real locator calls. Run the agent in a sandboxed Docker container with no network and no secrets.

What about visual healing instead of DOM-based healing?

Visual healing uses screenshot comparison and vision models to find elements by appearance. It works when the DOM changes completely but the UI looks the same. I use it as a secondary strategy when ARIA-based healing fails. The trade-off is cost: vision models are 10x more expensive than text models. I wrote about visual regression with AI judgment in my AI agents for QA article.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.