QASkills AI Browser Automation Lab Refresh

The QASkills AI browser automation lab now has a tighter assignment path for Stagehand 3.7.1 and Browser Use 0.13.6. The goal is simple: stop treating natural-language browser agents like magic demos, and start testing them the way SDETs test any risky automation layer.

Table of Contents

• What Changed in the QASkills AI Browser Automation Lab?
• Why This Refresh Matters for QA Teams
• Stagehand 3.7.1 and Browser Use 0.13.6: What Learners Compare
• The New Portfolio Task: Agent vs Deterministic Checks
• Evidence Gaps QA Engineers Must Document
• Hands-On Walkthrough for the Lab
• India Career Context: Why This Skill Pays Off
• Assessment Rubric for SDETs and Managers
• Key Takeaways
• FAQ

What Changed in the QASkills AI Browser Automation Lab?

The latest QASkills AI browser automation lab refresh adds a portfolio-first exercise around two agent-style browser automation tools: Stagehand and Browser Use. Learners now compare natural-language browser agents with explicit Playwright or Selenium checks, then write down where the agent gives evidence and where it leaves gaps.

This is not a cosmetic update. The lab moves from “run an agent and watch it click” to “prove whether the agent did the right thing.” That difference matters because AI browser agents can complete happy-path tasks while hiding weak assertions, missing audit trails, or vague pass criteria.

The short version

• Stagehand tasks now focus on observe, act, and extract style browser workflows.
• Browser Use tasks now focus on goal-driven navigation and result verification.
• Every agent task has a matching deterministic test check.
• Learners must submit an evidence matrix, not just screenshots.
• The final portfolio artifact is written like a real QA handoff note.

Why I changed the learning objective

I see many QA engineers get excited by browser agents because the first demo looks strong. “Go to the site, find the product, add it to cart” feels impressive when written in one sentence. But a production SDET does not get paid for a cool demo. A production SDET gets paid for repeatable evidence.

That is why the refreshed assignment asks learners to answer four questions after every run:

1. What did the agent intend to do?
2. What browser actions actually happened?
3. What assertion proves the outcome?
4. What evidence would fail a release gate?

Those four questions turn an AI automation lab into a useful portfolio project. The source track is QASkills.com, and the package references come from the npm, PyPI, GitHub, and PyPI Stats links cited below.

Why This Refresh Matters for QA Teams

The QASkills AI browser automation lab matters because AI browser automation is entering the same stage that Selenium entered years ago: teams are moving from experiments to maintenance. Once a tool enters CI, every missing assertion becomes a future incident.

Agents are flexible, but flexibility cuts both ways

Natural-language browser agents can interpret pages, recover from small UI shifts, and work with vague instructions. That makes them useful for exploration, workflow discovery, and test generation support. It also makes them risky when a team treats the agent’s final answer as proof.

A deterministic Playwright check behaves differently. It does not “understand” the page like an agent, but it gives you explicit selectors, explicit waits, and explicit assertions. When it fails, you can inspect the locator, trace, screenshot, console logs, and network events. That is boring in the best possible way.

The lab does not tell learners to pick one side. It teaches them to combine both:

• Use the agent to explore or execute a user-like task.
• Use Playwright or Selenium to verify the critical state.
• Capture the mismatch between “agent says done” and “test proves done.”
• Turn that mismatch into a QA recommendation.

The data says teams are paying attention

The ecosystem has moved fast. At the time of my research, the npm downloads API reported 4,661,505 last-month downloads for @browserbasehq/stagehand. GitHub’s repository API reported 23,656 stars for browserbase/stagehand. For Browser Use, GitHub’s repository API reported 107,042 stars, while PyPI Stats reported 58,808,322 last-month downloads for browser-use.

These numbers do not prove quality. Downloads can include CI installs, cache effects, bots, and dependency churn. Stars can spike because a tool is trendy. But they do prove attention, and attention creates hiring demand. If QA engineers want to stand out, they need to show judgment, not tool worship.

What this lab is not

The refresh is not a benchmark claiming one framework is faster. It is not a vendor endorsement. It is not a shortcut that replaces Playwright, Selenium, API checks, database validation, accessibility checks, or contract tests.

The lab is a structured way to answer a practical question: “When an AI browser agent completes a task, what evidence should QA trust?”

If you want a broader checklist for agent-style browser testing, read ScrollTest’s AI Browser Agent Testing: 3 Checks Before Trust. The new QASkills lab takes the same principle and turns it into hands-on portfolio work.

Stagehand 3.7.1 and Browser Use 0.13.6: What Learners Compare

The QASkills AI browser automation lab uses Stagehand 3.7.1 and Browser Use 0.13.6 as contrasting examples of agentic browser automation. The point is not to crown a winner. The point is to understand how each tool changes the QA evidence trail.

Stagehand: a structured bridge between Playwright and AI actions

Stagehand sits close to the Playwright mental model. The npm package describes it as an AI web browsing framework, and its GitHub repository calls it an SDK for browser agents. In the lab, learners use it for tasks where the instruction is natural language, but the surrounding test harness still feels familiar to automation engineers.

A typical Stagehand-style learning task looks like this:

import { Stagehand } from "@browserbasehq/stagehand";

const stagehand = new Stagehand({
  env: "LOCAL",
  modelName: "gpt-4.1-mini"
});

await stagehand.init();
const page = stagehand.page;

await page.goto("https://example.test/login");
await page.act("Log in with the test user credentials");
await page.act("Open the billing page and find the latest invoice");

const invoice = await page.extract({
  instruction: "Extract invoice number, amount, and status",
  schema: {
    invoiceNumber: "string",
    amount: "string",
    status: "string"
  }
});

console.log(invoice);
await stagehand.close();

The assignment then asks learners to add a deterministic check after the agent action. For example, if the agent says it found a paid invoice, the test must assert the visible invoice status, URL state, or API response. The agent’s extracted object is useful, but it is not the only proof.

Browser Use: goal-driven browser work with a stronger agent feel

Browser Use has a different feel. The PyPI summary says it makes websites accessible for AI agents. In the lab, it is useful for goal-driven tasks where the agent handles more planning. That makes it great for demonstrating autonomy, but it also increases the need for audit notes.

A simplified Browser Use learning task looks like this:

from browser_use import Agent
from langchain_openai import ChatOpenAI

agent = Agent(
    task="Open the demo shop, search for wireless mouse, and report the first in-stock product under ₹2000.",
    llm=ChatOpenAI(model="gpt-4.1-mini"),
)

result = await agent.run()
print(result)

The matching QA task is not “paste the result into the report.” The matching QA task is:

• Record the agent goal.
• Record the pages visited.
• Check whether the product is actually in stock.
• Validate the price against the rendered UI or API payload.
• Document any ambiguity in currency, filters, sorting, and page state.

Version note for learners

The queue topic refers to Browser Use 0.13.6 because that is the lab baseline. During research, PyPI showed 0.13.7 as the latest version, and GitHub release data showed 0.13.6 was published before 0.13.7. That is normal in a live ecosystem. For a portfolio lab, pin the version you tested, mention the date, and avoid claiming your result applies forever.

The New Portfolio Task: Agent vs Deterministic Checks

The central portfolio task in the QASkills AI browser automation lab is a comparison between natural-language browser agents and deterministic Playwright or Selenium checks. The learner runs the same business scenario twice: once with an agent, once with a conventional test. Then they compare evidence.

The business scenario

The default assignment uses a realistic ecommerce or SaaS workflow:

1. Log in as a test user.
2. Navigate to a product, invoice, report, or settings page.
3. Perform one state-changing action.
4. Confirm the business outcome.
5. Export an evidence note for QA review.

This structure works because it includes navigation, interpretation, action, and verification. A pure search task is too easy. A pure form-fill task does not expose enough judgment. A state-changing task forces the learner to think like a release owner.

The deterministic Playwright comparison

Here is the kind of explicit Playwright check I want learners to add next to an agent run:

import { test, expect } from "@playwright/test";

test("billing invoice stays paid after agent workflow", async ({ page, request }) => {
  await page.goto("https://example.test/login");
  await page.getByLabel("Email").fill("qa.user@example.test");
  await page.getByLabel("Password").fill(process.env.TEST_PASSWORD!);
  await page.getByRole("button", { name: "Sign in" }).click();

  await page.getByRole("link", { name: "Billing" }).click();
  await page.getByRole("row", { name: /INV-2026-07/ }).click();

  await expect(page.getByRole("heading", { name: /Invoice INV-2026-07/ })).toBeVisible();
  await expect(page.getByTestId("invoice-status")).toHaveText("Paid");

  const response = await request.get("/api/invoices/INV-2026-07");
  expect(response.ok()).toBeTruthy();
  expect((await response.json()).status).toBe("PAID");
});

This test is not as flexible as an agent. It does not read ambiguous UI like a person. But it tells the team exactly what passed. It has selectors, assertions, and an API cross-check. If a learner can explain the tradeoff between the two approaches, the portfolio becomes valuable.

The Selenium comparison option

Some learners still work in Selenium-heavy companies. That is fine. The refreshed lab accepts Selenium as the deterministic comparison if the evidence is clear. For teams maintaining enterprise Java frameworks, a Selenium comparison may be more useful than a Playwright sample.

If you are modernizing a Playwright-heavy stack, also read Playwright UI Mode: The Complete Interactive Testing Guide. UI Mode is excellent for reviewing the exact steps behind a deterministic test before comparing it with an agent run.

Evidence Gaps QA Engineers Must Document

Most AI browser automation demos fail at evidence. The QASkills AI browser automation lab makes this explicit by asking learners to write an evidence gap table after every run.

The evidence matrix

Here is the matrix learners now submit:

Evidence Area	Agent Output	Deterministic Check	Risk Note
Goal interpretation	Natural-language summary	Test title and tags	Can the goal be misunderstood?
Navigation	Agent step log	Trace, URL, locator path	Did it visit the right page?
Assertion	Agent says task complete	expect() or assert call	What exactly proves success?
Data validation	Extracted text or model answer	API, DB, or DOM check	Can stale UI fool the test?
Failure artifact	Run log	Screenshot, video, trace	Can another engineer debug it?

The matrix forces clear thinking. If the agent says “the payment succeeded” but there is no DOM assertion, no API check, and no trace, the learner must mark the evidence as weak. That is the kind of judgment hiring managers want to see.

Common evidence gaps

These are the gaps I expect learners to find:

• Ambiguous success criteria: The agent reports success because it reached a page, not because the business state changed.
• No negative check: The task passes even when a warning banner appears.
• Weak data proof: The agent reads visible text but does not validate the backend state.
• Missing run artifact: There is no screenshot, video, trace, or action log for debugging.
• Prompt drift: A small change in wording changes the agent path.
• Environment drift: The agent behaves differently across local, staging, and CI runs.

How to write the gap note

A good evidence note is short and sharp. It does not praise the tool. It explains risk.

Example: The Browser Use run completed the checkout goal and returned the correct order number, but the run did not capture an API confirmation or a trace artifact. I rate this as exploration-grade evidence, not release-gate evidence. I would add a Playwright API assertion for order status and attach the trace to CI.

That paragraph is portfolio material. It shows that the learner understands both AI agents and QA accountability.

Hands-On Walkthrough for the Lab

The hands-on path in the QASkills AI browser automation lab is built for learners who want a screenshot-able GitHub project. The final repo should not be a random notebook. It should look like a small test automation spike that a team could review.

Recommended repo structure

ai-browser-lab/
  README.md
  package.json
  pyproject.toml
  tests/
    playwright/
      invoice.spec.ts
    selenium/
      invoice_test.py
  agents/
    stagehand-invoice.ts
    browser-use-invoice.py
  evidence/
    run-notes.md
    evidence-matrix.md
    screenshots/
    traces/
  docs/
    qa-recommendation.md

The README should explain the scenario, versions, setup steps, and evidence rules. The evidence folder should contain the matrix. The docs folder should contain the final QA recommendation. That recommendation is the part most learners skip, and it is the part that proves maturity.

Step-by-step assignment

1. Pin versions. Record Node, Python, Stagehand, Browser Use, Playwright, Selenium, and browser versions.
2. Choose one workflow. Pick login, checkout, billing, report export, or settings update.
3. Run the agent task. Keep the prompt short and save the output.
4. Run the deterministic test. Add explicit assertions for the same outcome.
5. Capture artifacts. Save screenshots, traces, logs, or videos.
6. Fill the evidence matrix. Mark weak proof honestly.
7. Write the QA recommendation. State whether the agent is fit for exploration, release gating, or neither.

A good agent prompt

Keep prompts narrow. A vague prompt creates a vague test.

Open the billing page for qa.user@example.test.
Find invoice INV-2026-07.
Confirm whether the invoice status is Paid.
Return only: invoice number, visible status, current URL, and any warning text.

This prompt gives the agent a clear task and a constrained output. It still needs verification, but it reduces noise.

A bad agent prompt

Check billing and tell me if everything looks good.

This prompt is too broad. It hides the assertion inside the model’s judgment. A QA engineer cannot defend a release gate with “looks good.”

India Career Context: Why This Skill Pays Off

For QA engineers in India, this lab is useful because the market is splitting. Service companies still need stable Selenium, Java, API, and SQL skills. Product companies increasingly expect SDETs to understand Playwright, CI, cloud grids, observability, and now AI-assisted testing. The salary gap often appears when a candidate can explain tradeoffs, not when they name more tools.

What hiring managers hear

If a candidate says, “I used Browser Use to automate checkout,” that is interesting but incomplete. If the candidate says, “I compared Browser Use with Playwright, found three evidence gaps, added API validation, and marked the agent as exploration-grade,” that sounds like an SDET.

That difference matters for ₹25-40 LPA product-company conversations. At that level, teams do not want someone who only records scripts. They want someone who can protect the release while still adopting new tools.

How to present this in a portfolio

Use a simple project title:

AI Browser Agent Evidence Lab: Stagehand vs Browser Use vs Playwright Checks

Then include four links in the README:

• Agent run summary
• Deterministic Playwright or Selenium test
• Evidence matrix
• QA recommendation

Where QASkills fits

QASkills is meant to give QA engineers hands-on skill tracks they can finish and show. The refreshed lab fits that pattern: learn the tool, run the task, compare evidence, write the recommendation. If you are already building an AI testing portfolio, pair this lab with ScrollTest’s AI QA Portfolio Sprint: 7 Days for SDETs.

Assessment Rubric for SDETs and Managers

If you manage QA engineers, you can use this rubric to review the QASkills AI browser automation lab output. It is intentionally practical. I do not care whether the demo looks fancy. I care whether another engineer can trust the result.

Rubric

Area	Strong Submission	Weak Submission
Version control	Versions pinned and dated	No versions listed
Scenario	Business workflow with clear outcome	Random browsing task
Agent task	Prompt is narrow and reproducible	Prompt is vague
Deterministic check	Assertions prove business state	Only screenshots or print logs
Artifacts	Trace, screenshot, log, or video attached	No debug evidence
Risk analysis	Specific evidence gaps documented	Generic “AI is useful” note
Recommendation	Clear release-gate decision	No decision

Scoring guide

I score the assignment out of 10:

• 2 points for setup and version clarity
• 2 points for agent task design
• 2 points for deterministic assertion quality
• 2 points for artifacts and evidence matrix
• 2 points for QA recommendation

A score of 8 or above is portfolio-ready. A score of 6 means the learner understands the tools but needs stronger proof. Anything below 5 usually means the learner treated the agent output as truth.

CI recommendation

I would not put a raw agent run directly into a release-blocking CI stage on day one. A safer pipeline looks like this:

name: ai-browser-evidence
on: [workflow_dispatch]

jobs:
  agent-exploration:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm run agent:stagehand
      - run: npm run test:playwright -- --grep @critical
      - uses: actions/upload-artifact@v4
        with:
          name: ai-browser-evidence
          path: evidence/

Key Takeaways

The QASkills AI browser automation lab refresh gives QA engineers a practical way to test agentic browser automation without losing engineering discipline.

• Stagehand 3.7.1 and Browser Use 0.13.6 are useful baselines for comparing agent-style browser work.
• Natural-language browser agents are strong for exploration, but their output is not enough for release proof.
• Every agent task should have a matching Playwright or Selenium assertion for the critical business outcome.
• The evidence matrix is the real portfolio artifact because it shows judgment.
• For SDETs in India, this is a strong way to show AI testing skill without sounding like a prompt tourist.

My recommendation is direct: learn the agents, but do not outsource your QA brain to them. Use Stagehand or Browser Use to move faster. Use Playwright, Selenium, API checks, and evidence notes to stay honest.

FAQ

Is the QASkills AI browser automation lab only for advanced SDETs?

No. A manual tester with basic automation knowledge can complete the beginner path, but the portfolio-quality output requires some Playwright or Selenium skill. If you can write a locator, an assertion, and a short README, you can start.

Should I use Stagehand or Browser Use in production CI?

Use them carefully. I would start with exploration, smoke investigation, or test idea generation. For release gates, attach deterministic assertions, artifacts, and clear failure rules before trusting an agent run.

Why compare agents with Playwright or Selenium?

The comparison shows what the agent can infer and what a deterministic test can prove. That gap is the heart of AI testing work. A good SDET can explain both sides.

What should I put in my GitHub portfolio?

Include the agent script, the deterministic test, a pinned version list, screenshots or traces, the evidence matrix, and a one-page QA recommendation. Keep it small but complete.

Does the lab replace normal automation learning?

No. It builds on normal automation. You still need selectors, assertions, API checks, CI basics, and debugging habits. AI browser agents add a new layer, not a replacement for engineering fundamentals.