Day 20: Playwright CLI, Codegen, Trace Viewer, and CI

This is Day 20 of the 21-day JS to Playwright Framework series. One lesson a day. JavaScript first. TypeScript next. Playwright Test after that. Framework layers on Days 17 through 19. Today the command line becomes the product.

Days 1 through 7 were the language. Days 8 and 9 were objects and types. Days 10 through 16 opened the browser, found fields, asserted, looped CSV, and put login in a class. Days 17 through 19 moved into AdvancePlaywrightFramework1x on feat-cucumber: config, pages, fixtures, reporters, Cucumber levels 00 through 02, Restful Booker. Today we stop clicking green in the IDE and start owning the CLI that CI will actually run.

I am Pramod Dutta. I teach SDETs in India for a living. The week I open the Playwright CLI, someone always says they already know it because they ran the test command once from VS Code. That is not CLI mastery. That is a default. A default does not filter by title, does not record a trace zip, does not open Inspector, and does not fail a pull request when someone left test.only in the file.

This is not the already-published scrolltest Day 20. That live post is on codegen and MCP. This series Day 20 is CLI, codegen, Trace Viewer, and CI. Different slug. Different labs. Different gate. If you landed here from the TypeScript Challenge calendar, go back to the hub and start at Day 1 of this series. We earned the Test CLI from console.log.

All CLI labs today come from my public batch repo: LearningPlaywrightBatch on branch main, folder Lecture_Playwright_CLI/. I fetched learning 01 through 10 and exercises 01 through 05 from raw GitHub. I quote those files. I will not invent a learning 11 or an exercise 06.

The CI bits come from the advanced framework on feat-cucumber, not main: workflows playwright.yml and copilot-setup-steps.yml, plus the CI-aware fields in playwright.config.ts. I fetched those three files. I will say what they do. I will also say what they do not do: this pipeline runs the Test runner. It does not run cucumber-js. It does not run typecheck. It does not run lint. If you tell an interviewer the BDD track is gated in GitHub Actions, you are inventing a job that is not in the YAML.

Classroom spellings stay. The lecture config is playwright.config.js, not TypeScript. The recorder login button locator uses a role name with a leading space, because that is how the-internet labels the button. Intentional-fail describes are titled INTENTIONAL FAIL. I do not rename them.

If you want the video plus project path after you finish these 21 posts, the course is here: Playwright Automation Mastery. The series hub for every day lives here: JavaScript to TypeScript to Playwright Advanced Framework 21-Day Guide.

*Want to master this with real projects? Join the Playwright Automation Mastery course at The Testing Academy.*

Contents

What you will be able to do after Day 20

By the end of this post you can name the ten CLI entry points from learning 01 and say which are one-off overrides versus persistent config. You can filter, shard, and list tests without executing them. You can record a flow, then say why the output is a scaffold and not a Page Object. You can tell UI Mode from the HTML report: one is a local server you drive, the other is a static artifact you share.

You can step a failing spec with Inspector and then open the same failure as a zip. You can install browsers with OS deps on Linux CI. You can draw the precedence rule: flags beat the config file, which beats defaults.

The labs we are actually using

Clone LearningPlaywrightBatch, stay on main, open Lecture_Playwright_CLI. The lecture README is honest: expect 7 pass and 3 intentional failures. Those ten specs live in cli_project/tests. I fetched the GitHub tree. Files 01 through 06 and 10 are the green path. Files 07, 08, and 09 are titled INTENTIONAL FAIL: wrong heading on a 404, 100 homepage links, and a 1ms visibility timeout.

Demo scripts 01 through 10 sit in cli_project/scripts. Start with 01_run_basic.sh. Recorder samples that are on disk: codegen_output/login_codegen.spec.js and dropdown_codegen.spec.js plus a .gitkeep. I will not invent a third recorded spec.

The lecture config is Lecture_Playwright_CLI/playwright.config.js. testDir is ./cli_project/tests. retries is 0 because the lecture wants the three failures. workers is 1. trace is on so every run produces a zip. baseURL is the-internet.herokuapp.com. Three projects: chromium, firefox, webkit.

For framework CI, check out AdvancePlaywrightFramework1x on feat-cucumber. GitHub lists two workflows only. There is no cucumber.yml. I will not invent a sharded matrix. Firefox, WebKit, and Pixel 5 are commented out in playwright.config.ts. Exercise 04 asks you to create a CI config file. Exercise 05 asks you to create a simulation script and a Makefile. Those are homework, not committed source.

Clone and stay on the branches I fetched:

git clone https://github.com/PramodDutta/LearningPlaywrightBatch.git
cd LearningPlaywrightBatch && git checkout main && cd Lecture_Playwright_CLI

git clone https://github.com/PramodDutta/AdvancePlaywrightFramework1x.git
cd AdvancePlaywrightFramework1x && git checkout feat-cucumber

Learning 01 — the CLI is the product

The first lecture file is the map, not a trivia list. The Test package CLI is the product. npx must resolve the binary from this project’s node_modules so CI uses the lockfile version. The config file is the team default. Flags are the override for this run only. Module 01 says that precedence out loud. If you skip it, every later flag fight will feel like magic.

npx playwright <command> [options]

npx playwright <command> [subcommand] [options] [arguments]

From the lecture takeaways:

The CLI is the entry point for all Playwright operations: testing, recording, reporting, debugging, and browser management.
Use --help on any command to discover available options.
CLI flags always override playwright.config.js settings for the current run.
The config file is for defaults; the CLI is for overrides.

Learning 02 — running tests and the flags you will type

Module 02 is the flag bible. Path substring and title grep are different axes. Grepping login will not select the form_authentication file if the title says Form Authentication. The demo script 01_run_basic.sh swallows a non-zero exit because three tests are supposed to fail. That swallow is a classroom convenience. Do not copy it into a pull request gate. A gate that cannot go red is not a gate. Multiple reporters belong in the config array. A single reporter flag replaces, it does not append. Sharding is in the lecture textbook. The framework YAML you will read later does not shard.

# Run all tests found by the config's testDir
npx playwright test

# Run a single test file
npx playwright test tests/01_homepage.spec.js

# Run multiple specific files
npx playwright test tests/login.spec.js tests/signup.spec.js

# Run all tests in a directory
npx playwright test tests/smoke/

npx playwright test login
npx playwright test tests/smoke/

Learning 03 — the recorder is not a framework

Module 03 is the honest recorder lesson. Two windows: the site you click, and the Inspector that writes code. The lecture walkthrough is Form Authentication on the-internet. The committed sample in codegen_output is slightly different from the sketch in the markdown: CSS ids, a role locator with a leading space in the button name, and a click-then-fill pair a human should usually delete. Compare it to the hand-written 03_form_authentication.spec.js. Same credentials. Same flash assert. Extra clicks in the recording. Flat sequential code is not LoginPage.login(). Days 16 and 17 already taught that split. Do not undo them because Inspector made a spec in forty seconds.

npx playwright codegen https://the-internet.herokuapp.com

# iPhone 13
npx playwright codegen --device "iPhone 13" https://example.com

# iPad Mini
npx playwright codegen --device "iPad Mini" https://example.com

# Pixel 5
npx playwright codegen --device "Pixel 5" https://example.com

Learning 04 — the HTML report is the shareable artifact

Module 04 is post-run, not during-run. Default report folder is playwright-report. The lecture config does not use that default. It writes HTML to cli_project/reports/html-report with open never. Pass that path to show-report. The report is the index. The zip is the film. In CI, auto-open is already disabled when the CI environment variable is set. Still set open never. The lecture artifact snippet keeps HTML for 14 days. The framework workflow you will quote later keeps it for 30 days. Two files. Two retentions. I will not merge them.

// playwright.config.js
export default {
  reporter: 'html',
};

// playwright.config.js
export default {
  reporter: [['html', { outputFolder: 'my-custom-report' }]],
};

From the lecture takeaways:

The HTML reporter produces a rich, interactive report with filtering, search, screenshots, and traces.
Use npx playwright show-report to open it, optionally passing a custom path or port.
Combine multiple reporters (e.g., list + html + junit) for both terminal output and persistent reports.
In CI, upload the playwright-report/ directory as an artifact.

Learning 05 — UI Mode is a local loop, not CI

Module 05 is the interactive runner. Watch mode on by default. Locator picker on the snapshot. UI Mode versus the HTML report is the interview table: local loop versus shareable artifact. UI Mode versus Inspector is daily workflow versus one stubborn bug. The lecture is explicit: UI Mode needs a display and is not for CI. There is no ui true in config. The flag is CLI-only. Module 09 will repeat that. The framework package.json already aliases test:ui. That is a name, not a new engine.

npx playwright test --ui

# Default launch
npx playwright test --ui

# Specify a port
npx playwright test --ui-port 8080

# Specify a host (useful in containers)
npx playwright test --ui-host 0.0.0.0

# Combine host and port
npx playwright test --ui-port 8080 --ui-host 0.0.0.0

Learning 06 — Inspector is a stepper

Module 06 is the live stepper. Headed browser. Inspector window. Pause at the first action. Resume and step over. page.pause is the in-code breakpoint. The lecture says it only activates under debug. I still do not want it on a green CI path. Leave it on a branch you are debugging. The three intentional fails are the training set: 07 wants heading Page Found on a 404, 08 wants 100 links, 09 wants finish visible in 1 millisecond. Script 08_run_debug.sh exists. I listed it. I did not invent a ninth debug script. Pair workers 1 with Inspector so you do not get three windows. The lecture config already uses one worker. The framework uses four workers when CI is set. Override locally when you step.

npx playwright test --debug

# Opens Inspector for all tests
PWDEBUG=1 npx playwright test

# Windows (PowerShell)
$env:PWDEBUG=1; npx playwright test

# Windows (cmd)
set PWDEBUG=1 && npx playwright test

Learning 07 — the trace is the film of the run

Module 07 is the post-mortem. A trace is a zip: screenshots, DOM snapshots, network, console, source. You do not unzip it by hand. You hand it to the viewer. The lecture config uses trace on because this folder is a classroom. The framework config uses on-first-retry because CI storage is not a souvenir shop. The remote viewer at trace.playwright.dev is a client-side app. The lecture says the data stays in the browser. That sentence is the one you give a security reviewer. None of the ten cli_project tests use programmatic tracing.start. They rely on config. I will not pretend there is a manual-trace spec. Script 09_show_trace.sh runs the homepage spec with trace on, finds a zip, and opens it.

// playwright.config.js
export default {
  use: {
    trace: 'on-first-retry',
  },
};

# Force traces on for this run
npx playwright test --trace on

# Only on first retry
npx playwright test --trace on-first-retry

# Disable traces
npx playwright test --trace off

Learning 08 — browsers are a cache, CI is a clean disk

Module 08 is browser binaries. The with-deps flag is the Linux CI flag. It installs OS libraries and needs sudo on Linux. macOS and Windows generally do not need it. Channels such as chrome and msedge use the branded browser already on the machine. install does not download Chrome stable. The lecture config uses Desktop Chrome on bundled Chromium, not a channel. The framework chromium project is the same pattern. The lecture Docker tags pin 1.49. The batch repo at inventory time used 1.58 and 1.59. The framework on feat-cucumber uses 1.60. Pin via package-lock. The framework Dockerfile on this branch is 0 bytes. I will not teach containers from an empty file. The YAML you will quote uses ubuntu-latest plus install with deps.

// playwright.config.js
export default {
  projects: [
    {
      name: 'Google Chrome',
      use: { channel: 'chrome' },
    },
    {
      name: 'Microsoft Edge',
      use: { channel: 'msedge' },
    },
  ],
};

npx playwright test --project "Google Chrome"

Learning 09 — flags beat config, config beats defaults

Module 09 is one rule: flags beat config, config beats defaults. Mapping table: workers, retries, timeout, reporter, grep, shard, forbidOnly, trace, headed, project. Config-only: baseURL, storageState, viewport, video, screenshot, webServer, projects, globalSetup. CLI-only: ui, debug, list, last-failed, config path, plus the commands show-report, show-trace, codegen, install. Option B in the file is the pattern the framework ships: one config, process.env.CI and a resolved base URL. Option A, a separate CI config file, is what exercise 04 asks you to practise. I have not found playwright.ci.config.js committed in the lecture tree. Do not cite one.

CLI flags  >  Config file  >  Playwright defaults

# Default: uses playwright.config.js or playwright.config.ts
npx playwright test

# Use a custom config file
npx playwright test --config staging.config.js
npx playwright test --config configs/ci.config.ts

Learning 10 — CI textbooks versus the YAML on disk

Module 10 is the lecture CI textbook: a complete GitHub Actions example, a sharded blob merge, a browser matrix, Docker, GitLab, Jenkins, a checklist. I teach that file as the textbook. Then I open the files that exist on feat-cucumber and say where they diverge. That divergence is the Day 20 homework, not a failure of the lecture. When CI is true, HTML auto-open is off and missing browsers fail hard. The framework reads the same variable as isCI.

# .github/workflows/playwright.yml
name: Playwright Tests

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 30

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Install Playwright browsers
        run: npx playwright install --with-deps

      - name: Run Playwright tests
        run: npx playwright test

      - name: Upload HTML report
        uses: actions/upload-artifact@v4
        if: ${{ !cancelled() }}
        with:
          name: playwright-report
          path: playwright-report/
          retention-days: 14

      - name: Upload test results (traces)
        uses: actions/upload-artifact@v4
        if: ${{ !cancelled() }}
        with:
          name: test-results
          path: test-results/
          retention-days: 7

# .github/workflows/playwright-sharded.yml
name: Playwright Tests (Sharded)

on:
  push:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 20
    strategy:
      fail-fast: false
      matrix:
        shard: [1/4, 2/4, 3/4, 4/4]

    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: 'npm'

      - run: npm ci
      - run: npx playwright install --with-deps

      - name: Run tests (shard ${{ matrix.shard }})
        run: npx playwright test --shard ${{ matrix.shard }}

      - name: Upload blob report
        uses: actions/upload-artifact@v4
        if: ${{ !cancelled() }}
        with:
          name: blob-report-${{ strategy.job-index }}
          path: blob-report/
          retention-days: 1

  merge-reports:
    if: ${{ !cancelled() }}
    needs: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: 'npm'
      - run: npm ci

      - name: Download blob reports
        uses: actions/download-artifact@v4
        with:
          path: all-blob-reports
          pattern: blob-report-*
          merge-multiple: true

      - name: Merge reports
        run: npx playwright merge-reports --reporter html ./all-blob-reports

      - name: Upload merged HTML report
        uses: actions/upload-artifact@v4
        with:
          name: playwright-report
          path: playwright-report/
          retention-days: 14

Files I actually fetched and will now quote

The lecture README says expect 7 pass, 3 intentional failures. I quote the homepage spec, the hand-written login, the two committed recorder samples, the three red specs, three demo scripts, the lecture config, both workflows, and playwright.config.ts. If a path is not here, I did not treat it as a Day 20 teaching file.

Lecture playwright.config.js

// @ts-check
const { defineConfig, devices } = require('@playwright/test');
const path = require('path');

module.exports = defineConfig({
  testDir: './cli_project/tests',
  timeout: 30000,
  retries: 0, // No retries - we want to capture intentional failures
  workers: 1, // Run sequentially for clear output

  use: {
    baseURL: 'https://the-internet.herokuapp.com',
    headless: true,
    screenshot: 'only-on-failure',
    trace: 'on', // Always record traces for trace viewer demos
  },

  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },
    {
      name: 'firefox',
      use: { ...devices['Desktop Firefox'] },
    },
    {
      name: 'webkit',
      use: { ...devices['Desktop Safari'] },
    },
  ],

  reporter: [
    ['html', { outputFolder: path.resolve(__dirname, 'cli_project/reports/html-report'), open: 'never' }],
    ['json', { outputFile: path.resolve(__dirname, 'cli_project/reports/results.json') }],
    ['./cli_project/reporters/CustomTTAReporter.js'],
  ],
});

cli_project tests — homepage

const { test, expect } = require('@playwright/test');

test.describe('TC-001: Homepage Title Verification', () => {
  test('should have correct title and heading on homepage', async ({ page }) => {
    await page.goto('https://the-internet.herokuapp.com');
    await expect(page).toHaveTitle('The Internet');
    await expect(page.locator('h1')).toBeVisible();
    console.log('✅ TC-001: Homepage Title Verification - Passed');
  });
});

cli_project tests — login

const { test, expect } = require('@playwright/test');

test.describe('TC-003: Form Authentication', () => {
  test('should login successfully with valid credentials', async ({ page }) => {
    await page.goto('https://the-internet.herokuapp.com/login');
    await page.locator('#username').fill('tomsmith');
    await page.locator('#password').fill('SuperSecretPassword!');
    await page.getByRole('button', { name: ' Login' }).click();
    await expect(page.locator('#flash')).toContainText('You logged into a secure area!');
    console.log('✅ TC-003: Form Authentication - Passed');
  });
});

committed recorder samples

// This test was auto-generated by Playwright Codegen
// Command: npx playwright codegen https://the-internet.herokuapp.com/login
const { test, expect } = require('@playwright/test');

test('Login flow - recorded via codegen', async ({ page }) => {
  await page.goto('https://the-internet.herokuapp.com/login');
  await page.locator('#username').click();
  await page.locator('#username').fill('tomsmith');
  await page.locator('#password').click();
  await page.locator('#password').fill('SuperSecretPassword!');
  await page.getByRole('button', { name: ' Login' }).click();
  await expect(page.locator('#flash')).toContainText('You logged into a secure area!');
});

// This test was auto-generated by Playwright Codegen
// Command: npx playwright codegen https://the-internet.herokuapp.com/dropdown
const { test, expect } = require('@playwright/test');

test('Dropdown selection - recorded via codegen', async ({ page }) => {
  await page.goto('https://the-internet.herokuapp.com/dropdown');
  await page.locator('#dropdown').selectOption('2');
  await expect(page.locator('#dropdown')).toHaveValue('2');
});

intentional failures

const { test, expect } = require('@playwright/test');

/* INTENTIONAL FAILURE: Navigates to 404 page and asserts wrong heading - use trace viewer to debug */

test.describe('TC-007: INTENTIONAL FAIL - Nonexistent Page', () => {
  test('should find heading "Page Found" on 404 page', async ({ page }) => {
    await page.goto('https://the-internet.herokuapp.com/this-page-does-not-exist');
    await expect(page.locator('h1')).toHaveText('Page Found');
    console.log('❌ TC-007: Nonexistent Page - This should not print (test should fail)');
  });
});

const { test, expect } = require('@playwright/test');

/* INTENTIONAL FAILURE: Asserts wrong element count - use debug mode to count actual elements */

test.describe('TC-008: INTENTIONAL FAIL - Wrong Element Count', () => {
  test('should have exactly 100 links on homepage', async ({ page }) => {
    await page.goto('https://the-internet.herokuapp.com');
    const links = page.locator('#content ul li a');
    await expect(links).toHaveCount(100);
    console.log('❌ TC-008: Wrong Element Count - This should not print (test should fail)');
  });
});

const { test, expect } = require('@playwright/test');

/* INTENTIONAL FAILURE: Uses impossibly short timeout - use trace viewer to see timing */

test.describe('TC-009: INTENTIONAL FAIL - Impossible Timeout', () => {
  test('should load dynamic content with 1ms timeout', async ({ page }) => {
    await page.goto('https://the-internet.herokuapp.com/dynamic_loading/1');
    await page.getByRole('button', { name: 'Start' }).click();
    await expect(page.locator('#finish')).toBeVisible({ timeout: 1 });
    console.log('❌ TC-009: Impossible Timeout - This should not print (test should fail)');
  });
});

demo scripts

#!/bin/bash
# Demo: Basic test execution with npx playwright test
echo "=== Demo: Running All Tests ==="
echo "Command: npx playwright test --config=../../playwright.config.js"
echo ""
cd "$(dirname "$0")/../.."
npx playwright test --project chromium 2>&1 || true
echo ""
echo "=== Demo Complete ==="

#!/bin/bash
# Demo: Generating tests with npx playwright codegen
# NOTE: Requires a display. This opens a browser window and the Playwright Inspector.
# As you interact with the browser, Playwright records your actions as test code.
echo "=== Demo: Playwright Codegen ==="
echo "Command: npx playwright codegen https://the-internet.herokuapp.com"
echo ""
echo "NOTE: This opens a browser window and the Playwright Inspector."
echo "      Interact with the browser and Playwright will generate test code."
echo "      Close the browser window when done."
echo ""
echo "Pre-recorded codegen examples are available in codegen_output/"
echo ""
cd "$(dirname "$0")/../.."
npx playwright codegen https://the-internet.herokuapp.com 2>&1 || true
echo ""
echo "=== Demo Complete ==="

#!/bin/bash
# Demo: Recording and viewing trace files
# Runs a test with tracing enabled, then shows how to view the trace file.
echo "=== Demo: Trace Viewer ==="
echo ""
cd "$(dirname "$0")/../.."

echo "--- Step 1: Running a test with trace enabled ---"
echo "Command: npx playwright test --project chromium --trace on cli_project/tests/01_homepage_title.spec.js"
npx playwright test --project chromium --trace on cli_project/tests/01_homepage_title.spec.js 2>&1 || true
echo ""

echo "--- Step 2: Finding trace files ---"
echo "Trace files are stored in the test-results/ directory as .zip files."
TRACE_FILE=$(find test-results -name "trace.zip" 2>/dev/null | head -1)
if [ -n "$TRACE_FILE" ]; then
  echo "Found trace: $TRACE_FILE"
  echo ""
  echo "--- Step 3: Opening trace viewer ---"
  echo "Command: npx playwright show-trace $TRACE_FILE"
  npx playwright show-trace "$TRACE_FILE" 2>&1 || true
else
  echo "No trace file found. Trace files are generated in test-results/ when tests run with --trace on."
fi
echo ""
echo "=== Demo Complete ==="

The real YAML is smaller than the textbook

Read playwright.yml slowly. Triggers: push and pull_request to main or master. If you only live on feat-cucumber and never open a PR into those branches, this workflow will not run for you. One job, not a matrix. No shard. No browser matrix. No container image. Node is lts star, not pinned 20. Browsers install with deps. The test step is the Test runner. Not the BDD runner. Not typecheck. Not lint.

Artifact: playwright-report for 30 days if the job is not cancelled. The lecture example also uploaded test-results for 7 days. This YAML does not. I will not claim the zip folder is uploaded. It is not in the path.

CI may not run Cucumber. package.json has test:bdd and cucumber level scripts. Day 18 taught those. This workflow does not call them. A green check means src/tests ran. It does not mean the level 00 smoke feature ran. The quality-gate rule on the same branch is an IDE rule. It is not a CI step yet.

The Copilot workflow is not the test gate. Its build step does not match package.json.

On CI, forbidOnly is true, retries are 2, workers are 4. Video is always on. Firefox and WebKit stay commented out.

feat-cucumber test workflow

name: Playwright Tests
on:
  push:
    branches: [ main, master ]
  pull_request:
    branches: [ main, master ]
jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-node@v4
      with:
        node-version: lts/*
    - name: Install dependencies
      run: npm ci
    - name: Install Playwright Browsers
      run: npx playwright install --with-deps
    - name: Run Playwright tests
      run: npx playwright test
    - uses: actions/upload-artifact@v4
      if: ${{ !cancelled() }}
      with:
        name: playwright-report
        path: playwright-report/
        retention-days: 30

Copilot setup workflow

name: "Copilot Setup Steps"

on:
  workflow_dispatch:
  push:
    paths:
      - .github/workflows/copilot-setup-steps.yml
  pull_request:
    paths:
      - .github/workflows/copilot-setup-steps.yml

jobs:
  copilot-setup-steps:
    runs-on: ubuntu-latest

    permissions:
      contents: read

    steps:
    - uses: actions/checkout@v4

    - uses: actions/setup-node@v4
      with:
        node-version: lts/*

    - name: Install dependencies
      run: npm ci

    - name: Install Playwright Browsers
      run: npx playwright install --with-deps

    # Customize this step as needed
    - name: Build application
      run: npx run build

feat-cucumber playwright.config.ts

import { defineConfig, devices } from '@playwright/test';
import * as dotenv from 'dotenv';

dotenv.config();

function resolveBaseURL(): string {
  if (process.env.BASE_URL) return process.env.BASE_URL;
  const env = (process.env.TTA_ENV || 'qa').toLowerCase();
  switch (env) {
    case 'api':
      return process.env.API_BASE_URL || 'https://restful-booker.herokuapp.com';
    case 'dev':
    case 'local':
      return process.env.DEV_BASE_URL || 'http://localhost:3000';
    case 'stg':
    case 'stage':
    case 'staging':
      return process.env.STG_BASE_URL || 'https://stage.thetestingacademy.com';
    case 'prod':
    case 'production':
      return process.env.PROD_BASE_URL || 'https://app.thetestingacademy.com';
    case 'qa':
    default:
      return process.env.QA_BASE_URL || 'https://app.thetestingacademy.com';
  }
}

const isCI = !!process.env.CI;

export default defineConfig({
  testDir: './src/tests',
  timeout: 60_000,
  expect: { timeout: 10_000 },
  fullyParallel: true,
  forbidOnly: isCI,
  retries: isCI ? 2 : 0,
  workers: isCI ? 4 : undefined,
  reporter: [
    ['./src/utils/CustomReporter.ts'],
    ['html', { outputFolder: 'playwright-report' }],
    ['json', { outputFile: 'test-results/results.json' }],
    ['allure-playwright', {
      resultsDir: 'allure-results',
      reportName: 'TTACart Automation Report',
      environmentInfo: {
        Environment: process.env.TTA_ENV || 'qa',
        BaseURL: resolveBaseURL(),
        Node: process.version,
        OS: process.platform,
        CI: String(isCI),
      },
      categories: [
        { name: 'Assertion failures', matchedStatuses: ['failed'] },
        { name: 'Broken tests / errors', matchedStatuses: ['broken'] },
        {
          name: 'Timeouts',
          matchedStatuses: ['broken', 'failed'],
          messageRegex: '.*Timeout.*',
        },
      ],
    }],
    ['list'],
  ],
  use: {
    baseURL: resolveBaseURL(),
    screenshot: 'only-on-failure',
    video: 'on',
    trace: 'on-first-retry',
    actionTimeout: 15_000,
    navigationTimeout: 30_000,
    extraHTTPHeaders: {
      Accept: 'application/json',
      'Content-Type': 'application/json',
    },
  },
  projects: [
    {
      name: 'api',
      testMatch: /src\/tests\/apiTests\/.*\.spec\.ts/,
    },
    {
      name: 'chromium',
      testIgnore: /src\/tests\/apiTests\/.*\.spec\.ts/,
      use: { ...devices['Desktop Chrome'] },
    },
    // { name: 'firefox', use: { ...devices['Desktop Firefox'] } },
    // { name: 'webkit', use: { ...devices['Desktop Safari'] } },
    // { name: 'mobile-chrome', use: { ...devices['Pixel 5'] } },
  ],
});

Exercises 01 through 05

Five exercise files. I fetched all five. I will not replace them with a quiz I wrote. Create the files they ask for. Do not pretend those homework files are already in the lecture tree.

Exercise 01 — basic CLI

Use the ten cli_project tests as the 5 to 10 file suite the exercise asks for. Fill the comparison table with what you observe. Grepping the word login against this suite may match fewer titles than you expect. 03 is described as Form Authentication. Title grep is not path grep. That is the exercise biting. Precedence is module 09. Flags win.

Objective (from the exercise file): Get hands-on experience with the core Playwright CLI commands for running, filtering, and configuring test execution from the terminal.

Exercise 02 — recording tests

Record checkboxes and add-remove on the-internet. Save the two deliverable spec names the exercise gives you. They are not in cli_project/tests today. The committed cousins are the login and dropdown samples under codegen_output. Do not submit those two as the checkbox exercise. List three improvements as comments. After Day 16, one of those three should be: extract a page object.

Objective (from the exercise file): Learn to use npx playwright codegen to record browser interactions, generate test code, and critically evaluate the output for production readiness.

Exercise 03 — inspector and trace

Part A is the live stepper on a passing spec, a pause call, Pick Locator on five elements, then a broken spec. Part B is trace on, find zips, open locally, then drag a zip onto the public viewer. For this lecture, 07 08 and 09 are the broken specs. You do not need to invent a fourth unless you want one.

Objective (from the exercise file): Master Playwright’s built-in debugging tools: the Inspector for stepping through tests, and the Trace Viewer for post-mortem analysis of test runs.

Exercise 04 — config versus flags

Create the CI config the exercise pastes. Run it. Then override timeout, workers, and retries from the command line and watch flags win. Five config-only options. Five CLI-only options. I will not quote a committed CI config from GitHub because the lecture tree I fetched does not contain one at the lecture root.

Objective (from the exercise file): Understand the relationship between playwright.config.js settings and CLI flags, learn when each takes precedence, and create a CI-specific configuration.

Exercise 05 — toolkit mini project

Write the script aliases the exercise lists. Then write the simulation script and Makefile it pastes. After that, read the framework package.json. It already has headed, ui, debug, per-browser, tagged e2e, report, and the BDD names. Smoke on the Test runner is not there. BDD smoke is a different runner. The simulation script is homework, not one of the ten demo scripts.

Objective (from the exercise file): Create a complete developer toolkit around Playwright CLI: npm scripts for every common task, a CI simulation shell script, and a Makefile for convenience commands.

Common mistakes I see in this week of the batch

Calling the VS Code play button CLI mastery. If you cannot filter, record a zip, and pick one project without opening the docs, you are not done.

Treating recorder output as the framework. The committed login sample is a recording. src/pages/LoginPage.ts on feat-cucumber is a page object.

Leaving a swallowed exit after the test command in CI because you copied the basic demo script. That script expects failures. A pull request gate must be allowed to go red.

Enabling UI Mode in GitHub Actions. Module 05 says it needs a display. The HTML report is the artifact. The workflow already uploads it.

Setting lecture-style always-on traces on a large CI suite. Lecture is ten tests. Framework CI records video always and traces on first retry.

Telling the team Cucumber is gated because the workflow is green. The job runs the Test runner. Until someone adds a step, CI may not run Cucumber.

Assuming firefox and webkit run in CI because package.json has those script names. The projects are commented out.

Uploading the wrong folder. Lecture HTML lives under cli_project/reports/html-report. Framework HTML lives under playwright-report.

Using the lecture 1.49 Docker tag on a 1.60 lockfile. Pin the image to your version, or do what this YAML does: ubuntu-latest plus install with deps.

Forgetting forbidOnly on a laptop. On feat-cucumber it is isCI. GitHub sets CI. Your laptop does not, unless you export it.

What to run today, in order

In the lecture folder, list tests on chromium and count ten files. Run them. Confirm seven pass, three fail. Do not fix 07 through 09. Open the HTML report at cli_project/reports/html-report. Click a failed test. Find the trace link. Open a zip. Click the action that asserted Page Found. Step 08 in Inspector. Pick Locator on the homepage links. Write the real count in your notes. Record the login URL. Compare your recording to the committed login sample. Delete the extra click before fill.

On feat-cucumber, open playwright.yml. Highlight the test step. Write next to it: not cucumber. Open playwright.config.ts. Highlight forbidOnly, retries, workers, trace. Open package.json. Circle test:bdd and cucumber:level0. Confirm they are not in the YAML. Draw the three-card diagram without looking.

Optional, on your own branch: add a second job that runs BDD smoke. Do not ask me to pretend that job is already on feat-cucumber. It is not.

FAQ

Is this the same as the published scrolltest Day 20 on codegen and MCP?

No. The already-published scrolltest Day 20 is a different series post about codegen and MCP. This is Day 20 of the JS to Playwright Framework series: CLI, codegen, Trace Viewer, and CI. Slug: js-playwright-framework-day-20-cli-codegen-trace-ci. MCP and AI agents are Day 21 of this series.

Where do the CLI labs live?

In LearningPlaywrightBatch on main, under Lecture_Playwright_CLI. Learning modules 01 through 10. Exercises 01 through 05. Runnable specs in cli_project/tests. Config: playwright.config.js at the lecture root.

Why do lecture test runs fail?

The README says expect 7 pass and 3 intentional failures. The failing files are 07_nonexistent_page, 08_wrong_element_count, and 09_impossible_timeout. They exist so you have a red row for Inspector and Trace Viewer. retries is 0 in that config on purpose.

Does the recorder create Page Objects?

No. Module 03 says it generates flat sequential code with no POM and no route mocks. Committed samples: codegen_output/login_codegen.spec.js and dropdown_codegen.spec.js. Refactor after recording. Days 16 and 17 are the POM days.

What is the difference between UI Mode and the HTML report?

UI Mode is a local interactive runner. The HTML report is a static post-run artifact. Module 05: UI Mode is not for CI. The report is what you upload. feat-cucumber uploads playwright-report for 30 days.

When should I use Inspector versus Trace Viewer?

Inspector and page.pause are live steppers while the browser is open. Trace Viewer is post-mortem on a zip after the process has exited. Live for a bug you can reproduce on the laptop. Trace for a bug that happened on the runner.

Which trace setting should CI use?

The lecture recommends on-first-retry for CI. feat-cucumber sets that, and retries 2 when CI is set. The lecture config uses trace on because it is a demo folder with ten tests. Do not copy always-on into a large suite.

Does the framework GitHub Action run Cucumber?

No. playwright.yml on feat-cucumber runs the Test runner only. It does not call the BDD runner. CI may not run Cucumber until someone adds that step. BDD scripts exist in package.json. They are not in the workflow.

What does playwright.config.ts change when CI is set?

forbidOnly becomes true, retries become 2, workers become 4. trace is on-first-retry regardless. video is on regardless. Firefox, WebKit, and mobile stay commented out. Default run runs api and chromium.

Is copilot-setup-steps.yml the test pipeline?

No. It bootstraps a Copilot environment when that YAML changes. The test gate is playwright.yml.

Do I need Docker for this CI?

Not for this repo’s current workflow. The framework Dockerfile on feat-cucumber is empty (0 bytes). playwright.yml uses ubuntu-latest and install with deps.

What is Day 21 of this series?

MCP, AI agents, and the framework capstone. Planner, generator, healer, the src/ai factory, and one pass through UI plus BDD plus API plus optional LLM. Not another flags day.

Tomorrow — Day 21: MCP, AI agents, and the framework capstone

Today the diagram stopped at a GitHub Actions X. Tomorrow we put an agent on the same command line.

Day 21 is the capstone: Playwright MCP, the plan / generate / heal agents, the runtime src/ai factory, and one honest pass through the whole stack. I will quote files from Lecture_Playwright_MCP, Lecture_Playwright_AI_Agents, and AdvancePlaywrightFramework1x on feat-cucumber. I will not invent a server that is not in .mcp.json. I will not pretend CI already runs those agents.

The CLI you learned today is the cheap interface those agents use when MCP is too heavy. If you cannot draw local run then trace then CI gate, you will treat MCP as magic.

If you only remember one sentence from Day 20: a local test run is a demo. A trace zip is the film. playwright.yml is the gate — and that gate does not run Cucumber yet.

Series hub (bookmark this): JavaScript to TypeScript to Playwright Advanced Framework 21-Day Guide.

Master Playwright end to end

If you want these labs as a live classroom — every flag, a recording you then throw into a Page Object, Trace Viewer on a red CI artifact, and the MCP plus AI capstone on Day 21 — join Playwright Automation Mastery at The Testing Academy. Lifetime access. Real projects. A pipeline you can point at, including the jobs that are not wired yet.

*This is Day 20 of 21. Draft only. Not published.*