| |

Playwright Reports: Day 16 HTML, JUnit and CI Guide

ByPromode
Playwright reports Day 16 tutorial featured image showing HTML, JUnit, JSON and CI evidence

Table of Contents

Playwright reports turn a test run into a debugging story. Day 16 of this Playwright + TypeScript series shows how I set up HTML, list, JSON, and JUnit reports so a failed CI run gives the team evidence instead of a vague red cross.

If you completed Day 15 on Playwright sharding, you already know how fast a suite can become when it runs across multiple CI machines. Reporting is the next problem. Parallel runs are useful only when the result is readable, searchable, and easy to share with developers.

Contents

What Are Playwright Reports?

Playwright reports are the output generated by Playwright Test after it executes your spec files. They can be human-friendly, machine-friendly, or both. A tester reads an HTML report. A CI server reads a JUnit XML file. A dashboard or internal tool may read a JSON report.

The official Playwright reporters documentation says Playwright Test ships with built-in reporters and supports custom reporters. That matters because a real SDET team rarely has one reporting need. Local debugging, pull request checks, release gates, and management summaries all ask for different formats.

Why reporting becomes a framework decision

I see many teams treat reports as decoration. They run tests, generate a folder, and never open it unless something breaks. That habit creates slow triage. The report should answer these questions within 30 seconds:

  • Which test failed?
  • Which browser, project, worker, and shard executed it?
  • Was the failure a real product bug, test data issue, or automation bug?
  • Is there a trace, screenshot, or video?
  • Can a developer reproduce the issue without asking the QA engineer for five extra details?

Reporting is not separate from framework design. It is the visible face of the framework. If your report is messy, people assume your tests are messy too.

The data point I care about

Playwright is no longer a small tool used by a few early adopters. The GitHub API for microsoft/playwright reported 91,514 stars during this run, and the npm downloads API reported 162,742,966 downloads of @playwright/test for the last month ending 2026-06-22. Those numbers do not prove quality by themselves, but they do prove that Playwright reporting choices affect a large ecosystem of teams.

For Indian QA teams working across TCS, Infosys, service companies, and product companies, readable CI evidence is a career skill. A senior SDET is expected to move beyond “test failed” and show an evidence trail that a developer respects.

The Playwright Reporters You Should Know

Playwright has several reporter options. You do not need all of them in every project. You need the right combination for local runs and CI runs.

List reporter

The list reporter prints every test as it runs. It is useful when I want a clear terminal log during local debugging or while watching a CI job live.

npx playwright test --reporter=list

Use it when the team wants fast visibility in terminal output. Avoid it when your suite has thousands of tests and the CI log becomes too noisy.

Line reporter

The line reporter keeps output compact. It updates a single line during execution and works well for local runs where I only care about progress and final failures.

npx playwright test --reporter=line

This is a good default for developers who run a focused spec while coding. It is not the best final artifact for release pipelines.

HTML reporter

The HTML reporter is the format most QA engineers open after a failed run. It gives a searchable report with test status, duration, attachments, stdout, stderr, screenshots, traces, and videos when configured.

npx playwright test --reporter=html
npx playwright show-report

For a tutorial series, this is the most important reporter. It connects directly to what we covered on Day 7 with Trace Viewer.

JUnit reporter

JUnit XML is not exciting, but it is practical. CI systems understand it. Test management tools understand it. Many engineering dashboards understand it.

npx playwright test --reporter=junit

If your company expects trend charts, historical pass rates, or release gate integration, you will likely need JUnit XML.

JSON reporter

JSON is helpful when you want to build your own summary, send Slack messages, update a dashboard, or run a small script that groups failures by feature area.

npx playwright test --reporter=json

I do not use JSON as the only report. I use it as a source for automation around the test result.

Configure Playwright Reports in TypeScript

Command-line reporters are fine for experiments. A real framework should keep reporting rules in playwright.config.ts. That makes local and CI behavior predictable.

Start with a simple config

Create or update playwright.config.ts like this:

import { defineConfig, devices } from '@playwright/test';

const isCI = !!process.env.CI;

export default defineConfig({
  testDir: './tests',
  timeout: 30_000,
  expect: {
    timeout: 5_000,
  },
  fullyParallel: true,
  retries: isCI ? 2 : 0,
  workers: isCI ? 4 : undefined,

  reporter: isCI
    ? [
        ['list'],
        ['html', { outputFolder: 'playwright-report', open: 'never' }],
        ['junit', { outputFile: 'test-results/junit.xml' }],
        ['json', { outputFile: 'test-results/results.json' }],
      ]
    : [
        ['line'],
        ['html', { outputFolder: 'playwright-report', open: 'on-failure' }],
      ],

  use: {
    baseURL: process.env.BASE_URL || 'https://example.com',
    trace: 'retain-on-failure',
    screenshot: 'only-on-failure',
    video: isCI ? 'retain-on-failure' : 'off',
  },

  projects: [
    { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
    { name: 'firefox', use: { ...devices['Desktop Firefox'] } },
  ],
});

This config gives developers a clean local experience and gives CI enough artifacts for triage. It also avoids opening a browser report automatically inside CI, which is a small but common mistake.

Understand reporter arrays

The reporter property can accept one reporter or an array of reporters. I prefer arrays because one run can produce multiple outputs:

  1. Terminal output for the current job log.
  2. HTML report for human investigation.
  3. JUnit XML for CI test result panels.
  4. JSON for custom automation.

This is the reporting equivalent of writing both human-readable and machine-readable logs. Do not force one format to do every job.

Use environment-based behavior

Local runs and CI runs have different priorities. Local runs need speed and clarity. CI runs need auditability. That is why the config checks process.env.CI.

In a product company, this small split saves time. Developers are not annoyed by heavy artifacts on every local run, and release pipelines still keep the evidence that managers and developers expect.

Build a Practical HTML Report Workflow

The HTML report is the report I expect every Playwright beginner to understand. It is also the report I expect every SDET to attach to a failed pipeline.

Run and open the report locally

Use this flow during development:

npx playwright test tests/login.spec.ts --project=chromium
npx playwright show-report

Screenshot description: the HTML report opens with a left-side list of tests, status badges for passed and failed tests, duration values, and a detail panel that shows the error message plus attachments. A failed test should show a screenshot attachment and a trace link if your config uses retain-on-failure.

Read the report like a debugger

Do not open the report and randomly click. Follow a repeatable order:

  1. Check the failed test title and file path.
  2. Check the browser project and retry count.
  3. Open the error message and stack trace.
  4. Open the screenshot to confirm the UI state.
  5. Open the trace when the screenshot does not explain the failure.
  6. Check stdout or console logs only after the visual evidence.

This order keeps you honest. A stack trace may say a locator timed out, but the screenshot may show a feature flag banner, broken login state, or wrong environment.

Control where reports are stored

Use explicit folders. Random defaults become painful when CI uploads artifacts.

reporter: [
  ['html', { outputFolder: 'artifacts/html-report', open: 'never' }],
  ['junit', { outputFile: 'artifacts/junit/results.xml' }],
  ['json', { outputFile: 'artifacts/json/results.json' }],
],

Then add those folders to .gitignore:

artifacts/
playwright-report/
test-results/

Reports are generated outputs. Commit the config, not the report folders.

Use JUnit and JSON Reports in CI

Playwright reports become more valuable when the CI pipeline stores them correctly. This is where many teams lose evidence. The test ran, the report was created, but the artifact was never uploaded.

GitHub Actions example

If you followed Day 12 on Playwright CI with GitHub Actions, extend the workflow with report uploads:

name: Playwright Reports Day 16

on:
  pull_request:
  push:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 30

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Setup Node
        uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: npm

      - name: Install dependencies
        run: npm ci

      - name: Install Playwright browsers
        run: npx playwright install --with-deps

      - name: Run Playwright tests
        run: npx playwright test
        env:
          CI: true
          BASE_URL: ${{ secrets.STAGING_URL }}

      - name: Upload HTML report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: playwright-html-report
          path: playwright-report/
          retention-days: 14

      - name: Upload test results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: playwright-test-results
          path: test-results/
          retention-days: 14

The important line is if: always(). Without it, the upload step may not run after a failing test step. A report that disappears after failure is worse than no report because it gives the team false confidence.

Publish JUnit results

Some teams add a separate action to publish JUnit results in the pull request UI. The exact action depends on your organization policy, but the pattern is the same: generate test-results/junit.xml, upload it, and surface it where developers already review code.

Do not make developers download a zip for every small failure. Use the zip for detailed investigation, but put the first signal in the PR check when possible.

Parse JSON for a summary

Here is a small TypeScript script that reads the JSON report and prints a clean summary. Create scripts/playwright-summary.ts:

import { readFileSync } from 'node:fs';

type TestResult = {
  status: string;
  duration: number;
  error?: { message?: string };
};

type TestSpec = {
  title: string;
  ok: boolean;
  tests: Array<{ results: TestResult[] }>;
};

type Suite = {
  title: string;
  suites?: Suite[];
  specs?: TestSpec[];
};

const report = JSON.parse(readFileSync('test-results/results.json', 'utf-8'));

const failed: string[] = [];
let total = 0;

function walkSuite(suite: Suite, path: string[] = []) {
  const currentPath = suite.title ? [...path, suite.title] : path;

  for (const spec of suite.specs ?? []) {
    total += 1;
    if (!spec.ok) {
      failed.push([...currentPath, spec.title].filter(Boolean).join(' > '));
    }
  }

  for (const child of suite.suites ?? []) {
    walkSuite(child, currentPath);
  }
}

for (const suite of report.suites ?? []) {
  walkSuite(suite);
}

console.log(`Total specs: ${total}`);
console.log(`Failed specs: ${failed.length}`);

for (const title of failed.slice(0, 10)) {
  console.log(`- ${title}`);
}

Run it after the test step:

npx tsx scripts/playwright-summary.ts

This is a simple starting point. You can extend it to group failures by tag, component, or team owner.

Write a Small Custom Reporter

Playwright supports custom reporters, and that is useful when your organization has a specific reporting need. Do not start here on Day 1. Start with built-in reporters. Add a custom reporter when the pain is real.

Create a reporter file

Create reporters/failed-tests-reporter.ts:

import type {
  FullConfig,
  FullResult,
  Reporter,
  Suite,
  TestCase,
  TestResult,
} from '@playwright/test/reporter';

class FailedTestsReporter implements Reporter {
  private failures: string[] = [];

  onBegin(config: FullConfig, suite: Suite) {
    console.log(`Starting ${suite.allTests().length} Playwright tests`);
    console.log(`Workers configured: ${config.workers}`);
  }

  onTestEnd(test: TestCase, result: TestResult) {
    if (result.status !== 'passed') {
      const title = test.titlePath().join(' > ');
      const retryText = result.retry ? `retry ${result.retry}` : 'first attempt';
      this.failures.push(`${title} (${result.status}, ${retryText})`);
    }
  }

  onEnd(result: FullResult) {
    console.log(`Final status: ${result.status}`);

    if (this.failures.length === 0) {
      console.log('No failed tests.');
      return;
    }

    console.log('Failed tests:');
    for (const failure of this.failures) {
      console.log(`- ${failure}`);
    }
  }
}

export default FailedTestsReporter;

Add it to the config

reporter: [
  ['list'],
  ['html', { outputFolder: 'playwright-report', open: 'never' }],
  ['./reporters/failed-tests-reporter.ts'],
],

This custom reporter prints a compact failure list at the end of the run. It is intentionally small. Custom reporters become hard to maintain when they turn into a second test management platform.

When custom reporters make sense

Use a custom reporter when you need one of these:

  • A team-specific Slack or Teams message format.
  • A mapping between test tags and component owners.
  • A release gate summary for a deployment pipeline.
  • A failure digest that removes noisy logs.
  • A bridge to an internal dashboard.

Do not write a custom reporter because it looks senior. The senior move is to keep the reporting layer boring until the business needs more.

Attach Screenshots, Traces, and Videos

A report without artifacts is a table. A report with artifacts is evidence. Playwright gives you three important artifact types: screenshots, traces, and videos.

Use traces for failed tests

For most teams, I prefer this setting:

use: {
  trace: 'retain-on-failure',
  screenshot: 'only-on-failure',
  video: 'retain-on-failure',
}

This keeps successful runs lighter and stores evidence for failures. Trace files can grow across large suites, so do not turn on full trace collection for every local run unless you have a reason.

Add manual attachments

You can attach extra files to a test result through testInfo.attach. This is useful for API responses, generated test data, or screenshots that you capture at a meaningful business step.

import { test, expect } from '@playwright/test';

test('checkout shows order confirmation', async ({ page }, testInfo) => {
  await page.goto('/shop');
  await page.getByRole('button', { name: 'Add to cart' }).click();
  await page.getByRole('link', { name: 'Checkout' }).click();

  await page.screenshot({ path: 'test-results/checkout-before-pay.png' });
  await testInfo.attach('checkout before payment', {
    path: 'test-results/checkout-before-pay.png',
    contentType: 'image/png',
  });

  await page.getByRole('button', { name: 'Pay now' }).click();
  await expect(page.getByText('Order confirmed')).toBeVisible();
});

Use attachments carefully. If every test attaches five files, your report becomes heavy and hard to scan. Attach only evidence that helps triage.

Connect reporting with locators

Good reporting cannot fix bad locators. If a test fails because a CSS chain breaks every week, your report will only show repeated noise. Review Day 2 on Playwright locators and assertions if your reports keep showing timeout failures on fragile selectors.

The best report is the one that points to a real product issue. Stable locators, clear assertions, and meaningful test titles make the report useful before you add a single custom tool.

Common Reporting Pitfalls

I have made most of these mistakes in real frameworks. Fix them early.

Pitfall 1: Only generating reports locally

A local report helps one tester. A CI artifact helps the whole team. If your CI job does not upload playwright-report and test-results, the framework is incomplete.

Pitfall 2: Forgetting if: always()

In GitHub Actions, artifact upload steps should usually run with if: always(). Otherwise the test failure can skip the upload step, which removes the exact evidence you need.

Pitfall 3: Storing reports in Git

Generated reports do not belong in Git. They create noisy diffs, slow clones, and accidental commits of environment data. Store them as CI artifacts instead.

Pitfall 4: Keeping every video forever

Video is useful for hard UI failures, but it is expensive for storage. Use retain-on-failure in CI. Keep retention days short unless your compliance process says otherwise.

Pitfall 5: Weak test titles

A report is only as clear as the test title. This is bad:

test('test checkout', async ({ page }) => {
  // ...
});

This is better:

test('guest user can pay by UPI and see order confirmation', async ({ page }) => {
  // ...
});

The second title gives a developer context before opening the trace.

Pitfall 6: Mixing report folders across shards

If you run sharded jobs, each shard should write to a predictable folder or use Playwright blob reports for later merging. We covered the sharding side on Day 15. The reporting side is simple: do not let multiple jobs overwrite the same report path.

Key Takeaways

Playwright reports are not a final polish step. They are part of the automation design.

  • Use HTML reports for human debugging and JUnit XML for CI visibility.
  • Use JSON reports when you need custom summaries or dashboard automation.
  • Keep local reporting lighter than CI reporting.
  • Always upload reports and test results as CI artifacts.
  • Attach traces, screenshots, and videos on failure, not on every successful run.
  • Write clear test titles because they become the report headline.

For Day 16, your homework is simple: add HTML, JUnit, and JSON reporters to your config, run one failing test, upload the artifacts in CI, and check whether a developer can understand the failure without asking you for context.

FAQ

Which Playwright reporter should I use first?

Start with the HTML reporter and one terminal reporter such as line or list. Add JUnit when you connect the suite to CI. Add JSON when you need scripts or dashboards.

Should I enable screenshots, traces, and videos for every test?

No. For most teams, trace: 'retain-on-failure', screenshot: 'only-on-failure', and video: 'retain-on-failure' is a better balance. It keeps evidence for failures without making every run heavy.

How do I share Playwright reports with developers?

Upload the HTML report as a CI artifact and link it from the failed job. If your team uses pull request comments or Slack alerts, include the failed test names and the artifact link.

Can Playwright reports replace a test management tool?

Usually no. Playwright reports explain an automation run. A test management tool may track coverage, manual test cases, release sign-off, and audit history. Use the Playwright report as execution evidence.

What should Day 17 cover after Playwright reports?

The natural next topic is flaky test management: retries, quarantine rules, failure classification, and how to stop a test suite from hiding real bugs behind reruns.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.