Azure DevOps Pipeline for Playwright Tests

Your Playwright suite runs green on your laptop but you have no idea whether the latest pull request actually broke the checkout flow before it merges. A Playwright Azure DevOps pipeline closes that gap by running your end-to-end tests automatically on every push, publishing a clean HTML report, and blocking bad merges. In this guide you will build that pipeline from scratch with TypeScript, wire up sharding for speed, capture traces on failure, and publish results back into the Azure DevOps UI.

🎭 Want to master this with real projects? Join the Playwright Automation Mastery course at The Testing Academy.

Why run Playwright in Azure DevOps at all?

Local runs lie. A test that passes on macOS with your fonts, your timezone, and your warm browser cache can fail on a clean Linux agent. Continuous integration gives you a reproducible, headless environment that mirrors production far more closely than a developer machine ever will. Azure DevOps Pipelines specifically gives you Microsoft-hosted Linux agents with Node.js preinstalled, a generous free tier for public projects, native test-reporting tabs, and tight integration with branch policies so a red pipeline can physically prevent a merge.

The official mcr.microsoft.com/playwright Docker image ships every browser and OS dependency Playwright needs, which removes the single most common source of CI flakiness: missing system libraries. The rest of this article assumes a standard Playwright + TypeScript project created with npm init playwright@latest.

Prerequisites and project layout

Before touching YAML, make sure your repository contains a sane playwright.config.ts tuned for CI. The most important knobs are forbidOnly (fails the build if someone left a test.only behind), retries (one retry on CI smooths over genuine network blips without hiding real bugs), and a list of reporters that includes the JUnit format Azure DevOps can read natively.

// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  // Fail fast if a stray test.only is committed.
  forbidOnly: !!process.env.CI,
  // Retry once on CI to absorb transient flakiness, never locally.
  retries: process.env.CI ? 1 : 0,
  // Opt out of parallelism inside a single file on CI for determinism.
  workers: process.env.CI ? 2 : undefined,
  reporter: [
    ['list'],
    ['html', { open: 'never' }],
    ['junit', { outputFile: 'results/junit-results.xml' }],
  ],
  use: {
    baseURL: process.env.BASE_URL ?? 'https://playwright.dev',
    // Keep the trace only for the first retry of a failing test.
    trace: 'on-first-retry',
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
  },
  projects: [
    { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
    { name: 'firefox', use: { ...devices['Desktop Firefox'] } },
    { name: 'webkit', use: { ...devices['Desktop Safari'] } },
  ],
});

Two reporters do the heavy lifting here. The junit reporter produces an XML file that the Publish Test Results task ingests so individual test pass/fail status shows up in the Azure DevOps Tests tab. The html reporter produces the rich, clickable report you will upload as a build artifact for debugging.

Your first azure-pipelines.yml

Azure DevOps reads a file named azure-pipelines.yml at the repository root. The simplest working pipeline checks out the code, installs Node, installs dependencies, installs Playwright browsers, and runs the tests. Use npm ci rather than npm install so the build respects your lockfile exactly and stays reproducible.

# azure-pipelines.yml
trigger:
  - main

pool:
  vmImage: 'ubuntu-latest'

steps:
  - task: NodeTool@0
    inputs:
      versionSpec: '20.x'
    displayName: 'Install Node.js 20'

  - script: npm ci
    displayName: 'Install dependencies'

  - script: npx playwright install --with-deps
    displayName: 'Install Playwright browsers'

  - script: npx playwright test
    displayName: 'Run Playwright tests'
    env:
      CI: 'true'

  - task: PublishTestResults@2
    displayName: 'Publish JUnit results'
    condition: succeededOrFailed()
    inputs:
      testResultsFormat: 'JUnit'
      testResultsFiles: 'results/junit-results.xml'

  - task: PublishBuildArtifacts@1
    displayName: 'Publish HTML report'
    condition: succeededOrFailed()
    inputs:
      pathToPublish: 'playwright-report'
      artifactName: 'playwright-report'

The detail that trips up newcomers is condition: succeededOrFailed() on the two publish steps. Without it, a failing test run aborts the job and your report never gets published, which is precisely when you need it most. That condition forces both steps to run even after npx playwright test exits non-zero. The --with-deps flag tells Playwright to install the Linux system libraries the browsers need, so you do not have to hand-curate an apt-get list.

Going faster with sharding

Once your suite grows past a few hundred tests, a single agent becomes the bottleneck. Playwright supports sharding natively: pass --shard=1/3 to run the first third of tests, --shard=2/3 for the second, and so on. Azure DevOps lets you fan these out across parallel agents with a job strategy.matrix. Each shard is an independent job, so three shards cut wall-clock time roughly in three.

# azure-pipelines.yml (sharded)
trigger:
  - main

jobs:
  - job: e2e
    pool:
      vmImage: 'ubuntu-latest'
    strategy:
      matrix:
        shard_1: { SHARD: '1/3' }
        shard_2: { SHARD: '2/3' }
        shard_3: { SHARD: '3/3' }
    steps:
      - task: NodeTool@0
        inputs:
          versionSpec: '20.x'
      - script: npm ci
        displayName: 'Install dependencies'
      - script: npx playwright install --with-deps
        displayName: 'Install browsers'
      - script: npx playwright test --shard=$(SHARD)
        displayName: 'Run shard $(SHARD)'
        env:
          CI: 'true'
      - task: PublishTestResults@2
        condition: succeededOrFailed()
        inputs:
          testResultsFormat: 'JUnit'
          testResultsFiles: 'results/junit-results.xml'
          testRunTitle: 'Playwright shard $(SHARD)'

One nuance with sharding: each shard generates its own partial HTML report and its own JUnit XML. Giving each PublishTestResults@2 task a distinct testRunTitle keeps the Azure DevOps Tests tab readable. If you want a single unified HTML report instead of three partial ones, configure the blob reporter on each shard, publish those blobs as artifacts, then run npx playwright merge-reports in a final dependent job. The blob reporter exists specifically to make cross-shard report merging possible.

Running inside the official Playwright container

The --with-deps install adds a minute or two to every run. You can skip it entirely by running the whole job inside Microsoft’s prebuilt image, which already contains the browsers and their dependencies. Pin the image tag to the exact Playwright version in your package.json so the container browsers and your @playwright/test version never drift apart, a mismatch that produces confusing "executable doesn't exist" errors.

# azure-pipelines.yml (container)
trigger:
  - main

pool:
  vmImage: 'ubuntu-latest'

container: mcr.microsoft.com/playwright:v1.50.0-noble

steps:
  - script: npm ci
    displayName: 'Install dependencies'
  - script: npx playwright test
    displayName: 'Run tests in Playwright container'
    env:
      CI: 'true'
  - task: PublishTestResults@2
    condition: succeededOrFailed()
    inputs:
      testResultsFormat: 'JUnit'
      testResultsFiles: 'results/junit-results.xml'

Notice there is no playwright install step at all. The container already has Chromium, Firefox, and WebKit baked in. The -noble suffix refers to the Ubuntu 24.04 base; older images use -jammy or -focal. Always match the v1.50.0 portion to your installed Playwright release.

Caching, environments, and secrets

A real pipeline rarely tests against a hardcoded public URL. You will need a base URL per environment and credentials that must never appear in YAML. Azure DevOps offers pipeline variables for non-secret values and variable groups backed by Azure Key Vault for secrets. Reference them as environment variables in your test code through process.env, exactly as you would locally.

// tests/login.spec.ts
import { test, expect } from '@playwright/test';

test('user can sign in with CI-provided credentials', async ({ page }) => {
  // BASE_URL, TEST_USER and TEST_PASS come from an Azure DevOps variable group.
  await page.goto('/login');
  await page.getByLabel('Email').fill(process.env.TEST_USER!);
  await page.getByLabel('Password').fill(process.env.TEST_PASS!);
  await page.getByRole('button', { name: 'Sign in' }).click();

  await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();
});

To expose a secret variable to a script step you must map it explicitly into env:, because Azure DevOps deliberately does not auto-inject secrets into the process environment. That mapping looks like this:

  - script: npx playwright test
    displayName: 'Run tests with secrets'
    env:
      CI: 'true'
      BASE_URL: $(BASE_URL)
      TEST_USER: $(TEST_USER)
      TEST_PASS: $(TEST_PASS)   # secret from variable group

To shave time off repeat runs, cache the npm download directory with the Cache@2 task keyed on your lockfile hash. Note that you should cache ~/.npm and still run npm ci; caching node_modules directly is fragile across agent images and not recommended.

Capturing traces and debugging failures

The single biggest productivity win in CI is the Playwright trace. With trace: 'on-first-retry' set in the config, every test that fails once produces a trace.zip containing a full DOM snapshot timeline, network log, and console output. Those traces land inside the HTML report you already publish as an artifact. Download the artifact, unzip it, and open it with the trace viewer.

# After downloading the published artifact locally:
npx playwright show-trace path/to/trace.zip

# Or open the whole HTML report, which embeds every trace:
npx playwright show-report ./playwright-report

For assertions where you want the test to keep going and collect multiple failures rather than stop at the first, use expect.soft. Soft assertions are invaluable in CI because one run surfaces every broken expectation on a page instead of forcing you to fix-and-rerun one at a time.

import { test, expect } from '@playwright/test';

test('checkout summary shows all line items', async ({ page }) => {
  await page.goto('/checkout');

  // Soft assertions: collect every failure, fail the test at the end.
  await expect.soft(page.getByTestId('subtotal')).toHaveText('$120.00');
  await expect.soft(page.getByTestId('tax')).toHaveText('$9.60');
  await expect.soft(page.getByTestId('total')).toHaveText('$129.60');

  // A hard assertion still stops the test immediately if it fails.
  await expect(page.getByRole('button', { name: 'Place order' })).toBeEnabled();
});

Comparing the pipeline strategies

There is no single "best" setup; the right choice depends on suite size and how much you care about cold-start time. The table below summarizes the trade-offs between the three approaches covered above.

Approach	Browser install step	Best for	Wall-clock speed
Hosted agent + install –with-deps	Required each run	Small suites, simplest YAML	Baseline
Matrix sharding (3 agents)	Required per shard	Large suites needing speed	~3x faster
Official Playwright container	None (preinstalled)	Consistent deps, fewer flakes	Faster cold start

Enforcing the pipeline with branch policies

A pipeline that nobody has to pass is just decoration. The final step is wiring the Playwright Azure DevOps pipeline into a branch policy so a red build blocks the merge. In Project Settings → Repositories → Policies for your main branch, add a Build Validation policy that points at your pipeline and set it to required. Now every pull request must show a green Playwright run before the Complete button unlocks, which is the entire point of investing in CI in the first place.

From here you can layer on scheduled nightly runs against a staging environment, fan out across more shards as the suite grows, or post the HTML report URL into a pull-request comment. But the core loop, run on every push, publish results, block bad merges, is now complete and will pay for itself the first time it catches a regression before customers do.

FAQ

Why do my Playwright tests pass locally but fail in the Azure DevOps pipeline?

The usual culprits are timezone, locale, viewport, and missing system fonts on the clean Linux agent, plus genuine timing differences because hosted agents are often slower than your laptop. Run the official mcr.microsoft.com/playwright container to eliminate dependency drift, set an explicit timezoneId and locale in your config, and enable trace: 'on-first-retry' so you can open the failing run in the trace viewer and see exactly what the agent saw.

How do I see Playwright test results inside the Azure DevOps UI?

Add the JUnit reporter to your playwright.config.ts so it writes an XML file, then add a PublishTestResults@2 task with testResultsFormat: 'JUnit' pointing at that file. Pass-fail status then appears in the pipeline run’s Tests tab. Publish the playwright-report folder as a build artifact with PublishBuildArtifacts@1 to get the rich, clickable HTML report with embedded traces.

Should I use sharding or the Playwright container for my pipeline?

They solve different problems and you can use both together. The container removes the browser-install step and guarantees consistent dependencies, which mainly improves reliability and cold-start time. Sharding splits the test run across multiple parallel agents to cut wall-clock time and matters most for large suites. For a big project, run a sharded matrix where each shard executes inside the official container.