GitHub Actions for Test Automation

GitHub Actions is GitHub’s built-in CI/CD system. A workflow is a YAML file in .github/workflows/ that runs jobs on hosted or self-hosted machines when events happen, such as a push or a pull request. For testers, Actions is where your suite becomes a required check: every PR runs the tests, and the results decide whether it can merge.

Workflow anatomy

name: Tests

on:
  pull_request:
  push:
    branches: [main]
  workflow_dispatch:        # manual "Run workflow" button

jobs:
  unit:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: npm
      - run: npm ci
      - run: npm test

The building blocks:

• on: the triggering events. pull_request runs for PRs, push with a branch filter runs after merges, workflow_dispatch adds a manual trigger, and schedule (cron) runs nightly suites.
• jobs: independent units of work that run in parallel by default. Use needs to order them.
• runs-on: the runner image. ubuntu-latest is the usual choice for browser tests.
• steps: either uses (a reusable action) or run (a shell command).

actions/setup-node@v4 with cache: npm caches the npm download cache keyed on your lockfile, so npm ci is faster on later runs. Always use npm ci in CI: it installs exactly what package-lock.json specifies and fails if the lockfile and package.json disagree.

Set timeout-minutes on every job. The default is six hours, and a hung browser can burn a lot of runner time before anyone notices.

Secrets and environment variables

Never hard-code credentials in a workflow. Store them in the repository or environment settings as secrets and reference them through the secrets context:

    steps:
      - run: npx playwright test
        env:
          BASE_URL: https://staging.example.com
          TEST_USER_EMAIL: ${{ secrets.TEST_USER_EMAIL }}
          TEST_USER_PASSWORD: ${{ secrets.TEST_USER_PASSWORD }}

Actions masks secret values in logs, but masking is a safety net, not a guarantee: do not echo secrets or write them into reports. Note also that workflows triggered by pull_request from a fork do not receive repository secrets, which is a deliberate protection. Design your suite so that fork PRs can at least run tests that need no secrets.

Use dedicated test accounts with the least privilege needed, never real user or admin credentials.

Running Playwright in CI

A single-job Playwright workflow:

name: E2E

on:
  pull_request:
  push:
    branches: [main]

jobs:
  e2e:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: npm
      - run: npm ci
      - run: npx playwright install --with-deps
      - run: npx playwright test
      - uses: actions/upload-artifact@v4
        if: ${{ !cancelled() }}
        with:
          name: playwright-report
          path: playwright-report/
          retention-days: 14

npx playwright install --with-deps downloads the browsers and the operating system libraries they need. The if: ${{ !cancelled() }} condition matters: by default a step is skipped once an earlier step fails, which is exactly when you most need the report. With this condition the upload runs after failures but not when the run is cancelled.

In playwright.config.ts, a few CI-specific settings are worth having:

import { defineConfig } from '@playwright/test';

export default defineConfig({
  forbidOnly: !!process.env.CI,        // fail if a test.only slipped in
  retries: process.env.CI ? 2 : 0,
  reporter: process.env.CI ? 'blob' : 'html',
  use: {
    trace: 'on-first-retry',           // record a trace when a test is retried
  },
});

GitHub Actions sets CI=true automatically, so process.env.CI is truthy on runners.

Sharding with a matrix

When a suite takes too long on one machine, split it. Playwright’s --shard=x/y option runs one slice of the tests, and an Actions matrix starts one job per slice in parallel.

name: E2E (sharded)

on:
  pull_request:
  push:
    branches: [main]

jobs:
  e2e:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    strategy:
      fail-fast: false
      matrix:
        shardIndex: [1, 2, 3, 4]
        shardTotal: [4]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: npm
      - run: npm ci
      - run: npx playwright install --with-deps
      - run: npx playwright test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
      - uses: actions/upload-artifact@v4
        if: ${{ !cancelled() }}
        with:
          name: blob-report-${{ matrix.shardIndex }}
          path: blob-report/
          retention-days: 1

  merge-reports:
    if: ${{ !cancelled() }}
    needs: [e2e]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: npm
      - run: npm ci
      - uses: actions/download-artifact@v4
        with:
          path: all-blob-reports
          pattern: blob-report-*
          merge-multiple: true
      - run: npx playwright merge-reports --reporter html ./all-blob-reports
      - uses: actions/upload-artifact@v4
        with:
          name: html-report
          path: playwright-report/
          retention-days: 14

How it fits together:

• fail-fast: false lets all shards finish even if one fails, so you see every failure in one run.
• Each shard writes a blob report (because of reporter: 'blob' in CI) and uploads it under a unique artifact name. Artifact names must be unique per run with upload-artifact@v4, which is why the shard index is in the name.
• The merge-reports job downloads all blobs into one folder and runs npx playwright merge-reports to produce a single HTML report.

Sharding balances best when tests are independent and of similar length. If one spec file holds most of the slow tests, consider enabling fullyParallel: true so Playwright can split at the test level rather than the file level.

Making pipelines fast and trustworthy

Cancel superseded runs

When someone pushes three commits to a PR in a row, the first two runs are wasted. A concurrency group cancels them:

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

Run the right tests at the right time

• On every PR: lint, unit, API and a focused end-to-end set.
• After merge to main or nightly via schedule: the full cross-browser suite.
• Use paths filters sparingly. Skipping a workflow on a path filter can leave a required check “pending” forever, so check how your branch protection treats skipped workflows before relying on them.

Pin what you depend on

Use major version tags such as actions/checkout@v4 at minimum. Security-conscious teams pin third-party actions to a full commit SHA, since a tag can be moved. Also set the minimal permissions the workflow needs:

permissions:
  contents: read

Practice exercise

Your team’s single-job Playwright workflow takes 28 minutes and often hits its 30-minute timeout. Reports are missing when tests fail. Secrets are written directly in the YAML. Rewrite the plan: list the changes you would make and show the key YAML fragments.

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

permissions:
  contents: read

jobs:
  e2e:
    strategy:
      fail-fast: false
      matrix:
        shardIndex: [1, 2, 3, 4]
        shardTotal: [4]
    steps:
      # checkout, setup-node, npm ci, playwright install
      - run: npx playwright test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
        env:
          TEST_USER_PASSWORD: ${{ secrets.TEST_USER_PASSWORD }}
      - uses: actions/upload-artifact@v4
        if: ${{ !cancelled() }}
        with:
          name: blob-report-${{ matrix.shardIndex }}
          path: blob-report/

Key takeaways

• A workflow is events (on), jobs, runners and steps; set timeout-minutes on every job.
• Use actions/setup-node@v4 with cache: npm and always npm ci.
• Keep credentials in secrets; fork PRs do not get them, by design.
• Upload reports with if: ${{ !cancelled() }} so failures are debuggable.
• Shard Playwright with --shard and a matrix, use blob reports and merge them in a final job.
• Cancel superseded runs, set minimal permissions and treat retried-then-passed tests as flaky.

Generate a starting workflow with the GitHub Actions Generator, then adapt it using the patterns above.