CI/CD Testing Pipelines

Anatomy of a Test Pipeline

A well-designed test pipeline runs tests in order of speed: fast tests first, slow tests last. If fast tests fail, you skip the slow ones — saving time and compute.

GitHub Actions: Complete Pipeline

Here's a production-ready test pipeline using GitHub Actions. It runs lint, unit, integration, and E2E tests in separate jobs with proper dependencies.

name: Test Pipeline
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: '3.12' }
      - run: pip install ruff
      - run: ruff check .
      - run: ruff format --check .

  unit:
    needs: lint
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: '3.12' }
      - run: pip install -r requirements-test.txt
      - run: pytest tests/unit/ -v --junitxml=unit-results.xml --cov=src --cov-report=xml
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: unit-results
          path: unit-results.xml

  integration:
    needs: unit
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: testpass
          POSTGRES_DB: testdb
        ports: ['5432:5432']
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: '3.12' }
      - run: pip install -r requirements-test.txt
      - run: pytest tests/integration/ -v --junitxml=integration-results.xml
        env:
          DATABASE_URL: postgresql://postgres:testpass@localhost:5432/testdb

  e2e:
    needs: integration
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: '3.12' }
      - run: pip install -r requirements-test.txt
      - run: playwright install --with-deps chromium
      - run: pytest tests/e2e/ -v --junitxml=e2e-results.xml
      - uses: actions/upload-artifact@v4
        if: failure()
        with:
          name: playwright-traces
          path: test-results/

Parallel Execution

Slow test suites kill developer productivity. Parallel execution is the most impactful optimization — splitting tests across multiple workers or machines.

pytest-xdist (Python)

# Run tests across 4 CPU cores
pytest tests/ -n 4

# Auto-detect available cores
pytest tests/ -n auto

# Split by file (each worker gets complete test files)
pytest tests/ -n 4 --dist loadfile

GitHub Actions Matrix Strategy

# Split E2E tests across 3 parallel machines
e2e:
  strategy:
    matrix:
      shard: [1, 2, 3]
  steps:
    - uses: actions/checkout@v4
    - run: pip install -r requirements-test.txt
    - run: playwright install --with-deps chromium
    - run: |
        pytest tests/e2e/ \
          --splits 3 \
          --group ${{ matrix.shard }} \
          --splitting-algorithm least_duration

Quality Gates

Quality gates are automated checkpoints that block code from merging if it doesn't meet quality standards.

# Enforce coverage threshold in pytest
# pytest.ini or pyproject.toml
[tool.pytest.ini_options]
addopts = "--cov=src --cov-fail-under=80"

Test Artifacts & Reporting

When tests fail in CI, you need enough information to debug without re-running locally. Upload these artifacts:

• JUnit XML — Standard test result format, supported by all CI tools for summary views
• HTML reports — Human-readable test reports (pytest-html, Allure)
• Screenshots — Captured on failure for E2E tests
• Playwright traces — Full interaction replay with DOM snapshots, network logs, and console output
• Coverage reports — HTML or XML coverage data for tracking trends

# Upload Allure results for beautiful reporting
- run: pytest tests/ --alluredir=allure-results
- uses: actions/upload-artifact@v4
  if: always()
  with:
    name: allure-results
    path: allure-results/

Caching & Optimization

CI pipelines that install dependencies on every run waste minutes. Caching eliminates this.

# Cache pip dependencies
- uses: actions/setup-python@v5
  with: { python-version: '3.12' }
- uses: actions/cache@v4
  with:
    path: ~/.cache/pip
    key: ${{ runner.os }}-pip-${{ hashFiles('requirements*.txt') }}
    restore-keys: ${{ runner.os }}-pip-

# Cache Playwright browsers (saves ~1 min)
- uses: actions/cache@v4
  with:
    path: ~/.cache/ms-playwright
    key: playwright-${{ hashFiles('requirements*.txt') }}

Interview Questions