GitLab CI for Testing: Pipelines for QA Teams

A merge request reached staging after its pipeline showed green, yet the payment contract tests had never run. A path rule excluded them after a shared schema changed, and the UI displayed success because every created job passed. GitLab CI can give fast, precise feedback, but only if the pipeline expresses risk correctly, proves that expected tests executed, and preserves enough evidence to diagnose a failure.

Design gates around decisions, not tool categories

Start by mapping checks to the decision they inform. A merge request needs quick evidence that the change is safe to review and merge. The default branch may justify broader integration checks. A deployment needs smoke evidence against the deployed artifact.

A practical sequence is static checks, unit tests, integration tests, package, and post-deployment smoke. Stages impose broad ordering, while needs can create a faster dependency graph between jobs. Do not add a separate stage for every framework. Two tools answering the same risk can run in parallel within one stage.

Set a time budget for each gate and decide which failures block. A quarantined flaky suite should not silently become optional forever. Give it an owner, an expiry, and a visible nonblocking job while reliable checks remain mandatory.

Establish a small, explicit pipeline

Keep project behavior in .gitlab-ci.yml and use pinned, purpose-built images. The following pipeline installs dependencies once per job, publishes test reports even after failure, and makes the integration dependency explicit.

stages:
  - verify
  - integration

default:
  image: node:22-bookworm-slim
  interruptible: true
  before_script:
    - npm ci --cache .npm --prefer-offline
  cache:
    key:
      files:
        - package-lock.json
    paths:
      - .npm/

unit_tests:
  stage: verify
  script:
    - npm run test:unit -- --ci --reporters=default --reporters=jest-junit
  variables:
    JEST_JUNIT_OUTPUT_DIR: test-results/unit
    JEST_JUNIT_OUTPUT_NAME: junit.xml
  artifacts:
    when: always
    reports:
      junit: test-results/unit/junit.xml
    paths:
      - test-results/unit/
    expire_in: 7 days

api_tests:
  stage: integration
  needs: ["unit_tests"]
  services:
    - name: postgres:16.4-alpine
      alias: db
  variables:
    POSTGRES_DB: orders_test
    POSTGRES_USER: runner
    POSTGRES_PASSWORD: runner
    DATABASE_URL: postgresql://runner:runner@db:5432/orders_test
  script:
    - npm run db:migrate
    - npm run test:api
  artifacts:
    when: always
    reports:
      junit: test-results/api/junit.xml
    paths:
      - test-results/api/
      - logs/

The service alias db becomes the network hostname. A job container should not connect to the database through localhost. Verify this baseline on a deliberately failing assertion and confirm the job is red while its JUnit report remains visible.

Control when pipelines and jobs exist

Use workflow: rules to avoid duplicate branch and merge request pipelines. Use job rules to select checks based on pipeline source and relevant changes, but remember that excluded jobs provide no evidence.

workflow:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
    - if: '$CI_PIPELINE_SOURCE == "schedule"'
    - when: never

contract_tests:
  stage: integration
  script:
    - npm ci
    - npm run test:contracts
    - test -s test-results/contracts/junit.xml
  rules:
    - changes:
        - services/**/*
        - contracts/**/*
        - package-lock.json
    - if: '$CI_PIPELINE_SOURCE == "schedule"'

Path selection fails when dependency knowledge is incomplete. Shared libraries, base images, test fixtures, and CI templates may affect suites outside the edited service. Maintain the path map as architecture changes, and run the complete set on the default branch or a schedule to detect missed dependencies.

Review the created pipeline, not just the YAML source. For a documentation-only change, a service change, a shared-contract change, and a scheduled run, record which jobs appear. Compare that matrix with the team's intended policy. This catches rules that are individually valid but interact in surprising order. In a rules list, evaluation stops at the first matching rule, so place narrow exceptions before broad branch conditions and make the fallback deliberate.

The final test -s prevents “no tests discovered” from passing because a runner returned zero. Prefer framework options that fail on zero tests when available.

Manage variables and protected access safely

Put secrets in GitLab CI/CD variables, not YAML, repository files, command arguments, or artifacts. Mask values that can be masked and protect credentials intended only for protected refs. Scope environment credentials to the environment that needs them.

Merge requests from untrusted forks should not receive deployment or production-like secrets. Split trusted deployment checks from ordinary verification and use short-lived credentials when the surrounding platform supports them.

Nonsecret configuration still deserves validation. Fail early when required values are absent:

: "${TEST_BASE_URL:?TEST_BASE_URL must be set}"
: "${TEST_TENANT_ID:?TEST_TENANT_ID must be set}"
npm run test:smoke

Do not print all environment variables during troubleshooting. Logs and dotenv artifacts can expose tokens. Give test accounts the minimum permissions needed, rotate credentials, and make synthetic data visibly nonproduction.

Separate cache, artifacts, and reports

A cache speeds future jobs but is not guaranteed evidence. Artifacts carry files from a completed job. Report artifacts let GitLab interpret formats such as JUnit. Treat the three mechanisms differently.

Cache downloaded dependencies keyed by the lockfile. Do not cache node_modules across incompatible images, mutable databases, or generated test results. A cold cache must produce the same outcome as a warm cache.

Publish JUnit XML for merge request feedback, plus human diagnostic files such as screenshots, traces, and service logs. Set when: always so a failed test still uploads them. Keep paths repository-relative and confirm the test process writes them before exiting.

Use needs with artifacts when a downstream job consumes an upstream build. Package the application once, then test and deploy that exact artifact. If each stage rebuilds source independently, the released bytes may not be the tested bytes.

Reports also need stable test-case identities. If the same case name appears across several suites, merge request summaries can be difficult to interpret. Include the capability and meaningful scenario in test names, but keep volatile data such as random IDs out of them. Store volatile details in logs or attachments instead. Validate XML before upload when a custom reporter is involved; malformed output should fail the job rather than erase the evidence from an otherwise valid test run.

Parallelize suites without losing determinism

GitLab can create several identical jobs with parallel, exposing the node index and total. The test runner must split cases deterministically.

browser_tests:
  stage: integration
  image: mcr.microsoft.com/playwright:v1.52.0-noble
  parallel: 4
  script:
    - npm ci
    - npx playwright test --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL
  artifacts:
    when: always
    reports:
      junit: test-results/junit-$CI_NODE_INDEX.xml
    paths:
      - test-results/
      - playwright-report/

Keep shard output names unique or later uploads will overwrite evidence. Test accounts and data also need shard-specific identifiers. Parallel jobs that modify one shared customer or database trade speed for race conditions.

Balance by historical duration if the framework supports it, but retain stable ownership of cases. Verify parallelization by comparing the collected test IDs with a single-worker run. The union should match exactly, with neither gaps nor duplicates.

Use resource_group for operations that truly must serialize, such as tests against one exclusive environment. It is not a fix for avoidable shared-state design.

Test deployed environments with controlled jobs

Deployment smoke tests should target an environment URL passed from the deploy job, not a hard-coded shared host. Name review environments with stable GitLab variables and ensure they have a teardown path.

Use environment metadata so the pipeline records what a job targeted. Keep smoke checks small: health, authentication, one critical read, and perhaps one reversible transaction. Large end-to-end suites make rollback decisions slow and often fail for unrelated data reasons.

Manual jobs are appropriate for costly or destructive validation, but a required release control should not depend on someone remembering to click it. Encode the policy through protected environments, required jobs, or the project's merge controls.

When testing after deployment, record the commit SHA, artifact identifier, base URL, and test-data namespace in the report. That evidence prevents a green smoke run against the wrong environment from approving a release.

Diagnose and maintain the pipeline as product code

Lint the CI configuration, review template changes carefully, and test included templates at a fixed reference. A remote include that moves independently can change pipeline behavior without an application diff.

For every failed job, distinguish product failure, test defect, runner infrastructure, and environment setup. Preserve timestamps and dependency logs so that classification is evidence-based. Retrying is acceptable after an identified transient cause; blind retries distort reliability.

Track useful operational signals such as time to first actionable failure, queue delay, flaky-case ownership, and percentage of failures with complete artifacts. A lower duration is not an improvement if path rules stopped important checks from running.

Periodically simulate a failing unit assertion, an unavailable database, an empty test selection, and a missing report. Confirm each produces the intended red gate and useful evidence. A trustworthy GitLab pipeline does more than execute commands. It makes the required work visible, binds tests to the artifact and environment under judgment, and fails loudly when its own proof is missing.