Skip to main content

GitHub Actions

info

What you'll learn

  • How to run Cypress tests with GitHub Actions as part of CI/CD pipeline
  • How to parallelize Cypress test runs within GitHub Actions
  • How to cache build artifacts between installation jobs and worker jobs

GitHub offers developers Actions that provide a way to automate, customize, and execute your software development workflows within your GitHub repository. Detailed documentation is available in the GitHub Action Documentation.

Cypress GitHub Action

Workflows can be packaged and shared as GitHub Actions. GitHub maintains many, such as the checkout and Upload/Download Artifact Actions actions used below.

The Cypress team maintains the official Cypress GitHub Action for running Cypress tests. This action provides npm installation, custom caching, additional configuration options and simplifies setup of advanced workflows with Cypress in the GitHub Actions platform.

Basic Setup

The example below is basic CI setup and job using the Cypress GitHub Action to run Cypress tests within the Electron browser. This GitHub Action configuration is placed within .github/workflows/main.yml.

name: Cypress Tests

on: [push]

jobs:
cypress-run:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
# Install NPM dependencies, cache them correctly
# and run all Cypress tests
- name: Cypress run
uses: cypress-io/github-action@v2
with:
build: npm run build
start: npm start
tip
Try it out

To try out the example above yourself, fork the Cypress Kitchen Sink example project and place the above GitHub Action configuration in .github/workflows/main.yml.

How this action works:

  • On push to this repository, this job will provision and start GitHub-hosted Ubuntu Linux instance for running the outlined steps for the declared cypress-run job within the jobs section of the configuration.
  • The GitHub checkout Action is used to checkout our code from our GitHub repository.
  • Finally, our Cypress GitHub Action will:
    • Install npm dependencies
    • Build the project (npm run build)
    • Start the project web server (npm start)
    • Run the Cypress tests within our GitHub repository within Electron.

Testing in Chrome and Firefox with Cypress Docker Images

GitHub Actions provides the option to specify a container image for the job. Cypress offers various Docker Images for running Cypress locally and in CI.

Below we add the container attribute using a Cypress Docker Image built with Google Chrome and Firefox. For example, this allows us to run the tests in Firefox by passing the browser: firefox attribute to the Cypress GitHub Action.

name: Cypress Tests using Cypress Docker Image

on: [push]

jobs:
cypress-run:
runs-on: ubuntu-latest
container: cypress/browsers:node12.18.3-chrome87-ff82
steps:
- name: Checkout
uses: actions/checkout@v2

# Install NPM dependencies, cache them correctly
# and run all Cypress tests
- name: Cypress run
uses: cypress-io/github-action@v2
with:
# Specify Browser since container image is compile with Firefox
browser: firefox

Caching Dependencies and Build Artifacts

The Cypress team maintains the official Cypress GitHub Action for running Cypress tests. This action provides npm installation, custom caching, additional configuration options and simplifies setting up advanced workflows with Cypress in the GitHub Actions platform.

info

Caching of dependencies and build artifacts between installation and worker jobs can be accomplished with the Upload/Download Artifact Actions.

The install job below uses the upload-artifact action and will save the state of the build directory for the worker jobs.

name: Cypress Tests with installation job

on: [push]

jobs:
install:
runs-on: ubuntu-latest
container: cypress/browsers:node12.18.3-chrome87-ff82
steps:
- name: Checkout
uses: actions/checkout@v2

- name: Save build folder
uses: actions/upload-artifact@v2
with:
name: build
if-no-files-found: error
path: build

- name: Cypress install
uses: cypress-io/github-action@v2
with:
# Disable running of tests within install job
runTests: false
build: yarn build

The download-artifact action will retrieve the build directory saved in the install job, as seen below in a worker job.

name: Cypress Tests with Dependency and Artifact Caching

on: [push]

jobs:
# install:
# ....
cypress-run:
runs-on: ubuntu-latest
container: cypress/browsers:node12.18.3-chrome87-ff82
steps:
- name: Checkout
uses: actions/checkout@v2

- name: Download the build folders
uses: actions/download-artifact@v2
with:
name: build
path: build

# Install NPM dependencies, cache them correctly
# and run all Cypress tests
- name: Cypress run
uses: cypress-io/github-action@v2
with:
# Specify Browser since container image is compile with Firefox
browser: firefox
build: yarn build

Parallelization

The Cypress Dashboard offers the ability to parallelize and group test runs along with additional insights and analytics for Cypress tests.

GitHub Actions offers a matrix strategy for declaring different job configurations for a single job definition. Jobs declared within a matrix strategy can run in parallel which enables us run multiples instances of Cypress at same time as we will see later in this section.

Before diving into an example of a parallelization setup, it is important to understand the two different types of GitHub Action jobs that we will declare:

  • Install Job: A job that installs and caches dependencies that will be used by subsequent jobs later in the GitHub Action workflow.
  • Worker Job: A job that handles execution of Cypress tests and depends on the install job.

Install Job

The separation of installation from test running is necessary when running parallel jobs. It allows for reuse of various build steps aided by caching.

First, we'll define the install step that will be used by the worker jobs defined in the matrix strategy.

For the steps, notice that we pass runTests: false to the Cypress GitHub Action to instruct it to only install and cache Cypress and npm dependencies without running the tests.

The upload-artifact action will save the state of the build directory for the worker jobs.

name: Cypress Tests with installation job

on: [push]

jobs:
install:
runs-on: ubuntu-latest
container: cypress/browsers:node12.18.3-chrome87-ff82
steps:
- name: Checkout
uses: actions/checkout@v2

- name: Cypress install
uses: cypress-io/github-action@v2
with:
# Disable running of tests within install job
runTests: false
build: yarn build

- name: Save build folder
uses: actions/upload-artifact@v2
with:
name: build
if-no-files-found: error
path: build

Worker Jobs

Next, we define the worker job named ui-chrome-tests that will run Cypress tests with Chrome as part of a parallelized matrix strategy.

info
Note

Using our Cypress GitHub Action we specify install: false since our dependencies and build were cached in our install job.

The download-artifact action will retrieve the build directory saved in the install job.

name: Cypress Tests with Install Job and UI Chrome Job x 5

on: [push]

jobs:
install:
# ...

ui-chrome-tests:
runs-on: ubuntu-latest
container: cypress/browsers:node12.18.3-chrome87-ff82
needs: install
strategy:
fail-fast: false
matrix:
# run copies of the current job in parallel
containers: [1, 2, 3, 4, 5]
steps:
- name: Checkout
uses: actions/checkout@v2

- name: Download the build folders
uses: actions/download-artifact@v2
with:
name: build
path: build

- name: 'UI Tests - Chrome'
uses: cypress-io/github-action@v2
with:
# we have already installed all dependencies above
install: false
start: yarn start:ci
wait-on: 'http://localhost:3000'
wait-on-timeout: 120
browser: chrome
record: true
parallel: true
group: 'UI - Chrome'
spec: cypress/tests/ui/*
env:
CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
# Recommended: pass the GitHub token lets this action correctly
# determine the unique run id necessary to re-run the checks
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
note

The above configuration using the parallel and record options of the Cypress GitHub Action requires setting up recording to the Cypress Dashboard.

Setting up Parallelization

To setup multiple containers to run in parallel, the matrix option of the strategy configuration can be set to containers: [1, 2, 3, 4, 5], which will start 5 instances of the defined container image.

info
Note

The containers array is filled with filler (or dummy) items to provision the desired number of CI machine instances within GitHub Actions.

caution
Ensure Correct Container

The container attribute must be specified using the Cypress Docker Image in the configuration that was used in the install job.

#...
ui-chrome-tests:
runs-on: ubuntu-latest
container: cypress/browsers:node12.18.3-chrome87-ff82
needs: install
strategy:
fail-fast: false
matrix:
# run copies of the current job in parallel
containers: [1, 2, 3, 4, 5]

Using the Cypress Dashboard with GitHub Actions

In the GitHub Actions configuration we have defined in the previous section, we are leveraging three useful features of the Cypress Dashboard:

  1. Recording test results with the record: true option to the Cypress Dashboard:

  2. Parallelizing test runs and optimizing their execution via intelligent load-balancing of test specs across CI machines with the parallel: true option.

  3. Organizing and consolidating multiple cypress run calls by labeled groups into a single report within the. Cypress Dashboard. In the example above we use the group: "UI - Chrome" option to organize all UI tests for the Chrome browser into a group labeled "UI - Chrome" in the Cypress Dashboard report.

Cypress Real World Example with GitHub Actions

A complete CI workflow against multiple browsers, viewports and operating systems is available in the Real World App (RWA).

Clone the Real World App (RWA) and refer to the .github/workflows/main.yml file.

Cypress Real World App GitHub Actions Matrix

Common Problems and Solutions

Re-run jobs passing with empty tests

We recommend passing the GITHUB_TOKEN secret (created by the GH Action automatically) as an environment variable. This will allow the accurate identification of each build to avoid confusion when re-running a build.

name: Cypress tests
on: [push]
jobs:
cypress-run:
name: Cypress run
runs-on: ubuntu-20.04
steps:
- name: Checkout
uses: actions/checkout@v2

- name: Cypress run
uses: cypress-io/github-action@v2
with:
record: true
env:
# pass GitHub token to detect new build vs re-run build
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}

Pull requests commit message is merge SHA into SHA

You can overwrite the commit message sent to the Dashboard by setting an environment variable. See Issue #124 for more details.

name: Cypress tests
on: [push]
jobs:
cypress-run:
name: Cypress run
runs-on: ubuntu-20.04
steps:
- name: Checkout
uses: actions/checkout@v2

- name: Cypress run
uses: cypress-io/github-action@v2
with:
record: true
env:
# overwrite commit message sent to Dashboard
COMMIT_INFO_MESSAGE: ${{github.event.pull_request.title}}
# re-enable PR comment bot
COMMIT_INFO_SHA: ${{github.event.pull_request.head.sha}}
info

We also recommend adding COMMIT_INFO_SHA to re-enable Cypress bot PR comments. See this comment for more details.

See also