How to automate UI tests with Github Actions
Developers spend 4-8 hrs a week fixing bugs. Things only get worse if a bug sneaks its way into production. It takes 5-10x longer to fix it. UI testing is integral to delivering high-quality experiences but can be a huge time sink. It's too much work to run all your tests manually after every change.
Leading UI engineering teams at Twilio, Target and Adobe automate UI testing. Tests are triggered when a developer pushes code. They execute in the background and report results on completion. That allows you to detect regressions automatically.
This article shows you how to automate your testing pipeline with Github Actions. You'll learn how to report the status of your tests with pull request checks. Along the way, I'll point out common mistakes to avoid.
Continuous UI testing
Reviewing code is a big part of being a developer. It helps catch bugs early and maintains high code quality.
To ensure that a pull request (PR) won't break production, you’d typically pull code and run the test suite locally. That disrupts your workflow and takes a lot of time. With Continuous Integration (CI), you get all the benefits of testing without any manual intervention.
You can tweak the UI, build a new feature, or update a dependency. When you open a pull request, the CI server will automatically run comprehensive UI tests—visual, composition, accessibility, interaction and user flows.
You’ll get test results via PR badges, which provide a summary of all the checks.
At a glance, you can tell if the pull request passed your UI quality checks. If yes, move on to reviewing the actual code. If not, dive into the logs to find out what’s wrong.
Tutorial
In a previous post, I introduced the Taskbox app and demonstrated how to test the various aspects of this UI. Building on that, we’ll set up continuous integration using GitHub Actions. Grab the code and follow along.
Setup CI
Create a .github/workflows/ui-tests.yml file in your repository to get started. A workflow is a set of jobs that want to automate. It is triggered by events such as pushing a commit or creating a pull request.
Our workflow will run when code is pushed to any branch of our repository and it’ll have three jobs:
- Run interaction tests and the accessibility audit with Jest
- Run visual and composition tests with Chromatic
- Run user flow tests with Cypress
Note, to run Chromatic, you’ll need the CHROMATIC_PROJECT_TOKEN. You can grab it from the Chromatic manage page and add it to your repository secrets. While the GITHUB_TOKEN is available by default.
Finally, create a new commit, push your changes to GitHub, and you should see your workflow in action!
Cache dependencies
Each job runs independently, which means the CI server has to install dependencies in all three jobs. That slows down the test run. We can cache dependencies and only run yarn install if the lock file changes to avoid that. Let’s update the workflow to include the install-cache job.
We also tweaked the other three jobs to wait for the install-cache job complete. That way they can use the cached dependencies. Push another commit to re-run the workflow.
Success! You’ve automated your testing workflow. When you open up a PR it’ll run Jest, Chromatic and Cypress in parallel and display the results on the PR page itself.
Merge with confidence
The more often you run tests, the fewer bugs you'll have. Research-backed studies from Microsoft suggest that you can see a 20.9% reduction in defects with automated testing.
UI tests act as health checks for an app’s look and feel. They verify the visual appearance, confirm underlying logic, and even detect integration issues. Continuous integration helps you test each commit to reduce bugs with no extra effort from you.
When your tests pass, you’ll have confidence that your UI is bug-free.
📉 20% fewer defects with 0% extra effort
Automation is the secret sauce that allows you to reduce bugs by running checks on each commit.
Get started with our tutorial on automating UI tests with with @github actions.https://t.co/1jXReaUS5d pic.twitter.com/mMy63Uod7Z
— Storybook (@storybookjs) September 16, 2021