The power of visual testing
Learn how to perform visual tests to verify your application’s appearance
When you build software, it’s equally important to test how your application works and how it looks. Traditionally, though, the functional tests that we depend on for verifying application performance are so focused on functionality that they fail to catch rendering errors (these are often known as visual regressions). In fact, functional tests often pass even when a UI is obviously broken.
For example, consider a layout bug that hides your application's login button behind an image. Your functional test would still pass and indicate the button is clickable–because, behaviorally, it still is–even though the login flow has now become inaccessible to your users.
Today, we'll explore how you can resolve these false positives and prevent rendering bugs from reaching production through a pixel-perfect testing workflow known as visual testing. You'll learn how visual testing works, how to perform visual tests manually, and how to automate visual testing within Playwright, Cypress, and Storybook.
How does visual testing work?
Do you remember playing Spot the Difference when you were a child? Fundamentally, visual testing for developers uses the same principles as the game of taking two images and using the first as a baseline for catching changes in the second. Rather than referencing illustrations, developers performing visual tests use a correct implementation of a UI element as their baseline (this could be a design, a screenshot, or a Storybook story). Then, after making code changes, they compare the new UI of whatever they're testing against the previously accepted baseline.
It's very likely that you perform visual tests like these all the time, setting baselines from your memory of how your UI should appear, and then playing your own Spot the Difference to see if code changes affected rendering.
FAQ: Can you perform visual testing manually?
Of course! Manual visual testing is something that most developers do all the time to see if code changes have had a visual impact.
However, if you tried the Spot the Difference game we shared above, you'll realize that visual testing and finding differences between images can be a pretty time- and resource-intensive process. Granted, most UIs are far less busy than the beach scene pictured here, but developers face different challenges that raise the difficulty level. We'll explore these challenges next!
Catching visual regressions at scale
On a case-by-case basis, manual visual testing doesn't feel too exhaustive. If you're only working with a single component, you can quickly perform a visual test to see if that component has changed before or after code changes.
The problem is that developers are experts in complexity, and we never deal with such straightforward implementation. Instead, the frontend codebases we build can contain hundreds of components and thousands of variations, when you consider the fact that each component likely needs to support multiple data states, themes, browsers, viewports, accessibility, and more. Quickly, a single component becomes a score of variations.
The challenge gets worse because of the interconnected nature of application styling, inheritance, and composability. A change in one component can unexpectedly impact other components throughout your UI that depend on it. In turn, changes to those components can cascade into bugs that affect other UI aspects. In the world of cascading styles, it doesn't take much for a snowball to become an avalanche, and unless you're visual testing at high maintenance, it's often your users who end up finding the damage.
You can try to remedy this problem with traditional functional testing methods, like writing tests to assert the classes and styling of your UI components. However, manually asserting styles introduces a different problem altogether, requiring a huge amount of manual upkeep and test writing to cover all possibilities.
Fortunately, you can enable a much better way of catching rendering bugs through the use of specialized automated visual testing tools, whether with an open-source option like Playwright (which provides great visual tests) or an enterprise app like Chromatic.
Using visual testing tools
Visual testing tools enable you to perform automated visual regression tests that catch rendering changes across an entire application UI while requiring minimal maintenance or manual assertions. Rather than having to play Spot the Difference, your role is to review the test results, make fixes where changes were unintended, approve the test, and then ship!
Your first visual test
Chromatic is a visual testing tool that integrates with Storybook, Playwright, and Cypress. Whenever you make code changes, you can run Chromatic tests to capture new snapshots of your UI and compare them against pre-established baselines.
Once your visual test is complete, you can inspect your test results inside Chromatic’s UI Review interface with Chromatic’s diff inspector. The diff inspector highlights any visual changes in neon green, pinpointing all changes, bugs, or unexpected visual regressions. You, or other stakeholders, can also leave comments as part of the review process.
Chromatic's visual tests include coverage of:
- Functionality
- Cross-browser support
- Viewports
- Localization
- Themes
Additionally, all Chromatic visual tests run at the same time at no extra cost, keeping your testing workflow fast. In contrast, many visual testing tools charge per parallelized test. See how Chromatic compares to popular visual testing tools like Percy and Applitools.
When to perform visual tests
You can perform Chromatic's visual tests as an automated CI job, catching visual regressions on any new commit. Chromatic integrates with all major CI/CD tools, including:
- GitHub Actions
- GitLab Pipelines
- Bitbucket Pipelines
- CircleCI
- Travis CI
- Jenkins
- Azure Pipelines
- Other CI providers
Chromatic also integrates with three of the industry’s most popular testing tools: Storybook, Playwright, and Cypress. Here’s a little more about these tools and how to get started.
Visual testing with Storybook
Storybook is an open-source workshop for developing components and pages in isolation. Our team are also Storybook’s core maintainers.
Chromatic integrates with Storybook to enable on demand visual tests of composable UI through Storybook’s Visual Tests addon. This addon lets you perform visual tests without needing to wait on the other checks in your CI pipeline to complete. This workflow allows you to gain confidence in your UI without needing to switch out of development.
Visual testing with Playwright
Playwright is an open-source end-to-end test runner developed by Microsoft.
Chromatic integrates with Playwright to provide specialized visual test coverage within Playwright end-to-end tests. Playwright offers visual testing natively but these visual tests only capture static images of your UI. Chromatic, by contrast, saves full page archives of every test case, which you can view and interact with in the browser.
Visual testing with Cypress
Cypress is a popular end-to-end test runner like Playwright. However, unlike Playwright, it doesn’t offer any built-in visual testing features.
Cypress’s lack of visual coverage makes tools like Chromatic an essential part of end-to-end developer workflows. Chromatic and Cypress integrate fully to equip you with the same visibility of visual bugs as you have for functional bugs.
Get started with Chromatic
Try Chromatic today with Storybook, Playwright, or Cypress without providing any credit card information. You get 5,000 free snapshots per month, with more upgrade options available for higher allowances. Get started with Chromatic.