UI tests capture a visual snapshot of every story in a cloud browser environment. Whenever you push code, Chromatic generates a new set of snapshots and compares them against baselines. If there are visual changes, you verify if they’re intentional.
Enable visual tests for your project on the manage screen. All snapshots are taken in Chrome by default. This is also where you expand test coverage to additional browsers.
Once visual tests are enabled, you can establish baselines by running a Chromatic build in a new project or on a branch without an ancestor. This captures a snapshot of each story in a cloud browser and sets it as the baseline. Subsequent builds will generate new snapshots that are compared against existing baselines to detect UI changes.
View changes between baselines
Each build Chromatic compares new snapshots to existing baselines from previous builds. The list of changes are shown on the build page in the web app. The build will be marked “unreviewed” and the changes will be listed in the “Tests” table.
What about component errors?
When a story fails to render it will be badged with “Component Error”. You will not be able to “pass” a build that has component errors. Fix story errors in Storybook and run tests again.
Verify UI changes
Chromatic detects UI changes but it’s still up to you to verify if changes are intentional. For intentional changes, you need to update the baseline so future tests will be compared to the latest baseline for the story. If a change is unintentional it needs to be fixed.
✅Accept change: This updates the story baseline. When a snapshot is accepted it won’t need to be re-accepted until it changes, even through git branches or merges.
❌Deny change: This marks the change as “denied” indicating a regression and immediately fails the build. You can deny multiple changes per build.
What about baselines in other branches?
Chromatic automatically changes the baseline snapshots that it uses for each build depending on your branch. Each branch has a separate set of baselines. This means you can update UI components on multiple feature branches in parallel without conflicts. Learn more »
How do I reproduce the snapshot?
Sometimes you need a closer look to determine why a snapshot is rendering as it does. Along with pixel and DOM diffs, Chromatic displays the interactive stories just as they appear in Storybook. Click on the snapshot image to go to the component screen. Then click on the “Canvas” tab to view the fully interactive story.
How are changes on builds different from those listed on the PR Screen 'Changeset' tab?
UI tests (shown on the build screen) detect changes between builds, specifically, between the last accepted baseline and the latest build. This is useful for detecting defects during the development process and when merging to master to ship.
In contrast, the PR screen shows the changeset between the latest commit on the PR branch (head) and the ‘merge base’ (base). Think of it like code review, but for UI.
If you accept all the changes, the build will 🟢Pass. Future builds whose stories have the same appearance will pass.
If you deny any of the changes, the build will 🔴Fail. You will need to make code changes (and thus start a new build) to get the build to pass.
When your build is passed (all changes accepted), you’re ready to merge visual changes with confidence knowing that your UI is bug free. Chromatic will update the PR check for “UI Tests” to reflect the build status.
After you merge your code, Chromatic will also apply accepted baselines to stories on the target branch. That means you’ll only need to accept baselines a single time.
PR check for “UI Tests”
Chromatic adds a ‘UI Tests’ check within the status checks for your pull/merge requests. The badge shows errors or changes that need to be reviewed. Require the check in GitHub, GitLab, or Bitbucket to prevent accidental UI bugs from being merged.
Next: Learn about UI Review
Now that you can catch bugs during development, learn about how to invite stakeholders into Chromatic’s UI Review workflow to get the team sign off before merging.
Verify UI changes faster using keyboard shortcuts. Protip: Pressing 1 multiple times switches between the baseline and new snapshot in the 1up view.
It’s essential that your components and stories render in a consistent fashion between tests to prevent false positives. Some reasons your stories might not render consistently and ways you can avoid this include:
Randomness in stories: It’s not uncommon to use random number generators to generate data for complex component inputs. To avoid this, you can hard-code the input data, but often a more convenient solution is to use a tool like seedrandom which you can use to make your “random” number generator consistent.
Use of the current date/time: Dates and times are a testers bane! To get consistency in components or tests that use the current time, you can use a tool to also “seed” the time, like timemachine for the
Animations and video: Chromatic will attempt to pause all animations and video. However, you may need to configure Chromatic’s exact behaviour.
Unpredictable resource hosts: Resources that load from unpredictable or flaky sources may not load in time (15s) to capture. Workaround this by serving resources as static files in Storybook or using a placeholder service. Learn more about how we load resources.
Iframes rendering out of the viewport: Some browsers only visually render iframes when they are inside of the viewport, despite the fact that they have loaded with all of their resources. For this reason, if you have an iframe that is placed below the viewport of a tall story, it will appear blank. You may want to ignore that element and also test it in isolation so that it fits inside of the viewport.
Skip stories: Some stories may render unpredictably intentionally. If this is the case you may want to ignore the story from testing and move on.
Introduce a delay: As a last resort, you can try adding a delay. This will delay Chromatic’s snapshot for a specified amount of time. The trouble with this technique whilst it may make the problem less common, it may not eliminate it completely.
If you want to utilize the above techniques to ensure consistency for Chromatic’s tests, but you still want random elements for your local Storybook, you can use
isChromatic() exported from our package to check for the Chromatic environment.
Build failed because of component errors
Reviewing tests disabled on build screen
If a build isn’t the newest build on a branch, we disable reviewing the build; as any future builds will base themselves on the newest build, making approvals to this build pointless.
Note that in the case that there is a descendent build of this build on a different branch (for instance if the commit for this build was merged into that different branch), we do allow reviewing of this build. Future builds on this branch will use approved changes from the build; however future builds on the different branch will not—for this reason it is best to review builds before merging them.
"Didn't find any commits in this git repository in the last 100 builds."
This means that across the last 100 unique commits across all builds in your app, we didn’t find a single one that exists in the repository you ran this build against. Commits can go missing if you rebase or perform squash-merges, however, if all of the previous 100 builds’ commits are missing, it is likely something has gone wrong.
If you’ve reached this situation and can’t work out why, please let us know.
"Failed to find common ancestors with most recent builds within 1000 commits"
This means that although we found recent builds that were in your git repository history (see above), we couldn’t find any common history between your checked out build and those builds within 1000 commits.
Unless you are doing something unusual with your git repository, this is extremely unlikely. Either way, please let us know.
"Build X is based on a commit without ancestor builds."
When we create a build, we search your git history for a recent Chromatic build based on a commit that is an ancestor (i.e. a commit that is in the direct history of this commit). Unless this is the very first build, if we do not find one, we will show you this message.
This is typically unusual, because in order to run Chromatic on a commit, chances are the commit that added Chromatic to your app is an ancestor!
However, this situation can arise due to the following:
You switched branches and re-ran Chromatic, without checking-in the code changes that installed Chromatic. In this case you can safely ignore this message.
You rewrote history in merging the Chromatic installation code (e.g. using GitHub’s “Squash and Merge” or “Rebase and Merge” buttons). Learn how to resolve
You are using a shallow clone of your repository when running Chromatic. Chromatic needs access to your full git history in order to find baselines (or at least the history until the previous Chromatic build, which depends on how often you push code/run builds). Learn about how we use Git for baselines »
Something else happened—perhaps a bug at our end? Please let us know if this is the case.
Image size too large
We have a 25 million pixel size limit for image snapshots. This ensures fast and reliable performance for every build.
If your stories are larger than this, perhaps something has gone wrong? Let us know if you need this limit increased by chat or email.
Can I disable UI Tests if I prefer not to use them?
Yes. Go to the manage page for your project where you can disable UI Tests. Chromatic will no longer add status checks to your PRs for UI Tests once it is disabled.