Test-driven Frontend — Design Testing

We develop user interfaces for humans, not for robots. Which is why it's hard to remove manual testing from our applications and also not the overall goal. But there are some tools that can catch certain mistakes before wasting a persons time clicking through your application.

In the last part of test-driven frontend, we will cover visual regression testing, snapshots and semantic testing.

Part1 — Testing strategies
Part 2 — How to set up a unit testing Environment
Part 3 — Accessibility testing
Part 4 — Functional and behaviour testing

Visual regression test

AI might not be able to tell if you implemented the designer's idea right, but it can help to notice differences. Especially in places where we might not expect them. In a living design system, you are reusing smaller components to create complex ones, and those will be influenced too when parts are changed. If your design system is already evolved and holds many complex components it might take a lot of time to check the appearance of everything after you changed one component. In this case, a visual screenshot comparison can save you a lot of time.

We already installed some nice addons which will help us developing our storybook project. One other great addon for design testing purposes comes from percy.io. Percy is a cloud-based visual testing service. It offers a free plan for smaller projects.

Let’s start the code-along by installing and configuring Percy. You will need to register yourself at percy.io in order to use the addon

yarn add @percy/storybook --dev

For generating the screen snapshots we can create a new npm script command

"snapshot": "build-storybook && percy-storybook --widths=320,1024"

We can customize the command e.g. setting a min-height or set different breakpoints at the widths option. Keep in mind that for every breakpoint a new set of screenshots will be made, this has an influence on how fast your plan will be used up. Same for cross-browser testing, percy currently has Firefox and Chrome enabled.

To run the new npm script we just have written, you also have to add your personal percy-token as environment variable.

To get the PERCY_TOLEN you have to register to any plan on percy and then set up the project, where it will be displayed.

# Windows
set PERCY_TOKEN=<your_token>

# Unix / Mac
export PERCY_TOKEN=<your_token>

To generate the initial set of snapshots run:

yarn run snapshot

The script will now generate a build of storybook and create a visual snapshot of every story. It will then post in your terminal the URL where you can see the generated snapshots e.g.

Uploading 4 snapshots to Percy.
Percy snapshots uploaded. Visual diffs are now processing: https://percy.io/YOUR_ACC/designsystem-lib/builds/BUILDNUMBER

This is how the generated initial comparison looks like

To automate the generation and integrate it into your workflow you now can configure your CI to execute the snapshot command on every merge-request and integrate it into your git approval workflow.

Semantic Regression

The simplest unit-test to setup with jest is a snapshot test, which just compares the HTML output between your versions. It gives you a high test coverage in your coverage report and so a lot of people are tempted to throw a snapshot test at every component. But in the nature of this test, you will probably need to update these snapshot almost every time you make a change to your component. Even more often if you are using a CSS-in-JS approach like StyledComponents, where it generates you every time a new class name.

I recommend to only add a snapshot test for components which rely on props from other components. Or if your markup is changing based on the components state. In our counter component, we want to use the state to change the CSS classes, to ensure that further changes do not break this behaviour, we can use a snapshot test in combination with other assertions

So far we are using shallow rendering from enzyme. To use this renderer for snapshot matching it needs to be serialized into Json with the toJson function from enzyme-to-json npm package.

yarn add @enzyme-to-json --dev

To cover not only a regression we can combine the snapshot test with a classical assertion on the correct class-name we expect in each case.

Generate snapshot from shallow rendered component.

Let's run the test and have a look at the saved snapshot.

On the initial run, the test fails because of the second assertion, but it will not fail on the snapshot as it has nothing to compare it with. So a snapshot test is not helping us to determine the initial correctness. The second test for the correct written class name is. But neither of them can tell us that the CSS associated with these classes is correct. But we clearly see in the percy screenshot that there is no style applied yet.

To fix the test we implement the functionality — on state updates we do not only call the callback function but also a new functionchangeAppearance() in which we set a new state variable classMod, based on the current amount. This new variable will be used to add a second counter class counter--${classMod} . For an easier selection, also add a new data-attribute to the counter div.

If we now run the tests again it will still fail, because of the snapshots. Another point that shows — snapshot tests are not here for telling you whether your code is correct or not. Instead, it informs you about a change, and it's up to you to inspect them closely.

Snapshots failing

The Snapshot shows you the difference for inspection

So far we only changed functionality, not the actual visual appearance. If we run percy again it will regenerate all snapshots but not detect any differences.

Visual Changes

Finally, we add some CSS to our component and extend the story in storybook to showcase all three different states.

Storybook showcases three different states, accessibility passes

Rerun percy again. Like the semantic snapshots, percy can’t tell you if something is right or wrong, it is only able to show you what it saw and notify you about the differences. But if you have a higher number of components that are connected with each other, it will help you detect mistakes faster.

percy compare view marks everything that's different

Summary

Automated testing is not able to solve all of your problems, but they will give you more confidence about the code you write and the changes you make. Reading tests can already help to understand a feature or a certain piece of code for new members of your team.

Now we are at the end of the test-driven frontend series. I hope you learned something new and will find some of the presented tools helpful. These are of course not all possible tools and ways to test, but a set I found helpful to me in the past.