Visual Regression Testing for Elements
  • 14 Mar 2024
  • 3 Minutes to read
  • Contributors
  • Dark
    Light

Visual Regression Testing for Elements

  • Dark
    Light

Article Summary

Overview

In Autify, visual regression testing can be performed on elements. You can add the assertion command Element should look the same in a scenario, which compares the current screenshot of the target element and the reference image created from the past test run. Any kinds of elements on the screen, images or tables or divs, can be selected for the assertion as long as they can fit in the viewport.

When a test with the Element should look the same steps is run, the test will fail if the latest screenshot and the reference image did not match a pixel-by-pixel comparison. In such a case, review the result to see if the difference was caused by an expected change. If that was an expected change, click "Accept changes." This is all that is required to update the reference image that will be used for the next run!

Note

Visual Regression Testing for Elements works only on Chrome/Linux environment. When run on other environments, Element should look the same steps will be skipped.

Perform Visual Regression Testing for Elements

Record a scenario

Start recording from the New Scenario page or Scenario Details page. On Autify Recorder, click on the checkbox (☑️) and select the element on which you want to create an assertion. Then, select Element Should Look The Same from the assertion commands list.

Screen Shot 2023-04-25 at 8.37.39.png

When you save the scenario, you will see a new Element Should Look The Same step in your scenario. Note that the step has no thumbnail, but it will have a reference image after the first run.

Screen Shot 2023-04-11 at 15.03.44.png

Run the test twice

Once ready, run the scenario. During the first run, the Element Should Look The Same step is skipped with the following message as there is no reference image to compare with yet: This step is skipped on the first run. reference image is created, and future tests will compare the element with this reference image.
Screen Shot 2023-04-11 at 15.09.38.png

If you look at the step detail of the scenario, you can review the reference image created by the first run.

Screen Shot 2023-04-11 at 15.41.22.png

Next, run the scenario again to perform the assertion. This time, the current image will be compared with the reference image created during the previous successful test run at the Element Should Look The Same step!

Screen Shot 2023-04-11 at 15.33.00.png

Open the screenshot of Element Should Look The Same to see the side-by-side comparison. By clicking the "Diff" tab, you will be able to see the images of the element compared with each other.

Screen Shot 2023-04-23 at 21.43.15.png

Screen Shot 2023-04-23 at 21.42.13.png

Accept changes

The Element Should Look The Same step passes when the current screenshot exactly matches the reference image, otherwise, it fails. The following screenshot shows the failure of a Element Should Look The Same step.

Screen Shot 2023-04-22 at 6.47.08.png

However, there will be times when you will want to accept the change in appearance and use this result as the reference image in the future. That's why we have an "Accept changes" button.

Screen Shot 2023-04-22 at 6.47.31.png

When you click this button, the reference image for this Element Should Look The Same step will be updated, although it won't affect the status of the test result itself.

Screen Shot 2023-04-22 at 6.49.37.png

The next time you run the test, the new reference image will be used for the comparison. It is easy to change the expected appearance of the element.

Screen Shot 2023-04-22 at 6.57.46.png

Notes on use

Make assertions against text

You can use this feature to make assertions against text, but Autify may detect minute differences that occur due to font rendering, causing the assertion step to fail.
Therefore, if you wish to make assertions against text, we recommend that you use text assertion such as the "Element Text Should Be" command.

Make assertions against colors

While you can use this feature to make assertions against colors, Autify might miss changes in colors depending on the type of color change, resulting in a successful step even when there is a change. We recommend using JS steps for color assertions.

Locator

No locators can be registered for the Element should look the same step.

We hope you enjoy using our visual regression testing feature, and look forward to your feedback!


Was this article helpful?

Changing your password will log you out immediately. Use the new password to log back in.
First name must have atleast 2 characters. Numbers and special characters are not allowed.
Last name must have atleast 1 characters. Numbers and special characters are not allowed.
Enter a valid email
Enter a valid password
Your profile has been successfully updated.