Visual Regression Testing for Elements
In Autify, visual regression testing can be performed on elements. You can add the assertion command
Element should look the same in a scenario, which compares the current screenshot of the target element and the reference image created from the past test run. Any kinds of elements on the screen, images or tables or divs, can be selected for the assertion as long as they can fit in the viewport.
When a test with the
Element should look the same steps is run, the test will fail if the latest screenshot and the reference image did not match a pixel-by-pixel comparison. In such a case, review the result to see if the difference was caused by an expected change. If that was an expected change, click "Accept changes." This is all that is required to update the reference image that will be used for the next run!
As of April 2023, Visual Regression Testing for Elements works only on Chrome/Linux environment. When run on other environments,
Element should look the same steps will be skipped.
Perform Visual Regression Testing for Elements
Record a scenario
Start recording from the New Scenario page or Scenario Details page. On Autify Recorder, click on the checkbox (☑️) and select the element on which you want to create an assertion. Then, select
Element Should Look The Same from the assertion commands list.
When you save the scenario, you will see a new
Element Should Look The Same step in your scenario. Note that the step has no thumbnail, but it will have a reference image after the first run.
Run the test twice
Once ready, run the scenario. During the first run, the
Element Should Look The Same step is skipped with the following message as there is no reference image to compare with yet: This step is skipped on the first run. reference image is created, and future tests will compare the element with this reference image.
If you look at the step detail of the scenario, you can review the reference image created by the first run.
Next, run the scenario again to perform the assertion. This time, the current image will be compared with the reference image created during the previous successful test run at the
Element Should Look The Same step!
Open the screenshot of
Element Should Look The Same to see the side-by-side comparison. By clicking the "Diff" tab, you will be able to see the images of the element compared with each other.
Element Should Look The Same step passes when the current screenshot exactly matches the reference image, otherwise, it fails. The following screenshot shows the failure of a
Element Should Look The Same step.
However, there will be times when you will want to accept the change in appearance and use this result as the reference image in the future. That's why we have an "Accept changes" button.
When you click this button, the reference image for this
Element Should Look The Same step will be updated, although it won't affect the status of the test result itself.
The next time you run the test, the new reference image will be used for the comparison. It is easy to change the expected appearance of the element.
Notes on use
Visual Regression Testing for Elements is a feature designed to make assertions against images.
You can also use this feature to make assertions against text, but Autify may detect minute differences that occur due to font rendering, causing the assertion step to fail.
Therefore, if you wish to make assertions against text, we recommend that you use text assertion such as the "Element Text Should Be" command.
If the assertion step fails, please check the screenshot to see if there is a difference.
No locators can be registered for the
Element should look the same step.
We hope you enjoy using our visual regression testing feature, and look forward to your feedback!