Tips for selecting elements reliably (Image Recognition)

When Autify searches for an element using image recognition, it locates the element based on the appearance of the area that you selected during recording.

Please keep the following points in mind so that Autify can select elements reliably.

Select a distinctive area for the element

Reliability may be affected if the area you select has nothing displayed around it, or if there are multiple areas that have the same appearance.
You can make tests more reliable by interacting with areas that have an appearance that's distinctive to the target element.

As you can see in the example image below, it's best to tap an area that has distinguishing characteristics and avoid white areas.

Avoid areas where the appearance changes frequently

Reliability may be affected if there are elements close-by whose appearance changes at each test run, such as time, date, or dynamic content.
Whenever possible, choose an area within the target element where the appearance remains consistent.

(Advanced) Element Search Using Detection Pattern

By learning more about how element detection using image recognition works, you'll gain a deeper understanding of how to make tests reliable. Here, you'll learn about the detection patterns that the image recognition locator uses.

A detection pattern is a section of a screenshot that the image recognition locator uses to search for elements. The element search process is as follows.

Tap

1. A detection pattern is created by cropping the tapped area from the screenshot taken during recording.
2. During test runs, Autify searches the screen for a location that matches the detection pattern.

The detection pattern is created dynamically during test runs. You can find the detection pattern used during testing on the Test Results page.

When interacting with an element, tapping a section with visually distinctive features improves the test's reliability. This is because the distinctive features are included in the detection pattern.

If the element search is unreliable because the location you want to interact with lacks a distinctive appearance, or because its appearance frequently changes, you can define the detection pattern manually.

To define the detection pattern manually, go to the Quick Element Update page and select [Manual] under the Adjust Detection pattern section.

In this case, the location you tap can be outside the detection pattern. Even if the appearance of the tap area changes frequently, tapping outside the detection pattern will work as long as there's a stable element nearby.

Swipe to Element

To display the detection pattern during recording, swipe to the target element and select it. Then, adjust the detection pattern to cover all of its features. During test runs, Autify will locate the matching position on the screen and swipe to that position.

For Swipe to Element, the detection pattern must be adjusted manually.

See Swipe to Element for more information.

If element search remains unstable even after adjusting the detection pattern

We recommend using XPath or Accessibility ID locators if the target element's appearance changes frequently, or if the element is not reliably selected even after adjusting the detection pattern.