When Autify searches for an element using image recognition, it locates the element based on the appearance of the area that you selected during recording.
Please keep the following points in mind so that Autify can select elements reliably.
Select a distinctive area for the element
Reliability may be affected if the area you select has nothing displayed around it, or if there are multiple areas that have the same appearance.
You can make tests more reliable by interacting with areas that have an appearance that's distinctive to the target element.
As you can see in the example image below, it's best to tap an area that has distinguishing characteristics and avoid white areas.
Avoid areas where the appearance changes frequently
Reliability may be affected if there are elements close-by whose appearance changes at each test run, such as time, date, or dynamic content.
Whenever possible, choose an area within the target element where the appearance remains consistent.
(Advanced) Element Search Using Detection Pattern
By learning more about how element detection using image recognition works, you'll gain a deeper understanding of how to make tests reliable. Here, you'll learn about the detection patterns that the image recognition locator uses.
A detection pattern is a section of a screenshot that the image recognition locator uses to search for elements. The element search process is as follows.
Tap
- A detection pattern is created by cropping the tapped area from the screenshot taken during recording.
- During test runs, Autify searches the screen for a location that matches the detection pattern.
The detection pattern is created dynamically during test runs. You can find the detection pattern used during testing on the Test Results page.
When interacting with an element, tapping a section with visually distinctive features improves the test's reliability. This is because the distinctive features are included in the detection pattern.
If the element search is unreliable because the location you want to interact with lacks a distinctive appearance, or because its appearance frequently changes, you can define the detection pattern manually.
To define the detection pattern manually, go to the Quick Element Update page and select [Manual] under the Adjust Detection pattern section.
In this case, the location you tap can be outside the detection pattern. Even if the appearance of the tap area changes frequently, tapping outside the detection pattern will work as long as there's a stable element nearby.
💡Tips
For example, in the image below, suppose you want to tap the 「Check Availability・Reserve」 button element. In this case, since the elements above the target may change, setting the detection pattern to include the area in the pink frame (as shown in the example below) might lead to failure. To ensure stability, it is recommended to specify a range that only includes the part of the button element where the appearance does not change (as shown in the example below: blue frame).
Swipe to Element
To display the detection pattern during recording, swipe to the target element and select it. Then, adjust the detection pattern to cover all of its features. During test runs, Autify will locate the matching position on the screen and swipe to that position.
For Swipe to Element, the detection pattern must be adjusted manually.
See Swipe to Element for more information.
If element search remains unstable even after adjusting the detection pattern
We recommend using XPath or Accessibility ID locators if the target element's appearance changes frequently, or if the element is not reliably selected even after adjusting the detection pattern.
💡Tips
For example, suppose the target element is 「EAST NIHONBASHI：SANDWICH」. Although the text 「EAST NIHONBASHI：SANDWICH」 remains the same, if the application applies a mechanism where the text wraps depending on the width of the device being used, the visual appearance will change. In such cases, if you are using an image recognition locator, it may result in a failure. For these scenarios, we recommend using an XPath locator.
