How to speed up test execution?

This page describes the mechanisms related to execution time (the time from the start to the end of a test).

Test Execution Time and Mechanisms for Finding Elements

As described in this article, if Autify cannot find the element that exactly matches the element during recording or the element in the last successful test run (“exact match”), AI searches for an alternative element. When this happens, there is an idle time where Autify waits for an exact match to appear, and if it doesn’t appear during that time, the AI will begin searching.

Let us explain how Autify waits for a certain period of time.

As mentioned above, Autify waits for a certain period of time if the target element is different from when the scenario was recorded or when the test was last passed. If you execute the scenario for the first time and Autify doesn’t find an exact match at a certain step, it will wait 20 seconds (the default wait time*) for the element to appear. Then, if the test passes multiple times and Autify determines that it doesn’t have to wait for the element to appear, the wait time for that step is gradually reduced.

Since the AI works on JS Step for elements, the execution time will be reduced if the same conditions apply.

At first, Autify waits for the exact match and carefully looks for a similar element, but if the element is found reliably, you can expect the execution time to become shorter as the wait time is shortened.

In general, the element search feature should help gradually reduce execution time. However, if Autify can’t find the correct element or if a certain step continues to take time, you may be able to shorten the execution time using the methods described below.

*You can set the default waiting time for each Workspace. (For details, see Shorten the AI waiting time below.)

Ways to shorten execution time and their disadvantages

Thanks to the mechanism described above, it is usually best to just let Autify gradually optimize execution time. However, there are ways to deliberately shorten the execution time if needed. Each of these methods have different effects that you need to consider before implementing.

Add a locator to steps that are taking longer than 20 seconds

Adding a locator can save time by allowing you to select specific elements without relying on Autify's AI.

Note that it will disable the AI's search function on the step to which the locator is added, so if changes are made to the selector of the target element due to site modifications or anything else, the element will not be found and the step will fail.

For more information on the locator function, please refer to the following document.

• Add a locator to specify an element

Shorten the AI waiting time (default 20 seconds)

Note that for screens that originally take a long time to load, the test may fail because the steps are executed while the screen has not completed loading.

If the target application does not include asynchronous transitions, this is not a problem, but it is not recommended for applications with asynchronous transitions. Note that even if the AI wait time is changed from the default 20 seconds, the wait time for the target step may be automatically shortened if the test execution is judged to be stable.

Example: If the AI wait time is changed from the default 20 seconds to 10 seconds, the AI wait time may be reduced to 5 seconds for the target step if the test succeeds continuously (a shorter time is adopted).

If you would like to use this method, please contact our support team with your desired wait time (in seconds) and we will take care of this for you.

Processes that cannot be shortened

Environment start up

In Autify Cloud environment, an environment for test execution is created at the start of each run. It takes a certain amount of time to secure the environment, and as a result, there may be a wait time before the test execution actually starts.

This is integrated as part of the test execution and therefore displayed as "Running".

Step 1 execution

Step 1 is the initial access of the url to be tested and there are cases where it can take a long time to load.

For instance, some third-party Javascript libraries used in your web application may take a long time to load the first time for some reason and in extreme cases it may take more than 5 minutes to display.

This also includes the time it takes for Autify's test execution engine to start, so it may take longer than if you were to display the page manually.

URL transition steps

Similar to the "Execution Time for Step 1," it may take time for the new page to load.

Test plan post-processing

When executing a test plan, there are post-processes after the execution of each test scenario. Therefore, the total test execution time for each scenario may not necessarily match the test plan execution time.

Screenshot

A screenshot is taken at each step and it is included in the total time. It cannot be shortened.

If the test execution still takes a long time even after implementing the methods described in this article, or if you have any questions, please feel free to contact our support team.