Features to help troubleshoot issues in test results

On this page, you will learn about Autify's features that may be helpful when troubleshooting.

Error messages

When a test fails, the failed step and its error message are displayed at the top of the test results. This error message gives you important information for identifying why the test failed.

For common errors and how to solve them, please see Troubleshooting - Common Errors.

Screenshots

Autify takes a screenshot at each step, which you can view in the test results. If an error occurs, checking these screenshots may help identify where the problem occurred. You can view the full screenshot for failed steps.

The cause of the error may be in an earlier step, not the step where the error occurred. Therefore, make sure to carefully examine the screenshots of not only the failed step but also the steps before it.

Browser Log

The Browser Log Feature is available for tests run in the Chrome environment (excluding tests run on a real Android device).

If a test that keeps failing, or if a test that usually passes suddenly fails, check for error logs in the browser log of the failed step and previous steps. There may be a log that could have caused the error.

One common situation is where a request to the API of the Application Under Test failed. If you find a log that doesn't look right, share it with your development team to help find the cause of the error.

Video Log

Information that wasn't captured in screenshots may be found in the Video Log.

If you can't find the cause of the issue in the screenshots or browser logs, check the video log of the failed step and previous steps to see if there are any clues for why the step failed.

Run the test in a different environment

There are execution environments where browser logs and video logs are unavailable. If a test fails with an unknown error in those execution environments, try running the test in an environment where those features are available, such as Chrome / Linux. You may be able to identify the cause of the error from the browser log or video log.

Also, testing on a real device takes longer to execute compared to the Autify Cloud Environment. You can shorten the time spent on fixing flaky tests by running tests in the Autify Cloud Environment until they become reliable, and then running them in a real device.

Rerun

If a test fails with an Internal Error, you can often resolve it by re-running the test.

Even when it's not an internal error, re-running the test can be an effective way to troubleshoot, since tests can fail due to the application under test having a temporary API error.

Local Replay

If a test fails unexpectedly, it's important to check whether the error occurs even without Autify. Use Local Replay to navigate to the screen where the problem occurred and check whether the problem occurs on your local PC.