Why should I keep the number of steps in a test scenario small?

Overview

Keeping your test scenarios short is recommended for most if not all cases. This article describes the benefits of short test scenarios as well as how you can reduce the size of your scenarios.

Here are the benefits of shorter test scenarios:

• Shorter test scenarios are less complex and easier to read/understand.
• Replay doesn't take as much time. You can re-record or fine-tune the steps faster which reduces maintenance costs.
• Test execution time will be shorter. You can retry your test scenario after you update or fix an issue and see the result quickly.
• You can isolate issues more easily. If one test scenario fails due to a bug, you can still check the other parts of your product are healthy. As a result, your test scenario sets will stay stable.

Conversely, the longer your test scenario becomes, the more unstable (or "flaky") it will tend to be.

How to reduce the size of your test

1. List your criteria to check

Say you are trying to test a blog service. Your list might look something like this:

1. The blog service logo displays at the top of the page
2. Create a new user
3. Log in with an existing user account
4. Create a blog post
5. Edit a blog post
6. Delete a blog post
7. "Like" (or "upvote") a blog post
8. Undo "like" on a blog post
9. Log out after logging in
10. Delete a user

Each item on the list should equate to a part of your test scenario.

2. Divide your list into small groups

The smaller the better. If you can create a group with only one list item, please do so.

Or you can divide them into categories, such as user-related items and blog post-related items.

Some of the items in your list might rely on the results of a previous step. For example, to delete a user, you have to create one beforehand. You can put those list items into one group if you notice any such dependencies.

One of the advantages of E2E (end-to-end) testing is that you can test the functionalities that cannot be independent of each other or should be tested together at once. However, keep in mind that you want to minimize such dependencies. If you introduce a lot of dependencies in a test scenario, a single unexpected error can destroy everything. We recommend you to have fixtures instead of having a long preparation sequence in a test scenario.

Here is an example of how you can group your list.

• Scenario #1: Logo
  • The blog service logo displays at the top of the page.
• Scenario #2: User creation and deletion
  • Create a new user.
  • Delete a user.
• Scenario #3: Log in and log out
  • Log in with an existing user account.
  • Log out after logging in.
• Scenario #4: Blog post creation and deletion
  • Create a blog post.
  • Delete a blog post.
• Scenario #5: Blog post edit
  • Edit a blog post.
• Scenario #6: "Like" a blog post and cancel it.
  • "Like" (or "upvote") a blog post.
  • Undo "like" on a blog post

Afterward, create test scenarios for each group.

3. Use Step Groups

You may notice that you have to log in before you check something in most of the test scenarios. To avoid having to manually add those steps to every scenario that requires login, you can put the login sequence into a step group (a reusable component) and insert the step group where needed instead.

Although using step groups does not reduce the total number of steps, doing so reduces the redundancy of your test scenarios and makes them easier to maintain.

Note that you should aim to keep the step group small as well. For a login sequence step group, avoid including any steps irrelevant to the login action. For example, if you want to assert some UI components in the login page itself before you actually log in to the website, that should be done in another test scenario.

4. Prepare fixtures

The word "fixture" refers to a set of prerequisites that you need before executing the tests, including the test data you prepare before the test executions. Examples include having a sample user to test login/logout actions, or a sample blog post that you can use to test edit actions.

You may be thinking that you can just set up test data within the test scenario. But what if the development team updates the UI of the signup pages where user data is created? The UI updates can make "Scenario #2: User creation and deletion" unstable, but if you already have sample user data ready before test execution "Scenario #3: Log in and log out" can still run stably without being affected by the result of the other scenario. In short, you can increase the stability of your test scenarios by having consistent data.

Manually creating such fixture data is often the most simple approach. Alternatively, if you are generating test data automatically in the build/deployment process, you can leverage our CI/CD feature to execute tests more efficiently.