Bitbucket Pipelines integration

Autify provides a way to run a test plan or test scenario easily from your Bitbucket Pipelines with Autify CLI.

Getting started

This document describes how to use our Autify CLI in your Bitbucket Pipelines step-by-step.

Prerequisites

First, we assume you already have Bitbucket Pipelines to deploy your software to somewhere e.g. staging, production. We'll add an Autify test step right after the deployment step to confirm the newly deployed software doesn't break the end-to-end experience.

Secondly, you need to create a test plan or test scenario on Autify for Web. The test scenario should be recorded against your target website where the software is deployed by your Bitbucket Pipelines. Please note the URL of your test plan or scenario e.g. https://app.autify.com/projects/00/scenarios/000 and we'll use it later.

You also need to create a personal access token for Autify for Web. Please note the generated token somewhere so that we can store the value on Bitbucket Pipelines Variables later.

Note: Since the personal access token is associated with a single user, we recommend you to create a machine user in your organization and use its personal access token for CI/CD integration.

Store personal access token on Bitbucket Pipelines Variables

Store the personal access token you generated above on Variables by following Bitbucket's documentation. You should check "Secured" to avoid exposing the token value in the log just in case. You can use an arbitrary key for the variable but if you name it AUTIFY_WEB_ACCESS_TOKEN, you don't have to specify the key later because this is the default variable read by Autify CLI. Assuming you named so here.

Add step to run Autify for Web

Finally, let's add a new step to your existing Bitbucket Pipelines. See the examples below. The URL value (https://app.autify.com/projects/00/scenarios/000) should be the URL you noted above.

See more details of parameters in the Autify CLI documention: https://help.autify.com/docs/autify-command-line-interface

Simply start a test and finish the step (no waiting for the finish of the test):

- step:
      script:
        - curl https://autify-cli-assets.s3.amazonaws.com/autify-cli/channels/stable/install-standalone.sh | sh
        - autify web test run https://app.autify.com/projects/00/scenarios/000

Start a test and wait until the test finishes or timed out:

- step:
      script:
        - curl https://autify-cli-assets.s3.amazonaws.com/autify-cli/channels/stable/install-standalone.sh | sh
        - autify web test run https://app.autify.com/projects/00/scenarios/000 --wait --timeout 300

CAVEAT: This will consume your Bitbucket Pipelines' minutes while waiting. Be careful when extending the timeout value.

Use --url-replacements to start your test plan or scenario against a different URL:

- step:
      script:
        - curl https://autify-cli-assets.s3.amazonaws.com/autify-cli/channels/stable/install-standalone.sh | sh
        - autify web test run https://app.autify.com/projects/00/scenarios/000 --url-replacements "https://example.com https://example.net"

--url-replacements is useful when you want to test your staging website by a test scenaio recorded against your production website, for example.

Conclusion

By using Autify CLI, you can automatically run a test plan or scenario after deployments so that you can detect regressions right away and take an appropriate actions without wasting time.