Azure Pipelines integration

Autify provides an official way to integrate with Azure Pipelines to run a test plan or test scenario easily from your Azure Pipelines workflow.

Getting started

This document describes how to use our Azure Pipelines integrations with your Git repository step-by-step.

Prerequisites

First, we assume you already have an Azure Pipelines workflow to deploy your software to somewhere e.g. staging, production. We'll add an Autify test step right after the deployment step to confirm the newly deployed software doesn't break the end-to-end experience.

Secondly, you need to create a test plan or test scenario on Autify for Web. The test scenario should be recorded against your target website where the software is deployed by your Azure Pipelines workflow. Please note the URL of your test plan or scenario e.g. https://app.autify.com/projects/00/scenarios/000 and we'll use it later.

You also need to create a personal access token for Autify for Web. Please note the generated token somewhere so that we can store the value on Azure Pipelines variables later.

Note: Since the personal access token is associated with a single user, we recommend you to create a machine user in your organization and use its personal access token for CI/CD integration.

Store the personal access token on Azure Pipelines variables

Store the personal access token you generated above on secret variables by following Azure's documentation.

You can use an arbitrary name for the secret variable but that name will be referred by your Azure Pipelines definition later. The following step will assume you named it AUTIFY_WEB_ACCESS_TOKEN.

Add Azure Pipelines step to run Autify for Web

Finally, let's add a new step to your existing Azure Pipelines workflow. Open your azure-pipelines.yml and see the examples below. The URL value (https://app.autify.com/projects/00/scenarios/000) should be the URL you noted above.

Simply start a test and finish the step (no waiting for the finish of the test):

- script: curl https://autify-cli-assets.s3.amazonaws.com/autify-cli/channels/stable/install-standalone.sh | sh
  displayName: 'Install Autify CLI'

- script: autify web test run https://app.autify.com/projects/00/scenarios/000
  displayName: 'Run test on Autify for Web'
  env:
    AUTIFY_WEB_ACCESS_TOKEN: $(AUTIFY_WEB_ACCESS_TOKEN)

Start a test and wait until the test finishes or timed out:

- script: curl https://autify-cli-assets.s3.amazonaws.com/autify-cli/channels/stable/install-standalone.sh | sh
  displayName: 'Install Autify CLI'

- script: autify web test run https://app.autify.com/projects/00/scenarios/000 --wait
  displayName: 'Run test on Autify for Web'
  env:
    AUTIFY_WEB_ACCESS_TOKEN: $(AUTIFY_WEB_ACCESS_TOKEN)

CAVEAT: This will consume your Azure Pipelines runner's minutes while waiting. Be careful when extending the timeout value.

Use --url-replacements to start your test plan or scenario against a different URL:

- script: curl https://autify-cli-assets.s3.amazonaws.com/autify-cli/channels/stable/install-standalone.sh | sh
  displayName: 'Install Autify CLI'

- script: autify web test run https://app.autify.com/projects/00/scenarios/000 --url-replacements http://example.com=http://example.net
  displayName: 'Run test on Autify for Web'
  env:
    AUTIFY_WEB_ACCESS_TOKEN: $(AUTIFY_WEB_ACCESS_TOKEN)

--url-replacements is useful when you want to test your staging website using a test scenario recorded against your production website, for example.

Conclusion

By using Autify's Azure Pipelines integration, you can automatically run a test plan or scenario after deployments so that you can detect regressions right away and take appropriate actions without wasting time.