GitLab CI/CD integration

Autify provides an official way to integrate with GitLab CI/CD to run a test plan or test scenario easily from your GitLab CI/CD workflow.

Getting started

This document describes how to use our GitLab CI/CD integrations with your GitLab repository step-by-step.

Prerequisites

First, we assume you already have a GitLab CI/CD workflow to deploy your software to somewhere e.g. staging, production. We'll add an Autify test job right after the deployment job to confirm the newly deployed software doesn't break the end-to-end experience.

Secondly, you need to create a test plan or test scenario on Autify for Web. The test scenario should be recorded against your target website where the software is deployed by your GitLab CI/CD workflow. Please note the URL of your test plan or scenario e.g. https://app.autify.com/projects/00/scenarios/000 and we'll use it later.

You also need to create a personal access token for Autify for Web. Please note the generated token somewhere so that we can store the value on GitLab CI/CD variables later.

Note: Since the personal access token is associated with a single user, we recommend you to create a machine user in your organization and use its personal access token for CI/CD integration.

Store personal access token on GitLab CI/CD variables

Store the personal access token you generated above on Variables as Custom CI/CD variables by following GitLab's document. (You can store to a project or a group.) If you want to run the Autify job to non-protected branches/tags, uncheck "Protect variable". You should check "Mask variable" to avoid exposing the token value in the log just in case.

Do not store this value in .gitlab-ci.yml directly since it will be visible as plain text.

You can use an arbitrary key for the variable but if you name it AUTIFY_WEB_ACCESS_TOKEN, you don't have to specify the key later because this is the default variable read by Autify CLI. Assuming you named so here.

Add GitLab CI/CD job to run Autify for Web

Finally, let's add a new job to your existing GitLab CI/CD workflow. Open your .gitlab-ci.yml and see the examples below. The URL value (https://app.autify.com/projects/00/scenarios/000) should be the URL you noted above.

Simply start a test and finish the step (no waiting for the finish of the test):

autify-test-job:
  stage: <YOUR_STAGE>
  before_script:
    - curl https://autify-cli-assets.s3.amazonaws.com/autify-cli/channels/stable/install-standalone.sh | sh
  script:
    - autify web test run https://app.autify.com/projects/00/scenarios/000

Start a test and wait until the test finishes or timed out:

autify-test-job:
  stage: <YOUR_STAGE>
  before_script:
    - curl https://autify-cli-assets.s3.amazonaws.com/autify-cli/channels/stable/install-standalone.sh | sh
  script:
    - autify web test run https://app.autify.com/projects/00/scenarios/000 --wait --timeout 300

CAVEAT: This will consume your GitLab CI/CD minutes while waiting. Be careful when extending the timeout value.

Use --url-replacements to start your test plan or scenario against a different URL:

autify-test-job:
  stage: <YOUR_STAGE>
  before_script:
    - curl https://autify-cli-assets.s3.amazonaws.com/autify-cli/channels/stable/install-standalone.sh | sh
  script:
    - autify web test run https://app.autify.com/projects/00/scenarios/000 --url-replacements http://example.com=http://example.net

--url-replacements is useful when you want to test your staging website using a test scenaio recorded against your production website, for example.

Conclusion

By using Autify's GitLab CI/CD integration, you can automatically run a test plan or scenario after deployments so that you can detect regressions right away and take appropriate actions without wasting time.