Autify Command Line Interface (CLI)

Autify Command Line Interface (CLI) is a software application used to communicate with Autify services via text based command line. You can interact with Autify without opening a browser so that automated integrations such as CI/CD integration can easily be done by using Autify CLI.

This document explains about Autify CLI from the basics to a comprehensive list of commands.

Getting started

Summary

Autify CLI brings you yet another way to interface with the Autify system instead of the browser UI. The text based command line interface enables you to easily automate integration with Autify.

To use Autify CLI, first you need to install the Autify CLI onto your platform. Then, you can login to the CLI using your personal access token for Autify and run advanced commands provided by Autify CLI such as "Run a test scenaio and wait until it finishes." Let's go through this process in the next few sections.

Install

You can install Autify CLI via Homebrew (Linux/macOS), shell installer (Linux/macOS) or native installer (macOS/Windows). You can also install from the tarball directly.

Homebrew

For Linux and macOS platforms, you can use Homebrew to install autify-cli using our tap:

brew install autifyhq/tap/autify-cli

or

brew tap autifyhq/tap
brew install autify-cli

An example output:

$ brew install autifyhq/tap/autify-cli
==> Tapping autifyhq/tap
Cloning into '/usr/local/Homebrew/Library/Taps/autifyhq/homebrew-tap'...
remote: Enumerating objects: 19, done.
remote: Counting objects: 100% (19/19), done.
remote: Compressing objects: 100% (15/15), done.
remote: Total 19 (delta 1), reused 19 (delta 1), pack-reused 0
Receiving objects: 100% (19/19), done.
Resolving deltas: 100% (1/1), done.
Tapped 1 formula (16 files, 12.9KB).
==> Cloning https://github.com/autifyhq/autify-cli
Cloning into '/Users/runner/Library/Caches/Homebrew/autify-cli--git'...
==> Checking out revision b03bec6
HEAD is now at b03bec6 Release 0.2.0
==> Installing autify-cli from autifyhq/tap
==> curl https://autify-cli-assets.s3.amazonaws.com/autify-cli/versions/0.2.0/b03bec6/autify-v0.2.0-b03bec6-darwin-arm64.tar.gz | tar xz
🍺  /usr/local/Cellar/autify-cli/0.2.0: 10,082 files, 98.7MB, built in 29 seconds

Shell installer

For Linux and macOS platforms, you can use the shell installer via the command below:

curl https://autify-cli-assets.s3.amazonaws.com/autify-cli/channels/stable/install-standalone.sh | sh

An example output:

$ curl https://autify-cli-assets.s3.amazonaws.com/autify-cli/channels/stable/install-standalone.sh | sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2049  100  2049    0     0  10784      0 --:--:-- --:--:-- --:--:-- 10784
This script requires superuser access.
You will be prompted for your password by sudo.
Installing CLI from https://autify-cli-assets.s3.amazonaws.com/autify-cli/channels/stable/autify-linux-x64.tar.xz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 21.9M  100 21.9M    0     0  2157k      0  0:00:10  0:00:10 --:--:-- 2588k
v16.15.0
autify installed to /usr/local/bin/autify
autify-cli/0.1.0 linux-x64 node-v16.15.0

Native installer

For macOS and Windows, we provide native installers.

Note: They are not signed properly yet. Therefore, you'll get some warnings when installing. We'll sign them properly in the near future.

Tarball

We provide the tarballs below:

Here is an example command to download and extract a tar.gz file for Linux (Intel 64bit):

$ curl -O https://autify-cli-assets.s3.us-west-2.amazonaws.com/autify-cli/channels/stable/autify-linux-x64.tar.gz

$ tar -xzf ./autify-linux-x64.tar.gz

$ ./autify/bin/autify --version
autify-cli/0.1.0 linux-x64 node-v16.15.0

We recommend to link /path/to/autify/bin/autify to one of your PATH environment variables like below:

$ sudo ln -s /path/to/autify/bin/autify /usr/local/bin/autify

Note: /path/to/autify/bin includes a node executable. Therefore, if you add /path/to/autify/bin to your PATH environment variable, it might be prioritized over your existing node executable. That's the reason why we recommend to link only /path/to/autify/bin/autify and this is exactly what the shell installer does.

Usage

First, you need to generate a personal access token on Autify for Web and/or Autify for Mobile.

Then, you can login to Autify CLI. Run the command below and enter your personal access token:

## Autify for Web
$ autify web auth login
? Enter Access Token **********

## Autify for Mobile
$ autify mobile auth login
? Enter Access Token **********

Autify for Web

Now, let’s run a test scenario you have already created. Copy the URL of the test scenario or test plan you want to start (it will be something like https://app.autify.com/projects/000/scenarios/00000). Then, execute the command below:

autify web test run --wait <YOUR TEST SCENARIO URL>

An example output:

$ autify web test run --wait https://app.autify.com/projects/000/scenarios/00000
✅ Successfully started: https://app.autify.com/projects/000/results/000000 (Capability is Linux Chrome 101.0)
🕐 Waiting for the test result: https://app.autify.com/projects/000/results/000000
  ✔ Waiting... (timeout: 300 s)
✅ Test passed!: https://app.autify.com/projects/000/results/000000

$ echo $?
0

This command will execute the test scenario with the default capabilities (Linux/Chrome) and wait until it finishes up to the timeout (default 5 minutes). If the test passes, the command will exit with the exit code 0; otherwise, the exit code will be non-0. You can see the result as the output of $? above.

Autify for Mobile

If you use Autify for Mobile, you can run a test plan almost similar to Autify for Web. Copy the URL of a test plan you want to start (it will be something like https://mobile-app.autify.com/projects/AAA/test_plans/BBB) and also copy the file path for your application build file e.g. *.app for iOS or *.apk for Android. Then, execute the command below:

autify mobile test run --wait --build-path <YOUR BUILD PATH> <YOUR TEST SCENARIO URL>

$ autify mobile test run --wait --build-path /path/to/android.apk https://mobile-app.autify.com/projects/AAA/test_plans/BBB
✅ Successfully uploaded android.apk (ID: CCC). See https://mobile-app.autify.com/projects/AAA/builds?os=android&selectedBuildId=CCC&page=1
✅ Successfully started: https://mobile-app.autify.com/projects/AAA/results/DDD
🕐 Waiting for the test result: https://mobile-app.autify.com/projects/AAA/results/DDD
  ✔ Waiting... (timeout: 300 s)
✅ Test passed!: https://mobile-app.autify.com/projects/AAA/results/DDD

$ echo $?
0

This command will upload the build file and execute the test plan with the uploaded build and wait until it finishes up to the timeout (default 5 minutes). If the test passes, the command will exit with the exit code 0; otherwise, the exit code will be non-0. You can see the result as the output of $? above.

Update

Autify CLI uses the channel release for updating to avoid reinstalling everytime. By default, the stable channel is used and you can update your Autify CLI to the latest stable channel by running:

$ autify update
autify-cli: Updating CLI from 0.0.0-aaaaaa to 0.1.0-bbbbbb... done

Note: Homebrew users shouldn't use this command and should let Homebrew update instead. All stable releases will be shipped to our tap automatically, so this command should be sufficient to update to the latest version:

$ brew upgrade autify-cli

Note: There is a beta channel and release candidate channels for previewing the pre-release versions, but in most of the cases, the stable channel should be sufficient.

Configuration

The config file is stored at ~/.config/autify/config.env (macOS or Linux) / %LOCALAPPDATA%\autify\config.env (Windows) by default. You can override the configuration base directory via the XDG_CONFIG_HOME environment variable.

All configurations are formed as environment variables so that you can always override them at runtime by providing the same name of the environment variable such as AUTIFY_WEB_ACCESS_TOKEN=mytoken autify web test run ....

There are 4 configurations available:

AUTIFY_WEB_ACCESS_TOKEN

A personal access token to authenticate with Autify for Web.

AUTIFY_WEB_BASE_PATH

API base path to communicate with Autify for Web. Default is https://app.autify.com/api/v1.

AUTIFY_MOBILE_ACCESS_TOKEN

A personal access token to authenticate with Autify for Mobile.

AUTIFY_MOBILE_BASE_PATH

API base path to communicate with Autify for Mobile. Default is https://mobile-app.autify.com/api/v1.

Commands

Here is the list of commands. These commands are designed to be easy to use so that you don't need prior knowledge of our API specifications. They should cover most of the use cases and we’ll keep improving them by adding more commands/features. Feel free to request a new command for your specific use cases at GitHub Issues.

Autify for Web

autify web auth login

This command lets CLI login to Autify for Web interactively or non-interactivelly.

See GitHub for more details.

autify web test run

This command runs an existing test scenario or test plan with various options like capabilities or url replacements. It also supports the --wait option to wait until the test finishes.

See GitHub for more details.

autify web test wait

This command waits a test scenario or test plan that is already requested until it finishes. However, in most cases, you would use autify web test run --wait instead.

See GitHub for more details.

Autify for Mobile

autify mobile auth login

This command lets CLI login to Autify for Mobile interactively or non-interactively.

See GitHub for more details.

autify mobile build upload

This command just upoads your new build file to Autify for Mobile and returns the build id.

See GitHub for more details.

autify mobile test run

This command runs an existing test plan with options to use a new build file or already uploaded build id. It also supports the --wait option to wait until the test finishes.

See GitHub for more details.

autify mobile test wait

This command waits a test plan that is already requested until it finishes. However, in most cases, you would use autify mobile test run --wait instead.

See GitHub for more details.

API commands

The commands under api namespace (autify web api and autify mobile api) are not listed here intentionally because they are auto-generated commands from our API specs. See GitHub if you're interested in them. When you see a great use case to combine multiple APIs by api commands, we're happy to implement that in the use case specific command like autify web test run so that anyone can use the same logic easily. Please share your ideas at GitHub Issues.

CI/CD Integrations

We have serveal documentations about our official CI/CD integrations powerd by Autify CLI. To understand what is CI/CD integration and what integrations we have today, please read CI/CD with Autify first.

Contribution

Autify CLI is open source software hosted at GitHub. We’re planning to provide guidance for code contribution soon.