Autify Connect integration
  • 11 Oct 2022
  • 10 Minutes to read
  • Contributors
  • Dark
    Light

Autify Connect integration

  • Dark
    Light

In this document, we'll explain how to use Autify Connect with Autify Command Line Interface (CLI) and the other existing CI/CD integrations. Autify CLI provides unified and streamlined experiences for Autify Connect just like bunch of other commands and our CI/CD integrations already support it as well.

Read Autify Connect Overview first so that you understand the idea of what Autify Connect is.

Autify Connect's use cases

Before jumping into the details, let's set our tone about the use cases of Autify Connect. Autify Connect provides one significant functionality to you i.e. running Autify's test even if the endpoint is unavailable via the internet. This opens three different use cases for you:

  1. Test against endpoints in your private network with long-running static Access Point
  2. Test against endpoints in your private network via ephemeral Access Point on your local machine
  3. Test against your localhost endpoints via ephemeral Access Point on your local machine

Use case 1

autify-connect-case-1.png

The use case 1 is for the cases you want to run Autify's test against your private website such as staging environment behind your firewall, or corporate internal website, etc. In this case. you need to run at least one Autify Connect Client inside your private network with a static Access Point like "corp network". Then, you can use that Access Point for any test you want to run inside your private network as long as someone maintains the availability of the Access Point i.e. keeps the Autify Connect Client up and running for the static Access Point by daemonizing and moniroting it.

Use case 2

autify-connect-case-2.png

The use case 2 is for the same target endpoins as the use case 1 but Access Point is ephemeral and running on your local machine. In this case, you don't have to manage such long-running process. Instead, you can create an ephemeral Access Point when needed and run Autify Connect Client with the ephemeral one. Now you can run Autify test against private network via your local machine as long as you keep running the Autify Connect Client and your local machine has access to the target endpoints.

Use case 3

autify-connect-case-3.png

The use case 3 is for a completely different purpose from the use case 1 and 2. The target endpoints are localhost/127.0.0.1 e.g. local development server but you still want to run Autify test against such local endpoints. The ephemeral Access Point described in the use case 2 can be also useful for this case because your local machine has access to your localhost. Therefore, even though the target endpoints are totally different, the solution is identical for both the use case 2 and 3.

Autify CLI provides the easiest ways to support all the use cases above. We'll go though them one by one but you can jump into the section of the specific use case you want to know.

Getting started

Autify CLI provides a set of commands to operate Autify Connect on your local machine easily. Let's walk through them and you realize there are no frictions to get started with Autify Connect.

Prerequisites

Record a test scenario

You need to create a test scenario at Autify for Web. Record a scenario against your private endpoint or your localhost endpoint e.g. development server. Note the Autify's URL of the recorded test scenario e.g. https://app.autify.com/projects/0000/scenarios/0000.

All the following secitons use a test scenario as a target but you can also use a test plan by passing its URL like https://app.autify.com/projects/0000/test_plans/0000.

For localhost/127.0.0.1 recording, you should use localhost.autify.com while recording instead of localhost/127.0.0.1 for now.

If you record the scenario with localhost/127.0.0.1, you need to ask URL replacement to localhost.autify.com when you run the test. We're working on eliminating this restrictions. Until then, either recording with localhost.autify.com or URL replacement to localhost.autify.com are needed.

Install Autify CLI

You need to install Autify CLI and set a personal access token of Autify for Web. See Autify CLI's document for more details.

Install Autify Connect Client

Just run the command below. This will download an appropriate autifyconnect binary to the cache directory for Autify CLI:

autify connect client install

Example output:

$ autify connect client install
Installing Autify Connect Client (version: v0.6.1)... done
Successfully installed Autify Connect Client (path: /home/node/.cache/autify/autifyconnect, version: Autify Connect version v0.6.1, build ca0972e)

1. Test against endpoints in your private network with long-running static Access Point

The steps below are only for getting started or demo purpose. For the production use, you probably want to run autifyconnect binary directly from your supervisor like systemd, supervisord, etc. See this document about how to run and operate autifyconnect in production.

In this use case, you'll create a static Access Point once and re-use it. First, create an Access Point by Autify CLI:

autify connect access-point create --name ACCESS_POINT_NAME --web-workspace-id WORKSPACE_ID

This command stores the generated key in the config directory of Autify CLI automatically.

If you prefer to create an Access Point via UI, you can also do that. Once created, store the name and key of the Access Point by the command below. You'll be asked to enter the generated key via STDIN:

autify connect access-point set --name ACCESS_POINT_NAME

Let's start the Autify Connect Client with the created Access Point. Keep this terminal up and running somewhere:

autify connect client start

Now, the static Access Point is ready to use. Run the Autify test scenario you recorded with the Access Point:

autify web test run TEST_SCENARIO_URL --autify-connect ACCESS_POINT_NAME --wait

This command will start the test scenario with connecting to the Access Point and wait until the test finishes.

2. Test against endpoints in your private network via ephemeral Access Point on your local machine

In this case, you don't need a static Access Point but just an ephemeral one. Autify CLI supports automatic creating/dereting an ephemeral Access Point when running Autify Connect Client.

Run the command below. It will create an ephemeral Access Point, run Autify Connect Client with the generated key, run the test scenario. Once the test scenario finishes, it will also stop the Autify Connect Client and delete the ephemeral Access Point automatically:

autify web test run TEST_SCENARIO_URL --autify-connect-client --wait

Example output:

$ autify web test run https://app.autify.com/projects/000/scenarios/0000 --autify-connect-client --wait
[Autify Connect Manager]  2022-09-15T19:54:39.213Z	info	Ephemeral Access Point was created: autify-cli-UUID
Starting Autify Connect Client...
[Autify Connect Manager]  2022-09-15T19:54:39.253Z	info	Starting Autify Connect Client (accessPoint: autify-cli-UUID, debugServerPort: 51035, path: /Users/runner/Library/Caches/autify/autifyconnect, version: Autify Connect version v0.6.2, build 0913a76)
Waiting until Autify Connect Client is ready...
[Autify Connect Client]   2022-09-15T19:54:39.262Z	info	start serving a debug server on http://localhost:51035
[Autify Connect Client]   2022-09-15T19:54:39.262Z	info	Starting to establish a secure connection with the Autify connect server. Your session ID is "...".
[Autify Connect Client]   2022-09-15T19:54:40.509Z	info	Successfully connected!
Autify Connect Client is ready!
✅ Successfully started: https://app.autify.com/projects/000/results/1111 (Capability is Linux Chrome 104.0)
🕐 Waiting for the test result: https://app.autify.com/projects/000/results/1111
  ✔ Waiting... (timeout: 300 s)
✅ Test passed!: https://app.autify.com/projects/000/results/1111
Waiting until Autify Connect Client exits...
[Autify Connect Client]   2022-09-15T19:54:51.932Z	info	Interrupt received.
[Autify Connect Client]   2022-09-15T19:54:51.932Z	info	Shutdown completed.
[Autify Connect Manager]  2022-09-15T19:54:52.358Z	info	Ephemeral Access Point was deleted: autify-cli-UUID
[Autify Connect Manager]  2022-09-15T19:54:52.362Z	info	Autify Connect Client exited (code: 0, signal: null)
Autify Connect Client exited.

This command should cover most of the use case but if you want to run Autify Connect Client separately, first run it with your workspace ID to which your recorded test scenario belongs (the workspace ID is inside the URL after /projects/):

autify connect client start --web-workspace-id 000

Then, run the test scenario against the created ephemeral Access Point (the name should be shown in the previous command's output like Starting Autify Connect Client (accessPoint: autify-cli-UUID,...):

autify web test run TEST_SCENARIO_URL --autify-connect "autify-cli-UUID" --wait

3. Test against your localhost endpoints via ephemeral Access Point on your local machine

In this case, you just want an ephemeral Access Point on your local machine. Autify CLI supports automatic creating/dereting an ephemeral Access Point when running Autify Connect Client.

First of all, run your localhost server in a separate terminal. It would be like rails server, npm run start, etc. depending on your task runner. Also assuming your test scenario is recorded against the local endpoint like http://localhost.autify.com:3000 (Remember: We'd like to support localhost out-of-the-box but currently you need to use localhost.autify.com for recording or URL replacements.)

Then, run the same command as the use case 2:

autify web test run TEST_SCENARIO_URL --autify-connect-client --wait

If you recorded the test scenario against localhost/127.0.0.1, you need to request URL replacements like below (for now):

autify web test run TEST_SCENARIO_URL --autify-connect-client --wait --url-replacements http://localhost=http://localhost.autify.com

A demo of localhost development use case

Here is a step-by-step example to demonstrate a typical development cycle with the use case 3:

  1. Launch a dead simple localhost application.
    • The command bellow will launch a static http server at http://localhost:8000 on macOS. Keep it running in the foreground while going through the rest.
echo Hello world! > index.html
python3 -mhttp.server
  1. Record a test scenario against the http://localhost.autify.com:8000 server.
    • Add an assertion on the text element that the text shold be "Hello world!".
    • In real use cases, this test scenario should be shared among your team and optionally used for testing staging/production endpoints as well.
  2. Run the test scenario from Autify using Autify Connect Client.
autify web test run https://app.autify.com/projects/000/scenarios/000 --autify-connect-client --wait
  1. Confim the test passed.
  2. Modify your application.
echo Hello Autify! > index.html
  1. Run the same test scenario from Autify using Autify Connect Client by the same command.
autify web test run https://app.autify.com/projects/000/scenarios/000 --autify-connect-client --wait
  1. Confirm the test failed this time.
    • This imitated when your local change causes a regression failure.

As you can see above, you can verify your localhost application by Autify with a single command. You can embed that command to your task runner e.g. Rake/npm/etc. to run it natively if you want.

CI/CD integrations for Autify Connect

All of our official CI/CD integrations use Autify CLI under the hood. Therefore, you can do exactly the same things above with these CI/CD integrations, too.

Run a test scenario against localhost server inside CI/CD runners during code review process

CI_CD with Autify-local-EN.png

The use case 3 can be extended to do the same thing inside CI/CD runners e.g. GitHub Actions hosted runners. This allows you to run Autify test during code review processes without extra infastructure. It's a critical heavy lifting for the blocker of Autify integration with code review process. See more details about code review integration here.

To enable this integration, you need to ensure your localhost server can launch and work properly inside CI/CD runners as well. Typical pitfalls are database (seed), other microservices (mock, monorepo or Docker), credentials (must set in the secret store) and network (CI/CD managed runners might not reside in your private network).

Once it works just like your local machine, you can run a test scenario against the localhost server inside CI/CD runners.

For example, GitHub Actions and Rails application. You can add this workflow to your Rails applicaiton's GitHub repository. Then, each pull request will launch rails server inside CI/CD runners and run Autify test via Autify Connect Client:

on:
  pull_request:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: ruby/setup-ruby@v1
        with:
          ruby-version: '3.0'
      # Running your Rails server in the background. It might require additional setup like database or passing credentials via environment variables.
      - run: bundle exec rails server --daemon
      - uses: autifyhq/actions-web-test-run@v2
        with:
          access-token: ${{ secrets.AUTIFY_WEB_ACCESS_TOKEN }}
          # Assuming the test scenario was recorded against http://localhost.autify.com
          autify-test-url: https://app.autify.com/projects/000/scenarios/000
          autify-connect-client: true
          wait: true
          timeout: 300

This will consume your GitHub Actions hosted runner's minutes while waiting. Be careful when extending the timeout value.

If you want us to support running these processes inside our infrastructure on behalf of you, please contact us because we'd like to hear from you.

One concern here is that CI/CD providers might disconnect/reject long-lasting TCP connections because it looks like abuse e.g. backdoor. Self hosted runner can avoid such limitation as well as the cost issue (although they need to pay for the runner's platform like Amazon EC2.)

Run a test scenario with Autify Connect Client already running somewhere else

This is for use case 1 and it's pretty simple because Autify Connect Client is already running. Assuming you already deployed your software to your private network application. Then, just run a test scenario like you can do for a public network applicatoin.

Here is an example of GitHub Actions after your deployment step. This step will start Autify test using Autify Connect Access Point named YOUR_ACCESS_POINT_NAME.

- uses: autifyhq/actions-web-test-run@v2
  with:
    access-token: ${{ secrets.AUTIFY_WEB_ACCESS_TOKEN }}
    autify-test-url: https://app.autify.com/projects/000/scenarios/000
    autify-connect: YOUR_ACCESS_POINT_NAME

Conclusion

In this document, we explain the use cases of Autify Connect, how to use Autify Connect Client with Autify CLI and how to easily run Autify test along with Autify Connect Client. This integration should open various capabilites for you to test your websites uniquely like test against private network or test against localhost development server easily even inside CI/CD runners.


Changing your password will log you out immediately. Use the new password to log back in.
First name must have atleast 2 characters. Numbers and special characters are not allowed.
Last name must have atleast 1 characters. Numbers and special characters are not allowed.
Enter a valid email
Enter a valid password
Your profile has been successfully updated.