Azure DevOps

The Klarent Azure DevOps integration runs tests using the Klarent Action Docker image. Any Azure Pipelines agent with Docker available can run Klarent tests.

Prerequisites

1. Docker on the agent

• Microsoft-hosted agents (ubuntu-latest): Docker is pre-installed — no setup needed.
• Self-hosted agents: Docker must be installed and accessible on the agent machine.

2. Add variables in Azure Pipelines

Go to Pipelines → select your pipeline → Edit → Variables and add the following:

Variable name	Value	Secret?
CJ_SERVICE_ACCOUNT_KEY	Your Klarent service account key	Yes
CJ_TEST_ID	ID of the test to run (for single-test pipelines)	No
CJ_TEST_SUITE_ID	ID of the test suite to run (for suite pipelines)	No

Note

Mark sensitive values (such as CJ_SERVICE_ACCOUNT_KEY) as secret in the Variables panel to prevent them from appearing in logs.

Run a single test

Self-hosted agent:

trigger:
  - main

pool:
  name: Default

steps:
  - script: |
      docker run --rm \
        -e "INPUT_SERVICE_ACCOUNT_KEY=$(CJ_SERVICE_ACCOUNT_KEY)" \
        -e "INPUT_TEST_ID=$(CJ_TEST_ID)" \
        foreai/cj-action:latest
    displayName: 'Run Klarent Test'
    env:
      CJ_SERVICE_ACCOUNT_KEY: $(CJ_SERVICE_ACCOUNT_KEY)

Microsoft-hosted agent:

trigger:
  - main

pool:
  vmImage: ubuntu-latest

steps:
  - script: |
      docker run --rm \
        -e "INPUT_SERVICE_ACCOUNT_KEY=$(CJ_SERVICE_ACCOUNT_KEY)" \
        -e "INPUT_TEST_ID=$(CJ_TEST_ID)" \
        foreai/cj-action:latest
    displayName: 'Run Klarent Test'
    env:
      CJ_SERVICE_ACCOUNT_KEY: $(CJ_SERVICE_ACCOUNT_KEY)

Note

The env: block is required for secret variables. Azure Pipelines does not expose secret variables to scripts by default — the env: block explicitly maps them into the script’s environment so Docker can pass them to the container.

Run a test suite

Self-hosted agent:

trigger:
  - main

pool:
  name: Default

steps:
  - script: |
      docker run --rm \
        -e "INPUT_SERVICE_ACCOUNT_KEY=$(CJ_SERVICE_ACCOUNT_KEY)" \
        -e "INPUT_TEST_SUITE_ID=$(CJ_TEST_SUITE_ID)" \
        -e "INPUT_WAIT_TIMEOUT_SECONDS=360" \
        -e "INPUT_WEBSITE_URL_OVERRIDE=https://staging.my-app.com" \
        -e "INPUT_PARAMS_OVERRIDE={\"param1\":\"value1\",\"param2\":\"value2\"}" \
        -e "INPUT_BROWSER_TYPE_OVERRIDE=chromium" \
        foreai/cj-action:latest
    displayName: 'Run Klarent Test Suite'
    env:
      CJ_SERVICE_ACCOUNT_KEY: $(CJ_SERVICE_ACCOUNT_KEY)

Microsoft-hosted agent:

trigger:
  - main

pool:
  vmImage: ubuntu-latest

steps:
  - script: |
      docker run --rm \
        -e "INPUT_SERVICE_ACCOUNT_KEY=$(CJ_SERVICE_ACCOUNT_KEY)" \
        -e "INPUT_TEST_SUITE_ID=$(CJ_TEST_SUITE_ID)" \
        -e "INPUT_WAIT_TIMEOUT_SECONDS=360" \
        -e "INPUT_WEBSITE_URL_OVERRIDE=https://staging.my-app.com" \
        -e "INPUT_PARAMS_OVERRIDE={\"param1\":\"value1\",\"param2\":\"value2\"}" \
        -e "INPUT_BROWSER_TYPE_OVERRIDE=chromium" \
        foreai/cj-action:latest
    displayName: 'Run Klarent Test Suite'
    env:
      CJ_SERVICE_ACCOUNT_KEY: $(CJ_SERVICE_ACCOUNT_KEY)

Available inputs

Pass each input as a Docker environment variable using -e "INPUT_<NAME>=<value>".

Environment variable	Required	Default	Description
INPUT_TEST_ID	No*		ID of the test to run. Provide either this or INPUT_TEST_SUITE_ID.
INPUT_TEST_SUITE_ID	No*		ID of the test suite to run.
INPUT_SERVICE_ACCOUNT_KEY	Yes		Your Klarent service account key.
INPUT_WAIT_TIMEOUT_SECONDS	No	300	Max seconds to wait for completion. Must be between 30 and 900.
INPUT_WEBSITE_URL_OVERRIDE	No		Override the base website URL used during test execution.
INPUT_PARAMS_OVERRIDE	No		Override default parameter values. Must be a valid JSON string with string keys and values.
INPUT_BROWSER_TYPE_OVERRIDE	No	chromium	Browser engine: chromium, firefox, or webkit.
INPUT_CREATE_ISSUE_ON_FAILURE	No	false	If true, creates an issue on test failure (requires additional token configuration).

* Either INPUT_TEST_ID or INPUT_TEST_SUITE_ID must be provided.