.kanman/acceptance.json

Every key of the acceptance manifest: how kanman starts your app, waits for it, and runs the acceptance spec of one story.

The acceptance manifest tells kanman how to prove a story in your repository. It lives at .kanman/acceptance.json in the repository root. With it, kanman can run the outcome gate: it writes an executable acceptance spec for each story, checks that the spec fails before any code exists, and checks that it passes on a clean checkout after the change.

kanman stays repository-agnostic. The manifest declares the commands; kanman runs them in a fresh environment. Secrets stay in your CI settings or in the workspace and never go into the manifest; the manifest only names them (see Secrets and CI variables).

For a step-by-step introduction, see Add an acceptance manifest to your repo. For the local verify command, see .kanman/verify.json.

Example

This is the manifest of the public demo repository kanman-ai/pilot-sandbox, a small TypeScript service with Playwright specs:

{
  "version": 1,
  "provision": {
    "type": "docker_compose",
    "file": "docker-compose.acceptance.yml",
    "up": "docker compose -f docker-compose.acceptance.yml up -d --build --wait",
    "down": "docker compose -f docker-compose.acceptance.yml down -v"
  },
  "ready_url": "http://localhost:3000/api/invoices",
  "ready_timeout_seconds": 120,
  "setup": "npm ci && npx playwright install --with-deps chromium",
  "run": {
    "command": "npx playwright test",
    "spec_arg": ".kanman/acceptance/{KEY}/",
    "env": {
      "BASE_URL": "http://localhost:3000"
    }
  },
  "spec_dir": ".kanman/acceptance/{KEY}",
  "artifacts": [
    "test-results/**/trace.zip",
    "test-results/**/*.png",
    "test-results/**/*.webm",
    "test-results/results.json",
    "playwright-report/**"
  ]
}

For story SBX-12, kanman runs npx playwright test .kanman/acceptance/SBX-12/ against the app started with Docker Compose, and stores the trace, screenshots, video and JSON report as proof.

Top-level keys

Key Type Required Default Description
version number yes Always 1.
provision object yes How to start a fresh environment for the spec. See provision.
ready_url string (URL) no kanman polls this URL every two seconds after provisioning and continues once it answers with a 2xx or 3xx status. Leave it out if your provision command only returns when the app is ready (for example docker compose up --wait).
ready_timeout_seconds integer no 300 How long kanman waits for the environment to become ready. Maximum 900.
setup string no Shell command run once in the checkout after the environment is ready and before the spec, for example installing dependencies and browsers.
run object yes How to run the spec. See run.
spec_dir string no .kanman/acceptance/{KEY} Folder of one story’s spec. {KEY} is replaced with the story key.
context array of strings no [] Files or folders of your repository that kanman reads before it writes a spec, for example a login helper or page objects (folders one level deep). kanman always reads the example spec in the spec_dir folder named EXAMPLE and the files in _support/ folders, so list only what lives elsewhere.
artifacts array of strings no ["test-results/**"] Glob patterns of files kept as proof (traces, screenshots, video, reports, apply logs). They are linked from the evidence pack. The first entry that is a plain path ending in .json (no wildcards), for example test-results/results.json, is read as the framework’s JSON report.
framework string no playwright Test framework of the specs: playwright, vitest, jest, pytest or other. kanman uses it to write specs in the right style and to read results. With other, only the exit code counts.

Unknown keys are ignored, so a manifest written for a newer version keeps working with an older runner.

provision

provision.type selects one of two forms.

Docker Compose

Use this for applications that start as one or more containers.

Key Type Required Default Description
type string yes docker_compose
file string yes Path of the Compose file, relative to the repository root.
up string no docker compose -f <file> up -d --build --wait Full command that starts the environment. The default returns once the health checks pass.
down string no docker compose -f <file> down -v Full command that removes the environment after the run.
service string no Name of the main service, for your own reference. kanman accepts the key but does not use it.

Apply

Use this for repositories that are not a running app, for example infrastructure or configuration. The proof is the apply log plus whatever your spec checks.

Key Type Required Default Description
type string yes apply
command string yes Command that creates the environment, for example a plan and apply against a throwaway workspace.
down string no Command that tears it down again.

Example for an infrastructure repository that checks its modules against a local test stack:

{
  "version": 1,
  "provision": {
    "type": "apply",
    "command": "make test-env-up > apply.log 2>&1",
    "down": "make test-env-down"
  },
  "ready_timeout_seconds": 600,
  "setup": "make tools",
  "run": {
    "command": "pytest",
    "spec_arg": ".kanman/acceptance/{KEY}/",
    "env": {
      "TEST_ENV": "acceptance"
    }
  },
  "spec_dir": ".kanman/acceptance/{KEY}",
  "framework": "pytest",
  "artifacts": ["apply.log", "reports/**/*.xml"]
}

run

Key Type Required Default Description
command string yes Command that runs specs, for example npx playwright test.
spec_arg string no Argument appended to command to run only one story’s spec. {KEY} is replaced with the story key.
env object of strings no {} Extra environment variables for the spec run. Only plain values, never secrets.
ci_var_keys array of strings no [] Names of CI variables of your repository that setup and the spec run receive, for example the login of a test account. See Secrets and CI variables.
secrets array of strings no [] Names of workspace secrets granted to the team that setup and the spec run receive. See Secrets and CI variables.

The full command for one story is command plus a space plus spec_arg with {KEY} filled in.

kanman always sets CI=true and KANMAN_STORY_KEY (the story key) for the run. Values in run.env win over these.

Tip

Keep the trailing slash in spec_arg when you pass a folder. Playwright treats the argument as a pattern, so .kanman/acceptance/SBX-1 would also match SBX-12.

Secrets and CI variables

Specs often need values that must not be in the repository, for example the user name and password of a test account for your login page. Name them in run; the values stay where they are kept today:

  • run.ci_var_keys: variables from your repository’s CI settings. On GitHub these are Actions variables of the repository or of the organization (GitHub never hands out the values of Actions secrets; store such a value as a workspace secret instead). On GitLab these are the project’s CI/CD variables; the connected account needs the Maintainer role to read them. If your workspace runs everything on your own runners, the runner reads them with its own GitLab or GitHub token, and kanman’s cloud never sees the values.
  • run.secrets: workspace secrets. A name counts only if the team grants it under What I may change in Team settings > Limits and budgets.
"run": {
  "command": "npx playwright test",
  "spec_arg": ".kanman/acceptance/{KEY}/",
  "env": { "BASE_URL": "http://localhost:3000" },
  "ci_var_keys": ["E2E_KC_USERNAME", "E2E_KC_PASSWORD"],
  "secrets": ["E2E_API_TOKEN"]
}

How kanman handles them:

  • Only setup and the spec run receive the values. Provisioning and teardown do not.
  • kanman takes the names from the manifest on your default branch. Names a run branch adds count only once they are merged.
  • The values are removed from the run log, the command output and the test report kanman keeps, and they are never stored in the evidence pack, the run or the audit log. The run log names which variables the run received and which were not available, with the reason. Traces, screenshots and videos show what your application displays, so do not let a spec print a password onto the page.
  • A name kanman cannot read (not found, no access, not granted) is listed in the run log, and the run continues without it. Names kanman uses itself, such as model provider keys or PATH, are never filled.
  • On a self-hosted runner, these are the only credentials repository commands receive; see Runner secrets.

The {KEY} placeholder

{KEY} is the story key kanman shows on the story page, made of the team’s key prefix and a number, for example SBX-12. It is replaced in spec_dir and run.spec_arg.

The protected spec folder

Everything below .kanman/acceptance/ belongs to kanman’s spec author:

  • kanman writes the spec from the story’s Demonstrate block before the story moves to Ready. The files must stay inside the story’s own folder (.kanman/acceptance/<KEY>/).
  • Once the spec has failed as expected, kanman commits it to the branch kanman/spec/<KEY>. The implementation branch starts from there, so the spec ships with the pull request.
  • The implementation never edits these files. Every preset denies .kanman/acceptance/** to the executor, and the diff guard stops the run if a commit touches the folder anyway or the folder differs from the version approved at Ready.
  • A spec must not mock its own target. Network interception, MSW, nock and module mocks of the behaviour under test make the spec invalid.

How kanman uses the manifest

Moment What kanman runs
Before Ready provision, wait for ready_url, setup, then the new spec on your default branch. It must run and fail.
In Progress to Review The same in a freshly provisioned environment against the implementation branch, with the approved spec restored. The spec must pass.
Review to Done The same on a clean-room checkout (never the executor’s workspace) in a fresh environment. The spec must pass and artifacts are stored as proof.
After a deployment When your GitHub repository reports a successful deployment with an environment URL, kanman runs the specs of stories merged in the seven days before it against that URL, without provisioning. A failure is filed as a regression on the team’s intake.

Each run gets a fresh checkout in its own directory. down runs after each of these, also when the spec fails. One run (provisioning, setup and spec together) may take at most 30 minutes.

A spec that does not compile, finds no tests or crashes the test runner counts as an error, not as red. Problems before the spec starts (clone, provisioning, ready check, setup) count as environment failures; kanman retries them quietly before it asks you.

Validation errors

kanman reads the manifest from your default branch when a story moves toward Ready. If it is missing or invalid, the story stays in Refining and the story page shows The repository has no .kanman/acceptance.json, so no acceptance spec can run or The acceptance manifest is invalid: followed by the reason. Common causes:

Problem Fix
version must be 1 Add "version": 1.
provision.type must be docker_compose or apply Use one of the two forms.
ready_timeout_seconds is above 900 Lower it or make up return only when the app is ready.
run.command is empty Add the command that runs your specs.

Repositories without a manifest can only run on the Trial preset; with any other preset the story stays in Refining. kanman then falls back to CI, the test plan and the reviewer run, and the evidence pack says plainly that no outcome proof exists.

Last updated: January 1, 0001

Open kanman