Run a campaign

What a campaign is

A campaign gives a coding agent a target and the instructions to file what it finds. Three commands run it.

During the campaign, your agent writes Testkit tests. It files each bug it proves with testkit bug record and testkit bug file. What a bug report states describes the bug record and the issue.

Campaigns, bug filing, and CI reports need no Turnstone account. They use the CLI and your repo on GitHub or Gitea.

Before you start

You need these four things.

• Testkit 0.4.1 or later, with one test passing. Get started covers both.
• A clone whose origin remote is the repo on GitHub or Gitea. Testkit files issues in that repo. To file them in another repo, set bugs.repository.
• A token for that repo that can create issues and labels.
• A coding agent that can run commands in the clone.

To check the version, run npx testkit --version. It prints a line that starts with @gfdelarue/testkit/ and the installed version.

Set the token

Testkit reads the token from the environment.

With no token set, Testkit asks git credential fill for one.

Set the variable in the shell that your agent works in. Keep the token out of the repo and out of every bug record.

Testkit reaches GitHub Enterprise Server only from a GitHub Actions job on that server, so testkit bug file works there only in a job.

Name your product's areas

An area is a named part of your product and the paths that hold its code. Add areas to testkit.config.ts at the repo root. If the file does not exist, create it.

import { defineConfig } from "@gfdelarue/testkit/config";

export default defineConfig({
	areas: {
		checkout: { name: "Checkout", paths: ["src/checkout/**"] },
		billing: { name: "Billing", paths: ["src/billing/**", "src/invoices/**"] },
	},
});

Each key is an area id. An area id uses lowercase letters, digits, and single hyphens. Each path is a glob from the repo root. The same file holds the rest of your config, such as the services that your tests start.

Testkit puts an area:<id> label on each issue it files, and Turnstone reads the bug's area from that label. Without areas, Testkit files bugs with no area.

areas gives the rules for ids, names, and paths.

Pick a target

The target is one line of at most 200 characters that tells the agent what to hunt, such as Refund flows in checkout.

Keep the target narrow. A narrow target gets the agent reading code sooner and keeps its reports about one part of the product.

Pick code where the right behaviour is written down: an API reference, a spec, a ticket, or a contract with a customer. Every bug record names that source in its authority field. Without one, a report is an opinion.

Start the campaign

Run the command from the repo root.

$npx testkit campaign start --target "Refund flows in checkout" --id refunds-2026-10

--id sets the campaign id. Without it, Testkit makes the id from the target and the date, such as refund-flows-in-checkout-2026-10-14. The campaign id goes into the facts block of each issue that the campaign files.

A checkout holds one active campaign. If a campaign is active, campaign start stops and tells you to run testkit campaign finish first.

Give the brief to your agent

campaign start prints the agent's brief. Give that output to the agent.

# Testkit campaign refunds-2026-10

Target: Refund flows in checkout

## Area map

- `checkout`, Checkout: 1 file, 54.2% of source bytes
- `billing`, Billing: 2 files, 41.9% of source bytes
- Unclaimed: 1 file, 3.9% of source bytes

Unclaimed source files:

- `src/server.js`

The areas, the files, and the shares in this brief come from a small example repo. The area map lists each area's share of the repo's source, the source files that no area claims, and the files that two areas claim.

The brief asks the agent to write or refresh areas before it hunts, so that every source file belongs to exactly one area. Read the change it makes to testkit.config.ts.

The brief then has these parts.

What your agent does for each bug

For each bug that the agent confirms, it does these steps.

1. It writes a Testkit test that fails because of the bug.
2. It writes the bug record as a JSON file. npx testkit bug guide prints the fields and the cause, impact, and escape lists.
3. It runs testkit bug record. The command checks the record, stores a draft under .testkit/bugs/, and prints the draft id.
4. It commits the test and pushes the commit.
5. It runs testkit bug file with the draft id.

Recorded a-refund-pays-out-twice-d1c8d076
Draft: .testkit/bugs/a-refund-pays-out-twice-d1c8d076.json
Next: commit and push the test, then run `testkit bug file a-refund-pays-out-twice-d1c8d076`.

testkit bug file runs the test at the checked-out commit and files the issue only when the test fails. Replace DRAFT_ID with the draft id that testkit bug record prints.

Filed #12: https://github.com/acme/shop/issues/12
Area: checkout, 14.0% of source bytes
Introduced by: 9b7a4c3d8e6f

The repo, the issue number, and the commit in this output are examples.

The issue carries the testkit label and a severity:<level> label. With areas set, it also carries an area:<id> label. The issue body names the regression test, the commit that introduced the faulty lines, and the command that reproduces the failure.

If testkit bug file stops without filing, What testkit bug file checks lists each cause and what to do.

Check what comes back

Every bug arrives as an issue that names a failing test in the repo.

1. Run the command under Reproduce in the issue. The test must fail on your machine for the reason the issue gives.
2. Open the source that the issue cites under Authority. It must say what the issue claims.
3. Read the root cause. It must name code, not a symptom.

Close a report that fails any of the three. What a bug report states lists the sections to expect.

See what the campaign recorded

campaign status lists the bugs recorded so far.

Active campaign: refunds-2026-10
  Target: Refund flows in checkout
  Started: 2026-10-14T00:26:02.987Z
  Start commit: ac58e0f35137c177346285256a5559520a083f2f
Recorded bugs: 1
  high: A refund pays out twice
    Test: src/checkout/__testkit__/refund.contract.testkit.ts
    Issue: not filed, draft a-refund-pays-out-twice-d1c8d076
Area map: 3.9% of source bytes unclaimed, 0 double-claimed files.

The bug in this output has a draft and no issue, so testkit bug file has not filed it yet. The last line gives the share of the source that no area claims and the number of files that two areas claim.

Finish the campaign

When the agent finishes, write the summary.

$npx testkit campaign finish

campaign finish ends the campaign, writes a summary to .testkit/campaigns/<id>/summary.md, and prints it. The summary is for the pull request that adds the tests. It holds these parts.

• The target, the start and finish times, and the range of commits.
• The bugs that the campaign filed.
• The bugs that the campaign recorded and did not file, each with its draft id.
• The Testkit test files that the campaign added or changed.

Keep the test failing until the fix lands

The pull request that adds the tests stays red while a bug is open, and that is correct. You can merge it in two ways.

• Push the fix to the same branch, watch the test pass, and merge both together.
• Set bugs.knownFailures to "allow". A run then passes when an open issue with the testkit label covers each failure. A new failure still fails the run.

Do not skip the test to get a green check. Do not rewrite it to assert what the product does today. Either change hides a bug that is still there.

Stop a run

An agent can leave a run going. testkit run status shows active runs, and testkit stop ends every run that Testkit owns in the repo.

$npx testkit run status
$npx testkit stop --reason "campaign paused"