What a bug report states

From a bug record to an issue

A bug report starts as a bug record. The bug record is a JSON file that your coding agent writes during a campaign, or that you write yourself. Three commands turn it into an issue.

The commands work with or without an active campaign. Bug filing needs no Turnstone account.

The five sections

Every issue that Testkit files has five sections.

A report with all five can be checked by someone who was not there. Every field of the bug record is required, so no report lacks one.

Write the bug record

$npx testkit bug guide

The command prints the fields of a bug record, the severities, and the lists of cause, impact, and escape ids.

{
	"title": "A refund pays out twice",
	"severity": "high",
	"expected": "The second POST /orders/{id}/refunds for the same order returns 409 and pays nothing.",
	"authority": "`docs/api/refunds.md`, \"A refund is idempotent per order.\"",
	"observed": "The second request returns 201 and creates a second payout.",
	"rootCause": "`src/checkout/refunds.ts` retries the payout call without an idempotency key.",
	"test": {
		"file": "src/checkout/__testkit__/refund.contract.testkit.ts",
		"title": "a refund pays out once"
	},
	"fault": { "path": "src/checkout/refunds.ts", "lines": [41, 58] },
	"cause": "mishandled-error",
	"impact": "financial-error",
	"confidence": { "cause": "high", "impact": "high" },
	"escape": "no-test"
}

These rules apply to the fields.

• Write test.file and fault.path relative to the product directory, the directory that holds testkit.config.ts.
• test.title must be the failure's title exactly as Testkit reports it. In a Vitest or Playwright file, that is the names of the describe blocks around the test and the test's own name, joined by single spaces. In a k6 file, that is the label of the failed assertion.
• A k6 script that fails as a whole has no failure title. Write in test.title what the script tests. Testkit then files one issue that covers the whole file.
• Keep credentials, tokens, and passwords out of the record.

Severities

Cause, impact, and escape

Each of these fields takes one id from a list. testkit bug guide prints each id's definition and the rule that separates it from its neighbours.

A bug gets one id for each field. The impact list is in precedence order, and the first id that applies wins. Use other when none fits.

The test that comes with it

The bug record names one test. The test's name states the expected behaviour, so the name is true once the product is fixed.

import { defineFile } from "@gfdelarue/testkit";
import { contractScenario, expect } from "@gfdelarue/testkit/node-vitest/contract";

export const testkit = defineFile({ engine: "node-vitest" });

contractScenario("a refund pays out once", async ({ http }) => {
	const first = await http().post("/orders/8841/refunds");
	const second = await http().post("/orders/8841/refunds");
	expect(first.status).toBe(201);
	expect(second.status).toBe(409);
});

Against the broken product, the test fails on the line that states the expected behaviour.

[1/1] FAIL api contract src/checkout/__testkit__/refund.contract.testkit.ts 0s
    AssertionError: expected 201 to be 409 // Object.is equality at
    /repo/src/checkout/__testkit__/refund.contract.testkit.ts:10:24

The issue Testkit files

The issue's title is the title of the bug record.

Labels

Testkit creates a label that does not exist yet and never changes one that does. An issue carries an area:<id> label when areas is set.

Severity and area are labels because people re-triage both. To change a bug's severity or area, replace the label. An issue with two severity: labels counts as having none, and the same holds for area: labels.

Body

The body has the five sections, three lines that locate the bug, and the facts block.

## Expected behaviour

The second POST /orders/{id}/refunds for the same order returns 409 and pays nothing.

## Authority

`docs/api/refunds.md`, "A refund is idempotent per order."

## Observed behaviour

The second request returns 201 and creates a second payout.

## Root cause

`src/checkout/refunds.ts` retries the payout call without an idempotency key.

## Reproduce

```sh
npx testkit run --file src/checkout/__testkit__/refund.contract.testkit.ts
```

Test: `src/checkout/__testkit__/refund.contract.testkit.ts`, "a refund pays out once".
Found at `4f0c2d1e9b7a4c3d8e6f5a4b3c2d1e0f9a8b7c6d`.
Fault: `src/checkout/refunds.ts`, lines 41 to 58.

<details>
<summary>Testkit facts</summary>

<!-- testkit-facts:start -->

```json
{
  "v": 1,
  "cause": "mishandled-error",
  "impact": "financial-error",
  "confidence": { "cause": "high", "impact": "high" },
  "escape": "no-test",
  "caughtBy": "campaign",
  "campaign": "refunds-2026-10",
  "testType": "contract",
  "test": {
    "file": "src/checkout/__testkit__/refund.contract.testkit.ts",
    "title": "a refund pays out once"
  },
  "foundAt": "4f0c2d1e9b7a4c3d8e6f5a4b3c2d1e0f9a8b7c6d",
  "fault": { "path": "src/checkout/refunds.ts", "lines": [41, 58] },
  "area": { "id": "checkout", "name": "Checkout", "share": 0.14 },
  "introducedBy": "9b7a4c3d8e6f5a4b3c2d1e0f9a8b7c6d5e4f3a2b",
  "introducedMethod": "blame"
}
```

<!-- testkit-facts:end -->

</details>

The file names, the commits, and the campaign in this body are examples.

The reproduce command runs from the repo root. When testkit.config.ts is in a subdirectory, the command carries --dir <directory>, and the --file path is relative to that directory. The Test: and Fault: lines always give paths from the repo root. A fault on one line reads line 41.

Facts block

The facts block is one JSON object between two marker comments. It sits in a collapsed section named Testkit facts, and tools read it. Facts lists what Turnstone reads from it.

All paths in the block use / and are relative to the repo root. Commits are full commit ids.

The package ships the full format, with the rules for a tool that reads it, as node_modules/@gfdelarue/testkit/docs/bug-records.md.

What testkit bug file checks

testkit bug file runs the test at the checked-out commit and files the issue only when the test fails. It files nothing in these cases.

The token needs permission to create issues and labels. Set the token lists the variables that Testkit reads it from.

Secrets

Testkit redacts the secrets it recognises in the title, in every section, and in test.title. It does so when it records the bug and again when it files the issue. Testkit cannot recognise every secret, so keep credentials, tokens, and passwords out of the record.

Testkit does not rewrite a path or an area's name. It stops in these cases.

• testkit bug record rejects a record whose test.file or fault.path holds text that Testkit redacts.
• testkit bug file stops when the area's name or the reproduce command holds such text.
• Testkit files no bug under a failure title that holds such text. Rename the test, or give the assertion a label.

Known failures

A known failure is a failing test that an open issue with the testkit label covers. bugs.knownFailures in testkit.config.ts decides what a known failure does to a run.

export default defineConfig({
	bugs: { knownFailures: "allow" },
});

Use "allow" to keep CI passing while a filed bug waits for its fix. These rules apply.

• An issue covers a failure when its facts block gives the failing file as test.file and the failure's title as test.title. An issue with no test.title covers the whole file.
• Closing an issue makes its test count again.
• A failure that names no test is always new. A crash, a timeout, and a file that does not load are such failures.
• When Testkit cannot read the issues, every failure is new.
• A run that fails outside its tests fails under both values.

With "allow", the run reads the open issues, so testkit run needs a token. Pass a run whose failures are all filed shows the CI step.

Where the evidence lives

The issue points to evidence where it already is. Do not copy it somewhere new.

Read a run's results shows how to find a file's output in a run.

Turnstone holds none of this evidence. What Turnstone keeps lists the facts that Turnstone stores.

What a report never does

• It never skips the failing test.
• It never asserts the broken behaviour so that the test passes.
• It never reports a failure without a cause. A failing test with no root cause is a lead, and the agent keeps working on it.