Configuration reference

The file

import { app, defineConfig } from "@gfdelarue/testkit/config";

export default defineConfig({
	services: {
		api: app.node({
			build: null,
			start: "npm start",
			port: 4392,
		}),
	},
});

Import the builders from @gfdelarue/testkit/config.

Testkit 0.4.1 ignores a top-level key that it does not know, and prints no warning. A skip block placed beside services does nothing. To confirm that a rule took effect, run testkit discover and read the skipped count.

Top-level fields

execution

execution: {
	taskConcurrency: 4,
	fileTimeoutSeconds: 120,
},

services

A map from a service name to a service. The name appears in run output, and --service takes it. Build each service with app.node(...), app.next(...), or database.fixture(...).

profiles.http

Named k6 options. An HTTP test selects one with defineHttpSuite({ profile: "raw" }, ...).

profiles: {
	http: {
		raw: {
			options: { vus: 1, iterations: 1 },
		},
	},
},

When a run selects a k6 file, every profile must be plain JSON. A profile that holds a function fails config validation. Put auth and header functions in the test file or in a helper beside it.

ui

Targets for browser tests.

Options that a test passes to createUiSandbox win over this block.

lint

Settings for testkit lint. Omit the block to run the default rules.

Set a rule to true, false, or an options object. The options are enabled, include, and exclude. testkitBoundary also takes allowed and trackedOnly.

lint.ui sets the limits for browser test files.

lint: {
	rules: {
		unitTestBoundaries: true,
	},
	ui: { maxLines: 300 },
},

A finding from an enabled rule fails the command. There are no severity levels.

areas

A map from an area id to a named part of the product. testkit bug file uses it to name the area that holds a fault, and puts an area:<id> label on the issue.

areas: {
	checkout: { name: "Checkout", paths: ["src/checkout/**"] },
	billing: { name: "Billing", paths: ["src/billing/**", "src/invoices/**"] },
},

Each source file belongs to one area at most. testkit campaign start lists the source files that no area claims and the files that two areas claim. testkit bug file stops when no area claims the fault's file, or when two areas do, and names the file. Without areas, Testkit files bugs with no area.

An area's share is its part of the repo's source bytes at the commit that holds the bug. Tests, generated files, vendored code, and migrations do not count as source.

bugs

Settings for bug filing, and for runs that meet a filed bug.

bugs: {
	knownFailures: "allow",
	repository: "acme/shop-bugs",
},

Known failures gives the rule for knownFailures. With "allow", the run reads the open issues, so the testkit run step in CI needs a token.

testkit bug file files issues in repository, and testkit run and testkit ci report read the open issues from it. Commit statuses and pull request comments always go to the origin repo. If a Turnstone project reads this repo's bugs, enter the same repo in the project.

artifacts

With no retention block, Testkit keeps every run. It records retention decisions under .testkit/results/retention and each publication under .testkit/results/publications.

fingerprints

Testkit hashes the repo's files to decide when a cache is stale. exclude removes generated files from the hash, and include puts a path back below an excluded directory.

fingerprints: {
	exclude: ["next-env.d.ts"],
},

Patterns use / as the only separator. Testkit always excludes .testkit/ and .next-testkit/.

toolchains

Named rules for the Node and npm versions that run a service's commands. A service selects one with runtime.toolchain.

toolchains: {
	frontendNode: toolchain.node({
		cwd: "frontend",
		detect: "auto",
		install: "download",
	}),
},

With detect: "auto", Testkit reads the Node version from package.json#volta.node, .nvmrc, .node-version, .tool-versions, and package.json#engines.node. It reads the npm version from package.json#volta.npm, package.json#packageManager, and package.json#engines.npm.

Services

app.node(...)

app.node also accepts baseUrl, buildInputs, outDir, readyPath, readyTimeoutMs, readyUrl, runtime, and tsconfig, and every field under Service fields. The types in @gfdelarue/testkit/config define each one.

app.next(...)

Takes the same options as app.node, except entry, outDir, readyPath, and tsconfig. It adds mode, which is "dev" or "start", and api.

frontend: app.next({
	cwd: "frontend",
	mode: "start",
	port: 3000,
	dependsOn: ["api"],
	env: {
		values: { NEXT_PUBLIC_API_URL: "{baseUrl:api}" },
	},
}),

Service fields

env

Environment values for the service process and for every test that targets the service. app.node and app.next take { values, databases }.

Values can use template tokens.

databases expands one service's database into prefixed entries.

env: {
	databases: { api: { prefix: "API", service: "api" } },
},

That block sets API_DATABASE_HOST, API_DATABASE_PORT, API_DATABASE_NAME, API_DATABASE_USER, API_DATABASE_PASSWORD, and API_DATABASE_SSL. An entry in values with the same name wins.

A test reads the values from its engine's context.

envFile and envFiles

Env files to load into the same environment as env: envFile takes one path, and envFiles takes a list.

database

database.postgres(...) gives the service a local Postgres in Docker. The service process receives it as DATABASE_URL. Test the database has a working example.

A template stage is an ordered list of steps.

Docker also runs psql and pg_dump. To use another client image, set TESTKIT_POSTGRES_CLIENT_IMAGE.

databaseFrom

The name of another service. This service receives that service's DATABASE_URL.

dependsOn

The services that must start first.

runtime

runtime: {
	instances: 1,
	maxConcurrentTasks: 4,
	prepare: {
		inputs: ["frontend/src", "frontend/package.json"],
		steps: [{ kind: "command", run: "npm run build", cwd: "frontend" }],
	},
},

skip

Skip rules for the service. Every rule needs a reason.

services: {
	api: app.node({
		port: 4392,
		skip: {
			files: [
				{
					path: "src/checkout/__testkit__/payout.contract.testkit.ts",
					reason: "The payout sandbox is down until Friday",
				},
			],
			suites: [
				{ selector: "ui:billing", reason: "Billing UI is a placeholder" },
			],
		},
	}),
},

To skip one file, prefer defineFile({ skip }) in the file itself. testkit run --ignore-skip-rules runs skipped files anyway.

requirements

Lock rules for the service. Files that need the same lock do not run at the same time.

requirements: {
	suites: [{ selector: "ui:billing", locks: ["billing-ledger"] }],
	files: [
		{
			path: "src/jobs/__testkit__/worker.int.testkit.ts",
			locks: ["global-worker-loop"],
		},
	],
},

To lock one file, prefer defineFile({ locks }).

discovery

discovery.roots assigns directories to the service. Testkit normally gives a test file to the service with the deepest matching cwd, so set roots only when two services share a directory.

Template tokens

Strings in a service can hold these tokens.

These tokens work only in env values.

A bare database token reads the service's own database. {dbUrl:api} reads the database of the service api, which must be in dependsOn.

Testkit rejects a database token in start, cwd, runtime.prepare, and database.template strings. In those places, read DATABASE_URL when the command runs.

Environment limits

Each engine reserves some names and limits how much environment a test can receive.

Every engine also reserves APPDATA, ComSpec, HOME, LANG, LC_ALL, LOCALAPPDATA, PATH, PATHEXT, SystemRoot, TEMP, TMP, TMPDIR, USERPROFILE, XDG_CACHE_HOME, XDG_CONFIG_HOME, and XDG_DATA_HOME.

Testkit compares names without regard to case. It rejects an empty name, a name that contains =, and two names that differ only in case.