Citation Grounding Benchmark: Public Worked Example

Benchmark human-labeled answer claims against cited, locally exported evidence. The tool keeps three things separate: the supplied human label, a deliberately narrow deterministic numerical signal, and an optional locally supplied grader label. A citation's mere presence never proves a claim supported.

This walkthrough uses the tool's public README and checked-in example files. Run the command from a repository checkout with Node.js 22+.

Run the checked-in example

node bin/citation-grounding-benchmark.mjs --input examples/supported.json --json
node bin/citation-grounding-benchmark.mjs --input examples/unsupported.json --json
node bin/citation-grounding-benchmark.mjs --input examples/supported.json --grader examples/grader-disagrees.json --json
npm run check

Read the result

The first example passes (exit 0). The unsupported fixture and the local grader disagreement fail (exit 1). examples/missing-citation.json is incomplete (exit 2). The CLI writes only the JSON report to stdout. Without --json, a short operational summary goes to stderr. It reads files, but writes no files and never calls a model, provider, database, browser or network service.

Scope

Each bound accepts exactly N and reports N+1 as incomplete. The library accepts an injected clock for deterministic testing; the CLI uses a monotonic clock. Timeout is cooperative, not preemptive: a synchronous JSON parse, filesystem call, native string scan or sort can overrun it before the next checkpoint. No time value is written into the report.