An offline, dependency-free reporter that selects saved evaluation cases by a reproducible seed, explicit risk/category/failure-history/coverage quotas, and a validated holdout boundary. It returns only opaque case and source identifiers; it does not run evaluations or modify a dataset.
This walkthrough uses the tool's public README and checked-in example files. Run the command from a repository checkout with Node.js 22+.
Run the checked-in example
node bin/eval-dataset-sampler.mjs --root examples --dataset clean.dataset.json --plan plan.json
node bin/eval-dataset-sampler.mjs --root examples --dataset leak.dataset.json --plan plan.json
node bin/eval-dataset-sampler.mjs --root examples --dataset clean.dataset.json --plan plan.json --report sample.report.json
node bin/eval-dataset-sampler.mjs --help
npm run checkRead the result
The clean check exits 0 with status: pass and two selected IDs. The leak check exits 1 with status: fail and an empty sample. Completed and incomplete dataset checks emit one JSON report on stdout and a fixed, human-readable status and finding count on stderr. Add --json to suppress only that human report summary; invalid configuration still leaves stdout empty and emits a fixed diagnostic on stderr. --help alone prints usage and exits 0. The summary contains no input values or identifiers. Optional --report FILE writes the same JSON bytes to a named file under the declared root; without it, no file is written. Import sampleDataset, TOOLID, RULESEVERITY, or exitCodeFor from src/index.mjs; file-based callers can import checkSample from src/check.mjs. Both accept an injected now clock function, defaulting to Date.now.
Scope
The sampler does not judge case quality, calculate content digests, infer missing failure history, create evaluation data, run a model, guarantee statistical representativeness, or decide that a caller was authorized to expose selected opaque IDs. It enforces only the explicit evidence, quotas and holdout boundary described here.