An offline, dependency-free reporter that selects saved evaluation cases by a reproducible seed, explicit risk/category/failure-history/coverage quotas, and a validated holdout boundary. It returns only opaque case and source identifiers; it does not run evaluations or modify a dataset.
This walkthrough uses the tool's public README and checked-in example files. Run the command from a repository checkout with Node.js 22+; inspect the source before using it on your own files.
Run the checked-in example
node bin/eval-dataset-sampler.mjs --root examples --dataset clean.dataset.json --plan plan.json
node bin/eval-dataset-sampler.mjs --root examples --dataset leak.dataset.json --plan plan.json
node bin/eval-dataset-sampler.mjs --root examples --dataset clean.dataset.json --plan plan.json --report sample.report.json
node bin/eval-dataset-sampler.mjs --help
npm run checkRead the result
The clean check exits 0 with status: pass and two selected IDs. The leak check exits 1 with status: fail and an empty sample. Completed and incomplete dataset checks emit one JSON report on stdout and a fixed, human-readable status and finding count on stderr. Add --json to suppress only that human report summary; invalid configuration still leaves stdout empty and emits a fixed diagnostic on stderr. --help alone prints usage and exits 0. The summary contains no input values or identifiers. Optional --report FILE writes the same JSON bytes to a named file under the declared root; without it, no file is written. Import sampleDataset, TOOLID, RULESEVERITY, or exitCodeFor from src/index.mjs; file-based callers can import checkSample from src/check.mjs. Both accept an injected now clock function, defaulting to Date.now.
Where this check stops
The sampler does not judge case quality, calculate content digests, infer missing failure history, create evaluation data, run a model, guarantee statistical representativeness, or decide that a caller was authorized to expose selected opaque IDs. It enforces only the explicit evidence, quotas and holdout boundary described here.
Before adapting the command to your own workflow, review the accepted inputs, exit codes and safety boundaries in the README.
Compiled with AI assistance from checked-in public documentation and example scripts. Run the example and review the repository's current documentation before relying on its result.