Harness, scorer, fixtures and reference sets
The benchmark harness, the scorer (src/scorer.ts), the 20 fixtures with findings-attacker-reachable.json, and the scoring documentation (docs/benchmark.md).
2.0 · Data
Versioned evidence with traceable provenance
Dataset 2.0.0 · Snyk VulnBench 2.0 · 108 reference vulnerabilities · one run per configuration and application · Source: published 2.0 paper
Downloads
Published release assets
Main dataset
The authoritative execution, with its parent retained for audit, archive checksums and per-file checksums.
Supplementary dataset
Prompt-swap control, equal-effort arm and Snyk Code repeats; never merged into the main results.
Post-disclosure re-scan
A labelled Snyk Code re-scan after one rule update; never merged into the main results.
Reproducers
One directory per real vulnerability found outside the reference set.
Website evidence tables
The derived tables behind the 2.0 charts, each with a notes file naming its source archive and checksum.
Provenance
Exact source archive
- Dataset version
- 2.0.0
- Execution
- 20260907-vulnbench-v2-deepsec-150-574753a2
- Parent execution
- 20260901-vulnbench-v2-c08730c4
- Archive SHA-256
- b29c643079c8fddc361463ca39e29204e0510d0a16300631784b59772157acd6
- Parent archive SHA-256
- 8a3ee79fec516f6d723fe5d228faf155dfa590cc209b2d3f38e9c3daee0d4bcb
- License
- The harness code and benchmark data are released under the Apache License 2.0; the fixture applications keep their upstream licenses, listed in THIRD_PARTY_LICENSES.md.
Data dictionary
Core concepts
- Run
- One configuration scanning one application once.
- Reference vulnerability
- A curated, attacker-reachable vulnerability with its source, sink and flow.
- Match
- A finding credited to a reference vulnerability under the endpoint rule.
- Outside the reference set
- A finding that matched no reference vulnerability; not proven false.
- Known cost
- Cost recorded by the harness; unavailable is not zero.
Corrections
No corrections published
No corrections published for dataset 2.0.0. The one known reference-set defect is disclosed in the paper; all results use the reference set as frozen for the runs, and the public iguana answer file now carries the correction, so re-scoring with the current public files differs on that entry.