Skip to content

Conversation

@AdamKorcz
Copy link
Contributor

What kind of change does this PR introduce?

New check

  • PR title follows the guidelines defined in our pull request documentation

  • Tests for the changes have been added (for bug fixes/features)

Special notes for your reviewer

This PR adds a new check which analyzes a projects release history and looks for vulnerabilities in the direct dependencies of a project that were known at the time of release. A project can score 10 if none of its max 10 most recent releases had known vulnerabilities at the time of each release. This check will encourage projects to remove all known vulnerabilities from its direct dependencies which will lead to more secure products for downstream users. In addition, some security frameworks require that releases don't contain known vulnerabilities.

Under the hood, the check works like this:

  1. First, the check collects the commits of each of the projects last 10 releases.
  2. Next, the check collects the tar balls of each of the 10 commits.
  3. The check then downloads each tar ball.
  4. With each tar ball, the check scans the projects direct dependencies using osv-scalibr.
  5. After collecting all the dependencies, the check sends a batched query to osv.dev with the library names and versions.
  6. The check analyzes the responses from osv.dev to see if they contain vulnerabilities that were published before the release was issued.
  7. The check scores based on its findings.

Later, we can add a mechanism that checks if projects have SBOMs that describe whether a project is actually affected by known vulnerabilities as a way to give projects more control.

To test this with debugging: RELEASES_DEPS_DEBUG=1 SCALIBR_DEBUG=1 go run main.go --repo=https://github.com/owner/repo --checks=ReleasesDirectDepsVulnFree

Does this PR introduce a user-facing change?

Yes

Add new check: Direct dependencies in releases are free of known vulnerabilities at the time of release.

@AdamKorcz AdamKorcz requested a review from a team as a code owner December 9, 2025 17:40
@AdamKorcz AdamKorcz requested review from raghavkaul and spencerschrock and removed request for a team December 9, 2025 17:40
@dosubot dosubot bot added the size:XXL This PR changes 1000+ lines, ignoring generated files. label Dec 9, 2025
@codecov
Copy link

codecov bot commented Dec 9, 2025

Codecov Report

❌ Patch coverage is 59.23345% with 234 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.73%. Comparing base (353ed60) to head (20263fb).
⚠️ Report is 293 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4876      +/-   ##
==========================================
+ Coverage   66.80%   67.73%   +0.92%     
==========================================
  Files         230      255      +25     
  Lines       16602    16231     -371     
==========================================
- Hits        11091    10994      -97     
+ Misses       4808     4354     -454     
- Partials      703      883     +180     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@spencerschrock
Copy link
Member

  • With each tar ball, the check scans the projects direct dependencies using osv-scalibr.
  • After collecting all the dependencies, the check sends a batched query to osv.dev with the library names and versions.
  • The check analyzes the responses from osv.dev to see if they contain vulnerabilities that were published before the release was issued.

Any reason why you dont use osv-scanner for steps 4-5? Which is essentially a wrapper over osv-scalibr and osv.dev already? I'm also not sure of the limitation to direct dependencies here.

I think there are obviously similarities to the Vulnerabilities check, but instead of "are there any vulns now", the question is "Does this project have some process to avoid vulns when releases are made". Is that enough of a difference to warrant a separate check?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL This PR changes 1000+ lines, ignoring generated files.

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants