Skip to content

fix!: fail closed when pip-audit cannot run; reject package_manager pip - #85

Merged
lhoupert merged 5 commits into
mainfrom
fix/pip-audit-fail-closed
Oct 6, 2026
Merged

lhoupert merged 5 commits into
mainfrom
fix/pip-audit-fail-closed

Conversation

@lhoupert

@lhoupert lhoupert commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

When pip-audit could not run, the action reported "✅ No vulnerabilities found" and exited 0, and package_manager: pip audited the action's own environment. "Could not run" covers a failed export, a missing requirements file or a crash, because pip-audit uses exit 1 for both crashes and findings. Now:

  • a run without a pip-audit JSON report fails the step and says "pip-audit did NOT run" in the summary, the annotations and the PR comment;
  • pinned exports are audited as written (--no-emit-project, --disable-pip);
  • pip mode fails with a pointer to requirements mode;
  • the bandit fork is pinned to a commit without the nested checkout that deleted lockfiles generated in CI.

Fixes #84.

For reviewers: setups that were never really audited now turn red. Examples are default inputs with no requirements.txt, pipenv without a Pipfile.lock, and locks with fixable CVEs that were never reported.

Author attestation

  • I am a human, these are my changes, and I have reviewed and understood every change and can explain why each is correct.

AI-assisted: Claude Code wrote the change, the tests and the local end-to-end checks.

A failed uv/poetry/pipenv export, a missing requirements file or a
pip-audit crash (exit 1, the same code as "vulnerabilities found") was
reported as "No vulnerabilities found" with exit 0. Any run without a
pip-audit JSON report now exits non-zero and writes "pip-audit did NOT
run" to the step summary.

- uv export drops the caller's project (--no-emit-project), and pinned
  exports (uv, poetry, pipenv) are audited with --disable-pip, so
  pip-audit no longer builds the project in a venv (--no-deps was a
  no-op without it). requirements mode keeps pip resolution.
- pipenv without Pipfile.lock fails instead of auditing an empty list.
- package_manager pip ran `pip freeze` inside the action's own uv
  environment, so it audited the action's dependencies, not the
  caller's. It now fails with a pointer to requirements mode.
- validate_results.py errors when a pip-audit case has no report or
  audited zero dependencies.

Refs #84

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

✅ All test workflows behaved as expected

13 passed, 0 failed

Test Name Expected Actual Bandit pip-audit Result
01 requirements · flat · clean success success — — ✅
02 requirements · src/ · bandit HIGH failure failure B105, B404, B602 — ✅
03 requirements · src/+scripts/ · bandit HIGH + pip-audit failure failure B105, B404, B602 click, cryptography, idna, requests, urllib3 ✅
04 uv · flat · clean success success — — ✅
05 uv · src/ · pip-audit vuln failure failure — click, idna, requests, urllib3 ✅
06 uv · src/+scripts/ · bandit MEDIUM failure failure B324, B506 — ✅
07 poetry · flat · clean success success — — ✅
08 poetry · src/ · bandit MEDIUM + pip-audit failure failure B105, B324 cryptography, idna, requests, urllib3 ✅
09 pipenv · flat · clean success success — — ✅
10 pipenv · src/+scripts/ · bandit HIGH failure failure B404, B602 — ✅
11 requirements · flat · clean (root working dir) success success — — ✅
12 uv · flat · bandit-only (no pip-audit) failure failure B404, B602 disabled ✅
14 uv · flat · low threshold (B101 assert) failure failure B101 disabled ✅

@lhoupert lhoupert changed the title fix: fail closed when pip-audit cannot run; reject package_manager pip fix!: fail closed when pip-audit cannot run; reject package_manager pip Oct 6, 2026
lhoupert and others added 4 commits October 6, 2026 15:22
The Pipfile.lock check only looked in the current directory. pipenv keeps
the lock at <Pipfile>.lock, so it now honours PIPENV_PIPFILE.

The lock check and the executable lookup now run before the temp
requirements file is created, and a failed export deletes that file.

Refs #84

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…udits

When pip-audit could not run, the action exited before it wrote the
bandit section, emitted annotations or updated the PR comment. A red job
could then sit next to a stale "all clean" comment. It now renders the
whole report with a "pip-audit did NOT run" section, emits an ::error::
annotation with the reason, updates the comment and then exits non-zero.

A pip-audit-report.json left by an earlier run is deleted first, so the
always() upload never publishes stale results.

A valid report that audited zero dependencies, or skipped all of them,
no longer says "No vulnerabilities found". The section shows the
audited and skipped counts, and a ::warning:: is emitted when nothing
was audited. Such a run does not fail: a project with no dependencies
is legitimate.

Refs #84

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
It had no callers, and its pip_stdout="[]" default is no longer a valid
pip-audit report. The clean-audit mock now reports one audited
dependency, as a real clean run does.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
lhoupert/bandit-action e2d4893 is 18022d5 minus its "Checkout repository"
step. That nested actions/checkout ran with clean: true (git clean -ffdx
and git reset --hard) in the caller's workspace. It deleted lockfiles the
caller generated before this action (integration cases 04, 07 and 10),
and it reverted the `uv lock --upgrade` in ci.yml's self-audit before
pip-audit ran.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Comment thread action.yml
if: contains(inputs.tools, 'bandit')
continue-on-error: true
uses: lhoupert/bandit-action@18022d5292d04b21fae1bfa44597b94402ba7365
uses: lhoupert/bandit-action@e2d48932beda5cc8c50cb45ed39f8d873ef2d365
@lhoupert
lhoupert merged commit 3f6afd3 into main Oct 6, 2026
20 checks passed
@lhoupert
lhoupert deleted the fix/pip-audit-fail-closed branch October 6, 2026 14:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: pip-audit can pass without auditing the project's dependencies

2 participants