Repository navigation
feat: add maintainer-triggered PR description assessment - #4902
Open
KSchlobohm wants to merge 1 commit into
Open
KSchlobohm wants to merge 1 commit into
KSchlobohm wants to merge 1 commit into
Conversation
Port the complete pr-assess workflow with concise reviewer-facing comments, bounded outcome-label updates, focused tests, and usage guidance. Keep the reviewed gh-aw v0.89.21 runtime pin isolated from existing workflows. Assisted-by: GitHub Copilot (model: GPT-6.1 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9ee0ab16-f074-4303-82b9-d11bfad16175
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Authorization exceeds the stated maintainer scope, required labels are not provisioned, and the AI disclosure is incomplete.
3 open findings
What changed in this PR
Adds a maintainer-triggered workflow that evaluates PR description alignment and reports bounded comments and outcome labels.
Changes:
- Adds the
pr-assessagentic workflow and generated lock file. - Adds focused workflow configuration and prompt-contract tests.
- Documents usage and pins the isolated gh-aw runtime dependency.
| File | Description |
|---|---|
.github/workflows/pr-assess.md |
Defines assessment behavior and safeguards. |
.github/workflows/pr-assess.lock.yml |
Provides the compiled GitHub Actions workflow. |
.github/aw/actions-lock.json |
Pins gh-aw setup v0.89.21. |
tests/test_github_workflows.py |
Tests triggers, permissions, outputs, and reporting. |
docs/guides/agentic-sdlc.md |
Documents maintainer usage and outcomes. |
🧠 Review effort: Balanced
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+209
to
+212
| Applying the outcome label is your responsibility, not a recommendation for a | ||
| maintainer. Use `remove_labels` to remove any existing outcome labels other than | ||
| the selected verdict (up to two labels), then `add_labels` with exactly one | ||
| **plain string**: |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds an optional workflow that collaborators with write access or higher trigger with the
pr-assesslabel. It compares the PR description with the code changes, flags material omissions or contradictions, and posts a concise assessment with an outcome label to help reviewers spot gaps.Testing
I tested this workflow on my fork. Automated checks passed (137 tests, 16 skips), and live scenarios covered accurate descriptions, missing behavior, contradictory claims, reassessment, and labels applied to issues or closed PRs. I also tested three reproduced upstream PRs and reviewed the comments for accuracy and respectful wording; one result involved a reviewer judgment difference.
Automated check:
.\.venv\Scripts\python.exe -m pytest tests\test_github_workflows.py -q— 137 passed, 16 skipped.uv run specify --helpuv sync && uv run pytestAI Disclosure
AI disclosure: Prepared and created with GitHub Copilot (GPT-6.1 Sol) under human supervision.