Skip to content

feat: add preview-first external contribution triage workflow - #1476

Open
Wassim Chegham (manekinekko) wants to merge 2 commits into
mainfrom
manekinekko-external-contribution-triage
Open

Wassim Chegham (manekinekko) wants to merge 2 commits into
mainfrom
manekinekko-external-contribution-triage

Conversation

@manekinekko

Copy link
Copy Markdown
Member

Summary

Adds a GitHub Agentic Workflow (gh-aw) that triages external contributions, meaning issues and PRs from NONE, FIRST_TIMER, FIRST_TIME_CONTRIBUTOR, or CONTRIBUTOR authors. For each one it proposes up to 5 allowlisted type:/area: labels and a short triage comment.

  • Preview by default. Output stays staged unless all of these hold: the workflow source sets staged: false, vars.SCOPE_TRIAGE_PUBLISH_ENABLED == 'true', and (for manual dispatch) preview=false. The hourly schedule also requires vars.SCOPE_TRIAGE_SCHEDULE_ENABLED == 'true'.
  • Trust boundaries:
    • A trusted pre-activation step picks one candidate and passes it to the publisher as an artifact.
    • The agent has read-only access, no bash, and a limited set of GitHub tools scoped to microsoft/scope.
    • A separate publish-triage job re-checks eligibility and the revision fingerprint, then validates the proposal: body limits, no HTML, images, @mentions, or off-repo URLs, and only allowlisted labels.
    • That job runs only after threat detection succeeds.
  • Idempotent: a SHA-256 revision fingerprint is stored in a bot comment. A new revision updates that comment instead of posting another. Comment creation is not retried, so it can't post duplicates.
  • Prompt evaluation: adds a contribution-triage adapter, 5 synthetic cases, a rubric, and manifest/registry entries.
  • .lock.yml was compiled with gh-aw v0.89.21 (--strict --validate).

Demos

N/A. This is a CI workflow with no Portal or CLI changes.

Before

N/A

After

N/A

Testing

  • pnpm vitest run scripts/external-contribution-triage.test.ts scripts/ci-workflow.test.ts: ✅ 64 passed
  • evaluations/static-prompts: test:ts ✅ (35 passed), typecheck ✅, validate-data ✅ (227 cases, 16 targets)
  • Offline fake-transport generation for the 5 triage cases: ✅; deterministic Python evaluators: ✅ 20 observations passed
  • gh aw compile --strict --validate (v0.89.21): ✅
  • uv offline quality evaluation: ⚠️ not run (PyPI unreachable locally)
  • Hosted Copilot preview run: ⏳ not run yet. To check it after merge, dispatch the workflow manually with preview=true (the default).

Documentation and compatibility

  • New: docs/architecture/contribution-triage.md
  • Updated: AGENTS.md, CONTRIBUTING.md (new "Automated triage assistance" section and labels row), docs/architecture/prompt-evaluations.md
  • Breaking changes: none. The workflow does nothing until a maintainer dispatches it or sets the repo variables.

Checklist

  • If Portal features changed, keep CLI capabilities in sync. (N/A)
  • If Portal components changed, update their Storybook stories. (N/A)
  • If database changes require a migration, include up() / down() and keep it CosmosDB-compatible. (N/A)
  • If dependencies changed, update the lockfile and regenerate NOTICE / NOTICE-REVIEW.txt with pnpm notice as needed. (N/A)
  • Video showing the behavior before the suggested change (N/A)
  • Video showing the behavior after the suggested change (N/A)

Adds a gh-aw agentic workflow that proposes type/area labels and a
triage comment for external issues and PRs. Preview (staged) by default;
publishing requires explicit maintainer opt-in via repo variables.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot added type: documentation Documentation additions, corrections, and improvements. area: cicd Build, test, release, and deployment pipelines. language: javascript Work involving JavaScript code, tooling, or dependencies. topic: testing Test coverage, test infrastructure, and validation quality. labels Oct 8, 2026
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown

Test Results (Node.js 22)

test: Run #217

Tests 📝 Passed ✅ Failed ❌ Skipped ⏭️ Pending ⏳ Other ❓ Flaky 🍂 Duration ⏱️
3242 3242 0 0 0 0 0 1m29s

🎉 All tests passed!

Github Test Reporter

Disable public diagnostic reporting and remove conclusion write permissions while preserving threat detection and failure signals.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown

Test Results (Node.js 22)

test: Run #224

Tests 📝 Passed ✅ Failed ❌ Skipped ⏭️ Pending ⏳ Other ❓ Flaky 🍂 Duration ⏱️
3243 3243 0 0 0 0 0 55.6s

🎉 All tests passed!

Github Test Reporter

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: cicd Build, test, release, and deployment pipelines. language: javascript Work involving JavaScript code, tooling, or dependencies. topic: testing Test coverage, test infrastructure, and validation quality. type: documentation Documentation additions, corrections, and improvements.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant