Repository navigation
docs: add Factory code review recipe #897
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
rachaelrenk
merged 6 commits into
factory/factory-use-cases-structure
from
factory/factory-use-case-code-review
Oct 21, 2026
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
fe605af
docs: add Factory code review recipe
warp-agent-staging[bot] 009f116
docs: remove scaffold marker
warp-agent-staging[bot] 4204e5c
docs: clarify provider review triggers
warp-agent-staging[bot] dea46bf
docs: clarify code review recipe and provider setup
rachaelrenk 5f8da74
docs: route review reports through the factory foreman
rachaelrenk 71659bb
docs: clarify review setup and test instructions
rachaelrenk File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,8 +1,129 @@ | ||
| --- | ||
| title: Review pull requests with a factory | ||
| title: Configuring pull request review in your factory | ||
| description: >- | ||
| Configure a factory to inspect pull requests, identify potential bugs, and | ||
| return actionable review findings. | ||
| Automate pull request reviews with your factory to identify bugs, security | ||
| risks, and missing tests and post actionable findings. | ||
| --- | ||
| import { Tabs, TabItem } from '@astrojs/starlight/components'; | ||
|
|
||
| {/* This scaffold is completed by its dependent guide PR. */} | ||
| Automate pull request review in your factory to check code changes against the acceptance criteria recorded in the issue or pull request, repository conventions, and security requirements. Use this pattern for independent review of code written by people or agents. | ||
|
|
||
| This [recipe](/factories/use-cases/) routes pull requests to your factory's foreman, which asks the review agent to inspect the change and publishes its report. The review agent reports to the foreman rather than posting on the pull request. The automation prompt limits this workflow to review without implementation, approval, or merging. Keep required approvals and merge permissions in the code host. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| * A [factory](/factories/quickstart/) connected to [GitHub](/factories/integrations/github/), [GitLab](/factories/integrations/gitlab/), or [Azure DevOps](/factories/integrations/azure-devops/). | ||
| * Repository credentials that can read code, checks, and review discussions, and post findings. | ||
| * A review policy with required validation commands, security checks, and finding thresholds. | ||
| * A test pull request or merge request with a known defect. | ||
|
|
||
| ## Recommended configuration | ||
|
|
||
| Use the following table to choose the trigger, agent, skills, settings, and expected output for your review automation. | ||
|
|
||
| | Part | Recommendation | | ||
| | --- | --- | | ||
| | Trigger | A pull request becomes ready or is created, depending on the provider | | ||
| | Agents | Your factory's foreman coordinates the review agent and publishes its findings | | ||
| | Skills | Repository-specific review policy and validation commands | | ||
| | Settings | Filter by repository and target branch. Choose a different model or harness from implementation when possible | | ||
| | Output | Actionable findings with affected lines, a recommendation, and validation results | | ||
|
|
||
| ## Set up pull request review | ||
|
|
||
| 1. Use your factory's existing [foreman and review agents](/factories/factory-agents/#the-default-agents). Keep both agents' shared instructions intact. | ||
| 2. Add a review-policy skill under `agents/review/skills/`, using your agent's directory name if different. Include validation commands, relevant security or migration checks, and which findings require changes. See [factory skills](/factories/factory-skills/) for the file format. | ||
| 3. Add or edit an automation for your code host. Target the foreman with `agent: foreman`, replacing `foreman` if yours has a different name. Put this workflow's review-only instructions in the automation prompt. | ||
|
|
||
| Reuse an existing automation if it already handles those requests. Check for overlapping triggers before adding another route. | ||
|
|
||
| Select your code host from the tabs below, then follow the instructions to configure the automation. | ||
|
|
||
| <Tabs> | ||
| <TabItem label="GitHub"> | ||
|
|
||
| This automation starts when a pull request is opened ready for review or changes from draft to ready. Replace `acme/api-service` and `main` with your repository and target branch. | ||
|
|
||
| ```markdown title="automations/review-ready-prs/automation.md" | ||
| --- | ||
| agent: foreman | ||
| triggers: | ||
| - provider: github | ||
| event: pull_request_ready | ||
| filter: | ||
| repos: [acme/api-service] | ||
| base_branches: [main] | ||
| labels: | ||
| not_in: [skip-factory-review] | ||
| --- | ||
|
|
||
| Coordinate a review-only task. If the pull request is a draft or is no | ||
| longer open, stop without posting a review. Treat this event as a fresh | ||
| review request, even if an earlier revision was reviewed. Ask the factory's | ||
| review agent to inspect the current head commit against acceptance criteria | ||
| in the source issue or pull request and repository guidance, run relevant | ||
| validation, and return its report to you. Publish the report on the pull | ||
| request as comments, not an approval or rejection. Include affected lines, | ||
| impact, recommended corrections, | ||
| validation results, and anything unverified. Stop after publishing. | ||
| Do not edit code, dispatch implementation, approve, or merge. | ||
| ``` | ||
|
|
||
| The optional `skip-factory-review` label excludes a pull request when the event fires. For bot-authored events, see the published [GitHub bot-event policy](https://docs.warp.dev/factories/integrations/github/#events-from-bots-dont-start-work). | ||
|
|
||
| </TabItem> | ||
| <TabItem label="GitLab"> | ||
|
|
||
| Copy `agent: foreman` and the Markdown review prompt from the GitHub tab's `automations/review-ready-prs/automation.md` example. Replace its entire `triggers` list with the configuration below, using your GitLab project path and target branch in place of `acme/api-service` and `main`. | ||
|
|
||
| ```yaml title="GitLab review trigger" | ||
| triggers: | ||
| - provider: gitlab | ||
| event: merge_request | ||
| filter: | ||
| repos: [acme/api-service] | ||
| base_branches: [main] | ||
| actions: [open] | ||
| ``` | ||
|
|
||
| `merge_request` also covers updates, approvals, and other actions. The `actions: [open]` filter limits initial review to newly opened merge requests. This trigger has no label or draft filter, so keep the prompt's instruction to skip drafts. | ||
|
|
||
| </TabItem> | ||
| <TabItem label="Azure DevOps"> | ||
|
|
||
| Copy `agent: foreman` and the Markdown review prompt from the GitHub tab's `automations/review-ready-prs/automation.md` example. Replace its entire `triggers` list with the configuration below. Replace `AZURE_DEVOPS_REPOSITORY_ID` with the repository's UUID, not a GitHub-style `owner/repo` path, and replace `main` with the target branch. | ||
|
|
||
| ```yaml title="Azure DevOps review trigger" | ||
| triggers: | ||
| - provider: azure_devops | ||
| event: pull_request_created | ||
| filter: | ||
| repos: [AZURE_DEVOPS_REPOSITORY_ID] | ||
| base_branches: [main] | ||
| ``` | ||
|
|
||
| This trigger has no label or draft filter. Keep the prompt's instruction to skip drafts. See [Azure DevOps events and filters](/factories/integrations/azure-devops/#supported-events-and-filters) for setup and scope. | ||
| </TabItem> | ||
| </Tabs> | ||
|
|
||
| 4. Apply the definition changes before testing. Save edits in the factory dashboard for a Warp-managed factory, or merge them into the configured production branch for a GitHub-backed factory. Confirm validation succeeds and the changes are applied. See [where the factory definition lives](/factories/factory-as-code/#where-the-definition-lives). | ||
| 5. Test the selected event with a pull request or merge request that contains a known defect and matches every filter. | ||
| * **GitHub** - Mark a draft pull request as ready for review, or open a non-draft pull request. To test the optional exclusion, repeat with `skip-factory-review` applied before opening the pull request or marking it ready. | ||
| * **GitLab** - Open a non-draft merge request. | ||
| * **Azure DevOps** - Create a non-draft pull request. | ||
|
|
||
| Confirm the review agent returns its findings to the foreman, the foreman posts the defect on the pull request, and neither agent changes the branch or starts implementation. Test exclusions too: GitHub's skip label should prevent a run, while draft requests on GitLab and Azure DevOps may start a run but should produce no review. | ||
| 6. Add re-review events only if the workflow needs them. Use `pull_request_synchronized` for GitHub, `merge_request` with `actions: [update]` for GitLab, or `pull_request_updated` for Azure DevOps. GitLab and Azure DevOps update events are not limited to new commits. Keep the same repository and target-branch filters, retain the draft and closed-request checks, and decide how the agent should reply to or update earlier findings instead of duplicating them. Push a fix and confirm the next review checks the revised code. | ||
|
|
||
| ## Example workflow | ||
|
|
||
| A pull request changes a cache key from `account_id:region` to `account_id`. When the author marks it ready, the foreman requests a review. The review agent finds that two regions now share one entry and reports the collision scenario and expected key shape to the foreman, which posts the finding on the pull request. | ||
|
|
||
| After the author pushes a fix, a configured re-review event requests another check of the current head commit. The review agent verifies the new key and the foreman publishes the result. A person still decides whether to approve and merge. | ||
|
|
||
| ## Related pages | ||
|
|
||
| * [Factory use cases](/factories/use-cases/) - Compare review with triage, implementation, and verification recipes. | ||
| * [Factory automations](/factories/automations/) - Configure pull request events and filters. | ||
| * [Factory agents](/factories/factory-agents/) - Configure the review role and choose a separate model or harness. | ||
| * [GitHub integration](/factories/integrations/github/) - Review supported pull request events and permissions. | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.