Review Agents Work Best in Testable Review Environments

GitHub's 17 July update to Copilot code review customization quietly changes what a review agent is. Copilot code review now reads custom instructions from the pull request head branch, including copilot-instructions.md, *.instructions.md, agent skills, and AGENTS.md. It also reads REVIEW.md, GEMINI.md, and CLAUDE.md, can run with a repository-level .github/workflows/copilot-code-review.yml setup file, uses a configurable firewall by default, and separates code-review runner settings from Copilot cloud agent runner settings.

That lands beside two other fresh GitHub signals from the same week. Repository-level Copilot usage metrics now show where Copilot coding agent and Copilot code review create pull request activity. Repository admins can also archive pull requests, closing and locking spammy, abusive, or policy-sensitive PRs while preserving admin-visible history.

Read together, the pattern is useful for operators: AI review is becoming a repository-specific runtime, not a generic comment bot. The review agent now has instructions, environment setup, network policy, runner choice, metrics, and moderation edge cases. Those should be designed and tested like the rest of the delivery system.

The repeated angle to avoid

Recent posts here have covered Copilot repository metrics, project views, runtime credentials, dependency cooldowns, Copilot security reviews, alert playbooks, repository overviews, prompt-injection triage, pull request dashboards, AI coding telemetry, accessibility-tree testing, feature flags, cache variants, pointer events, and agent audit trails.

The weak version of this article would repeat the old X needs Y pattern: review agents need better configuration. The sharper thesis is that Copilot code review configuration has crossed from static guidance into executable review infrastructure. Builders should now treat review instructions, setup files, firewall settings, runner types, and PR visibility rules as one operating surface.

Head-branch instructions are powerful and risky

Reading instructions from the pull request head branch is a practical improvement. A team can now propose and validate review guidance in the same branch as the code change. That makes review policy easier to iterate on. It also makes the instruction file itself a change that deserves review.

For a small web product, this matters because repository guidance often encodes business boundaries:

• forms must validate phone and email before CRM sync;
• checkout changes require payment sandbox evidence;
• booking changes need timezone and mobile tests;
• analytics events must preserve attribution fields;
• content changes must avoid raw HTML or unsupported markdown;
• AI-generated changes need owner review before merge.

If a pull request can alter the instructions the reviewer uses to judge that pull request, the team needs a simple rule: instruction changes are not just documentation. They are policy changes.

A safe default is to classify review-instruction diffs separately from product-code diffs:

Changed file	Treat as	Review posture
AGENTS.md, REVIEW.md, CLAUDE.md, GEMINI.md	Review policy	Require repository owner approval before relying on new guidance
*.instructions.md	Model behaviour contract	Check for weakened constraints, broad permissions, or missing evidence rules
.github/workflows/copilot-code-review.yml	Review runtime	Review like CI because it changes what tools and dependencies the reviewer sees
Product code only	Normal code change	Apply the current approved review policy

That does not mean every instruction tweak needs a committee. It means an agent should not be allowed to relax its own guardrails without a human noticing.

Setup files make reviews reproducible

The new copilot-code-review.yml support is important because many useful reviews require context that is not obvious from static code. A reviewer may need generated types, installed dependencies, schema files, lint rules, build artifacts, or framework-specific checks before it can make high-quality comments.

That moves Copilot code review closer to CI. The setup file should answer the same questions a good build job answers:

review_environment_receipt:
  purpose: make Copilot code review evaluate the repository as it actually runs
  setup_file: .github/workflows/copilot-code-review.yml
  required_context:
    - package install command
    - generated API or GraphQL types
    - lint and formatting configuration
    - schema or route manifest generation
  forbidden_context:
    - production secrets
    - customer records
    - payment data
    - broad cloud credentials
  expected_review_checks:
    - form validation paths are visible
    - analytics event schema is available
    - accessibility and security rules can be referenced
  owner_review_required_if:
    - setup adds network access
    - setup installs unpinned tools
    - setup changes generated files used by the reviewer
    - setup differs from normal CI in a customer-path repository

The key is not to make the agent run every test. It is to make the context intentional. A review that cannot see generated types may leave noisy comments. A review setup that downloads broad tools from the network may introduce unnecessary supply-chain risk. A review setup that differs from CI may produce advice that looks authoritative but does not match the real build.

Firewalls create a policy split

GitHub says Copilot code review now runs behind a firewall by default, with network access configurable separately from Copilot cloud agent. That split is healthy. A code reviewer and a coding agent do different jobs and should not automatically share the same network assumptions.

For most small teams, the default review posture should be boring:

1. No production secrets in the review environment. A reviewer should not need live CRM, payment, email, or hosting credentials to comment on a pull request.
2. Network access only when the review genuinely requires it. Package installation, private registry access, or documentation fetching may be valid. Open internet access for every review is not a default requirement.
3. Separate review and write agents. A reviewer that comments on code should not inherit the same runner or network policy as an agent that creates branches, edits files, or opens PRs.
4. Self-hosted runner exceptions need explicit evidence. GitHub notes that self-hosted runners do not currently support the firewall for Copilot code review. If a repository uses them, record why and what compensating controls exist.

This is especially relevant for ecommerce, local-service funnels, and analytics code. The agent may need schemas and tests. It should not need live lead data, production API keys, or customer files.

Metrics should measure instruction quality

Repository-level Copilot metrics add the feedback loop. If review agents are now configurable by repository, operators should measure whether each repository's review environment is improving the right outcomes.

Do not use suggestion count as the primary success metric. More comments can mean the agent is helping, but it can also mean instructions are too vague, setup is incomplete, or the agent keeps detecting the same issue because the team never turns it into a test.

A better weekly review looks like this:

Signal	Useful question	Action if unhealthy
Copilot-reviewed PRs by repository	Is AI review active on the repos where review debt hurts?	Enable or tune review only where it maps to business risk
Repeated comment class	Is the same problem appearing every week?	Convert the rule into tests, lint, schema validation, or component guidance
Comments dismissed as irrelevant	Is setup missing context or are instructions too broad?	Update REVIEW.md or the review setup file
Agent comments on instruction-changing PRs	Did the agent rely on unapproved branch guidance?	Require owner review for instruction diffs
High-risk PRs with no useful comments	Is the review environment blind to customer-path context?	Add schemas, fixtures, generated types, or explicit risk labels

The outcome to watch is not whether the reviewer talks. It is whether human reviewers spend less time catching routine defects and more time judging product, security, and customer-path risk.

Archived PRs are part of the review boundary

The pull request archive feature looks separate, but it belongs in the same operating model. Public pull requests can contain spam, abuse, leaked context, policy-sensitive material, or prompt-like instructions aimed at humans and tools. Archiving closes and locks a PR, hides it from non-admin visitors with a 404, and preserves administrator access.

That gives teams a moderation option between leaving bad input visible and deleting useful historical context. For AI-assisted review, it also creates a boundary: archived PRs should not keep influencing normal review queues, metrics interpretation, or agent context unless an administrator deliberately includes them.

Use a short archival rule:

• archive PRs that are abusive, spammy, legally sensitive, or unsafe to keep public;
• keep an admin-visible note explaining why it was archived;
• exclude archived PRs from normal review-performance reporting;
• review whether any linked issues, comments, or agent summaries copied unsafe content elsewhere;
• unarchive only when visibility is safe and the reason is recorded.

Small teams often think moderation is only a community-management issue. Once agents read repository discussions and pull request context, moderation becomes part of operational hygiene.

What to change this week

Start by adding one explicit review policy file, even if it is short. Put the business-critical surfaces in it: checkout, booking, quote forms, lead routing, auth, customer messaging, analytics attribution, deployment settings, and secrets. Then decide which instruction files are authoritative and which are legacy duplicates.

Next, create or review .github/workflows/copilot-code-review.yml only if the agent needs setup beyond static code reading. Keep it minimal, pin tools where possible, and avoid production credentials. If the repository uses self-hosted runners for review, record the firewall gap and why the choice is still acceptable.

Finally, use repository-level metrics as a calibration loop. The question is not whether Copilot code review is enabled everywhere. The question is whether each repository's review agent has the right instructions, the right environment, the right network boundary, and a measurable effect on the review work humans actually care about.