Codex PR Review

Why a native reviewer

The obvious version is a model call with a review prompt I maintain. I wanted OpenAI’s own review behaviour instead, so codex exec review does the reading and the action does everything around it. A prompt does not settle who may ask, which credentials the model sees, what it actually read, or whether to trust the answer. Those questions are most of the code. With agents doing most of my typing, review is the gate I refuse to skip, as I wrote in how I work with coding agents. Nothing runs on open or push, so nobody pays for a review nobody asked for.

From comment to review

scripts/trigger.py parses @gpt review [openai|bedrock] [low|medium|high|xhigh] [full] [force] from a comment. It checks write access, an open pull request, a same-repository head and an allowed base branch, then reacts with eyes. scripts/history.py reads a hidden codex-pr-review-state marker from the last review: the commit, a settings digest, and one fingerprint per finding. scripts/rereview.py stops the run before anything is installed when both match. scripts/write_config.py writes a config.toml with a read-only sandbox, approval_policy = "never" and shell_environment_policy.inherit = "core", so commands Codex runs never see the API key. Then codex exec review --base "$BASE_REF" --json runs. Review mode reports zero token usage, so scripts/usage.py reads the real counts from the session rollouts Codex wrote. scripts/publish_review.py parses the findings and posts the review, and scripts/status.py clears the progress note.

Reading the whole diff

One review pass samples. On a 48,773-line pull request across 406 files, it made 29 model requests in about four minutes, and all six findings sat in two areas. Under review-mode: full, scripts/shards.py cuts the diff into shards of about 2,500 changed lines, never splitting a file and packing tests after their source. Each shard is reviewed in the full head against a synthetic base: HEAD’s tree with only that shard’s files restored to the merge base. Git plumbing builds those commits against a temporary index, so no ref is created. Four passes run at once by default, each with its own CODEX_HOME. A follow-up pass takes any file nobody read, and a final pass looks only for issues that span areas. scripts/coverage.py counts a file as inspected only when a rollout shows Codex reading it, or reading a diff that contained it.

Decisions and costs

• Compute the verdict in scripts/verdict.py, since Codex’s overall_correctness field comes back empty on the pinned versions. Health loses 40, 20, 5 or 1 per P0 to P3 finding, capped so all nits together cost at most one P1. Low confidence turns a clean result into “Needs a full review”. The weights are my judgement, and the verdict is advisory.
• Fingerprint a finding by its path and the sorted meaningful words of its title, so it survives line shifts. Codex is not deterministic, so before v1.3.0 a reworded finding on an unchanged commit counted as fixed. A finding now resolves only when its file changed.
• Keep fetch-depth: 0 with persist-credentials: false. On one repository, a blob:none partial clone fetched 4.4 MB instead of 81 MB. It loads blobs lazily, though, and git diff then fails on a private repository without a credential. I would revisit this, since a huge repository is left with a warm mirror or a skipped review.

Asking for a review

Add an OPENAI_API_KEY secret, or an AWS_ROLE_TO_ASSUME secret naming a role trusted only for the default branch. Then call the reusable workflow from the default branch, where comment triggers run.

on:
  issue_comment: { types: [created] }
  pull_request: { types: [labeled] }
jobs:
  codex:
    permissions:
      contents: read
      pull-requests: write
      issues: write
      id-token: write
    uses: piyush-gambhir/codex-pr-review/.github/workflows/review.yml@v1
    with: { base-branches: main }
    secrets: inherit

Comment @gpt review on an open pull request, @gpt review bedrock xhigh to change provider and effort, or @gpt review full for a sharded pass. A second request for the same commit and settings links the existing review and calls no model, unless it adds force.

Limits and status

Only trusted contributors get reviews, and fork pull requests are skipped because the job holds model credentials. Coverage is a floor: an inspected file was in front of the reviewer, not proven free of bugs. Costs on the meta line are a price table estimate, not a bill. The reusable workflow depends on job.workflow_*, which GitHub Enterprise Server lacks, so a standalone example ships beside it. Seven releases, v1.0.0 to v1.4.0, went out on 29 and 30 September 2026. It is MIT licensed and independent of OpenAI and AWS. Its scripts need only the Python standard library, and 457 unit tests cover them.