Review a pull request

A review is an agent thread that reads one pull request at one commit and reports the problems it finds. The portal posts them on the pull request as a GitHub review: findings on changed lines become inline comments, and the rest go in the review’s summary.

A review starts in one of three ways:

• from the portal, with Review (see Start a review),
• automatically, when a pull request opens or gets new commits, if the repository is set to (see Automatic reviews), or
• from GitHub, with a comment such as @pcsmate review on the pull request (see Ask from GitHub).

Before you start

• The project’s repositories are connected through the GitHub App (see Connect GitHub).
• Daytona is connected (see Connect Daytona). Each review runs in its own sandbox.
• For automatic reviews and review comments, the portal receives the GitHub App’s webhooks (GITHUB_WEBHOOK_SECRET; see Set up the GitHub Apps). Until then, the Automatic reviews card says so, and only Review works.

Start a review

1. Click Reviews, the pull request icon in the icon rail.
2. Choose the project in the menu at the top right. Open pull requests lists the open pull requests of every repository in the project, most recently updated first.
3. Click Review next to a pull request.

The row shows Reviewing while the agent works. A review takes a few minutes, longer for a large pull request. When the pull request was reviewed at its current commit, the row shows Reviewed at and the commit, and the button reads Review again.

The portal refuses a review when:

• the pull request is closed or merged,
• a review of the same commit is already running, or
• the project has spent its review budget for the month (see Cost).

Automatic reviews

On Reviews, the Automatic reviews card has a menu for each of the project’s repositories:

Choice	Reviews
Off (the default)	Only when someone asks
When a pull request opens	When a pull request opens, is reopened, or is marked ready for review
On every push	All of the above, and again each time the pull request gets new commits

Draft pull requests are not reviewed automatically until they are marked ready. A commit that already has a review is not reviewed again automatically, so GitHub redelivering an event does not cost a second review. When the month’s budget is spent, automatic reviews stop without a comment.

The review belongs to the person who opened the pull request when they are a member of the workspace, otherwise to the person who connected the GitHub account or organization.

Ask from GitHub

Comment @<app> review on a pull request, where <app> is the GitHub App’s slug: @pcsmate review for the app named pcsmate. The Automatic reviews card shows the handle. Upper and lower case do not matter, and the request can be anywhere in the comment.

• Only people with write access to the repository can ask: owners, organization members and collaborators. Other comments, and comments from bots, are ignored.
• A comment always starts a review, whatever the repository’s setting, even of a commit that was reviewed before.
• The app reacts to the comment with 👀 when the review starts. When it cannot start, the app answers with a comment that says why, such as Could not start a review: Pull request #7 is closed.

What the agent does

The review’s sandbox checks out the pull request’s head commit, with the base branch available to compare against. The agent:

• reads the changed code and the code it touches, beyond the diff,
• follows the repository’s review rules and security guidance in AGENTS.md, REVIEW.md and SECURITY.md files (see Add review rules to a repository), read from the base branch so a pull request cannot change its own rules,
• does not run tests, builds or installs, and does not change files or push, and
• ends with a summary and a list of findings.

Each finding has a priority:

Priority	Meaning
P0	Blocks the merge: data loss, a security hole, or a crash or outage on a main path
P1	A bug users will hit, or a change that breaks the build, deploys or other callers
P2	A real bug in an edge case, or a clear performance or maintainability problem
P3	Minor. Not reported.

The portal posts P0 to P2 findings and drops the rest.

Add review rules to a repository

The agent reads three kinds of file. Each can be at the root of the repository, where it covers every file, or in a folder, where it covers that folder and the folders below it.

File	What the agent uses	Which files apply to a changed file
AGENTS.md	Only the rules under a ## Code Review Rules heading	Every one that covers it
REVIEW.md	The whole file, as review rules	Every one that covers it
SECURITY.md	The whole file, as security guidance and threat model: what an attacker can reach, what must stay secret, what counts as a security problem and how severe it is	Only the nearest one: in the file’s own folder, else the closest parent folder

When two rules disagree, the one in the file nearer to the changed file wins. The agent reads every one of these files from the pull request’s base branch, so a pull request that adds or changes them is reviewed by the rules from before it; its changes apply to the pull requests after it merges.

Use REVIEW.md when you keep review rules apart from the instructions for coding agents, and an AGENTS.md section when you keep them together. For example, backend/REVIEW.md:

- Every query must filter by the viewer's workspace.
- A schema change needs an Alembic migration.

or the same rules in backend/AGENTS.md:

## Code Review Rules

- Every query must filter by the viewer's workspace.
- A schema change needs an Alembic migration.

A SECURITY.md at the root, such as:

## Threat model

- Users are untrusted. Each belongs to one workspace and must never read another workspace's rows.
- API keys stay on the server. The browser gets masked values and signed URLs.
- Sandboxes run untrusted code and must not hold the portal's credentials.

Keep the rules short, and leave formatting and lint checks to CI. A SECURITY.md in a folder replaces the root one for that folder’s files, so repeat what still applies there.

On GitHub

The portal posts one review on the pull request, as the GitHub App (<app>[bot]), with Comment as its type: it never approves or requests changes.

While the review runs, the pull request shows a MateFactory review check, linked to the review thread. When the review ends, the check passes with No issues found, or ends as neutral with the count of findings, such as Found 1 P0, 2 P2, or with The review did not finish. It never fails, so it does not block a merge. The check needs the app’s Checks permission; without it, reviews run with no check.

• The review’s text starts with MateFactory review and the agent’s summary. Findings on lines that the pull request did not change are listed under Findings outside the changed lines. It ends with the reviewed commit and a link to the review thread.
• Each finding on changed lines is an inline comment titled with its priority, such as [P1] Charges twice on retry. When the agent proposes a fix for exactly those lines, the comment has a suggestion you can commit from GitHub.

Read a review in the portal

Reviews lists the project’s reviews, newest first, and opens the newest one. Click a review to open or close it. Each shows:

• the count of findings for each priority, or No issues,
• its status: Reviewing, Posted or Failed, and its model cost,
• what started it: from the portal, when the pull request opened, on a new commit or on request in a comment, and whose it is,
• links to its Thread, where you can follow the agent’s work and ask it questions, and to the Review on GitHub,
• the summary and every finding, with its file and lines and any suggested fix.

The iPhone and Android apps have the same page in their Reviews tab, for one project at a time: pick the project at the top.

A review fails when the agent stops or errors, when its reply has no result in the expected form, or when GitHub refuses the review. The review shows why. Its thread stays, so you can read what the agent did.

Cost

A review costs model usage and sandbox time. When a review finishes, its sandbox is stopped and archived, so finished reviews do not fill the Daytona organization’s disk limit. To ask the agent about a review, open its Thread and start the sandbox; an archived sandbox takes longer to start. The model cost of each review is shown on the review.

Each project has a review budget of $10.00 of model spend per calendar month (UTC). The Reviews card shows how much of it is used. Once the month’s spend reaches the budget, new reviews are refused until the next month. A review that is already running finishes, even when it takes the spend past the budget.

Not available yet

These are planned:

• personal review settings, and a priority threshold and budget for each repository,
• Fix buttons that have an agent fix a finding and push the fix,
• a separate security review.