FiretrailDocs
Open app

Connecting GitHub

Firetrail plugs into GitHub the polite way: it installs as a GitHub App on the repositories you choose, reads pull requests as they open, and posts its findings back as a check and a comment. This page explains exactly what that connection does — and just as importantly, what it doesn't do.

Connecting a repository

  1. In the app, create a project and choose Connect GitHub.
  2. GitHub walks you through installing the Firetrail app. You pick which repositories it can see — one, several, or all of them.
  3. Back in Firetrail, pick the repository this project should analyze.

The number of repositories you can connect depends on your plan: one on Free, two on Starter, unlimited on Pro and Business.

What Firetrail can (and can't) do

Firetrail requests the minimum GitHub permissions it needs:

  • Read pull requests — the diff, title, description, and commits, so it has something to analyze.
  • Post checks and comments — so it can report what it found on the PR itself.

It cannot push code, merge PRs, or modify your repository. You can see and manage the installation from your GitHub settings at any time, and uninstalling it there cuts off access immediately — and the organization's owners get an email listing the affected repositories, the same as when repositories are removed from the App or the App is suspended. In the app, the GitHub accounts section of your settings shows every connected installation and lets you disconnect one.

What happens when a PR opens

When a pull request opens (or gets new commits), GitHub notifies Firetrail and an analysis starts automatically:

  1. A check named "Firetrail — Future Debug Cost" appears on the PR in an in-progress state.
  2. Firetrail runs the full analysis pipeline — the built-in checks, the score, the coverage estimate, and (on paid plans) the review team.
  3. The check completes with a summary of the result. Its conclusion is always neutral — GitHub-speak for "informational" — so it can never block the merge, no matter how high the score is. (Teams that want real blocking can opt into Merge Checks.)

What happens when you push more commits

Pushing to an open PR re-analyzes it, with two small courtesies so a busy PR doesn't burn one analysis per commit:

  • Pushes are batched. Firetrail waits for a few minutes of quiet after your last push, then analyzes the newest commit. In the meantime that commit shows the Firetrail check in a queued state, so you can tell the push was seen.
  • One analysis at a time. If an analysis of the same PR is still running when you push, the new commit waits its turn and is analyzed right after — it is never skipped. A commit that got superseded by a newer push before its turn has its check closed as skipped; the newest commit always carries the real result.

The summary comment

Firetrail maintains one comment per pull request — no comment spam. The comment shows:

  • the Future Debug Cost score and its one-line summary,
  • a PR Quiz status line, where available,
  • the review team's verdicts — first, because that is the part worth reading,
  • what the PR ships that its description doesn't mention (intent drift),
  • the top findings Firetrail is confident about — each one as a cause (what in the diff creates the risk), the consequence (what breaks later if it ships as-is), and the solution (the one action that removes it), and
  • the itemized score breakdown (what added points and why), collapsed.

When an analysis re-runs, the same comment is updated in place. Your PR conversation stays readable.

Syncing existing pull requests

Connecting a repository doesn't automatically analyze PRs that were already open. To bring those in, click "↻ Sync open PRs" on the Pull requests list. Firetrail imports every open PR from the repository and queues each one for analysis.

Re-running an analysis

Open any PR in the app and click "↻ Re-run analysis". Firetrail fetches the latest commit and analyzes it fresh. This is useful after pushing new commits, or to refresh a PR that was analyzed before you enabled a feature (like the review team). Results are cached per commit, so re-running a PR that hasn't changed is instant and costs nothing.