How analysis works
Between the moment a pull request opens and the moment its result appears — usually a minute or two — Firetrail runs a pipeline of independent steps. You don't need to know any of this to use the product, but knowing what happens under the hood makes the results much easier to trust and interpret.
One pass through this pipeline is called an analysis run. Runs start automatically when a PR opens or gets new commits (pushes in quick succession are batched, and a push that arrives while a run is still going waits for it rather than being skipped), and manually when you click "↻ Re-run analysis".
The pipeline, step by step
1. Fetch. Firetrail pulls the PR's latest commit, diff, title, and description from GitHub. This is the raw material for everything that follows.
2. Analyze — everything in parallel. Two groups of AI readers now examine the diff at the same time:
- The classifiers — ten built-in checks, each hunting for one specific risk pattern (mixed intents, untested logic, risky config, hard-to-read code, and so on). Each returns what it found, the evidence, and how confident it is. Your custom classifiers run here too.
- The review team (on every plan) — AI reviewers who each read the whole PR through their own specialty and form an independent verdict.
3. Score. Once every check and reviewer has finished, the confident findings are added up into the Future Debug Cost and its itemized breakdown. Firetrail deliberately waits for everyone — the result you see published is the complete picture, not a draft that quietly changes later.
4. Estimate coverage (when your organization has switched it on). Firetrail reads every line the PR added and estimates how much of it is exercised by the tests in that same PR — the patch coverage number.
5. Reconcile intent. Firetrail compares what the PR said it does (title and description) against what the analysis actually found, producing the "said vs. did" view on the PR analysis page — including the undisclosed-change warning when the diff quietly does something the description never mentioned.
6. Publish. The GitHub check completes, the summary comment updates, and the full result appears in the app.
The statuses you'll see
While a run is in flight, the PR shows a live status:
queued → fetching → analyzing → scoring → completed
Two other statuses can appear: failed (something went wrong — re-running usually fixes it) and skipped (the PR wasn't eligible for analysis). The Pull requests list and the PR view both update in real time, so you can watch a run move through the stages.
If Firetrail's own workers restart in the middle of a run (a deploy on our side, for example), the run picks up where it left off within a few minutes. A run that cannot be resumed is marked failed with a note, on the PR's check and in the app — it is never left spinning.
Very large PRs: partial analysis
A huge PR can exceed what the AI can read in one sitting. When that happens, Firetrail doesn't fail and doesn't silently pretend — it analyzes the most important slice: the files with the most change, plus all config, migration, and dependency files (the ones with the widest blast radius).
When a run is partial, we tell you. The PR shows a "Partial analysis" notice and confidence is reduced accordingly. This is a principle for us: you should never mistake a partial read for a full one.
Want the full read anyway? A ⚡ Turbo Analysis re-runs the entire pipeline on a model with a much larger reading window, so even a very large PR is analyzed in full instead of sampled. Each paid plan includes a monthly allowance of Turbo Analyses per seat — see Plans & billing.
Caching and re-runs
Results are cached per commit. Re-running a PR whose commit hasn't changed reuses the cached results — instant, and no AI cost. Pushing a new commit (or re-running after one) produces a genuinely fresh analysis.
One subtlety for admins: changing which AI model powers a check (from the admin console) affects future runs only. Already-cached results keep the model they were produced with until a new commit triggers a fresh analysis.