Two models. One job each.
Almost everything Limn shows you is arithmetic — counted, not predicted. Two questions resist counting, and each has a model of its own built for it: how serious is this comment, and is that good compared to everybody else?
Both are built and run by us. Neither sends your code or your comments anywhere.
The grader
On your machinethe grade runs on your own CPU. No comment leaves it to be graded.
How serious is this comment?
A review bot leaves forty comments on a pull request. Two of them will change whether the code works. The rest are preferences, tidy-ups and things somebody already knew. Nothing in the comments themselves tells you which is which — and the vendor’s own labels are marking their own homework.
So the first model reads each comment and says how serious it is: a nit, something minor, something major, or something critical. It is the same judgement a senior engineer makes in a second and cannot make four hundred times a week.
How it was built
We collected millions of real review-bot comments from public repositories on GitHub — every major vendor, across projects of every size — and taught the model on those rather than on anything invented. Under the hood it is a fine-tuned ModernBERT, which is a language model small enough to run on the ordinary processor in a laptop or a server: no graphics card, no cloud, nothing sent out, and nothing metered.
It does one thing, and that is the point. It is not a general assistant being asked nicely to have an opinion; it was trained for this single question and does nothing else.
How we know it works
We took 300 real bot comments, hid every label they already carried, and had them graded from scratch by an independent grader. Then we compared.
The model chose the same severity as the grader seven times in ten.
To know whether that is good, we needed something to compare it against — so we checked how often a human reviewer agreed with that same grader on the same comments. Also seven times in ten. The model is reading these comments about as well as the question can be answered.
On the ones CodeRabbit had labelled, its own severity labels matched the grader fewer than five times in ten. That gap is the whole reason we grade independently instead of passing the vendor’s label through.
What it feeds
- The severity badge on every bot comment, wherever you read it.
- The “needs a look” counts on a pull request, so a wall of forty threads has a top.
- The keep / tune / rarely used verdict on each vendor, and the over-calling line beside it.
And is that good, or normal?
Say your team acts on half of what CodeRabbit tells it. Is that good? There is no way to know from inside your own repositories. Half might be excellent. Half might mean the bot is badly configured and everyone has quietly started ignoring it.
The second model answers that by building the missing half of the comparison: what the same bot does in other people’s repositories.
The benchmark
Bundledthe cohort ships with the app, so placing your bots sends nothing anywhere.
How it was built
Every public action on GitHub is published as a public event stream. We read it to find repositories running each review vendor, drew a sample of 2,204 of them across seven vendors, and then visited each one to collect what actually happened: how much the bot said, how much of it people replied to, resolved, or left sitting.
Repositories are then grouped with others of a similar size and pace, because a quiet internal service and a busy monorepo are not the same test. Your repository is placed against its own group, never against the average of all of them.
Which repositories to sample was decided in code, with a fixed seed and a written record of every choice, so the cohort is reproducible rather than hand-picked. It is rebuilt as the corpus grows, and the page tells you how old the current one is.
How we know it works
By refusing.
Some groups have plenty of comparable repositories behind them and some have a handful, and a comparison drawn from a handful is a number that will move on its own next month. Where a group is too thin, the panel says so in words instead of printing a figure.
There are fourteen different ways it can decline to answer, and each one names which comparison it is declining and why. Every figure it does print carries how many repositories are behind it and which group they were.
What it feeds
- Where each of your bots sits against the same bot running in comparable repositories.
- What a typical team gets out of that vendor, so your own rate has something to mean.
- A written refusal wherever the comparison would not be honest.
And nowhere else
Everything else here is counting, and counting is better at it.
Two models, on two questions, and that is the end of the list. How many pull requests merged, how long they waited, whose turn it is, what is stalled, what a change touches, who was holding it — all of that is arithmetic over your own data. It is exact, it is reproducible, and you can check it.
A model would make every one of those worse. It would turn a number you can verify into an estimate you have to trust, and estimates are how dashboards stop being believed.
There is a third place Limn uses AI, and it is separate from both models on this page: writing. Short notes — what a change could break, the reason beside an item in your queue — and, on your own machine, your AI agent’s review of a pull request and its fix for the comments you pick. Those are language, not measurement, and they are clearly marked wherever they appear.
No number on any screen comes from them, and nothing in the product acts on a model’s output on its own. A grade changes what a list is sorted by. It never resolves a thread, merges a pull request or closes anything. By default, a review is posted, and a fix pushed, only when you press the button.
Free. One command.
Limn runs only on your machine. It reads GitHub through your gh login and keeps its data in a file on your disk. There is no account and nothing to pay.
Nothing is sent to us. We run no server for the app, and it has no tracking. Your code and pull requests go only between your machine and GitHub — and to your AI provider, on your own account, when a review or fix runs.
npx limn-review- 100% free. No sign-in, no card, no repository limit.
- Runs only on your machine. No data is sent to our servers.
- Available only from npm:
npx limn-review. Needs Node 20 or later. - Runs on your own AI agent tooling. Reviews and fixes use your Claude Code login, an Anthropic API key or Codex on your ChatGPT plan.
- Free for individuals, and for organisations where no more than 3 people use it. More users than that need a licence — ask through the contact form.
- The demo runs in your browser on sample data. No install, no sign-in.