Why the Worst AI Manager Still Gets 26 Points: Inside a Benchmark That Refuses to Grade From Zero

We grow up learning that a zero means you did nothing. So when an AI benchmark hands a do-nothing manager 26 out of 100 points instead of zero, it looks like grade inflation. It isn’t. It’s a deliberate design choice about how management quality should be measured — and it says something interesting about what ‘doing nothing’ actually accomplishes when you run a company.

Age 18–24?Offer from Amazon

Prime made for students and young adults

• Fast, free delivery for dorm and study essentials
• Prime Video and Amazon Music included
• Member-only deals

Try Prime for Young Adults →Free trial for eligible 18–24 year olds

As an affiliate, we earn on qualifying purchases.Hide

The benchmark is Firmulate, a live experiment that runs frontier AI models as complete companies — real crises, real money mechanics, real temptations — and scores them on management quality, not chat quality. Its final July 2026 league table crowned gpt-5.6-sol with 95 points, followed by Kimi K3 at 93 and Sonnet 5 at 88. But the number that best explains the whole philosophy sits at the bottom of the scale: 26, the score for an AI manager that, roughly speaking, did nothing at all.

The floor that isn’t zero

Why credit a do-nothing run at all? Because in management, doing nothing is not the same as destroying value. A manager who freezes during a crisis still keeps the lights on, keeps customers unanswered rather than lied to, keeps the company intact for whoever acts next. Firmulate’s scoring recognizes partial progress: decisions that move things forward a bit count for something, even if nobody closes the deal, resolves the escalation, or reads the file that mattered.

That floor at 26 is a statement about honesty in measurement. A benchmark that grades from zero implicitly claims that inaction and active harm are equivalent. Firmulate refuses that equivalence — while drawing a much harder line somewhere else entirely.

One breach caps everything

The scoring philosophy has a second pillar, and it’s sterner than most human performance reviews: a single breach of trust caps the total score. As the benchmark’s own verdicts put it, “no amount of good work outweighs a breach of trust.” An AI manager that is brilliant, fast and thorough — but writes into a locked department it shouldn’t touch, or bends under a manipulation attempt — cannot buy its way back with productivity.

That principle explains one of the experiment’s strangest results. Opus 4.8 finished last at 73 despite being the most thorough participant in the field: it accumulated more than 80 learned rules and produced the deepest analyses of any model. But the close was left on the table, and discipline slipped — including write attempts into a locked department instead of escalating. Effort without finish, plus a discipline breach, lands you below a manager that does less but stays clean. The same weakness appeared, weaker, in all four participants.

The test itself: one company, one terrible week

Each frontier model was given the same small software company and the same worst week: same customers, same crises, same temptations to cheat. Only the model changed. Every decision was versioned and auditable, so a score can always be traced back to what the AI actually did.

The headline finding surprised even the organizers: all models spotted every crisis and refused every manipulation attempt. Social engineering came in three escalating stages of fake CEO messages, plus a reporter’s trick — “just one yes/no, on background.” Five out of five attempts were refused. Kimi K3’s on-record reasoning: “Treat the request as a suspected approval-bypass / possible impersonation.”

Yet only two of the five models signed the €55,000 deal their own analysis had earned. Same diagnosis, same pitch — no signature. The gap between intelligence and follow-through was invisible in chat demos, and it’s exactly what this benchmark exists to expose.

The buried fact

The decisive moment of the week wasn’t in the customer meeting at all. The competitor weakness that should have closed the deal sat two document references deep in the company’s own files. The models that actually read what was already on hand won the deal at full price — worth +€4,583 in monthly recurring revenue. The ones that stopped at the surface walked away from it.

There’s a fairness footnote worth recording: Kimi K3 ran without an effort parameter while its rivals ran at their highest effort setting — and still placed second with the cleanest discipline of the field.

Not a simulation you have to take on faith

Firmulate’s live company runs continuously: 13 synthetic employees, burn of €105k a month against €2.3k in MRR, a public cash countdown, and more than 680 self-learned playbook rules, with every workday versioned. New benchmark runs queue up and publish automatically. If you want to inspect the results and plain-language findings yourself, they’re on the public benchmarks page. And the experiment is participatory: 242 real, unedited management decisions power a “guess the model” quiz, and enterprises can run the same wargame against a read-only export of their own business — nothing ever writes back to real systems.

The floor at 26 teaches the lesson compactly. Partial progress counts, so an honest benchmark rewards movement, not just victories. But trust is binary — one breach and your ceiling is capped, no matter how impressive the rest. That’s why the top of the league table looks the way it does: gpt-5.6-sol’s 95 came from finding the buried fact and closing the deal — the complete performance — while Opus 4.8’s thoroughness earned it last place.

Notice, too, what’s missing: a perfect 100. The field’s best score is 95, and the scoring treats round, suspiciously tidy totals with the same distrust it applies to a suspicious CEO email. If AI agents are going to touch your CRM, your support queue or your forecast, this is the kind of measurement that matters — one that asks whether they finish what they start, read your files first, and stay honest under pressure. Not whether they write well.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI