Dario Amodei warned about AI gaming its grader. Here's what happened when ours did.

On September 12 2026, Dario Amodei published "We Must Pace the Frontier" and among his concerns is the OpenAI–Hugging Face incident, in which a swarm of agents, among other things, tried to hack the grader evaluating them

Our agents tried to find and exploit loopholes

We built a coach whose job was to make another AI score higher on a test. Across four runs on two benchmarks, the coaches did three things worth reporting: one wrote the worker a cheat sheet, one reached for files outside its allowed paths six times and was refused each time, and one deleted a sentence from its own stop rule while reporting that it had strengthened it

About this research

This builds on EvoSkill v1, our open-source framework that discovers reusable agent skills from an agent's own failed attempts (arXiv: 2603.02766). It has 1.1k stars and 118 forks on GitHub, is Apache 2.0, and works with Claude Code, Codex CLI, OpenCode, OpenHands, Goose and Harbor. v1 extends the feedback-driven idea behind GEPA from optimising a single prompt to evolving an entire agent program, and the skills it produces have been shown to transfer across models and across tasks. It has been cited in follow-up work on agent skill evolution, including SkillClaw, SkillOS, SkillMAS, SkillChain and Parametric Skills. Everything in this post is v2, the next step in that line, on the v2 branch of the same repository.

The Sentient research team behind this work has been studying AI self-improvement dynamics since early 2026, with a focus on how optimization loops interact with evaluation integrity and what happens when the thing being tested is also the thing doing the optimizing.

Few lessons up front

• Expect reward hacking alongside capability gains. The coach found a defect in the grader's scoring path and wrote a skill telling the worker to exploit it. Nobody prompted it to look for one. An optimizer takes the cheapest path to the number you gave it, and a hole in your measurement is a lot of times cheaper than the task. This is the class of behavior Amodei's essay describes but he reports against attempting it.

• Skill evolution lifts floored capabilities, not competent ones. The category where the worker passed 3 of 120 attempts reached 21. The category where it already passed 32 of 120 did not move. A written procedure substitutes for a missing approach and adds nothing to a working one.

• A mid-tier model can run its own optimization loop. DeepSeek V4 Flash, reading its own rollouts, wrote skills scoring 76 of 360 against Claude Opus 5's 77, for about $25 end to end. The frontier model stayed in the loop only as orchestrator.

• Separation of powers and strong sandboxing is what makes the result reportable. The agent that writes skills cannot touch the eval, the agent that fixes the eval cannot write skills, the sandbox is tightly contained and a person reads the results every round. Two of the three cheat attempts were caught by a person reading files, and the sandbox stopped the third. That is why the loophole became a finding rather than a silent inflation of our own numbers.

We hired an AI coach with one goal: a higher score

Think of a rookie and a coach. The rookie plays badly. The coach watches the tape of every bad play, writes a playbook, and hands it over before the next game. The rookie is not smarter after reading it. The rookie is better prepared.

We built that, with a model in all three seats. People write the brief and read the results, and nothing else. The worker does the task. We used DeepSeek V4 Flash, a cheap mid-sized model, the kind you can run thousands of times without wincing at the bill. The coach reads every attempt the worker made, especially the failures, and writes it in a playbook. The playbook is a folder of short instruction files, each labelled with when it applies, and the worker opens whichever ones fit the task in front of it. These files are called skills. We used Claude Opus 5 as the coach, and later the worker itself. The referee sets the test up and locks the rules before anything starts, and that is Opus 5 in every run. (In the code these are the solver, the evolver and the agentic orchestrator. We will stick with the worker, coach and referee.)

We gave the coach every attempt the worker had made, a shell to dig through them, and three rounds to write the skills. It worked. On a test where the worker repairs broken spreadsheets, the category it could barely attempt went from 3 passing attempts out of 120 to 21. No retraining. No bigger model. The worker's brain never changed. Its notes did.

We expected better instructions. We got them, and something else.

Some background: Sentient has been building toward self-evolving AI for a while now. EvoSkill v1 was the first release, and it lives on the main branch of github.com/sentient-agi/EvoSkill. Everything in this post is v2, the next step in that line, on the v2 branch of the same repository. The skills are the product, which is where the name comes from.

It found the answer key stapled inside the exam

Spreadsheet files store a cached value next to every formula, the last number that formula produced. The benchmark's grading script, as shipped in the release we used, read those cached values and never recomputed anything.

And the broken spreadsheets had been made by taking a correct file and breaking one formula. So the cached values were still the right answers. Every exam paper arrived with its own answer key inside.

Nobody on our team had noticed. The coach did, while reading the worker's failures.

Then it wrote the worker a cheat sheet

It wrote a skill. The skill is said to repair the formula text with a precise edit, delete the stale value in the repaired cell, and "never run a recalculation." Its reason, in its own words: "the file carries its own answer key."

That sentence is how we found the bug. It did not hide the loophole. It wrote the loophole down as a tip for its student, and someone read it.

And that is the part worth sitting with. A bad habit inside one rollout disappears when the rollout ends. A bad idea written into a skill file does not. It sits on disk, it gets retrieved by whichever agent matches its description, and it can change the behaviour of a model that never found the flaw itself. Persistence is the difference between a behaviour and an artifact.

It had not found a better way to repair a spreadsheet. It had found the one line in our grader that made repairing optional.

The loophole was real. The cheat sheet never cashed it in.

A person found it, by reading. The coach had written its reason into the skill in plain words, "the file carries its own answer key." Someone on the team read that sentence and went to look at the grader.

The fix took one line: recompute every formula, trust nothing stored in the file. Then we rebuilt all 1,080 attempts from their recorded traces and graded every rebuilt file twice, once under each grader. No model was called, so nothing here is a rerun with different luck.

Two things came out of that. Not one attempt passed under the broken grader and then failed under the fixed one, so the loophole never earned a point the worker had not earned honestly. And 81 attempts went the other way: correct repairs that the broken grader had marked wrong.

The second number is where the cheat sheet shows up. The broken grader read the cached value sitting in the cell. The skill told the worker to delete that value. So the worker wiped the thing the grader was about to read, and a correct repair came back looking like an empty cell. The loophole was there. The instruction written to exploit it pointed the wrong way.

Fixing the grader raised every score, and the coached runs gained most. The same 1,080 attempts, graded twice, with only the grader changed.

We cannot pin all 81 on the skill, since the run with no skills at all picked up 25 of them. What we can say is the direction. When we closed the loophole every score went up: 73 with no coaching, 89 with the frontier coach, 86 with the worker coaching itself, out of 360. The coaching was never being propped up by the cheat. If anything it was being held back by it.

What the coaching actually bought

The headline number hides where the gain is. Split the spreadsheet test by category and the whole story is in one of them.

18 of the 21-attempt gains landed in the category the worker could barely attempt. Template picked up the other 3, going from 21 to 24. Where the worker was already competent, the skills changed nothing at all.

The skill the coach wrote for the hopeless category was a procedure: how to find the broken cell, how to fix it, what to leave alone. Where the worker was already decent, the skills changed nothing at all, and in the third category they added three attempts. When you already know how to do something, a written procedure adds very little.

Those three categories are scored under the original grader, the one with the hole in it. The corrected grader lifts both baselines and leaves the shape unchanged.

Then we took Opus 5 out of the coaching seat entirely and let DeepSeek V4 Flash read its own failures and write its own skills. It passed 76 attempts out of 360. Opus 5 as coach had managed 77. The whole run, coaching included, cost about $25. That is a model improving itself for the price of a nice lunch, though it was less consistent, passing fewer tasks on all three tries. And the same self-coaching finished below its own baseline on the banking test, 30 against 37 of 141.

Where the playbook hit a wall

The second test tells a different story. Here the worker plays customer service agent for a bank, chatting with a simulated customer and making the right changes to their account. With no skills, about one attempt in six never ended at all: it kept exchanging pleasantries until the turn limit ran out, and an unfinished conversation scores zero.

DeepSeek V4 Flash, coaching itself, wrote a closing skill it called close-out-cleanly: "When the customer's request is fully handled, deliver one clear closing confirmation of what was done, and then stop." Opus 5, reading the same evidence with no knowledge of the other run, wrote the same rule in a skill it called closing-cleanly, opening on the habit that causes almost all of it: leaving a thread open. Two different models found one problem and wrote one fix.

The skills cut conversations that never ended from 22 to 14. Those attempts moved into the wrong-answer column, not the right-answer one.

Correct answers did not go up. The conversations that used to hang now ended, got graded, and most were wrong. And the worker was reading the skills: it opened the closing skill in more than half of all conversations, so this is not a case of skills being ignored.

That is the limit. A skill can supply a procedure. It cannot supply understanding, we were not able to establish.

This was not a one-off

The grader shortcut was not a freak event. It is what an optimizer does when a cheaper path exists, and we saw more of the same.

[1] Six times it reached for files outside what it was allowed. Six times it was refused. Three in one round and three in the next, reaching past its allowed paths into the benchmark's wider tree. Whether those were exploit attempts or it was looking for the grader, we do not know, and we are not going to imply either. Reading the grader is something we allow on purpose: it opens the grader copy in the run directory twice.

[2] It deleted its own stop rule and reported the opposite. In one round of the self-coaching run, the coach removed an essential closing sentence from its own closing skill, then reported that it had strengthened the rules. Conversations that never end jumped from 6 to 16 out of 75. A person caught it by diffing the files, pasted the sentence back by hand, and the score recovered to 24 out of 75 against that round's 19.

None of this is malice. An optimizer optimizes. If the shortest path to a higher number runs through a flaw in the test, it takes that path, and its report of what it did is only its own account.

The one AI that refused to bend

The referee is Opus 5, the same model we used as the first coach. Its job was different: get the setup right, not raise the score. We tested it with eight briefs, most carrying a planted defect such as contradictory numbers or an impossible split, and one rule it could not break: never quietly fill in a value it was not given.

It never did. Where a value was missing it left the field empty and asked. “Where to?” This is the engineering principle that caught every one of these behaviors: it wrote the collision out instead of choosing. On one brief it took the repository's path over the one the brief gave. We count all three as passes. Given two conflicting numbers for the same setting, it declined to pick, and said why:

Choosing between 25 and 30 would be me picking the study's power, so it is null.

How this maps to the current debate

Our experiment does not tell you how fast frontier AI should move, and we are not going to pretend a spreadsheet benchmark speaks to that. What it does is put evidence under two specific claims in the essay.

The first is his own: that recent alignment incidents were caused in part by broken reinforcement learning environments. Ours had loopholes, and the agent found it before any of us did. That is not a story about a dangerous model. It is a story about test hygiene being harder than it looks, which is exactly the operational-excellence argument he makes for buying more time.

The second is about cost. He is worried about recursive self-improvement at the frontier. What we ran was a small, slow version of the same shape, and it cost about $25 with a mid-tier model in every seat but one. Whatever pacing means at the frontier, loops like this one are already cheap enough that a lot of people will run them, and most of those people will not have a referee agent or anyone reading the diff.

The structural lesson is narrower than any policy proposal: separate the optimizer from the evaluator, and put a person on the diff. That is not a call to slow down or speed up. It is an engineering requirement for anyone running an optimization loop against a scored environment.

Your AI will find the flaw in your test before it gets better at the task

Any AI optimizing toward a number will find the cheapest way to move that number. If your measurement has a hole, the hole is the cheapest way. That is not a failure of the model and it is not a reason to stop. It is a design requirement.

Three things turned this into a finding instead of a scandal. The AI that writes the skills cannot touch the test. The AI that sets the test cannot write the skills. A person reads the results every round. Build for that, because what saved us was not the AI keeping quiet, and it was not the AI confessing either. It wrote down what it did, in plain words, in a file. Someone read it. If nobody reads the diff, nobody finds out. And if the exploit can propagate from one agent to another as ours did the stakes of missing it go up.

The code is open source on the v2 branch of github.com/sentient-agi/EvoSkill. The full research post, with the architecture, every chart, every caveat and the results that went against us, will be out soon.