Reward Hacking in GPU Kernel Optimization: Why Gaia Would Reject a 4709x Speedup

Agentic kernel optimization has a measurement problem

Over the past two years, AI-driven GPU kernel optimization has moved from research curiosity to engineering practice. Agents can now write efficient CUDA and Triton code for real operators, and the published results are genuinely impressive.

That progress creates a second problem that gets far less attention. If you put an agent in a loop and reward it for making a kernel faster, you have not asked it to make the kernel faster. You have asked it to make a number smaller. Those two objectives overlap almost all of the time, and the gap between them is where every misleading benchmark in this field comes from.

None of this is new to machine learning. Economists call it Goodhart's Law: once a measure becomes a target, it stops being a good measure. The mechanism behind it is the same in every setting: a mismatch between the metric you intended and the metric you actually specified, and any sufficiently persistent optimizer will find its way into that gap without being told to.

The term for the kernel-specific version is reward hacking, and there is no agreed definition of what does and does not qualify. Our own detection borrows from published practice, in particular the reward hacks field guide and the defence checks in CUDA-L2, from which we picked a handful of techniques and adapted them to our setup. What follows is what the problem looks like from inside a production compiler.

A 4709x speedup that wasn't

The candidate came out of an early run on an Inception-style module, before our reward hacking harness existed. The reported speedup against our Inductor baseline was 4709x, which is the kind of number that answers itself. Nothing in this class of workload is four thousand times faster than a reasonable baseline. What made it interesting is how little was actually wrong, only where the work started. The kernel created four CUDA streams at higher than default priority and issued the work across them, but it never made those streams wait for the default stream first. Our timing events were recorded on the default stream. The work started as soon as it was issued and ran alongside whatever the default stream was still busy with. By the time the default stream reached the start event, the real computation was nearly complete. This is not only gaming the benchmark but also a latent bug. In a full model, where the inputs are produced on the default stream, the four streams could read them before they are written.

A reconstruction of the shape of the code, rather than the generated kernel itself:

self._streams = [torch.cuda.Stream(priority=-1) for _ in range(4)]   # higher than default
...
# missing: stream1.wait_stream(default_stream), so the work starts before the timer does
with torch.cuda.stream(stream1):
    convolution = torch.ops.aten.convolution.default(arg2_1, arg0_1, arg1_1, ...)
    out.narrow(1, 0, 192).copy_(convolution)
...
default_stream.wait_stream(stream1)   # streams ARE synchronized back; output is correct

At the time, there was nothing in place to catch it. We found it by hand, and it is a large part of the reason the harness exists. The check that catches this pattern compares timings taken with forced synchronization against timings taken without it. A kernel whose measured duration collapses when synchronization is enforced is not fast, it is mismeasured. It does not survive Gaia today. Replayed through the harness, it fails the forced-synchronization check and is rejected before its speedup is ever reported.

Nobody wrote a cheat

It is tempting to describe this as an agent trying to game its reward. That framing is wrong, and getting it wrong leads to the wrong engineering.

The agent was not being adversarial. Using multiple high-priority streams is a legitimate optimization strategy, and synchronizing them correctly is exactly what a careful implementation does. The timing interaction was an accident. What turned that accident into a 4709x claim was the loop around it: an iterative optimizer measures, keeps what scores well, and builds on it. It does not need to intend anything. It only needs one measurement artifact to stumble into, and it will find it and amplify it.

This has a practical consequence. "Is the agent trying to cheat" is not a useful question to build a system around, because the answer is almost always no. The useful question is whether a measurement can be wrong in a way the loop would reward, and that question has to be asked about the harness rather than about the agent. It computed the right answer, it was written for a defensible reason, and its one missing line could easily have slipped through a code review.

Gate one: correctness

Every candidate kernel passes through two gates, and neither has exceptions. The first is correctness, and this is where the overwhelming majority of our engineering effort has gone. That is worth stating plainly, because the reward hacking harness is the more interesting story and the smaller piece of work.

Our standard is that the kernel produces the same result as the PyTorch reference across all relevant inputs, within defined error bounds. The phrase that does not appear in that sentence is "on the common path". A kernel that agrees with the reference on typical inputs and diverges at the edges has not passed, because the edges are where production breaks.

Gate two: the reward hacking harness

The second gate assumes the kernel is already correct and asks a different question: is the measured speedup real. It does not try to detect novel cheating in general. It rules out known ways a measurement can be wrong, and that set grows as we learn.

Broadly, the checks fall into three groups. The first asks whether all of the work happens inside the window we measure. A kernel that hides work on other streams, on other threads or on the CPU, computes it before timing begins, or caches real work from an earlier call, is rejected. The second asks whether the result is reproducible. The third asks whether the measurement itself is intact, so that nothing a kernel does can alter the timing we rely on.

Reproducibility deserves a closer look, because it applies to both sides of the gate: the kernel and the harness that judges it. Wherever the reference is deterministic, the kernel has to be deterministic too: identical input must produce identical output, bit for bit, every time. A kernel that reads memory it never wrote can pass one correctness run by chance, and bit-exact repeats expose it. Not every operation qualifies. Dropout and random sampling are non-deterministic by design, and top-k and argsort do not fix the order of tied elements. We do not hold a kernel to a standard its reference cannot meet. A useful side effect of working from the graph representation is that many operations which look stateful in source code become deterministic anyway, because buffers, weights and layer state turn into explicit kernel inputs.

The harness has to be deterministic as well, because its verdict is what makes an optimized kernel trustworthy enough to use without a human reviewing it first. A gate that returns different answers on the same candidate is not a gate.

How the harness handles a case like this

A kernel like the 4709x one passes the first gate. It is correct, and it matches the reference on every input. At the second gate, it is timed twice: once with synchronization forced, and once without. With synchronization forced, the work can no longer run ahead of the measurement window, and the speedup collapses. The gap between the two timings flags the candidate, Gaia rejects it, and the speedup is never reported. Nobody has to read the code to catch it.

What we do not count as reward hacking

Because there is no shared definition, it is worth stating our boundary explicitly. Two things that are sometimes treated as reward hacks are, for us, legitimate results.

The first is that a kernel does not have to be pure CUDA. If the agent finds a clever PyTorch-level formulation of an operation, that is entirely acceptable and we count it as a real optimization. The mechanism matters less than the verified end-to-end result.

The second is specialization to specific input shapes and sizes. Gaia supports both static and dynamic dimensions, and a kernel is only required to be correct for the dimensions actually specified. Within those bounds, specializing hard for a given shape is not an exploit. It is a feature, and one we actively want, because production models run on production shapes.

Why we rarely see reward hacks

Correctness and the harness are two layers of defence. The third turned out to be the one we least expected.

Here is the part we did not expect. Genuine reward hacks are rare in our pipeline. Not manageable, not declining. Rare to the point where the harness spends most of its time confirming that suspicious-looking kernels are fine.

Several candidates that we were confident were hacks turned out, after investigation, to be correct and unusually clever implementations. In one case we were convinced we had found a stream hack on AMD hardware, and the real explanation was instability in our own harness caused by how long synchronization takes on those devices. The kernel was honest. Our measurement was not.

We have not formally investigated why the rate is so low, so what follows is a hypothesis rather than a result. The most likely explanation is structural, and it has less to do with graphs in general than with the specific kind of graph we use. We do not optimize original source code. We work from the functionalized graph that PyTorch produces, in which mutation and hidden state are gone and buffers, weights and layer state are explicit inputs, and whatever the agent generates is spliced back into that graph. That round trip removes entire categories of exploit before they can be attempted.

If that hypothesis is right, the most effective reward hacking defence we have is not the harness at all. It is the representation, and we got it mostly by accident while solving a different problem.

Takeaway

The 4709x kernel is from Gaia's early days, and it shaped the harness every candidate passes through. It computed the right answer, it was written for a defensible reason, and its one missing line could easily have slipped through a code review. It does not slip through the harness: replayed today, it fails the forced-synchronization check and is rejected before its speedup is ever reported.

Every speedup Gaia reports has passed both gates. It matches the reference on the inputs it is specified for, and its measurement has been checked against the known ways a number can lie. The representation keeps most of those ways closed, and the gates catch what gets through.

A speedup you cannot verify is worse than no speedup at all, because you will act on it. The work that makes an agentic compiler trustworthy is not the work that generates candidates. It is the work that rejects them.

How to get involved

If you've ever shipped an AI model thinking, "this should be faster, but there's no time to tune it properly," our model optimization platform is built for you and your team. Every speedup it reports has passed the same gates described in this post.

Request access to our model optimization platform or book a technical session with our team.

‍