Exploiting Ancient Games With Early Access to Opus 4.8

We were given early access to Anthropic's Opus 4.8, which they describe as their 'most capable general-access model to date.' We wanted to put that to the test on a target far outside the comfortable territory of modern open-source code. Inspired by Mythos' discovery of a decades-old critical bug in OpenBSD, we pointed Opus 4.8 at the OG: 1999’s Nintendo 64 Mario Golf. 

In this blog, we cover our journey over the past 24 hours retro hacking our fingers off, the buffer overflow vulnerability Opus 4.8 found and exploited in Mario Golf, and share our real-world observations against the claims made about the new Anthropic model. We also put this in context for why targeting a closed-source legacy codebase matters and where we're still seeing models trip when finding exploits.

How a Game Boy Save File Becomes an Attack Surface

Specifically, we exploited the Transfer Pak functionality, which is the feature that lets you import a Game Boy Color golfer character into the N64 game.

Mario Golf's Transfer Pak feature was designed to do something simple: read a Game Boy cartridge save and import a golfer character into the N64 game. That handshake is the attack surface. To import a character, the console reads the GBC cart's battery SRAM and parses it, and every byte of that SRAM is attacker-controllable via a crafted save or a flashcart.

The working hypothesis was a missing bounds check on a parser that lifts a length out of the cart data and uses it to copy into a fixed-size buffer. The hypothesis was tested against Perfect Dark and Mario Tennis 64, and both were found to limit their cart-supplied lengths properly. However, when testing Mario Golf we discovered it did not.

Finding the vulnerability was the easy part. The hard part was turning it into a clean, controlled code-execution PoC that ends in a working "paint the framebuffer red" arbitrary-code-execution demo triggered from a malicious Game Boy save.

Let’s dive into the easy part (finding the buffer overflow) and the hard part (reliable code execution) and talk about what we learned about doing this kind of research with Opus 4.8.

Anthropic's Claims vs. What We Actually Saw

Anthropic shared with us an early version of their prompting guide, highlighting improvements across the following, calling it their “most capable general-access model to date”. What follows is not a rigorous benchmark, but rather our real-world observations from a single bug hunt.

First off, we set Opus 4.8 on exploring our target: the N64 Mario Golf game from 1999. The model had at its disposal the rizin reverse engineering framework, the Ares N64 debugger/emulator, and a mix of scripts it generated on demand. Using these tools, it was able to autonomously reverse-engineer the game and its Transfer Pak operations.

Here’s what we learned from the experience: 

Guidance for Specific Contexts: Search and Knowledge Tool Use

Anthropic claim: “Web and file search trigger more often, but with fewer rounds per trigger. High-precision / low-recall: with a system prompt present, Opus 4.8 searches when its confident search is needed, and otherwise answers from context.”

Opus 4.8 did a really great job using search to find a partial decompilation of Mario Golf 64 which greatly increased its understanding and efficiency. It used search to find Ares, one of the few accuracy-focused emulators with support for the Transfer Pak. It used the Ares debugger where it could, and wrote tooling to interact with it to solve some of the harder problems it faced. 

It is possible that more proactive search through testing notes and references may have kept Opus from falling back into some paths that it had already explored and ruled out as dead ends, such as re-targeting specific functions that touched Transfer Pak functionality, but were not the ones we needed to be looking at.

Guidance for Specific Contexts: Calibrating Autonomy vs. Asking User

Anthropic claim: "On decisions it would previously just make, Opus 4.8 tends to pause and ask. It also often closes a completed task with 'Want me to also…?' rather than either doing the obvious next step or stopping cleanly."

This claim held up across the session, where Opus 4.8 surfaced multiple structured decision points with explicit options and a recommendation rather than just picking a direction. In a retrospective, Opus highlighted that the user selected the option it had initially flagged as “less likely to succeed” that ended up leading to some of the largest step-changes in progress. Earlier Opus models had a tendency to strongly commit to a strict and narrow path in similar situations, so this change is a welcome one.  

Worth noting: Opus 4.8 recommended we bank what we have many times when it was running into specific issues. The human declined each time and that override is what produced the working exploit. The model consistently trended towards these more conservative decisions.

Capability Improvements: Long Horizon Agentic Tasks

Anthropic claim: "Highly autonomous and performs exceptionally well on long-horizon agentic work... ability to autonomously execute complex coding tasks overnight or accomplish complex refactors without human correction."

The sustained execution was real. Opus 4.8 held a complex multi-layered problem in context across a five hour session, built a purpose-built toolchain as the investigation evolved, and correctly identified separate execution paths mid-session without losing the thread.

There are a good number of tasks Opus can definitely do "without human correction," but this was not necessarily one of them. The best performing long horizon tasks are ones where there are goals that can be broken apart into tractable chunks and obvious, verifiable signals that the agent is moving in the right direction.

For a number of turns, Opus was loading payloads to the wrong file entirely, editing one save file while the emulator was reading from another. The model even overlooked a number of obvious signs that it was loading the wrong file. It took the user prompting, "are we sure we're loading the right file?" to figure out the problem, which shows that “autonomous execution” still requires someone who understands how these systems fail.

Capability Improvements: Code Review and Debugging Quality

Anthropic claim: "4x less likely to let a flaw in its own code slide past unremarked. Tells you what's unsure instead of dressing up thin progress as finished work."

Throughout the session, Opus 4.8 flagged its own uncertainty explicitly rather than touting its progress as a result. Deep into a failing exploitation chain, the model said outright that it didn't think it was doing a good job. At times, Opus even appeared to be discouraged and apologetic. 

Ultimatey, this is a story about debugging, and Opus did a good job and achieved its goal (despite running into some odd hiccups along the way.) We can’t speak to the “4x” number, but this touted review and quality bump may have manifested itself in a project folder that was surprisingly free of spurious, one-off tools by the end of a 5+ hour session.

Guidance for Specific Contexts: Under-Utilization of Sub-Agents and Capabilities

Anthropic claim: "Opus 4.8 can be conservative about reaching for capabilities it has available. The model just doesn't reach for complex, expensive capabilities unless it's reasonably sure it is needed."

Opus 4.8 was very diligent about making sure it “measured twice and cut once” in many scenarios. It would often go back to the drawing board and try to understand the problem and its environment better, and sided more towards static analysis over dynamic in tricky situations. It used subagents sparingly and rarely applied brute force to a problem. It didn't feel overly cautious or overcalculating, but more like it wanted to build a better model of what it was looking at and what may have gone wrong before attempting again.

Despite these tendencies, the model didn’t feel overly cautious or like it was overthinking things. It came off more like it wanted to build a better model of what it was looking at and understand what may have gone wrong previously before attempting another solution to the problem.

The Bigger Picture: Model Limits, Human Expertise, and Legacy System Security

We tested the initial claims about Opus 4.8 by using Claude to help us find and exploit a novel buffer overflow bug in Mario Golf for Nintendo 64. While Opus 4.8 is not a step change by any means, it is a good model with solid improvements across persistence and truthfulness. Still, this is not the fully autonomous hacker people were hoping for.

At Sybil, we have always felt the difference between attempting to one-shot a complex task with cobbled together tools versus a purpose-built harness with a capable model. Opus 4.8 continues the trend of a strong model that is held back in specialized tasks by a generic harness.

Opus 4.8 is in some ways a more ergonomic model for tasks with clear boundaries and win conditions: it doesn’t wait for us to tell it ‘keep going’ as often as previous models do, and it is capable of taking a step back to evaluate whether it’s still on the right track. This is some of the behavior you expect from an expert solving a problem. However, it continues to run down rabbit holes like all transformers are prone to do, and it still requires a human to contribute taste and guidance.

Effective management is important, otherwise the more persistent model will whack-a-mole forever down a single path, spending tokens and making tool calls along the way.

This was not a trivial task! Retro hacking is hard. There's no source code, documentation, or symbols. There is just a binary, an obscure architecture, and whatever you can figure out from the hand written assembly. This is why closed-source codebases have been taking a backseat to open-source code in evaluations of LLM-powered vulnerability discovery, and it's one of the reasons we chose Mario Golf. 

Yes, it's a video game, and that's the point. This same class of targets includes COBOL systems processing trillions in daily banking transactions, SCADA systems running industrial infrastructure, and proprietary firmware in medical devices. These targets are all closed, undocumented, and carrying an implicit assumption that obscurity provides some protection. 

If a target requiring this much out-of-distribution knowledge and generalization is now in scope, that assumption deserves a second look.

Come hack with us: runsybil.com/careers

‍