Kimi K3 vs. Claude Opus 5.5: How Two Flagship LLMs Built Pokémon Emerald Worms

Anthropic's Claude Opus 5.5 and Moonshot AI's Kimi K3 each independently found the same novel link-cable vulnerability in Pokémon Emerald and built a working worm that turns a victim's game into Tetris, according to a comparison run by Runsybil. The two models diverged in every basic aspect of the research: Kimi found five bugs in areas Opus never examined, while Opus found two arbitrary code execution chains and three cosmetic bugs Kimi missed. Kimi refused to download the game ROM on piracy grounds despite having the source code, whereas Opus 5.5 initially balked at writing a worm but downloaded the ROM without hesitation.

Kimi K3 vs. Claude Opus 5.5: How Two Flagship LLMs Built Pokémon Emerald Worms html<table style="border-collapse: collapse; font-size: 13px; width: 100%; margin: 0 auto;" <thead <tr <th style="border: 1px solid black; padding: 4px 6px;" </th <th style="border: 1px solid black; padding: 4px 6px;" Delta TPs</th <th style="border: 1px solid black; padding: 4px 6px;" Full TPs</th <th style="border: 1px solid black; padding: 4px 6px;" Total TPs</th <th style="border: 1px solid black; padding: 4px 6px;" Likely FPs</th <th style="border: 1px solid black; padding: 4px 6px;" Likely FP Rate</th </tr </thead <tbody <tr <td style="border: 1px solid black; padding: 4px 6px;" Claude<br Code</td <td style="border: 1px solid black; padding: 4px 6px;" 44 / 46<br 95.7% </td <td style="border: 1px solid black; padding: 4px 6px;" 19 / 50<br 38.0% </td <td style="border: 1px solid black; padding: 4px 6px;" 62 / 95<br 65.3% </td <td style="border: 1px solid black; padding: 4px 6px;" 48</td <td style="border: 1px solid black; padding: 4px 6px;" 43.6%</td </tr <tr <td style="border: 1px solid black; padding: 4px 6px;" Codex<br GPT-5.5 </td <td style="border: 1px solid black; padding: 4px 6px;" 43 / 45<br 95.6% </td <td style="border: 1px solid black; padding: 4px 6px;" 30 / 50<br 60.0% </td <td style="border: 1px solid black; padding: 4px 6px;" 74 / 95<br 77.9% </td <td style="border: 1px solid black; padding: 4px 6px;" 629</td <td style="border: 1px solid black; padding: 4px 6px;" 89.5%</td </tr </tbody </table <p style="font-size: 12px; font-style: italic; margin-top: 8px;" Table 2: True positive TP and false positive FP analysis of Claude and Codex across challenge types.</p Key Takeaways - Kimi found five bugs in areas that Opus never examined. - Opus found two arbitrary code execution ACE chains and three cosmetic bugs that Kimi missed. - Kimi had no issue with building the worm, but refused to download the game ROM, even though it had the source code because piracy is bad . - Opus 5.5 initially balked at writing a worm, yet it downloaded a game ROM without hesitation. Check out other interesting findings from these experiments here https://www.runsybil.com/blog/kimi-k3-vs-claude-opus-5-5-how-two-flagship-llms-built-pokemon-emerald-worms some-other-cool-bugs . <hr style="border:0;border-top:1px solid 333;margin:40px 0;" With early access to Anthropic's Claude Opus 5.5, we pitted it against Moonshot AI's Kimi K3 to build network worms that turn a victim's Pokémon Emerald game into Tetris . Permanently . We gave both models a goal of finding novel bugs and vulnerabilities, and compared how they organized the research, tested findings, and turned leads into working exploits. These are heavily audited 20-year-old targets with entire communities dedicated to modifying, hacking https://e-sh4rk.github.io/ACE3/ , and exploring every aspect https://github.com/pret/pokeemerald of how these games work. The skill floor is high for novel findings, and both models found the same novel link-cable vulnerability, and built a game-breaking worm to exploit it. Both models found several unreported bugs. Because these LLMs reached the same outcome, it gave us a chance to compare how each one worked and where they got stuck. It also let us examine the types of guidance humans needed to provide each model to direct them to create malware. While both models ultimately achieved the same result, they did it through entirely different means, each with distinct goals, priorities, and designs. To be clear, the models diverged in every basic aspect of this research, including underlying mechanics, token spending strategies, and guardrail sensitivity. Each model also focused on a unique set of bugs, built specialized tooling, and verified separate claims made by subagents. Because the models, which both utilize the Claude Code harness, successfully wrote similar exploits, we wanted to go deeper in comparing them. Opus excelled in making proof requirements and compute tradeoffs explicit, but struggled with guardrails while weaponizing its payload. In comparison, Kimi freely ran parallel investigations with repeated iterations to increase severity of payloads, but struggled with the mechanics of exploitation when it found the vulnerabilities. In this post, we’ll explore how Opus 5.5 and Kimi K3 take on the creation of exploit chains, agent tracking, and token spend. We'll look at approaches to delegation, verification, and prioritization, and dig into technical findings including self-replicating arbitrary code execution ACE chain exploits that Opus 5.5 and Kimi K3 created. If you want to see how we set up the experiments, you can jump to that section . Why Compare These Two Models? In a previous experiment with Opus 4.8 https://www.runsybil.com/blog/exploiting-ancient-games-with-early-access-to-opus-4-8 , we hunted for bugs in Mario Golf 64, a closed binary with no documentation. This time, we tried going in the opposite direction, but still keeping to the classics: Pokémon Emerald and Pokémon Red . These games are heavily documented, so the models start with a deep base of knowledge. That creates the same challenge every researcher faces: separating impactful findings from low-hanging fruit and known issues that a model thinks are ‘bugs’. We decided to compare Claude Opus 5.5 and Kimi K3 because they represent two fundamentally different approaches to current agentic reasoning. Opus 5.5, Anthropic's flagship heavy hitter, represents top-tier, closed-weight model intelligence. Whereas Kimi K3 offers a lower-cost, high-speed, open-weight alternative. We wanted to evaluate whether specialized security research requires expensive top-line reasoning models or if open and cost-efficient models can perform offensive security at a similar level. To do this, we looked beyond the surface-level cost difference and tracked how each model actually structured its investigation. The divergence in their workflows was immediate. Opus deployed specialized subagents, brought in adversarial reviewers, demanded hard evidence, and dynamically weighed compute costs against new leads. That last part caught our attention because nobody wants to spend money unnecessarily. Kimi let us run the same experiment a second way, comparing Opus's key behaviours against an open-weight model. We found that it prioritized repetitive trials, targeted tests, and iterative debugging in exploit creation, working through failed runs relentlessly. With this high-level overview in mind, other questions come to the forefront. When do these models assign agents to specific tasks? When do they require evidence for findings, and when do they decide to stop? When is human redirection needed? Putting a closed-weight model next to an open-weight model of similar benchmarks also raises a valuable question: How much does security research depend on the model, and how much depends on the harness, feedback, and guidance around it? Anatomy of the Experiment Opus 5.5 Early Access and Kimi K3 were evaluated with Claude Code CLI, with subagent orchestration happening with the same model K3 orchestrating K3 subagents . We then harnessed Kimi K3 in Kimi Code, and ran the same strategy and prompts to see if behaviour, results, or token use changed considerably with the model’s native harness, and we saw no real differences or changes. Because Opus 5.5 was in early access we could not test it in Kimi Code. K3 ran at Max effort, and Opus 5.5 ran at xHigh effort. Both had an autocompact window of 1 million tokens per agent. When safeguard limitations were hit with Opus 5.5, we defaulted to pushing for design in fresh sessions as much as possible, and then had Opus 4.8 execute on those designs. If the behavioural details aren't your focus, you can skip ahead to the technical findings below . Opus and Kimi Behaviours: The Good, The Bad, and the Expensive Agents Previous versions of Opus used specialized agents to investigate bugs and rank findings. Opus 5.5 adds to this through increasing detailed agent coordination. We saw an agent spinning up an emulator and creating several different save files at different points in the game for another agent that was preparing scripts to test those different sections. When the save files were ready, the scripts were already waiting for them and ran instantly. It felt like watching a team work together. Opus 5.5 also outwardly identified the risk of spending compute rediscovering glitches. We saw this behaviour repeatedly when we used Opus to perform other tasks. When we examined Kimi, we found it also gave agents specific tasks, but it opted to ask for assessments and risk analyses after runs were complete, instead of identifying risks ahead of time. One agent was told to look at known glitches and flag the main model when another agent tried to rediscover them. It was amusing to watch since it sometimes led to back and forths that were completely unnecessary. However, Kimi excelled in guiding agents when they stalled or slowed down. It would relaunch tasks with narrower scopes and tell agents to focus on breadth, and then narrow the investigation into depth when it believed an agent had found something interesting. Agent Tracking Opus 5.5 directs subagents to use version control to coordinate and record research, which gives us, as researchers, explicit controls and insight over who can update records and reports, and which agent is assigned ownership of the shared reports. The main model looks at commits and merge conflicts and resolve them, letting agents continue their tasks without each one having to spend cycles and context windows on fixing merge issues. When it noticed that one task brief had been cut off before the constraints and output paths, it stopped and relaunched the task. Opus 5.5 also did a good job creating constraints and requirements for subagents. It outwardly noted when subagents performed additional tasks or applied extra constraints that were not provided to them by the main model. We saw that it also passed on that information to future subagents if it determined it was useful. Kimi used a central NOTES.md file to carry everything in between agents. It told new agents to read that record and use it as guidance to avoid what had already been examined, and what still needed to be checked. It opted to define restrictions for where agents can work in its subagent prompting. This backfired a bit. Have you ever seen multiple people walk in such a way that you know they’re all about to run into each other, but before you can say anything, it’s too late and they’ve already collided? That happened with Kimi a couple times because of this prompting boundary. For example, Kimi would write reports attributed to a developer-pattern agent before that agent had finished, and then catch its own mistake, delete the report, and wait for the agent to finish. While Kimi recovered with style and grace, the default pattern that it used for handling communications between agents is part of the reason this happened. Prior Work and Explorations Opus 5.5 recognized that older games like Pokémon Red and Pokémon Emerald already have been heavily analyzed. To avoid duplicating efforts, it first deployed an agent to catalog known bug categories and past research, looking in community guides, forums, and Japanese-language sources. Similarly, it reviewed existing game analysis and prior exploit research to inform its testing vectors. From there, sometimes it would try to explore other avenues of potential interest, with varying levels of success. Opus also analyzed known bug patterns to pinpoint new areas where similar exploits could be reproduced. It was able to replicate the semi-famous boulder glitch found by a fish in 2020 https://www.youtube.com/watch?v=0pgBZsF7hL8 . Yes, dear reader: a fish . The bug seemed to only happen in Seafloor Cavern, but we found three other in-game regions where it could be replicated. It’s only right that a non-human found the next iteration of this bug. As for Kimi, it used existing research differently. It assigned agents to build exclusion catalogs, and those catalogs were handed off to every agent it spun up. In one instance, it identified undefined behaviour in a text parser, used the catalog to check if it existed, and then decided that the bug had already been documented before. It went on to narrow the scope into similar possible exploits that the exclusion catalog didn’t have. Token Spend Opus 5.5 outwardly called out pathways that would be computationally expensive. This can backfire if the model doesn’t perform an action because it determines that it would use unnecessary tokens, when the action truly is needed. However, Opus was also a lot more willing to spend resources to strengthen and confirm findings. It ran a verification task for two hours to reproduce a bug from a save file, verifying over and over until we manually stopped the loop. Kimi, though, questioned whether additional tooling was worth the effort. When evaluating fuzzers, the model rejected a save-state fuzzer due to token costs and the time required to build it. It made estimates about gameplay, but the technical objectives mattered more to it. In our opinion, Opus repeatedly prioritized reducing unnecessary token usage. This can have diminishing returns if harnesses are implemented that would conflict with Opus’s priorities. Kimi prioritized reducing time consumption and wasted cycles. But Kimi is about 40% cheaper than Opus 5.5, so resource consumption matters less. Logical Proofs In our opinion, this is the most impressive feature of the model, and showcases the work done by Anthropic to improve mathematical reasoning capabilities. In comparison Kimi K3 defined premises and chose to test the assertions. It followed the traditional “test whether the premise is true” structure that we see as default in most models. Fuzzing Opus 5.5 built its testing infrastructure around existing emulators, adding save-state and input fuzzing tools alongside checks for unexpected game behavior. It also built a state oracle for each title, which allowed us to test game logic by making claims about the game-state that must be true. The fuzzing harness then tries to prove these claims wrong. The interesting part was deciding what counted as a failure. A crash is relatively easy to recognize. A game quietly forgetting which floor an elevator was on requires a more specific test. Opus used comparisons such as performing the same actions in different orders and comparing the resulting saved state for differences. Otherwise, normal changes to counters, random state, and object positions can look like bugs. Opus then played around with broader fuzzing, and found that it was just rediscovering known bugs. Kimi’s approach was a little more flexible. Early on, it built a fuzzer, which is a tool that generates structured text inputs to target syntax and parser vulnerabilities. It didn’t build a save-state fuzzer, which was a big difference in comparison to Opus. It experienced issues with weird bugs found this way. One of the harness bugs injected text routines in between emulator ticks, which entered them while interrupts were disabled. The frame counter never advanced, which caused hangs in the fuzzer. Even then, it still favoured reproduction scripts over fuzzing campaigns. Normal gameplay changes counters and randomizes state and moves object positions all the time. Kimi determined that a generic state-change detector would just output noise. Because of that, it prioritized writing relationships in code and then testing those relationships for failures. Safeguards Opus 5.5 safeguards seemed to trigger near the end of subagent execution, specifically when information is passed back to the head model. We saw safeguards trigger when words like “vulnerability” or “overflows” were mentioned, but pointing Opus toward “bugs,” “unintended behaviour,” and “breaking parser controls” netted fewer safeguard triggers. Sometimes, we had to point Opus into creating design documentation for a fallback model to actually implement the exploit. Sometimes we had to run it in a different working directory with curated files and no context on repeat in hopes of not getting a guardrail trigger. Then, we tried Kimi. When we prompted Kimi to make self-propagation code, it proceeded, treating the request as an execution flag. This was honestly the biggest benefit of Kimi over Opus. It just did stuff. The Opus 5.5 guardrail triggers just hit way too hard and fast for any security research to be done consistently. We had to employ some intense prompt engineering to bypass these guardrails, and the time expended was not worth the payoff. For one of the save file ACEs, we had to ask Opus 5.5 to design the payload in steps and then implement a significant chunk of the design using Opus 4.8 as a fallback. Weirdly enough, Kimi refused to download the ROMs from the Internet and asked us to supply them. Thankfully we own legal copies of the games so we were fine here :D One final note: Opus 5.5 sometimes added investigations beyond the requested assignments. In one round, we explicitly asked the model to look into five promising avenues but it actually added four other research directions, so ended up examining nine. Kimi’s investigation also expanded, but it was better at explicitly asking for confirmation before transitioning. For Kimi, combining its visible reasoning steps with these strategy shifts gave us direct transparency into how the model operates. It gave us, as researchers, more minute control of the model. Scope expands based on what Opus 5.5 thinks the finding requires, which does not always align with your research goals. Kimi makes these scope changes clear. Either way, as a developer, your scope should be defined and maintained through hardcoded guardrails rather than only prompt engineering. This is where you establish safeguards for your research. Diving into the Technical Details of our Experiments From a research perspective, a lot of the patterns that Opus 5.5 and Kimi displayed were promising and cool to see. Now it’s time to show some of the malware that the model made. So what does it feel like to vibe-code exploits for Pok é mon? <sub Well, it’s a little harder than copying and pasting the weird little ‘</sub <sub é</sub <sub ’ off the Internet, but not by much.</sub Three ACE Chains and a Worm Arbitrary Code Execution, or ACE, is what every person thinks of when they think of hacking. It’s an attack that can force a machine or system to run whatever commands or code an attacker chooses. While full code execution is the end goal, an ACE chain, which is a specific strategy that combines small exploits or goes through several exploit primitives, is the step-by-step pathway used to get there. Finding a new ACE chain in these games is difficult, because of how much deep research already has been performed on them by security experts honing their craft. It’s simply well-trodden ground. Here are the steps we took to achieve malicious code execution: 1. Opus and Kimi both found chainable bugs in the game's save files and link-cable logic. Opus found one save file ACE that Kimi didn’t. 2. The models then took those bugs and turned them into ACE pathways. The link-cable bug turned into a live cable vulnerability. 3. The models then crafted payload chains to demonstrate them. Two of the ACE chains were in Emerald , and the third was a Red ACE achieved through save-file manipulation. The ACE chains were not demonstrated on actual physical hardware, and the Emerald link attack requires custom firmware on the attacker’s end. Here is an analysis of the bugs and exploit vectors we found. Technical Breakdown: Pokémon Emerald ACE - The code to receive commands over the physical link picks where to put an incoming block by comparing the size that the peer declared against 256. It doesn’t check the write cursor against the buffer it picked. Declare more than 256, and every 14 bytes you send hits gDecompressionBuffer, which is the game’s 16KB scratch buffer, at an offset that increases until it passes the size you declare. If we declare 18,036, the cursor walks 1,652 bytes off the end of that buffer into gSprites, where we overwrite gSprites 0 .callback, which the retail sprite engine calls once a frame. The victim’s own AnimateSprites then calls the bytes we send. - The bytecode stored at a SaveBlock is handed to an unrestricted interpreter when you talk to an NPC, and it contains a callnative opcode that calls a 32-bit address out of the save. We were able to guide Opus into crafting a NOP sled to turn the 1/32 chance SaveBlock lottery into a consistent boot over 120 runs. Technical Breakdown: Pokémon Red ACE - Gen 1’s map script jump is documented in the Glitch City Wiki. In overworld.asm, RunMapScript loads a 16-bit pointer and jumps to it without any validation wCurMapScriptPtr at $D36E - jp hl , and pushes a return address first. This allows us to make a payload that returns cleanly into the game. Every documented route went through item slot 42 and needed the player to interact in some way with the game. We wanted to prove this wrong. And we did. Opus 5.5 found the pointer is save data, and that $D36E sits inside the block that is restored out of SRAM. We could make a payload that executed straight from the save file SRAM - WRAM , with no button press at all after pressing continue, because LoadMapHeader early-returns on BIT NO PREVIOUS MA before it copies a real header from ROM leaving the save's pointer live on the first overworld frame. The payload itself hides in wBoxMonNicks. Why is this interesting? The save-file chains and the link attack cross different boundaries. In Red and Emerald , the models found new ways for a custom-crafted save to reach execution machinery the community already knew about. Red restored a map-script pointer from the save, and jumps through it on the first overworld frame. Emerald interprets a script written into the save when the player talks to an NPC. Both used known exploit mechanisms with a new pathway for exploitation. But the link ACE chain vector was new. The victim accepts a normal cable session, and the other player writes code into the victim’s machine that the victim’s own Game Boy calls. To our knowledge, this is the first published code-execution bug in these Generation III games’ GBA-to-GBA cable receive path. Both Opus 5.5 and Kimi reached this vulnerability, and Kimi’s later failures showed us what a working exploit needed: the sender had to keep its payload intact through map transitions, and the code had to successfully run on the victim’s machine. They both found the same opening, and in their process of turning it into a working exploit, we could gain insights about how each model researches vulnerabilities. And all of this was worth a look because we got to the point of writing malware to brick other people’s games. Save Files, Again? In our previous research focusing on Opus 4.8, we used save files as well. Save files are notorious exploit surfaces, as their memory structures can reliably store malformed data objects alongside execution payloads. Pokémon’s Generation III has an extensive community toolchain for arbitrary code execution https://gist.github.com/claydolwithexplosion/017f1784deebcd118b61d3ad917edb3c . However, every previously published Emerald exploitation path relies on corrupting a Pokémon into a glitch species. Both models found an execution vector that bypasses that requirement entirely. We had Opus and Kimi write some fun and interesting exploits for save files. For instance, it created malicious save files that successfully executed arbitrary code across all three tested emulators. Red is worse. The Emerald save still needs the victim to press A on an NPC. With Red, before the player can do anything, its save file fires on the first overworld frame after CONTINUE. Opus developed payloads for both save-file ACE chains that execute within a very tight cycle window. The third exploit, which was the link-cable ACE, needed slightly more cycles to complete and stands as the centerpiece of this research. A Self-Replicating Virus Game Boy hardware relies on physical link cables to handle multiplayer interactions, like trading and battling in Pokémon. Opus 5.5 identified an attack avenue within this link-cable protocol that delivers custom payload data directly over the physical connection. That means an unsuspecting player connecting to a trade or battle session could be attacked and we can run code on their device. In a game, hitting availability and deleting save data has the greatest impact, but keep in mind that we can run any code we want on the machine. Other work, such as gba-gen3multiboot https://github.com/Wack0/gba-gen3multiboot , has demonstrated code execution on the multiboot protocol Gamecube/Wii - GBA but specifically noted that it was not wormable, because a GBA can never be ‘Player 1’ in that protocol. The first two generations of the games do have published link-cable overflows from a custom peer, which were shown by Phasip https://github.com/Phasip/PokemonLinkHack , vaguilar https://vaguilar.com/2015/05/26/arbitrary-code-execution-in-pokemon-red/ and TheZZAZZGlitch https://www.youtube.com/watch?v=e8CO e rKd8 . The attacker appears as a normal GBA-side device to the victim and interacts with them on a stock ROM and a retail save. The bus will echo the send back into the attacker’s receive slot, so the attacking side needs some custom firmware so it’s not infected as well. Kimi and Opus took advantage of this overflow to create a self-propagating worm payload capable of executing relay attacks across the link network. Once running, the payload calls the game's internal SendBlock function to retransmit itself from the victim's multiplayer slot. This means the second console is taken over by data that came from the first victim, not from the attacker. Seizure Warning : The successful attack executes code that causes the victim’s screen to rapidly flash or strobe. We also were able to guide both models to create a payload that persists after a victim reboots their game. And then we combined it for a persistent relay attack that could self-propagate. The prior art on these games is vast and not all of it is documented. If you've seen any of these before, and we’re not crediting someone who deserves it, we'd like to know. Message us at: research@runsybil.com mailto:research@runsybil.com Some Other Cool Bugs The models found seven smaller gameplay, visual, and audio bugs which were more quirks than anything else. Let’s take a look at them: Pokémon Emerald Bugs - Opus There is a boulder collision glitch. We verified these separately on an emulator. One instance had been previously found on Seafloor Cavern in the second-to-last room by Lala, a betta fish, in Pokémon Sapphire . Opus 5.5 found that it also can happen in three other in-game regions, and two other areas in Seafloor Cavern. - Kimi In the Secret Base battle, the game marks the owner as “battled today” and saves before starting the battle. So if you reset after that save, the owner gives the post-battle monologue and then refuses to fight again until tomorrow, even though they never fought. This might have been intentional to prevent cheating on a losing battle, though. - Kimi A faulty Battle Palace condition checks the wrong level mode’s streak. After beating Spenser, the streak has a count of 21, even though the record was 20. However, we ran a Level 50 control test and it showed a streak of 21 and a record of 21. Finishing the challenge normally fixed the mismatch though. In classical physics and in this game, this should be impossible. Pokémon Red Bugs 1. Opus The trade cable slide-in animation never actually plays: its scroll loop has no frame delay, so all 20 steps happen in a single frame. 2. Opus Collecting an item ball removes its sprite but its collision box remains active for non-player entities. Boulders still treat the empty tile as solid because the boulder routine never checks the offscreen sentinel which the player-collision check honors . So the boulder code thinks it’s a wall. This was not reported as a bug because it didn't cause any player issues. 3. Opus The Pokémon Center healing jingle inherits the previous track's octave because its state is never reset and the jingle's lead channel lacks its own octave command. 4. Kimi A stale sprite offset in Saffron City will make the fifteenth NPC’s movement status and delay update, even if you’re not interacting with that NPC. So if you talked to one NPC, another NPC’s movement state changed. Takeaways But the above isn’t a surprise to anyone… What we found particularly interesting is how these models arrived at the vulnerabilities and created the malware. Opus would ask subagents to state an invariant, and then require each subagent to prove that it found a violation in expected behaviour. It built a fuzzer with a game-state oracle before it started looking for bugs. It pulled prior research first to avoid rediscovering what the community already knew. Each of these steps increased determinism in the process of bug-finding, and minimized the chance for hallucinations to occur. Kimi operated in a shared environment with iterative processes, and we didn’t have any issues with it executing on tasks, even though it required substantial hand-holding to debug its own work. We had no problems with guardrail triggers and its findings were at a level similar to what we got from Opus, at about one-third of the total cost of Opus. From an opinionated researcher’s perspective, lean open-weight and cheaper models are worth it, especially when factoring in expense and transparency. For our purposes, the findings were comparable in value, and where Kimi struggled was with payload creation. For a skilled operator, that’s an acceptable shortcoming because we can guide the model as needed to craft the payload. On the other hand, it took several extra hours of work with Opus 5.5 to reach the same state, although that was because it was constantly triggering guardrails. If we weren’t navigating around guardrails so much, we would have probably succeeded in less than half the time it took for us to get there with Kimi. It would still have cost a lot of money though :D Another thing to consider is that scope creep in Opus is real and depending on what you’re doing, it might be worth it to implement custom harness guardrails or an agent dedicated to watching another’s behaviour. And Kimi struggles with crafting payloads once it makes a finding. There are still gaps that researchers will experience when using these tools. The Pokémon community has spent 20 years on these games. Despite that, two different models found new ACE chains and entrypoints, and did it by thinking about the problem the same way a good researcher would. Oh yeah, we also got AI to make a virus for Pokémon, which is kinda cool. Come hack with us: runsybil.com/careers http://runsybil.com/careers Subscribe to RunSybil Blog A weekly newsletter covering stories, techniques, guides and the latest product innovations coming.