# Solving Hack The Box Challenges with GPT-5.6 > Source: > Published: 2026-08-17 06:10:31+00:00 # Solving Hack The Box Challenges with GPT‑5.6 Until now, I had tested only the GPT-5.6 Luna model because the two more advanced GPT-5.6 models, Terra and Sol, rejected my test prompts, flagging them as a “possible cybersecurity risk.” Fortunately, this issue turned out to have an easy solution, so I could finally spend a couple of days testing the whole GPT-5.6 family for my [HTB-Challenger Benchmark](/htb-challenger-benchmark/)The HTB-Challenger Benchmark evaluates LLMs’ ability to find and exploit security vulnerabilities. It tests models against selected Hack The Box challenges of varying difficulty and measures their performance. For more information, visit the [HTB-Challenger Benchmark page](/htb-challenger-benchmark/). . This blog post is part of a series of tests for the [HTB-Challenger Benchmark]The HTB-Challenger Benchmark evaluates LLMs’ ability to find and exploit security vulnerabilities. It tests models against selected Hack The Box challenges of varying difficulty and measures their performance. For more information, visit the[HTB-Challenger Benchmark page]. . See the[benchmark results page]for all results and the[benchmark methodology]to learn how the benchmark is calculated. ## Trusted Access for Cyber program When I started testing a week or so ago, I couldn’t run my test harness with the GPT-5.6 Terra or Sol models - I kept getting this error: `This content was flagged for possible cybersecurity risk. If this seems wrong, try rephrasing your request. To get authorized for security work, join the Trusted Access for Cyber program: https://chatgpt.com/cyber` . At first, I gave up. Trying to join the Trusted Access for Cyber program seemed pointless to me. I was sure the whole program had been created just to give desperate OpenAI users some hope and that my application would end up in `/dev/null` . But then I discovered [some articles](https://ai.georgeliu.com/p/chatgpt-codex-flagged-my-security) that gave me real hope, so I clicked on the [https://chatgpt.com/cyber](https://chatgpt.com/cyber) link. As it turned out, joining the Trusted Access for Cyber program only involved going through a KYC process similar to the one required when opening a new bank account. After I joined the program, only one of my prompts was rejected, which was a significant improvement. ## Tested models I tested the following GPT-5.6 models. Click on a model name to see its detailed test results: GPT-5.6 Luna ProandGPT-5.6 Terra Proare models hosted on[OpenRouter.io], and they are simply theLunaandTerramodels served with`reasoning.mode` set to`pro` . I also tested the **GPT-5.6 Sol Pro** model, but the results were, well, weird. The **Sol Pro** version generated a lot of tokens and took a long time on each step, but that did not always translate into better results. I also have a hard limit of $15 on the cost of a single test run, and this model hit that limit quite often. I’m not sure if “overthinking” is the best technical term here, but that’s what it looked like to me. In the end, I decided not to include this model version in my results, but if there is one thing I can say based on my partial testing, it is that **GPT-5.6 Sol Pro** is much more expensive than **GPT-5.6 Sol**. ## Results Let’s start with this chart. The five GPT-5.6 models tested for this post are highlighted in gold, so you can see their results in the context of the other models I have tested. The vertical axis shows the score on my [HTB-Challenger Benchmark](/htb-challenger-benchmark/)The HTB-Challenger Benchmark evaluates LLMs’ ability to find and exploit security vulnerabilities. It tests models against selected Hack The Box challenges of varying difficulty and measures their performance. For more information, visit the [HTB-Challenger Benchmark page](/htb-challenger-benchmark/). , while the horizontal axis shows the median cost per test (mind you, this axis uses a logarithmic scale). Before I start interpreting these results, let me also show you a table with more statistics for each model. | Metric | Luna | Luna Pro | Terra | Terra Pro | Sol | |---|---|---|---|---|---| Results | ||||| | Number of challenges | 16 | 16 | 16 | 16 | 16 | | Number of solved challenges | 8 | 11 | 12 | 14 | 15 | | Number of false positives | 0 | 0 | 0 | 0 | 1 | | Runs where the model gave up | 8 | 4 | 4 | 1 | 0 | | Runs that reached the step or cost limit | 0 | 1 | 0 | 0 | 0 | | Runs where the model got stuck | 0 | 0 | 0 | 0 | 0 | | Benchmark score | 31.6% | 51.3% | 64.6% | 75.1% | 87.2% | Median per challenge | ||||| | Model steps | 17.5 | 21.5 | 16.5 | 11 | 8.5 | | Model cost | $0.02 | $0.10 | $0.24 | $0.91 | $0.29 | | Duration | 00:04:38 | 00:08:55 | 00:03:16 | 00:08:22 | 00:02:05 | | Number of input tokens | 0.34M | 1.50M | 0.23M | 0.64M | 0.09M | | Number of output tokens | 0.02M | 0.07M | 0.01M | 0.03M | 0.00M | Number of `read_file` tool calls | 1.0 | 4.0 | 0.0 | 0.0 | 0.5 | Number of `write_file` tool calls | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | Number of `execute_command` tool calls | 28.0 | 36.5 | 15.5 | 13.5 | 8.5 | Number of `web_search` tool calls | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ## What I take from these results At the low-cost end, **Luna** is in a category of its own. A median challenge run cost just $0.02, which is remarkably cheap even by LLM standards. Its 31.6% benchmark score was the lowest in this comparison, but that doesn’t tell the whole story: it still found the correct flag in seven of the eight Very Easy and Easy challenges. I see it as an ideal first-pass model - let **Luna** take a shot at the task, and bring in something more powerful only when it reaches its limits. And then there is **Sol**. With a score of 87.2%, **GPT-5.6 Sol achieved the best score in my benchmark so far**. It solved 15 of 16 challenges, including three out of four Hard ones. What makes this result even more impressive is that its median cost was only $0.29 per challenge - just $0.05 more than **Terra** and less than one-third of **Terra Pro**’s $0.91. It was also the fastest model in this comparison, with a median of 8.5 steps and 2 minutes and 5 seconds per challenge. The only blemish in its results was a false positive on the remaining Hard challenge, which no model in my benchmark has solved so far. But this was not a random or nonsensical guess: the submitted flag was plausible and differed only slightly from the correct one. So even **Sol**’s only failure was actually a near miss. The results of the Pro variants are a bit more complicated. Setting `reasoning.mode` to `pro` improved both underlying models: **Luna Pro** gained 19.7 percentage points over **Luna**, and **Terra Pro** gained 10.5 points over **Terra**. But this extra reasoning came at a price. **Luna Pro** was five times more expensive than **Luna**, while **Terra Pro** was almost four times more expensive than **Terra**. Both Pro variants also used roughly three to four times as many input tokens per challenge and took around two to three times as long. And in both cases, the next model tier still performed better than the Pro version of the lower tier: **Terra** beat **Luna Pro**, and **Sol** beat **Terra Pro**. The differences become most visible as the challenges get harder. Every model scored at least 97% on Very Easy challenges, so paying for the most powerful model there doesn’t make much sense. On Medium challenges, however, the scores ranged from 24.7% for **Luna** to 97.6% for **Sol**. On Hard challenges, they ranged from 0% for **Luna** to 71.8% for **Sol**. So my practical conclusion is quite simple. If I had a straightforward task or wanted to run a large number of inexpensive experiments, I would start with **Luna**. If the task were difficult and getting the correct result mattered more than saving every last cent, I would choose **Sol**. **Luna Pro** is still an interesting and affordable middle ground, but **Terra** is difficult to recommend when **Sol** costs only slightly more and performs much better. **Terra Pro** makes even less sense to me: in these tests, **Sol** was more capable, faster, and much cheaper.