Evaluating GLM 5.3 Flash on Hack The Box Challenges

Z.ai's GLM 5.3 Flash scored 81.0% on the HTB-Challenger Benchmark, solving 15 of 16 Hack The Box challenges, second only to OpenAI's GPT-5.6 Sol at 87.2%. Despite requiring more steps (median 24.5 vs. 8.5), Flash's median cost per challenge was $0.0140, about one-twentieth of Sol's $0.2898, making it the most cost-effective high-scoring model tested. The model, a 320B-total, 18B-active sparse-linear attention architecture trained on a new 30T-token multimodal base, is distinct from the larger GLM 5.3, which scored only 47.2%.

Evaluating GLM 5.3 Flash on Hack The Box Challenges I had already evaluated GLM 5.3 Flash when Z.ai previewed it anonymously as the mysterious Ox Alpha /2026/08/21/evaluating-stealth-ox-alpha-on-hack-the-box-challenges.html on OpenRouter, and its performance was fantastic. The one thing I couldn’t measure was cost-effectiveness because the preview was free. Now that Z.ai has revealed Ox Alpha as GLM 5.3 Flash https://z.ai/blog/glm-5.3-flash and published its pricing, I could return to it and measure the final missing piece. This blog post is part of a series of tests for the HTB-Challenger Benchmark The HTB-Challenger Benchmark evaluates LLMs’ ability to find and exploit security vulnerabilities. It tests models against selected Hack The Box challenges of varying difficulty and measures their performance. For more information, visit the HTB-Challenger Benchmark page . . See the benchmark results page for all results and the benchmark methodology to learn how the benchmark is calculated. My new test closely reproduced the earlier Ox Alpha result: 81.4% in the anonymous preview run and 81.0% after release. It means, that GLM 5.3 Flash has the second-highest score after GPT-5.6 Sol /2026/08/16/solving-htb-challenges-with-openai-gpt-5.6-sol.html 87.2% . Both models correctly solved the same 15 out of 16 challenges. Flash’s lower score came from requiring more model steps on the solved challenges: its median was 24.5, compared with Sol’s 8.5. Anyway, despite needing more steps and tokens, GLM 5.3 Flash’s median cost per challenge was about one-twentieth of GPT-5.6 Sol’s $0.0140 vs. $0.2898 . In this benchmark, that meant the same number of correct solves at a fraction of the measured cost. Flash was even cheaper than GPT-5.6 Luna /2026/08/10/solving-htb-challenges-with-openai-gpt-5.6-luna.html , whose median cost was $0.0181. As a side note, Flash was under a temporary 50% launch discount at the time of my test, so this should not be read as the permanent list-price ratio. But even at 100% of its price, it would still be very competitive. At the moment, GLM 5.3 Flash has replaced GPT-5.6 Luna as my go-to model, not just for cybersecurity work. It was less step-efficient than Sol, but no other included model in this benchmark achieved both a higher score and a lower median cost. GLM 5.3 Flash is not a smaller GLM 5.3 One final note on this model’s name. GLM 5.3 Flash may sound like a smaller version of GLM 5.3 /2026/08/21/evaluating-z-ai-glm-5.3-on-hack-the-box-challenges.html , but the two models have different origins. According to its official model card https://huggingface.co/zai-org/GLM-5.3-Flash , Flash is a 320B-total, 18B-active model trained from a new 30T-token multimodal base, with a new architecture combining sparse and linear attention. By contrast, Z.ai says GLM 5.3 https://z.ai/blog/glm-5.3 reuses the GLM 5.2 base and derives all its improvements from additional post-training. The published GLM 5.2 artifact https://huggingface.co/zai-org/GLM-5.2 - the base reused by GLM 5.3 - is listed at 753B parameters. They are therefore separate branches of the GLM family, not full-size and compressed versions of the same model. I tested GLM 5.3 just a few days earlier, and it performed much worse in this benchmark: it scored 47.2% and solved 10 challenges, compared with Flash’s 81.0% and 15 solves. Its measured median cost was also $0.4915, compared with Flash’s $0.0140, so I’m not sure why Z.ai decided to use its brand for the new model. GLM 5.4 Flash would have made much more sense to me. Cost vs. Benchmark Score The highlighted point is this model. Models closer to the upper-left achieve a higher benchmark score at a lower median cost per challenge. Overall benchmark results Number of challenges: 16 Number of solved challenges: 15 Number of false positives: 0 Runs where the model gave up: 0 Runs that reached the step or cost limit: 0 Runs where the model got stuck: 1 Benchmark score: 81.0% | Metric | Per challenge median | Total | |---|---|---| | Model steps | 24.5 | 507 | | Model cost | $0.01 | $0.39 | | Duration | 00:10:52 | 04:50:09 | | Number of input tokens | 0.45M | 12.73M | | Number of output tokens | 0.02M | 0.50M | Number of read file tool calls | 1.0 | 29 | Number of write file tool calls | 2.5 | 80 | Number of execute command tool calls | 22.0 | 481 | Number of web search tool calls | 0.0 | 1 | Results by challenge difficulty All resource-usage metrics are medians per challenge. | Metric | Very Easy | Easy | Medium | Hard | |---|---|---|---|---| Results | |||| | Number of challenges | 4 | 4 | 4 | 4 | | Number of solved challenges | 4 | 4 | 4 | 3 | | Number of false positives | 0 | 0 | 0 | 0 | | Runs where the model gave up | 0 | 0 | 0 | 0 | | Runs that reached the step or cost limit | 0 | 0 | 0 | 0 | | Runs where the model got stuck | 0 | 0 | 0 | 1 | | Benchmark score | 94.1% | 93.2% | 89.8% | 64.9% | Median per challenge | |||| | Model steps | 17.5 | 24 | 30 | 25 | | Model cost | $0.01 | $0.01 | $0.03 | $0.03 | | Duration | 00:06:07 | 00:10:52 | 00:18:19 | 00:25:38 | | Number of input tokens | 0.32M | 0.21M | 0.86M | 0.68M | | Number of output tokens | 0.01M | 0.01M | 0.04M | 0.03M | Number of read file tool calls | 0.5 | 1.0 | 1.0 | 2.0 | Number of write file tool calls | 1.0 | 3.0 | 3.0 | 2.5 | Number of execute command tool calls | 19.5 | 20.0 | 29.5 | 27.0 | Number of web search tool calls | 0.0 | 0.0 | 0.0 | 0.0 |