{"slug": "evaluating-glm-5-3-flash-on-hack-the-box-challenges", "title": "Evaluating GLM 5.3 Flash on Hack The Box Challenges", "summary": "Z.ai's GLM 5.3 Flash scored 81.0% on the HTB-Challenger Benchmark, solving 15 of 16 Hack The Box challenges, second only to OpenAI's GPT-5.6 Sol at 87.2%. Despite requiring more steps (median 24.5 vs. 8.5), Flash's median cost per challenge was $0.0140, about one-twentieth of Sol's $0.2898, making it the most cost-effective high-scoring model tested. The model, a 320B-total, 18B-active sparse-linear attention architecture trained on a new 30T-token multimodal base, is distinct from the larger GLM 5.3, which scored only 47.2%.", "body_md": "# Evaluating GLM 5.3 Flash on Hack The Box Challenges\n\nI had already evaluated **GLM 5.3 Flash** when Z.ai previewed it anonymously as the mysterious [Ox Alpha](/2026/08/21/evaluating-stealth-ox-alpha-on-hack-the-box-challenges.html) on OpenRouter, and its performance was fantastic. The one thing I couldn’t measure was cost-effectiveness because the preview was free. Now that Z.ai has [revealed Ox Alpha as GLM 5.3 Flash](https://z.ai/blog/glm-5.3-flash) and published its pricing, I could return to it and measure the final missing piece.\n\nThis blog post is part of a series of tests for the\n\n[HTB-Challenger Benchmark]The HTB-Challenger Benchmark evaluates LLMs’ ability to find and exploit security vulnerabilities. It tests models against selected Hack The Box challenges of varying difficulty and measures their performance. For more information, visit the[HTB-Challenger Benchmark page]. . See the[benchmark results page]for all results and the[benchmark methodology]to learn how the benchmark is calculated.\n\nMy new test closely reproduced the earlier **Ox Alpha** result: 81.4% in the anonymous preview run and 81.0% after release. It means, that **GLM 5.3 Flash** has the second-highest score after [GPT-5.6 Sol](/2026/08/16/solving-htb-challenges-with-openai-gpt-5.6-sol.html) (87.2%). Both models correctly solved the same 15 out of 16 challenges. Flash’s lower score came from requiring more model steps on the solved challenges: its median was 24.5, compared with Sol’s 8.5.\n\nAnyway, despite needing more steps and tokens, **GLM 5.3 Flash’s** median cost per challenge was about one-twentieth of **GPT-5.6 Sol’s** ($0.0140 vs. $0.2898). In this benchmark, that meant the same number of correct solves at a fraction of the measured cost. Flash was even cheaper than [GPT-5.6 Luna](/2026/08/10/solving-htb-challenges-with-openai-gpt-5.6-luna.html), whose median cost was $0.0181. As a side note, Flash was under a temporary 50% launch discount at the time of my test, so this should not be read as the permanent list-price ratio. But even at 100% of its price, it would still be very competitive.\n\nAt the moment, **GLM 5.3 Flash** has replaced **GPT-5.6 Luna** as my go-to model, not just for cybersecurity work. It was less step-efficient than Sol, but no other included model in this benchmark achieved both a higher score and a lower median cost.\n\n## GLM 5.3 Flash is not a smaller GLM 5.3\n\nOne final note on this model’s name. **GLM 5.3 Flash** may sound like a smaller version of [GLM 5.3](/2026/08/21/evaluating-z-ai-glm-5.3-on-hack-the-box-challenges.html), but the two models have different origins. According to its [official model card](https://huggingface.co/zai-org/GLM-5.3-Flash), Flash is a 320B-total, 18B-active model trained from a new 30T-token multimodal base, with a new architecture combining sparse and linear attention. By contrast, [Z.ai says GLM 5.3](https://z.ai/blog/glm-5.3) reuses the GLM 5.2 base and derives all its improvements from additional post-training. The published [GLM 5.2 artifact](https://huggingface.co/zai-org/GLM-5.2) - the base reused by GLM 5.3 - is listed at 753B parameters. They are therefore separate branches of the GLM family, not full-size and compressed versions of the same model.\n\nI tested **GLM 5.3** just a few days earlier, and it performed much worse in this benchmark: it scored 47.2% and solved 10 challenges, compared with Flash’s 81.0% and 15 solves. Its measured median cost was also $0.4915, compared with Flash’s $0.0140, so I’m not sure why Z.ai decided to use its brand for the new model. **GLM 5.4 Flash** would have made much more sense to me.\n\n## Cost vs. Benchmark Score\n\nThe highlighted point is this model. Models closer to the upper-left achieve a higher benchmark score at a lower median cost per challenge.\n\n## Overall benchmark results\n\n**Number of challenges:** 16**Number of solved challenges:** 15**Number of false positives:** 0**Runs where the model gave up:** 0**Runs that reached the step or cost limit:** 0**Runs where the model got stuck:** 1**Benchmark score:** 81.0%\n\n| Metric | Per challenge (median) | Total |\n|---|---|---|\n| Model steps | 24.5 | 507 |\n| Model cost | $0.01 | $0.39 |\n| Duration | 00:10:52 | 04:50:09 |\n| Number of input tokens | 0.45M | 12.73M |\n| Number of output tokens | 0.02M | 0.50M |\nNumber of `read_file` tool calls |\n1.0 | 29 |\nNumber of `write_file` tool calls |\n2.5 | 80 |\nNumber of `execute_command` tool calls |\n22.0 | 481 |\nNumber of `web_search` tool calls |\n0.0 | 1 |\n\n## Results by challenge difficulty\n\nAll resource-usage metrics are medians per challenge.\n\n| Metric | Very Easy | Easy | Medium | Hard |\n|---|---|---|---|---|\nResults |\n||||\n| Number of challenges | 4 | 4 | 4 | 4 |\n| Number of solved challenges | 4 | 4 | 4 | 3 |\n| Number of false positives | 0 | 0 | 0 | 0 |\n| Runs where the model gave up | 0 | 0 | 0 | 0 |\n| Runs that reached the step or cost limit | 0 | 0 | 0 | 0 |\n| Runs where the model got stuck | 0 | 0 | 0 | 1 |\n| Benchmark score | 94.1% | 93.2% | 89.8% | 64.9% |\nMedian per challenge |\n||||\n| Model steps | 17.5 | 24 | 30 | 25 |\n| Model cost | $0.01 | $0.01 | $0.03 | $0.03 |\n| Duration | 00:06:07 | 00:10:52 | 00:18:19 | 00:25:38 |\n| Number of input tokens | 0.32M | 0.21M | 0.86M | 0.68M |\n| Number of output tokens | 0.01M | 0.01M | 0.04M | 0.03M |\nNumber of `read_file` tool calls |\n0.5 | 1.0 | 1.0 | 2.0 |\nNumber of `write_file` tool calls |\n1.0 | 3.0 | 3.0 | 2.5 |\nNumber of `execute_command` tool calls |\n19.5 | 20.0 | 29.5 | 27.0 |\nNumber of `web_search` tool calls |\n0.0 | 0.0 | 0.0 | 0.0 |", "url": "https://wpnews.pro/news/evaluating-glm-5-3-flash-on-hack-the-box-challenges", "canonical_source": "https://theaq.blog/2026/08/28/evaluating-z-ai-glm-5.3-flash-on-hack-the-box-challenges.html", "published_at": "2026-08-28 07:57:22+00:00", "updated_at": "2026-08-28 08:49:35.954272+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "ai-products"], "entities": ["Z.ai", "GLM 5.3 Flash", "GPT-5.6 Sol", "GPT-5.6 Luna", "Hack The Box", "HTB-Challenger Benchmark", "OpenRouter", "GLM 5.3"], "alternates": {"html": "https://wpnews.pro/news/evaluating-glm-5-3-flash-on-hack-the-box-challenges", "markdown": "https://wpnews.pro/news/evaluating-glm-5-3-flash-on-hack-the-box-challenges.md", "text": "https://wpnews.pro/news/evaluating-glm-5-3-flash-on-hack-the-box-challenges.txt", "jsonld": "https://wpnews.pro/news/evaluating-glm-5-3-flash-on-hack-the-box-challenges.jsonld"}}