Evaluating GLM 5.3 Flash on Hack The Box Challenges

I had already evaluated GLM 5.3 Flash when Z.ai previewed it anonymously as the mysterious Ox Alpha on OpenRouter, and its performance was fantastic. The one thing I couldn’t measure was cost-effectiveness because the preview was free. Now that Z.ai has revealed Ox Alpha as GLM 5.3 Flash and published its pricing, I could return to it and measure the final missing piece.

This blog post is part of a series of tests for the

[HTB-Challenger Benchmark]The HTB-Challenger Benchmark evaluates LLMs’ ability to find and exploit security vulnerabilities. It tests models against selected Hack The Box challenges of varying difficulty and measures their performance. For more information, visit the[HTB-Challenger Benchmark page]. . See the[benchmark results page]for all results and the[benchmark methodology]to learn how the benchmark is calculated.

My new test closely reproduced the earlier Ox Alpha result: 81.4% in the anonymous preview run and 81.0% after release. It means, that GLM 5.3 Flash has the second-highest score after GPT-5.6 Sol (87.2%). Both models correctly solved the same 15 out of 16 challenges. Flash’s lower score came from requiring more model steps on the solved challenges: its median was 24.5, compared with Sol’s 8.5.

Anyway, despite needing more steps and tokens, GLM 5.3 Flash’s median cost per challenge was about one-twentieth of GPT-5.6 Sol’s ($0.0140 vs. $0.2898). In this benchmark, that meant the same number of correct solves at a fraction of the measured cost. Flash was even cheaper than GPT-5.6 Luna, whose median cost was $0.0181. As a side note, Flash was under a temporary 50% launch discount at the time of my test, so this should not be read as the permanent list-price ratio. But even at 100% of its price, it would still be very competitive.

At the moment, GLM 5.3 Flash has replaced GPT-5.6 Luna as my go-to model, not just for cybersecurity work. It was less step-efficient than Sol, but no other included model in this benchmark achieved both a higher score and a lower median cost.

GLM 5.3 Flash is not a smaller GLM 5.3 #

One final note on this model’s name. GLM 5.3 Flash may sound like a smaller version of GLM 5.3, but the two models have different origins. According to its official model card, Flash is a 320B-total, 18B-active model trained from a new 30T-token multimodal base, with a new architecture combining sparse and linear attention. By contrast, Z.ai says GLM 5.3 reuses the GLM 5.2 base and derives all its improvements from additional post-training. The published GLM 5.2 artifact - the base reused by GLM 5.3 - is listed at 753B parameters. They are therefore separate branches of the GLM family, not full-size and compressed versions of the same model.

I tested GLM 5.3 just a few days earlier, and it performed much worse in this benchmark: it scored 47.2% and solved 10 challenges, compared with Flash’s 81.0% and 15 solves. Its measured median cost was also $0.4915, compared with Flash’s $0.0140, so I’m not sure why Z.ai decided to use its brand for the new model. GLM 5.4 Flash would have made much more sense to me.

Cost vs. Benchmark Score #

The highlighted point is this model. Models closer to the upper-left achieve a higher benchmark score at a lower median cost per challenge.

Overall benchmark results #

Number of challenges: 16Number of solved challenges: 15Number of false positives: 0Runs where the model gave up: 0Runs that reached the step or cost limit: 0Runs where the model got stuck: 1Benchmark score: 81.0%

Results by challenge difficulty #

All resource-usage metrics are medians per challenge.