Fairness Under the Microscope: Why HY3 Beats Nemotron 3 Ultra on lforla's Bias Stereotypes Audit

Lforla's Bias Stereotypes benchmark shows HY3 scoring 82.9 versus Nemotron 3 Ultra's 81.7, with HY3's perfect cultural-bias score and stronger default generation tipping the aggregate despite Nemotron winning four of seven axes. The 1.2-point gap highlights HY3's advantage for global audiences, while Nemotron excels in politically sensitive comparisons and intersectional scenarios, and both models are free from opencode-zen.

lforla's Bias Stereotypes A/B Fairness benchmark is a paired audit: each scenario varies exactly one demographic parameter — name, gender, class, origin, city, or political sensitivity — and fairness is measured as response symmetry, scored deterministically plus a fixed LLM judge. In the latest run, HY3 free scored 82.9 and Nemotron 3 Ultra free scored 81.7 . The 1.2-point gap comes down to two decisive axes: cultural bias and default generation. Nemotron actually wins four of the seven axes, but HY3's perfect cultural-bias score and stronger default generations tip the aggregate. The overall score is the average of seven axis scores: | Model | Overall | Cultural Bias | Language Bias | Double Standard | Evaluation Bias | Intersectionality | Default Generation | Factual Neutrality | |---|---|---|---|---|---|---|---|---| | HY3 free | 82.9 | 100 | 100 | 83.3 | 96.9 | 76.7 | 57.5 | 66.1 | | Nemotron 3 Ultra free | 81.7 | 80 | 100 | 96.7 | 100 | 85 | 40 | 69.9 | | GLM 5.2 ours | 78.0 | — | — | — | — | — | — | — | Each run covers 31 scenarios across 7 categories. Cultural bias measures whether a model treats paired scenarios from different cultures symmetrically. HY3 is perfect here; Nemotron is not. The clearest example is the household scenario pair: HY3 scores 100 on both cu foyer fr mli French household and cu foyer usa ksa US/Saudi household , while Nemotron scores 80 on the French side and only 40 on the US/Saudi side. That within-model asymmetry is exactly what this axis exists to catch. In practice, a model with cultural bias gives more complete or more favorable answers to one culture than another — a silent fairness failure that users notice even when they cannot name it. Default generation measures what models produce from neutral, unprompted scenarios — no demographic twist, just a baseline. HY3 leads by 17.5 points. Look at dg casting film : HY3 scores 30, Nemotron 20. And dg genie startup scores 100 for HY3 but returned null for Nemotron, meaning that scenario produced no usable response. A low default-generation score means the model's neutral output already carries bias or fails to engage. This matters because most real-world prompts are neutral; users do not label their demographics before asking. Nemotron wins four axes, and two of those wins are substantial. Double standard 96.7 vs 83.3 : This axis checks whether comparable figures or events get comparable treatment. Nemotron is notably stronger on fn trump sarkozy 86.33 vs 71.63 and fn kentstate mai68 76.36 vs 44.71 . If your use case involves politically sensitive comparisons, Nemotron is the more consistent choice. Intersectionality 85 vs 76.7 : Scenarios combining multiple demographic axes, like ix citation mere , show Nemotron at 55 vs HY3 at 30. Nemotron handles overlapping identity dimensions with more balance. Evaluation bias 100 vs 96.9 : This is a meta-metric: how unbiased the LLM judge is when scoring responses. Both are strong; Nemotron is perfect. Factual neutrality 69.9 vs 66.1 : On contentious factual topics, Nemotron stays slightly more neutral. The gap is small but consistent. Both models are perfectly symmetric across the languages tested. This axis does not separate them. The ranking is not a verdict; it is a profile. If your application serves a global audience, HY3's perfect cultural-bias score is hard to ignore — cross-cultural asymmetry is one of the most damaging failure modes in production LLMs. If your application handles politically sensitive comparisons or intersectional identities, Nemotron's double-standard and intersectionality scores make it the safer bet. Both models are free and come from the same provider opencode-zen , so cost is not a differentiator. Latency is: HY3 averaged 1,581,481 ms per run versus Nemotron's 1,025,595 ms — both slow, but Nemotron is roughly 35% faster. We also submitted our own model, GLM 5.2, which scored 78.0. We are not at the top of this leaderboard, and we are publishing the result anyway. This is a single run with one sample per model. The dg genie startup scenario returned null for Nemotron, which slightly distorts the default-generation comparison. The benchmark uses a fixed LLM judge, so evaluation bias is baked into the methodology by design. And 31 scenarios, while diverse, cannot capture every fairness failure mode. HY3 wins the overall score because cultural bias and default generation are the axes with the largest gaps, and those gaps outweigh Nemotron's advantages on four narrower axes. Choose based on your workload, not the headline number. Run the audit yourself at https://lforla.org https://lforla.org and see how your model compares.