{"slug": "conceptnet-4-layer-enterprise-voice-intent-classifier-98-6-accuracy-9-languages", "title": "ConceptNet — 4-layer enterprise voice intent classifier, 98.6% accuracy, 9 languages, token-free, open source", "summary": "Independent testing of the open-source ConceptNet enterprise voice intent classifier, developed by wushu75 and hosted on Hugging Face as conceptnetUk/intent-classifier, reproduced its reported 98.6% accuracy across 9 languages on a reconstructed 80/20 split, and the neural model's performance held under a grouped holdout, suggesting the high accuracy is not due to template leakage. The evaluator found that the L3/Predictive layer shows less diversity in expressing its core temporal relation than L2 or L4, and recommended further testing on varied expressions and multi-relation utterances.", "body_md": "For now, here’s what I found from some light testing in Colab:\n\nI tried to reproduce the public numbers first, then deliberately looked for easy failure modes rather than assuming the headline accuracy was telling the whole story.\n\nThe short version is: **the basic result held up better than I expected**.\n\nOn the public 80/20-style split I reconstructed, I got:\n\nThat is broadly consistent with the results described in the [repository](https://github.com/wushu75/ConceptNet) and [model card](https://huggingface.co/conceptnetUk/intent-classifier).\n\nI also tried a stronger grouped holdout where related lexical families were kept together rather than randomly split. The neural model was still essentially unchanged:\n\nSo I **didn’t** find evidence for the simple explanation that the reported neural accuracy is mostly coming from generic near-duplicate/template leakage.\n\nThe more interesting signal was narrower: **L3 / Predictive seems to have much less diversity in how its core temporal relation is expressed than L2 or L4**.\n\nIf I were extending the evaluation, my default route would therefore be quite small:\n\nThat would probably tell more than simply adding another large random test split.\n\n1. What reproduced, including the negative resultOverall, the part I found most encouraging was actually the failed attempt to break the neural result with a generic lexical-family split. **The obvious “99% only because the random split leaked the templates” explanation did not survive that check.**\n\nThe next useful question therefore seems narrower: whether the four execution semantics remain stable when the same relation is expressed differently, and when multiple relations appear in one utterance.\n\nIf this were my evaluation budget, I would spend the next small increment on:\n\nThose are all relatively cheap, and each one answers a different deployment question without requiring a redesign of the core model.", "url": "https://wpnews.pro/news/conceptnet-4-layer-enterprise-voice-intent-classifier-98-6-accuracy-9-languages", "canonical_source": "https://discuss.huggingface.co/t/conceptnet-4-layer-enterprise-voice-intent-classifier-98-6-accuracy-9-languages-token-free-open-source/179274#post_4", "published_at": "2026-08-27 14:59:22+00:00", "updated_at": "2026-08-27 15:19:03.424662+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "natural-language-processing", "ai-research", "ai-products"], "entities": ["ConceptNet", "wushu75", "Hugging Face", "conceptnetUk/intent-classifier"], "alternates": {"html": "https://wpnews.pro/news/conceptnet-4-layer-enterprise-voice-intent-classifier-98-6-accuracy-9-languages", "markdown": "https://wpnews.pro/news/conceptnet-4-layer-enterprise-voice-intent-classifier-98-6-accuracy-9-languages.md", "text": "https://wpnews.pro/news/conceptnet-4-layer-enterprise-voice-intent-classifier-98-6-accuracy-9-languages.txt", "jsonld": "https://wpnews.pro/news/conceptnet-4-layer-enterprise-voice-intent-classifier-98-6-accuracy-9-languages.jsonld"}}