ConceptNet — 4-layer enterprise voice intent classifier, 98.6% accuracy, 9 languages, token-free, open source

Independent testing of the open-source ConceptNet enterprise voice intent classifier, developed by wushu75 and hosted on Hugging Face as conceptnetUk/intent-classifier, reproduced its reported 98.6% accuracy across 9 languages on a reconstructed 80/20 split, and the neural model's performance held under a grouped holdout, suggesting the high accuracy is not due to template leakage. The evaluator found that the L3/Predictive layer shows less diversity in expressing its core temporal relation than L2 or L4, and recommended further testing on varied expressions and multi-relation utterances.

For now, here’s what I found from some light testing in Colab: I tried to reproduce the public numbers first, then deliberately looked for easy failure modes rather than assuming the headline accuracy was telling the whole story. The short version is: the basic result held up better than I expected . On the public 80/20-style split I reconstructed, I got: That is broadly consistent with the results described in the repository https://github.com/wushu75/ConceptNet and model card https://huggingface.co/conceptnetUk/intent-classifier . I also tried a stronger grouped holdout where related lexical families were kept together rather than randomly split. The neural model was still essentially unchanged: So I didn’t find evidence for the simple explanation that the reported neural accuracy is mostly coming from generic near-duplicate/template leakage. The more interesting signal was narrower: L3 / Predictive seems to have much less diversity in how its core temporal relation is expressed than L2 or L4 . If I were extending the evaluation, my default route would therefore be quite small: That would probably tell more than simply adding another large random test split. 1. What reproduced, including the negative resultOverall, the part I found most encouraging was actually the failed attempt to break the neural result with a generic lexical-family split. The obvious “99% only because the random split leaked the templates” explanation did not survive that check. The next useful question therefore seems narrower: whether the four execution semantics remain stable when the same relation is expressed differently, and when multiple relations appear in one utterance. If this were my evaluation budget, I would spend the next small increment on: Those are all relatively cheap, and each one answers a different deployment question without requiring a redesign of the core model.