18:45
2026-08-01
discuss.huggingface.co
artificial-intelligence
Reliability check on my own dataset's annotation layer: five machine raters, one definition, answers from 0 to 78
A reliability benchmark of machine-generated annotations on a Turkish narrative corpus found that five of six automated raters performed at or near chance (Cohen's κ between 0.000 and 0.027) on the on…