Aura đŸ’«: A Self-Hostable Platform for Human Evaluation of NLG for Under-resourced Languages and Beyond Researchers led by Rudali Huidrom released Aura, an open-source, self-hostable platform that unifies five human-evaluation protocols (Likert/DA, pairwise, ranking, error-span, post-editing) across eight task families for Natural Language Generation, with a built-in randomized experiment measuring how an advisory-only LLM assistant affects inter-annotator agreement and time-on-task. In a human evaluation of three English→Indian-language ranking campaigns with N=9 annotators, Aura returned a mean usability rating of 4.06/5, and its agreement statistics matched reference implementations to machine precision. The work appears in the AACL-IJCNLP 2026 System Demonstrations proceedings, pages 9–16, and targets indigenous and under-resourced languages where paid crowds and references are scarce and data governance matters. Abstract Human evaluation is the gold standard for Natural Language Generation NLG , but the tooling is fragmented: most platforms serve a single task or a single protocol, and none treat the growing practice of assisting annotators with an LLM as a measurable object of study. These gaps fall hardest on indigenous and under-resourced languages, where paid crowds are scarce, references are unavailable, and data governance matters. We present Aura, an open, self-hostable platform that unifies five evaluation protocols Likert/DA, pairwise, ranking, error-span, post-editing across eight task families under one segment schema and one result model, records writing-system and speaker provenance as first-class fields, and runs entirely offline. Its distinguishing feature is a grounded, advisory-only, logged LLM assistant paired with a built-in randomised experiment that estimates the effect of AI assistance on inter-annotator agreement and time-on-task. Aura’s agreement statistics match reference implementations to machine precision. We conducted a human evaluation experiment on three English→Indian-language ranking campaigns N=9 annotators which returned a mean usability rating of 4.06/5. Aura is released as open-source software. - Anthology ID: - 2026.aacl-demo.2 - Volume: - Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics: System Demonstrations https://aclanthology.org/volumes/2026.aacl-demo/ - Month: - November - Year: - 2026 - Address: - Hengqin, China - Editors: - Ningyu Zhang https://aclanthology.org/people/ningyu-zhang/ , Derry Wijaya https://aclanthology.org/people/derry-tanti-wijaya/ - Venues: - AACL https://aclanthology.org/venues/aacl/ | IJCNLP https://aclanthology.org/venues/ijcnlp/ - SIG: - Publisher: - Association for Computational Linguistics - Note: - Pages: - 9–16 - Language: - URL: - https://aclanthology.org/2026.aacl-demo.2/ https://aclanthology.org/2026.aacl-demo.2/ - DOI: - Cite ACL : - Rudali Huidrom, Khangembam Alex D Nelson, Nicholas Khumukcham, Hoomexsun Pangsatabam, Jesu Nazarene Roy L, Kanal Trivedi, Vikas Kumar, Pinaki Das, Kshetrimayum Boynao Singh, Tarun Bisht, Justice Konjengbam, Konjengbam Anand, and Vikram Goyal. 2026. Aura đŸ’«: A Self-Hostable Platform for Human Evaluation of NLG for Under-resourced Languages and Beyond https://aclanthology.org/2026.aacl-demo.2/ . In Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics: System Demonstrations , pages 9–16, Hengqin, China. Association for Computational Linguistics. - Cite Informal : - Aura đŸ’«: A Self-Hostable Platform for Human Evaluation of NLG for Under-resourced Languages and Beyond https://aclanthology.org/2026.aacl-demo.2/ Huidrom et al., AACL-IJCNLP 2026 - PDF: - https://aclanthology.org/2026.aacl-demo.2.pdf https://aclanthology.org/2026.aacl-demo.2.pdf