Emergent Language: A Survey and Taxonomy

A survey published in Autonomous Agents and Multi-Agent Systems defines emergent language as a research area in artificial intelligence where multi-agent reinforcement learning enables agents to autonomously develop communication forms, potentially in natural language style. The paper reviews scientific publications, analyzes evaluation methods and metrics, and identifies research gaps, serving as a reference for researchers.

Abstract The field of emergent language represents a novel area of research within the domain of artificial intelligence, particularly within the context of multi-agent reinforcement learning. Although the concept of studying language emergence is not new, early approaches were primarily concerned with explaining human language formation, with little consideration given to its potential utility for artificial agents. In contrast, studies based on reinforcement learning aim to develop communicative capabilities in agents that are comparable to or even superior to human language. Thus, they extend beyond the learned statistical representations that are common in natural language processing research. This gives rise to a number of fundamental questions, from the prerequisites for language emergence to the criteria for measuring its success. This paper addresses these questions by providing a comprehensive review of relevant scientific publications on emergent language in artificial intelligence. Its objective is to serve as a reference for researchers interested in or proficient in the field. Consequently, the main contributions are the definition and overview of the prevailing terminology, the analysis of existing evaluation methods and metrics, and the description of the identified research gaps. Similar content being viewed by others 1 Introduction Communication between individual entities is based on conventions and rules that emerge from the necessity or advantage of coordination. Accordingly, Lewis 1 /article/10.1007/s10458-025-09691-y ref-CR1 formalized settings that facilitate the emergence of language as “coordination problems” 1 /article/10.1007/s10458-025-09691-y ref-CR1 and introduced a simple signaling game. This game, in which a speaker describes an object and a listener confronted with multiple options has to identify the indicated one, extensively shaped the field of emergent language EL research in computer science. Early works examined narrowly defined questions regarding the characteristics of emergent communication EC via hand-crafted simulations 2 ref-CR2 , 3 ref-CR3 , 4 ref-CR4 , 5 ref-CR5 , 6 ref-CR6 , 7 ref-CR7 , 8 ref-CR8 , 9 ref-CR9 , 10 ref-CR10 , 11 ref-CR11 , 12 /article/10.1007/s10458-025-09691-y ref-CR12 . These approaches mostly utilized supervised learning methods and non-situated settings, limiting them in their ability to examine the origins and development of complex linguistic features 2 /article/10.1007/s10458-025-09691-y ref-CR2 . However, EL research experienced an upsurge in the period between 2016 and 2018 13 ref-CR13 , 14 ref-CR14 , 15 ref-CR15 , 16 ref-CR16 , 17 ref-CR17 , 18 ref-CR18 , 19 ref-CR19 , 20 /article/10.1007/s10458-025-09691-y ref-CR20 with a focus on MARL approaches 21 ref-CR21 , 22 ref-CR22 , 23 ref-CR23 , 24 ref-CR24 , 25 ref-CR25 , 26 ref-CR26 , 27 ref-CR27 , 28 ref-CR28 , 29 ref-CR29 , 30 ref-CR30 , 31 ref-CR31 , 32 /article/10.1007/s10458-025-09691-y ref-CR32 to enable the examination of more complex features. One fundamental goal of EL research from the multi-agent reinforcement learning MARL perspective is to have agents autonomously develop a communication form that allows not only agent-to-agent but also agent-to-human communication in natural language NL style fashion 2 /article/10.1007/s10458-025-09691-y ref-CR2 , 16 /article/10.1007/s10458-025-09691-y ref-CR16 , 24 /article/10.1007/s10458-025-09691-y ref-CR24 , 29 /article/10.1007/s10458-025-09691-y ref-CR29 , 33 /article/10.1007/s10458-025-09691-y ref-CR33 , 34 /article/10.1007/s10458-025-09691-y ref-CR34 . Therefore, reinforcement learning RL methods are attractive from two points of view. First, successful communication settings might lead to agents that are “more flexible and useful in everyday life” 35 /article/10.1007/s10458-025-09691-y ref-CR35 . Furthermore, they may provide insights into the evolution of NL itself 36 /article/10.1007/s10458-025-09691-y ref-CR36 . However, encouraging communication alone will not automatically produce a language with natural language characteristics 37 /article/10.1007/s10458-025-09691-y ref-CR37 . Providing the right incentives for language development is therefore crucial. EL is the methodological attempt to enable agents to not only statistically understand and use NL, like natural language processing NLP models that learn on text alone 38 /article/10.1007/s10458-025-09691-y ref-CR38 , 39 /article/10.1007/s10458-025-09691-y ref-CR39 , but rather to design, acquire, develop, and learn their own language 40 /article/10.1007/s10458-025-09691-y ref-CR40 , 41 /article/10.1007/s10458-025-09691-y ref-CR41 . The autonomy and independent active experience of RL learning settings is a crucial difference to the data-driven approaches in the field of NLP 42 ref-CR42 , 43 ref-CR43 , 44 /article/10.1007/s10458-025-09691-y ref-CR44 and its large language model LLM . According to Browning and LeCun, “we should not confuse the shallow understanding LLM possess for the deep understanding humans acquire” 41 /article/10.1007/s10458-025-09691-y ref-CR41 through their experiences in life. In EL settings, the agents experience the benefits of communication through goal-oriented tasks 45 /article/10.1007/s10458-025-09691-y ref-CR45 just like it happens naturally 1 /article/10.1007/s10458-025-09691-y ref-CR1 and therefore have the opportunity to develop a deeper understanding of the world 33 /article/10.1007/s10458-025-09691-y ref-CR33 , 46 /article/10.1007/s10458-025-09691-y ref-CR46 . Hence, advances in EL research enable novel applications of multi-agent systems and a considerably advanced form of human-centric AI 35 /article/10.1007/s10458-025-09691-y ref-CR35 . In the current state of EL research, numerous different methods and metrics are already established but they are complex to structure and important issues remain regarding the analysis and comparison of achieved results 29 /article/10.1007/s10458-025-09691-y ref-CR29 , 35 /article/10.1007/s10458-025-09691-y ref-CR35 , 47 /article/10.1007/s10458-025-09691-y ref-CR47 . Therefore, we see a need for a taxonomy to prevent misunderstandings and incorrect use of established metrics. In this paper, we address these issues by providing a comprehensive overview of publications in EL research and by introducing a taxonomy for discrete EL that encompasses key concepts and terminologies of this field. Additionally, we present established and recent metrics for discrete EL categorized according to the taxonomy and discuss their utility. Our goal is to provide a clear and concise description that researchers can use as a shared resource for guidance. Finally, we create a summary of EL research that highlights its achievements and provides an outlook on future research directions. We base our work on a comprehensive and systematic literature search with reproducible search terms on well-known databases. We follow the PRISMA 48 /article/10.1007/s10458-025-09691-y ref-CR48 specifications and show a corresponding flow diagram in Fig. 11 /article/10.1007/s10458-025-09691-y Fig11 in Appendix B /article/10.1007/s10458-025-09691-y Sec48 . The literature search and review process as well as its results are described in detail in Sect. 4 /article/10.1007/s10458-025-09691-y Sec7 . All identified work has been reviewed and categorized according to an extensive list of specific characteristics, e.g. regarding communication setting, game composition, environment configuration, language design, language metrics, and more. Previous surveys of EL in computer science focused only on a subgroup of characteristics or specific parts of this research area. Some of these earlier surveys focus on specific learning settings 45 /article/10.1007/s10458-025-09691-y ref-CR45 , 49 /article/10.1007/s10458-025-09691-y ref-CR49 , 50 /article/10.1007/s10458-025-09691-y ref-CR50 , on methodological summaries and criticism 29 /article/10.1007/s10458-025-09691-y ref-CR29 , 40 /article/10.1007/s10458-025-09691-y ref-CR40 , 51 ref-CR51 , 52 ref-CR52 , 53 ref-CR53 , 54 ref-CR54 , 55 /article/10.1007/s10458-025-09691-y ref-CR55 , or provide a more general overview 24 /article/10.1007/s10458-025-09691-y ref-CR24 , 35 /article/10.1007/s10458-025-09691-y ref-CR35 , 36 /article/10.1007/s10458-025-09691-y ref-CR36 , 47 /article/10.1007/s10458-025-09691-y ref-CR47 , 56 ref-CR56 , 57 ref-CR57 , 58 /article/10.1007/s10458-025-09691-y ref-CR58 . The most similar ones to our work are 35 /article/10.1007/s10458-025-09691-y ref-CR35 and 58 /article/10.1007/s10458-025-09691-y ref-CR58 . Lazaridou 35 /article/10.1007/s10458-025-09691-y ref-CR35 gives an introduction and overview of the EL field before 2021, however, it is mostly a summary of previous work and does not provide a taxonomy or review of existing metrics in the field as we do. Brandizzi 58 /article/10.1007/s10458-025-09691-y ref-CR58 focuses on common characteristics in EC research and the development of emergent human–machine communication strategies. They discuss distinctions and connections of EC research to linguistics, cognitive science, computer science, and sociology, while we focus on emergent language and its analysis. We describe and discuss all relevant surveys in more detail in Sect. 3 /article/10.1007/s10458-025-09691-y Sec6 . Based on this preliminary work, the current state of research on EL misses an overarching review and a comprehensive compilation and alignment of proposed quantification and comparability methods. Accordingly, the key contributions of the present survey are: - A taxonomy of the EL field, in particular regarding the properties of discrete EL, see Sect. 5 /article/10.1007/s10458-025-09691-y Sec8 . - A list of categorized quantification approaches and metrics in a consistent notation, see Sect. 6 /article/10.1007/s10458-025-09691-y Sec20 . - A summary of open questions and an outlook on potential future work, see Sect. 7 /article/10.1007/s10458-025-09691-y Sec42 . In addition, we introduce the fundamental concepts of NL and EC that underlie our survey in Sect. 2 /article/10.1007/s10458-025-09691-y Sec2 . As mentioned, we provide a detailed summary of related surveys in Sect. 3 /article/10.1007/s10458-025-09691-y Sec6 . Section 4 /article/10.1007/s10458-025-09691-y Sec7 describes our study methodology, including the keywords and terms of our systematic literature search. Finally, Sect. 8 /article/10.1007/s10458-025-09691-y Sec45 offers a concluding discussion and final remarks. 2 Background To contextualize the presented taxonomy and analysis, this section summarizes the key concepts of communication and linguistics and provides an overview of EL research. 2.1 Communication Communication at its very basis is the transfer or exchange of signals, which can be interpreted to form some information. These signals include both intended, such as deliberate utterances, and unintended, such as uncontrolled bodily reactions, and include both explicit and implicit parts 59 /article/10.1007/s10458-025-09691-y ref-CR59 . According to Watzlawik’s “Interactional View” 60 /article/10.1007/s10458-025-09691-y ref-CR60 , “one cannot not communicate”. In this regard, communication is ubiquitous and necessary, occurring through various channels and modes 41 /article/10.1007/s10458-025-09691-y ref-CR41 , 61 ref-CR61 , 62 ref-CR62 , 63 ref-CR63 , 64 /article/10.1007/s10458-025-09691-y ref-CR64 . Depending on the specific channel and purpose, communication can be roughly divided into the five forms depicted in Fig. 1 /article/10.1007/s10458-025-09691-y Fig1 . In the context of EL, two of these forms are actively studied, namely interpersonal communication and group communication. Interpersonal communication is communication between entities that mutually influence each other, and its general setting is depicted in Fig. 2 /article/10.1007/s10458-025-09691-y Fig2 . This form of communication is based on individual entities, each within its perceivable environment. Although these environments are agent-specific, they overlap and allow communication through a common channel. In addition, there may be noise in this process that affects the perception of the environment or the communication itself. Group communication, on the other hand, differs only in the number of entities involved and the communication goal. Usually, group communication is more formal and focuses on a common goal or group task while interpersonal communication has a social character and might only relate to a goal or task of one of the participants. Accordingly, the group communication setting can be found in most population-based EL research. Intrapersonal communication e.g., internal vocalization , public communication e.g., lectures , and mass communication e.g., blog entries are not currently examined in the EL literature. Communication has been studied in many different disciplines from many different perspectives, including animals 66 ref-CR66 , 67 ref-CR67 , 68 /article/10.1007/s10458-025-09691-y ref-CR68 , pre-linguistic infants 69 /article/10.1007/s10458-025-09691-y ref-CR69 , 70 /article/10.1007/s10458-025-09691-y ref-CR70 , and sign language 71 /article/10.1007/s10458-025-09691-y ref-CR71 . However, in order to keep the present work concise, we will refer mainly to research in the fields of linguistics and computer science. Generally, communication can be seen as a utility to coordinate with others 72 ref-CR72 , 73 ref-CR73 , 74 ref-CR74 , 75 /article/10.1007/s10458-025-09691-y ref-CR75 . Conversely, the necessity for collaboration within a collective may be a fundamental precursor to the evolution and sustained functionality of explicit communication 4 /article/10.1007/s10458-025-09691-y ref-CR4 , 53 /article/10.1007/s10458-025-09691-y ref-CR53 . This theory leads to an essential differentiation regarding context-dependent communication. Meaningful communication might emerge in a cooperative but not in a fully competitive or manipulative setting. However, a partially competitive setting might be vital for the emergence of resilient and comprehensive communication, e.g. to enable the detection and use of lies 34 /article/10.1007/s10458-025-09691-y ref-CR34 . Accordingly, the level of cooperation is a defining element of the communication setting in EL research. NL is a tool that allows us to encode very complex information within a discrete and humanly manageable amount of utterances. A lot of artificial intelligence research aims to develop NL models, with applications ranging from translation to coherent full-text generation based on single-word input 44 /article/10.1007/s10458-025-09691-y ref-CR44 , 77 ref-CR77 , 78 ref-CR78 , 79 /article/10.1007/s10458-025-09691-y ref-CR79 . However, current research is mostly based on LLM which “achieve a sophisticated level of inductive learning and inference” 80 /article/10.1007/s10458-025-09691-y ref-CR80 but are also “far from human abilities in natural language inference, analogical reasoning, and interpretation” 80 /article/10.1007/s10458-025-09691-y ref-CR80 . A lot of research from the EL community is based on the theory, that models which learn language statistically based on static datasets are limited in their communicative and cognitive abilities 18 /article/10.1007/s10458-025-09691-y ref-CR18 , 41 /article/10.1007/s10458-025-09691-y ref-CR41 , 72 /article/10.1007/s10458-025-09691-y ref-CR72 , 81 /article/10.1007/s10458-025-09691-y ref-CR81 , 82 /article/10.1007/s10458-025-09691-y ref-CR82 . Recent publications have shown that LLM have “weak reasoning and decision-making abilities” 83 /article/10.1007/s10458-025-09691-y ref-CR83 , their “reasoning is fragile” 84 /article/10.1007/s10458-025-09691-y ref-CR84 , and that current LLM face “reliability issues” 85 /article/10.1007/s10458-025-09691-y ref-CR85 . Correspondingly, the field of EL research in AI aims to enable agents to utilize intended communication in the same way humans use it to increase cooperation, performance, and generalization and, in the long run, enable direct meaningful communication between humans and artificial systems 18 /article/10.1007/s10458-025-09691-y ref-CR18 , 34 /article/10.1007/s10458-025-09691-y ref-CR34 . In line with this, multiple explicit forms of EC in artificial intelligence research have been investigated as shown in Sect. 5.4.2 /article/10.1007/s10458-025-09691-y Sec14 . In contrast, work focusing on implicit communication, like the information content of spatial positioning of agents in a multi-agent setting 86 /article/10.1007/s10458-025-09691-y ref-CR86 , is not part of the present survey. 2.2 Natural language NL is a prime example of a versatile and comprehensive form of communication designed to convey meaning 87 /article/10.1007/s10458-025-09691-y ref-CR87 . The flexibility of NL allows humans to be exact but also deliberately ambiguous in their communication 88 /article/10.1007/s10458-025-09691-y ref-CR88 . It is a vital feature that distinguishes us from other species and gives us a great advantage in terms of knowledge storage, sharing, and acquisition 88 /article/10.1007/s10458-025-09691-y ref-CR88 . However, the origin and evolution of language is still a mystery 89 /article/10.1007/s10458-025-09691-y ref-CR89 , 90 /article/10.1007/s10458-025-09691-y ref-CR90 . In the field of linguistics, many conflicting theories have been introduced so far 90 ref-CR90 , 91 ref-CR91 , 92 ref-CR92 , 93 ref-CR93 , 94 /article/10.1007/s10458-025-09691-y ref-CR94 , ranging from behavioral to biological explanations. Additionally, accompanying research in the field of computer science has a long history 31 /article/10.1007/s10458-025-09691-y ref-CR31 with a comparable range of theories. Even though there is still a debate around this topic, it is commonly agreed upon that a very intricate evolutionary process was involved 88 /article/10.1007/s10458-025-09691-y ref-CR88 , 90 /article/10.1007/s10458-025-09691-y ref-CR90 . This evolution most likely took place in two different areas simultaneously, biologically and linguistically. On the biological side, the human brain most likely developed specific areas and functionalities specifically for more complex language-based communication, that are studied in the scientific field of neurolinguistics 95 /article/10.1007/s10458-025-09691-y ref-CR95 . On the linguistics side, this evolution can be seen in language development itself, which is a constantly ongoing process 88 /article/10.1007/s10458-025-09691-y ref-CR88 that might be strongly connected to the development of cognitive skills 96 /article/10.1007/s10458-025-09691-y ref-CR96 and the social environment 97 /article/10.1007/s10458-025-09691-y ref-CR97 . Similarly, EL is concerned with the research of suitable model structures for the processing of language, while concurrently developing and evaluating language. While the exact origin of language is highly debatable, the actual communication process via NL is generally easier to conceptualize. For example, it can be modeled by the semiotic cycle depicted in Fig. 3 /article/10.1007/s10458-025-09691-y Fig3 45 /article/10.1007/s10458-025-09691-y ref-CR45 , 98 /article/10.1007/s10458-025-09691-y ref-CR98 . This depiction applies to multiple expressive channels, e.g. speech and writing. It assumes at least two involved parties, a speaker and a listener. The speaker produces an utterance based on the meaning to be conveyed. This meaning results from the combined conceptualization of the speaker’s goal and model of the world. On the other hand, the listener receives the utterance and comprehends it to derive a meaning, which is not a direct copy of the initial one by the speaker but it still refers to the shared world. The interpretation of the meaning, which the listener’s world model informs, leads to some action by the listener. At the center of this process are the shared world and the respective world models of speaker and listener that function as grounding for the information exchange via language. Further, both linguistic level components of production and comprehension allow the respective agent to participate in the language process. The semiotic cycle puts the utterance as an externalized information carrier into focus. While the other components are internalized and thus difficult to define and measure, the utterance itself is external and available for analysis. Fundamentally, this specific utterance is based on the underlying communication process and specifically, the language used. Accordingly, most research papers investigate characteristics of the utilized language to analyze the communication possibilities and capabilities of users. To this end, linguistics subdivides the language structure into six major levels 99 ref-CR99 , 100 ref-CR100 , 101 /article/10.1007/s10458-025-09691-y ref-CR101 , as illustrated in Fig. 4 /article/10.1007/s10458-025-09691-y Fig4 . This structure was originally developed for spoken language, as indicated by the terms ‘phonetics’ and ‘phonology’ derived from the Greek word ‘phon’ meaning ‘sound’. However, the levels are also applicable to written language in the context of EL. Therefore, the following description will address both spoken and written language within this framework. The phonetics level includes the entire bandwidth of the chosen, often continuous, language channel. For example, it comprises the full range of possible speech sounds available to humans. Consequently, it is fundamental for the general transfer range and describes it without any limitation. At the phonology level are the atomic building blocks of the spoken or written language, defined as phonemes or graphemes. A phoneme or grapheme enables the creation of meaning as well as the necessary distinction at the lowest level of language. However, in a NL with an alphabetic writing system, phonemes and graphemes, which in this case correspond to letters, are often not a direct match and are only roughly related. Nevertheless, these individual units comprise the set of used elements from the continuous channel range for a specific language. These are used and combined at the morphology level to create and assign meaning by making words, in linguistics called lexemes. In this context, word-forming rules and underlying structures are of interest. Utilizing these meaningful building blocks, sentences can be realized at the syntax level. This level only concerns the structure of sentences and in particular, their assembly rules and the word categories that are used. The meaning of these sentences is relevant at the next level, semantics . At this level, the literal meaning of language constructions is of interest while the final level, pragmatics , focuses on how context contributes to the meaning. Accordingly, it analyzes how language is used in interactions and the relationship between the involved parties. Overall, the presented levels are not only important to describe language functionally and structurally but also to distinguish language characteristics and metrics. Thus, we use them to organize parts of the taxonomy in Sect. 5 /article/10.1007/s10458-025-09691-y Sec8 and the metrics in Sect. 6 /article/10.1007/s10458-025-09691-y Sec20 . 2.3 Emergent language EL refers to a form of communication that develops among artificial agents through interaction, without being explicitly pre-programmed. Thus, it is a bottom-up approach, arising from the agents’ need to cooperate and solve tasks within a given environment 58 /article/10.1007/s10458-025-09691-y ref-CR58 . This process involves the agents creating, adapting, and refining linguistic structures and meanings to enhance their ability to exchange information effectively and efficiently 49 /article/10.1007/s10458-025-09691-y ref-CR49 . EL research aims to understand the principles and mechanisms underlying this spontaneous development of communication. It explores how linguistic elements such as syntax 102 /article/10.1007/s10458-025-09691-y ref-CR102 , semantics 103 /article/10.1007/s10458-025-09691-y ref-CR103 , 104 /article/10.1007/s10458-025-09691-y ref-CR104 , and pragmatics 29 /article/10.1007/s10458-025-09691-y ref-CR29 can arise from the interaction of artificial agents and how these elements contribute to the agents’ performance and cooperation. A NL-like communication form would make artificial agents and computer systems, in general, more accessible, simpler to comprehend, and altogether more powerful 24 /article/10.1007/s10458-025-09691-y ref-CR24 , 34 /article/10.1007/s10458-025-09691-y ref-CR34 , 35 /article/10.1007/s10458-025-09691-y ref-CR35 . EL research originally focused on the question of language origin 3 /article/10.1007/s10458-025-09691-y ref-CR3 . Recently, this focus shifted to the more functional aspect of EL, focusing on how to enable agent systems to benefit from a mechanism that helped humanity thrive and how to achieve communication capabilities as close as possible to NL 35 /article/10.1007/s10458-025-09691-y ref-CR35 . Today, EL within computer science is about self-learned 4 /article/10.1007/s10458-025-09691-y ref-CR4 , reusable 46 /article/10.1007/s10458-025-09691-y ref-CR46 , teachable 105 /article/10.1007/s10458-025-09691-y ref-CR105 , 106 /article/10.1007/s10458-025-09691-y ref-CR106 , interpretable 14 /article/10.1007/s10458-025-09691-y ref-CR14 , and powerful 18 /article/10.1007/s10458-025-09691-y ref-CR18 communication protocols. In the long run, EL aims to enable machines to communicate with each other and with humans in a more seamless and extendable manner 35 /article/10.1007/s10458-025-09691-y ref-CR35 , 107 /article/10.1007/s10458-025-09691-y ref-CR107 . Accordingly, various research questions and areas were derived. For example, recent papers have addressed issues around the nature of the setting, which can be semi-cooperative 34 /article/10.1007/s10458-025-09691-y ref-CR34 , 108 /article/10.1007/s10458-025-09691-y ref-CR108 , include adversaries 22 /article/10.1007/s10458-025-09691-y ref-CR22 , 32 /article/10.1007/s10458-025-09691-y ref-CR32 , have message-influencing noise 109 /article/10.1007/s10458-025-09691-y ref-CR109 , or incorporate social structures 110 /article/10.1007/s10458-025-09691-y ref-CR110 , 111 /article/10.1007/s10458-025-09691-y ref-CR111 . Moreover, some are concerned with the challenge of grounding EL, e.g. using representation learning as basis 112 /article/10.1007/s10458-025-09691-y ref-CR112 , combining supervised learning and self-play 113 /article/10.1007/s10458-025-09691-y ref-CR113 , or utilizing EL agents as the basis for NL finetuning approaches 107 /article/10.1007/s10458-025-09691-y ref-CR107 . Others tackle the direct emergence of language with NL characteristics, e.g. looking at internal and external pressures 114 ref-CR114 , 115 ref-CR115 , 116 ref-CR116 , 117 ref-CR117 , 118 /article/10.1007/s10458-025-09691-y ref-CR118 , evaluating factors to enforce semantic conveyance 53 /article/10.1007/s10458-025-09691-y ref-CR53 , looking at compositionality 119 /article/10.1007/s10458-025-09691-y ref-CR119 , generalization 25 /article/10.1007/s10458-025-09691-y ref-CR25 , or expressivity 120 /article/10.1007/s10458-025-09691-y ref-CR120 , or questioning the importance of characteristics like compositionality 121 /article/10.1007/s10458-025-09691-y ref-CR121 and the connection between compositionality and generalization 122 /article/10.1007/s10458-025-09691-y ref-CR122 . Based on these examples and the introduced goals and approaches, the difference in comparison to NLP research becomes apparent. Current approaches in NLP, namely LLM, learn language imitation via statistics, but they might not capture the functional aspects and the purpose of communication itself 18 /article/10.1007/s10458-025-09691-y ref-CR18 , 41 /article/10.1007/s10458-025-09691-y ref-CR41 . In contrast, EL uses language not as the sole objective but as a means to achieve something with meaning 23 /article/10.1007/s10458-025-09691-y ref-CR23 . Accordingly, agents have to learn their own EL to enable functionality beyond simple statistical reproduction. Specifically, agents should learn communication by necessity or benefits 114 /article/10.1007/s10458-025-09691-y ref-CR114 and they need a setting that rewards or encourages communication, e.g., an at least partially cooperative setting 34 /article/10.1007/s10458-025-09691-y ref-CR34 . While the EL concept sounds simple, it comes with many challenges. Encouraging communication alone can lead to simple gibberish that helps with task completion but does not represent the intended natural language characteristics 37 /article/10.1007/s10458-025-09691-y ref-CR37 . Providing the right incentives for language development is therefore crucial. In addition, it is important to examine how agents use communication and the opportunity to send and receive information, raising the question of how to measure successful communication 29 /article/10.1007/s10458-025-09691-y ref-CR29 . The measurability of language properties such as syntax, semantics, and pragmatics is also important for assessing the emergence of desirable language properties 122 /article/10.1007/s10458-025-09691-y ref-CR122 . The following sections explore these challenges and related constructs and approaches in detail. 3 Related surveys As briefly mentioned in Sect. 1 /article/10.1007/s10458-025-09691-y Sec1 , our literature review identified 19 publications that we classified as surveys. We adopted a broad definition of what constitutes a survey, categorizing any publication as a survey if it either explicitly described itself as such or provided a particularly comprehensive and structured review of previous research. These publications conduct similar investigations on EL research but with different scopes. We focus on discrete language emergence, associated taxonomy, characteristics, metrics, and research gaps. In contrast, in our review of the existing survey work, three distinct interpretive directions emerge, which we categorize as summarized in Table 1 /article/10.1007/s10458-025-09691-y Tab1 : Surveys that focus on the learning settings 45 /article/10.1007/s10458-025-09691-y ref-CR45 , 49 /article/10.1007/s10458-025-09691-y ref-CR49 , 50 /article/10.1007/s10458-025-09691-y ref-CR50 , surveys that summarize and review utilized methods 29 /article/10.1007/s10458-025-09691-y ref-CR29 , 40 /article/10.1007/s10458-025-09691-y ref-CR40 , 51 ref-CR51 , 52 ref-CR52 , 53 ref-CR53 , 54 ref-CR54 , 55 /article/10.1007/s10458-025-09691-y ref-CR55 , and surveys that provide a general discussion or overview of the EL field 24 /article/10.1007/s10458-025-09691-y ref-CR24 , 35 /article/10.1007/s10458-025-09691-y ref-CR35 , 36 /article/10.1007/s10458-025-09691-y ref-CR36 , 47 /article/10.1007/s10458-025-09691-y ref-CR47 , 56 ref-CR56 , 57 ref-CR57 , 58 /article/10.1007/s10458-025-09691-y ref-CR58 . The following section briefly summarizes these surveys within these categories. 3.1 Settings Surveys within the settings category primarily focus on the design of language learning environments and the general structure of learning problems. For instance, van Eecke and Beuls 45 /article/10.1007/s10458-025-09691-y ref-CR45 explored the language game paradigm, categorizing experiments and identifying properties critical for MARL research, such as symmetric agent roles and autonomous behavior. While their work offers a foundational perspective, our survey extends beyond the language game paradigm to analyze a broader range of approaches in greater depth see Sect. 7 /article/10.1007/s10458-025-09691-y Sec42 . Similarly, Lipowska and Lipowski 49 /article/10.1007/s10458-025-09691-y ref-CR49 emphasized sociocultural aspects, such as migration and teachability, within simple naming games. While these are part of our analysis, our review situates them within a unified framework, providing a more comprehensive perspective. Denamganaï and Walker 50 /article/10.1007/s10458-025-09691-y ref-CR50 introduced ReferentialGym as a tool for studying referential games and their associated metrics, like positive signaling and positive listening 29 /article/10.1007/s10458-025-09691-y ref-CR29 . In contrast, our survey goes beyond referential games, offering a broader exploration of EL metrics and their applications across diverse frameworks. 3.2 Methods The methods category encompasses surveys that primarily examine learning and evaluation methodologies in EL, each offering unique perspectives on key challenges. A recurring theme in this category is the need for more comprehensive evaluation metrics that capture the complexity of emergent communication 51 ref-CR51 , 52 ref-CR52 , 53 /article/10.1007/s10458-025-09691-y ref-CR53 . For example, Korbak et al. 51 /article/10.1007/s10458-025-09691-y ref-CR51 highlighted the limitations of existing compositionality metrics, introducing the tree reconstruction error to address semantic compositionality, a challenge we contextualize further in Section 5.4.5 /article/10.1007/s10458-025-09691-y Sec17 . In contrast, LaCroix 52 /article/10.1007/s10458-025-09691-y ref-CR52 critiqued the overemphasis on compositionality, advocating for a shift towards reflexivity, though metrics for this remain unexplored. This gap underscores the fragmented nature of current evaluation practices. Grounding and utility also feature prominently in this category. Lemon 40 /article/10.1007/s10458-025-09691-y ref-CR40 emphasized the interplay of symbolic and conversational grounding, highlighting data-related challenges, while Lowe et al. 29 /article/10.1007/s10458-025-09691-y ref-CR29 proposed pragmatic metrics such as positive signaling and positive listening, which inspired our taxonomy of language utility in Sect. 5.4.6 /article/10.1007/s10458-025-09691-y Sec18 . These works collectively underscore the necessity of balancing semantic depth with practical utility, a balance our survey seeks to achieve by integrating diverse perspectives into a unified framework. Further, recent studies like those by Galke and Raviv 54 /article/10.1007/s10458-025-09691-y ref-CR54 explored the role of linguistic pressures and biases in bridging the gap between EL and human NL, providing insights into the origins of NL phenomena in EL. Vanneste et al. 55 /article/10.1007/s10458-025-09691-y ref-CR55 tackled discretization methods critical for EL learning, offering a comparative analysis that complements our work. 3.3 General The general category includes surveys that provide broad overviews or address themes not confined to specific settings or methods . Key contributions in this category highlight the interdisciplinary perspectives, interaction paradigms, and structural dimensions of emergent communication research. Hernandez-Leal et al. 47 /article/10.1007/s10458-025-09691-y ref-CR47 offered a foundational overview of multi-agent deep reinforcement learning MARL , including emergent behavior and communication. While their survey provides valuable historical context and practical challenges, our work builds upon this by focusing specifically on emergent language EL within MARL, analyzing it with finer granularity and from a metrics-driven perspective. Similarly, Brandizzi and Iocchi 24 /article/10.1007/s10458-025-09691-y ref-CR24 emphasized the underrepresentation of human-in-the-loop paradigms, proposing novel interaction settings but lacking the systematic categorization and metrics-oriented discussion presented in our survey. Moulin-Frier and Oudeyer 56 /article/10.1007/s10458-025-09691-y ref-CR56 , Fernando et al. 57 /article/10.1007/s10458-025-09691-y ref-CR57 , and Galke et al. 36 /article/10.1007/s10458-025-09691-y ref-CR36 explored interdisciplinary connections and cognitive constraints in EL, with the latter focusing on the perceived gaps between EL and NL. While these works underscore key challenges, our survey contextualizes such limitations across a broader set of metrics and emergent properties, providing actionable insights for bridging these gaps. Other surveys, such as those by Suglia et al. 123 /article/10.1007/s10458-025-09691-y ref-CR123 and Zhu et al. 124 /article/10.1007/s10458-025-09691-y ref-CR124 , structured EL research into multimodal and dimensional frameworks, respectively. These works serve as useful complements to our survey, which introduces an extensive taxonomy Sect. 5 /article/10.1007/s10458-025-09691-y Sec8 that synthesizes and organizes findings from diverse sources. Similarly, Lazaridou and Baroni 35 /article/10.1007/s10458-025-09691-y ref-CR35 and Brandizzi 58 /article/10.1007/s10458-025-09691-y ref-CR58 provided comprehensive overviews of the field but lacked the detailed quantification and taxonomy of metrics that form the core of our work. By synthesizing these contributions, our survey is distinguished by its focus on emergent language metrics and quantification, complemented by a systematic taxonomy to address fragmentation in the field. This integrated approach provides a structured roadmap for advancing EL research, with an emphasis on both practical measurability and interdisciplinary relevance. 4 Study methodology The literature search that resulted in the body of work surveyed in this paper was conducted on the 17th of June 2024. The used libraries and databases are: ScienceDirect https://www.sciencedirect.com/ , IEEE Xplore https://ieeexplore.ieee.org/ , ACM Digital Library https://dl.acm.org/ , WebOfScience https://www.webofscience.com/ , arXiv https://arxiv.org/ , and SemanticScholar https://www.semanticscholar.org/ . SemanticScholar https://www.semanticscholar.org/ is a special case, due to the nature of the provided search machine that does not allow complex queries and filtering like the others. Consequently, we hand-picked suitable papers from the first 50 entries of the search result list. A PRISMA 48 /article/10.1007/s10458-025-09691-y ref-CR48 flow diagram of the publication selection process is provided in Fig. 11 /article/10.1007/s10458-025-09691-y Fig11 in Appendix B /article/10.1007/s10458-025-09691-y Sec48 . Additionally, the individual queries and results of all services are summarized in Table 2 /article/10.1007/s10458-025-09691-y Tab2 . The queries delivered 613 hits in total which resulted in 516 unique papers. A first quick read of these papers led to 23 additional papers, referenced by some of the originally found work. Accordingly, the literature review started with a corpus consisting of 539 individual papers. Of the 539 papers, 327 were sorted out due to the substantial divergence from the searched topic, often focusing on domains like 5G, networking, and radio. Of the remaining 212 papers, 181 directly address the field of interest, while 31 are only partially relevant. Papers were deemed partially relevant if they mentioned the surveyed topic but primarily focused on different areas such as datasets, language theory, simulation, or unrelated case studies. In conclusion, this survey mainly reviews 181 papers that directly discuss or contribute to the topic of EL in computer science. Figure 5 /article/10.1007/s10458-025-09691-y Fig5 presents the distribution of the 181 relevant publications over the years, categorized by publication type. The topic of EL has maintained a steady presence in conference publications, peaking in 2020. The subsequent decline in total publications may be attributed to the absence of recent topic-specific workshops. Additionally, the surge in interest in LLM technologies might have diverted attention from EL research. It is also worth noting that some recent studies may not have been openly published at the time of our literature search. We therefore expect the publication count to increase by 2024. 5 Taxonomy of emergent language In the course of our comprehensive literature review, we identified recurrent instances of taxonomic inconsistencies due to missing standardization 125 /article/10.1007/s10458-025-09691-y ref-CR125 and “ill-adapted metrics” 25 /article/10.1007/s10458-025-09691-y ref-CR25 . Particular concern arises from the discrepancy between the concepts intended for measurement and their corresponding metrics, or the absence of such metrics 46 /article/10.1007/s10458-025-09691-y ref-CR46 , 51 /article/10.1007/s10458-025-09691-y ref-CR51 , 126 ref-CR126 , 127 ref-CR127 , 128 /article/10.1007/s10458-025-09691-y ref-CR128 . This section is dedicated to the formulation of a systematic taxonomy aimed at enhancing comparability and mitigating confusion within the field. This taxonomy forms the basis for the following sections and is designed to ensure consistent representation throughout the survey. It is created with the hope that it will serve as a cornerstone for future research, promoting the use of standardized terminology, particularly in the domain of language characteristics. The taxonomy first describes the main factors influencing the EL, before categorizing the language characteristics. These influencing factors have a significant impact on the investigative possibilities of EL research and are therefore of particular importance when analyzing EL. Thus, the taxonomy introduces a classification system for the communication setting Sect. 5.1 /article/10.1007/s10458-025-09691-y Sec9 and communication games Sect. 5.2 /article/10.1007/s10458-025-09691-y Sec10 that agents encounter during language emergence. The communication setting encompasses factors such as the number of agents and the type of communication available to them. The communication game involves the environmental configuration and crucial factors influencing challenges and the complexity of multi-task learning. Furthermore, a short discussion on the concept of language priors is provided in Sect. 5.3 /article/10.1007/s10458-025-09691-y Sec11 , considering that the presence of a prior significantly influences the characteristics of the emerging language 14 /article/10.1007/s10458-025-09691-y ref-CR14 , 113 /article/10.1007/s10458-025-09691-y ref-CR113 . We conclude this section with a comprehensive overview of the concepts and characteristics examined within EL research Sect. 5.4 /article/10.1007/s10458-025-09691-y Sec12 . The taxonomy adheres to the six major linguistic structural levels introduced in Sect. 2.2 /article/10.1007/s10458-025-09691-y Sec4 and illustrated in Fig. 4 /article/10.1007/s10458-025-09691-y Fig4 . 5.1 Comunication setting In the literature, several communication settings are represented. One distinguishing factor is the number of agents involved. We derived three classes—the single agent, dual agent, and population setting. While the single agent setting is rare, the other two are well represented in the examined literature, as shown in Table 3 /article/10.1007/s10458-025-09691-y Tab3 . A single agent is typically used to train human–machine interfaces 129 /article/10.1007/s10458-025-09691-y ref-CR129 or fine-tune existing models 38 /article/10.1007/s10458-025-09691-y ref-CR38 . In contrast, dual-agent settings are more common and often involve a pair of speaker-listener agents, with one agent designated as the speaker and the other as the listener exclusively 14 /article/10.1007/s10458-025-09691-y ref-CR14 . The population setting involves larger groups of agents in the language emergence process. This requires more computational resources but also enables more possibilities for regularization 21 /article/10.1007/s10458-025-09691-y ref-CR21 and language evolution 117 /article/10.1007/s10458-025-09691-y ref-CR117 . Accordingly, the population setting offers more opportunities to actively shape the process 21 /article/10.1007/s10458-025-09691-y ref-CR21 , 45 /article/10.1007/s10458-025-09691-y ref-CR45 , 130 /article/10.1007/s10458-025-09691-y ref-CR130 . An additional factor that shapes the communication setting is the type of cooperation inherent in the setup. Determining the level of cooperation or competition feasible within the setting is a fundamental decision and closely related to the choice of the language game. We derived three options—the cooperative, semi-cooperative, and competitive type. In the literature reviewed, the majority of studies adopted a fully cooperative setting approach, where agents fully share their rewards and lack individual components. The emphasis on strongly cooperative settings is justified given that AI agents utilize a common language to coordinate and will not learn to communicate if they dominate without communication 34 /article/10.1007/s10458-025-09691-y ref-CR34 . Only a few publications explore semi-cooperative settings that incorporate individual rewards alongside shared rewards, introducing the challenge of balancing tasks and rewards 29 /article/10.1007/s10458-025-09691-y ref-CR29 , 108 /article/10.1007/s10458-025-09691-y ref-CR108 . A semi-cooperative setup can be compared to a simplified social scenario with overarching societal objectives, while also encompassing additional individual interests and goals. In contrast, investigations of fully competitive settings are rare, with only one work in which agents compete for rewards without a common goal 34 /article/10.1007/s10458-025-09691-y ref-CR34 . This scarcity likely arises from the fact that such settings inherently favor deceptive language as the only advantageous strategy, making its emergence improbable without any cooperative element 34 /article/10.1007/s10458-025-09691-y ref-CR34 . The third important factor in communications settings is symmetry. Agents should treat messages similarly to regular observations; otherwise, they risk devolving into mere directives 146 /article/10.1007/s10458-025-09691-y ref-CR146 . Building on this premise, the symmetry is important for promoting robust language emergence, as opposed to languages that consist primarily of directives. An illustrative example of asymmetric settings is the commonly used, and aforementioned, speaker-listener paradigm 14 /article/10.1007/s10458-025-09691-y ref-CR14 , 51 /article/10.1007/s10458-025-09691-y ref-CR51 , 168 /article/10.1007/s10458-025-09691-y ref-CR168 . Languages developed in such settings are severely limited compared to NL, lacking the capacity for diverse discourse or even basic information exchange beyond directives 146 /article/10.1007/s10458-025-09691-y ref-CR146 . Contrary to promoting informed choices by the listener, the speaker-listener approach emphasizes obedience to commands. Conversely, a symmetric setting facilitates bi-directional communication, thereby allowing for more comprehensive language development 110 /article/10.1007/s10458-025-09691-y ref-CR110 , 202 /article/10.1007/s10458-025-09691-y ref-CR202 . For instance, symmetry may result from agents being randomly assigned roles within the interaction 202 /article/10.1007/s10458-025-09691-y ref-CR202 . Additionally, symmetry can emerge from tasks that are inherently balanced, such as negotiations between equal partners where both parties have equivalent roles and objectives 17 /article/10.1007/s10458-025-09691-y ref-CR17 . At the population level, another important consideration is the choice of recipients, i.e., between targeted and broadcast communication. While broadcast communication facilitates broader information dissemination across the agent group, targeted communication promotes the development of social group dynamics and regularization 222 /article/10.1007/s10458-025-09691-y ref-CR222 , 226 /article/10.1007/s10458-025-09691-y ref-CR226 . For example, targeted communication strategies can be learned through mechanisms such as attention 222 /article/10.1007/s10458-025-09691-y ref-CR222 , and agents can develop minimized communication strategies that optimize group performance 229 /article/10.1007/s10458-025-09691-y ref-CR229 . Table 3 /article/10.1007/s10458-025-09691-y Tab3 provides a summary of these settings and their variations. The setting categories presented and their implementation are not inherently tied to the language itself but are crucial in determining the likelihood of meaningful language emergence and in shaping the features and experimental possibilities. These initial choices dictate the options for the language development process, the opportunities for regularization 21 /article/10.1007/s10458-025-09691-y ref-CR21 , 130 /article/10.1007/s10458-025-09691-y ref-CR130 , and the requirements regarding computational resources. 5.2 Language games Distinct communication settings are implemented through different communication games. In this section, we provide an overview of the games used in EL literature. Specifically, we focus on a subset of these games known as language games, that emphasize explicit communication via a predefined language channel. The literature identifies several categories of language games, such as referential games, reconstruction games, question-answer games, grid-world games, among others. Our review indicates that these categories represent the most commonly used game types. To give a comprehensive view, Table 4 /article/10.1007/s10458-025-09691-y Tab4 lists the publications that focus on these game types. In the following, we offer a concise overview of each category to provide a clearer understanding of their characteristics. Referential game Generally, a referential game, also called signaling game, consists of two agents, a sender and a receiver 14 /article/10.1007/s10458-025-09691-y ref-CR14 . The objective of this game is for the receiver to correctly identify a particular sample from a set, which may include distractors, solely based on the message received from the sender. This set can consist of images 14 /article/10.1007/s10458-025-09691-y ref-CR14 , 16 /article/10.1007/s10458-025-09691-y ref-CR16 , 149 /article/10.1007/s10458-025-09691-y ref-CR149 , object feature vectors 140 /article/10.1007/s10458-025-09691-y ref-CR140 , texts 140 /article/10.1007/s10458-025-09691-y ref-CR140 , or even graphs 186 /article/10.1007/s10458-025-09691-y ref-CR186 . To accomplish this selection task, the sender must first encode a message that contains information about the correct sample. In game design, a fundamental decision arises regarding whether the sender should only view the correct sample or also some distractors that may differ from those presented to the receiver 14 /article/10.1007/s10458-025-09691-y ref-CR14 . Another design decision concerns the receiver’s side, specifically the number of distractors and whether to provide the original sample shown to the sender or only a similar one for selection 160 /article/10.1007/s10458-025-09691-y ref-CR160 . However, only the encoded message is transmitted to the receiver, who then selects an item from their given collection. Reconstruction game The reconstruction game is similar to the referential game, but with a key difference: the receiver does not have a collection to choose from. Instead, the receiver must construct a sample based on the message from the sender, aiming to replicate the original sample shown to the sender as closely as possible 118 /article/10.1007/s10458-025-09691-y ref-CR118 , 122 /article/10.1007/s10458-025-09691-y ref-CR122 . Consequently, this game setup resembles an autoencoder-based approach, with a latent space tailored to mimic or facilitate language 162 /article/10.1007/s10458-025-09691-y ref-CR162 . Therefore, the key distinction between reconstruction and referential games, often used interchangeably in early literature, lies in the collection’s presence referential or absence reconstruction for the receiver to select from 155 /article/10.1007/s10458-025-09691-y ref-CR155 . Question-answer game The question-answer game is a variant of the referential game, but without strict adherence to previously established rules. It operates as a multi-round referential game, allowing for iterative and bilateral communication 202 /article/10.1007/s10458-025-09691-y ref-CR202 . Unlike referential and reconstruction games, the question-answer game explicitly incorporates provisions for multiple rounds with follow-up or clarifying queries from the receiver 19 /article/10.1007/s10458-025-09691-y ref-CR19 , 21 /article/10.1007/s10458-025-09691-y ref-CR21 . Question-answer games have introduced intriguing inquiries and avenues for exploring the symmetry of EL, although they are not as widely adopted 19 /article/10.1007/s10458-025-09691-y ref-CR19 . Grid world game Grid world games use a simplified 2D environment to model various scenarios like warehouse path planning 22 /article/10.1007/s10458-025-09691-y ref-CR22 , movement of objects 205 /article/10.1007/s10458-025-09691-y ref-CR205 , traffic junctions 226 /article/10.1007/s10458-025-09691-y ref-CR226 , 231 /article/10.1007/s10458-025-09691-y ref-CR231 , or mazes 20 /article/10.1007/s10458-025-09691-y ref-CR20 , 115 /article/10.1007/s10458-025-09691-y ref-CR115 . They offer design flexibility, allowing agents to be part of the environment or act as external supervisors. Design choices also include environment complexity and the extent of agents’ observations. Although common in the literature surveyed, implementations of grid world games vary widely in their design choices and are thus a very heterogeneous group. Continuous world game Continuous environments add complexity to the learning process 229 /article/10.1007/s10458-025-09691-y ref-CR229 , 236 /article/10.1007/s10458-025-09691-y ref-CR236 . In EL approaches, the learning landscape involves multi-task settings where one task is tackled directly within the environment while another involves language formation. Playing continuous world games, whether in two or three dimensions, presents challenges and adds a greater sense of realism and intricacy. These environments have the potential to make it more feasible to deploy EL agents in real-world scenarios compared to discrete environments 207 /article/10.1007/s10458-025-09691-y ref-CR207 . Other The literature on EL also covers various other game types besides those mentioned earlier, such as matrix communication games 29 /article/10.1007/s10458-025-09691-y ref-CR29 , 30 /article/10.1007/s10458-025-09691-y ref-CR30 , social deduction games 23 /article/10.1007/s10458-025-09691-y ref-CR23 , 235 /article/10.1007/s10458-025-09691-y ref-CR235 , or lever games 20 /article/10.1007/s10458-025-09691-y ref-CR20 , 234 /article/10.1007/s10458-025-09691-y ref-CR234 . These game types contribute to the creation of new language emergence settings, often designed to target specific aspects or characteristics of language development. They are valuable tools to explore and understand the complexities of EL in different contexts. In summary, although many language games have been developed, comparing different games can be complex and understanding the nuances of each game can prove challenging. A promising direction would be for the research community to collectively agree on a standardized subset of these games as benchmarks. By focusing on a representative set of games from different categories, researchers could systematically explore different settings, ensuring that new approaches are rigorously tested and their results are directly comparable across studies. This would accelerate the maturation of the field of EL research, foster collaboration, and enable the community to better identify and address key challenges. 5.3 Language prior EL research occasionally utilizes a concept known as a language prior to incorporate structures from human NLs into the emerging language. A language prior is used to impose specific linguistic structures on the emerging language, making it easier to align with human NL and improve interpretability and performance. This prior can be implemented through supervised learning 14 /article/10.1007/s10458-025-09691-y ref-CR14 , 19 /article/10.1007/s10458-025-09691-y ref-CR19 , 113 /article/10.1007/s10458-025-09691-y ref-CR113 , also known as injection, or through divergence estimation 15 /article/10.1007/s10458-025-09691-y ref-CR15 . An overview of prior usage in the literature surveyed is given in Table 11 /article/10.1007/s10458-025-09691-y Tab11 in Appendix A /article/10.1007/s10458-025-09691-y Sec47 . Given this context, research on EL can be divided into two main areas. The first area focuses on independent situated learning and does not use priors, so that communication and language emerge spontaneously 18 /article/10.1007/s10458-025-09691-y ref-CR18 . The second area explores imitation learning-based approaches, which aim to replicate NL behavior in artificial agents using priors 103 /article/10.1007/s10458-025-09691-y ref-CR103 . However, it is important to note that these approaches differ from LLM because language acquisition in EL is generally task-oriented. In academic literature, the independent situated learning environment is often referred to as the evolution-based approach, while the imitation learning-related approach is commonly known as the acquisition-based approach. The term evolution implies starting from scratch, while acquisition involves learning an existing language 168 /article/10.1007/s10458-025-09691-y ref-CR168 . The terminology and different approaches are depicted in Fig. 6 /article/10.1007/s10458-025-09691-y Fig6 . In addition, the concepts of community and generational learning are closely related 21 /article/10.1007/s10458-025-09691-y ref-CR21 , 116 /article/10.1007/s10458-025-09691-y ref-CR116 , 130 /article/10.1007/s10458-025-09691-y ref-CR130 . In these methods, language emerges through iterative learning across and within agent sub-groups called communities. Generational learning additionally involves older generations of agents training younger ones using previously developed communication as a foundation 105 /article/10.1007/s10458-025-09691-y ref-CR105 , 117 /article/10.1007/s10458-025-09691-y ref-CR117 . Language transfer across groups or generations can be interpreted as an iterative prior. However, this method remains a fully evolutionary approach in the absence of a deliberately designed prior. 5.4 Language characteristics As discussed in Sect. 2.2 /article/10.1007/s10458-025-09691-y Sec4 , language is a complex, multifaceted system 88 /article/10.1007/s10458-025-09691-y ref-CR88 , 90 /article/10.1007/s10458-025-09691-y ref-CR90 . Therefore, it is essential to establish a comprehensive taxonomy of its properties to provide a unified framework for EL research. This taxonomy will not only facilitate the unambiguous categorization of metrics used in EL studies cf. Section 6 /article/10.1007/s10458-025-09691-y Sec20 but will also enhance the comparability and comprehensibility of approaches and results within the field. As shown previously in Fig. 4 /article/10.1007/s10458-025-09691-y Fig4 , NL can be divided hierarchically into distinct characteristics 99 ref-CR99 , 100 ref-CR100 , 101 /article/10.1007/s10458-025-09691-y ref-CR101 . The following sections provide a categorization of the reviewed publications along these characteristics, occasionally breaking them down into smaller sub-characteristics if relevant. 5.4.1 Phonetics The phonetics of a language inherently represents its medium, delineating the constraints of the specific communication channel 3 /article/10.1007/s10458-025-09691-y ref-CR3 , 101 /article/10.1007/s10458-025-09691-y ref-CR101 . These media or channels can be either discrete or continuous; for example, an audio channel is continuous, while a symbolic channel is typically discrete. Regardless of the type, they lay the foundation for the nature of communication. However, for EL research the discrete case is of particular importance, as it closely mirrors NL as we understand it 35 /article/10.1007/s10458-025-09691-y ref-CR35 . Although humans use a continuous phonetic medium for communication, some degree of discretization is essential to establish a common ground for efficient communication 100 /article/10.1007/s10458-025-09691-y ref-CR100 . Table 5 /article/10.1007/s10458-025-09691-y Tab5 provides an overview of the reviewed papers, categorized according to the continuous or discrete approach. Notably, some papers explore both approaches, providing valuable insights for researchers interested in the basic aspects of phonetics research in EL. 5.4.2 Phonology Phonology encompasses the actively used vocabulary and determines the part of the medium that is utilized for communication. We identified five different types of vocabulary actively researched, however, some of them are rare to find in the literature. Table 6 /article/10.1007/s10458-025-09691-y Tab6 summarizes the results of our survey regarding vocabulary types in EL research. One commonly used phonological type in EL is a binary encoding, while an even more prominent type is a token-based vocabulary. However, these two phonological classes are not always distinct, as a token-based vocabulary often builds upon a binary encoded representation 112 /article/10.1007/s10458-025-09691-y ref-CR112 . The other three types, which are distinct from the two most prominent, are rarely mentioned in the literature reviewed. One of these types involves using NL vocabulary, such as all the words from an English dictionary. While this approach enforces the NL resemblance of the EL, it also drastically limits the emergence and associated benefits 19 /article/10.1007/s10458-025-09691-y ref-CR19 . Essentially, this phonological preset strips the agents of the possibility to shape phonology and morphology. The other two vocabulary types being referred to are sound and graphics. The former enables agents to produce and process sound 200 /article/10.1007/s10458-025-09691-y ref-CR200 , while the latter focuses on enabling agents to draw and analyze graphical representations 104 /article/10.1007/s10458-025-09691-y ref-CR104 , 174 /article/10.1007/s10458-025-09691-y ref-CR174 . Both mediums present challenges in ensuring discretization, which may be the reason why they are not as extensively researched in EL. 5.4.3 Morphology Morphology governs the rules for constructing words and sentences, meaning the overall ability to combine individual elements, also called tokens, into words and to combine those words into sentences 101 /article/10.1007/s10458-025-09691-y ref-CR101 . This is particularly relevant in the field of EL due to the prominent division of existing work based on morphological setup and options. The most significant differentiation is between the use of a fixed or flexible message length. Table 7 /article/10.1007/s10458-025-09691-y Tab7 demonstrates that much of the existing work employs fixed message lengths, despite this setup not being comparable to NL 15 /article/10.1007/s10458-025-09691-y ref-CR15 . For instance, NL users, such as humans, have the ability to adjust the length of their message to fit their intention, which may vary depending on the audience, medium, or communicative goal. When communicating with colleagues, they may use shorter sentences to be efficient, while more detailed explanations may be used when conversing with friends. Accordingly, this characteristic can be measured using metrics that assess word formation and vocabulary. Based on the metrics found in the literature, distinct features of language morphology can be quantified. Specifically, this refers to the compression of language and the presence of redundancy or ambiguity. Compression Compression 105 /article/10.1007/s10458-025-09691-y ref-CR105 , also known as combinatoriality 224 /article/10.1007/s10458-025-09691-y ref-CR224 , refers to the ability of a communication system to combine a small number of basic elements to create a vast range of words that can carry meaning. This feature of discrete communication is crucial in producing comprehensive and flexible communication with limited resources, and is an essential characteristic of NL. We assume that using compressed language is generally favorable for language learners as it reduces the burden of learning 105 /article/10.1007/s10458-025-09691-y ref-CR105 . Redundancy or ambiguity In NL, words and phrases can have redundant or ambiguous meanings. Redundancy occurs when multiple words convey the same meaning, while ambiguity arises from a limited vocabulary 14 /article/10.1007/s10458-025-09691-y ref-CR14 , 114 /article/10.1007/s10458-025-09691-y ref-CR114 . The addition of this characteristic in the morphology subsection rather than the semantics subsection may be controversial. We argue that any metric measuring redundancy or ambiguity provides more useful information about the morphology, encompassing the form and size of the vocabulary, than it does about the semantic range and capabilities of the language. However, to quantify redundancy or ambiguity, we must establish semantic meaning first. 5.4.4 Syntax The syntax of a language establishes the grammatical rules that govern sentence formation. Consequently, syntax plays a central role in establishing a functional correspondence between emerged language and NL 245 /article/10.1007/s10458-025-09691-y ref-CR245 . This specific characteristic of a discrete language is underrepresented in current EL literature. However, we found two examples in the body of literature discussing syntax in EL. Ueda et al. 102 /article/10.1007/s10458-025-09691-y ref-CR102 introduced a method to examine the syntactic structure of an EL using categorial grammar induction CGI , which is based on the induction of categorial grammars from sentence-meaning pairs. This method is straightforward in simple referential games. Additionally, van der Wal et al. 193 /article/10.1007/s10458-025-09691-y ref-CR193 introduced unsupervised grammar induction UGI techniques for syntax analysis in EL research. We discuss the methods they use to measure and analyze syntax in an EL briefly in Sect. 6.3 /article/10.1007/s10458-025-09691-y Sec27 . 5.4.5 Semantics Semantics is concerned with the literal meaning of language constructs and is a dominant topic in current EL research, as shown in Table 12 /article/10.1007/s10458-025-09691-y Tab12 in Appendix A /article/10.1007/s10458-025-09691-y Sec47 . EL studies often focus on establishing useful and meaningful communication between agents, making semantics a central feature 35 /article/10.1007/s10458-025-09691-y ref-CR35 . It serves as a crucial tool for distinguishing actual information exchange from mere noise utterances 168 /article/10.1007/s10458-025-09691-y ref-CR168 , 202 /article/10.1007/s10458-025-09691-y ref-CR202 . Given the complexity of capturing the meaning of literal language in a single metric, several features have been introduced to measure the semantics of EL. In particular, these features include grounding, compositionality, consistency, and generalization, as shown in Fig. 7 /article/10.1007/s10458-025-09691-y Fig7 . Table 8 /article/10.1007/s10458-025-09691-y Tab8 provides an overview of the literature addressing the individual semantic features in EL. Grounding A language is considered grounded when it is deeply intertwined with the environment, for example, when it is tightly bound to environmental concepts and objects 149 /article/10.1007/s10458-025-09691-y ref-CR149 , 184 /article/10.1007/s10458-025-09691-y ref-CR184 , 246 /article/10.1007/s10458-025-09691-y ref-CR246 . Grounding is essential for the interoperability of individuals and is particularly important in NL communication, where meaningful interaction requires shared understanding 33 /article/10.1007/s10458-025-09691-y ref-CR33 , 36 /article/10.1007/s10458-025-09691-y ref-CR36 , 112 /article/10.1007/s10458-025-09691-y ref-CR112 . While in theory, an EL can establish a unique form of grounding using self-emerged concepts distinct from those in NL, deriving a useful metric for such a scenario proves challenging. This difficulty arises from the need to compare ELs to existing and comprehensible grounding principles typically found in NLs 50 /article/10.1007/s10458-025-09691-y ref-CR50 , 112 /article/10.1007/s10458-025-09691-y ref-CR112 , 127 /article/10.1007/s10458-025-09691-y ref-CR127 . Compositionality When a language exhibits compositionality, its components can be rearranged or replaced by conceptually equivalent words without changing the overall meaning 121 /article/10.1007/s10458-025-09691-y ref-CR121 , 122 /article/10.1007/s10458-025-09691-y ref-CR122 . Compositionality facilitates the construction of higher-level concepts, using conceptual foundations to enable efficient language expression 46 /article/10.1007/s10458-025-09691-y ref-CR46 , 51 /article/10.1007/s10458-025-09691-y ref-CR51 , 135 /article/10.1007/s10458-025-09691-y ref-CR135 . For example, NLs partition concepts such as objects and their attributes to allow compositional constructions 18 /article/10.1007/s10458-025-09691-y ref-CR18 , 155 /article/10.1007/s10458-025-09691-y ref-CR155 . As a result, we can describe variations of a single object using different words from the same semantic concept, such as ‘blue towel and ‘red towel for the object towel and the semantic concept of color. Similarly, we can attribute specific properties to different objects using the same phrase, as in ‘green towel and ‘green car. Ultimately, compositionality is beneficial for the learning process 36 /article/10.1007/s10458-025-09691-y ref-CR36 , 122 /article/10.1007/s10458-025-09691-y ref-CR122 and promotes efficient and rich language use, even in systems with limited memory capacity 18 /article/10.1007/s10458-025-09691-y ref-CR18 , 108 /article/10.1007/s10458-025-09691-y ref-CR108 , 119 /article/10.1007/s10458-025-09691-y ref-CR119 . Consistency Merely having grounded words in a language does not necessarily guarantee its semantic quality. In addition, consistency is essential for a language to convey meaningful and practical information effectively 36 /article/10.1007/s10458-025-09691-y ref-CR36 , 247 /article/10.1007/s10458-025-09691-y ref-CR247 . If the words within a language lack consistency in their literal meanings, they will not facilitate effective communication. Therefore, even if a language is semantically grounded and compositional, its utility is compromised if the words exhibit inconsistent literal meanings 127 /article/10.1007/s10458-025-09691-y ref-CR127 . While words can change their general meaning to fit the context, their literal meaning should remain consistent to keep their usefulness 33 /article/10.1007/s10458-025-09691-y ref-CR33 . Generalization Generalization serves as a cornerstone of NL, allowing humans to communicate about topics ranging from simple to complex, broad to specific, and known to unknown, all with a relatively limited vocabulary 25 /article/10.1007/s10458-025-09691-y ref-CR25 , 135 /article/10.1007/s10458-025-09691-y ref-CR135 . A language that excels at generalization enables its users to navigate different levels of complexity, facilitating hierarchical descriptions of concepts and relationships 37 /article/10.1007/s10458-025-09691-y ref-CR37 . Consequently, generalization and compositionality are closely related, as they both contribute to the flexibility and expressiveness of language 36 /article/10.1007/s10458-025-09691-y ref-CR36 , 117 /article/10.1007/s10458-025-09691-y ref-CR117 , 168 /article/10.1007/s10458-025-09691-y ref-CR168 . This ability to generalize not only enriches communication but also underscores the adaptability and robustness of human language. 5.4.6 Pragmatics The final dimension of EL research is pragmatics. This field of study examines how language is employed in context, particularly in interactions, and how it conveys information 101 /article/10.1007/s10458-025-09691-y ref-CR101 , 160 /article/10.1007/s10458-025-09691-y ref-CR160 , 248 /article/10.1007/s10458-025-09691-y ref-CR248 . By evaluating the pragmatics of the linguistic structure, we can ascertain whether EL is itself useful and utilized effectively. While this assessment may be feasible based on rewards in a standard RL setting, integrating communication into such environments increases the complexity. This is because most setups do not separate the agent’s environment interaction from its communication capabilities, thereby expanding the network’s capacity, and making it difficult to attribute an increase in reward directly to EL 35 /article/10.1007/s10458-025-09691-y ref-CR35 . As outlined in Table 9 /article/10.1007/s10458-025-09691-y Tab9 and depicted in Fig. 8 /article/10.1007/s10458-025-09691-y Fig8 , five distinct features have been identified: predictability, efficiency, positive signaling, positive listening, and symmetry. These features are essential for assessing the constructive impact and utilization of EL. Understanding how agents employ language is crucial in evaluating its effectiveness and overall benefit. Predictability Predictability is concerned with the assessment of the complexity of the context, including the action space within the environment. When actions exhibit less diversity, it becomes more feasible to coordinate without communication 110 /article/10.1007/s10458-025-09691-y ref-CR110 . For instance, in a simple grid-based environment where agents have only two possible actions-moving left or right-agents can often achieve their objectives without the need for communication. In such a scenario, the limited action space reduces the necessity for EL, as agents can predict each other’s movements based on past behavior or simple rules. However, in a more complex environment where agents have multiple actions, such as navigating a maze with numerous paths and obstacles, the need for effective communication increases. Here, EL can significantly enhance coordination by allowing agents to share information about their positions, plans, or discoveries, thus improving their overall performance in navigating the maze. Therefore, it is essential to compare the diversity of signaling and context attributes to evaluate the potential benefit of EL. Efficiency Efficiency is a critical aspect considered whenever communication entails a cost. This is particularly true in the context of modeling the emergence of NL and the broader objective of employing EL for human-computer interaction HCI . In EL settings, the achievement of concise communication is contingent upon the presence of an opportunity cost 115 /article/10.1007/s10458-025-09691-y ref-CR115 . Without such a cost, there is no incentive to communicate concisely, making EL ineffective as an intermediary for HCI. When communication is accompanied by a cost the necessity for efficiency in communication becomes paramount. In such scenarios, the objective is to minimize the cost while maximizing the effectiveness of communication within a given task. Positive signaling The concept of positive signaling is concerned with the degree of alignment between the observations, knowledge, and experience of the message producer and their communication output 29 /article/10.1007/s10458-025-09691-y ref-CR29 . The objective is to guarantee the transmission of useful information, or at the very least, information that the speaker can discern through knowledge or observation 181 /article/10.1007/s10458-025-09691-y ref-CR181 . This feature assumes that all communication should be relevant to something observable, known or tangible to the speaker. Thus, it stipulates that the situational information content of the produced signal is crucial for a language that can be used in a contextually meaningful way. Positive listening Positive listening, in contrast to positive signaling, focuses on the role of the message receiver, to evaluate the active processing of incoming information 29 /article/10.1007/s10458-025-09691-y ref-CR29 . From a pragmatic point of view, it makes sense to process incoming messages in a meaningful way. However, the definition of meaningful processing is broad. Positive listening as defined in this taxonomy, contrary to earlier work 29 /article/10.1007/s10458-025-09691-y ref-CR29 , 202 /article/10.1007/s10458-025-09691-y ref-CR202 , does not necessarily require a connection between the incoming message and the subsequent action; it is much more about active processing, which may or may not lead to inclusion in the choice of action. Thus, active engagement followed by rejection or disregard is also considered positive listening in the context of this taxonomy. Symmetry Symmetry in EL is defined as the consistency in language usage among participating agents 105 /article/10.1007/s10458-025-09691-y ref-CR105 , 110 /article/10.1007/s10458-025-09691-y ref-CR110 . This concept applies to MARL settings where agents can assume multiple roles, such as message producer and message receiver. Symmetry plays a crucial role in achieving convergence on a shared and aligned EL. For instance, if an agent employs language differently depending on whether it is sending or receiving messages so that words have varying meanings based on the assigned role the EL setting is considered asymmetric. In such instances, rather than learning a collectively and contextually grounded language, agents develop individual protocols specific to their respective roles 110 /article/10.1007/s10458-025-09691-y ref-CR110 . This would suggest that there is no common language, but rather separate codes that can only be applied to specific combinations and conditions. For this reason, this pragmatic feature is particularly relevant, since the aim of EL is a common language. 5.5 Summary of the taxonomy Our proposed taxonomy systematically categorizes the key features of EL systems, including communication settings, language games, language priors, and language characteristics. The latter is particularly detailed, with sub-characteristics and their features aligned with the major levels of linguistic structure, as previously illustrated in Fig. 4 /article/10.1007/s10458-025-09691-y Fig4 . This comprehensive taxonomy enables a standardized comparison of approaches in the EL literature, highlighting the opportunities and properties associated with individual options and topics in EL research. Specifically, by applying this taxonomy, especially in terms of language characteristics, we can uncover the capabilities and potentials of various EL approaches. This facilitates a more detailed, comparable, and insightful analysis of EL. 6 Metrics This section provides a comprehensive categorization and review of existing metrics used in EL research. The section is organized along the same categorization used in Sect. 5.4 /article/10.1007/s10458-025-09691-y Sec12 . Note that the categories of phonetics and phonology are excluded from this discussion, as these aspects are predetermined settings in the current EL literature and thus not yet targeted by metrics. We begin by introducing the notational system used for all metrics to ensure consistency and facilitate ease of use. We then describe the metrics within each category, detailing the individual metric and adapting it to our notation. For each metric, we provide references to both original sources and additional literature, if available, to enable further exploration beyond the scope of this work. Figure 9 /article/10.1007/s10458-025-09691-y Fig9 provides a visual summary of the existing metrics and their correspondence to the language characteristics. An extended version including all references for the individual metrics is provided in Fig. 12 /article/10.1007/s10458-025-09691-y Fig12 in Appendix B /article/10.1007/s10458-025-09691-y Sec48 6.1 Notation Given the complexity and variability within the EL field, it is crucial to establish a unified and coherent notation system. In this section we present a standardized mathematical notation designed to be consistent across the various aspects of EL research, thereby facilitating clearer communication and comparison of results within the community. This approach aligns with our broader goal of advancing the field through a common taxonomy that supports the development of measurable and interpretable ELs. Throughout this section we focus on finite and discrete languages, although some of the definitions and metrics discussed here are also applicable to continuous languages. These languages offer a more straightforward mapping to NLs, making them particularly relevant to the study of EL systems. 6.1.1 Definition In alignment with the semiotic cycle introduced in Sect. 2.2 /article/10.1007/s10458-025-09691-y Sec4 , our notation is organized into three interconnected spaces: setting, meaning, and language. The setting space encompasses the typical elements of RL, providing the foundational environment in which agents operate. The meaning space incorporates a representation learning endeavor, whereby sensory input is integrated with decision-relevant information to generate a coherent internal representation. Finally, the language space encompasses both the production and comprehension of discrete messages, encapsulating the communication process. These components, illustrated in Fig. 10 /article/10.1007/s10458-025-09691-y Fig10 , will be introduced and explored in detail in the following paragraphs. Setting The overall setting, consisting of the environment, actions, goals, and other typical RL elements, is denoted by \ \Omega\ . Let \ \xi\ denote the set of all entities in the system, with an individual entity represented as \ \xi i \in \xi\ . Each entity can assume specific roles, such as the sender S or receiver R in a communication scenario. An entity can assume several roles over the course of the entire communication scenario. However, for an individual message exchange, an entity assumes one specific role. We represent the role of an individual entity i by \ \xi {i,j} \in \xi i\ , where j specifies the role e.g., \ j = S\ or \ j = R\ . Entities interact with their environment \ {\mathcal {E}}\ through actions, denoted as a , which belong to the set of possible actions A , such that \ a \in A\ . The action taken by a specific entity \ \xi i\ is represented as \ a {\xi i}\ . The state of the environment at any given time is denoted by s , which is an element of the state space S , so that \ s \in S\ . As the system progresses over time, denoted by discrete points in time \ \left 0, \dots , t \right \ , the sequence of states and actions forms a trajectory \ \tau\ , generally expressed as \ \tau = \left\{ s^0, a^0, \dots , s^t, a^t \right\}\ . It is important to note that the entities described here do not necessarily correspond to autonomous agents in the traditional sense; they could also represent ground truth models, human participants, or abstract constructs that lack the direct interaction capabilities typically associated with agents. Despite this distinction, for the sake of clarity and consistency, we will refer to these entities as agents in the following sections. Given the importance of partial observability in EL research 18 /article/10.1007/s10458-025-09691-y ref-CR18 , 50 /article/10.1007/s10458-025-09691-y ref-CR50 , 172 /article/10.1007/s10458-025-09691-y ref-CR172 , 228 /article/10.1007/s10458-025-09691-y ref-CR228 , it is essential to consider that agents only have access to their own observations, denoted as \ o {\xi }\ , which are derived from the underlying state s . An individual observation \ o {\xi }\ is an element of the collection of observations of an agent \ O {\xi }\ , which is a subset of the observation space O , so that \ o {\xi } \in O {\xi } \subseteq O\ . In our framework, an observation \ o {\xi }\ effectively replaces the ‘world model’ component from the traditional semiotic cycle, highlighting the localized and subjective nature of an agent’s perception in partially observable environments. Referential games cf. Table 4 /article/10.1007/s10458-025-09691-y Tab4 are frequently employed in EL literature. They often operate on individual, static samples that are drawn from a corresponding dataset or distribution. In doing so, they differ from traditional RL setups that emphasize sequential decision-making and environmental interactions over time. In such cases, rather than speaking of a state s or an observation o , we use the term sample k , which is an element of the collection of all samples K , so that \ k \in K\ . The specific nature of a sample depends on the environment; for example, in an image-based sender-receiver game, the sample would be an image. Each sample is represented by its feature vector f , which belongs to the feature space F , so that \ f \in F\ . The feature vector corresponding to a specific sample k is denoted by \ f k\ . In EL settings, the communicative goal g of an agent may differ from the reinforcement learning task goal. In addition, depending on the game, the sender and receiver may have distinct goals. These are important factors to consider when evaluating the communicative behavior. Meaning In our notation, the meaning space, denoted by \ \Phi\ , serves as the critical intermediary between the setting space and the language space. The meaning space represents the semantic connections derived from the provided information. Each element within this space, represented by a specific meaning vector \ \varphi \in \Phi\ , captures the essence of concepts or objects as understood by the agent. These meaning vectors are critical to the processes of language comprehension and production, as well as to the processes of conceptualization and interpretation, that allow an agent to effectively use inputs and generate outputs in the setting space cf. Figure 10 /article/10.1007/s10458-025-09691-y Fig10 . The representation mappings \ \Psi\ within the meaning space are agent-specific and referred to as \ \Psi {\text {con}}\ and \ \Psi {\text {int}}\ , given in Eq. 1 /article/10.1007/s10458-025-09691-y Equ1 . These mappings enable the transition between an arbitrary space \ \chi\ , such as sensory inputs or raw data, and the meaning space, where the data acquires semantic meaning. \ \Psi {\text {con}}\ refers to the conceptualization process that transforms raw, uninterpreted data into meaningful representations within \ \Phi\ . Conversely, \ \Psi {\text {int}}\ denotes the interpretation process that translates these meaning vectors back into the arbitrary space that can represent any external or internal stimuli. These mappings are critical to the agent’s ability to both understand its environment and communicate effectively within it through language that is both grounded in and reflective of the underlying reality with which the agents interact. Language In our proposed framework, a message m belongs to the message space M , such that \ m \in M\ . Each message encapsulates semantic and pragmatic content, serving as a vehicle for meaningful communication between agents. A message is composed of individual words w , which are elements of a finite collection W , commonly referred to as vocabulary, lexicon, or dictionary. In this context, each word is considered a semantic unit that carries intrinsic meaning. At the lowest level, a word is composed of characters or symbols \ \upsilon \in \Upsilon\ . These atomic characters, while essential for constructing words, do not independently carry semantic meaning. Instead, they function as elements of a finite set \ \Upsilon\ from which any number of meaningful words can be composed. Building on the formalization from 119 /article/10.1007/s10458-025-09691-y ref-CR119 , we describe the message space \ M {\xi }\ of an agent \ \xi\ , which represents the agent’s language capabilities from a compositional standpoint. The message space \ M {\xi } \subseteq M\ is composed of a set of messages or strings \ m {\xi }\ , each constructed from words within \ W {\xi }\ , as shown in Eq. 2 /article/10.1007/s10458-025-09691-y Equ2 . Further, each \ w {\xi } \in m {\xi }\ is composed of a set of characters \ \upsilon {\xi } \in \Upsilon {\xi } \subseteq \Upsilon\ utilized by the agent, given by Eq. 3 /article/10.1007/s10458-025-09691-y Equ3 . A language \ {\mathcal {L}}\ encompasses a set of mapping functions that facilitate the transformation between the message space M and other arbitrary spaces \ \chi\ . These mappings are agent-specific and enable both the production of messages, denoted as \ {\mathcal {L}} {\text {prod}}\ , and the comprehension of messages, denoted as \ {\mathcal {L}} {\text {comp}}\ . This framework aligns with the linguistic level description of the semiotic cycle presented in Fig. 3 /article/10.1007/s10458-025-09691-y Fig3 . Within this context, we formally define a language \ {\mathcal {L}}\ in Eq. 4 /article/10.1007/s10458-025-09691-y Equ4 . These emerging mapping functions are not necessarily injective, meaning that distinct inputs from the space \ \chi\ could potentially be mapped to an identical message within M 117 /article/10.1007/s10458-025-09691-y ref-CR117 , 120 /article/10.1007/s10458-025-09691-y ref-CR120 . Conversely, distinct messages within M could also be mapped to the same value in \ \chi\ . While this non-injectivity adds a layer of complexity to the expressiveness of the language, it also introduces a degree of flexibility that can be advantageous in certain communication scenarios. For example, it allows for synonymy where different messages convey the same meaning , which can provide redundancy and flexibility in communication, and homonymy where the same message may have multiple interpretations depending on context , which can facilitate more nuanced and context-dependent communication. These natural phenomena, though challenging, are well-documented in NLs and are of particular interest in the design and evaluation of artificial communication systems 101 /article/10.1007/s10458-025-09691-y ref-CR101 , 249 /article/10.1007/s10458-025-09691-y ref-CR249 . However, managing these complexities effectively is crucial, as unchecked non-injectivity could lead to ambiguities that complicate communication rather than simplifying it. 6.1.2 Important notes The notation presented here is designed to be comprehensible and thorough; however, it may not be directly applicable in all cases to existing works, as these employ different wordings. a lot of existing work uses the term ‘word’, which in our notation describes element carrying semantic meaning, and ‘symbols’, which in our notation serve as fundamental building blocks without inherent semantic meaning, interchangeably 19 /article/10.1007/s10458-025-09691-y ref-CR19 , 35 /article/10.1007/s10458-025-09691-y ref-CR35 , 134 /article/10.1007/s10458-025-09691-y ref-CR134 , 150 /article/10.1007/s10458-025-09691-y ref-CR150 , 186 /article/10.1007/s10458-025-09691-y ref-CR186 . Furthermore, a considerable proportion of existing literature utilizes a multitude of different definitions for concepts such as ‘meaning space’ 128 /article/10.1007/s10458-025-09691-y ref-CR128 , 138 /article/10.1007/s10458-025-09691-y ref-CR138 , 181 /article/10.1007/s10458-025-09691-y ref-CR181 , ‘ground-truth oracle’ 134 /article/10.1007/s10458-025-09691-y ref-CR134 , 148 /article/10.1007/s10458-025-09691-y ref-CR148 , and other pivotal elements. In our endeavor to establish a unified framework, we have occasionally adopted terminology that differs from that used by the original authors. While this may initially lead to some confusion, we intend to mitigate this by providing transparent and detailed descriptions. Our objective is a consistent application of these concepts across the field of EL research, thereby promoting coherence between different studies. The following sections attempt to align existing research and metrics with the proposed framework. While this alignment has required some linguistic adjustments to existing terminology and procedures, it is important to note that no substantive changes have been made to the underlying methodologies. 6.2 Morphology Morphological metrics aim to evaluate the structure and formation of words within a language, as well as the richness and diversity of its vocabulary. The identified metrics focus on aspects such as language compression, redundancy, and ambiguity. The morphology of a language significantly influences the complexity of language based tasks 250 /article/10.1007/s10458-025-09691-y ref-CR250 . Therefore, the evaluation of morphological features is a crucial component for understanding and evaluating the effectiveness of ELs. 6.2.1 Compression The concept of compression within a language refers to its ability to efficiently combine and reuse a limited set of characters to generate a large collection of words or meanings 105 /article/10.1007/s10458-025-09691-y ref-CR105 , 224 /article/10.1007/s10458-025-09691-y ref-CR224 . Several metrics can be used to quantify compression in ELs. A straightforward approach for these metrics is to use statistical measures, as shown in the following paragraphs. These metrics provide insight into the efficiency of the language, indicating how well it minimizes redundancy while maximizing expressiveness. Efficient compression is a key indicator of a communication system, especially in scenarios where resources such as memory or bandwidth are constrained. Distinct appearances The metric of distinct appearances \ {DA}\ was proposed by Loreto et al. 224 /article/10.1007/s10458-025-09691-y ref-CR224 . It is formalized in Eq. 5 /article/10.1007/s10458-025-09691-y Equ5 and designed to quantify the capacity of a communication system to name a diverse set of objects or categories using its available symbols 224 /article/10.1007/s10458-025-09691-y ref-CR224 . Specifically, this metric evaluates how frequently characters \ \upsilon \in \Upsilon\ are reused across different words or names w within the lexicon W . By examining the set \ W {\upsilon }\ , which includes all words containing a given character \ \upsilon\ , we can assess the system’s flexibility in recombining basic units to generate a broad spectrum of expressions. A high \ {DA}\ value, approaching 1, indicates that the characters are highly versatile and reused extensively across different words, thereby reflecting a flexible communication system. Conversely, a low \ {DA}\ value suggests limited reuse of characters, which may imply constraints in the system’s expressiveness or a less efficient use of its symbolic resources. This metric provides insights into how efficiently a system can balance the trade-off between a compact character set and the richness of its vocabulary. Average message length Another way to assess the degree of compression achieved by agents in their communication is to analyze the average message length 72 /article/10.1007/s10458-025-09691-y ref-CR72 , 114 /article/10.1007/s10458-025-09691-y ref-CR114 , 142 /article/10.1007/s10458-025-09691-y ref-CR142 , 163 /article/10.1007/s10458-025-09691-y ref-CR163 , 169 /article/10.1007/s10458-025-09691-y ref-CR169 , 193 /article/10.1007/s10458-025-09691-y ref-CR193 . This metric, which appears for the first time in Choi et al. 72 /article/10.1007/s10458-025-09691-y ref-CR72 , captures the typical length of generated messages and provides insight into the efficiency of the EL in terms of information density 114 /article/10.1007/s10458-025-09691-y ref-CR114 . By tracking the average number of words in the messages, we can quantify how effectively the agents compress their language. This metric is computed at the word level, meaning each word within a message is counted. The average message length \ \overline{ |m |}\ for a set of messages M is calculated as follows: Active words The active words metric, introduced by Lazaridou et al. 14 /article/10.1007/s10458-025-09691-y ref-CR14 , complements the average message length by quantifying the diversity of word usage within the vocabulary 114 /article/10.1007/s10458-025-09691-y ref-CR114 . Specifically, this metric measures the variety and utilization of distinct words in a communication system. A high number of active words indicates a diverse vocabulary, reflecting a more complex or redundant EL. Conversely, a lower number suggests that the communication system relies on a limited set of words, which may indicate a more efficient and compressed language with less synonyms 18 /article/10.1007/s10458-025-09691-y ref-CR18 . This metric is widely used in the literature 14 /article/10.1007/s10458-025-09691-y ref-CR14 , 18 /article/10.1007/s10458-025-09691-y ref-CR18 , 32 /article/10.1007/s10458-025-09691-y ref-CR32 , 114 /article/10.1007/s10458-025-09691-y ref-CR114 , 116 /article/10.1007/s10458-025-09691-y ref-CR116 , 119 /article/10.1007/s10458-025-09691-y ref-CR119 , 137 /article/10.1007/s10458-025-09691-y ref-CR137 , 140 /article/10.1007/s10458-025-09691-y ref-CR140 , 142 /article/10.1007/s10458-025-09691-y ref-CR142 , 144 /article/10.1007/s10458-025-09691-y ref-CR144 , 151 /article/10.1007/s10458-025-09691-y ref-CR151 , 187 /article/10.1007/s10458-025-09691-y ref-CR187 . Mathematically, the active word value \ {AW}\ for an agent \ \xi {i}\ can be defined as the size of the collection of words actively used by the agent \ W {\xi {i}}\ , as given in Eq. 7 /article/10.1007/s10458-025-09691-y Equ7 . In multi-agent setups, this metric can be averaged across all agents to provide a collective measure of vocabulary diversity within the joint system. 6.2.2 Redundancy or ambiguity Redundancy in language occurs when multiple words are associated with the same meaning, providing alternative expressions for the same concept. Conversely, ambiguity occurs when a single word is associated with multiple meanings, creating the potential for different interpretations depending on the context. Both redundancy and ambiguity are characteristic features of NLs, reflecting the complexity and flexibility inherent in human communication 114 /article/10.1007/s10458-025-09691-y ref-CR114 , 132 /article/10.1007/s10458-025-09691-y ref-CR132 . Perplexity Perplexity, introduced by Havrylov and Titov 15 /article/10.1007/s10458-025-09691-y ref-CR15 , measures how often a word was used in a message to describe the same object 72 /article/10.1007/s10458-025-09691-y ref-CR72 , 114 /article/10.1007/s10458-025-09691-y ref-CR114 . “A lower perplexity shows that the same |words| are consistently used to describe the same objects.” 114 /article/10.1007/s10458-025-09691-y ref-CR114 . Mathematically, \ P \left w | \varphi \right \ represents the probability or score of a word for a specific concept or meaning, e.g., derived from an affine transformation of the sender’s hidden state 114 /article/10.1007/s10458-025-09691-y ref-CR114 or from a ground truth label 72 /article/10.1007/s10458-025-09691-y ref-CR72 . Thus, perplexity, given in Eq. 8 /article/10.1007/s10458-025-09691-y Equ8 , quantifies the predictability of word usage, with lower values reflecting a less redundant communication system. It is usually calculated based on a sampled set of meanings \ \Phi {\text {test}}\ for which the word probability can be generated. Singular value decomposition Another approach to quantitatively assess the redundancy of the vocabulary used in a communication system is outlined by Lazaridou et al. 14 /article/10.1007/s10458-025-09691-y ref-CR14 . This method involves constructing a matrix where the rows correspond to distinct meanings, the columns represent individual words, and the matrix entries indicate the frequency with which each word is used for a given meaning. The rows are thus constructed based on a predefined ground truth classification. By applying Singular Value Decomposition SVD to this matrix, we can examine the dimensionality of the underlying communication strategy. If the communication system relies on a limited set of highly synonymous words, we would expect the SVD to reveal a low-dimensional structure. Conversely, a higher-dimensional decomposition would indicate a more diverse use of vocabulary, reflecting a potentially less synonymous and more redundant language. Message distinctness Message distinctness evaluates the linguistic representation of distinct features and thus aims to quantify ambiguity 72 /article/10.1007/s10458-025-09691-y ref-CR72 , 114 /article/10.1007/s10458-025-09691-y ref-CR114 , 168 /article/10.1007/s10458-025-09691-y ref-CR168 , 225 /article/10.1007/s10458-025-09691-y ref-CR225 . The metric, first suggested in Lazaridou et al. 168 /article/10.1007/s10458-025-09691-y ref-CR168 and Choi et al. 72 /article/10.1007/s10458-025-09691-y ref-CR72 , quantifies the diversity of messages generated by the agent by assessing how well it differentiates between various inputs. Specifically, message distinctness \ {MD}\ is calculated as the ratio of the number of unique messages generated within a batch cf. 9 /article/10.1007/s10458-025-09691-y Equ9 to the batch size cf. 10 /article/10.1007/s10458-025-09691-y Equ10 . A higher message distinctness indicates less ambiguity of the language. 6.3 Syntax Despite the significance of structural properties in ELs, particularly regarding their syntax and its relation to semantics, research in this area remains limited 245 /article/10.1007/s10458-025-09691-y ref-CR245 . Recurrent syntactical patterns are central to the robustness and versatility of NLs 50 /article/10.1007/s10458-025-09691-y ref-CR50 . Exploring these properties within the context of EL could provide valuable insights into their development and alignment with NL. 6.3.1 Syntax tree Van der Wal et al. 193 /article/10.1007/s10458-025-09691-y ref-CR193 introduced unsupervised grammar induction UGI techniques for syntax analysis in EL research, describing a two-stage approach to deriving grammar and syntax. The first phase involves the induction of unlabeled constituent tree structures, explained below, and the labeling of these structures. The second phase extracts a probabilistic context-free grammar PCFG from the labeled data. Two methods were compared for constituency structure induction: the Common Cover Link CCL , a pre-neural statistical parser that makes assumptions about NL such as the Zipfian distribution, and the Deep Inside-Outside Recursive Auto-encoder DIORA , a neural parser. For the labeling process, Van der Wal et al. 193 /article/10.1007/s10458-025-09691-y ref-CR193 used Bayesian Model Merging BMM , to consolidate probabilistic models to label the induced syntax trees. In syntax trees, the structure of the language is represented in a hierarchical manner, where nodes represent grammatical constructs such as sentences, phrases, and words and edges represent the rules or relationships that connect these constructs. Analysis of these trees helps to understand how well grammar induction methods match the true syntactic nature of ELs. There are several metrics associated with syntax trees that are used to measure the complexity of the grammar 193 /article/10.1007/s10458-025-09691-y ref-CR193 . First, tree depth measures the maximum distance from the root of the tree to its deepest leaf. Tree depth reflects the hierarchical complexity of the grammar. Shallow trees indicate a simpler grammar, while deeper trees suggest a more complex syntactic structure. Second, the number of unique preterminal groups is a metric that counts the different sets of preterminals intermediate symbols that appear to the right of production rules in a grammar. A larger number of unique preterminal groups indicates a richer and more diverse syntactic organization, suggesting that the grammar can generate a greater variety of structures. 6.3.2 Categorical grammar induction Ueda et al. 102 /article/10.1007/s10458-025-09691-y ref-CR102 proposed a novel approach for analyzing the syntactic structure of ELs using Categorial Grammar Induction CGI . This technique focuses on deriving categorial grammars from message-meaning pairs, making it particularly well-suited for simple referential or signaling games. In this method, derivation trees are constructed using lexical entries and application rules, mapping messages to atomic syntactical representations. Given that multiple derivations might exist for a single message, “the most likely derivation is selected using a log-linear model” 102 /article/10.1007/s10458-025-09691-y ref-CR102 . CGI is particularly valuable for assessing the syntactic structure of an EL using the generated trees. 6.4 Semantics Capturing the semantic properties of ELs is inherently complex, making it difficult to encapsulate nuances in a single metric. To address this, several key features have been introduced, including grounding, compositionality, consistency, and generalization. These are important because agents can develop representations that are well aligned with task performance but fail to capture the underlying conceptual properties 16 /article/10.1007/s10458-025-09691-y ref-CR16 . Thus, an EL might enable successful task completion without truly encoding semantic meaning. Therefore, evaluating these semantic features is essential to evaluate the value and validity of the EL. 6.4.1 Grounding Grounding is essential for the development of meaning and for systematic generalization to novel combinations of concepts 123 /article/10.1007/s10458-025-09691-y ref-CR123 . It forms the basis of human-agent communication 33 /article/10.1007/s10458-025-09691-y ref-CR33 , and without proper grounding, meaningful communication cannot be effectively learned 112 /article/10.1007/s10458-025-09691-y ref-CR112 . However, in general dialog settings, grounding does not emerge naturally without specific regularization techniques 127 /article/10.1007/s10458-025-09691-y ref-CR127 . The grounding problem, which concerns how words acquire semantic meaning, is central to this challenge 184 /article/10.1007/s10458-025-09691-y ref-CR184 . Thus, grounding metrics are vital as they largely define the usability of a language. However, a significant limitation of these metrics is their reliance on some form of oracle or a NL-grounded precursor 15 /article/10.1007/s10458-025-09691-y ref-CR15 , 26 /article/10.1007/s10458-025-09691-y ref-CR26 , 134 /article/10.1007/s10458-025-09691-y ref-CR134 , 207 /article/10.1007/s10458-025-09691-y ref-CR207 . Divergence Havrylov and Titov 15 /article/10.1007/s10458-025-09691-y ref-CR15 proposed a weak form of grounding. Weak grounding means that the same word can correspond to completely different concepts in the induced EL and NL. They used the Kullback–Leibler divergence \ {D {KL}}\ cf. Equation 11 /article/10.1007/s10458-025-09691-y Equ11 of an EL and a NL distribution to ensure that the statistical properties of EL messages resemble those of NL. They introduced this approach as an indirect supervision measure during training but it can also serve as a metric for evaluating the alignment between EL and NL. For a given sample k and the message \ m {\xi {S}}\ produced by the sender, the grounding divergence \ {G {Div}}\ calculation is shown in Eq. 12 /article/10.1007/s10458-025-09691-y Equ12 . Since the true NL distribution \ P {\text {NL}} \left m {\xi {S}} \right \ is inaccessible, a language model is trained to approximate this distribution. The KL divergence yields a value in the range \ \left 0, \infty \right \ , with lower values indicating a closer resemblance between the generated messages and NL. Purity Purity, proposed by Lazaridou et al. 14 /article/10.1007/s10458-025-09691-y ref-CR14 , is a metric used to assess the alignment between predefined semantic categories and those observed in an EL. It measures the effectiveness of a communication system in consistently mapping signals or words to specific concepts 198 /article/10.1007/s10458-025-09691-y ref-CR198 . Thus, purity quantifies the extent to which the clustering of words reflects meaningful and coherent categories, as determined by ground-truth labels. To assess purity, we first form clusters by grouping samples based on the most frequently activated words to describe them. The quality of these clusters is then evaluated using the purity metric, which calculates the proportion of labels in each cluster that match the majority category of that cluster. A higher purity score indicates that the sender is producing words that are semantically aligned with predefined categories, as opposed to arbitrary or agnostic symbol usage, as demonstrated in 14 /article/10.1007/s10458-025-09691-y ref-CR14 . However, this metric requires the existence of predefined ground-truth labels, limiting its applicability in scenarios where such labels are unavailable or ambiguous. Formally, given a set of clusters \ \{ C {k} \}\ where each cluster of samples \ C {k}\ has a corresponding majority ground-truth label \ c {k}\ , the purity of a cluster \ C {k}\ is defined as: Here, \ \{ w \mid w \in C {k} \}\ is the collection of all words used to describe the samples in the cluster and \ \{ w {c} \mid w {c} \in C {k} \wedge w {c} = c {k} \}\ is the collection of words within the cluster that fit the majority label of that cluster. The purity metric ranges from 0 to 1, where a value of 1 indicates perfect alignment with the ground-truth categories. Representational similarity analysis Representational Similarity Analysis RSA emerged in the field of neuroscience and was proposed by Kriegeskorte et al. 251 /article/10.1007/s10458-025-09691-y ref-CR251 . It has since been adapted for the evaluation of the similarity of neural representations across different modalities, including computational models and brain activity patterns. This technique has been effectively applied in EL research 16 /article/10.1007/s10458-025-09691-y ref-CR16 , 114 /article/10.1007/s10458-025-09691-y ref-CR114 , 216 /article/10.1007/s10458-025-09691-y ref-CR216 , where the focus shifts from analyzing neural activity to exploring the structural relationships between different embedding spaces. For example, RSA has been employed to compare the similarity of embedding space structures between input, sender, and receiver in a referential game 16 /article/10.1007/s10458-025-09691-y ref-CR16 , 114 /article/10.1007/s10458-025-09691-y ref-CR114 . By calculating pairwise cosine similarities within these spaces and then computing the Spearman correlation between the resulting similarity vectors, we can calculate an RSA score that measures the global agreement between these spaces, independent of their dimensionality. The agreement of an agent’s embedding space with the input embedding space as such provides an intuitive measure of the grounding of the EL. This approach offers the advantage of being applicable to heterogeneous agents and arbitrary input spaces. In our framework, this corresponds to any ground truth structured embedding \ e \left o {\xi } \right \ of an agent’s observation \ o {\xi }\ and its internal meaning representation \ \varphi {\xi }\ . Nevertheless, a significant limitation is the necessity for an embedding, which provides a structured description of the observation oriented towards a ground truth, for example, based on a NL model. Furthermore, RSA is not directly applicable to the language itself, particularly for discrete languages. Instead, it operates at the level of earlier meaning representations. Despite this, RSA provides valuable insights into whether the EL can be grounded by evaluating the grounding of the meaning space. The methodology of 16 /article/10.1007/s10458-025-09691-y ref-CR16 utilizes a collection K of samples, comprising k observations, images, or feature vectors, to compute representational similarities between input and meaning space. First, we generate input or ground truth embeddings \ e {GT} = e \left o {\xi } \right \ using an appropriate model and generate the corresponding internal representations \ \varphi {\xi }\ from the appropriate architecture part of agent \ \xi\ . Next, we compute pairwise similarities within each embedding space, denoted as \ {S {e}}\ for the ground truth embeddings and \ {S {\varphi }}\ for the agent representations, typically using cosine similarity \ S {cos}\ as defined in Eq. 14 /article/10.1007/s10458-025-09691-y Equ14 . This yields a similarity vector of size \ N \cdot N - 1 \ for each embedding space. The vectors are converted into rank vectors \ {R}\left {S {e}} \right \ and \ {R}\left {S {\varphi }} \right \ . Finally, we calculate the Spearman rank correlation \ \rho\ 252 /article/10.1007/s10458-025-09691-y ref-CR252 between the ranked similarity vectors, using the covariance \ {cov}\ and standard deviation \ \sigma\ , to assess the alignment between the input and agent representation spaces cf. Equation 15 /article/10.1007/s10458-025-09691-y Equ15 . The correlation coefficient \ \rho\ takes on values between \ -1\ and 1. A high absolute value of this coefficient indicates a strong alignment between the two variables. 6.4.2 Compositionality In EL research, achieving compositionality often requires deliberate guidance, as it does not naturally arise without specific interventions 119 /article/10.1007/s10458-025-09691-y ref-CR119 . For instance, training models on diverse tasks and varying environmental configurations can facilitate the development of compositional structures. This occurs as atomic concepts, learned in simpler contexts, are recombined in more complex scenarios 18 /article/10.1007/s10458-025-09691-y ref-CR18 . When a language is truly compositional, its components can be systematically rearranged or substituted with conceptually equivalent components without altering the overall meaning 121 /article/10.1007/s10458-025-09691-y ref-CR121 , 122 /article/10.1007/s10458-025-09691-y ref-CR122 . The formalization of compositionality can be framed using the comprehension \ {\mathcal {L}} {\text {comp}}\ or production \ {\mathcal {L}} {\text {prod}}\ function that map expressions from a language \ {\mathcal {L}}\ to a space of meanings \ \Phi\ or vice versa 138 /article/10.1007/s10458-025-09691-y ref-CR138 . For example, the function \ {\mathcal {L}} {\text {comp}}: {\mathcal {L}} \rightarrow \Phi\ reflects “all the things that the language can denote” 138 /article/10.1007/s10458-025-09691-y ref-CR138 . A language is compositional if these functions act as a homomorphism, e.g., there exist binary operators \ \circ\ on \ {\mathcal {L}} {\text {comp}}\ and \ \times\ on \ \Phi\ such that for any expression composed of two constituents \ m 1\ and \ m 2\ in \ {\mathcal {L}}\ , the following condition holds: Topographic similarity Topographic similarity topsim , originally proposed by Brighton and Kirby 253 /article/10.1007/s10458-025-09691-y ref-CR253 and first applied to EL by Lazaridou et al. 168 /article/10.1007/s10458-025-09691-y ref-CR168 , is a metric designed to quantify the structural alignment between the internal representations of meanings and the corresponding generated messages in a communication system. Unlike RSA cf. Section 6.4.1 /article/10.1007/s10458-025-09691-y Sec31 , which compares the meaning space against a ground truth, topsim focuses on the internal alignment within an agent’s meaning and message spaces. “The intuition behind this measure is that semantically similar objects should have similar messages” 168 /article/10.1007/s10458-025-09691-y ref-CR168 . It has become a widely used metric in the study of EL, as depicted in Fig. 12 /article/10.1007/s10458-025-09691-y Fig12 in Appendix B /article/10.1007/s10458-025-09691-y Sec48 . To compute topsim, we start by sampling k meaning representations denoted by \ \varphi\ , typically embedded feature vectors, from the meaning space \ \Phi\ . Let \ \phi = \{ \varphi {1}, \dots , \varphi {k} \}\ denote the collection of these samples, with \ \varphi \in \Phi\ . Using the sender’s policy \ \pi {\xi {S}}^{M}\ , we generate corresponding messages \ m {i} = \pi {\xi {S}}^{M} \varphi i \ for each sample \ \varphi {i} \in \phi\ . We then compute distances within the meaning and language spaces using suitable distance functions for language \ \Delta {{\mathcal {L}}}\ and meaning \ \Delta {\Phi }\ space. The choice of distance function \ \Delta\ depends on the nature of the spaces involved. For discrete communication, typical choices include Hamming 254 /article/10.1007/s10458-025-09691-y ref-CR254 or Levenshtein 255 /article/10.1007/s10458-025-09691-y ref-CR255 distance, whereas for continuous spaces, cosine or Euclidean distance are often used 51 /article/10.1007/s10458-025-09691-y ref-CR51 . Finally, we compute the Spearman rank correlation \ \rho\ 252 /article/10.1007/s10458-025-09691-y ref-CR252 using the ranked distances to get the topsim value of the language: Positional disentanglement Positional Disentanglement posdis was introduced by Chaabouni et al. 122 /article/10.1007/s10458-025-09691-y ref-CR122 as a metric to evaluate the extent to which words in specific positions within a message uniquely correspond to particular attributes of the input. This metric operates on an order-dependent strategy, which is normalized by the message length and calculated as the ratio of mutual information to entropy. The underlying assumption is that the language leverages positional information to disambiguate words, such that “each position of the message should only be informative about a single attribute” 122 /article/10.1007/s10458-025-09691-y ref-CR122 . Thus, “posdis assumes a message whose length equals the number of attributes in the input object, and where each message token, in a specific position, represents a single attribute” 180 /article/10.1007/s10458-025-09691-y ref-CR180 . This order-dependence is a characteristic feature of NL structures and is essential for the emergence of sophisticated syntactic patterns 122 /article/10.1007/s10458-025-09691-y ref-CR122 . The metric begins by identifying each word \ w p\ at position p in a message m , where f represents the feature vector of the ground truth. The mutual information \ I w p, f i \ between \ w p\ and a specific feature \ f i\ is calculated to determine how informative the position p is about the attribute \ f i\ cf. Equation 18 /article/10.1007/s10458-025-09691-y Equ18 . The two most informative features \ f i^{1}\ and \ f i^{2}\ are then identified based on the mutual information value cf. Equation 19 /article/10.1007/s10458-025-09691-y Equ19 . To quantify positional disentanglement, the mutual information difference between the two most informative features is normalized by the entropy \ H w p \ of the word at position p , as defined in Eq. 20 /article/10.1007/s10458-025-09691-y Equ20 and Eq. 21 /article/10.1007/s10458-025-09691-y Equ21 . Finally, the overall posdis value for a language is calculated by averaging the posdis scores across all positions in the messages within the dataset. For messages of varying lengths, the posdis score is normalized by the average message length \ \overline{|m |}\ , as given in Eq. 22 /article/10.1007/s10458-025-09691-y Equ22 . Bag of symbols disentanglement Bag of Symbols Disentanglement bosdis is a metric introduced by Chaabouni et al. 122 /article/10.1007/s10458-025-09691-y ref-CR122 to assess the degree to which words in a language unambiguously correspond to different input elements, regardless of their position within a message. While positional disentanglement posdis relies on the assumption that positional information is crucial for disambiguating words cf. Section 6.4.2 /article/10.1007/s10458-025-09691-y Sec32 , bosdis relaxes this assumption and captures the intuition behind a permutation-invariant language. In such a language, the order of words is irrelevant, and only the frequency of words carries meaning 122 /article/10.1007/s10458-025-09691-y ref-CR122 . The metric normalizes the mutual information between symbols and input features by the entropy summed over the entire vocabulary. This approach maintains the requirement that each symbol uniquely refers to a distinct meaning, but shifts the focus to symbol counts as the primary informative element. Tree reconstruct error Tree Reconstruct Error TRE assumes prior knowledge of the compositional structure within the input data, enabling the construction of tree-structured derivations 134 /article/10.1007/s10458-025-09691-y ref-CR134 . As defined by Andreas 134 /article/10.1007/s10458-025-09691-y ref-CR134 , a language is considered compositional if it functions as a homomorphism from inputs to their representations. The compositionality of a language should be evaluated by identifying representations that allow an explicitly compositional language to closely approximate the true underlying structure 134 /article/10.1007/s10458-025-09691-y ref-CR134 . One metric for this assessment is TRE, which quantifies the discrepancy between a compositional approximation and the actual structure, using a composition function and a distance metric. A TRE value of zero indicates perfect reproduction of compositionality. The compositional nature of a sender’s language is affirmed if there exists an assignment of representations to predefined primitives e.g., categories, concepts, or words such that for each input, the composition of primitive representations according to the oracle’s derivation precisely reproduces the sender’s prediction 134 /article/10.1007/s10458-025-09691-y ref-CR134 . TRE specifically measures the accuracy with which a given communication protocol can be reconstructed while adhering to the compositional structure of the derivation or embedding of the input \ e \in E\ 51 /article/10.1007/s10458-025-09691-y ref-CR51 . One of the key advantages of the TRE framework is its flexibility across different settings, whether discrete or continuous. It allows for various choices of compositionality functions, distance metrics, and other parameters. However, this flexibility comes with challenges, including the requirement for an oracle-provided ground truth and the necessity of pre-trained continuous embeddings. It is defined in a way that allows the choice of the distance metric \ \delta\ and the compositionality function \ \circ\ to be determined by the evaluator 134 /article/10.1007/s10458-025-09691-y ref-CR134 . When the exact form of the compositionality function is not known a priori, it is common to define \ \circ\ with free parameters, as suggested by Andreas 134 /article/10.1007/s10458-025-09691-y ref-CR134 , treating these parameters as part of the learned model and optimizing them jointly with the other parameters \ \eta\ . However, care must be taken when learning the compositional function to avoid degenerate solutions 134 /article/10.1007/s10458-025-09691-y ref-CR134 . Given a data sample k from the dataset K \ k \in K\ and a corresponding message m from the set of all possible messages M \ m \in M\ , TRE requires a distance function \ \delta\ and learnable parameters \ \eta\ . Additionally, it employs a compositionality function \ \circ\ and pre-trained embeddings of ground truth, denoted by \ e \in E\ , which can be obtained using models like word2vec. The functions involved in the TRE calculation are as follows: - Pre-trained ground truth oracle e.g., word2vec : \ {\mathcal {E}}: K \rightarrow E\ - Learned language speaker: \ \xi S: K \rightarrow M\ - Learnable approximation function for TRE: \ {\widetilde{f}} {\eta }: E \rightarrow M\ In the discrete message setting, which is the focus here, a discrete distance metric such as \ L 1\ is typically chosen, along with a compositional function \ \circ\ defined by a weighted linear combination 51 /article/10.1007/s10458-025-09691-y ref-CR51 , 134 /article/10.1007/s10458-025-09691-y ref-CR134 : To compute the TRE, an optimized approximation function \ {\widetilde{f}} {\eta }\ is required. This function must satisfy two key properties: embedding consistency, meaning that the learned parameters \ \eta\ are specific to an embedding, and compositionality, which ensures that the function behaves according to: The optimization process involves minimizing the distance between the output of the learned language speaker \ \xi {S} k {i} \ and the approximation function \ {\widetilde{f}} {\eta } e {i} \ , based on the ground truth: With the optimized parameters \ \eta ^{ }\ , TRE can be calculated at two levels: the datum level, which assesses individual instances: and the dataset level, which measures the overall communication performance across the dataset: Conflict count Conflict count, introduced by Kuciński et al. 166 /article/10.1007/s10458-025-09691-y ref-CR166 , is designed to quantify the extent to which the assignment of features to words in a language deviates from the word’s principal meaning. This metric is particularly useful in scenarios where the language employs synonyms, as it accounts for the possibility of multiple words referring to the same concept. The conflict count metric operates under the assumption that the number of concepts or features \ f {i}\ given in a feature vector f of a sample k in the collection of samples K is equal to the message length \ |m |\ , and that there exists a one-to-one mapping between a concept \ f i \in f\ and a word \ w \in W\ . The metric counts how frequently this one-to-one mapping is violated, with a value of 0 indicating no conflicts and, therefore, high compositionality. An advantage of this metric is its ability to accommodate redundancy in the language. However, it also has limitations, such as the assumption that the number of features or attributes equals the message length, i.e., \ |f |= |m |\ . Additionally, because conflict count assumes the number of concepts in a derivation to be equal to the message length, it becomes undefined for languages or protocols that violate this assumption, such as those involving negation or context-sensitive constructions presented in 51 /article/10.1007/s10458-025-09691-y ref-CR51 . The primary objective of conflict count is to quantify the number of times the mapping from a word w to its principal meaning \ \varphi w\ is violated. This requires the assumption that a mapping \ \alpha\ exists from the position p of word w in message m to an individual feature in feature vector f , such that: In this framework, the meaning of a word, denoted by \ \varphi {w}\ , is determined by both the word w itself and its position p within the message. This meaning corresponds to a specific instance j of a particular feature i within the feature vector f , such that \ f {i,j} = \varphi w,p \ . The process of calculating the conflict count begins by identifying the principal meaning of each word-position pair: using the count function: where \ m \left k \right \ is the message produced for sample k and \ {pos} {m} \left w \right \ computes the position of word w in message m . Finally, the conflict count value \ {conf}\ is determined by finding the mapping \ \alpha\ that minimizes the score: where the score function is defined as: 6.4.3 Consistency For a language to be effective, the meaning of each word must be consistent across different contexts. Inconsistent word meanings can render a language practically useless, even if the language is semantically grounded and exhibits compositional properties 127 /article/10.1007/s10458-025-09691-y ref-CR127 . In dialogue settings, particularly in the absence of explicit regularization mechanisms, words often fail to maintain consistent groundings across different instances, leading to ambiguity and reduced communicative effectiveness 127 /article/10.1007/s10458-025-09691-y ref-CR127 . Thus, it is crucial to carefully monitor this language characteristic in EL settings. Mutual information Consistency in language can be quantitatively assessed by examining the mutual information between messages and their corresponding input features. Ideally, a consistent language will exhibit a high degree of overlap between messages and features, leading to a high mutual information value, indicating strong correspondence 150 /article/10.1007/s10458-025-09691-y ref-CR150 . Formally, mutual information between two random variables, say X and Y , with joint distribution \ P { X,Y }\ and marginal distributions \ P X\ and \ P Y\ , is defined as the Kullback–Leibler divergence \ {D {KL}}\ see Eq. 11 /article/10.1007/s10458-025-09691-y Equ11 between the joint distribution and the product of the marginals: In the context of discrete communication, where both messages and sample features are represented as discrete variables, the mutual information between the set of messages M and the set of features F is computed using a double summation over all possible message-feature pairs: where \ P { \left M, F \right } \left m, f \right \ is the joint probability of message m and feature f , and \ P {M} \left m \right \ and \ P {F} \left f \right \ are the marginal probabilities of m and f , respectively. Correlation Various studies employ different statistical techniques to measure consistency using correlations 114 /article/10.1007/s10458-025-09691-y ref-CR114 , 116 /article/10.1007/s10458-025-09691-y ref-CR116 , 173 /article/10.1007/s10458-025-09691-y ref-CR173 , 176 /article/10.1007/s10458-025-09691-y ref-CR176 , 184 /article/10.1007/s10458-025-09691-y ref-CR184 , 195 /article/10.1007/s10458-025-09691-y ref-CR195 . For example, consistency within a language system can be quantified by analyzing the variability of words produced for a given sample k . Specifically, given the set of all words representing k , a heatmap is generated using the mean of this set. The sharpness of the heatmap is then quantified by computing the Variance of the Laplacian VoL . The average consistency score is obtained by dividing the VoL of the heatmap by the count of all samples considered, as introduced by Verma and Dhar 195 /article/10.1007/s10458-025-09691-y ref-CR195 . Additionally, Mul et al. 176 /article/10.1007/s10458-025-09691-y ref-CR176 explored the correlation between messages and actions as well as between messages and salient properties of the environment. The analysis reveals correlations by examining the conditional probability distribution of actions given the messages produced by a pretrained or fine-tuned receiver. This distribution, denoted as \ P \left a \mid m \right \ , was visualized using bin bar plots to highlight the prominent correlations 176 /article/10.1007/s10458-025-09691-y ref-CR176 . Similarly, the relationship between input and messages is analyzed by examining the conditional distribution of a pretrained sender’s messages given the observational input, represented as \ P \left m \mid o \right \ 176 /article/10.1007/s10458-025-09691-y ref-CR176 . Coherence Coherence is often assessed through context independence, a metric initially proposed by Bogin et al. 33 /article/10.1007/s10458-025-09691-y ref-CR33 . Context independence examines whether words within a language maintain consistent semantics across varying contexts. However, context independence may be considered restrictive, particularly in languages where synonyms are prevalent 29 /article/10.1007/s10458-025-09691-y ref-CR29 , 164 /article/10.1007/s10458-025-09691-y ref-CR164 . The context independence metric aims to measure the alignment between words \ w \in W\ and features \ f \in F\ of the input samples by analyzing their probabilistic associations. Specifically, \ P \left w \mid f \right \ denotes the probability that a word w is used when a feature f is present, while \ P \left f \mid w \right \ represents the probability that a feature f appears when a word w is used. For each feature f , we identify the word \ w {f}\ most frequently associated with it by maximizing \ P \left f \mid w \right \ : The context independence or coherence metric \ {CI}\ is then computed as the average product of these probabilities across all features: This metric ranges from 0 to 1, with 1 indicating perfect alignment, meaning that each word retains its meaning consistently across different contexts and is thus used coherently. Entropy Entropy metrics are instrumental in analyzing the variability and predictability within linguistic systems. The most fundamental use of entropy involves marginal probabilities, which capture the variability in the number of words in a language 108 /article/10.1007/s10458-025-09691-y ref-CR108 , 114 /article/10.1007/s10458-025-09691-y ref-CR114 . More advanced applications of entropy focus on sender language entropy, which examines the conditional entropy of messages given features and vice versa 32 /article/10.1007/s10458-025-09691-y ref-CR32 , 118 /article/10.1007/s10458-025-09691-y ref-CR118 . Specifically, low conditional entropy \ H \left M \mid F \right \ indicates that a unique message is used for a specific feature, whereas high \ H \left M \mid F \right \ reflects the generation of synonyms for the same feature 118 /article/10.1007/s10458-025-09691-y ref-CR118 . Recent approaches further extend this analysis by combining conditional entropies 37 /article/10.1007/s10458-025-09691-y ref-CR37 , 177 /article/10.1007/s10458-025-09691-y ref-CR177 . For example, \ H \left M \mid F \right \ quantifies the uncertainty remaining about messages after knowing the concepts, while \ H \left F \mid M \right \ measures the uncertainty about concepts given the messages. A negative correlation between these measures and agent performance is expected 177 /article/10.1007/s10458-025-09691-y ref-CR177 . However, a notable limitation of these entropy-based methods is that they focus on complete messages rather than individual words, which can limit the evaluation of more complex languages. For example, Ohmer et al. 177 /article/10.1007/s10458-025-09691-y ref-CR177 provide the following comprehensive evaluation approach. First, the conditional entropy of messages given features \ H \left M \mid F \right \ , see Eq. 41 /article/10.1007/s10458-025-09691-y Equ41 , and \ H \left F \mid M \right \ are calculated. Additionally, the marginal entropies are calculated using Eq. 42 /article/10.1007/s10458-025-09691-y Equ42 , where X represents either messages M or features F . Using these entropies, \ {consistency}\ , see Eq. 43 /article/10.1007/s10458-025-09691-y Equ43 , measures how much uncertainty about the message is reduced when the feature is known, with lower values indicating more consistent message usage. \ {effectiveness}\ , on the other hand, see Eq. 44 /article/10.1007/s10458-025-09691-y Equ44 , evaluates the reduction in uncertainty about the feature when the message is known, with lower values reflecting more unique messages for individual features. Finally, the normalized mutual information \ {NI}\ provides a combined score: A high \ {NI}\ score indicates a strong predictive relationship between messages and features, reflecting high consistency. Similarity The Jaccard similarity coefficient is a another metric for evaluating the consistency of language usage among agents 72 /article/10.1007/s10458-025-09691-y ref-CR72 , 116 /article/10.1007/s10458-025-09691-y ref-CR116 . It quantifies the similarity between two sets by comparing the size of their intersection to the size of their union 116 /article/10.1007/s10458-025-09691-y ref-CR116 . To measure language consistency, the Jaccard similarity is computed by sampling messages for each input and averaging the similarity scores across the population 116 /article/10.1007/s10458-025-09691-y ref-CR116 . This approach reflects how consistently words are used across different messages. Specifically, Jaccard similarity \ J M {\xi {i}}, M {\xi {j}} \ is defined in Eq. 46 /article/10.1007/s10458-025-09691-y Equ46 , where \ M {\xi {i}}\ and \ M {\xi {j}}\ represent sets of messages generated by different agents based on the same input. The similarity ranges from 0 to 1, with 1 indicating complete overlap and thus perfect similarity. In practice, Jaccard similarity helps to assess the coherence of languages emerging from agent-based systems. For instance, in referential game experiments, high perplexity cf. Section 6.2.2 /article/10.1007/s10458-025-09691-y Sec26 and low Jaccard similarity have been observed, suggesting that agents assign unique but incoherent strings to object types to gain an advantage in the game without producing a consistent language 72 /article/10.1007/s10458-025-09691-y ref-CR72 . However, Jaccard similarity is only applicable to scenarios where multiple agents generate messages about the same set of objects. Thus, its application is limited to cases where the goal is to compare the overlap of message sets between agents attempting to convey similar meanings. 6.4.4 Generalization A language’s ability to generalize is crucial for describing objects and concepts at different levels of complexity, allowing for effective clustering and hierarchical representation. Generalization in ELs reflects their ability to extend beyond specific training instances to novel situations. “If the emergent languages can be generalised, we then could say that these languages do capture the structure of meaning spaces” 155 /article/10.1007/s10458-025-09691-y ref-CR155 . Research shows that languages capable of generalization tend to emerge only when the input is sufficiently varied 122 /article/10.1007/s10458-025-09691-y ref-CR122 . In contrast, a large dictionary size often indicates a lack of generalization 108 /article/10.1007/s10458-025-09691-y ref-CR108 . Human languages have evolved under the pressure of a highly complex environment, fostering their generalization capabilities 122 /article/10.1007/s10458-025-09691-y ref-CR122 . However, deep learning models often exploit dataset-specific regularities rather than developing systematic solutions 46 /article/10.1007/s10458-025-09691-y ref-CR46 . To address this, much research is being done on the systematic generalization abilities of ELs. Zero shot evaluation Zero-shot evaluation, which assesses the ability of an agent to generalize to novel stimuli 72 /article/10.1007/s10458-025-09691-y ref-CR72 , 168 /article/10.1007/s10458-025-09691-y ref-CR168 , has become a standard metric in the study of EL as illustrated in Fig. 12 /article/10.1007/s10458-025-09691-y Fig12 in Appendix B /article/10.1007/s10458-025-09691-y Sec48 . This evaluation is critical to understand the generalization capabilities of an agent. Zero-shot evaluation can be done in two different scenarios, one with unseen input and the other with an unseen partner. In the unseen input scenario, models are tested on a zero-shot test set consisting of samples with feature combinations not encountered during training. Performance, such as accuracy, is reported for these unseen samples 114 /article/10.1007/s10458-025-09691-y ref-CR114 , 122 /article/10.1007/s10458-025-09691-y ref-CR122 , 127 /article/10.1007/s10458-025-09691-y ref-CR127 , 164 /article/10.1007/s10458-025-09691-y ref-CR164 , 168 /article/10.1007/s10458-025-09691-y ref-CR168 . Different methods for constructing novel inputs include exposing models to objects that resemble training data but have unseen properties or entirely novel combinations of features 168 /article/10.1007/s10458-025-09691-y ref-CR168 . Moreover, a more drastic approach may involve moving to entirely new input scenarios, such as testing the ability of agents to generalize across different game types 37 /article/10.1007/s10458-025-09691-y ref-CR37 . The unseen partner scenario, also known as cross-play or zero-shot coordination, evaluates models by pairing agents that did not communicate during training. Again, performance is measured, typically in terms of accuracy 211 /article/10.1007/s10458-025-09691-y ref-CR211 , 256 /article/10.1007/s10458-025-09691-y ref-CR256 . However, these approaches also have drawbacks. The unseen input scenario requires a ground truth oracle to withhold feature combinations, which is necessary to accurately define novel combinations. Meanwhile, the unseen partner setup can introduce inefficiencies by requiring additional resources to train novel communication partners for testing. Ease and transfer learning Ease and Transfer Learning ETL , as proposed by Chaabouni et al. 25 /article/10.1007/s10458-025-09691-y ref-CR25 , evaluates how easily new listeners can adapt to an EL on distinct tasks. ETL extends the concept of ease-of-teaching 106 /article/10.1007/s10458-025-09691-y ref-CR106 by assessing how effectively a deterministic language, developed by a fixed set of speakers, can be transferred to new listeners who are trained on tasks different from the original one for which the language was optimized 25 /article/10.1007/s10458-025-09691-y ref-CR25 . This metric not only gauges the language’s generality but also its transferability across tasks 25 /article/10.1007/s10458-025-09691-y ref-CR25 . To measure ETL, after convergence, a fixed number of speakers produce a deterministic language by selecting symbols using an \ \mathop {\textrm{argmax}}\limits\ operation over their distributions. This language is then used to train newly initialized listeners on a new task. The training curve is tracked to observe how quickly and accurately the listeners learn the task, which may involve more challenging objectives than the former training tasks 25 /article/10.1007/s10458-025-09691-y ref-CR25 , 154 /article/10.1007/s10458-025-09691-y ref-CR154 . 6.5 Pragmatics Pragmatics is a critical aspect of language that examines how context influences meaning 199 /article/10.1007/s10458-025-09691-y ref-CR199 . It goes beyond the literal interpretation of words and requires the listener to infer the speaker’s intentions, beliefs, and mental states, an ability known as Theory of Mind ToM 199 /article/10.1007/s10458-025-09691-y ref-CR199 . In human interactions, this contextual reasoning is essential for predicting and understanding behavior. In the context of EL, pragmatics focuses on how effectively agents use the communication ability in their environment. Empirical studies have shown that agents may initially fail to use communication meaningfully, but, once they do communicate, they can reach a locally optimal solution to the communication problem 230 /article/10.1007/s10458-025-09691-y ref-CR230 . Thus, evaluating the pragmatics of EL is essential to determining its utility and effectiveness in real-world applications. 6.5.1 Predictability Predictability evaluates the complexity of an environment and its effect on the need for communication. Thus, it is a central metric for the probability of emergence and the use of EL. In simple environments with limited actions, agents can often coordinate without communication 110 /article/10.1007/s10458-025-09691-y ref-CR110 . Behavioral divergence Behavioral divergence, introduced by Dubova et al. 110 /article/10.1007/s10458-025-09691-y ref-CR110 , posits that less diversity in actions or messages correlates with more predictable behavior, potentially reducing the need for communication. To quantify this, we calculate Behavioral Action Predictability \ {BAP}\ and Behavioral Message Predictability \ {BMP}\ . Both use the Jensen-Shannon Divergence JSD see Eq. 47 /article/10.1007/s10458-025-09691-y Equ47 which itself uses the Kullback–Leibler Divergence \ {D {KL}}\ cf. Equation 11 /article/10.1007/s10458-025-09691-y Equ11 . BAP see Eq. 48 /article/10.1007/s10458-025-09691-y Equ48 and BMP see Eq. 49 /article/10.1007/s10458-025-09691-y Equ49 both use a uniform distribution Q for comparison. BAP further uses the distribution of actions by the agent \ P \left a {\xi } \right \ while BMP uses the distribution of messages by the agent \ P \left m {\xi } \right \ . Based on that, these metrics provide a robust measure of how predictable agent behaviors and messages are, with higher values indicating less predictability and greater need for beneficial communication 110 /article/10.1007/s10458-025-09691-y ref-CR110 . 6.5.2 Efficiency In EL settings, efficient communication arises only when there is an opportunity cost 115 /article/10.1007/s10458-025-09691-y ref-CR115 . Without such a cost, there is no drive towards brevity, which limits the effectiveness and efficiency of EL in HCI. Sparsity Sparsity, as proposed by Kalinowska et al. 115 /article/10.1007/s10458-025-09691-y ref-CR115 , measures the extent to which agents minimize their communication during task execution. This metric requires only the collection of messages exchanged per episode for computation. However, its applicability is limited to scenarios where communication is not strictly necessary for task completion, i.e., agents have the option to send no messages at all or to send messages that contain no meaningful information. A sparsity value of 0 indicates that an agent can solve the task using only a single message throughout an episode, reflecting a highly efficient communication strategy. Conversely, higher sparsity values indicate more frequent or verbose communication, which may indicate inefficiencies in the EL. Communication sparsity \ {ComSpar}\ is mathematically defined as: In this equation, \ M {\text {ep}, i}\ represents the set of all messages exchanged during episode i , and \ n {\text {ep}}\ is the total number of episodes observed. The collection \ \left\{ m \mid m \in M {\text {ep}, i} \wedge m \ne 0 \right\}\ consists of all messages m of episode i that are non-zero and thus contributing. 6.5.3 Positive signaling Positive signaling evaluates the alignment between an agent’s observations and its communication output 29 /article/10.1007/s10458-025-09691-y ref-CR29 . The goal is to ensure that the outgoing transmitted information is both relevant and observable by the agent 181 /article/10.1007/s10458-025-09691-y ref-CR181 . Speaker consistency Speaker Consistency SC , introduced by Jaques et al. 240 /article/10.1007/s10458-025-09691-y ref-CR240 , measures how effectively an agent’s messages reflect its state or trajectory, thereby ensuring the communication is meaningful. This is quantified using mutual information. For an agent \ \xi i\ , the trajectory \ \tau {\xi i}^t\ represents the sequence of states and actions up to time step t . The message produced at time t is denoted by \ m {\xi i}^t\ . The mutual information \ I m {\xi i}^t, \tau {\xi i}^t \ between the message and trajectory is calculated as: Here, \ H m {\xi i}^t \ is the entropy of the message distribution, \ H m {\xi i}^t | \tau {\xi i}^t \ is the conditional entropy given the trajectory, \ \overline{P {\xi i}} m \ as marginal distribution of message m over all trajectories, and \ P {\xi i} m|\tau {\xi i}^t \ as conditional distribution of message m given the trajectory \ \tau {\xi i}^t\ . This way, the mutual information value reflects how much information the message carries about the agent’s trajectory. Lowe et al. 29 /article/10.1007/s10458-025-09691-y ref-CR29 built on this concept and provided the following formula for Speaker Consistency \ {SC}\ : In this equation, \ P \left a, m \right \ is the joint probability of action a and message m , calculated empirically by averaging their co-occurrences across episodes. In general, SC is a valuable metric for evaluating whether the EL is both informative and aligned with the behavioral patterns of the sender. 6.5.4 Positive listening Positive listening evaluates the effectiveness of how a message receiver utilizes and applies incoming information 29 /article/10.1007/s10458-025-09691-y ref-CR29 . However, agents should not simply process messages similarly to other observations to avoid treating them as mere directives 146 /article/10.1007/s10458-025-09691-y ref-CR146 . Nevertheless, the metrics presented in this section focus on evaluating the receiver’s ability to effectively integrate and use the information received, rather than evaluating the receiver’s ability to do more than just follow instructions. Instantaneous coordination Instantaneous Coordination IC , also referred to as listener consistency 29 /article/10.1007/s10458-025-09691-y ref-CR29 , was introduced by Jaques et al. 240 /article/10.1007/s10458-025-09691-y ref-CR240 as a metric to evaluate how effectively an agent’s message influences another agent’s subsequent action. IC is computed similarly to Speaker Consistency cf. Section 6.5.3 /article/10.1007/s10458-025-09691-y Sec38 , but differs in that it measures the mutual information between one agent’s message and the other agent’s next action, averaged over episodes. This metric directly captures the receiver’s immediate reaction to an incoming message, making it a measure of positive listening. However, it primarily captures situations where the receiver’s action is directly changed by the sender’s message, without considering the broader context or long-term dependencies 29 /article/10.1007/s10458-025-09691-y ref-CR29 . Accordingly, “IC can miss many positive listening relationships” 29 /article/10.1007/s10458-025-09691-y ref-CR29 . Jaques et al. 240 /article/10.1007/s10458-025-09691-y ref-CR240 proposed two specific measures for IC: One that quantifies the mutual information between the sender’s message and the receiver’s next action see Eq. 53 /article/10.1007/s10458-025-09691-y Equ53 , and another one that measures the mutual information between the sender’s current action and the receiver’s next action see Eq. 54 /article/10.1007/s10458-025-09691-y Equ54 . These measures are calculated by averaging over all trajectory steps and taking the maximum value between any two agents, focusing on short-term dependencies between consecutive timesteps. A unified equation for \ {IC}\ is provided by Lowe et al. 29 /article/10.1007/s10458-025-09691-y ref-CR29 : Here, \ P \left a {\xi {R}}^{t+1}, m {\xi {S}}^{t} \right \ is the empirical joint probability of the sender’s message and the receiver’s subsequent action, averaged over episodes within each epoch. Message effect The Message Effect \ {ME}\ metric, introduced by Bouchacourt and Baroni 202 /article/10.1007/s10458-025-09691-y ref-CR202 , quantifies the influence of a message sent by one agent on the subsequent actions and messages of another agent. This metric explicitly considers bidirectional communication, so in the following we use generic agents \ \xi {A}\ and \ \xi {B}\ instead of sender and receiver. A notable challenge of this metric is the requirement for counterfactual analysis. Given an agent \ \xi {A}\ at timestep t sending a message \ m {\xi {A}}^t\ , we define \ z {\xi {B}}^{t+1}\ as the combination of the action and message produced by agent \ \xi {B}\ at the following timestep. Accordingly, the conditional distribution \ P \left z {\xi {B}}^{t+1} \mid m {\xi A}^t \right \ represents the response of \ \xi {B}\ to the message from \ \xi {A}\ . To account for counterfactuals, which encode what might have happened had \ \xi {A}\ sent a different message \ {\widetilde{m}} {\xi {A}}^t\ , we define the counterfactual distribution \ {\widetilde{P}} \left z {\xi B}^{t+1} \right \ see Eq. 56 /article/10.1007/s10458-025-09691-y Equ56 . The \ {ME}\ is then measured by the Kullback–Leibler divergence between the actual response and the counterfactual response see Eq. 57 /article/10.1007/s10458-025-09691-y Equ57 . The computation involves sampling \ z {\xi {B}}^{t+1, k}\ from the conditional distribution for the actual message and sampling counterfactuals \ {\widetilde{m}} {\xi {A}}^t\ to estimate \ {\widetilde{P}} \left z {\xi {B}}^{t+1, k} \right \ see Eq. 58 /article/10.1007/s10458-025-09691-y Equ58 . The final ME is calculated as the average KL divergence over the collection of samples K see Eq. 59 /article/10.1007/s10458-025-09691-y Equ59 . Causal influence of communication The Causal Influence of Communication CIC metric, introduced independently by Jaques et al. 240 /article/10.1007/s10458-025-09691-y ref-CR240 and Lowe et al. 29 /article/10.1007/s10458-025-09691-y ref-CR29 , provides a direct measure of positive listening by quantifying the causal effect that one agent’s message has on another agent’s behavior. Traditional methods of evaluating communication often fall short, as simply testing for a decrease in reward after removing the communication channel does not adequately capture the utility of communication 29 /article/10.1007/s10458-025-09691-y ref-CR29 . CIC is computed using the mutual information between an agent’s message and the subsequent action of the receiving agent. Unlike Instantaneous Coordination cf. Section 6.5.4 /article/10.1007/s10458-025-09691-y Sec39 , CIC considers the probabilities \ P \left a, m \right = \pi {\xi R} \left a \mid m \right \pi {\xi S} \left m \right \ that represent changes in the action distribution of the receiver \ \xi R\ when the message m from the sender \ \xi S\ is altered. These probabilities are normalized within each game to accurately reflect the influence of messages on actions within the same context 29 /article/10.1007/s10458-025-09691-y ref-CR29 . For multi-time-step causal influence, the CIC metric is defined as the difference between the entropy of the receiver’s actions with and without communication: Here, \ \tau {\xi R}\ denotes the standard trajectory of the receiver, comprising state-action pairs, while \ \tau {\xi R}^{+M}\ includes the communicated messages. The CIC is estimated by learning an approximate policy function \ \pi \cdot | \tau {\xi R} \ . For more details on the multistep version, refer to Eccles et al. 230 /article/10.1007/s10458-025-09691-y ref-CR230 , and for the single-step version, see Jaques et al. 240 /article/10.1007/s10458-025-09691-y ref-CR240 . 6.5.5 Symmetry Symmetry in EL refers to consistent language use across agents in settings, where agents alternate between roles such as message sender and receiver 105 /article/10.1007/s10458-025-09691-y ref-CR105 , 110 /article/10.1007/s10458-025-09691-y ref-CR110 . Thus, symmetry ensures convergence to a common language rather than distinct dialects 110 /article/10.1007/s10458-025-09691-y ref-CR110 . Inter-agent divergence Inter-Agent Divergence IAD , introduced by Dubova et al. 110 /article/10.1007/s10458-025-09691-y ref-CR110 , 257 /article/10.1007/s10458-025-09691-y ref-CR257 , quantifies the similarity in how different agents map messages to actions. Let \ a {\xi i}\ denote the action of agent \ \xi i\ . The first step involves computing the marginal action distributions for each agent given a message m , represented as \ P a {\xi i}|m \ . The divergence between two agents, \ \xi i\ and \ \xi j\ , based on their responses to the same message, is then calculated using the Jensen-Shannon Divergence JSD as follows: Finally, the overall IAD is computed by averaging these divergences across all possible agent pairs \ \left \xi i, \xi j \right \in \xi {\text {comb}}\ and messages \ m \in M\ : While IAD effectively captures the consistency of inter-agent communication, it may have limitations when applied to more complex languages where message-level comparisons become difficult. Within-agent divergence Within-Agent Divergence WAD , proposed by Dubova et al. 110 /article/10.1007/s10458-025-09691-y ref-CR110 , 257 /article/10.1007/s10458-025-09691-y ref-CR257 , measures the consistency of an agent’s communication behavior when it changes roles, such as from sender to receiver. This metric captures the internal symmetry in an agent’s behavior and is crucial in complex systems where agents can assume different roles within the same environment. To compute WAD, we again first consider the action distribution \ P \left a {\xi i} \mid m \right \ for each agent \ \xi i\ over a set of messages \ m \in M\ . This distribution reflects how an agent’s actions are conditioned on receiving or sending a specific message. Given this, the Jensen-Shannon Divergence JSD is used to assess the divergence between an agent’s behavior when acting as a sender \ \xi {i,S}\ versus as a receiver \ \xi {i,R}\ : Finally, the overall WAD is computed by averaging this divergence across all agents \ \xi i \in \xi\ based on the WAD for individual agents and their messages \ m \in M {\xi i}\ : 6.6 Summary of the metrics While some EL features are quantifiable by multiple metrics and have been investigated in multiple studies, others remain underexplored, as illustrated in Fig. 12 /article/10.1007/s10458-025-09691-y Fig12 in Appendix B /article/10.1007/s10458-025-09691-y Sec48 . Metrics such as topographic similarity and zero shot evaluation , both of which assess semantic properties, are well established and widely utilized across multiple studies. In contrast, metrics related to pragmatics, such as speaker consistency and instantaneous coordination , are fairly well established but are less frequently used. Morphology metrics, particularly active words and average message length , are more commonly used, whereas syntax remains a peripheral concern, with only two isolated metrics proposed and not adopted in subsequent research. This imbalance indicates that while semantic metrics dominate EL research, morphology and pragmatics receive moderate attention, and syntax is mostly neglected. Furthermore, the optimality of these metrics is not straightforward. Rather than being simply minimized or maximized, their ideal values are likely to lie at a nuanced balance point that varies depending on the specific EL system and application. This uncertainty leaves the critical question of what constitutes a ‘good’ EL system largely unanswered. Addressing this gap will require a deeper exploration of underrepresented metrics and a more refined understanding of how to evaluate EL systems holistically. 7 Future work In this section, we outline potential future directions for the research field of EL, based on our vision outlined in Sect. 7.1 /article/10.1007/s10458-025-09691-y Sec43 . We present major research opportunities, organized along key research dimensions, in Sect. 7.2 /article/10.1007/s10458-025-09691-y Sec44 . Along with future research directions, we have summarized a list of open source code repositories in Table 10 /article/10.1007/s10458-025-09691-y Tab10 in Appendix A /article/10.1007/s10458-025-09691-y Sec47 that can serve as convenient starting points for experimenting with these directions, for example, comprehensive frameworks such as the EGG toolkit 162 /article/10.1007/s10458-025-09691-y ref-CR162 and BabyAI 131 /article/10.1007/s10458-025-09691-y ref-CR131 are included. 7.1 Vision Our vision for EL research is grounded in a functional perspective, aiming to achieve significant breakthroughs in human-agent interaction 24 /article/10.1007/s10458-025-09691-y ref-CR24 , 26 /article/10.1007/s10458-025-09691-y ref-CR26 , 29 /article/10.1007/s10458-025-09691-y ref-CR29 , 33 /article/10.1007/s10458-025-09691-y ref-CR33 , 35 /article/10.1007/s10458-025-09691-y ref-CR35 , 176 /article/10.1007/s10458-025-09691-y ref-CR176 , 256 /article/10.1007/s10458-025-09691-y ref-CR256 . This means developing communication systems that enable HCI at the human level, addressing the purpose, cost, and value of communication with intuitive and effective interfaces 27 /article/10.1007/s10458-025-09691-y ref-CR27 , 36 /article/10.1007/s10458-025-09691-y ref-CR36 , 58 /article/10.1007/s10458-025-09691-y ref-CR58 , 148 /article/10.1007/s10458-025-09691-y ref-CR148 , 189 /article/10.1007/s10458-025-09691-y ref-CR189 , 233 /article/10.1007/s10458-025-09691-y ref-CR233 . A key goal is to ensure that ELs are grounded in real-world contexts, allowing agents to understand and interact with human-like comprehension and vice versa 19 /article/10.1007/s10458-025-09691-y ref-CR19 , 21 /article/10.1007/s10458-025-09691-y ref-CR21 , 40 /article/10.1007/s10458-025-09691-y ref-CR40 , 258 /article/10.1007/s10458-025-09691-y ref-CR258 . This includes creating hierarchical, compositional conceptualization capabilities that allow agents to discuss and understand novel concepts in a structured, human-relevant manner 25 /article/10.1007/s10458-025-09691-y ref-CR25 , 133 /article/10.1007/s10458-025-09691-y ref-CR133 , 182 /article/10.1007/s10458-025-09691-y ref-CR182 , 187 /article/10.1007/s10458-025-09691-y ref-CR187 . In addition, exploring the potential for AI explainability through communication is an exciting area 21 /article/10.1007/s10458-025-09691-y ref-CR21 , 127 /article/10.1007/s10458-025-09691-y ref-CR127 , 189 /article/10.1007/s10458-025-09691-y ref-CR189 . Finally, in the long term, creation and creativity through EL comparable to human capabilities would be a milestone. This would allow agents to truly communicate on a human level and enhance their ability to perceive and adapt to their environment through the use of language 103 /article/10.1007/s10458-025-09691-y ref-CR103 . 7.2 Dimensions and opportunities The development, evaluation, and application of EL in communication systems can be systematically analyzed along several critical dimensions. Given the relative youth of the field, with the majority of research emerging within the last eight years, specific areas of focus have gained prominence, particularly in the context of semantic metrics, such as topographic similarity and zero-shot evaluation, as highlighted in Fig. 12 /article/10.1007/s10458-025-09691-y Fig12 in Appendix B /article/10.1007/s10458-025-09691-y Sec48 . However, Fig. 13 /article/10.1007/s10458-025-09691-y Fig13 in Appendix B /article/10.1007/s10458-025-09691-y Sec48 illustrates that there is no discernible chronological trend or evolution in the way the different language characteristics are addressed. This absence of a historical trajectory is likely attributable to the relatively brief history of the field and the considerable diversity of proposed approaches and methodologies. Despite this, we identified nine key dimensions that, to the best of our knowledge, represent the primary areas of focus in EL research. 7.2.1 Evaluation metrics Evaluation metrics are essential for rigorously assessing the characteristics and effectiveness of ELs. As detailed in our taxonomy cf. Section 5.4 /article/10.1007/s10458-025-09691-y Sec12 , we have identified key characteristics and their associated metrics. While some EL features are quantifiable through multiple metrics and have been examined in multiple studies, others remain underexplored, as illustrated in Fig. 12 /article/10.1007/s10458-025-09691-y Fig12 in Appendix B /article/10.1007/s10458-025-09691-y Sec48 . We emphasize the need to develop comprehensive and quantitative metrics that accurately capture these features, which are critical to determining the practical utility of ELs. Previous studies have similarly highlighted this need 25 /article/10.1007/s10458-025-09691-y ref-CR25 , 29 /article/10.1007/s10458-025-09691-y ref-CR29 , 46 /article/10.1007/s10458-025-09691-y ref-CR46 , 72 /article/10.1007/s10458-025-09691-y ref-CR72 , 109 /article/10.1007/s10458-025-09691-y ref-CR109 , 126 /article/10.1007/s10458-025-09691-y ref-CR126 , 179 /article/10.1007/s10458-025-09691-y ref-CR179 , 234 /article/10.1007/s10458-025-09691-y ref-CR234 , 241 /article/10.1007/s10458-025-09691-y ref-CR241 . In addition, further research is needed to systematically investigate existing metrics, especially with respect to their sensitivity to variations in settings, algorithms, and agent architectures 29 /article/10.1007/s10458-025-09691-y ref-CR29 , 50 /article/10.1007/s10458-025-09691-y ref-CR50 . It is imperative that these metrics be subjected to more rigorous investigation to ensure that they enable meaningful quantitative comparisons and support well-founded conclusions about the capabilities and utility of ELs. Thus, we endorse more comprehensive studies, more edge case testing and, in particular, more analysis of actual human-agent interaction. We see this as a critical priority for advancing the field. 7.2.2 Emergent language and natural language alignment This dimension addresses the convergence and divergence between EL and NL. A key approach to this challenge, discussed in Sect. 5.3 /article/10.1007/s10458-025-09691-y Sec11 , involves leveraging language priors to guide this alignment. Achieving robust EL-NL alignment is essential for advancing human-agent interaction. Thus, future research should explore the integration of NL-centered metrics and regularization techniques to enhance this alignment 14 /article/10.1007/s10458-025-09691-y ref-CR14 , 107 /article/10.1007/s10458-025-09691-y ref-CR107 . However, this alignment presents a fundamental dilemma. On the one hand, agents need the autonomy to develop languages organically, tailored to their specific interactions and requirements. On the other hand, to facilitate seamless human-agent communication, these ELs must closely resemble NLs, which imposes significant constraints on their development. This tension creates what we call the Evolution-Acquisition Dilemma, where the evolutionary process fosters intrinsically motivated language emergence, while the acquisition process necessitates alignment with NL. Balancing these competing needs is a critical challenge for future research in this area. 7.2.3 Emergent language and large language models The remarkable performance of LLM on various benchmarks has established them as a cornerstone of modern NLP 80 /article/10.1007/s10458-025-09691-y ref-CR80 and potential foundation for more complex agents 259 ref-CR259 , 260 ref-CR260 , 261 ref-CR261 , 262 /article/10.1007/s10458-025-09691-y ref-CR262 . Despite their success, however, LLM face fundamental limitations, particularly in grounding language use in shared environments and experiences 263 /article/10.1007/s10458-025-09691-y ref-CR263 as well as agency 264 /article/10.1007/s10458-025-09691-y ref-CR264 and truthfulness 265 /article/10.1007/s10458-025-09691-y ref-CR265 . Addressing these shortcomings may require insights from EL research. A key challenge in EL is the evolution-acquisition dilemma - the need to ground language in shared, incremental experiences, which current learning systems cannot achieve due to resource and technology constraints. While most LLM applications rely on fine-tuning 266 /article/10.1007/s10458-025-09691-y ref-CR266 and scaling 85 /article/10.1007/s10458-025-09691-y ref-CR85 , these methods do not inherently address this challenge or the broader issues of grounding and adaptability. One promising avenue lies in the concept of agentic LLM 263 /article/10.1007/s10458-025-09691-y ref-CR263 , 267 /article/10.1007/s10458-025-09691-y ref-CR267 , 268 /article/10.1007/s10458-025-09691-y ref-CR268 or cognitive language agents 269 /article/10.1007/s10458-025-09691-y ref-CR269 , which combine the representational strength of LLMs with the adaptive, experiential learning processes of RL. Here, looking at opportunities for EL related research, LLM might act as language priors, providing a foundation that can be iteratively refined through agentic interaction and experience 263 /article/10.1007/s10458-025-09691-y ref-CR263 , 270 /article/10.1007/s10458-025-09691-y ref-CR270 . This approach mirrors human language acquisition, where teachers provide guidance based on shared experience. In the absence of such a teacher, agentic LLM provide a synthetic framework for combining supervised and EL paradigms, allowing agents to relax static supervised training regimes and develop more adaptive communication protocols. We argue for further exploration of cognitive language agents, focusing not only on established benchmarks but also on challenges central to EL research. Bridging these fields could open up new opportunities for developing systems that combine the scalability of LLM with the adaptability and grounding capabilities of EL. 7.2.4 Representation learning EL can be viewed as a complex representation learning task, focusing on how agents encode, interpret, and construct internal representations of observations and linguistic data. While representation learning is a well-established area in artificial intelligence research, its application in the context of EL remains underexplored. This dimension is central to the analysis of meaning and language space as outlined in our framework, which is based on the semiotic cycle cf. Figure 10 /article/10.1007/s10458-025-09691-y Fig10 . Advancing this dimension requires advanced latent space analyses to elucidate the relationships between ELs, underlying world models, and NL structures. In addition, evaluating the impact of discrete versus continuous representations is critical to refining our understanding of EL dynamics. Future research directions include developing methodologies to ensure that agent representations more accurately reflect the input they receive 16 /article/10.1007/s10458-025-09691-y ref-CR16 , exploring efficient representation of multimodal information 124 /article/10.1007/s10458-025-09691-y ref-CR124 , conducting in-depth analyses to uncover and mitigate influencing factors and biases in learned representations 46 /article/10.1007/s10458-025-09691-y ref-CR46 , and assessing the efficacy of these representations for downstream tasks 111 /article/10.1007/s10458-025-09691-y ref-CR111 . 7.2.5 Agent design Agent design is a critical aspect in EL research, directly influencing the linguistic capabilities and adaptability of artificial agents. Prominent research directions include the investigation of advanced neural network architectures tailored for EL 25 /article/10.1007/s10458-025-09691-y ref-CR25 , 170 /article/10.1007/s10458-025-09691-y ref-CR170 , 186 /article/10.1007/s10458-025-09691-y ref-CR186 , the creation of architectures optimized for heterogeneous and dynamic agent populations, and the refinement of structures that enhance language emergence and linguistic properties 111 /article/10.1007/s10458-025-09691-y ref-CR111 , 167 /article/10.1007/s10458-025-09691-y ref-CR167 . In addition, modular designs rather than monolithic ones potentially offer advantages by separating language processing from other task-specific computations. Addressing these design challenges is critical to advancing both EL research and broader artificial intelligence goals. 7.2.6 Setting design The environment in which agents operate is central to shaping the EL, encompassing interaction rules, agent goals, and communication dynamics cf. Table 3 /article/10.1007/s10458-025-09691-y Tab3 . This dimension is integral to the setting space outlined in our framework cf. Figure 10 /article/10.1007/s10458-025-09691-y Fig10 . Important future research directions include scaling up experimental settings to include larger and more complex tasks 14 /article/10.1007/s10458-025-09691-y ref-CR14 , 25 /article/10.1007/s10458-025-09691-y ref-CR25 , 130 /article/10.1007/s10458-025-09691-y ref-CR130 , 146 /article/10.1007/s10458-025-09691-y ref-CR146 , 202 /article/10.1007/s10458-025-09691-y ref-CR202 , 230 /article/10.1007/s10458-025-09691-y ref-CR230 with a focus on realistic perceptually grounded game environments 110 /article/10.1007/s10458-025-09691-y ref-CR110 , 202 /article/10.1007/s10458-025-09691-y ref-CR202 . In addition, the study of the impact of populations as such 179 /article/10.1007/s10458-025-09691-y ref-CR179 and the use of heterogeneous agent populations 118 /article/10.1007/s10458-025-09691-y ref-CR118 are crucial areas of research. While some benchmarks have been established and utilized 131 /article/10.1007/s10458-025-09691-y ref-CR131 , 162 /article/10.1007/s10458-025-09691-y ref-CR162 , there remains a significant need for the development and widespread dissemination of comprehensive benchmarks in area of research. 7.2.7 Communication design The design of the communication channel in EL systems is critical, focusing on how agents exchange and structure information through the channels available to them. This aspect is directly related to the phonetics and phonology components outlined in our taxonomy cf. Section 5.4.1 /article/10.1007/s10458-025-09691-y Sec13 and Sect. 5.4.2 /article/10.1007/s10458-025-09691-y Sec14 . For discrete ELs, it is essential to establish channels that support word-based communication, with considerations such as vocabulary size and variable message length being fundamental to enabling effective and scalable human-agent interaction. Future research directions in this area include the exploration of topology-aware variable communication channels, the integration of heterogeneous channels within multi-agent systems, and the evolution of communication channels over time. Moreover, the incorporation of multimodal communication channels could provide more realistic and contextually rich stimuli, which may significantly enhance the sophistication and applicability of ELs in NL-oriented human-agent coordination 25 /article/10.1007/s10458-025-09691-y ref-CR25 . 7.2.8 Learning strategies Learning strategies focus on how agents acquire, adapt, and refine their linguistic capabilities over time, including the development of language rules and adaptation through interactions with other agents. While MARL serves as the foundational framework, there is significant potential to enhance the learning process through strategic design choices. Future research directions include the exploration of advanced regularization techniques 107 /article/10.1007/s10458-025-09691-y ref-CR107 , 240 /article/10.1007/s10458-025-09691-y ref-CR240 , the adoption of tailored optimization strategies 25 /article/10.1007/s10458-025-09691-y ref-CR25 , and the integration of supervised or self-supervised learning objectives using appropriate loss designs 15 /article/10.1007/s10458-025-09691-y ref-CR15 , 151 /article/10.1007/s10458-025-09691-y ref-CR151 . Additionally, the application of meta-learning 155 /article/10.1007/s10458-025-09691-y ref-CR155 , decentralized learning approaches 56 /article/10.1007/s10458-025-09691-y ref-CR56 , and curriculum learning methodologies 56 /article/10.1007/s10458-025-09691-y ref-CR56 offer promising avenues for optimizing the EL learning process. 7.2.9 Human-agent interaction The final dimension focuses on the interpretability of ELs by humans and the degree to which humans can shape their development. This aspect is critical for creating human-agent interaction systems where communication is intuitive and effective 124 /article/10.1007/s10458-025-09691-y ref-CR124 . To advance this dimension, future research should prioritize the integration of human-in-the-loop feedback mechanisms to ensure that ELs are not only practical, but also comprehensible to human users 18 /article/10.1007/s10458-025-09691-y ref-CR18 , 24 /article/10.1007/s10458-025-09691-y ref-CR24 . This will improve the usability and adoption of these systems in real-world applications. Key research directions include designing experiments that create incentives for agents to develop communication strategies more closely aligned with human language 16 /article/10.1007/s10458-025-09691-y ref-CR16 . Additionally, exploring the resilience of communication protocols to deception through training with competing agents can lead to more robust and realistic interactions 34 /article/10.1007/s10458-025-09691-y ref-CR34 . Exploring adaptive communication strategies to optimize the sparsity and clarity of messages based on individual or group needs within human-agent teams is another promising direction 233 /article/10.1007/s10458-025-09691-y ref-CR233 . 8 Limitations and discussion In this section, we critically evaluate the limitations of our survey and identify areas for future improvement. Through our review, we aimed to develop a detailed taxonomy for the field of EL, focusing on its key properties cf. Section 5 /article/10.1007/s10458-025-09691-y Sec8 , and to analyze as well as categorize quantification approaches and metrics cf. Section 6 /article/10.1007/s10458-025-09691-y Sec20 . In addition, we curated a summary of open questions and suggestions for future research cf. Section 7 /article/10.1007/s10458-025-09691-y Sec42 . Despite considerable efforts to establish a viable taxonomy and framework in the most systematic and unbiased manner, there are several potential limitations to our research approach and methodology. First, while we have provided an extensive overview of 181 scientific publications in EL research, it is important to acknowledge that our search process, despite being thorough, may have overlooked significant contributions. Consequently, we do not claim completeness. However, we are very confident that our review represents a fair and well-balanced reflection of the existing body of work and the current state of the art. Second, our review includes sources that are not peer-reviewed, such as preprints from arXiv https://arxiv.org/ , to ensure that our work captures the most recent developments and diverse perspectives, including those that might be controversial. While we have carefully examined each paper included in this review, we cannot guarantee that every detail in non-peer-reviewed papers is entirely accurate. Consequently, we focused on concepts, findings, and metrics that are supported by multiple studies. Third, we have introduced a taxonomy and a comprehensive metrics categorization for EL research, a field that is still in its early stages. This effort comes with inherent challenges, and while we have addressed many of these, it is important to note that our proposed framework does not represent a consensus within the wider research community. We are transparent about this limitation and encourage further discussion and validation. Fourth, in order to maintain focus and conciseness, we have deliberately excluded ideas that lack associated metrics. As a result, some conceptual ideas from the reviewed research literature that are difficult to quantify in this early stage may not be fully explored in this survey. Finally, we have incorporated several existing metrics into our proposed framework. While many of these metrics are well established in the field, we acknowledge that a more rigorous and critical experimental evaluation of these metrics would be beneficial. We strongly recommend that future research conduct such evaluations to further refine and validate the tools and methods used in EL research. 9 Conclusion In this paper, we present a comprehensive taxonomy of EL , an overview of applicable metrics, and a summary of open challenges and potential research directions. Additionally, we provide a list of open source code repositories of the field in Table 10 /article/10.1007/s10458-025-09691-y Tab10 in Appendix A /article/10.1007/s10458-025-09691-y Sec47 . Our overall goal is to create a standardised yet dynamic framework that not only facilitates progress in this area of research, but also stimulates further interest and exploration. Section 2 /article/10.1007/s10458-025-09691-y Sec2 introduces the foundational linguistic concepts that underpin our taxonomy. Section 5 /article/10.1007/s10458-025-09691-y Sec8 offers a comprehensive taxonomy of EL based on the review of 181 scientific publications. Section 6 /article/10.1007/s10458-025-09691-y Sec20 presents a unified categorization and notation for various metrics, depicted in Fig. 9 /article/10.1007/s10458-025-09691-y Fig9 , ensuring consistency and clarity. Section 7 /article/10.1007/s10458-025-09691-y Sec42 provides a summary of current achievements and outlines research opportunities. By providing a structured overview and systematic categorization of linguistic concepts relevant to EL we have created a common ground for research and discussion. The detailed presentation of metrics and their unified notation ensures readability and usability, making it easier for researchers to navigate related topics and identify potential research opportunities and blind spots of future publications and the research field as a whole. This survey provides a valuable perspective on the development and analysis of EL, serving as both a guide and a resource for advancing this area of study. EL is a fascinating and promising way to achieve grounded and goal-oriented communication among agents and between humans and agents. Despite its significant progress in recent years, the field faces many open questions and requires further evaluation methods and metrics. Critical questions remain about the measurability of linguistic features, the validity of proposed metrics, their utility, and their necessity. Aligning EL with NLP for HCI presents additional opportunities and challenges. We encourage continued contributions and interdisciplinary research to address these issues and advance the field. Data availability Not applicable. Materials availability Not applicable. Code availability Not applicable. References Lewis, D. K. 1969 . Convention: A philosophical study 1st ed. . Harvard University Press.Wagner, K., Reggia, J. A., Uriagereka, J., & Wilkinson, G. S. 2003 . Progress in the simulation of emergent communication and language. Adaptive Behavior, 11 1 , 37–69. https://doi.org/10.1177/10597123030111003 https://doi.org/10.1177/10597123030111003 Steels, L. 1997 . The synthetic modeling of language origins. Evolution of Communication, 1 1 , 1–34. https://doi.org/10.1075/eoc.1.1.02ste https://doi.org/10.1075/eoc.1.1.02ste Nowak, M. A., & Krakauer, D. C. 1999 . The evolution of language. Proceedings of the National Academy of Sciences, 96 14 , 8028–8033. https://doi.org/10.1073/pnas.96.14.8028 https://doi.org/10.1073/pnas.96.14.8028 Kirby, S. 2002 . Natural language from artificial life. Artificial Life, 8 2 , 185–215. https://doi.org/10.1162/106454602320184248 https://doi.org/10.1162/106454602320184248 Cangelosi, A., & Parisi, D. 2002 . Simulating the Evolution of Language . Springer London. https://doi.org/10.1007/978-1-4471-0663-0 https://doi.org/10.1007/978-1-4471-0663-0 Christiansen, M. H., & Kirby, S. 2003 . Language Evolution . Oxford University Press.Batali, J. 1998 . Computational simulations of the emergence of grammar. In J. Hurford, C. Knight, & M. Studdert-Kennedy Eds. , Approaches to the Evolution of Language pp. 405–426 . Cambridge University Press.Oliphant, M., & Batali, J. 1997 . Learning and the emergence of coordinated communication. Center for Research on Language Newsletter, 11 1 , 1–46.Steels, L. 1995 . A self-organizing spatial vocabulary. Artificial Life, 2 3 , 319–332. https://doi.org/10.1162/artl.1995.2.3.319 https://doi.org/10.1162/artl.1995.2.3.319 Skyrms, B. 2002 . Signals, evolution and the explanatory power of transient information. Philosophy of Science, 69 3 , 407–428. https://doi.org/10.1086/342451 https://doi.org/10.1086/342451 Smith, K., Kirby, S., & Brighton, H. 2003 . Iterated learning: A framework for the emergence of language. Artificial Life, 9 4 , 371–386. https://doi.org/10.1162/106454603322694825 https://doi.org/10.1162/106454603322694825 Foerster, J. N., Assael, Y. M., Freitas, N. D., & Whiteson, S. 2017 . Learning to communicate with deep multi-agent reinforcement learning. In D. D. Lee, U. Luxburg, R. Garnett, M. Sugiyama, & I. Guyon Eds. , Advances in neural information processing systems 29th ed., pp. 2145–2153 . Curran Associates Inc.Lazaridou, A., Peysakhovich, A., Baroni, M. 2017 . Multi-agent cooperation and the emergence of natural language. In: OpenReview.net ed. 5th international conference on learning representations. https://openreview.net/forum?id=Hk8N3Sclg https://openreview.net/forum?id=Hk8N3Sclg Havrylov, S., & Titov, I. 2017 . Emergence of language with multi-agent games: Learning to communicate with sequences of symbols. In U. Luxburg, I. Guyon, S. Bengio, H. Wallach, R. Fergus, S. V. N. Vishwanathan, & R. Garnett Eds. , Advances in neural information processing systems 30th ed., pp. 2146–2156 . Curran Associates Inc.Bouchacourt, D., Baroni, M. 2018 . How agents see things: On visual representations in an emergent language game. In: Association for Computational Linguistics ed. Proceedings of the 2018 conference on empirical methods in natural language processing, pp. 981–985. https://doi.org/10.18653/v1/D18-1119 https://doi.org/10.18653/v1/D18-1119 Cao, K., Lazaridou, A., Lanctot, M., Leibo, J.Z., Tuyls, K., Clark, S. 2018 . Emergent communication through negotiation. In: OpenReview.net ed. 6th international conference on learning representations: Conference track proceedings. https://openreview.net/forum?id=Hk6WhagRW https://openreview.net/forum?id=Hk6WhagRW Mordatch, I., Abbeel, P. 2018 Emergence of grounded compositional language in multi-agent populations. In: Association for the advancement of artificial intelligence ed. Proceedings of the thirty-second AAAI conference on artificial intelligence and thirtieth innovative applications of artificial intelligence conference and eighth AAAI symposium on educational advances in artificial intelligence, pp. 1495–1502. AAAI Press. https://cdn.aaai.org/ojs/11492/11492-13-15020-1-2-20201228.pdf https://cdn.aaai.org/ojs/11492/11492-13-15020-1-2-20201228.pdf Das, A., Kottur, S., Moura, J.M.F., Lee, S., Batra, D. 2017 . Learning cooperative visual dialog agents with deep reinforcement learning. In: 2017 IEEE International conference on computer vision ICCV , pp. 2970–2979. IEEE. https://doi.org/10.1109/ICCV.2017.321 https://doi.org/10.1109/ICCV.2017.321 . http://arxiv.org/pdf/1703.06585v2 http://arxiv.org/pdf/1703.06585v2 Sukhbaatar, S., Szlam, A., & Fergus, R. 2017 . Learning multiagent communication with backpropagation. In D. D. Lee, U. Luxburg, R. Garnett, M. Sugiyama, & I. Guyon Eds. , Advances in neural information processing systems 29th ed., pp. 2252–2260 . Curran Associates Inc.Agarwal, A., Gurumurthy, S., Sharma, V., Lewis, M., Sycara, K. 2019 . Community regularization of visually-grounded dialog. In: International foundation for autonomous agents and multiagent systems ed. proceedings of the 18th international conference on autonomous agents and multiagent systems. ACM digital library, pp. 1042–1050. International Foundation for Autonomous Agents and Multiagent Systems. https://www.ifaamas.org/Proceedings/aamas2019/pdfs/p1042.pdf https://www.ifaamas.org/Proceedings/aamas2019/pdfs/p1042.pdf Blumenkamp, J., Prorok, A. 2020 . The emergence of adversarial communication in multi-agent reinforcement learning. In: PMLR ed. 4th Conference on robot learning. Proceedings of machine learning research, pp. 1394–1414. https://proceedings.mlr.press/v155/blumenkamp21a.html https://proceedings.mlr.press/v155/blumenkamp21a.html Brandizzi, N., Grossi, D., Iocchi, L. 2021 . Rlupus: Cooperation through emergent communication in the werewolf social deduction game. In: 13th Adaptive and learning agents workshop at AAMAS 2021. http://arxiv.org/pdf/2106.05018v2 http://arxiv.org/pdf/2106.05018v2 Brandizzi, N., Iocchi, L. 2022 . Emergent communication in human-machine games. In: 5th Workshop on emergent communication at ICLR 2022. https://openreview.net/forum?id=rqLgeQWCXZ9 https://openreview.net/forum?id=rqLgeQWCXZ9 Chaabouni, R., Strub, F., Altché, F., Tarassov, E., Tallec, C., Davoodi, E., Mathewson, K.W., Tieleman, O., Lazaridou, A., Piot, B. 2022 . Emergent communication at scale. In: OpenReview.net ed. 10th International conference on learning representations. https://openreview.net/forum?id=AUGBfDIV9rL https://openreview.net/forum?id=AUGBfDIV9rL Gupta, A., Lanctot, M., & Lazaridou, A. 2021 . Dynamic population-based meta-learning for multi-agent communication with natural language. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. S. Liang, & J. Wortman Vaughan Eds. , Advances in neural information processing systems NeurIPS 2021 , 34th ed. . Curran Associates Inc.Karten, S., Agrawal, S., Tucker, M., Hughes, D., Lewis, M., Shah, J., Sycara, K. 2022 . The enforcers: Consistent sparse-discrete methods for constraining informative emergent communication. http://arxiv.org/pdf/2201.07452v1 http://arxiv.org/pdf/2201.07452v1 Lo, Y.L., Sengupta, B., Lo Long, Y. 2022 . Learning to ground decentralized multi-agent communication with contrastive learning. In: 5th Workshop on emergent communication at ICLR 2022. https://openreview.net/forum?id=rLceWXWCmZc https://openreview.net/forum?id=rLceWXWCmZc Lowe, R., Foerster, J., Boureau, Y.-L., Pineau, J., Dauphin, Y. 2019 . On the pitfalls of measuring emergent communication. In: International foundation for autonomous agents and multiagent systems ed. Proceedings of the 18th international conference on autonomous agents and multiagent systems. ACM digital library, pp. 693–701. International foundation for autonomous agents and multiagent systems. https://www.ifaamas.org/Proceedings/aamas2019/pdfs/p693.pdf https://www.ifaamas.org/Proceedings/aamas2019/pdfs/p693.pdf Vanneste, S., Vanneste, A., Mets, K., Schepper, T.D., Anwar, A., Mercelis, S., Latré, S., Hellinckx, P. 2022 . Learning to communicate using counterfactual reasoning. In: 14th Workshop on adaptive and learning agents at AAMAS 2022. https://ala2022.github.io/papers/ALA2022 paper 17.pdf https://ala2022.github.io/papers/ALA2022 paper 17.pdf Verma, S. 2021 . Towards sample efficient learners in population based referential games through action advising: Extended abstract. In: Proceedings of the 20th international conference on autonomous agents and multiagent systems. AAMAS ’21. International foundation for autonomous agents and multiagent systems. https://www.ifaamas.org/Proceedings/aamas2021/pdfs/p1689.pdf https://www.ifaamas.org/Proceedings/aamas2021/pdfs/p1689.pdf Yu, D., Mu, J., Goodman, N. 2022 . Emergent covert signaling in adversarial reference games. In: 5th Workshop on emergent communication at ICLR 2022. https://openreview.net/forum?id=H-eMQbR7Z5 https://openreview.net/forum?id=H-eMQbR7Z5 Bogin, B., Geva, M., Berant, J. 2018 . Emergence of communication in an interactive world with consistent speakers. In: 2nd Workshop on emergent communication at NeurIPS 2018. http://arxiv.org/pdf/1809.00549v2 http://arxiv.org/pdf/1809.00549v2 Noukhovitch, M., LaCroix, T., Lazaridou, A., Courville, A. 2021 . Emergent communication under competition. In: Proceedings of the 20th international conference on autonomous agents and multiagent systems. AAMAS ’21. International foundation for autonomous agents and multiagent systems. https://www.ifaamas.org/Proceedings/aamas2021/pdfs/p974.pdf https://www.ifaamas.org/Proceedings/aamas2021/pdfs/p974.pdf Lazaridou, A., Baroni, M. 2020 . Emergent multi-agent communication in the deep learning era. http://arxiv.org/pdf/2006.02419v2 http://arxiv.org/pdf/2006.02419v2 Galke, L., Ram, Y., Raviv, L. 2022 . Emergent communication for understanding human language evolution: What’s missing? In: 5th Workshop on emergent communication at ICLR 2022. https://openreview.net/forum?id=rqUGZQ-0XZ5 https://openreview.net/forum?id=rqUGZQ-0XZ5 Mu, J., Goodman, N. 2021 . Emergent communication of generalizations. In Neural Information Processing Systems Foundation ed. Advances in neural information processing systems. Advances in neural information processing systems, 34, pp 17994–18007. Curran Associates Inc. https://papers.nips.cc/paper/2021/file/9597353e41e6957b5e7aa79214fcb256-Paper.pdf https://papers.nips.cc/paper/2021/file/9597353e41e6957b5e7aa79214fcb256-Paper.pdf Steinert-Threlkeld, S., Zhou, X., Liu, Z., Downey, C.M. 2022 . Emergent communication fine-tuning ec-ft for pretrained language models. In: 5th Workshop on emergent communication at ICLR 2022. https://openreview.net/forum?id=SUqrM7WR7W5 https://openreview.net/forum?id=SUqrM7WR7W5 Bender, E.M., Koller, A. 2020 . Climbing towards nlu: On meaning, form, and understanding in the age of data. In: Jurafsky, D., Chai, J., Schluter, N., Tetreault, J. eds. Proceedings of the 58th annual meeting of the association for computational linguistics, pp. 5185–5198. Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.acl-main.463 https://doi.org/10.18653/v1/2020.acl-main.463 Lemon, O. 2022 . Conversational grounding in emergent communication—data and divergence. In: 5th Workshop on emergent communication at ICLR 2022. https://openreview.net/forum?id=BbG-m-0Xbq https://openreview.net/forum?id=BbG-m-0Xbq Browning, J., Lecun, Y. 2022 . AI and the limits of language: An artificial intelligence system trained on words and sentences alone will never approximate human understanding., Online. https://www.noemamag.com/ai-and-the-limits-of-language/ https://www.noemamag.com/ai-and-the-limits-of-language/ Manning, C. D., & Schütze, H. 2005 . Foundations of Statistical Natural Language Processing 8th ed. . MIT Press.Qiu, X., Sun, T., Xu, Y., Shao, Y., Dai, N., & Huang, X. 2020 . Pre-trained models for natural language processing: A survey. Science China Technological Sciences, 63 10 , 1872–1897. https://doi.org/10.1007/s11431-020-1647-3 https://doi.org/10.1007/s11431-020-1647-3 Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Le Scao, T., Gugger, S., Drame, M., Lhoest, Q., Rush, A. 2020 . Transformers: State-of-the-art natural language processing. In: Liu, Q., Schlangen, D. eds. Proceedings of the 2020 conference on empirical methods in natural language processing: System demonstrations, pp. 38–45. Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.emnlp-demos.6 https://doi.org/10.18653/v1/2020.emnlp-demos.6 van Eecke, P., Beuls, K. 2021 . Re-conceptualising the language game paradigm in the framework of multi-agent reinforcement learning. In: Association for the Advancement of Artificial ed. COMARL AAAI 2020-2021—Challenges and opportunities for multi-agent reinforcement learning, AAAI Spring Symposium Series. https://arxiv.org/pdf/2004.04722 https://arxiv.org/pdf/2004.04722 Keresztury, B., Bruni, E. 2020 . Compositional properties of emergent languages in deep learning. http://arxiv.org/pdf/2001.08618v1 http://arxiv.org/pdf/2001.08618v1 Hernandez-Leal, P., Kartal, B., & Taylor, M. E. 2019 . A survey and critique of multiagent deep reinforcement learning. Autonomous Agents and Multi-Agent Systems, 33 6 , 750–797. https://doi.org/10.1007/s10458-019-09421-1 https://doi.org/10.1007/s10458-019-09421-1 Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., … Moher, D. 2021 . The prisma 2020 statement: An updated guideline for reporting systematic reviews. BMJ Clinical Research Edition , 372 , 71. https://doi.org/10.1136/bmj.n71 https://doi.org/10.1136/bmj.n71 Lipowska, D., & Lipowski, A. 2022 . Emergence and evolution of language in multi-agent systems. Lingua, 272 , 103331. https://doi.org/10.1016/j.lingua.2022.103331 https://doi.org/10.1016/j.lingua.2022.103331 Denamganaï, K., Walker, J.A. 2020 . Referentialgym: A nomenclature and framework for language emergence & grounding in visual referential games. In: 4th Workshop on emergent communication at NeurIPS 2020. http://arxiv.org/pdf/2012.09486v1 http://arxiv.org/pdf/2012.09486v1 Korbak, T., Zubek, J., Rączaszek-Leonardi, J. 2020 . Measuring non-trivial compositionality in emergent communication. In: 4th Workshop on emergent communication at NeurIPS 2020. http://arxiv.org/pdf/2010.15058v2 http://arxiv.org/pdf/2010.15058v2 LaCroix, T. 2019 . Biology and compositionality: Empirical considerations for emergent-communication protocols. In: 3rd workshop on emergent communication at NeurIPS 2019. http://arxiv.org/pdf/1911.11668v2 http://arxiv.org/pdf/1911.11668v2 Mihai, D., Hare, J. 2021 . The emergence of visual semantics through communication games. http://arxiv.org/pdf/2101.10253v1 http://arxiv.org/pdf/2101.10253v1 Galke, L., Raviv, L. 2024 . Emergent communication and learning pressures in language models: a language evolution perspective. http://arxiv.org/pdf/2403.14427v1 http://arxiv.org/pdf/2403.14427v1 Vanneste, A., Vanneste, S., Mets, K., Schepper, T.D., Mercelis, S., Latré, S., Hellinckx, P. 2022 . An analysis of discretization methods for communication learning with multi-agent reinforcement learning. In: 14th Workshop on adaptive and learning agents at AAMAS 2022. http://arxiv.org/pdf/2204.05669v1 http://arxiv.org/pdf/2204.05669v1 Moulin-Frier, C., Oudeyer, P.-Y. 2021 . Multi-agent reinforcement learning as a computational tool for language evolution research: Historical context and future challenges. In: Association for the Advancement of Artificial ed. COMARL AAAI 2020-2021—Challenges and opportunities for multi-agent reinforcement learning, AAAI Spring Symposium Series. https://arxiv.org/pdf/2002.08878 https://arxiv.org/pdf/2002.08878 Fernando, C., Zenkova, D., Nikolov, S., Osindero, S. 2020 . From language games to drawing games. http://arxiv.org/pdf/2010.02820v2 http://arxiv.org/pdf/2010.02820v2 Brandizzi, N. 2023 . Toward more human-like ai communication: A review of emergent communication research. IEEE Access, 11 , 142317–142340. https://doi.org/10.1109/ACCESS.2023.3339656 https://doi.org/10.1109/ACCESS.2023.3339656 Carston, R. 2009 . The explicit/implicit distinction in pragmatics and the limits of explicit communication. International Review of Pragmatics, 1 1 , 35–62. https://doi.org/10.1163/187731009X455839 https://doi.org/10.1163/187731009X455839 Watzlawick, P., Bavelas, J. B., & Jackson, D. D. 1967 . Pragmatics of human communication: A study of interactional patterns, pathologies, and paradoxes . Norton.Andersen, P. A. 1991 . When one cannot not communicate: A challenge to motley’s traditional communication postulates. Communication Studies, 42 4 , 309–325. https://doi.org/10.1080/10510979109368346 https://doi.org/10.1080/10510979109368346 Antos, G., Ventola, E., & Weber, T. 2008 . Handbook of interpersonal communication . De Gruyter Mouton. https://doi.org/10.1515/9783110211399 https://doi.org/10.1515/9783110211399 Witt, P. 2016 . Communication and Learning . De Gruyter Mouton. https://doi.org/10.1515/9781501502446 https://doi.org/10.1515/9781501502446 Hartley, P. 1993 . Interpersonal Communication 1st ed. . Routledge.Jones, R. G. 2018 . Communication in the real world 2nd ed. . Flat World Knowledge.Bossert, W. H., & Wilson, E. O. 1963 . The analysis of olfactory communication among animals. Journal of Theoretical Biology, 5 3 , 443–469. https://doi.org/10.1016/0022-5193 63 90089-4 https://doi.org/10.1016/0022-5193 63 90089-4 Sales, G., & Pye, D. 1974 . Ultrasonic Communication by Animals . Springer. https://doi.org/10.1007/978-94-011-6901-1 https://doi.org/10.1007/978-94-011-6901-1 Rauschecker, J. P., & Scott, S. K. 2009 . Maps and streams in the auditory cortex: Nonhuman primates illuminate human speech processing. Nature Neuroscience, 12 6 , 718–724. https://doi.org/10.1038/nn.2331 https://doi.org/10.1038/nn.2331 Tronick, E. Z. 1989 . Emotions and emotional communication in infants. The American Psychologist, 44 2 , 112–119. https://doi.org/10.1037/0003-066X.44.2.112 https://doi.org/10.1037/0003-066X.44.2.112 Grosse, G., Behne, T., Carpenter, M., & Tomasello, M. 2010 . Infants communicate in order to be understood. Developmental Psychology, 46 6 , 1710–1722. https://doi.org/10.1037/a0020727 https://doi.org/10.1037/a0020727 Stokoe, W. C. 1980 . Sign language structure. Annual Review of Anthropology, 9 1 , 365–390. https://doi.org/10.1146/annurev.an.09.100180.002053 https://doi.org/10.1146/annurev.an.09.100180.002053 Choi, E., Lazaridou, A., Freitas, N.d. 2018 Compositional obverter communication learning from raw visual input. In: OpenReview.net ed. 6th International conference on learning representations: Conference track proceedings. https://openreview.net/forum?id=rknt2Be0- https://openreview.net/forum?id=rknt2Be0- Austin, J. L. 1975 . How to do Things with Words: The William James lectures delivered at Harvard University in 1955 2nd ed. . Clarendon Press.Clark, H. H. 1996 . Using language 1st ed. . Cambridge University Press.Wittgenstein, L. 1989 . Philosophical investigations 3rd ed. . Blackwell.Adler, R. B. 2012 . Interplay: The process of interpersonal communication 3rd ed. . Oxford University Press.Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., iegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D. 2020 Language models are few-shot learners. In: Neural Information Processing Systems Foundation ed. Advances in neural information processing systems 33. Advances in neural information processing systems, vol. 33, pp. 1877–1901. Curran Associates Inc. https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf Lauriola, I., Lavelli, A., & Aiolli, F. 2022 . An introduction to deep learning in natural language processing: Models, techniques, and tools. Neurocomputing, 470 , 443–456. https://doi.org/10.1016/j.neucom.2021.05.103 https://doi.org/10.1016/j.neucom.2021.05.103 Khurana, D., Koli, A., Khatter, K., & Singh, S. 2022 . Natural language processing: State of the art, current trends and challenges. Multimedia Tools and Applications . https://doi.org/10.1007/s11042-022-13428-4 https://doi.org/10.1007/s11042-022-13428-4 Lappin, S. 2023 . Assessing the strengths and weaknesses of large language models. Journal of Logic, Language and Information . https://doi.org/10.1007/s10849-023-09409-x https://doi.org/10.1007/s10849-023-09409-x Lazaridou, A., Kuncoro, A., Gribovskaya, E., Agrawal, D., Liska, A., Terzi, T., Gimenez, M., Massond Autume, C., Kocisky, T., Ruder, S., Yogatama, D., Cao, K., Young, S., & Blunsom, P. 2021 . Mind the gap: Assessing temporal generalization in neural language models. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. .S. Liang, & J. Wortman Vaughan Eds. , Advances in neural information processing systems NeurIPS pp. 29348–29363 . Curran Associates Inc.Merrill, W., Goldberg, Y., Schwartz, R., & Smith, N. A. 2021 . Provable limitations of acquiring meaning from ungrounded form: What will future language models understand? Transactions of the Association for Computational Linguistics, 9 , 1047–1060.Liu, X., Yu, H., Zhang, H., Xu, Y., Lei, X., Lai, H., Gu, Y., Ding, H., Men, K., Yang, K., Zhang, S., Deng, X., Zeng, A., Du Zhengxiao, Zhang, C., Shen, S., Zhang, T., Su, Y., Sun, H., Huang, M., Dong, Y., Tang, J. 2024 . Agentbench: Evaluating llms as agents. In: OpenReview.net ed. 12th International conference on learning representations. https://openreview.net/forum?id=zAdUB0aCTQ</ https://openreview.net/forum?id=zAdUB0aCTQ