{"slug": "emergent-language-a-survey-and-taxonomy", "title": "Emergent Language: A Survey and Taxonomy", "summary": "A survey published in Autonomous Agents and Multi-Agent Systems defines emergent language as a research area in artificial intelligence where multi-agent reinforcement learning enables agents to autonomously develop communication forms, potentially in natural language style. The paper reviews scientific publications, analyzes evaluation methods and metrics, and identifies research gaps, serving as a reference for researchers.", "body_md": "## Abstract\n\nThe field of emergent language represents a novel area of research within the domain of artificial intelligence, particularly within the context of multi-agent reinforcement learning. Although the concept of studying language emergence is not new, early approaches were primarily concerned with explaining human language formation, with little consideration given to its potential utility for artificial agents. In contrast, studies based on reinforcement learning aim to develop communicative capabilities in agents that are comparable to or even superior to human language. Thus, they extend beyond the learned statistical representations that are common in natural language processing research. This gives rise to a number of fundamental questions, from the prerequisites for language emergence to the criteria for measuring its success. This paper addresses these questions by providing a comprehensive review of relevant scientific publications on emergent language in artificial intelligence. Its objective is to serve as a reference for researchers interested in or proficient in the field. Consequently, the main contributions are the definition and overview of the prevailing terminology, the analysis of existing evaluation methods and metrics, and the description of the identified research gaps.\n\n### Similar content being viewed by others\n\n## 1 Introduction\n\nCommunication between individual entities is based on conventions and rules that emerge from the necessity or advantage of coordination. Accordingly, Lewis [[1](/article/10.1007/s10458-025-09691-y#ref-CR1)] formalized settings that facilitate the emergence of language as “coordination problems” [[1](/article/10.1007/s10458-025-09691-y#ref-CR1)] and introduced a simple signaling game. This game, in which a speaker describes an object and a listener confronted with multiple options has to identify the indicated one, extensively shaped the field of emergent language (EL) research in computer science. Early works examined narrowly defined questions regarding the characteristics of emergent communication (EC) via hand-crafted simulations [[2](#ref-CR2),[3](#ref-CR3),[4](#ref-CR4),[5](#ref-CR5),[6](#ref-CR6),[7](#ref-CR7),[8](#ref-CR8),[9](#ref-CR9),[10](#ref-CR10),[11](#ref-CR11),[12](/article/10.1007/s10458-025-09691-y#ref-CR12)]. These approaches mostly utilized supervised learning methods and non-situated settings, limiting them in their ability to examine the origins and development of complex linguistic features [[2](/article/10.1007/s10458-025-09691-y#ref-CR2)]. However, EL research experienced an upsurge in the period between 2016 and 2018 [[13](#ref-CR13),[14](#ref-CR14),[15](#ref-CR15),[16](#ref-CR16),[17](#ref-CR17),[18](#ref-CR18),[19](#ref-CR19),[20](/article/10.1007/s10458-025-09691-y#ref-CR20)] with a focus on MARL approaches [[21](#ref-CR21),[22](#ref-CR22),[23](#ref-CR23),[24](#ref-CR24),[25](#ref-CR25),[26](#ref-CR26),[27](#ref-CR27),[28](#ref-CR28),[29](#ref-CR29),[30](#ref-CR30),[31](#ref-CR31),[32](/article/10.1007/s10458-025-09691-y#ref-CR32)] to enable the examination of more complex features.\n\nOne fundamental goal of EL research from the multi-agent reinforcement learning (MARL) perspective is to have agents autonomously develop a communication form that allows not only agent-to-agent but also agent-to-human communication in natural language (NL) style fashion [[2](/article/10.1007/s10458-025-09691-y#ref-CR2), [16](/article/10.1007/s10458-025-09691-y#ref-CR16), [24](/article/10.1007/s10458-025-09691-y#ref-CR24), [29](/article/10.1007/s10458-025-09691-y#ref-CR29), [33](/article/10.1007/s10458-025-09691-y#ref-CR33), [34](/article/10.1007/s10458-025-09691-y#ref-CR34)]. Therefore, reinforcement learning (RL) methods are attractive from two points of view. First, successful communication settings might lead to agents that are “more flexible and useful in everyday life” [[35](/article/10.1007/s10458-025-09691-y#ref-CR35)]. Furthermore, they may provide insights into the evolution of NL itself [[36](/article/10.1007/s10458-025-09691-y#ref-CR36)]. However, encouraging communication alone will not automatically produce a language with natural language characteristics [[37](/article/10.1007/s10458-025-09691-y#ref-CR37)]. Providing the right incentives for language development is therefore crucial.\n\nEL is the methodological attempt to enable agents to not only statistically understand and use NL, like natural language processing(NLP) models that learn on text alone [[38](/article/10.1007/s10458-025-09691-y#ref-CR38), [39](/article/10.1007/s10458-025-09691-y#ref-CR39)], but rather to design, acquire, develop, and learn their own language [[40](/article/10.1007/s10458-025-09691-y#ref-CR40), [41](/article/10.1007/s10458-025-09691-y#ref-CR41)]. The autonomy and independent active experience of RL learning settings is a crucial difference to the data-driven approaches in the field of NLP [[42](#ref-CR42),[43](#ref-CR43),[44](/article/10.1007/s10458-025-09691-y#ref-CR44)] and its large language model (LLM). According to Browning and LeCun, “we should not confuse the shallow understanding LLM possess for the deep understanding humans acquire” [[41](/article/10.1007/s10458-025-09691-y#ref-CR41)] through their experiences in life. In EL settings, the agents experience the benefits of communication through goal-oriented tasks [[45](/article/10.1007/s10458-025-09691-y#ref-CR45)] just like it happens naturally [[1](/article/10.1007/s10458-025-09691-y#ref-CR1)] and therefore have the opportunity to develop a deeper understanding of the world [[33](/article/10.1007/s10458-025-09691-y#ref-CR33), [46](/article/10.1007/s10458-025-09691-y#ref-CR46)]. Hence, advances in EL research enable novel applications of multi-agent systems and a considerably advanced form of human-centric AI [[35](/article/10.1007/s10458-025-09691-y#ref-CR35)].\n\nIn the current state of EL research, numerous different methods and metrics are already established but they are complex to structure and important issues remain regarding the analysis and comparison of achieved results [[29](/article/10.1007/s10458-025-09691-y#ref-CR29), [35](/article/10.1007/s10458-025-09691-y#ref-CR35), [47](/article/10.1007/s10458-025-09691-y#ref-CR47)]. Therefore, we see a need for a taxonomy to prevent misunderstandings and incorrect use of established metrics. In this paper, we address these issues by providing a comprehensive overview of publications in EL research and by introducing a taxonomy for discrete EL that encompasses key concepts and terminologies of this field. Additionally, we present established and recent metrics for discrete EL categorized according to the taxonomy and discuss their utility. Our goal is to provide a clear and concise description that researchers can use as a shared resource for guidance. Finally, we create a summary of EL research that highlights its achievements and provides an outlook on future research directions. We base our work on a comprehensive and systematic literature search with reproducible search terms on well-known databases. We follow the PRISMA [[48](/article/10.1007/s10458-025-09691-y#ref-CR48)] specifications and show a corresponding flow diagram in Fig. [11](/article/10.1007/s10458-025-09691-y#Fig11) in Appendix [B](/article/10.1007/s10458-025-09691-y#Sec48). The literature search and review process as well as its results are described in detail in Sect. [4](/article/10.1007/s10458-025-09691-y#Sec7). All identified work has been reviewed and categorized according to an extensive list of specific characteristics, e.g. regarding communication setting, game composition, environment configuration, language design, language metrics, and more.\n\nPrevious surveys of EL in computer science focused only on a subgroup of characteristics or specific parts of this research area. Some of these earlier surveys focus on specific learning settings [[45](/article/10.1007/s10458-025-09691-y#ref-CR45), [49](/article/10.1007/s10458-025-09691-y#ref-CR49), [50](/article/10.1007/s10458-025-09691-y#ref-CR50)], on methodological summaries and criticism [[29](/article/10.1007/s10458-025-09691-y#ref-CR29), [40](/article/10.1007/s10458-025-09691-y#ref-CR40), [51](#ref-CR51),[52](#ref-CR52),[53](#ref-CR53),[54](#ref-CR54),[55](/article/10.1007/s10458-025-09691-y#ref-CR55)], or provide a more general overview [[24](/article/10.1007/s10458-025-09691-y#ref-CR24), [35](/article/10.1007/s10458-025-09691-y#ref-CR35), [36](/article/10.1007/s10458-025-09691-y#ref-CR36), [47](/article/10.1007/s10458-025-09691-y#ref-CR47), [56](#ref-CR56),[57](#ref-CR57),[58](/article/10.1007/s10458-025-09691-y#ref-CR58)]. The most similar ones to our work are [[35](/article/10.1007/s10458-025-09691-y#ref-CR35)] and [[58](/article/10.1007/s10458-025-09691-y#ref-CR58)]. Lazaridou [[35](/article/10.1007/s10458-025-09691-y#ref-CR35)] gives an introduction and overview of the EL field before 2021, however, it is mostly a summary of previous work and does not provide a taxonomy or review of existing metrics in the field as we do. Brandizzi [[58](/article/10.1007/s10458-025-09691-y#ref-CR58)] focuses on common characteristics in EC research and the development of emergent human–machine communication strategies. They discuss distinctions and connections of EC research to linguistics, cognitive science, computer science, and sociology, while we focus on emergent language and its analysis. We describe and discuss all relevant surveys in more detail in Sect. [3](/article/10.1007/s10458-025-09691-y#Sec6).\n\nBased on this preliminary work, the current state of research on EL misses an overarching review and a comprehensive compilation and alignment of proposed quantification and comparability methods. Accordingly, the key contributions of the present survey are:\n\n-\nA taxonomy of the EL field, in particular regarding the properties of discrete EL, see Sect.\n\n[5](/article/10.1007/s10458-025-09691-y#Sec8). -\nA list of categorized quantification approaches and metrics in a consistent notation, see Sect.\n\n[6](/article/10.1007/s10458-025-09691-y#Sec20). -\nA summary of open questions and an outlook on potential future work, see Sect.\n\n[7](/article/10.1007/s10458-025-09691-y#Sec42).\n\nIn addition, we introduce the fundamental concepts of NL and EC that underlie our survey in Sect. [2](/article/10.1007/s10458-025-09691-y#Sec2). As mentioned, we provide a detailed summary of related surveys in Sect. [3](/article/10.1007/s10458-025-09691-y#Sec6). Section [4](/article/10.1007/s10458-025-09691-y#Sec7) describes our study methodology, including the keywords and terms of our systematic literature search. Finally, Sect. [8](/article/10.1007/s10458-025-09691-y#Sec45) offers a concluding discussion and final remarks.\n\n## 2 Background\n\nTo contextualize the presented taxonomy and analysis, this section summarizes the key concepts of communication and linguistics and provides an overview of EL research.\n\n### 2.1 Communication\n\nCommunication at its very basis is the transfer or exchange of signals, which can be interpreted to form some information. These signals include both intended, such as deliberate utterances, and unintended, such as uncontrolled bodily reactions, and include both explicit and implicit parts [[59](/article/10.1007/s10458-025-09691-y#ref-CR59)]. According to Watzlawik’s “Interactional View” [[60](/article/10.1007/s10458-025-09691-y#ref-CR60)], “one cannot not communicate”. In this regard, communication is ubiquitous and necessary, occurring through various channels and modes [[41](/article/10.1007/s10458-025-09691-y#ref-CR41), [61](#ref-CR61),[62](#ref-CR62),[63](#ref-CR63),[64](/article/10.1007/s10458-025-09691-y#ref-CR64)]. Depending on the specific channel and purpose, communication can be roughly divided into the five forms depicted in Fig. [1](/article/10.1007/s10458-025-09691-y#Fig1).\n\nIn the context of EL, two of these forms are actively studied, namely interpersonal communication and group communication. Interpersonal communication is communication between entities that mutually influence each other, and its general setting is depicted in Fig. [2](/article/10.1007/s10458-025-09691-y#Fig2). This form of communication is based on individual entities, each within its perceivable environment. Although these environments are agent-specific, they overlap and allow communication through a common channel. In addition, there may be noise in this process that affects the perception of the environment or the communication itself. Group communication, on the other hand, differs only in the number of entities involved and the communication goal. Usually, group communication is more formal and focuses on a common goal or group task while interpersonal communication has a social character and might only relate to a goal or task of one of the participants. Accordingly, the group communication setting can be found in most population-based EL research. Intrapersonal communication (e.g., internal vocalization), public communication (e.g., lectures), and mass communication (e.g., blog entries) are not currently examined in the EL literature.\n\nCommunication has been studied in many different disciplines from many different perspectives, including animals [[66](#ref-CR66),[67](#ref-CR67),[68](/article/10.1007/s10458-025-09691-y#ref-CR68)], pre-linguistic infants [[69](/article/10.1007/s10458-025-09691-y#ref-CR69), [70](/article/10.1007/s10458-025-09691-y#ref-CR70)], and sign language [[71](/article/10.1007/s10458-025-09691-y#ref-CR71)]. However, in order to keep the present work concise, we will refer mainly to research in the fields of linguistics and computer science. Generally, communication can be seen as a utility to coordinate with others [[72](#ref-CR72),[73](#ref-CR73),[74](#ref-CR74),[75](/article/10.1007/s10458-025-09691-y#ref-CR75)]. Conversely, the necessity for collaboration within a collective may be a fundamental precursor to the evolution and sustained functionality of explicit communication [[4](/article/10.1007/s10458-025-09691-y#ref-CR4), [53](/article/10.1007/s10458-025-09691-y#ref-CR53)]. This theory leads to an essential differentiation regarding context-dependent communication. Meaningful communication might emerge in a cooperative but not in a fully competitive or manipulative setting. However, a partially competitive setting might be vital for the emergence of resilient and comprehensive communication, e.g. to enable the detection and use of lies [[34](/article/10.1007/s10458-025-09691-y#ref-CR34)]. Accordingly, the level of cooperation is a defining element of the communication setting in EL research.\n\nNL is a tool that allows us to encode very complex information within a discrete and humanly manageable amount of utterances. A lot of artificial intelligence research aims to develop NL models, with applications ranging from translation to coherent full-text generation based on single-word input [[44](/article/10.1007/s10458-025-09691-y#ref-CR44), [77](#ref-CR77),[78](#ref-CR78),[79](/article/10.1007/s10458-025-09691-y#ref-CR79)]. However, current research is mostly based on LLM which “achieve a sophisticated level of inductive learning and inference” [[80](/article/10.1007/s10458-025-09691-y#ref-CR80)] but are also “far from human abilities in natural language inference, analogical reasoning, and interpretation” [[80](/article/10.1007/s10458-025-09691-y#ref-CR80)]. A lot of research from the EL community is based on the theory, that models which learn language statistically based on static datasets are limited in their communicative and cognitive abilities [[18](/article/10.1007/s10458-025-09691-y#ref-CR18), [41](/article/10.1007/s10458-025-09691-y#ref-CR41), [72](/article/10.1007/s10458-025-09691-y#ref-CR72), [81](/article/10.1007/s10458-025-09691-y#ref-CR81), [82](/article/10.1007/s10458-025-09691-y#ref-CR82)]. Recent publications have shown that LLM have “weak reasoning and decision-making abilities” [[83](/article/10.1007/s10458-025-09691-y#ref-CR83)], their “reasoning is fragile” [[84](/article/10.1007/s10458-025-09691-y#ref-CR84)], and that current LLM face “reliability issues” [[85](/article/10.1007/s10458-025-09691-y#ref-CR85)]. Correspondingly, the field of EL research in AI aims to enable agents to utilize intended communication in the same way humans use it to increase cooperation, performance, and generalization and, in the long run, enable direct meaningful communication between humans and artificial systems [[18](/article/10.1007/s10458-025-09691-y#ref-CR18), [34](/article/10.1007/s10458-025-09691-y#ref-CR34)]. In line with this, multiple explicit forms of EC in artificial intelligence research have been investigated as shown in Sect. [5.4.2](/article/10.1007/s10458-025-09691-y#Sec14). In contrast, work focusing on implicit communication, like the information content of spatial positioning of agents in a multi-agent setting [[86](/article/10.1007/s10458-025-09691-y#ref-CR86)], is not part of the present survey.\n\n### 2.2 Natural language\n\nNL is a prime example of a versatile and comprehensive form of communication designed to convey meaning [[87](/article/10.1007/s10458-025-09691-y#ref-CR87)]. The flexibility of NL allows humans to be exact but also deliberately ambiguous in their communication [[88](/article/10.1007/s10458-025-09691-y#ref-CR88)]. It is a vital feature that distinguishes us from other species and gives us a great advantage in terms of knowledge storage, sharing, and acquisition [[88](/article/10.1007/s10458-025-09691-y#ref-CR88)]. However, the origin and evolution of language is still a mystery [[89](/article/10.1007/s10458-025-09691-y#ref-CR89), [90](/article/10.1007/s10458-025-09691-y#ref-CR90)]. In the field of linguistics, many conflicting theories have been introduced so far [[90](#ref-CR90),[91](#ref-CR91),[92](#ref-CR92),[93](#ref-CR93),[94](/article/10.1007/s10458-025-09691-y#ref-CR94)], ranging from behavioral to biological explanations. Additionally, accompanying research in the field of computer science has a long history [[31](/article/10.1007/s10458-025-09691-y#ref-CR31)] with a comparable range of theories. Even though there is still a debate around this topic, it is commonly agreed upon that a very intricate evolutionary process was involved [[88](/article/10.1007/s10458-025-09691-y#ref-CR88), [90](/article/10.1007/s10458-025-09691-y#ref-CR90)]. This evolution most likely took place in two different areas simultaneously, biologically and linguistically. On the biological side, the human brain most likely developed specific areas and functionalities specifically for more complex language-based communication, that are studied in the scientific field of neurolinguistics [[95](/article/10.1007/s10458-025-09691-y#ref-CR95)]. On the linguistics side, this evolution can be seen in language development itself, which is a constantly ongoing process [[88](/article/10.1007/s10458-025-09691-y#ref-CR88)] that might be strongly connected to the development of cognitive skills [[96](/article/10.1007/s10458-025-09691-y#ref-CR96)] and the social environment [[97](/article/10.1007/s10458-025-09691-y#ref-CR97)]. Similarly, EL is concerned with the research of suitable model structures for the processing of language, while concurrently developing and evaluating language.\n\nWhile the exact origin of language is highly debatable, the actual communication process via NL is generally easier to conceptualize. For example, it can be modeled by the semiotic cycle depicted in Fig. [3](/article/10.1007/s10458-025-09691-y#Fig3) [[45](/article/10.1007/s10458-025-09691-y#ref-CR45), [98](/article/10.1007/s10458-025-09691-y#ref-CR98)]. This depiction applies to multiple expressive channels, e.g. speech and writing. It assumes at least two involved parties, a speaker and a listener. The speaker produces an utterance based on the meaning to be conveyed. This meaning results from the combined conceptualization of the speaker’s goal and model of the world. On the other hand, the listener receives the utterance and comprehends it to derive a meaning, which is not a direct copy of the initial one by the speaker but it still refers to the shared world. The interpretation of the meaning, which the listener’s world model informs, leads to some action by the listener. At the center of this process are the shared world and the respective world models of speaker and listener that function as grounding for the information exchange via language. Further, both linguistic level components of production and comprehension allow the respective agent to participate in the language process.\n\nThe semiotic cycle puts the utterance as an externalized information carrier into focus. While the other components are internalized and thus difficult to define and measure, the utterance itself is external and available for analysis. Fundamentally, this specific utterance is based on the underlying communication process and specifically, the language used. Accordingly, most research papers investigate characteristics of the utilized language to analyze the communication possibilities and capabilities of users. To this end, linguistics subdivides the language structure into six major levels [[99](#ref-CR99),[100](#ref-CR100),[101](/article/10.1007/s10458-025-09691-y#ref-CR101)], as illustrated in Fig. [4](/article/10.1007/s10458-025-09691-y#Fig4). This structure was originally developed for spoken language, as indicated by the terms ‘phonetics’ and ‘phonology’ derived from the Greek word ‘phon’ meaning ‘sound’. However, the levels are also applicable to written language in the context of EL. Therefore, the following description will address both spoken and written language within this framework.\n\nThe *phonetics* level includes the entire bandwidth of the chosen, often continuous, language channel. For example, it comprises the full range of possible speech sounds available to humans. Consequently, it is fundamental for the general transfer range and describes it without any limitation. At the *phonology* level are the atomic building blocks of the spoken or written language, defined as phonemes or graphemes. A phoneme or grapheme enables the creation of meaning as well as the necessary distinction at the lowest level of language. However, in a NL with an alphabetic writing system, phonemes and graphemes, which in this case correspond to letters, are often not a direct match and are only roughly related. Nevertheless, these individual units comprise the set of used elements from the continuous channel range for a specific language. These are used and combined at the *morphology* level to create and assign meaning by making words, in linguistics called lexemes. In this context, word-forming rules and underlying structures are of interest. Utilizing these meaningful building blocks, sentences can be realized at the *syntax* level. This level only concerns the structure of sentences and in particular, their assembly rules and the word categories that are used. The meaning of these sentences is relevant at the next level, *semantics*. At this level, the literal meaning of language constructions is of interest while the final level, *pragmatics*, focuses on how context contributes to the meaning. Accordingly, it analyzes how language is used in interactions and the relationship between the involved parties. Overall, the presented levels are not only important to describe language functionally and structurally but also to distinguish language characteristics and metrics. Thus, we use them to organize parts of the taxonomy in Sect. [5](/article/10.1007/s10458-025-09691-y#Sec8) and the metrics in Sect. [6](/article/10.1007/s10458-025-09691-y#Sec20).\n\n### 2.3 Emergent language\n\nEL refers to a form of communication that develops among artificial agents through interaction, without being explicitly pre-programmed. Thus, it is a bottom-up approach, arising from the agents’ need to cooperate and solve tasks within a given environment [[58](/article/10.1007/s10458-025-09691-y#ref-CR58)]. This process involves the agents creating, adapting, and refining linguistic structures and meanings to enhance their ability to exchange information effectively and efficiently [[49](/article/10.1007/s10458-025-09691-y#ref-CR49)]. EL research aims to understand the principles and mechanisms underlying this spontaneous development of communication. It explores how linguistic elements such as syntax [[102](/article/10.1007/s10458-025-09691-y#ref-CR102)], semantics [[103](/article/10.1007/s10458-025-09691-y#ref-CR103), [104](/article/10.1007/s10458-025-09691-y#ref-CR104)], and pragmatics [[29](/article/10.1007/s10458-025-09691-y#ref-CR29)] can arise from the interaction of artificial agents and how these elements contribute to the agents’ performance and cooperation.\n\nA NL-like communication form would make artificial agents and computer systems, in general, more accessible, simpler to comprehend, and altogether more powerful [[24](/article/10.1007/s10458-025-09691-y#ref-CR24), [34](/article/10.1007/s10458-025-09691-y#ref-CR34), [35](/article/10.1007/s10458-025-09691-y#ref-CR35)]. EL research originally focused on the question of language origin [[3](/article/10.1007/s10458-025-09691-y#ref-CR3)]. Recently, this focus shifted to the more functional aspect of EL, focusing on how to enable agent systems to benefit from a mechanism that helped humanity thrive and how to achieve communication capabilities as close as possible to NL [[35](/article/10.1007/s10458-025-09691-y#ref-CR35)]. Today, EL within computer science is about self-learned [[4](/article/10.1007/s10458-025-09691-y#ref-CR4)], reusable [[46](/article/10.1007/s10458-025-09691-y#ref-CR46)], teachable [[105](/article/10.1007/s10458-025-09691-y#ref-CR105), [106](/article/10.1007/s10458-025-09691-y#ref-CR106)], interpretable [[14](/article/10.1007/s10458-025-09691-y#ref-CR14)], and powerful [[18](/article/10.1007/s10458-025-09691-y#ref-CR18)] communication protocols. In the long run, EL aims to enable machines to communicate with each other and with humans in a more seamless and extendable manner [[35](/article/10.1007/s10458-025-09691-y#ref-CR35), [107](/article/10.1007/s10458-025-09691-y#ref-CR107)].\n\nAccordingly, various research questions and areas were derived. For example, recent papers have addressed issues around the nature of the setting, which can be semi-cooperative [[34](/article/10.1007/s10458-025-09691-y#ref-CR34), [108](/article/10.1007/s10458-025-09691-y#ref-CR108)], include adversaries [[22](/article/10.1007/s10458-025-09691-y#ref-CR22), [32](/article/10.1007/s10458-025-09691-y#ref-CR32)], have message-influencing noise [[109](/article/10.1007/s10458-025-09691-y#ref-CR109)], or incorporate social structures [[110](/article/10.1007/s10458-025-09691-y#ref-CR110), [111](/article/10.1007/s10458-025-09691-y#ref-CR111)]. Moreover, some are concerned with the challenge of grounding EL, e.g. using representation learning as basis [[112](/article/10.1007/s10458-025-09691-y#ref-CR112)], combining supervised learning and self-play [[113](/article/10.1007/s10458-025-09691-y#ref-CR113)], or utilizing EL agents as the basis for NL finetuning approaches [[107](/article/10.1007/s10458-025-09691-y#ref-CR107)]. Others tackle the direct emergence of language with NL characteristics, e.g. looking at internal and external pressures [[114](#ref-CR114),[115](#ref-CR115),[116](#ref-CR116),[117](#ref-CR117),[118](/article/10.1007/s10458-025-09691-y#ref-CR118)], evaluating factors to enforce semantic conveyance [[53](/article/10.1007/s10458-025-09691-y#ref-CR53)], looking at compositionality [[119](/article/10.1007/s10458-025-09691-y#ref-CR119)], generalization [[25](/article/10.1007/s10458-025-09691-y#ref-CR25)], or expressivity [[120](/article/10.1007/s10458-025-09691-y#ref-CR120)], or questioning the importance of characteristics like compositionality [[121](/article/10.1007/s10458-025-09691-y#ref-CR121)] and the connection between compositionality and generalization [[122](/article/10.1007/s10458-025-09691-y#ref-CR122)].\n\nBased on these examples and the introduced goals and approaches, the difference in comparison to NLP research becomes apparent. Current approaches in NLP, namely LLM, learn language imitation via statistics, but they might not capture the functional aspects and the purpose of communication itself [[18](/article/10.1007/s10458-025-09691-y#ref-CR18), [41](/article/10.1007/s10458-025-09691-y#ref-CR41)]. In contrast, EL uses language not as the sole objective but as a means to achieve something with meaning [[23](/article/10.1007/s10458-025-09691-y#ref-CR23)]. Accordingly, agents have to learn their own EL to enable functionality beyond simple statistical reproduction. Specifically, agents should learn communication by necessity or benefits [[114](/article/10.1007/s10458-025-09691-y#ref-CR114)] and they need a setting that rewards or encourages communication, e.g., an at least partially cooperative setting [[34](/article/10.1007/s10458-025-09691-y#ref-CR34)].\n\nWhile the EL concept sounds simple, it comes with many challenges. Encouraging communication alone can lead to simple gibberish that helps with task completion but does not represent the intended natural language characteristics [[37](/article/10.1007/s10458-025-09691-y#ref-CR37)]. Providing the right incentives for language development is therefore crucial. In addition, it is important to examine how agents use communication and the opportunity to send and receive information, raising the question of how to measure successful communication [[29](/article/10.1007/s10458-025-09691-y#ref-CR29)]. The measurability of language properties such as syntax, semantics, and pragmatics is also important for assessing the emergence of desirable language properties [[122](/article/10.1007/s10458-025-09691-y#ref-CR122)]. The following sections explore these challenges and related constructs and approaches in detail.\n\n## 3 Related surveys\n\nAs briefly mentioned in Sect. [1](/article/10.1007/s10458-025-09691-y#Sec1), our literature review identified 19 publications that we classified as surveys. We adopted a broad definition of what constitutes a survey, categorizing any publication as a survey if it either explicitly described itself as such or provided a particularly comprehensive and structured review of previous research. These publications conduct similar investigations on EL research but with different scopes. We focus on discrete language emergence, associated taxonomy, characteristics, metrics, and research gaps. In contrast, in our review of the existing survey work, three distinct interpretive directions emerge, which we categorize as summarized in Table [1](/article/10.1007/s10458-025-09691-y#Tab1): Surveys that focus on the learning *settings* [[45](/article/10.1007/s10458-025-09691-y#ref-CR45), [49](/article/10.1007/s10458-025-09691-y#ref-CR49), [50](/article/10.1007/s10458-025-09691-y#ref-CR50)], surveys that summarize and review utilized *methods* [[29](/article/10.1007/s10458-025-09691-y#ref-CR29), [40](/article/10.1007/s10458-025-09691-y#ref-CR40), [51](#ref-CR51),[52](#ref-CR52),[53](#ref-CR53),[54](#ref-CR54),[55](/article/10.1007/s10458-025-09691-y#ref-CR55)], and surveys that provide a *general* discussion or overview of the EL field [[24](/article/10.1007/s10458-025-09691-y#ref-CR24), [35](/article/10.1007/s10458-025-09691-y#ref-CR35), [36](/article/10.1007/s10458-025-09691-y#ref-CR36), [47](/article/10.1007/s10458-025-09691-y#ref-CR47), [56](#ref-CR56),[57](#ref-CR57),[58](/article/10.1007/s10458-025-09691-y#ref-CR58)]. The following section briefly summarizes these surveys within these categories.\n\n### 3.1 Settings\n\nSurveys within the *settings* category primarily focus on the design of language learning environments and the general structure of learning problems. For instance, van Eecke and Beuls [[45](/article/10.1007/s10458-025-09691-y#ref-CR45)] explored the language game paradigm, categorizing experiments and identifying properties critical for MARL research, such as symmetric agent roles and autonomous behavior. While their work offers a foundational perspective, our survey extends beyond the language game paradigm to analyze a broader range of approaches in greater depth (see Sect. [7](/article/10.1007/s10458-025-09691-y#Sec42)). Similarly, Lipowska and Lipowski [[49](/article/10.1007/s10458-025-09691-y#ref-CR49)] emphasized sociocultural aspects, such as migration and teachability, within simple naming games. While these are part of our analysis, our review situates them within a unified framework, providing a more comprehensive perspective. Denamganaï and Walker [[50](/article/10.1007/s10458-025-09691-y#ref-CR50)] introduced ReferentialGym as a tool for studying referential games and their associated metrics, like positive signaling and positive listening [[29](/article/10.1007/s10458-025-09691-y#ref-CR29)]. In contrast, our survey goes beyond referential games, offering a broader exploration of EL metrics and their applications across diverse frameworks.\n\n### 3.2 Methods\n\nThe *methods* category encompasses surveys that primarily examine learning and evaluation methodologies in EL, each offering unique perspectives on key challenges. A recurring theme in this category is the need for more comprehensive evaluation metrics that capture the complexity of emergent communication [[51](#ref-CR51),[52](#ref-CR52),[53](/article/10.1007/s10458-025-09691-y#ref-CR53)]. For example, Korbak et al. [[51](/article/10.1007/s10458-025-09691-y#ref-CR51)] highlighted the limitations of existing compositionality metrics, introducing the *tree reconstruction error* to address semantic compositionality, a challenge we contextualize further in Section [5.4.5](/article/10.1007/s10458-025-09691-y#Sec17). In contrast, LaCroix [[52](/article/10.1007/s10458-025-09691-y#ref-CR52)] critiqued the overemphasis on compositionality, advocating for a shift towards reflexivity, though metrics for this remain unexplored. This gap underscores the fragmented nature of current evaluation practices.\n\nGrounding and utility also feature prominently in this category. Lemon [[40](/article/10.1007/s10458-025-09691-y#ref-CR40)] emphasized the interplay of symbolic and conversational grounding, highlighting data-related challenges, while Lowe et al. [[29](/article/10.1007/s10458-025-09691-y#ref-CR29)] proposed pragmatic metrics such as positive signaling and positive listening, which inspired our taxonomy of language utility in Sect. [5.4.6](/article/10.1007/s10458-025-09691-y#Sec18). These works collectively underscore the necessity of balancing semantic depth with practical utility, a balance our survey seeks to achieve by integrating diverse perspectives into a unified framework.\n\nFurther, recent studies like those by Galke and Raviv [[54](/article/10.1007/s10458-025-09691-y#ref-CR54)] explored the role of linguistic pressures and biases in bridging the gap between EL and human NL, providing insights into the origins of NL phenomena in EL. Vanneste et al. [[55](/article/10.1007/s10458-025-09691-y#ref-CR55)] tackled discretization methods critical for EL learning, offering a comparative analysis that complements our work.\n\n### 3.3 General\n\nThe *general* category includes surveys that provide broad overviews or address themes not confined to specific *settings* or *methods*. Key contributions in this category highlight the interdisciplinary perspectives, interaction paradigms, and structural dimensions of emergent communication research.\n\nHernandez-Leal et al. [[47](/article/10.1007/s10458-025-09691-y#ref-CR47)] offered a foundational overview of multi-agent deep reinforcement learning (MARL), including emergent behavior and communication. While their survey provides valuable historical context and practical challenges, our work builds upon this by focusing specifically on emergent language (EL) within MARL, analyzing it with finer granularity and from a metrics-driven perspective. Similarly, Brandizzi and Iocchi [[24](/article/10.1007/s10458-025-09691-y#ref-CR24)] emphasized the underrepresentation of human-in-the-loop paradigms, proposing novel interaction settings but lacking the systematic categorization and metrics-oriented discussion presented in our survey.\n\nMoulin-Frier and Oudeyer [[56](/article/10.1007/s10458-025-09691-y#ref-CR56)], Fernando et al. [[57](/article/10.1007/s10458-025-09691-y#ref-CR57)], and Galke et al. [[36](/article/10.1007/s10458-025-09691-y#ref-CR36)] explored interdisciplinary connections and cognitive constraints in EL, with the latter focusing on the perceived gaps between EL and NL. While these works underscore key challenges, our survey contextualizes such limitations across a broader set of metrics and emergent properties, providing actionable insights for bridging these gaps.\n\nOther surveys, such as those by Suglia et al. [[123](/article/10.1007/s10458-025-09691-y#ref-CR123)] and Zhu et al. [[124](/article/10.1007/s10458-025-09691-y#ref-CR124)], structured EL research into multimodal and dimensional frameworks, respectively. These works serve as useful complements to our survey, which introduces an extensive taxonomy (Sect. [5](/article/10.1007/s10458-025-09691-y#Sec8)) that synthesizes and organizes findings from diverse sources. Similarly, Lazaridou and Baroni [[35](/article/10.1007/s10458-025-09691-y#ref-CR35)] and Brandizzi [[58](/article/10.1007/s10458-025-09691-y#ref-CR58)] provided comprehensive overviews of the field but lacked the detailed quantification and taxonomy of metrics that form the core of our work.\n\nBy synthesizing these contributions, our survey is distinguished by its focus on emergent language metrics and quantification, complemented by a systematic taxonomy to address fragmentation in the field. This integrated approach provides a structured roadmap for advancing EL research, with an emphasis on both practical measurability and interdisciplinary relevance.\n\n## 4 Study methodology\n\nThe literature search that resulted in the body of work surveyed in this paper was conducted on the 17th of June 2024. The used libraries and databases are: [ScienceDirect](https://www.sciencedirect.com/), [IEEE Xplore](https://ieeexplore.ieee.org/), [ACM Digital Library](https://dl.acm.org/), [WebOfScience](https://www.webofscience.com/), [arXiv](https://arxiv.org/), and [SemanticScholar](https://www.semanticscholar.org/). [SemanticScholar](https://www.semanticscholar.org/) is a special case, due to the nature of the provided search machine that does not allow complex queries and filtering like the others. Consequently, we hand-picked suitable papers from the first 50 entries of the search result list. A PRISMA [[48](/article/10.1007/s10458-025-09691-y#ref-CR48)] flow diagram of the publication selection process is provided in Fig. [11](/article/10.1007/s10458-025-09691-y#Fig11) in Appendix [B](/article/10.1007/s10458-025-09691-y#Sec48). Additionally, the individual queries and results of all services are summarized in Table [2](/article/10.1007/s10458-025-09691-y#Tab2). The queries delivered 613 hits in total which resulted in 516 unique papers. A first quick read of these papers led to 23 additional papers, referenced by some of the originally found work. Accordingly, the literature review started with a corpus consisting of 539 individual papers.\n\nOf the 539 papers, 327 were sorted out due to the substantial divergence from the searched topic, often focusing on domains like 5G, networking, and radio. Of the remaining 212 papers, 181 directly address the field of interest, while 31 are only partially relevant. Papers were deemed partially relevant if they mentioned the surveyed topic but primarily focused on different areas such as datasets, language theory, simulation, or unrelated case studies. In conclusion, this survey mainly reviews 181 papers that directly discuss or contribute to the topic of EL in computer science.\n\nFigure [5](/article/10.1007/s10458-025-09691-y#Fig5) presents the distribution of the 181 relevant publications over the years, categorized by publication type. The topic of EL has maintained a steady presence in conference publications, peaking in 2020. The subsequent decline in total publications may be attributed to the absence of recent topic-specific workshops. Additionally, the surge in interest in LLM technologies might have diverted attention from EL research. It is also worth noting that some recent studies may not have been openly published at the time of our literature search. We therefore expect the publication count to increase by 2024.\n\n## 5 Taxonomy of emergent language\n\nIn the course of our comprehensive literature review, we identified recurrent instances of taxonomic inconsistencies due to missing standardization [[125](/article/10.1007/s10458-025-09691-y#ref-CR125)] and “ill-adapted metrics” [[25](/article/10.1007/s10458-025-09691-y#ref-CR25)]. Particular concern arises from the discrepancy between the concepts intended for measurement and their corresponding metrics, or the absence of such metrics [[46](/article/10.1007/s10458-025-09691-y#ref-CR46), [51](/article/10.1007/s10458-025-09691-y#ref-CR51), [126](#ref-CR126),[127](#ref-CR127),[128](/article/10.1007/s10458-025-09691-y#ref-CR128)]. This section is dedicated to the formulation of a systematic taxonomy aimed at enhancing comparability and mitigating confusion within the field. This taxonomy forms the basis for the following sections and is designed to ensure consistent representation throughout the survey. It is created with the hope that it will serve as a cornerstone for future research, promoting the use of standardized terminology, particularly in the domain of language characteristics.\n\nThe taxonomy first describes the main factors influencing the EL, before categorizing the language characteristics. These influencing factors have a significant impact on the investigative possibilities of EL research and are therefore of particular importance when analyzing EL. Thus, the taxonomy introduces a classification system for the communication setting (Sect. [5.1](/article/10.1007/s10458-025-09691-y#Sec9)) and communication games (Sect. [5.2](/article/10.1007/s10458-025-09691-y#Sec10)) that agents encounter during language emergence. The communication setting encompasses factors such as the number of agents and the type of communication available to them. The communication game involves the environmental configuration and crucial factors influencing challenges and the complexity of multi-task learning. Furthermore, a short discussion on the concept of language priors is provided in Sect. [5.3](/article/10.1007/s10458-025-09691-y#Sec11), considering that the presence of a prior significantly influences the characteristics of the emerging language [[14](/article/10.1007/s10458-025-09691-y#ref-CR14), [113](/article/10.1007/s10458-025-09691-y#ref-CR113)]. We conclude this section with a comprehensive overview of the concepts and characteristics examined within EL research (Sect. [5.4](/article/10.1007/s10458-025-09691-y#Sec12)). The taxonomy adheres to the six major linguistic structural levels introduced in Sect. [2.2](/article/10.1007/s10458-025-09691-y#Sec4) and illustrated in Fig. [4](/article/10.1007/s10458-025-09691-y#Fig4).\n\n### 5.1 Comunication setting\n\nIn the literature, several communication settings are represented. One distinguishing factor is the number of agents involved. We derived three classes—the single agent, dual agent, and population setting. While the single agent setting is rare, the other two are well represented in the examined literature, as shown in Table [3](/article/10.1007/s10458-025-09691-y#Tab3). A single agent is typically used to train human–machine interfaces [[129](/article/10.1007/s10458-025-09691-y#ref-CR129)] or fine-tune existing models [[38](/article/10.1007/s10458-025-09691-y#ref-CR38)]. In contrast, dual-agent settings are more common and often involve a pair of speaker-listener agents, with one agent designated as the speaker and the other as the listener exclusively [[14](/article/10.1007/s10458-025-09691-y#ref-CR14)]. The population setting involves larger groups of agents in the language emergence process. This requires more computational resources but also enables more possibilities for regularization [[21](/article/10.1007/s10458-025-09691-y#ref-CR21)] and language evolution [[117](/article/10.1007/s10458-025-09691-y#ref-CR117)]. Accordingly, the population setting offers more opportunities to actively shape the process [[21](/article/10.1007/s10458-025-09691-y#ref-CR21), [45](/article/10.1007/s10458-025-09691-y#ref-CR45), [130](/article/10.1007/s10458-025-09691-y#ref-CR130)].\n\nAn additional factor that shapes the communication setting is the type of cooperation inherent in the setup. Determining the level of cooperation or competition feasible within the setting is a fundamental decision and closely related to the choice of the language game. We derived three options—the cooperative, semi-cooperative, and competitive type. In the literature reviewed, the majority of studies adopted a fully cooperative setting approach, where agents fully share their rewards and lack individual components. The emphasis on strongly cooperative settings is justified given that AI agents utilize a common language to coordinate and will not learn to communicate if they dominate without communication [[34](/article/10.1007/s10458-025-09691-y#ref-CR34)]. Only a few publications explore semi-cooperative settings that incorporate individual rewards alongside shared rewards, introducing the challenge of balancing tasks and rewards [[29](/article/10.1007/s10458-025-09691-y#ref-CR29), [108](/article/10.1007/s10458-025-09691-y#ref-CR108)]. A semi-cooperative setup can be compared to a simplified social scenario with overarching societal objectives, while also encompassing additional individual interests and goals. In contrast, investigations of fully competitive settings are rare, with only one work in which agents compete for rewards without a common goal [[34](/article/10.1007/s10458-025-09691-y#ref-CR34)]. This scarcity likely arises from the fact that such settings inherently favor deceptive language as the only advantageous strategy, making its emergence improbable without any cooperative element [[34](/article/10.1007/s10458-025-09691-y#ref-CR34)].\n\nThe third important factor in communications settings is symmetry. Agents should treat messages similarly to regular observations; otherwise, they risk devolving into mere directives [[146](/article/10.1007/s10458-025-09691-y#ref-CR146)]. Building on this premise, the symmetry is important for promoting robust language emergence, as opposed to languages that consist primarily of directives. An illustrative example of asymmetric settings is the commonly used, and aforementioned, speaker-listener paradigm [[14](/article/10.1007/s10458-025-09691-y#ref-CR14), [51](/article/10.1007/s10458-025-09691-y#ref-CR51), [168](/article/10.1007/s10458-025-09691-y#ref-CR168)]. Languages developed in such settings are severely limited compared to NL, lacking the capacity for diverse discourse or even basic information exchange beyond directives [[146](/article/10.1007/s10458-025-09691-y#ref-CR146)]. Contrary to promoting informed choices by the listener, the speaker-listener approach emphasizes obedience to commands. Conversely, a symmetric setting facilitates bi-directional communication, thereby allowing for more comprehensive language development [[110](/article/10.1007/s10458-025-09691-y#ref-CR110), [202](/article/10.1007/s10458-025-09691-y#ref-CR202)]. For instance, symmetry may result from agents being randomly assigned roles within the interaction [[202](/article/10.1007/s10458-025-09691-y#ref-CR202)]. Additionally, symmetry can emerge from tasks that are inherently balanced, such as negotiations between equal partners where both parties have equivalent roles and objectives [[17](/article/10.1007/s10458-025-09691-y#ref-CR17)].\n\nAt the population level, another important consideration is the choice of recipients, i.e., between targeted and broadcast communication. While broadcast communication facilitates broader information dissemination across the agent group, targeted communication promotes the development of social group dynamics and regularization [[222](/article/10.1007/s10458-025-09691-y#ref-CR222), [226](/article/10.1007/s10458-025-09691-y#ref-CR226)]. For example, targeted communication strategies can be learned through mechanisms such as attention [[222](/article/10.1007/s10458-025-09691-y#ref-CR222)], and agents can develop minimized communication strategies that optimize group performance [[229](/article/10.1007/s10458-025-09691-y#ref-CR229)].\n\nTable [3](/article/10.1007/s10458-025-09691-y#Tab3) provides a summary of these settings and their variations. The setting categories presented and their implementation are not inherently tied to the language itself but are crucial in determining the likelihood of meaningful language emergence and in shaping the features and experimental possibilities. These initial choices dictate the options for the language development process, the opportunities for regularization [[21](/article/10.1007/s10458-025-09691-y#ref-CR21), [130](/article/10.1007/s10458-025-09691-y#ref-CR130)], and the requirements regarding computational resources.\n\n### 5.2 Language games\n\nDistinct communication settings are implemented through different communication games. In this section, we provide an overview of the games used in EL literature. Specifically, we focus on a subset of these games known as language games, that emphasize explicit communication via a predefined language channel. The literature identifies several categories of language games, such as referential games, reconstruction games, question-answer games, grid-world games, among others. Our review indicates that these categories represent the most commonly used game types. To give a comprehensive view, Table [4](/article/10.1007/s10458-025-09691-y#Tab4) lists the publications that focus on these game types. In the following, we offer a concise overview of each category to provide a clearer understanding of their characteristics.\n\n*Referential game* Generally, a referential game, also called signaling game, consists of two agents, a sender and a receiver [[14](/article/10.1007/s10458-025-09691-y#ref-CR14)]. The objective of this game is for the receiver to correctly identify a particular sample from a set, which may include distractors, solely based on the message received from the sender. This set can consist of images [[14](/article/10.1007/s10458-025-09691-y#ref-CR14), [16](/article/10.1007/s10458-025-09691-y#ref-CR16), [149](/article/10.1007/s10458-025-09691-y#ref-CR149)], object feature vectors [[140](/article/10.1007/s10458-025-09691-y#ref-CR140)], texts [[140](/article/10.1007/s10458-025-09691-y#ref-CR140)], or even graphs [[186](/article/10.1007/s10458-025-09691-y#ref-CR186)]. To accomplish this selection task, the sender must first encode a message that contains information about the correct sample. In game design, a fundamental decision arises regarding whether the sender should only view the correct sample or also some distractors that may differ from those presented to the receiver [[14](/article/10.1007/s10458-025-09691-y#ref-CR14)]. Another design decision concerns the receiver’s side, specifically the number of distractors and whether to provide the original sample shown to the sender or only a similar one for selection [[160](/article/10.1007/s10458-025-09691-y#ref-CR160)]. However, only the encoded message is transmitted to the receiver, who then selects an item from their given collection.\n\n*Reconstruction game* The reconstruction game is similar to the referential game, but with a key difference: the receiver does not have a collection to choose from. Instead, the receiver must construct a sample based on the message from the sender, aiming to replicate the original sample shown to the sender as closely as possible [[118](/article/10.1007/s10458-025-09691-y#ref-CR118), [122](/article/10.1007/s10458-025-09691-y#ref-CR122)]. Consequently, this game setup resembles an autoencoder-based approach, with a latent space tailored to mimic or facilitate language [[162](/article/10.1007/s10458-025-09691-y#ref-CR162)]. Therefore, the key distinction between reconstruction and referential games, often used interchangeably in early literature, lies in the collection’s presence (referential) or absence (reconstruction) for the receiver to select from [[155](/article/10.1007/s10458-025-09691-y#ref-CR155)].\n\n*Question-answer game* The question-answer game is a variant of the referential game, but without strict adherence to previously established rules. It operates as a multi-round referential game, allowing for iterative and bilateral communication [[202](/article/10.1007/s10458-025-09691-y#ref-CR202)]. Unlike referential and reconstruction games, the question-answer game explicitly incorporates provisions for multiple rounds with follow-up or clarifying queries from the receiver [[19](/article/10.1007/s10458-025-09691-y#ref-CR19), [21](/article/10.1007/s10458-025-09691-y#ref-CR21)]. Question-answer games have introduced intriguing inquiries and avenues for exploring the symmetry of EL, although they are not as widely adopted [[19](/article/10.1007/s10458-025-09691-y#ref-CR19)].\n\n*Grid world game* Grid world games use a simplified 2D environment to model various scenarios like warehouse path planning [[22](/article/10.1007/s10458-025-09691-y#ref-CR22)], movement of objects [[205](/article/10.1007/s10458-025-09691-y#ref-CR205)], traffic junctions [[226](/article/10.1007/s10458-025-09691-y#ref-CR226), [231](/article/10.1007/s10458-025-09691-y#ref-CR231)], or mazes [[20](/article/10.1007/s10458-025-09691-y#ref-CR20), [115](/article/10.1007/s10458-025-09691-y#ref-CR115)]. They offer design flexibility, allowing agents to be part of the environment or act as external supervisors. Design choices also include environment complexity and the extent of agents’ observations. Although common in the literature surveyed, implementations of grid world games vary widely in their design choices and are thus a very heterogeneous group.\n\n*Continuous world game* Continuous environments add complexity to the learning process [[229](/article/10.1007/s10458-025-09691-y#ref-CR229), [236](/article/10.1007/s10458-025-09691-y#ref-CR236)]. In EL approaches, the learning landscape involves multi-task settings where one task is tackled directly within the environment while another involves language formation. Playing continuous world games, whether in two or three dimensions, presents challenges and adds a greater sense of realism and intricacy. These environments have the potential to make it more feasible to deploy EL agents in real-world scenarios compared to discrete environments [[207](/article/10.1007/s10458-025-09691-y#ref-CR207)].\n\n*Other* The literature on EL also covers various other game types besides those mentioned earlier, such as matrix communication games [[29](/article/10.1007/s10458-025-09691-y#ref-CR29), [30](/article/10.1007/s10458-025-09691-y#ref-CR30)], social deduction games [[23](/article/10.1007/s10458-025-09691-y#ref-CR23), [235](/article/10.1007/s10458-025-09691-y#ref-CR235)], or lever games [[20](/article/10.1007/s10458-025-09691-y#ref-CR20), [234](/article/10.1007/s10458-025-09691-y#ref-CR234)]. These game types contribute to the creation of new language emergence settings, often designed to target specific aspects or characteristics of language development. They are valuable tools to explore and understand the complexities of EL in different contexts.\n\nIn summary, although many language games have been developed, comparing different games can be complex and understanding the nuances of each game can prove challenging. A promising direction would be for the research community to collectively agree on a standardized subset of these games as benchmarks. By focusing on a representative set of games from different categories, researchers could systematically explore different settings, ensuring that new approaches are rigorously tested and their results are directly comparable across studies. This would accelerate the maturation of the field of EL research, foster collaboration, and enable the community to better identify and address key challenges.\n\n### 5.3 Language prior\n\nEL research occasionally utilizes a concept known as a *language prior* to incorporate structures from human NLs into the emerging language. A language prior is used to impose specific linguistic structures on the emerging language, making it easier to align with human NL and improve interpretability and performance. This prior can be implemented through supervised learning [[14](/article/10.1007/s10458-025-09691-y#ref-CR14), [19](/article/10.1007/s10458-025-09691-y#ref-CR19), [113](/article/10.1007/s10458-025-09691-y#ref-CR113)], also known as injection, or through divergence estimation [[15](/article/10.1007/s10458-025-09691-y#ref-CR15)]. An overview of prior usage in the literature surveyed is given in Table [11](/article/10.1007/s10458-025-09691-y#Tab11) in Appendix [A](/article/10.1007/s10458-025-09691-y#Sec47).\n\nGiven this context, research on EL can be divided into two main areas. The first area focuses on independent situated learning and does not use priors, so that communication and language emerge spontaneously [[18](/article/10.1007/s10458-025-09691-y#ref-CR18)]. The second area explores imitation learning-based approaches, which aim to replicate NL behavior in artificial agents using priors [[103](/article/10.1007/s10458-025-09691-y#ref-CR103)]. However, it is important to note that these approaches differ from LLM because language acquisition in EL is generally task-oriented. In academic literature, the independent situated learning environment is often referred to as the evolution-based approach, while the imitation learning-related approach is commonly known as the acquisition-based approach. The term *evolution* implies starting from scratch, while *acquisition* involves learning an existing language [[168](/article/10.1007/s10458-025-09691-y#ref-CR168)]. The terminology and different approaches are depicted in Fig. [6](/article/10.1007/s10458-025-09691-y#Fig6).\n\nIn addition, the concepts of community and generational learning are closely related [[21](/article/10.1007/s10458-025-09691-y#ref-CR21), [116](/article/10.1007/s10458-025-09691-y#ref-CR116), [130](/article/10.1007/s10458-025-09691-y#ref-CR130)]. In these methods, language emerges through iterative learning across and within agent sub-groups called communities. Generational learning additionally involves older generations of agents training younger ones using previously developed communication as a foundation [[105](/article/10.1007/s10458-025-09691-y#ref-CR105), [117](/article/10.1007/s10458-025-09691-y#ref-CR117)]. Language transfer across groups or generations can be interpreted as an iterative prior. However, this method remains a fully evolutionary approach in the absence of a deliberately designed prior.\n\n### 5.4 Language characteristics\n\nAs discussed in Sect. [2.2](/article/10.1007/s10458-025-09691-y#Sec4), language is a complex, multifaceted system [[88](/article/10.1007/s10458-025-09691-y#ref-CR88), [90](/article/10.1007/s10458-025-09691-y#ref-CR90)]. Therefore, it is essential to establish a comprehensive taxonomy of its properties to provide a unified framework for EL research. This taxonomy will not only facilitate the unambiguous categorization of metrics used in EL studies (cf. Section [6](/article/10.1007/s10458-025-09691-y#Sec20)) but will also enhance the comparability and comprehensibility of approaches and results within the field. As shown previously in Fig. [4](/article/10.1007/s10458-025-09691-y#Fig4), NL can be divided hierarchically into distinct characteristics [[99](#ref-CR99),[100](#ref-CR100),[101](/article/10.1007/s10458-025-09691-y#ref-CR101)]. The following sections provide a categorization of the reviewed publications along these characteristics, occasionally breaking them down into smaller sub-characteristics if relevant.\n\n#### 5.4.1 Phonetics\n\nThe phonetics of a language inherently represents its medium, delineating the constraints of the specific communication channel [[3](/article/10.1007/s10458-025-09691-y#ref-CR3), [101](/article/10.1007/s10458-025-09691-y#ref-CR101)]. These media or channels can be either discrete or continuous; for example, an audio channel is continuous, while a symbolic channel is typically discrete. Regardless of the type, they lay the foundation for the nature of communication. However, for EL research the discrete case is of particular importance, as it closely mirrors NL as we understand it [[35](/article/10.1007/s10458-025-09691-y#ref-CR35)]. Although humans use a continuous phonetic medium for communication, some degree of discretization is essential to establish a common ground for efficient communication [[100](/article/10.1007/s10458-025-09691-y#ref-CR100)].\n\nTable [5](/article/10.1007/s10458-025-09691-y#Tab5) provides an overview of the reviewed papers, categorized according to the continuous or discrete approach. Notably, some papers explore both approaches, providing valuable insights for researchers interested in the basic aspects of phonetics research in EL.\n\n#### 5.4.2 Phonology\n\nPhonology encompasses the actively used vocabulary and determines the part of the medium that is utilized for communication. We identified five different types of vocabulary actively researched, however, some of them are rare to find in the literature. Table [6](/article/10.1007/s10458-025-09691-y#Tab6) summarizes the results of our survey regarding vocabulary types in EL research. One commonly used phonological type in EL is a binary encoding, while an even more prominent type is a token-based vocabulary. However, these two phonological classes are not always distinct, as a token-based vocabulary often builds upon a binary encoded representation [[112](/article/10.1007/s10458-025-09691-y#ref-CR112)].\n\nThe other three types, which are distinct from the two most prominent, are rarely mentioned in the literature reviewed. One of these types involves using NL vocabulary, such as all the words from an English dictionary. While this approach enforces the NL resemblance of the EL, it also drastically limits the emergence and associated benefits [[19](/article/10.1007/s10458-025-09691-y#ref-CR19)]. Essentially, this phonological preset strips the agents of the possibility to shape phonology and morphology. The other two vocabulary types being referred to are sound and graphics. The former enables agents to produce and process sound [[200](/article/10.1007/s10458-025-09691-y#ref-CR200)], while the latter focuses on enabling agents to draw and analyze graphical representations [[104](/article/10.1007/s10458-025-09691-y#ref-CR104), [174](/article/10.1007/s10458-025-09691-y#ref-CR174)]. Both mediums present challenges in ensuring discretization, which may be the reason why they are not as extensively researched in EL.\n\n#### 5.4.3 Morphology\n\nMorphology governs the rules for constructing words and sentences, meaning the overall ability to combine individual elements, also called tokens, into words and to combine those words into sentences [[101](/article/10.1007/s10458-025-09691-y#ref-CR101)]. This is particularly relevant in the field of EL due to the prominent division of existing work based on morphological setup and options. The most significant differentiation is between the use of a fixed or flexible message length. Table [7](/article/10.1007/s10458-025-09691-y#Tab7) demonstrates that much of the existing work employs fixed message lengths, despite this setup not being comparable to NL [[15](/article/10.1007/s10458-025-09691-y#ref-CR15)]. For instance, NL users, such as humans, have the ability to adjust the length of their message to fit their intention, which may vary depending on the audience, medium, or communicative goal. When communicating with colleagues, they may use shorter sentences to be efficient, while more detailed explanations may be used when conversing with friends.\n\nAccordingly, this characteristic can be measured using metrics that assess word formation and vocabulary. Based on the metrics found in the literature, distinct features of language morphology can be quantified. Specifically, this refers to the compression of language and the presence of redundancy or ambiguity.\n\n*Compression* Compression [[105](/article/10.1007/s10458-025-09691-y#ref-CR105)], also known as combinatoriality [[224](/article/10.1007/s10458-025-09691-y#ref-CR224)], refers to the ability of a communication system to combine a small number of basic elements to create a vast range of words that can carry meaning. This feature of discrete communication is crucial in producing comprehensive and flexible communication with limited resources, and is an essential characteristic of NL. We assume that using compressed language is generally favorable for language learners as it reduces the burden of learning [[105](/article/10.1007/s10458-025-09691-y#ref-CR105)].\n\n*Redundancy or ambiguity* In NL, words and phrases can have redundant or ambiguous meanings. Redundancy occurs when multiple words convey the same meaning, while ambiguity arises from a limited vocabulary [[14](/article/10.1007/s10458-025-09691-y#ref-CR14), [114](/article/10.1007/s10458-025-09691-y#ref-CR114)]. The addition of this characteristic in the morphology subsection rather than the semantics subsection may be controversial. We argue that any metric measuring redundancy or ambiguity provides more useful information about the morphology, encompassing the form and size of the vocabulary, than it does about the semantic range and capabilities of the language. However, to quantify redundancy or ambiguity, we must establish semantic meaning first.\n\n#### 5.4.4 Syntax\n\nThe syntax of a language establishes the grammatical rules that govern sentence formation. Consequently, syntax plays a central role in establishing a functional correspondence between emerged language and NL [[245](/article/10.1007/s10458-025-09691-y#ref-CR245)]. This specific characteristic of a discrete language is underrepresented in current EL literature. However, we found two examples in the body of literature discussing syntax in EL. Ueda et al. [[102](/article/10.1007/s10458-025-09691-y#ref-CR102)] introduced a method to examine the syntactic structure of an EL using categorial grammar induction (CGI), which is based on the induction of categorial grammars from sentence-meaning pairs. This method is straightforward in simple referential games. Additionally, van der Wal et al. [[193](/article/10.1007/s10458-025-09691-y#ref-CR193)] introduced unsupervised grammar induction (UGI) techniques for syntax analysis in EL research. We discuss the methods they use to measure and analyze syntax in an EL briefly in Sect. [6.3](/article/10.1007/s10458-025-09691-y#Sec27).\n\n#### 5.4.5 Semantics\n\nSemantics is concerned with the literal meaning of language constructs and is a dominant topic in current EL research, as shown in Table [12](/article/10.1007/s10458-025-09691-y#Tab12) in Appendix [A](/article/10.1007/s10458-025-09691-y#Sec47). EL studies often focus on establishing useful and meaningful communication between agents, making semantics a central feature [[35](/article/10.1007/s10458-025-09691-y#ref-CR35)]. It serves as a crucial tool for distinguishing actual information exchange from mere noise utterances [[168](/article/10.1007/s10458-025-09691-y#ref-CR168), [202](/article/10.1007/s10458-025-09691-y#ref-CR202)]. Given the complexity of capturing the meaning of literal language in a single metric, several features have been introduced to measure the semantics of EL. In particular, these features include grounding, compositionality, consistency, and generalization, as shown in Fig. [7](/article/10.1007/s10458-025-09691-y#Fig7). Table [8](/article/10.1007/s10458-025-09691-y#Tab8) provides an overview of the literature addressing the individual semantic features in EL.\n\n** Grounding** A language is considered grounded when it is deeply intertwined with the environment, for example, when it is tightly bound to environmental concepts and objects [\n\n[149](/article/10.1007/s10458-025-09691-y#ref-CR149),\n\n[184](/article/10.1007/s10458-025-09691-y#ref-CR184),\n\n[246](/article/10.1007/s10458-025-09691-y#ref-CR246)]. Grounding is essential for the interoperability of individuals and is particularly important in NL communication, where meaningful interaction requires shared understanding [\n\n[33](/article/10.1007/s10458-025-09691-y#ref-CR33),\n\n[36](/article/10.1007/s10458-025-09691-y#ref-CR36),\n\n[112](/article/10.1007/s10458-025-09691-y#ref-CR112)]. While in theory, an EL can establish a unique form of grounding using self-emerged concepts distinct from those in NL, deriving a useful metric for such a scenario proves challenging. This difficulty arises from the need to compare ELs to existing and comprehensible grounding principles typically found in NLs [\n\n[50](/article/10.1007/s10458-025-09691-y#ref-CR50),\n\n[112](/article/10.1007/s10458-025-09691-y#ref-CR112),\n\n[127](/article/10.1007/s10458-025-09691-y#ref-CR127)].\n\n** Compositionality** When a language exhibits compositionality, its components can be rearranged or replaced by conceptually equivalent words without changing the overall meaning [\n\n[121](/article/10.1007/s10458-025-09691-y#ref-CR121),\n\n[122](/article/10.1007/s10458-025-09691-y#ref-CR122)]. Compositionality facilitates the construction of higher-level concepts, using conceptual foundations to enable efficient language expression [\n\n[46](/article/10.1007/s10458-025-09691-y#ref-CR46),\n\n[51](/article/10.1007/s10458-025-09691-y#ref-CR51),\n\n[135](/article/10.1007/s10458-025-09691-y#ref-CR135)]. For example, NLs partition concepts such as objects and their attributes to allow compositional constructions [\n\n[18](/article/10.1007/s10458-025-09691-y#ref-CR18),\n\n[155](/article/10.1007/s10458-025-09691-y#ref-CR155)]. As a result, we can describe variations of a single object using different words from the same semantic concept, such as ‘blue towel and ‘red towel for the object towel and the semantic concept of color. Similarly, we can attribute specific properties to different objects using the same phrase, as in ‘green towel and ‘green car. Ultimately, compositionality is beneficial for the learning process [\n\n[36](/article/10.1007/s10458-025-09691-y#ref-CR36),\n\n[122](/article/10.1007/s10458-025-09691-y#ref-CR122)] and promotes efficient and rich language use, even in systems with limited memory capacity [\n\n[18](/article/10.1007/s10458-025-09691-y#ref-CR18),\n\n[108](/article/10.1007/s10458-025-09691-y#ref-CR108),\n\n[119](/article/10.1007/s10458-025-09691-y#ref-CR119)].\n\n** Consistency** Merely having grounded words in a language does not necessarily guarantee its semantic quality. In addition, consistency is essential for a language to convey meaningful and practical information effectively [\n\n[36](/article/10.1007/s10458-025-09691-y#ref-CR36),\n\n[247](/article/10.1007/s10458-025-09691-y#ref-CR247)]. If the words within a language lack consistency in their literal meanings, they will not facilitate effective communication. Therefore, even if a language is semantically grounded and compositional, its utility is compromised if the words exhibit inconsistent literal meanings [\n\n[127](/article/10.1007/s10458-025-09691-y#ref-CR127)]. While words can change their general meaning to fit the context, their literal meaning should remain consistent to keep their usefulness [\n\n[33](/article/10.1007/s10458-025-09691-y#ref-CR33)].\n\n** Generalization** Generalization serves as a cornerstone of NL, allowing humans to communicate about topics ranging from simple to complex, broad to specific, and known to unknown, all with a relatively limited vocabulary [\n\n[25](/article/10.1007/s10458-025-09691-y#ref-CR25),\n\n[135](/article/10.1007/s10458-025-09691-y#ref-CR135)]. A language that excels at generalization enables its users to navigate different levels of complexity, facilitating hierarchical descriptions of concepts and relationships [\n\n[37](/article/10.1007/s10458-025-09691-y#ref-CR37)]. Consequently, generalization and compositionality are closely related, as they both contribute to the flexibility and expressiveness of language [\n\n[36](/article/10.1007/s10458-025-09691-y#ref-CR36),\n\n[117](/article/10.1007/s10458-025-09691-y#ref-CR117),\n\n[168](/article/10.1007/s10458-025-09691-y#ref-CR168)]. This ability to generalize not only enriches communication but also underscores the adaptability and robustness of human language.\n\n#### 5.4.6 Pragmatics\n\nThe final dimension of EL research is pragmatics. This field of study examines how language is employed in context, particularly in interactions, and how it conveys information [[101](/article/10.1007/s10458-025-09691-y#ref-CR101), [160](/article/10.1007/s10458-025-09691-y#ref-CR160), [248](/article/10.1007/s10458-025-09691-y#ref-CR248)]. By evaluating the pragmatics of the linguistic structure, we can ascertain whether EL is itself useful and utilized effectively. While this assessment may be feasible based on rewards in a standard RL setting, integrating communication into such environments increases the complexity. This is because most setups do not separate the agent’s environment interaction from its communication capabilities, thereby expanding the network’s capacity, and making it difficult to attribute an increase in reward directly to EL [[35](/article/10.1007/s10458-025-09691-y#ref-CR35)].\n\nAs outlined in Table [9](/article/10.1007/s10458-025-09691-y#Tab9) and depicted in Fig. [8](/article/10.1007/s10458-025-09691-y#Fig8), five distinct features have been identified: predictability, efficiency, positive signaling, positive listening, and symmetry. These features are essential for assessing the constructive impact and utilization of EL. Understanding how agents employ language is crucial in evaluating its effectiveness and overall benefit.\n\n** Predictability** Predictability is concerned with the assessment of the complexity of the context, including the action space within the environment. When actions exhibit less diversity, it becomes more feasible to coordinate without communication [\n\n[110](/article/10.1007/s10458-025-09691-y#ref-CR110)]. For instance, in a simple grid-based environment where agents have only two possible actions-moving left or right-agents can often achieve their objectives without the need for communication. In such a scenario, the limited action space reduces the necessity for EL, as agents can predict each other’s movements based on past behavior or simple rules. However, in a more complex environment where agents have multiple actions, such as navigating a maze with numerous paths and obstacles, the need for effective communication increases. Here, EL can significantly enhance coordination by allowing agents to share information about their positions, plans, or discoveries, thus improving their overall performance in navigating the maze. Therefore, it is essential to compare the diversity of signaling and context attributes to evaluate the potential benefit of EL.\n\n** Efficiency** Efficiency is a critical aspect considered whenever communication entails a cost. This is particularly true in the context of modeling the emergence of NL and the broader objective of employing EL for human-computer interaction(HCI). In EL settings, the achievement of concise communication is contingent upon the presence of an opportunity cost [\n\n[115](/article/10.1007/s10458-025-09691-y#ref-CR115)]. Without such a cost, there is no incentive to communicate concisely, making EL ineffective as an intermediary for HCI. When communication is accompanied by a cost the necessity for efficiency in communication becomes paramount. In such scenarios, the objective is to minimize the cost while maximizing the effectiveness of communication within a given task.\n\n** Positive signaling** The concept of positive signaling is concerned with the degree of alignment between the observations, knowledge, and experience of the message producer and their communication output [\n\n[29](/article/10.1007/s10458-025-09691-y#ref-CR29)]. The objective is to guarantee the transmission of useful information, or at the very least, information that the speaker can discern through knowledge or observation [\n\n[181](/article/10.1007/s10458-025-09691-y#ref-CR181)]. This feature assumes that all communication should be relevant to something observable, known or tangible to the speaker. Thus, it stipulates that the situational information content of the produced signal is crucial for a language that can be used in a contextually meaningful way.\n\n** Positive listening** Positive listening, in contrast to positive signaling, focuses on the role of the message receiver, to evaluate the active processing of incoming information [\n\n[29](/article/10.1007/s10458-025-09691-y#ref-CR29)]. From a pragmatic point of view, it makes sense to process incoming messages in a meaningful way. However, the definition of meaningful processing is broad. Positive listening as defined in this taxonomy, contrary to earlier work [\n\n[29](/article/10.1007/s10458-025-09691-y#ref-CR29),\n\n[202](/article/10.1007/s10458-025-09691-y#ref-CR202)], does not necessarily require a connection between the incoming message and the subsequent action; it is much more about active processing, which may or may not lead to inclusion in the choice of action. Thus, active engagement followed by rejection or disregard is also considered positive listening in the context of this taxonomy.\n\n** Symmetry** Symmetry in EL is defined as the consistency in language usage among participating agents [\n\n[105](/article/10.1007/s10458-025-09691-y#ref-CR105),\n\n[110](/article/10.1007/s10458-025-09691-y#ref-CR110)]. This concept applies to MARL settings where agents can assume multiple roles, such as message producer and message receiver. Symmetry plays a crucial role in achieving convergence on a shared and aligned EL. For instance, if an agent employs language differently depending on whether it is sending or receiving messages so that words have varying meanings based on the assigned role the EL setting is considered asymmetric. In such instances, rather than learning a collectively and contextually grounded language, agents develop individual protocols specific to their respective roles [\n\n[110](/article/10.1007/s10458-025-09691-y#ref-CR110)]. This would suggest that there is no common language, but rather separate codes that can only be applied to specific combinations and conditions. For this reason, this pragmatic feature is particularly relevant, since the aim of EL is a common language.\n\n### 5.5 Summary of the taxonomy\n\nOur proposed taxonomy systematically categorizes the key features of EL systems, including communication settings, language games, language priors, and language characteristics. The latter is particularly detailed, with sub-characteristics and their features aligned with the major levels of linguistic structure, as previously illustrated in Fig. [4](/article/10.1007/s10458-025-09691-y#Fig4). This comprehensive taxonomy enables a standardized comparison of approaches in the EL literature, highlighting the opportunities and properties associated with individual options and topics in EL research. Specifically, by applying this taxonomy, especially in terms of language characteristics, we can uncover the capabilities and potentials of various EL approaches. This facilitates a more detailed, comparable, and insightful analysis of EL.\n\n## 6 Metrics\n\nThis section provides a comprehensive categorization and review of existing metrics used in EL research. The section is organized along the same categorization used in Sect. [5.4](/article/10.1007/s10458-025-09691-y#Sec12). Note that the categories of phonetics and phonology are excluded from this discussion, as these aspects are predetermined settings in the current EL literature and thus not yet targeted by metrics.\n\nWe begin by introducing the notational system used for all metrics to ensure consistency and facilitate ease of use. We then describe the metrics within each category, detailing the individual metric and adapting it to our notation. For each metric, we provide references to both original sources and additional literature, if available, to enable further exploration beyond the scope of this work. Figure [9](/article/10.1007/s10458-025-09691-y#Fig9) provides a visual summary of the existing metrics and their correspondence to the language characteristics. An extended version including all references for the individual metrics is provided in Fig. [12](/article/10.1007/s10458-025-09691-y#Fig12) in Appendix [B](/article/10.1007/s10458-025-09691-y#Sec48)\n\n### 6.1 Notation\n\nGiven the complexity and variability within the EL field, it is crucial to establish a unified and coherent notation system. In this section we present a standardized mathematical notation designed to be consistent across the various aspects of EL research, thereby facilitating clearer communication and comparison of results within the community. This approach aligns with our broader goal of advancing the field through a common taxonomy that supports the development of measurable and interpretable ELs. Throughout this section we focus on finite and discrete languages, although some of the definitions and metrics discussed here are also applicable to continuous languages. These languages offer a more straightforward mapping to NLs, making them particularly relevant to the study of EL systems.\n\n#### 6.1.1 Definition\n\nIn alignment with the semiotic cycle introduced in Sect. [2.2](/article/10.1007/s10458-025-09691-y#Sec4), our notation is organized into three interconnected spaces: setting, meaning, and language. The setting space encompasses the typical elements of RL, providing the foundational environment in which agents operate. The meaning space incorporates a representation learning endeavor, whereby sensory input is integrated with decision-relevant information to generate a coherent internal representation. Finally, the language space encompasses both the production and comprehension of discrete messages, encapsulating the communication process. These components, illustrated in Fig. [10](/article/10.1007/s10458-025-09691-y#Fig10), will be introduced and explored in detail in the following paragraphs.\n\n** Setting** The overall setting, consisting of the environment, actions, goals, and other typical RL elements, is denoted by \\(\\Omega\\). Let \\(\\xi\\) denote the set of all entities in the system, with an individual entity represented as \\(\\xi _i \\in \\xi\\). Each entity can assume specific roles, such as the sender (\n\n*S*) or receiver (\n\n*R*) in a communication scenario. An entity can assume several roles over the course of the entire communication scenario. However, for an individual message exchange, an entity assumes one specific role. We represent the role of an individual entity\n\n*i*by \\(\\xi _{i,j} \\in \\xi _i\\), where\n\n*j*specifies the role (e.g., \\(j = S\\) or \\(j = R\\)).\n\nEntities interact with their environment \\({\\mathcal {E}}\\) through actions, denoted as *a*, which belong to the set of possible actions *A*, such that \\(a \\in A\\). The action taken by a specific entity \\(\\xi _i\\) is represented as \\(a_{\\xi _i}\\). The state of the environment at any given time is denoted by *s*, which is an element of the state space *S*, so that \\(s \\in S\\). As the system progresses over time, denoted by discrete points in time \\(\\left[ 0, \\dots , t \\right]\\), the sequence of states and actions forms a trajectory \\(\\tau\\), generally expressed as \\(\\tau = \\left\\{ s^0, a^0, \\dots , s^t, a^t \\right\\}\\). It is important to note that the entities described here do not necessarily correspond to autonomous agents in the traditional sense; they could also represent ground truth models, human participants, or abstract constructs that lack the direct interaction capabilities typically associated with agents. Despite this distinction, for the sake of clarity and consistency, we will refer to these entities as agents in the following sections.\n\nGiven the importance of partial observability in EL research [[18](/article/10.1007/s10458-025-09691-y#ref-CR18), [50](/article/10.1007/s10458-025-09691-y#ref-CR50), [172](/article/10.1007/s10458-025-09691-y#ref-CR172), [228](/article/10.1007/s10458-025-09691-y#ref-CR228)], it is essential to consider that agents only have access to their own observations, denoted as \\(o_{\\xi }\\), which are derived from the underlying state *s*. An individual observation \\(o_{\\xi }\\) is an element of the collection of observations of an agent \\(O_{\\xi }\\), which is a subset of the observation space *O*, so that \\(o_{\\xi } \\in O_{\\xi } \\subseteq O\\). In our framework, an observation \\(o_{\\xi }\\) effectively replaces the ‘world model’ component from the traditional semiotic cycle, highlighting the localized and subjective nature of an agent’s perception in partially observable environments.\n\nReferential games (cf. Table [4](/article/10.1007/s10458-025-09691-y#Tab4)) are frequently employed in EL literature. They often operate on individual, static samples that are drawn from a corresponding dataset or distribution. In doing so, they differ from traditional RL setups that emphasize sequential decision-making and environmental interactions over time. In such cases, rather than speaking of a state *s* or an observation *o*, we use the term sample *k*, which is an element of the collection of all samples *K*, so that \\(k \\in K\\). The specific nature of a sample depends on the environment; for example, in an image-based sender-receiver game, the sample would be an image. Each sample is represented by its feature vector *f*, which belongs to the feature space *F*, so that \\(f \\in F\\). The feature vector corresponding to a specific sample *k* is denoted by \\(f_k\\).\n\nIn EL settings, the communicative goal *g* of an agent may differ from the (reinforcement) learning task goal. In addition, depending on the game, the sender and receiver may have distinct goals. These are important factors to consider when evaluating the communicative behavior.\n\n** Meaning** In our notation, the meaning space, denoted by \\(\\Phi\\), serves as the critical intermediary between the setting space and the language space. The meaning space represents the semantic connections derived from the provided information. Each element within this space, represented by a specific meaning vector \\(\\varphi \\in \\Phi\\), captures the essence of concepts or objects as understood by the agent. These meaning vectors are critical to the processes of language comprehension and production, as well as to the processes of conceptualization and interpretation, that allow an agent to effectively use inputs and generate outputs in the setting space (cf. Figure\n\n[10](/article/10.1007/s10458-025-09691-y#Fig10)).\n\nThe representation mappings \\(\\Psi\\) within the meaning space are agent-specific and referred to as \\(\\Psi _{\\text {con}}\\) and \\(\\Psi _{\\text {int}}\\), given in Eq. [1](/article/10.1007/s10458-025-09691-y#Equ1). These mappings enable the transition between an arbitrary space \\(\\chi\\), such as sensory inputs or raw data, and the meaning space, where the data acquires semantic meaning. \\(\\Psi _{\\text {con}}\\) refers to the conceptualization process that transforms raw, uninterpreted data into meaningful representations within \\(\\Phi\\). Conversely, \\(\\Psi _{\\text {int}}\\) denotes the interpretation process that translates these meaning vectors back into the arbitrary space that can represent any external or internal stimuli. These mappings are critical to the agent’s ability to both understand its environment and communicate effectively within it through language that is both grounded in and reflective of the underlying reality with which the agents interact.\n\n** Language** In our proposed framework, a message\n\n*m*belongs to the message space\n\n*M*, such that \\(m \\in M\\). Each message encapsulates semantic and pragmatic content, serving as a vehicle for meaningful communication between agents. A message is composed of individual words\n\n*w*, which are elements of a finite collection\n\n*W*, commonly referred to as vocabulary, lexicon, or dictionary. In this context, each word is considered a semantic unit that carries (intrinsic) meaning. At the lowest level, a word is composed of characters or symbols \\(\\upsilon \\in \\Upsilon\\). These atomic characters, while essential for constructing words, do not independently carry semantic meaning. Instead, they function as elements of a finite set \\(\\Upsilon\\) from which any number of meaningful words can be composed.\n\nBuilding on the formalization from [[119](/article/10.1007/s10458-025-09691-y#ref-CR119)], we describe the message space \\(M_{\\xi }\\) of an agent \\(\\xi\\), which represents the agent’s language capabilities from a compositional standpoint. The message space \\(M_{\\xi } \\subseteq M\\) is composed of a set of messages or strings \\(m_{\\xi }\\), each constructed from words within \\(W_{\\xi }\\), as shown in Eq. [2](/article/10.1007/s10458-025-09691-y#Equ2). Further, each \\(w_{\\xi } \\in m_{\\xi }\\) is composed of a set of characters \\(\\upsilon _{\\xi } \\in \\Upsilon _{\\xi } \\subseteq \\Upsilon\\) utilized by the agent, given by Eq. [3](/article/10.1007/s10458-025-09691-y#Equ3).\n\nA language \\({\\mathcal {L}}\\) encompasses a set of mapping functions that facilitate the transformation between the message space *M* and other arbitrary spaces \\(\\chi\\). These mappings are agent-specific and enable both the production of messages, denoted as \\({\\mathcal {L}}_{\\text {prod}}\\), and the comprehension of messages, denoted as \\({\\mathcal {L}}_{\\text {comp}}\\). This framework aligns with the linguistic level description of the semiotic cycle presented in Fig. [3](/article/10.1007/s10458-025-09691-y#Fig3). Within this context, we formally define a language \\({\\mathcal {L}}\\) in Eq. [4](/article/10.1007/s10458-025-09691-y#Equ4).\n\nThese emerging mapping functions are not necessarily injective, meaning that distinct inputs from the space \\(\\chi\\) could potentially be mapped to an identical message within *M* [[117](/article/10.1007/s10458-025-09691-y#ref-CR117), [120](/article/10.1007/s10458-025-09691-y#ref-CR120)]. Conversely, distinct messages within *M* could also be mapped to the same value in \\(\\chi\\). While this non-injectivity adds a layer of complexity to the expressiveness of the language, it also introduces a degree of flexibility that can be advantageous in certain communication scenarios. For example, it allows for synonymy (where different messages convey the same meaning), which can provide redundancy and flexibility in communication, and homonymy (where the same message may have multiple interpretations depending on context), which can facilitate more nuanced and context-dependent communication. These natural phenomena, though challenging, are well-documented in NLs and are of particular interest in the design and evaluation of artificial communication systems [[101](/article/10.1007/s10458-025-09691-y#ref-CR101), [249](/article/10.1007/s10458-025-09691-y#ref-CR249)]. However, managing these complexities effectively is crucial, as unchecked non-injectivity could lead to ambiguities that complicate communication rather than simplifying it.\n\n#### 6.1.2 Important notes\n\nThe notation presented here is designed to be comprehensible and thorough; however, it may not be directly applicable in all cases to existing works, as these employ different wordings. a lot of existing work uses the term ‘word’, which in our notation describes element carrying semantic meaning, and ‘symbols’, which in our notation serve as fundamental building blocks without inherent semantic meaning, interchangeably [[19](/article/10.1007/s10458-025-09691-y#ref-CR19), [35](/article/10.1007/s10458-025-09691-y#ref-CR35), [134](/article/10.1007/s10458-025-09691-y#ref-CR134), [150](/article/10.1007/s10458-025-09691-y#ref-CR150), [186](/article/10.1007/s10458-025-09691-y#ref-CR186)]. Furthermore, a considerable proportion of existing literature utilizes a multitude of different definitions for concepts such as ‘meaning space’ [[128](/article/10.1007/s10458-025-09691-y#ref-CR128), [138](/article/10.1007/s10458-025-09691-y#ref-CR138), [181](/article/10.1007/s10458-025-09691-y#ref-CR181)], ‘ground-truth oracle’ [[134](/article/10.1007/s10458-025-09691-y#ref-CR134), [148](/article/10.1007/s10458-025-09691-y#ref-CR148)], and other pivotal elements. In our endeavor to establish a unified framework, we have occasionally adopted terminology that differs from that used by the original authors. While this may initially lead to some confusion, we intend to mitigate this by providing transparent and detailed descriptions. Our objective is a consistent application of these concepts across the field of EL research, thereby promoting coherence between different studies. The following sections attempt to align existing research and metrics with the proposed framework. While this alignment has required some linguistic adjustments to existing terminology and procedures, it is important to note that no substantive changes have been made to the underlying methodologies.\n\n### 6.2 Morphology\n\nMorphological metrics aim to evaluate the structure and formation of words within a language, as well as the richness and diversity of its vocabulary. The identified metrics focus on aspects such as language compression, redundancy, and ambiguity. The morphology of a language significantly influences the complexity of language based tasks [[250](/article/10.1007/s10458-025-09691-y#ref-CR250)]. Therefore, the evaluation of morphological features is a crucial component for understanding and evaluating the effectiveness of ELs.\n\n#### 6.2.1 Compression\n\nThe concept of compression within a language refers to its ability to efficiently combine and reuse a limited set of characters to generate a large collection of words or meanings [[105](/article/10.1007/s10458-025-09691-y#ref-CR105), [224](/article/10.1007/s10458-025-09691-y#ref-CR224)]. Several metrics can be used to quantify compression in ELs. A straightforward approach for these metrics is to use statistical measures, as shown in the following paragraphs. These metrics provide insight into the efficiency of the language, indicating how well it minimizes redundancy while maximizing expressiveness. Efficient compression is a key indicator of a communication system, especially in scenarios where resources (such as memory or bandwidth) are constrained.\n\n*Distinct appearances*\n\nThe metric of distinct appearances (\\({DA}\\)) was proposed by Loreto et al. [[224](/article/10.1007/s10458-025-09691-y#ref-CR224)]. It is formalized in Eq. [5](/article/10.1007/s10458-025-09691-y#Equ5) and designed to quantify the capacity of a communication system to name a diverse set of objects or categories using its available symbols [[224](/article/10.1007/s10458-025-09691-y#ref-CR224)]. Specifically, this metric evaluates how frequently characters \\(\\upsilon \\in \\Upsilon\\) are reused across different words or names *w* within the lexicon *W*. By examining the set \\(W_{\\upsilon }\\), which includes all words containing a given character \\(\\upsilon\\), we can assess the system’s flexibility in recombining basic units to generate a broad spectrum of expressions.\n\nA high \\({DA}\\) value, approaching 1, indicates that the characters are highly versatile and reused extensively across different words, thereby reflecting a flexible communication system. Conversely, a low \\({DA}\\) value suggests limited reuse of characters, which may imply constraints in the system’s expressiveness or a less efficient use of its symbolic resources. This metric provides insights into how efficiently a system can balance the trade-off between a compact character set and the richness of its vocabulary.\n\n*Average message length*\n\nAnother way to assess the degree of compression achieved by agents in their communication is to analyze the average message length [[72](/article/10.1007/s10458-025-09691-y#ref-CR72), [114](/article/10.1007/s10458-025-09691-y#ref-CR114), [142](/article/10.1007/s10458-025-09691-y#ref-CR142), [163](/article/10.1007/s10458-025-09691-y#ref-CR163), [169](/article/10.1007/s10458-025-09691-y#ref-CR169), [193](/article/10.1007/s10458-025-09691-y#ref-CR193)]. This metric, which appears for the first time in Choi et al. [[72](/article/10.1007/s10458-025-09691-y#ref-CR72)], captures the typical length of generated messages and provides insight into the efficiency of the EL in terms of information density [[114](/article/10.1007/s10458-025-09691-y#ref-CR114)]. By tracking the average number of words in the messages, we can quantify how effectively the agents compress their language. This metric is computed at the word level, meaning each word within a message is counted. The average message length \\(\\overline{ |m |}\\) for a set of messages *M* is calculated as follows:\n\n*Active words*\n\nThe active words metric, introduced by Lazaridou et al. [[14](/article/10.1007/s10458-025-09691-y#ref-CR14)], complements the average message length by quantifying the diversity of word usage within the vocabulary [[114](/article/10.1007/s10458-025-09691-y#ref-CR114)]. Specifically, this metric measures the variety and utilization of distinct words in a communication system. A high number of active words indicates a diverse vocabulary, reflecting a more complex or redundant EL. Conversely, a lower number suggests that the communication system relies on a limited set of words, which may indicate a more efficient and compressed language with less synonyms [[18](/article/10.1007/s10458-025-09691-y#ref-CR18)]. This metric is widely used in the literature [[14](/article/10.1007/s10458-025-09691-y#ref-CR14), [18](/article/10.1007/s10458-025-09691-y#ref-CR18), [32](/article/10.1007/s10458-025-09691-y#ref-CR32), [114](/article/10.1007/s10458-025-09691-y#ref-CR114), [116](/article/10.1007/s10458-025-09691-y#ref-CR116), [119](/article/10.1007/s10458-025-09691-y#ref-CR119), [137](/article/10.1007/s10458-025-09691-y#ref-CR137), [140](/article/10.1007/s10458-025-09691-y#ref-CR140), [142](/article/10.1007/s10458-025-09691-y#ref-CR142), [144](/article/10.1007/s10458-025-09691-y#ref-CR144), [151](/article/10.1007/s10458-025-09691-y#ref-CR151), [187](/article/10.1007/s10458-025-09691-y#ref-CR187)]. Mathematically, the active word value \\({AW}\\) for an agent \\(\\xi _{i}\\) can be defined as the size of the collection of words actively used by the agent \\(W_{\\xi _{i}}\\), as given in Eq. [7](/article/10.1007/s10458-025-09691-y#Equ7). In multi-agent setups, this metric can be averaged across all agents to provide a collective measure of vocabulary diversity within the joint system.\n\n#### 6.2.2 Redundancy or ambiguity\n\nRedundancy in language occurs when multiple words are associated with the same meaning, providing alternative expressions for the same concept. Conversely, ambiguity occurs when a single word is associated with multiple meanings, creating the potential for different interpretations depending on the context. Both redundancy and ambiguity are characteristic features of NLs, reflecting the complexity and flexibility inherent in human communication [[114](/article/10.1007/s10458-025-09691-y#ref-CR114), [132](/article/10.1007/s10458-025-09691-y#ref-CR132)].\n\n*Perplexity*\n\nPerplexity, introduced by Havrylov and Titov [[15](/article/10.1007/s10458-025-09691-y#ref-CR15)], measures how often a word was used in a message to describe the same object [[72](/article/10.1007/s10458-025-09691-y#ref-CR72), [114](/article/10.1007/s10458-025-09691-y#ref-CR114)]. “A lower perplexity shows that the same |words| are consistently used to describe the same objects.” [[114](/article/10.1007/s10458-025-09691-y#ref-CR114)]. Mathematically, \\(P \\left( w | \\varphi \\right)\\) represents the probability or score of a word for a specific concept or meaning, e.g., derived from an affine transformation of the sender’s hidden state [[114](/article/10.1007/s10458-025-09691-y#ref-CR114)] or from a ground truth label [[72](/article/10.1007/s10458-025-09691-y#ref-CR72)]. Thus, perplexity, given in Eq. [8](/article/10.1007/s10458-025-09691-y#Equ8), quantifies the predictability of word usage, with lower values reflecting a less redundant communication system. It is usually calculated based on a sampled set of meanings \\(\\Phi _{\\text {test}}\\) for which the word probability can be generated.\n\n*Singular value decomposition*\n\nAnother approach to quantitatively assess the redundancy of the vocabulary used in a communication system is outlined by Lazaridou et al. [[14](/article/10.1007/s10458-025-09691-y#ref-CR14)]. This method involves constructing a matrix where the rows correspond to distinct meanings, the columns represent individual words, and the matrix entries indicate the frequency with which each word is used for a given meaning. The rows are thus constructed based on a predefined ground truth classification. By applying Singular Value Decomposition (SVD) to this matrix, we can examine the dimensionality of the underlying communication strategy. If the communication system relies on a limited set of highly synonymous words, we would expect the SVD to reveal a low-dimensional structure. Conversely, a higher-dimensional decomposition would indicate a more diverse use of vocabulary, reflecting a potentially less synonymous and more redundant language.\n\n*Message distinctness*\n\nMessage distinctness evaluates the linguistic representation of distinct features and thus aims to quantify ambiguity [[72](/article/10.1007/s10458-025-09691-y#ref-CR72), [114](/article/10.1007/s10458-025-09691-y#ref-CR114), [168](/article/10.1007/s10458-025-09691-y#ref-CR168), [225](/article/10.1007/s10458-025-09691-y#ref-CR225)]. The metric, first suggested in Lazaridou et al. [[168](/article/10.1007/s10458-025-09691-y#ref-CR168)] and Choi et al. [[72](/article/10.1007/s10458-025-09691-y#ref-CR72)], quantifies the diversity of messages generated by the agent by assessing how well it differentiates between various inputs. Specifically, message distinctness \\({MD}\\) is calculated as the ratio of the number of unique messages generated within a batch (cf. [9](/article/10.1007/s10458-025-09691-y#Equ9)) to the batch size (cf. [10](/article/10.1007/s10458-025-09691-y#Equ10)). A higher message distinctness indicates less ambiguity of the language.\n\n### 6.3 Syntax\n\nDespite the significance of structural properties in ELs, particularly regarding their syntax and its relation to semantics, research in this area remains limited [[245](/article/10.1007/s10458-025-09691-y#ref-CR245)]. Recurrent syntactical patterns are central to the robustness and versatility of NLs [[50](/article/10.1007/s10458-025-09691-y#ref-CR50)]. Exploring these properties within the context of EL could provide valuable insights into their development and alignment with NL.\n\n#### 6.3.1 Syntax tree\n\nVan der Wal et al. [[193](/article/10.1007/s10458-025-09691-y#ref-CR193)] introduced unsupervised grammar induction (UGI) techniques for syntax analysis in EL research, describing a two-stage approach to deriving grammar and syntax. The first phase involves the induction of unlabeled constituent tree structures, explained below, and the labeling of these structures. The second phase extracts a probabilistic context-free grammar (PCFG) from the labeled data. Two methods were compared for constituency structure induction: the Common Cover Link (CCL), a pre-neural statistical parser that makes assumptions about NL such as the Zipfian distribution, and the Deep Inside-Outside Recursive Auto-encoder (DIORA), a neural parser. For the labeling process, Van der Wal et al. [[193](/article/10.1007/s10458-025-09691-y#ref-CR193)] used Bayesian Model Merging (BMM), to consolidate probabilistic models to label the induced syntax trees.\n\nIn syntax trees, the structure of the language is represented in a hierarchical manner, where nodes represent grammatical constructs (such as sentences, phrases, and words) and edges represent the rules or relationships that connect these constructs. Analysis of these trees helps to understand how well grammar induction methods match the true syntactic nature of ELs. There are several metrics associated with syntax trees that are used to measure the complexity of the grammar [[193](/article/10.1007/s10458-025-09691-y#ref-CR193)]. First, tree depth measures the maximum distance from the root of the tree to its deepest leaf. Tree depth reflects the hierarchical complexity of the grammar. Shallow trees indicate a simpler grammar, while deeper trees suggest a more complex syntactic structure. Second, the number of unique preterminal groups is a metric that counts the different sets of preterminals (intermediate symbols) that appear to the right of production rules in a grammar. A larger number of unique preterminal groups indicates a richer and more diverse syntactic organization, suggesting that the grammar can generate a greater variety of structures.\n\n#### 6.3.2 Categorical grammar induction\n\nUeda et al. [[102](/article/10.1007/s10458-025-09691-y#ref-CR102)] proposed a novel approach for analyzing the syntactic structure of ELs using Categorial Grammar Induction (CGI). This technique focuses on deriving categorial grammars from message-meaning pairs, making it particularly well-suited for simple referential or signaling games.\n\nIn this method, derivation trees are constructed using lexical entries and application rules, mapping messages to atomic syntactical representations. Given that multiple derivations might exist for a single message, “the most likely derivation [is selected] using a log-linear model” [[102](/article/10.1007/s10458-025-09691-y#ref-CR102)]. CGI is particularly valuable for assessing the syntactic structure of an EL using the generated trees.\n\n### 6.4 Semantics\n\nCapturing the semantic properties of ELs is inherently complex, making it difficult to encapsulate nuances in a single metric. To address this, several key features have been introduced, including grounding, compositionality, consistency, and generalization. These are important because agents can develop representations that are well aligned with task performance but fail to capture the underlying conceptual properties [[16](/article/10.1007/s10458-025-09691-y#ref-CR16)]. Thus, an EL might enable successful task completion without truly encoding semantic meaning. Therefore, evaluating these semantic features is essential to evaluate the value and validity of the EL.\n\n#### 6.4.1 Grounding\n\nGrounding is essential for the development of meaning and for systematic generalization to novel combinations of concepts [[123](/article/10.1007/s10458-025-09691-y#ref-CR123)]. It forms the basis of human-agent communication [[33](/article/10.1007/s10458-025-09691-y#ref-CR33)], and without proper grounding, meaningful communication cannot be effectively learned [[112](/article/10.1007/s10458-025-09691-y#ref-CR112)]. However, in general dialog settings, grounding does not emerge naturally without specific regularization techniques [[127](/article/10.1007/s10458-025-09691-y#ref-CR127)]. The grounding problem, which concerns how words acquire semantic meaning, is central to this challenge [[184](/article/10.1007/s10458-025-09691-y#ref-CR184)]. Thus, grounding metrics are vital as they largely define the usability of a language. However, a significant limitation of these metrics is their reliance on some form of oracle or a NL-grounded precursor [[15](/article/10.1007/s10458-025-09691-y#ref-CR15), [26](/article/10.1007/s10458-025-09691-y#ref-CR26), [134](/article/10.1007/s10458-025-09691-y#ref-CR134), [207](/article/10.1007/s10458-025-09691-y#ref-CR207)].\n\n*Divergence*\n\nHavrylov and Titov [[15](/article/10.1007/s10458-025-09691-y#ref-CR15)] proposed a weak form of grounding. Weak grounding means that the same word can correspond to completely different concepts in the induced EL and NL. They used the Kullback–Leibler divergence \\({D_{KL}}\\) (cf. Equation [11](/article/10.1007/s10458-025-09691-y#Equ11)) of an EL and a NL distribution to ensure that the statistical properties of EL messages resemble those of NL. They introduced this approach as an indirect supervision measure during training but it can also serve as a metric for evaluating the alignment between EL and NL. For a given sample *k* and the message \\(m_{\\xi _{S}}\\) produced by the sender, the grounding divergence \\({G_{Div}}\\) calculation is shown in Eq. [12](/article/10.1007/s10458-025-09691-y#Equ12). Since the true NL distribution \\(P_{\\text {NL}} \\left( m_{\\xi _{S}} \\right)\\) is inaccessible, a language model is trained to approximate this distribution. The KL divergence yields a value in the range \\(\\left[ 0, \\infty \\right)\\), with lower values indicating a closer resemblance between the generated messages and NL.\n\n*Purity*\n\nPurity, proposed by Lazaridou et al. [[14](/article/10.1007/s10458-025-09691-y#ref-CR14)], is a metric used to assess the alignment between predefined semantic categories and those observed in an EL. It measures the effectiveness of a communication system in consistently mapping signals or words to specific concepts [[198](/article/10.1007/s10458-025-09691-y#ref-CR198)]. Thus, purity quantifies the extent to which the clustering of words reflects meaningful and coherent categories, as determined by ground-truth labels. To assess purity, we first form clusters by grouping samples based on the most frequently activated words to describe them. The quality of these clusters is then evaluated using the purity metric, which calculates the proportion of labels in each cluster that match the majority category of that cluster. A higher purity score indicates that the sender is producing words that are semantically aligned with predefined categories, as opposed to arbitrary or agnostic symbol usage, as demonstrated in [[14](/article/10.1007/s10458-025-09691-y#ref-CR14)]. However, this metric requires the existence of predefined ground-truth labels, limiting its applicability in scenarios where such labels are unavailable or ambiguous.\n\nFormally, given a set of clusters \\(\\{ C_{k} \\}\\) where each cluster of samples \\(C_{k}\\) has a corresponding majority ground-truth label \\(c_{k}\\), the purity of a cluster \\(C_{k}\\) is defined as:\n\nHere, \\(\\{ w \\mid w \\in C_{k} \\}\\) is the collection of all words used to describe the samples in the cluster and \\(\\{ w_{c} \\mid w_{c} \\in C_{k} \\wedge w_{c} = c_{k} \\}\\) is the collection of words within the cluster that fit the majority label of that cluster. The purity metric ranges from 0 to 1, where a value of 1 indicates perfect alignment with the ground-truth categories.\n\n*Representational similarity analysis*\n\nRepresentational Similarity Analysis (RSA) emerged in the field of neuroscience and was proposed by Kriegeskorte et al. [[251](/article/10.1007/s10458-025-09691-y#ref-CR251)]. It has since been adapted for the evaluation of the similarity of neural representations across different modalities, including computational models and brain activity patterns. This technique has been effectively applied in EL research [[16](/article/10.1007/s10458-025-09691-y#ref-CR16), [114](/article/10.1007/s10458-025-09691-y#ref-CR114), [216](/article/10.1007/s10458-025-09691-y#ref-CR216)], where the focus shifts from analyzing neural activity to exploring the structural relationships between different embedding spaces. For example, RSA has been employed to compare the similarity of embedding space structures between input, sender, and receiver in a referential game [[16](/article/10.1007/s10458-025-09691-y#ref-CR16), [114](/article/10.1007/s10458-025-09691-y#ref-CR114)]. By calculating pairwise cosine similarities within these spaces and then computing the Spearman correlation between the resulting similarity vectors, we can calculate an RSA score that measures the global agreement between these spaces, independent of their dimensionality. The agreement of an agent’s embedding space with the input embedding space as such provides an intuitive measure of the grounding of the EL.\n\nThis approach offers the advantage of being applicable to heterogeneous agents and arbitrary input spaces. In our framework, this corresponds to any ground truth structured embedding \\(e \\left( o_{\\xi } \\right)\\) of an agent’s observation \\(o_{\\xi }\\) and its internal meaning representation \\(\\varphi _{\\xi }\\). Nevertheless, a significant limitation is the necessity for an embedding, which provides a structured description of the observation oriented towards a ground truth, for example, based on a NL model. Furthermore, RSA is not directly applicable to the language itself, particularly for discrete languages. Instead, it operates at the level of earlier meaning representations. Despite this, RSA provides valuable insights into whether the EL can be grounded by evaluating the grounding of the meaning space.\n\nThe methodology of [[16](/article/10.1007/s10458-025-09691-y#ref-CR16)] utilizes a collection *K* of samples, comprising *k* observations, images, or feature vectors, to compute representational similarities between input and meaning space. First, we generate input or ground truth embeddings \\(e_{GT} = e \\left( o_{\\xi } \\right)\\) using an appropriate model and generate the corresponding internal representations \\(\\varphi _{\\xi }\\) from the appropriate architecture part of agent \\(\\xi\\). Next, we compute pairwise similarities within each embedding space, denoted as \\({S_{e}}\\) for the ground truth embeddings and \\({S_{\\varphi }}\\) for the agent representations, typically using cosine similarity \\(S_{cos}\\) as defined in Eq. [14](/article/10.1007/s10458-025-09691-y#Equ14). This yields a similarity vector of size \\(N \\cdot (N - 1)\\) for each embedding space. The vectors are converted into rank vectors \\({R}\\left( {S_{e}} \\right)\\) and \\({R}\\left( {S_{\\varphi }} \\right)\\). Finally, we calculate the Spearman rank correlation \\(\\rho\\) [[252](/article/10.1007/s10458-025-09691-y#ref-CR252)] between the ranked similarity vectors, using the covariance \\({cov}\\) and standard deviation \\(\\sigma\\), to assess the alignment between the input and agent representation spaces (cf. Equation [15](/article/10.1007/s10458-025-09691-y#Equ15)). The correlation coefficient \\(\\rho\\) takes on values between \\(-1\\) and 1. A high absolute value of this coefficient indicates a strong alignment between the two variables.\n\n#### 6.4.2 Compositionality\n\nIn EL research, achieving compositionality often requires deliberate guidance, as it does not naturally arise without specific interventions [[119](/article/10.1007/s10458-025-09691-y#ref-CR119)]. For instance, training models on diverse tasks and varying environmental configurations can facilitate the development of compositional structures. This occurs as atomic concepts, learned in simpler contexts, are recombined in more complex scenarios [[18](/article/10.1007/s10458-025-09691-y#ref-CR18)]. When a language is truly compositional, its components can be systematically rearranged or substituted with conceptually equivalent components without altering the overall meaning [[121](/article/10.1007/s10458-025-09691-y#ref-CR121), [122](/article/10.1007/s10458-025-09691-y#ref-CR122)].\n\nThe formalization of compositionality can be framed using the comprehension \\({\\mathcal {L}}_{\\text {comp}}\\) or production \\({\\mathcal {L}}_{\\text {prod}}\\) function that map expressions from a language \\({\\mathcal {L}}\\) to a space of meanings \\(\\Phi\\) or vice versa [[138](/article/10.1007/s10458-025-09691-y#ref-CR138)]. For example, the function \\({\\mathcal {L}}_{\\text {comp}}: {\\mathcal {L}} \\rightarrow \\Phi\\) reflects “all the things that the language can denote” [[138](/article/10.1007/s10458-025-09691-y#ref-CR138)]. A language is compositional if these functions act as a homomorphism, e.g., there exist binary operators \\(\\circ\\) on \\({\\mathcal {L}}_{\\text {comp}}\\) and \\(\\times\\) on \\(\\Phi\\) such that for any expression composed of two constituents \\(m_1\\) and \\(m_2\\) in \\({\\mathcal {L}}\\), the following condition holds:\n\n*Topographic similarity*\n\nTopographic similarity (topsim), originally proposed by Brighton and Kirby [[253](/article/10.1007/s10458-025-09691-y#ref-CR253)] and first applied to EL by Lazaridou et al. [[168](/article/10.1007/s10458-025-09691-y#ref-CR168)], is a metric designed to quantify the structural alignment between the internal representations of meanings and the corresponding generated messages in a communication system. Unlike RSA (cf. Section [6.4.1](/article/10.1007/s10458-025-09691-y#Sec31)), which compares the meaning space against a ground truth, topsim focuses on the internal alignment within an agent’s meaning and message spaces. “The intuition behind this measure is that semantically similar objects should have similar messages” [[168](/article/10.1007/s10458-025-09691-y#ref-CR168)]. It has become a widely used metric in the study of EL, as depicted in Fig. [12](/article/10.1007/s10458-025-09691-y#Fig12) in Appendix [B](/article/10.1007/s10458-025-09691-y#Sec48)).\n\nTo compute topsim, we start by sampling *k* meaning representations denoted by \\(\\varphi\\), typically embedded feature vectors, from the meaning space \\(\\Phi\\). Let \\(\\phi = \\{ \\varphi _{1}, \\dots , \\varphi _{k} \\}\\) denote the collection of these samples, with \\(\\varphi \\in \\Phi\\). Using the sender’s policy \\(\\pi _{\\xi _{S}}^{M}\\), we generate corresponding messages \\(m_{i} = \\pi _{\\xi _{S}}^{M}(\\varphi _i)\\) for each sample \\(\\varphi _{i} \\in \\phi\\). We then compute distances within the meaning and language spaces using suitable distance functions for language \\(\\Delta _{{\\mathcal {L}}}\\) and meaning \\(\\Delta _{\\Phi }\\) space. The choice of distance function \\(\\Delta\\) depends on the nature of the spaces involved. For discrete communication, typical choices include Hamming [[254](/article/10.1007/s10458-025-09691-y#ref-CR254)] or Levenshtein [[255](/article/10.1007/s10458-025-09691-y#ref-CR255)] distance, whereas for continuous spaces, cosine or Euclidean distance are often used [[51](/article/10.1007/s10458-025-09691-y#ref-CR51)]. Finally, we compute the Spearman rank correlation \\(\\rho\\) [[252](/article/10.1007/s10458-025-09691-y#ref-CR252)] using the ranked distances to get the topsim value of the language:\n\n*Positional disentanglement*\n\nPositional Disentanglement (posdis) was introduced by Chaabouni et al. [[122](/article/10.1007/s10458-025-09691-y#ref-CR122)] as a metric to evaluate the extent to which words in specific positions within a message uniquely correspond to particular attributes of the input. This metric operates on an order-dependent strategy, which is normalized by the message length and calculated as the ratio of mutual information to entropy. The underlying assumption is that the language leverages positional information to disambiguate words, such that “each position of the message should only be informative about a single attribute” [[122](/article/10.1007/s10458-025-09691-y#ref-CR122)]. Thus, “posdis assumes a message whose length equals the number of attributes in the input object, and where each message token, in a specific position, represents a single attribute” [[180](/article/10.1007/s10458-025-09691-y#ref-CR180)]. This order-dependence is a characteristic feature of NL structures and is essential for the emergence of sophisticated syntactic patterns [[122](/article/10.1007/s10458-025-09691-y#ref-CR122)].\n\nThe metric begins by identifying each word \\(w_p\\) at position *p* in a message *m*, where *f* represents the feature vector of the ground truth. The mutual information \\(I(w_p, f_i)\\) between \\(w_p\\) and a specific feature \\(f_i\\) is calculated to determine how informative the position *p* is about the attribute \\(f_i\\) (cf. Equation [18](/article/10.1007/s10458-025-09691-y#Equ18)). The two most informative features \\(f_i^{1}\\) and \\(f_i^{2}\\) are then identified based on the mutual information value (cf. Equation [19](/article/10.1007/s10458-025-09691-y#Equ19)). To quantify positional disentanglement, the mutual information difference between the two most informative features is normalized by the entropy \\(H(w_p)\\) of the word at position *p*, as defined in Eq. [20](/article/10.1007/s10458-025-09691-y#Equ20) and Eq. [21](/article/10.1007/s10458-025-09691-y#Equ21). Finally, the overall posdis value for a language is calculated by averaging the posdis scores across all positions in the messages within the dataset. For messages of varying lengths, the posdis score is normalized by the average message length \\(\\overline{|m |}\\), as given in Eq. [22](/article/10.1007/s10458-025-09691-y#Equ22).\n\n*Bag of symbols disentanglement*\n\nBag of Symbols Disentanglement (bosdis) is a metric introduced by Chaabouni et al. [[122](/article/10.1007/s10458-025-09691-y#ref-CR122)] to assess the degree to which words in a language unambiguously correspond to different input elements, regardless of their position within a message. While positional disentanglement (posdis) relies on the assumption that positional information is crucial for disambiguating words (cf. Section [6.4.2](/article/10.1007/s10458-025-09691-y#Sec32)), bosdis relaxes this assumption and captures the intuition behind a permutation-invariant language. In such a language, the order of words is irrelevant, and only the frequency of words carries meaning [[122](/article/10.1007/s10458-025-09691-y#ref-CR122)]. The metric normalizes the mutual information between symbols and input features by the entropy summed over the entire vocabulary. This approach maintains the requirement that each symbol uniquely refers to a distinct meaning, but shifts the focus to symbol counts as the primary informative element.\n\n*Tree reconstruct error*\n\nTree Reconstruct Error (TRE) assumes prior knowledge of the compositional structure within the input data, enabling the construction of tree-structured derivations [[134](/article/10.1007/s10458-025-09691-y#ref-CR134)]. As defined by Andreas [[134](/article/10.1007/s10458-025-09691-y#ref-CR134)], a language is considered compositional if it functions as a homomorphism from inputs to their representations. The compositionality of a language should be evaluated by identifying representations that allow an explicitly compositional language to closely approximate the true underlying structure [[134](/article/10.1007/s10458-025-09691-y#ref-CR134)]. One metric for this assessment is TRE, which quantifies the discrepancy between a compositional approximation and the actual structure, using a composition function and a distance metric. A TRE value of zero indicates perfect reproduction of compositionality.\n\nThe compositional nature of a sender’s language is affirmed if there exists an assignment of representations to predefined primitives (e.g., categories, concepts, or words) such that for each input, the composition of primitive representations according to the oracle’s derivation precisely reproduces the sender’s prediction [[134](/article/10.1007/s10458-025-09691-y#ref-CR134)]. TRE specifically measures the accuracy with which a given communication protocol can be reconstructed while adhering to the compositional structure of the derivation or embedding of the input \\(e \\in E\\) [[51](/article/10.1007/s10458-025-09691-y#ref-CR51)].\n\nOne of the key advantages of the TRE framework is its flexibility across different settings, whether discrete or continuous. It allows for various choices of compositionality functions, distance metrics, and other parameters. However, this flexibility comes with challenges, including the requirement for an oracle-provided ground truth and the necessity of pre-trained continuous embeddings.\n\nIt is defined in a way that allows the choice of the distance metric \\(\\delta\\) and the compositionality function \\(\\circ\\) to be determined by the evaluator [[134](/article/10.1007/s10458-025-09691-y#ref-CR134)]. When the exact form of the compositionality function is not known a priori, it is common to define \\(\\circ\\) with free parameters, as suggested by Andreas [[134](/article/10.1007/s10458-025-09691-y#ref-CR134)], treating these parameters as part of the learned model and optimizing them jointly with the other parameters \\(\\eta\\). However, care must be taken when learning the compositional function to avoid degenerate solutions [[134](/article/10.1007/s10458-025-09691-y#ref-CR134)].\n\nGiven a data sample *k* from the dataset *K* (\\(k \\in K\\)) and a corresponding message *m* from the set of all possible messages *M* (\\(m \\in M\\)), TRE requires a distance function \\(\\delta\\) and learnable parameters \\(\\eta\\). Additionally, it employs a compositionality function \\(\\circ\\) and pre-trained embeddings of ground truth, denoted by \\(e \\in E\\), which can be obtained using models like word2vec.\n\nThe functions involved in the TRE calculation are as follows:\n\n-\nPre-trained ground truth oracle (e.g., word2vec): \\({\\mathcal {E}}: K \\rightarrow E\\)\n\n-\nLearned language speaker: \\(\\xi _S: K \\rightarrow M\\)\n\n-\nLearnable approximation function for TRE: \\({\\widetilde{f}}_{\\eta }: E \\rightarrow M\\)\n\nIn the discrete message setting, which is the focus here, a discrete distance metric such as \\(L_1\\) is typically chosen, along with a compositional function \\(\\circ\\) defined by a weighted linear combination [[51](/article/10.1007/s10458-025-09691-y#ref-CR51), [134](/article/10.1007/s10458-025-09691-y#ref-CR134)]:\n\nTo compute the TRE, an optimized approximation function \\({\\widetilde{f}}_{\\eta }\\) is required. This function must satisfy two key properties: embedding consistency, meaning that the learned parameters \\(\\eta\\) are specific to an embedding, and compositionality, which ensures that the function behaves according to:\n\nThe optimization process involves minimizing the distance between the output of the learned language speaker \\(\\xi _{S}(k_{i})\\) and the approximation function \\({\\widetilde{f}}_{\\eta }(e_{i})\\), based on the ground truth:\n\nWith the optimized parameters \\(\\eta ^{*}\\), TRE can be calculated at two levels: the datum level, which assesses individual instances:\n\nand the dataset level, which measures the overall communication performance across the dataset:\n\n*Conflict count*\n\nConflict count, introduced by Kuciński et al. [[166](/article/10.1007/s10458-025-09691-y#ref-CR166)], is designed to quantify the extent to which the assignment of features to words in a language deviates from the word’s principal meaning. This metric is particularly useful in scenarios where the language employs synonyms, as it accounts for the possibility of multiple words referring to the same concept.\n\nThe conflict count metric operates under the assumption that the number of concepts or features \\(f_{i}\\) given in a feature vector *f* of a sample *k* in the collection of samples *K* is equal to the message length \\(|m |\\), and that there exists a one-to-one mapping between a concept \\(f_i \\in f\\) and a word \\(w \\in W\\). The metric counts how frequently this one-to-one mapping is violated, with a value of 0 indicating no conflicts and, therefore, high compositionality. An advantage of this metric is its ability to accommodate redundancy in the language. However, it also has limitations, such as the assumption that the number of features or attributes equals the message length, i.e., \\(|f |= |m |\\). Additionally, because conflict count assumes the number of concepts in a derivation to be equal to the message length, it becomes undefined for languages or protocols that violate this assumption, such as those involving negation or context-sensitive constructions presented in [[51](/article/10.1007/s10458-025-09691-y#ref-CR51)].\n\nThe primary objective of conflict count is to quantify the number of times the mapping from a word *w* to its principal meaning \\(\\varphi _w\\) is violated. This requires the assumption that a mapping \\(\\alpha\\) exists from the position *p* of word *w* in message *m* to an individual feature in feature vector *f*, such that:\n\nIn this framework, the meaning of a word, denoted by \\(\\varphi _{w}\\), is determined by both the word *w* itself and its position *p* within the message. This meaning corresponds to a specific instance *j* of a particular feature *i* within the feature vector *f*, such that \\(f_{i,j} = \\varphi (w,p)\\).\n\nThe process of calculating the conflict count begins by identifying the principal meaning of each word-position pair:\n\nusing the count function:\n\nwhere \\(m \\left( k \\right)\\) is the message produced for sample *k* and \\({pos}_{m} \\left( w \\right)\\) computes the position of word *w* in message *m*.\n\nFinally, the conflict count value \\({conf}\\) is determined by finding the mapping \\(\\alpha\\) that minimizes the score:\n\nwhere the score function is defined as:\n\n#### 6.4.3 Consistency\n\nFor a language to be effective, the meaning of each word must be consistent across different contexts. Inconsistent word meanings can render a language practically useless, even if the language is semantically grounded and exhibits compositional properties [[127](/article/10.1007/s10458-025-09691-y#ref-CR127)]. In dialogue settings, particularly in the absence of explicit regularization mechanisms, words often fail to maintain consistent groundings across different instances, leading to ambiguity and reduced communicative effectiveness [[127](/article/10.1007/s10458-025-09691-y#ref-CR127)]. Thus, it is crucial to carefully monitor this language characteristic in EL settings.\n\n*Mutual information*\n\nConsistency in language can be quantitatively assessed by examining the mutual information between messages and their corresponding input features. Ideally, a consistent language will exhibit a high degree of overlap between messages and features, leading to a high mutual information value, indicating strong correspondence [[150](/article/10.1007/s10458-025-09691-y#ref-CR150)].\n\nFormally, mutual information between two random variables, say *X* and *Y*, with joint distribution \\(P_{(X,Y)}\\) and marginal distributions \\(P_X\\) and \\(P_Y\\), is defined as the Kullback–Leibler divergence \\({D_{KL}}\\) (see Eq. [11](/article/10.1007/s10458-025-09691-y#Equ11)) between the joint distribution and the product of the marginals:\n\nIn the context of discrete communication, where both messages and sample features are represented as discrete variables, the mutual information between the set of messages *M* and the set of features *F* is computed using a double summation over all possible message-feature pairs:\n\nwhere \\(P_{ \\left( M, F \\right) } \\left( m, f \\right)\\) is the joint probability of message *m* and feature *f*, and \\(P_{M} \\left( m \\right)\\) and \\(P_{F} \\left( f \\right)\\) are the marginal probabilities of *m* and *f*, respectively.\n\n*Correlation*\n\nVarious studies employ different statistical techniques to measure consistency using correlations [[114](/article/10.1007/s10458-025-09691-y#ref-CR114), [116](/article/10.1007/s10458-025-09691-y#ref-CR116), [173](/article/10.1007/s10458-025-09691-y#ref-CR173), [176](/article/10.1007/s10458-025-09691-y#ref-CR176), [184](/article/10.1007/s10458-025-09691-y#ref-CR184), [195](/article/10.1007/s10458-025-09691-y#ref-CR195)]. For example, consistency within a language system can be quantified by analyzing the variability of words produced for a given sample *k*. Specifically, given the set of all words representing *k*, a heatmap is generated using the mean of this set. The sharpness of the heatmap is then quantified by computing the Variance of the Laplacian (VoL). The average consistency score is obtained by dividing the VoL of the heatmap by the count of all samples considered, as introduced by Verma and Dhar [[195](/article/10.1007/s10458-025-09691-y#ref-CR195)].\n\nAdditionally, Mul et al. [[176](/article/10.1007/s10458-025-09691-y#ref-CR176)] explored the correlation between messages and actions as well as between messages and salient properties of the environment. The analysis reveals correlations by examining the conditional probability distribution of actions given the messages produced by a pretrained or fine-tuned receiver. This distribution, denoted as \\(P \\left( a \\mid m \\right)\\), was visualized using bin bar plots to highlight the prominent correlations [[176](/article/10.1007/s10458-025-09691-y#ref-CR176)]. Similarly, the relationship between input and messages is analyzed by examining the conditional distribution of a pretrained sender’s messages given the observational input, represented as \\(P \\left( m \\mid o \\right)\\) [[176](/article/10.1007/s10458-025-09691-y#ref-CR176)].\n\n*Coherence*\n\nCoherence is often assessed through context independence, a metric initially proposed by Bogin et al. [[33](/article/10.1007/s10458-025-09691-y#ref-CR33)]. Context independence examines whether words within a language maintain consistent semantics across varying contexts. However, context independence may be considered restrictive, particularly in languages where synonyms are prevalent [[29](/article/10.1007/s10458-025-09691-y#ref-CR29), [164](/article/10.1007/s10458-025-09691-y#ref-CR164)]. The context independence metric aims to measure the alignment between words \\(w \\in W\\) and features \\(f \\in F\\) of the input samples by analyzing their probabilistic associations. Specifically, \\(P \\left( w \\mid f \\right)\\) denotes the probability that a word *w* is used when a feature *f* is present, while \\(P \\left( f \\mid w \\right)\\) represents the probability that a feature *f* appears when a word *w* is used. For each feature *f*, we identify the word \\(w_{f}\\) most frequently associated with it by maximizing \\(P \\left( f \\mid w \\right)\\):\n\nThe context independence or coherence metric \\({CI}\\) is then computed as the average product of these probabilities across all features:\n\nThis metric ranges from 0 to 1, with 1 indicating perfect alignment, meaning that each word retains its meaning consistently across different contexts and is thus used coherently.\n\n*Entropy*\n\nEntropy metrics are instrumental in analyzing the variability and predictability within linguistic systems. The most fundamental use of entropy involves marginal probabilities, which capture the variability in the number of words in a language [[108](/article/10.1007/s10458-025-09691-y#ref-CR108), [114](/article/10.1007/s10458-025-09691-y#ref-CR114)]. More advanced applications of entropy focus on sender language entropy, which examines the conditional entropy of messages given features and vice versa [[32](/article/10.1007/s10458-025-09691-y#ref-CR32), [118](/article/10.1007/s10458-025-09691-y#ref-CR118)]. Specifically, low conditional entropy \\(H \\left( M \\mid F \\right)\\) indicates that a unique message is used for a specific feature, whereas high \\(H \\left( M \\mid F \\right)\\) reflects the generation of synonyms for the same feature [[118](/article/10.1007/s10458-025-09691-y#ref-CR118)].\n\nRecent approaches further extend this analysis by combining conditional entropies [[37](/article/10.1007/s10458-025-09691-y#ref-CR37), [177](/article/10.1007/s10458-025-09691-y#ref-CR177)]. For example, \\(H \\left( M \\mid F \\right)\\) quantifies the uncertainty remaining about messages after knowing the concepts, while \\(H \\left( F \\mid M \\right)\\) measures the uncertainty about concepts given the messages. A negative correlation between these measures and agent performance is expected [[177](/article/10.1007/s10458-025-09691-y#ref-CR177)]. However, a notable limitation of these entropy-based methods is that they focus on complete messages rather than individual words, which can limit the evaluation of more complex languages.\n\nFor example, Ohmer et al. [[177](/article/10.1007/s10458-025-09691-y#ref-CR177)] provide the following comprehensive evaluation approach. First, the conditional entropy of messages given features \\(H \\left( M \\mid F \\right)\\), see Eq. [41](/article/10.1007/s10458-025-09691-y#Equ41), and \\(H \\left( F \\mid M \\right)\\) are calculated. Additionally, the marginal entropies are calculated using Eq. [42](/article/10.1007/s10458-025-09691-y#Equ42), where *X* represents either messages *M* or features *F*.\n\nUsing these entropies, \\({consistency}\\), see Eq. [43](/article/10.1007/s10458-025-09691-y#Equ43), measures how much uncertainty about the message is reduced when the feature is known, with lower values indicating more consistent message usage. \\({effectiveness}\\), on the other hand, see Eq. [44](/article/10.1007/s10458-025-09691-y#Equ44), evaluates the reduction in uncertainty about the feature when the message is known, with lower values reflecting more unique messages for individual features.\n\nFinally, the normalized mutual information \\({NI}\\) provides a combined score:\n\nA high \\({NI}\\) score indicates a strong predictive relationship between messages and features, reflecting high consistency.\n\n*Similarity*\n\nThe Jaccard similarity coefficient is a another metric for evaluating the consistency of language usage among agents [[72](/article/10.1007/s10458-025-09691-y#ref-CR72), [116](/article/10.1007/s10458-025-09691-y#ref-CR116)]. It quantifies the similarity between two sets by comparing the size of their intersection to the size of their union [[116](/article/10.1007/s10458-025-09691-y#ref-CR116)]. To measure language consistency, the Jaccard similarity is computed by sampling messages for each input and averaging the similarity scores across the population [[116](/article/10.1007/s10458-025-09691-y#ref-CR116)]. This approach reflects how consistently words are used across different messages. Specifically, Jaccard similarity \\(J(M_{\\xi _{i}}, M_{\\xi _{j}})\\) is defined in Eq. [46](/article/10.1007/s10458-025-09691-y#Equ46), where \\(M_{\\xi _{i}}\\) and \\(M_{\\xi _{j}}\\) represent sets of messages generated by different agents based on the same input. The similarity ranges from 0 to 1, with 1 indicating complete overlap and thus perfect similarity.\n\nIn practice, Jaccard similarity helps to assess the coherence of languages emerging from agent-based systems. For instance, in referential game experiments, high perplexity (cf. Section [6.2.2](/article/10.1007/s10458-025-09691-y#Sec26)) and low Jaccard similarity have been observed, suggesting that agents assign unique but incoherent strings to object types to gain an advantage in the game without producing a consistent language [[72](/article/10.1007/s10458-025-09691-y#ref-CR72)]. However, Jaccard similarity is only applicable to scenarios where multiple agents generate messages about the same set of objects. Thus, its application is limited to cases where the goal is to compare the overlap of message sets between agents attempting to convey similar meanings.\n\n#### 6.4.4 Generalization\n\nA language’s ability to generalize is crucial for describing objects and concepts at different levels of complexity, allowing for effective clustering and hierarchical representation. Generalization in ELs reflects their ability to extend beyond specific training instances to novel situations. “If the emergent languages can be generalised, we then could say that these languages do capture the structure of meaning spaces” [[155](/article/10.1007/s10458-025-09691-y#ref-CR155)]. Research shows that languages capable of generalization tend to emerge only when the input is sufficiently varied [[122](/article/10.1007/s10458-025-09691-y#ref-CR122)]. In contrast, a large dictionary size often indicates a lack of generalization [[108](/article/10.1007/s10458-025-09691-y#ref-CR108)]. Human languages have evolved under the pressure of a highly complex environment, fostering their generalization capabilities [[122](/article/10.1007/s10458-025-09691-y#ref-CR122)]. However, deep learning models often exploit dataset-specific regularities rather than developing systematic solutions [[46](/article/10.1007/s10458-025-09691-y#ref-CR46)]. To address this, much research is being done on the systematic generalization abilities of ELs.\n\n*Zero shot evaluation*\n\nZero-shot evaluation, which assesses the ability of an agent to generalize to novel stimuli [[72](/article/10.1007/s10458-025-09691-y#ref-CR72), [168](/article/10.1007/s10458-025-09691-y#ref-CR168)], has become a standard metric in the study of EL as illustrated in Fig. [12](/article/10.1007/s10458-025-09691-y#Fig12) in Appendix [B](/article/10.1007/s10458-025-09691-y#Sec48). This evaluation is critical to understand the generalization capabilities of an agent. Zero-shot evaluation can be done in two different scenarios, one with unseen input and the other with an unseen partner.\n\nIn the unseen input scenario, models are tested on a zero-shot test set consisting of samples with feature combinations not encountered during training. Performance, such as accuracy, is reported for these unseen samples [[114](/article/10.1007/s10458-025-09691-y#ref-CR114), [122](/article/10.1007/s10458-025-09691-y#ref-CR122), [127](/article/10.1007/s10458-025-09691-y#ref-CR127), [164](/article/10.1007/s10458-025-09691-y#ref-CR164), [168](/article/10.1007/s10458-025-09691-y#ref-CR168)]. Different methods for constructing novel inputs include exposing models to objects that resemble training data but have unseen properties or entirely novel combinations of features [[168](/article/10.1007/s10458-025-09691-y#ref-CR168)]. Moreover, a more drastic approach may involve moving to entirely new input scenarios, such as testing the ability of agents to generalize across different game types [[37](/article/10.1007/s10458-025-09691-y#ref-CR37)].\n\nThe unseen partner scenario, also known as cross-play or zero-shot coordination, evaluates models by pairing agents that did not communicate during training. Again, performance is measured, typically in terms of accuracy [[211](/article/10.1007/s10458-025-09691-y#ref-CR211), [256](/article/10.1007/s10458-025-09691-y#ref-CR256)].\n\nHowever, these approaches also have drawbacks. The unseen input scenario requires a ground truth oracle to withhold feature combinations, which is necessary to accurately define novel combinations. Meanwhile, the unseen partner setup can introduce inefficiencies by requiring additional resources to train novel communication partners for testing.\n\n*Ease and transfer learning*\n\nEase and Transfer Learning (ETL), as proposed by Chaabouni et al. [[25](/article/10.1007/s10458-025-09691-y#ref-CR25)], evaluates how easily new listeners can adapt to an EL on distinct tasks. ETL extends the concept of ease-of-teaching [[106](/article/10.1007/s10458-025-09691-y#ref-CR106)] by assessing how effectively a deterministic language, developed by a fixed set of speakers, can be transferred to new listeners who are trained on tasks different from the original one for which the language was optimized [[25](/article/10.1007/s10458-025-09691-y#ref-CR25)]. This metric not only gauges the language’s generality but also its transferability across tasks [[25](/article/10.1007/s10458-025-09691-y#ref-CR25)].\n\nTo measure ETL, after convergence, a fixed number of speakers produce a deterministic language by selecting symbols using an \\(\\mathop {\\textrm{argmax}}\\limits\\) operation over their distributions. This language is then used to train newly initialized listeners on a new task. The training curve is tracked to observe how quickly and accurately the listeners learn the task, which may involve more challenging objectives than the former training tasks [[25](/article/10.1007/s10458-025-09691-y#ref-CR25), [154](/article/10.1007/s10458-025-09691-y#ref-CR154)].\n\n### 6.5 Pragmatics\n\nPragmatics is a critical aspect of language that examines how context influences meaning [[199](/article/10.1007/s10458-025-09691-y#ref-CR199)]. It goes beyond the literal interpretation of words and requires the listener to infer the speaker’s intentions, beliefs, and mental states, an ability known as Theory of Mind (ToM) [[199](/article/10.1007/s10458-025-09691-y#ref-CR199)]. In human interactions, this contextual reasoning is essential for predicting and understanding behavior. In the context of EL, pragmatics focuses on how effectively agents use the communication ability in their environment. Empirical studies have shown that agents may initially fail to use communication meaningfully, but, once they do communicate, they can reach a locally optimal solution to the communication problem [[230](/article/10.1007/s10458-025-09691-y#ref-CR230)]. Thus, evaluating the pragmatics of EL is essential to determining its utility and effectiveness in real-world applications.\n\n#### 6.5.1 Predictability\n\nPredictability evaluates the complexity of an environment and its effect on the need for communication. Thus, it is a central metric for the probability of emergence and the use of EL. In simple environments with limited actions, agents can often coordinate without communication [[110](/article/10.1007/s10458-025-09691-y#ref-CR110)].\n\n*Behavioral divergence*\n\nBehavioral divergence, introduced by Dubova et al. [[110](/article/10.1007/s10458-025-09691-y#ref-CR110)], posits that less diversity in actions or messages correlates with more predictable behavior, potentially reducing the need for communication. To quantify this, we calculate Behavioral Action Predictability \\({BAP}\\) and Behavioral Message Predictability \\({BMP}\\). Both use the Jensen-Shannon Divergence (JSD) (see Eq. [47](/article/10.1007/s10458-025-09691-y#Equ47)) which itself uses the Kullback–Leibler Divergence \\({D_{KL}}\\) (cf. Equation [11](/article/10.1007/s10458-025-09691-y#Equ11)).\n\nBAP (see Eq. [48](/article/10.1007/s10458-025-09691-y#Equ48)) and BMP (see Eq. [49](/article/10.1007/s10458-025-09691-y#Equ49)) both use a uniform distribution *Q* for comparison. BAP further uses the distribution of actions by the agent \\(P \\left( a_{\\xi } \\right)\\) while BMP uses the distribution of messages by the agent \\(P \\left( m_{\\xi } \\right)\\). Based on that, these metrics provide a robust measure of how predictable agent behaviors and messages are, with higher values indicating less predictability and greater need for beneficial communication [[110](/article/10.1007/s10458-025-09691-y#ref-CR110)].\n\n#### 6.5.2 Efficiency\n\nIn EL settings, efficient communication arises only when there is an opportunity cost [[115](/article/10.1007/s10458-025-09691-y#ref-CR115)]. Without such a cost, there is no drive towards brevity, which limits the effectiveness and efficiency of EL in HCI.\n\n*Sparsity*\n\nSparsity, as proposed by Kalinowska et al. [[115](/article/10.1007/s10458-025-09691-y#ref-CR115)], measures the extent to which agents minimize their communication during task execution. This metric requires only the collection of messages exchanged per episode for computation. However, its applicability is limited to scenarios where communication is not strictly necessary for task completion, i.e., agents have the option to send no messages at all or to send messages that contain no meaningful information. A sparsity value of 0 indicates that an agent can solve the task using only a single message throughout an episode, reflecting a highly efficient communication strategy. Conversely, higher sparsity values indicate more frequent or verbose communication, which may indicate inefficiencies in the EL.\n\nCommunication sparsity \\({ComSpar}\\) is mathematically defined as:\n\nIn this equation, \\(M_{\\text {ep}, i}\\) represents the set of all messages exchanged during episode *i*, and \\(n_{\\text {ep}}\\) is the total number of episodes observed. The collection \\(\\left\\{ m \\mid m \\in M_{\\text {ep}, i} \\wedge m \\ne 0 \\right\\}\\) consists of all messages *m* of episode *i* that are non-zero and thus contributing.\n\n#### 6.5.3 Positive signaling\n\nPositive signaling evaluates the alignment between an agent’s observations and its communication output [[29](/article/10.1007/s10458-025-09691-y#ref-CR29)]. The goal is to ensure that the outgoing transmitted information is both relevant and observable by the agent [[181](/article/10.1007/s10458-025-09691-y#ref-CR181)].\n\n*Speaker consistency*\n\nSpeaker Consistency (SC), introduced by Jaques et al. [[240](/article/10.1007/s10458-025-09691-y#ref-CR240)], measures how effectively an agent’s messages reflect its state or trajectory, thereby ensuring the communication is meaningful. This is quantified using mutual information. For an agent \\(\\xi _i\\), the trajectory \\(\\tau _{\\xi _i}^t\\) represents the sequence of states and actions up to time step *t*. The message produced at time *t* is denoted by \\(m_{\\xi _i}^t\\). The mutual information \\(I(m_{\\xi _i}^t, \\tau _{\\xi _i}^t)\\) between the message and trajectory is calculated as:\n\nHere, \\(H(m_{\\xi _i}^t)\\) is the entropy of the message distribution, \\(H(m_{\\xi _i}^t | \\tau _{\\xi _i}^t)\\) is the conditional entropy given the trajectory, \\(\\overline{P_{\\xi _i}}(m)\\) as marginal distribution of message *m* over all trajectories, and \\(P_{\\xi _i}(m|\\tau _{\\xi _i}^t)\\) as conditional distribution of message *m* given the trajectory \\(\\tau _{\\xi _i}^t\\). This way, the mutual information value reflects how much information the message carries about the agent’s trajectory.\n\nLowe et al. [[29](/article/10.1007/s10458-025-09691-y#ref-CR29)] built on this concept and provided the following formula for Speaker Consistency (\\({SC}\\)):\n\nIn this equation, \\(P \\left( a, m \\right)\\) is the joint probability of action *a* and message *m*, calculated empirically by averaging their co-occurrences across episodes. In general, SC is a valuable metric for evaluating whether the EL is both informative and aligned with the behavioral patterns of the sender.\n\n#### 6.5.4 Positive listening\n\nPositive listening evaluates the effectiveness of how a message receiver utilizes and applies incoming information [[29](/article/10.1007/s10458-025-09691-y#ref-CR29)]. However, agents should not simply process messages similarly to other observations to avoid treating them as mere directives [[146](/article/10.1007/s10458-025-09691-y#ref-CR146)]. Nevertheless, the metrics presented in this section focus on evaluating the receiver’s ability to effectively integrate and use the information received, rather than evaluating the receiver’s ability to do more than just follow instructions.\n\n*Instantaneous coordination*\n\nInstantaneous Coordination (IC), also referred to as listener consistency [[29](/article/10.1007/s10458-025-09691-y#ref-CR29)], was introduced by Jaques et al. [[240](/article/10.1007/s10458-025-09691-y#ref-CR240)] as a metric to evaluate how effectively an agent’s message influences another agent’s subsequent action. IC is computed similarly to Speaker Consistency (cf. Section [6.5.3](/article/10.1007/s10458-025-09691-y#Sec38)), but differs in that it measures the mutual information between one agent’s message and the other agent’s next action, averaged over episodes. This metric directly captures the receiver’s immediate reaction to an incoming message, making it a measure of positive listening. However, it primarily captures situations where the receiver’s action is directly changed by the sender’s message, without considering the broader context or long-term dependencies [[29](/article/10.1007/s10458-025-09691-y#ref-CR29)]. Accordingly, “IC can miss many positive listening relationships” [[29](/article/10.1007/s10458-025-09691-y#ref-CR29)].\n\nJaques et al. [[240](/article/10.1007/s10458-025-09691-y#ref-CR240)] proposed two specific measures for IC: One that quantifies the mutual information between the sender’s message and the receiver’s next action (see Eq. [53](/article/10.1007/s10458-025-09691-y#Equ53)), and another one that measures the mutual information between the sender’s current action and the receiver’s next action (see Eq. [54](/article/10.1007/s10458-025-09691-y#Equ54)). These measures are calculated by averaging over all trajectory steps and taking the maximum value between any two agents, focusing on short-term dependencies between consecutive timesteps.\n\nA unified equation for \\({IC}\\) is provided by Lowe et al. [[29](/article/10.1007/s10458-025-09691-y#ref-CR29)]:\n\nHere, \\(P \\left( a_{\\xi _{R}}^{t+1}, m_{\\xi _{S}}^{t} \\right)\\) is the empirical joint probability of the sender’s message and the receiver’s subsequent action, averaged over episodes within each epoch.\n\n*Message effect*\n\nThe Message Effect (\\({ME}\\)) metric, introduced by Bouchacourt and Baroni [[202](/article/10.1007/s10458-025-09691-y#ref-CR202)], quantifies the influence of a message sent by one agent on the subsequent actions and messages of another agent. This metric explicitly considers bidirectional communication, so in the following we use generic agents \\(\\xi _{A}\\) and \\(\\xi _{B}\\) instead of sender and receiver. A notable challenge of this metric is the requirement for counterfactual analysis.\n\nGiven an agent \\(\\xi _{A}\\) at timestep *t* sending a message \\(m_{\\xi _{A}}^t\\), we define \\(z_{\\xi _{B}}^{t+1}\\) as the combination of the action and message produced by agent \\(\\xi _{B}\\) at the following timestep. Accordingly, the conditional distribution \\(P \\left( z_{\\xi _{B}}^{t+1} \\mid m_{\\xi _A}^t \\right)\\) represents the response of \\(\\xi _{B}\\) to the message from \\(\\xi _{A}\\). To account for counterfactuals, which encode what might have happened had \\(\\xi _{A}\\) sent a different message \\({\\widetilde{m}}_{\\xi _{A}}^t\\), we define the counterfactual distribution \\({\\widetilde{P}} \\left( z_{\\xi _B}^{t+1} \\right)\\) (see Eq. [56](/article/10.1007/s10458-025-09691-y#Equ56)).\n\nThe \\({ME}\\) is then measured by the Kullback–Leibler divergence between the actual response and the counterfactual response (see Eq. [57](/article/10.1007/s10458-025-09691-y#Equ57)). The computation involves sampling \\(z_{\\xi _{B}}^{t+1, k}\\) from the conditional distribution for the actual message and sampling counterfactuals \\({\\widetilde{m}}_{\\xi _{A}}^t\\) to estimate \\({\\widetilde{P}} \\left( z_{\\xi _{B}}^{t+1, k} \\right)\\) (see Eq. [58](/article/10.1007/s10458-025-09691-y#Equ58)). The final ME is calculated as the average KL divergence over the collection of samples *K* (see Eq. [59](/article/10.1007/s10458-025-09691-y#Equ59)).\n\n*Causal influence of communication*\n\nThe Causal Influence of Communication (CIC) metric, introduced independently by Jaques et al. [[240](/article/10.1007/s10458-025-09691-y#ref-CR240)] and Lowe et al. [[29](/article/10.1007/s10458-025-09691-y#ref-CR29)], provides a direct measure of positive listening by quantifying the causal effect that one agent’s message has on another agent’s behavior. Traditional methods of evaluating communication often fall short, as simply testing for a decrease in reward after removing the communication channel does not adequately capture the utility of communication [[29](/article/10.1007/s10458-025-09691-y#ref-CR29)].\n\nCIC is computed using the mutual information between an agent’s message and the subsequent action of the receiving agent. Unlike Instantaneous Coordination (cf. Section [6.5.4](/article/10.1007/s10458-025-09691-y#Sec39)), CIC considers the probabilities \\(P \\left( a, m \\right) = \\pi _{\\xi _R} \\left( a \\mid m \\right) \\pi _{\\xi _S} \\left( m \\right)\\) that represent changes in the action distribution of the receiver \\(\\xi _R\\) when the message *m* from the sender \\(\\xi _S\\) is altered. These probabilities are normalized within each game to accurately reflect the influence of messages on actions within the same context [[29](/article/10.1007/s10458-025-09691-y#ref-CR29)].\n\nFor multi-time-step causal influence, the CIC metric is defined as the difference between the entropy of the receiver’s actions with and without communication:\n\nHere, \\(\\tau _{\\xi _R}\\) denotes the standard trajectory of the receiver, comprising state-action pairs, while \\(\\tau _{\\xi _R}^{+M}\\) includes the communicated messages. The CIC is estimated by learning an approximate policy function \\(\\pi (\\cdot | \\tau _{\\xi _R})\\). For more details on the multistep version, refer to Eccles et al. [[230](/article/10.1007/s10458-025-09691-y#ref-CR230)], and for the single-step version, see Jaques et al. [[240](/article/10.1007/s10458-025-09691-y#ref-CR240)].\n\n#### 6.5.5 Symmetry\n\nSymmetry in EL refers to consistent language use across agents in settings, where agents alternate between roles such as message sender and receiver [[105](/article/10.1007/s10458-025-09691-y#ref-CR105), [110](/article/10.1007/s10458-025-09691-y#ref-CR110)]. Thus, symmetry ensures convergence to a common language rather than distinct dialects [[110](/article/10.1007/s10458-025-09691-y#ref-CR110)].\n\n*Inter-agent divergence*\n\nInter-Agent Divergence (IAD), introduced by Dubova et al. [[110](/article/10.1007/s10458-025-09691-y#ref-CR110), [257](/article/10.1007/s10458-025-09691-y#ref-CR257)], quantifies the similarity in how different agents map messages to actions. Let \\(a_{\\xi _i}\\) denote the action of agent \\(\\xi _i\\). The first step involves computing the marginal action distributions for each agent given a message *m*, represented as \\(P(a_{\\xi _i}|m)\\).\n\nThe divergence between two agents, \\(\\xi _i\\) and \\(\\xi _j\\), based on their responses to the same message, is then calculated using the Jensen-Shannon Divergence (JSD) as follows:\n\nFinally, the overall IAD is computed by averaging these divergences across all possible agent pairs \\(\\left( \\xi _i, \\xi _j \\right) \\in \\xi _{\\text {comb}}\\) and messages \\(m \\in M\\):\n\nWhile IAD effectively captures the consistency of inter-agent communication, it may have limitations when applied to more complex languages where message-level comparisons become difficult.\n\n*Within-agent divergence*\n\nWithin-Agent Divergence (WAD), proposed by Dubova et al. [[110](/article/10.1007/s10458-025-09691-y#ref-CR110), [257](/article/10.1007/s10458-025-09691-y#ref-CR257)], measures the consistency of an agent’s communication behavior when it changes roles, such as from sender to receiver. This metric captures the internal symmetry in an agent’s behavior and is crucial in complex systems where agents can assume different roles within the same environment. To compute WAD, we again first consider the action distribution \\(P \\left( a_{\\xi _i} \\mid m \\right)\\) for each agent \\(\\xi _i\\) over a set of messages \\(m \\in M\\). This distribution reflects how an agent’s actions are conditioned on receiving or sending a specific message.\n\nGiven this, the Jensen-Shannon Divergence (JSD) is used to assess the divergence between an agent’s behavior when acting as a sender \\(\\xi _{i,S}\\) versus as a receiver \\(\\xi _{i,R}\\):\n\nFinally, the overall WAD is computed by averaging this divergence across all agents \\(\\xi _i \\in \\xi\\) based on the WAD for individual agents and their messages \\(m \\in M_{\\xi _i}\\):\n\n### 6.6 Summary of the metrics\n\nWhile some EL features are quantifiable by multiple metrics and have been investigated in multiple studies, others remain underexplored, as illustrated in Fig. [12](/article/10.1007/s10458-025-09691-y#Fig12) in Appendix [B](/article/10.1007/s10458-025-09691-y#Sec48). Metrics such as *topographic similarity* and *zero shot evaluation*, both of which assess semantic properties, are well established and widely utilized across multiple studies. In contrast, metrics related to pragmatics, such as *speaker consistency* and *instantaneous coordination*, are fairly well established but are less frequently used. Morphology metrics, particularly *active words* and *average message length*, are more commonly used, whereas syntax remains a peripheral concern, with only two isolated metrics proposed and not adopted in subsequent research. This imbalance indicates that while semantic metrics dominate EL research, morphology and pragmatics receive moderate attention, and syntax is mostly neglected.\n\nFurthermore, the optimality of these metrics is not straightforward. Rather than being simply minimized or maximized, their ideal values are likely to lie at a nuanced balance point that varies depending on the specific EL system and application. This uncertainty leaves the critical question of what constitutes a ‘good’ EL system largely unanswered. Addressing this gap will require a deeper exploration of underrepresented metrics and a more refined understanding of how to evaluate EL systems holistically.\n\n## 7 Future work\n\nIn this section, we outline potential future directions for the research field of EL, based on our vision outlined in Sect. [7.1](/article/10.1007/s10458-025-09691-y#Sec43). We present major research opportunities, organized along key research dimensions, in Sect. [7.2](/article/10.1007/s10458-025-09691-y#Sec44).\n\nAlong with future research directions, we have summarized a list of open source code repositories in Table [10](/article/10.1007/s10458-025-09691-y#Tab10) in Appendix [A](/article/10.1007/s10458-025-09691-y#Sec47) that can serve as convenient starting points for experimenting with these directions, for example, comprehensive frameworks such as the EGG toolkit [[162](/article/10.1007/s10458-025-09691-y#ref-CR162)] and BabyAI [[131](/article/10.1007/s10458-025-09691-y#ref-CR131)] are included.\n\n### 7.1 Vision\n\nOur vision for EL research is grounded in a functional perspective, aiming to achieve significant breakthroughs in human-agent interaction [[24](/article/10.1007/s10458-025-09691-y#ref-CR24), [26](/article/10.1007/s10458-025-09691-y#ref-CR26), [29](/article/10.1007/s10458-025-09691-y#ref-CR29), [33](/article/10.1007/s10458-025-09691-y#ref-CR33), [35](/article/10.1007/s10458-025-09691-y#ref-CR35), [176](/article/10.1007/s10458-025-09691-y#ref-CR176), [256](/article/10.1007/s10458-025-09691-y#ref-CR256)]. This means developing communication systems that enable HCI at the human level, addressing the purpose, cost, and value of communication with intuitive and effective interfaces [[27](/article/10.1007/s10458-025-09691-y#ref-CR27), [36](/article/10.1007/s10458-025-09691-y#ref-CR36), [58](/article/10.1007/s10458-025-09691-y#ref-CR58), [148](/article/10.1007/s10458-025-09691-y#ref-CR148), [189](/article/10.1007/s10458-025-09691-y#ref-CR189), [233](/article/10.1007/s10458-025-09691-y#ref-CR233)]. A key goal is to ensure that ELs are grounded in real-world contexts, allowing agents to understand and interact with human-like comprehension and vice versa [[19](/article/10.1007/s10458-025-09691-y#ref-CR19), [21](/article/10.1007/s10458-025-09691-y#ref-CR21), [40](/article/10.1007/s10458-025-09691-y#ref-CR40), [258](/article/10.1007/s10458-025-09691-y#ref-CR258)]. This includes creating hierarchical, compositional conceptualization capabilities that allow agents to discuss and understand novel concepts in a structured, human-relevant manner [[25](/article/10.1007/s10458-025-09691-y#ref-CR25), [133](/article/10.1007/s10458-025-09691-y#ref-CR133), [182](/article/10.1007/s10458-025-09691-y#ref-CR182), [187](/article/10.1007/s10458-025-09691-y#ref-CR187)]. In addition, exploring the potential for AI explainability through communication is an exciting area [[21](/article/10.1007/s10458-025-09691-y#ref-CR21), [127](/article/10.1007/s10458-025-09691-y#ref-CR127), [189](/article/10.1007/s10458-025-09691-y#ref-CR189)]. Finally, in the long term, creation and creativity through EL comparable to human capabilities would be a milestone. This would allow agents to truly communicate on a human level and enhance their ability to perceive and adapt to their environment through the use of language [[103](/article/10.1007/s10458-025-09691-y#ref-CR103)].\n\n### 7.2 Dimensions and opportunities\n\nThe development, evaluation, and application of EL in communication systems can be systematically analyzed along several critical dimensions. Given the relative youth of the field, with the majority of research emerging within the last eight years, specific areas of focus have gained prominence, particularly in the context of semantic metrics, such as topographic similarity and zero-shot evaluation, as highlighted in Fig. [12](/article/10.1007/s10458-025-09691-y#Fig12) in Appendix [B](/article/10.1007/s10458-025-09691-y#Sec48). However, Fig. [13](/article/10.1007/s10458-025-09691-y#Fig13) in Appendix [B](/article/10.1007/s10458-025-09691-y#Sec48) illustrates that there is no discernible chronological trend or evolution in the way the different language characteristics are addressed. This absence of a historical trajectory is likely attributable to the relatively brief history of the field and the considerable diversity of proposed approaches and methodologies. Despite this, we identified nine key dimensions that, to the best of our knowledge, represent the primary areas of focus in EL research.\n\n#### 7.2.1 Evaluation metrics\n\nEvaluation metrics are essential for rigorously assessing the characteristics and effectiveness of ELs. As detailed in our taxonomy (cf. Section [5.4](/article/10.1007/s10458-025-09691-y#Sec12)), we have identified key characteristics and their associated metrics. While some EL features are quantifiable through multiple metrics and have been examined in multiple studies, others remain underexplored, as illustrated in Fig. [12](/article/10.1007/s10458-025-09691-y#Fig12) in Appendix [B](/article/10.1007/s10458-025-09691-y#Sec48). We emphasize the need to develop comprehensive and quantitative metrics that accurately capture these features, which are critical to determining the practical utility of ELs. Previous studies have similarly highlighted this need [[25](/article/10.1007/s10458-025-09691-y#ref-CR25), [29](/article/10.1007/s10458-025-09691-y#ref-CR29), [46](/article/10.1007/s10458-025-09691-y#ref-CR46), [72](/article/10.1007/s10458-025-09691-y#ref-CR72), [109](/article/10.1007/s10458-025-09691-y#ref-CR109), [126](/article/10.1007/s10458-025-09691-y#ref-CR126), [179](/article/10.1007/s10458-025-09691-y#ref-CR179), [234](/article/10.1007/s10458-025-09691-y#ref-CR234), [241](/article/10.1007/s10458-025-09691-y#ref-CR241)]. In addition, further research is needed to systematically investigate existing metrics, especially with respect to their sensitivity to variations in settings, algorithms, and agent architectures [[29](/article/10.1007/s10458-025-09691-y#ref-CR29), [50](/article/10.1007/s10458-025-09691-y#ref-CR50)]. It is imperative that these metrics be subjected to more rigorous investigation to ensure that they enable meaningful quantitative comparisons and support well-founded conclusions about the capabilities and utility of ELs. Thus, we endorse more comprehensive studies, more edge case testing and, in particular, more analysis of actual human-agent interaction. We see this as a critical priority for advancing the field.\n\n#### 7.2.2 Emergent language and natural language alignment\n\nThis dimension addresses the convergence and divergence between EL and NL. A key approach to this challenge, discussed in Sect. [5.3](/article/10.1007/s10458-025-09691-y#Sec11), involves leveraging language priors to guide this alignment. Achieving robust EL-NL alignment is essential for advancing human-agent interaction. Thus, future research should explore the integration of NL-centered metrics and regularization techniques to enhance this alignment [[14](/article/10.1007/s10458-025-09691-y#ref-CR14), [107](/article/10.1007/s10458-025-09691-y#ref-CR107)]. However, this alignment presents a fundamental dilemma. On the one hand, agents need the autonomy to develop languages organically, tailored to their specific interactions and requirements. On the other hand, to facilitate seamless human-agent communication, these ELs must closely resemble NLs, which imposes significant constraints on their development. This tension creates what we call the Evolution-Acquisition Dilemma, where the evolutionary process fosters intrinsically motivated language emergence, while the acquisition process necessitates alignment with NL. Balancing these competing needs is a critical challenge for future research in this area.\n\n#### 7.2.3 Emergent language and large language models\n\nThe remarkable performance of LLM on various benchmarks has established them as a cornerstone of modern NLP [[80](/article/10.1007/s10458-025-09691-y#ref-CR80)] and potential foundation for more complex agents [[259](#ref-CR259),[260](#ref-CR260),[261](#ref-CR261),[262](/article/10.1007/s10458-025-09691-y#ref-CR262)]. Despite their success, however, LLM face fundamental limitations, particularly in grounding language use in shared environments and experiences [[263](/article/10.1007/s10458-025-09691-y#ref-CR263)] as well as agency [[264](/article/10.1007/s10458-025-09691-y#ref-CR264)] and truthfulness [[265](/article/10.1007/s10458-025-09691-y#ref-CR265)]. Addressing these shortcomings may require insights from EL research. A key challenge in EL is the evolution-acquisition dilemma - the need to ground language in shared, incremental experiences, which current learning systems cannot achieve due to resource and technology constraints. While most LLM applications rely on fine-tuning [[266](/article/10.1007/s10458-025-09691-y#ref-CR266)] and scaling [[85](/article/10.1007/s10458-025-09691-y#ref-CR85)], these methods do not inherently address this challenge or the broader issues of grounding and adaptability. One promising avenue lies in the concept of agentic LLM [[263](/article/10.1007/s10458-025-09691-y#ref-CR263), [267](/article/10.1007/s10458-025-09691-y#ref-CR267), [268](/article/10.1007/s10458-025-09691-y#ref-CR268)] or cognitive language agents [[269](/article/10.1007/s10458-025-09691-y#ref-CR269)], which combine the representational strength of LLMs with the adaptive, experiential learning processes of RL. Here, looking at opportunities for EL related research, LLM might act as language priors, providing a foundation that can be iteratively refined through agentic interaction and experience [[263](/article/10.1007/s10458-025-09691-y#ref-CR263), [270](/article/10.1007/s10458-025-09691-y#ref-CR270)]. This approach mirrors human language acquisition, where teachers provide guidance based on shared experience. In the absence of such a teacher, agentic LLM provide a synthetic framework for combining supervised and EL paradigms, allowing agents to relax static supervised training regimes and develop more adaptive communication protocols. We argue for further exploration of cognitive language agents, focusing not only on established benchmarks but also on challenges central to EL research. Bridging these fields could open up new opportunities for developing systems that combine the scalability of LLM with the adaptability and grounding capabilities of EL.\n\n#### 7.2.4 Representation learning\n\nEL can be viewed as a complex representation learning task, focusing on how agents encode, interpret, and construct internal representations of observations and linguistic data. While representation learning is a well-established area in artificial intelligence research, its application in the context of EL remains underexplored. This dimension is central to the analysis of meaning and language space as outlined in our framework, which is based on the semiotic cycle (cf. Figure [10](/article/10.1007/s10458-025-09691-y#Fig10)). Advancing this dimension requires advanced latent space analyses to elucidate the relationships between ELs, underlying world models, and NL structures. In addition, evaluating the impact of discrete versus continuous representations is critical to refining our understanding of EL dynamics. Future research directions include developing methodologies to ensure that agent representations more accurately reflect the input they receive [[16](/article/10.1007/s10458-025-09691-y#ref-CR16)], exploring efficient representation of (multimodal) information [[124](/article/10.1007/s10458-025-09691-y#ref-CR124)], conducting in-depth analyses to uncover and mitigate influencing factors and biases in learned representations [[46](/article/10.1007/s10458-025-09691-y#ref-CR46)], and assessing the efficacy of these representations for downstream tasks [[111](/article/10.1007/s10458-025-09691-y#ref-CR111)].\n\n#### 7.2.5 Agent design\n\nAgent design is a critical aspect in EL research, directly influencing the linguistic capabilities and adaptability of artificial agents. Prominent research directions include the investigation of advanced neural network architectures tailored for EL [[25](/article/10.1007/s10458-025-09691-y#ref-CR25), [170](/article/10.1007/s10458-025-09691-y#ref-CR170), [186](/article/10.1007/s10458-025-09691-y#ref-CR186)], the creation of architectures optimized for heterogeneous and dynamic agent populations, and the refinement of structures that enhance language emergence and linguistic properties [[111](/article/10.1007/s10458-025-09691-y#ref-CR111), [167](/article/10.1007/s10458-025-09691-y#ref-CR167)]. In addition, modular designs rather than monolithic ones potentially offer advantages by separating language processing from other task-specific computations. Addressing these design challenges is critical to advancing both EL research and broader artificial intelligence goals.\n\n#### 7.2.6 Setting design\n\nThe environment in which agents operate is central to shaping the EL, encompassing interaction rules, agent goals, and communication dynamics (cf. Table [3](/article/10.1007/s10458-025-09691-y#Tab3)). This dimension is integral to the setting space outlined in our framework (cf. Figure [10](/article/10.1007/s10458-025-09691-y#Fig10)). Important future research directions include scaling up experimental settings to include larger and more complex tasks [[14](/article/10.1007/s10458-025-09691-y#ref-CR14), [25](/article/10.1007/s10458-025-09691-y#ref-CR25), [130](/article/10.1007/s10458-025-09691-y#ref-CR130), [146](/article/10.1007/s10458-025-09691-y#ref-CR146), [202](/article/10.1007/s10458-025-09691-y#ref-CR202), [230](/article/10.1007/s10458-025-09691-y#ref-CR230)] with a focus on realistic perceptually grounded game environments [[110](/article/10.1007/s10458-025-09691-y#ref-CR110), [202](/article/10.1007/s10458-025-09691-y#ref-CR202)]. In addition, the study of the impact of populations as such [[179](/article/10.1007/s10458-025-09691-y#ref-CR179)] and the use of heterogeneous agent populations [[118](/article/10.1007/s10458-025-09691-y#ref-CR118)] are crucial areas of research. While some benchmarks have been established and utilized [[131](/article/10.1007/s10458-025-09691-y#ref-CR131), [162](/article/10.1007/s10458-025-09691-y#ref-CR162)], there remains a significant need for the development and widespread dissemination of comprehensive benchmarks in area of research.\n\n#### 7.2.7 Communication design\n\nThe design of the communication channel in EL systems is critical, focusing on how agents exchange and structure information through the channels available to them. This aspect is directly related to the phonetics and phonology components outlined in our taxonomy (cf. Section [5.4.1](/article/10.1007/s10458-025-09691-y#Sec13) and Sect. [5.4.2](/article/10.1007/s10458-025-09691-y#Sec14)). For discrete ELs, it is essential to establish channels that support word-based communication, with considerations such as vocabulary size and variable message length being fundamental to enabling effective and scalable human-agent interaction. Future research directions in this area include the exploration of topology-aware variable communication channels, the integration of heterogeneous channels within multi-agent systems, and the evolution of communication channels over time. Moreover, the incorporation of multimodal communication channels could provide more realistic and contextually rich stimuli, which may significantly enhance the sophistication and applicability of ELs in NL-oriented human-agent coordination [[25](/article/10.1007/s10458-025-09691-y#ref-CR25)].\n\n#### 7.2.8 Learning strategies\n\nLearning strategies focus on how agents acquire, adapt, and refine their linguistic capabilities over time, including the development of language rules and adaptation through interactions with other agents. While MARL serves as the foundational framework, there is significant potential to enhance the learning process through strategic design choices. Future research directions include the exploration of advanced regularization techniques [[107](/article/10.1007/s10458-025-09691-y#ref-CR107), [240](/article/10.1007/s10458-025-09691-y#ref-CR240)], the adoption of tailored optimization strategies [[25](/article/10.1007/s10458-025-09691-y#ref-CR25)], and the integration of supervised or self-supervised learning objectives using appropriate loss designs [[15](/article/10.1007/s10458-025-09691-y#ref-CR15), [151](/article/10.1007/s10458-025-09691-y#ref-CR151)]. Additionally, the application of meta-learning [[155](/article/10.1007/s10458-025-09691-y#ref-CR155)], decentralized learning approaches [[56](/article/10.1007/s10458-025-09691-y#ref-CR56)], and curriculum learning methodologies [[56](/article/10.1007/s10458-025-09691-y#ref-CR56)] offer promising avenues for optimizing the EL learning process.\n\n#### 7.2.9 Human-agent interaction\n\nThe final dimension focuses on the interpretability of ELs by humans and the degree to which humans can shape their development. This aspect is critical for creating human-agent interaction systems where communication is intuitive and effective [[124](/article/10.1007/s10458-025-09691-y#ref-CR124)]. To advance this dimension, future research should prioritize the integration of human-in-the-loop feedback mechanisms to ensure that ELs are not only practical, but also comprehensible to human users [[18](/article/10.1007/s10458-025-09691-y#ref-CR18), [24](/article/10.1007/s10458-025-09691-y#ref-CR24)]. This will improve the usability and adoption of these systems in real-world applications. Key research directions include designing experiments that create incentives for agents to develop communication strategies more closely aligned with human language [[16](/article/10.1007/s10458-025-09691-y#ref-CR16)]. Additionally, exploring the resilience of communication protocols to deception through training with competing agents can lead to more robust and realistic interactions [[34](/article/10.1007/s10458-025-09691-y#ref-CR34)]. Exploring adaptive communication strategies to optimize the sparsity and clarity of messages based on individual or group needs within human-agent teams is another promising direction [[233](/article/10.1007/s10458-025-09691-y#ref-CR233)].\n\n## 8 Limitations and discussion\n\nIn this section, we critically evaluate the limitations of our survey and identify areas for future improvement. Through our review, we aimed to develop a detailed taxonomy for the field of EL, focusing on its key properties (cf. Section [5](/article/10.1007/s10458-025-09691-y#Sec8)), and to analyze as well as categorize quantification approaches and metrics (cf. Section [6](/article/10.1007/s10458-025-09691-y#Sec20)). In addition, we curated a summary of open questions and suggestions for future research (cf. Section [7](/article/10.1007/s10458-025-09691-y#Sec42)). Despite considerable efforts to establish a viable taxonomy and framework in the most systematic and unbiased manner, there are several potential limitations to our research approach and methodology.\n\nFirst, while we have provided an extensive overview of 181 scientific publications in EL research, it is important to acknowledge that our search process, despite being thorough, may have overlooked significant contributions. Consequently, we do not claim completeness. However, we are very confident that our review represents a fair and well-balanced reflection of the existing body of work and the current state of the art.\n\nSecond, our review includes sources that are not peer-reviewed, such as preprints from [arXiv](https://arxiv.org/), to ensure that our work captures the most recent developments and diverse perspectives, including those that might be controversial. While we have carefully examined each paper included in this review, we cannot guarantee that every detail in non-peer-reviewed papers is entirely accurate. Consequently, we focused on concepts, findings, and metrics that are supported by multiple studies.\n\nThird, we have introduced a taxonomy and a comprehensive metrics categorization for EL research, a field that is still in its early stages. This effort comes with inherent challenges, and while we have addressed many of these, it is important to note that our proposed framework does not represent a consensus within the wider research community. We are transparent about this limitation and encourage further discussion and validation.\n\nFourth, in order to maintain focus and conciseness, we have deliberately excluded ideas that lack associated metrics. As a result, some conceptual ideas from the reviewed research literature that are difficult to quantify in this early stage may not be fully explored in this survey.\n\nFinally, we have incorporated several existing metrics into our proposed framework. While many of these metrics are well established in the field, we acknowledge that a more rigorous and critical experimental evaluation of these metrics would be beneficial. We strongly recommend that future research conduct such evaluations to further refine and validate the tools and methods used in EL research.\n\n## 9 Conclusion\n\nIn this paper, we present a comprehensive taxonomy of (EL), an overview of applicable metrics, and a summary of open challenges and potential research directions. Additionally, we provide a list of open source code repositories of the field in Table [10](/article/10.1007/s10458-025-09691-y#Tab10) in Appendix [A](/article/10.1007/s10458-025-09691-y#Sec47). Our overall goal is to create a standardised yet dynamic framework that not only facilitates progress in this area of research, but also stimulates further interest and exploration.\n\nSection [2](/article/10.1007/s10458-025-09691-y#Sec2) introduces the foundational linguistic concepts that underpin our taxonomy. Section [5](/article/10.1007/s10458-025-09691-y#Sec8) offers a comprehensive taxonomy of EL based on the review of 181 scientific publications. Section [6](/article/10.1007/s10458-025-09691-y#Sec20) presents a unified categorization and notation for various metrics, depicted in Fig. [9](/article/10.1007/s10458-025-09691-y#Fig9), ensuring consistency and clarity. Section [7](/article/10.1007/s10458-025-09691-y#Sec42) provides a summary of current achievements and outlines research opportunities.\n\nBy providing a structured overview and systematic categorization of linguistic concepts relevant to EL we have created a common ground for research and discussion. The detailed presentation of metrics and their unified notation ensures readability and usability, making it easier for researchers to navigate related topics and identify potential research opportunities and blind spots of future publications and the research field as a whole. This survey provides a valuable perspective on the development and analysis of EL, serving as both a guide and a resource for advancing this area of study.\n\nEL is a fascinating and promising way to achieve grounded and goal-oriented communication among agents and between humans and agents. Despite its significant progress in recent years, the field faces many open questions and requires further evaluation methods and metrics. Critical questions remain about the measurability of linguistic features, the validity of proposed metrics, their utility, and their necessity. Aligning EL with (NLP) for (HCI) presents additional opportunities and challenges. We encourage continued contributions and interdisciplinary research to address these issues and advance the field.\n\n## Data availability\n\nNot applicable.\n\n## Materials availability\n\nNot applicable.\n\n## Code availability\n\nNot applicable.\n\n## References\n\nLewis, D. K. (1969).\n\n*Convention: A philosophical study*(1st ed.). Harvard University Press.Wagner, K., Reggia, J. A., Uriagereka, J., & Wilkinson, G. S. (2003). Progress in the simulation of emergent communication and language.\n\n*Adaptive Behavior,**11*(1), 37–69.[https://doi.org/10.1177/10597123030111003](https://doi.org/10.1177/10597123030111003)Steels, L. (1997). The synthetic modeling of language origins.\n\n*Evolution of Communication,**1*(1), 1–34.[https://doi.org/10.1075/eoc.1.1.02ste](https://doi.org/10.1075/eoc.1.1.02ste)Nowak, M. A., & Krakauer, D. C. (1999). The evolution of language.\n\n*Proceedings of the National Academy of Sciences,**96*(14), 8028–8033.[https://doi.org/10.1073/pnas.96.14.8028](https://doi.org/10.1073/pnas.96.14.8028)Kirby, S. (2002). Natural language from artificial life.\n\n*Artificial Life,**8*(2), 185–215.[https://doi.org/10.1162/106454602320184248](https://doi.org/10.1162/106454602320184248)Cangelosi, A., & Parisi, D. (2002).\n\n*Simulating the Evolution of Language*. Springer London.[https://doi.org/10.1007/978-1-4471-0663-0](https://doi.org/10.1007/978-1-4471-0663-0)Christiansen, M. H., & Kirby, S. (2003).\n\n*Language Evolution*. Oxford University Press.Batali, J. (1998). Computational simulations of the emergence of grammar. In J. Hurford, C. Knight, & M. Studdert-Kennedy (Eds.),\n\n*Approaches to the Evolution of Language*(pp. 405–426). Cambridge University Press.Oliphant, M., & Batali, J. (1997). Learning and the emergence of coordinated communication.\n\n*Center for Research on Language Newsletter,**11*(1), 1–46.Steels, L. (1995). A self-organizing spatial vocabulary.\n\n*Artificial Life,**2*(3), 319–332.[https://doi.org/10.1162/artl.1995.2.3.319](https://doi.org/10.1162/artl.1995.2.3.319)Skyrms, B. (2002). Signals, evolution and the explanatory power of transient information.\n\n*Philosophy of Science,**69*(3), 407–428.[https://doi.org/10.1086/342451](https://doi.org/10.1086/342451)Smith, K., Kirby, S., & Brighton, H. (2003). Iterated learning: A framework for the emergence of language.\n\n*Artificial Life,**9*(4), 371–386.[https://doi.org/10.1162/106454603322694825](https://doi.org/10.1162/106454603322694825)Foerster, J. N., Assael, Y. M., Freitas, N. D., & Whiteson, S. (2017). Learning to communicate with deep multi-agent reinforcement learning. In D. D. Lee, U. Luxburg, R. Garnett, M. Sugiyama, & I. Guyon (Eds.),\n\n*Advances in neural information processing systems*(29th ed., pp. 2145–2153). Curran Associates Inc.Lazaridou, A., Peysakhovich, A., Baroni, M. (2017). Multi-agent cooperation and the emergence of (natural) language. In: OpenReview.net (ed.) 5th international conference on learning representations.\n\n[https://openreview.net/forum?id=Hk8N3Sclg](https://openreview.net/forum?id=Hk8N3Sclg)Havrylov, S., & Titov, I. (2017). Emergence of language with multi-agent games: Learning to communicate with sequences of symbols. In U. Luxburg, I. Guyon, S. Bengio, H. Wallach, R. Fergus, S. V. N. Vishwanathan, & R. Garnett (Eds.),\n\n*Advances in neural information processing systems*(30th ed., pp. 2146–2156). Curran Associates Inc.Bouchacourt, D., Baroni, M. (2018). How agents see things: On visual representations in an emergent language game. In: Association for Computational Linguistics (ed.) Proceedings of the 2018 conference on empirical methods in natural language processing, pp. 981–985.\n\n[https://doi.org/10.18653/v1/D18-1119](https://doi.org/10.18653/v1/D18-1119)Cao, K., Lazaridou, A., Lanctot, M., Leibo, J.Z., Tuyls, K., Clark, S. (2018). Emergent communication through negotiation. In: OpenReview.net (ed.) 6th international conference on learning representations: Conference track proceedings.\n\n[https://openreview.net/forum?id=Hk6WhagRW](https://openreview.net/forum?id=Hk6WhagRW)Mordatch, I., Abbeel, P. (2018) Emergence of grounded compositional language in multi-agent populations. In: Association for the advancement of artificial intelligence (ed.) Proceedings of the thirty-second AAAI conference on artificial intelligence and thirtieth innovative applications of artificial intelligence conference and eighth AAAI symposium on educational advances in artificial intelligence, pp. 1495–1502. AAAI Press.\n\n[https://cdn.aaai.org/ojs/11492/11492-13-15020-1-2-20201228.pdf](https://cdn.aaai.org/ojs/11492/11492-13-15020-1-2-20201228.pdf)Das, A., Kottur, S., Moura, J.M.F., Lee, S., Batra, D. (2017). Learning cooperative visual dialog agents with deep reinforcement learning. In: 2017 IEEE International conference on computer vision (ICCV), pp. 2970–2979. IEEE.\n\n[https://doi.org/10.1109/ICCV.2017.321](https://doi.org/10.1109/ICCV.2017.321).[http://arxiv.org/pdf/1703.06585v2](http://arxiv.org/pdf/1703.06585v2)Sukhbaatar, S., Szlam, A., & Fergus, R. (2017). Learning multiagent communication with backpropagation. In D. D. Lee, U. Luxburg, R. Garnett, M. Sugiyama, & I. Guyon (Eds.),\n\n*Advances in neural information processing systems*(29th ed., pp. 2252–2260). Curran Associates Inc.Agarwal, A., Gurumurthy, S., Sharma, V., Lewis, M., Sycara, K. (2019). Community regularization of visually-grounded dialog. In: International foundation for autonomous agents and multiagent systems (ed.) proceedings of the 18th international conference on autonomous agents and multiagent systems. ACM digital library, pp. 1042–1050. International Foundation for Autonomous Agents and Multiagent Systems.\n\n[https://www.ifaamas.org/Proceedings/aamas2019/pdfs/p1042.pdf](https://www.ifaamas.org/Proceedings/aamas2019/pdfs/p1042.pdf)Blumenkamp, J., Prorok, A. (2020). The emergence of adversarial communication in multi-agent reinforcement learning. In: PMLR (ed.) 4th Conference on robot learning. Proceedings of machine learning research, pp. 1394–1414.\n\n[https://proceedings.mlr.press/v155/blumenkamp21a.html](https://proceedings.mlr.press/v155/blumenkamp21a.html)Brandizzi, N., Grossi, D., Iocchi, L. (2021). Rlupus: Cooperation through emergent communication in the werewolf social deduction game. In: 13th Adaptive and learning agents workshop at AAMAS 2021.\n\n[http://arxiv.org/pdf/2106.05018v2](http://arxiv.org/pdf/2106.05018v2)Brandizzi, N., Iocchi, L. (2022). Emergent communication in human-machine games. In: 5th Workshop on emergent communication at ICLR 2022.\n\n[https://openreview.net/forum?id=rqLgeQWCXZ9](https://openreview.net/forum?id=rqLgeQWCXZ9)Chaabouni, R., Strub, F., Altché, F., Tarassov, E., Tallec, C., Davoodi, E., Mathewson, K.W., Tieleman, O., Lazaridou, A., Piot, B. (2022). Emergent communication at scale. In: OpenReview.net (ed.) 10th International conference on learning representations.\n\n[https://openreview.net/forum?id=AUGBfDIV9rL](https://openreview.net/forum?id=AUGBfDIV9rL)Gupta, A., Lanctot, M., & Lazaridou, A. (2021). Dynamic population-based meta-learning for multi-agent communication with natural language. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. S. Liang, & J. Wortman Vaughan (Eds.),\n\n*Advances in neural information processing systems (NeurIPS 2021),*(34th ed.). Curran Associates Inc.Karten, S., Agrawal, S., Tucker, M., Hughes, D., Lewis, M., Shah, J., Sycara, K. (2022). The enforcers: Consistent sparse-discrete methods for constraining informative emergent communication.\n\n[http://arxiv.org/pdf/2201.07452v1](http://arxiv.org/pdf/2201.07452v1)Lo, Y.L., Sengupta, B., Lo Long, Y. (2022). Learning to ground decentralized multi-agent communication with contrastive learning. In: 5th Workshop on emergent communication at ICLR 2022.\n\n[https://openreview.net/forum?id=rLceWXWCmZc](https://openreview.net/forum?id=rLceWXWCmZc)Lowe, R., Foerster, J., Boureau, Y.-L., Pineau, J., Dauphin, Y. (2019). On the pitfalls of measuring emergent communication. In: International foundation for autonomous agents and multiagent systems (ed.) Proceedings of the 18th international conference on autonomous agents and multiagent systems. ACM digital library, pp. 693–701. International foundation for autonomous agents and multiagent systems.\n\n[https://www.ifaamas.org/Proceedings/aamas2019/pdfs/p693.pdf](https://www.ifaamas.org/Proceedings/aamas2019/pdfs/p693.pdf)Vanneste, S., Vanneste, A., Mets, K., Schepper, T.D., Anwar, A., Mercelis, S., Latré, S., Hellinckx, P. (2022). Learning to communicate using counterfactual reasoning. In: 14th Workshop on adaptive and learning agents at AAMAS 2022.\n\n[https://ala2022.github.io/papers/ALA2022_paper_17.pdf](https://ala2022.github.io/papers/ALA2022_paper_17.pdf)Verma, S. (2021). Towards sample efficient learners in population based referential games through action advising: Extended abstract. In: Proceedings of the 20th international conference on autonomous agents and multiagent systems. AAMAS ’21. International foundation for autonomous agents and multiagent systems.\n\n[https://www.ifaamas.org/Proceedings/aamas2021/pdfs/p1689.pdf](https://www.ifaamas.org/Proceedings/aamas2021/pdfs/p1689.pdf)Yu, D., Mu, J., Goodman, N. (2022). Emergent covert signaling in adversarial reference games. In: 5th Workshop on emergent communication at ICLR 2022.\n\n[https://openreview.net/forum?id=H-eMQbR7Z5](https://openreview.net/forum?id=H-eMQbR7Z5)Bogin, B., Geva, M., Berant, J. (2018). Emergence of communication in an interactive world with consistent speakers. In: 2nd Workshop on emergent communication at NeurIPS 2018.\n\n[http://arxiv.org/pdf/1809.00549v2](http://arxiv.org/pdf/1809.00549v2)Noukhovitch, M., LaCroix, T., Lazaridou, A., Courville, A. (2021). Emergent communication under competition. In: Proceedings of the 20th international conference on autonomous agents and multiagent systems. AAMAS ’21. International foundation for autonomous agents and multiagent systems.\n\n[https://www.ifaamas.org/Proceedings/aamas2021/pdfs/p974.pdf](https://www.ifaamas.org/Proceedings/aamas2021/pdfs/p974.pdf)Lazaridou, A., Baroni, M. (2020). Emergent multi-agent communication in the deep learning era.\n\n[http://arxiv.org/pdf/2006.02419v2](http://arxiv.org/pdf/2006.02419v2)Galke, L., Ram, Y., Raviv, L. (2022). Emergent communication for understanding human language evolution: What’s missing? In: 5th Workshop on emergent communication at ICLR 2022.\n\n[https://openreview.net/forum?id=rqUGZQ-0XZ5](https://openreview.net/forum?id=rqUGZQ-0XZ5)Mu, J., Goodman, N. (2021). Emergent communication of generalizations. In Neural Information Processing Systems Foundation (ed.) Advances in neural information processing systems. Advances in neural information processing systems, 34, pp 17994–18007. Curran Associates Inc.\n\n[https://papers.nips.cc/paper/2021/file/9597353e41e6957b5e7aa79214fcb256-Paper.pdf](https://papers.nips.cc/paper/2021/file/9597353e41e6957b5e7aa79214fcb256-Paper.pdf)Steinert-Threlkeld, S., Zhou, X., Liu, Z., Downey, C.M. (2022). Emergent communication fine-tuning (ec-ft) for pretrained language models. In: 5th Workshop on emergent communication at ICLR 2022.\n\n[https://openreview.net/forum?id=SUqrM7WR7W5](https://openreview.net/forum?id=SUqrM7WR7W5)Bender, E.M., Koller, A. (2020). Climbing towards nlu: On meaning, form, and understanding in the age of data. In: Jurafsky, D., Chai, J., Schluter, N., Tetreault, J. (eds.) Proceedings of the 58th annual meeting of the association for computational linguistics, pp. 5185–5198. Association for Computational Linguistics.\n\n[https://doi.org/10.18653/v1/2020.acl-main.463](https://doi.org/10.18653/v1/2020.acl-main.463)Lemon, O. (2022). Conversational grounding in emergent communication—data and divergence. In: 5th Workshop on emergent communication at ICLR 2022.\n\n[https://openreview.net/forum?id=BbG-m-0Xbq](https://openreview.net/forum?id=BbG-m-0Xbq)Browning, J., Lecun, Y. (2022). AI and the limits of language: An artificial intelligence system trained on words and sentences alone will never approximate human understanding., Online.\n\n[https://www.noemamag.com/ai-and-the-limits-of-language/](https://www.noemamag.com/ai-and-the-limits-of-language/)Manning, C. D., & Schütze, H. (2005).\n\n*Foundations of Statistical Natural Language Processing*(8th ed.). MIT Press.Qiu, X., Sun, T., Xu, Y., Shao, Y., Dai, N., & Huang, X. (2020). Pre-trained models for natural language processing: A survey.\n\n*Science China Technological Sciences,**63*(10), 1872–1897.[https://doi.org/10.1007/s11431-020-1647-3](https://doi.org/10.1007/s11431-020-1647-3)Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Le Scao, T., Gugger, S., Drame, M., Lhoest, Q., Rush, A. (2020). Transformers: State-of-the-art natural language processing. In: Liu, Q., Schlangen, D. (eds.) Proceedings of the 2020 conference on empirical methods in natural language processing: System demonstrations, pp. 38–45. Association for Computational Linguistics.\n\n[https://doi.org/10.18653/v1/2020.emnlp-demos.6](https://doi.org/10.18653/v1/2020.emnlp-demos.6)van Eecke, P., Beuls, K. (2021). Re-conceptualising the language game paradigm in the framework of multi-agent reinforcement learning. In: Association for the Advancement of Artificial (ed.) COMARL AAAI 2020-2021—Challenges and opportunities for multi-agent reinforcement learning, AAAI Spring Symposium Series.\n\n[https://arxiv.org/pdf/2004.04722](https://arxiv.org/pdf/2004.04722)Keresztury, B., Bruni, E. (2020). Compositional properties of emergent languages in deep learning.\n\n[http://arxiv.org/pdf/2001.08618v1](http://arxiv.org/pdf/2001.08618v1)Hernandez-Leal, P., Kartal, B., & Taylor, M. E. (2019). A survey and critique of multiagent deep reinforcement learning.\n\n*Autonomous Agents and Multi-Agent Systems,**33*(6), 750–797.[https://doi.org/10.1007/s10458-019-09421-1](https://doi.org/10.1007/s10458-019-09421-1)Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., … Moher, D. (2021). The prisma 2020 statement: An updated guideline for reporting systematic reviews.\n\n*BMJ (Clinical Research Edition),**372*, 71.[https://doi.org/10.1136/bmj.n71](https://doi.org/10.1136/bmj.n71)Lipowska, D., & Lipowski, A. (2022). Emergence and evolution of language in multi-agent systems.\n\n*Lingua,**272*, 103331.[https://doi.org/10.1016/j.lingua.2022.103331](https://doi.org/10.1016/j.lingua.2022.103331)Denamganaï, K., Walker, J.A. (2020). Referentialgym: A nomenclature and framework for language emergence & grounding in (visual) referential games. In: 4th Workshop on emergent communication at NeurIPS 2020.\n\n[http://arxiv.org/pdf/2012.09486v1](http://arxiv.org/pdf/2012.09486v1)Korbak, T., Zubek, J., Rączaszek-Leonardi, J. (2020). Measuring non-trivial compositionality in emergent communication. In: 4th Workshop on emergent communication at NeurIPS 2020.\n\n[http://arxiv.org/pdf/2010.15058v2](http://arxiv.org/pdf/2010.15058v2)LaCroix, T. (2019). Biology and compositionality: Empirical considerations for emergent-communication protocols. In: 3rd workshop on emergent communication at NeurIPS 2019.\n\n[http://arxiv.org/pdf/1911.11668v2](http://arxiv.org/pdf/1911.11668v2)Mihai, D., Hare, J. (2021). The emergence of visual semantics through communication games.\n\n[http://arxiv.org/pdf/2101.10253v1](http://arxiv.org/pdf/2101.10253v1)Galke, L., Raviv, L. (2024). Emergent communication and learning pressures in language models: a language evolution perspective.\n\n[http://arxiv.org/pdf/2403.14427v1](http://arxiv.org/pdf/2403.14427v1)Vanneste, A., Vanneste, S., Mets, K., Schepper, T.D., Mercelis, S., Latré, S., Hellinckx, P. (2022). An analysis of discretization methods for communication learning with multi-agent reinforcement learning. In: 14th Workshop on adaptive and learning agents at AAMAS 2022.\n\n[http://arxiv.org/pdf/2204.05669v1](http://arxiv.org/pdf/2204.05669v1)Moulin-Frier, C., Oudeyer, P.-Y. (2021). Multi-agent reinforcement learning as a computational tool for language evolution research: Historical context and future challenges. In: Association for the Advancement of Artificial (ed.) COMARL AAAI 2020-2021—Challenges and opportunities for multi-agent reinforcement learning, AAAI Spring Symposium Series.\n\n[https://arxiv.org/pdf/2002.08878](https://arxiv.org/pdf/2002.08878)Fernando, C., Zenkova, D., Nikolov, S., Osindero, S. (2020). From language games to drawing games.\n\n[http://arxiv.org/pdf/2010.02820v2](http://arxiv.org/pdf/2010.02820v2)Brandizzi, N. (2023). Toward more human-like ai communication: A review of emergent communication research.\n\n*IEEE Access,**11*, 142317–142340.[https://doi.org/10.1109/ACCESS.2023.3339656](https://doi.org/10.1109/ACCESS.2023.3339656)Carston, R. (2009). The explicit/implicit distinction in pragmatics and the limits of explicit communication.\n\n*International Review of Pragmatics,**1*(1), 35–62.[https://doi.org/10.1163/187731009X455839](https://doi.org/10.1163/187731009X455839)Watzlawick, P., Bavelas, J. B., & Jackson, D. D. (1967).\n\n*Pragmatics of human communication: A study of interactional patterns, pathologies, and paradoxes*. Norton.Andersen, P. A. (1991). When one cannot not communicate: A challenge to motley’s traditional communication postulates.\n\n*Communication Studies,**42*(4), 309–325.[https://doi.org/10.1080/10510979109368346](https://doi.org/10.1080/10510979109368346)Antos, G., Ventola, E., & Weber, T. (2008).\n\n*Handbook of interpersonal communication*. De Gruyter Mouton.[https://doi.org/10.1515/9783110211399](https://doi.org/10.1515/9783110211399)Witt, P. (2016).\n\n*Communication and Learning*. De Gruyter Mouton.[https://doi.org/10.1515/9781501502446](https://doi.org/10.1515/9781501502446)Hartley, P. (1993).\n\n*Interpersonal Communication*(1st ed.). Routledge.Jones, R. G. (2018).\n\n*Communication in the real world*(2nd ed.). Flat World Knowledge.Bossert, W. H., & Wilson, E. O. (1963). The analysis of olfactory communication among animals.\n\n*Journal of Theoretical Biology,**5*(3), 443–469.[https://doi.org/10.1016/0022-5193(63)90089-4](https://doi.org/10.1016/0022-5193(63)90089-4)Sales, G., & Pye, D. (1974).\n\n*Ultrasonic Communication by Animals*. Springer.[https://doi.org/10.1007/978-94-011-6901-1](https://doi.org/10.1007/978-94-011-6901-1)Rauschecker, J. P., & Scott, S. K. (2009). Maps and streams in the auditory cortex: Nonhuman primates illuminate human speech processing.\n\n*Nature Neuroscience,**12*(6), 718–724.[https://doi.org/10.1038/nn.2331](https://doi.org/10.1038/nn.2331)Tronick, E. Z. (1989). Emotions and emotional communication in infants.\n\n*The American Psychologist,**44*(2), 112–119.[https://doi.org/10.1037/0003-066X.44.2.112](https://doi.org/10.1037/0003-066X.44.2.112)Grosse, G., Behne, T., Carpenter, M., & Tomasello, M. (2010). Infants communicate in order to be understood.\n\n*Developmental Psychology,**46*(6), 1710–1722.[https://doi.org/10.1037/a0020727](https://doi.org/10.1037/a0020727)Stokoe, W. C. (1980). Sign language structure.\n\n*Annual Review of Anthropology,**9*(1), 365–390.[https://doi.org/10.1146/annurev.an.09.100180.002053](https://doi.org/10.1146/annurev.an.09.100180.002053)Choi, E., Lazaridou, A., Freitas, N.d. (2018) Compositional obverter communication learning from raw visual input. In: OpenReview.net (ed.) 6th International conference on learning representations: Conference track proceedings.\n\n[https://openreview.net/forum?id=rknt2Be0-](https://openreview.net/forum?id=rknt2Be0-)Austin, J. L. (1975).\n\n*How to do Things with Words: The William James lectures delivered at Harvard University in 1955*(2nd ed.). Clarendon Press.Clark, H. H. (1996).\n\n*Using language*(1st ed.). Cambridge University Press.Wittgenstein, L. (1989).\n\n*Philosophical investigations*(3rd ed.). Blackwell.Adler, R. B. (2012).\n\n*Interplay: The process of interpersonal communication*(3rd ed.). Oxford University Press.Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., iegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D. (2020) Language models are few-shot learners. In: Neural Information Processing Systems Foundation (ed.) Advances in neural information processing systems 33. Advances in neural information processing systems, vol. 33, pp. 1877–1901. Curran Associates Inc. https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf\n\nLauriola, I., Lavelli, A., & Aiolli, F. (2022). An introduction to deep learning in natural language processing: Models, techniques, and tools.\n\n*Neurocomputing,**470*, 443–456.[https://doi.org/10.1016/j.neucom.2021.05.103](https://doi.org/10.1016/j.neucom.2021.05.103)Khurana, D., Koli, A., Khatter, K., & Singh, S. (2022). Natural language processing: State of the art, current trends and challenges.\n\n*Multimedia Tools and Applications*.[https://doi.org/10.1007/s11042-022-13428-4](https://doi.org/10.1007/s11042-022-13428-4)Lappin, S. (2023). Assessing the strengths and weaknesses of large language models.\n\n*Journal of Logic, Language and Information*.[https://doi.org/10.1007/s10849-023-09409-x](https://doi.org/10.1007/s10849-023-09409-x)Lazaridou, A., Kuncoro, A., Gribovskaya, E., Agrawal, D., Liska, A., Terzi, T., Gimenez, M., Massond Autume, C., Kocisky, T., Ruder, S., Yogatama, D., Cao, K., Young, S., & Blunsom, P. (2021). Mind the gap: Assessing temporal generalization in neural language models. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. .S. Liang, & J. Wortman Vaughan (Eds.),\n\n*Advances in neural information processing systems (NeurIPS)*(pp. 29348–29363). Curran Associates Inc.Merrill, W., Goldberg, Y., Schwartz, R., & Smith, N. A. (2021). Provable limitations of acquiring meaning from ungrounded form: What will future language models understand?\n\n*Transactions of the Association for Computational Linguistics,**9*, 1047–1060.Liu, X., Yu, H., Zhang, H., Xu, Y., Lei, X., Lai, H., Gu, Y., Ding, H., Men, K., Yang, K., Zhang, S., Deng, X., Zeng, A., Du Zhengxiao, Zhang, C., Shen, S., Zhang, T., Su, Y., Sun, H., Huang, M., Dong, Y., Tang, J. (2024). Agentbench: Evaluating llms as agents. In: OpenReview.net (ed.) 12th International conference on learning representations.\n\n[https://openreview.net/forum?id=zAdUB0aCTQ</](https://openreview.net/forum?id=zAdUB0aCTQ)", "url": "https://wpnews.pro/news/emergent-language-a-survey-and-taxonomy", "canonical_source": "https://link.springer.com/article/10.1007/s10458-025-09691-y", "published_at": "2026-09-04 11:32:28+00:00", "updated_at": "2026-09-04 11:53:29.700678+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "natural-language-processing", "ai-research"], "entities": ["Autonomous Agents and Multi-Agent Systems", "Lewis"], "alternates": {"html": "https://wpnews.pro/news/emergent-language-a-survey-and-taxonomy", "markdown": "https://wpnews.pro/news/emergent-language-a-survey-and-taxonomy.md", "text": "https://wpnews.pro/news/emergent-language-a-survey-and-taxonomy.txt", "jsonld": "https://wpnews.pro/news/emergent-language-a-survey-and-taxonomy.jsonld"}}