EmCom-Diffusion: Probing Visual Reflection in Emergent Languages via Image Generation

Researchers propose EmCom-Diffusion, a framework that measures visual reflection in emergent languages by reconstructing images from emergent messages using a finetuned diffusion model and comparing them to originals. Validated on MS-COCO, it captures visual content missed by existing metrics like CBM and TopSim.

arXiv:2607.03752v1 Announce Type: cross Abstract: Measuring the extent to which emergent languages encode the visual content of their inputs is an open problem. We refer to this property as visual reflection: the extent to which emergent messages preserve information about their source images that can be recovered without appeal to the speaker-listener pair that produced them. Existing metrics measure it only indirectly, through proxies such as human-defined concept inventories, natural-language captions, structural distance correlations, or Referential Game accuracy, each of which can either miss visual content the message encodes or credit content it does not. We propose EmCom-Diffusion, an evaluation framework that measures visual reflection directly: it reconstructs each input image from its emergent message and compares the reconstruction with the original image itself, rather than with human-defined targets. Concretely, it finetunes a pretrained text-to-image diffusion model on image, emergent-message pairs and scores visual reflection as the perceptual similarity between the reconstructed and original images, operating generatively rather than discriminatively. Instantiating it on MS-COCO with a Referential Game, we validate the metric against random and fixed-token baselines under three pretrained visual encoders, and compare it against four existing metrics CBM, supervised translation, TopSim, and R@1 . EmCom-Diffusion captures visual content the other metrics miss.