RealCompanion: Benchmarking Human Understanding from Reasoning over Longitudinal Real-World Conversations

Researchers introduced RealCompanion, a benchmark for evaluating whether AI companions can reason over a real person's longitudinal conversation history, testing memory of past statements, inference about who the person is, and recognition of when past context bears on a new message. The benchmark addresses the privacy barrier that prevents using genuine personal records by generating the persona data synthetically.

A companion that talks with a person for months should come to understand them. It should remember what they said, infer who they are, and know when the past bears on the message in front of it. Testing this requires a real person's record, and such records are private, so benchmarks generate the pe