{"slug": "silicon-samples-match-the-answers-but-not-the-respondents", "title": "Silicon Samples Match the Answers but Not the Respondents", "summary": "A new arXiv paper (2610.30900v1) introduces the respondent coverage gap (RCG), a metric that compares synthetic survey respondents generated by large language models against real human respondents by their complete answer profiles rather than per-question distributions. On two US social surveys, all six LLMs evaluated left human respondents uncovered, and under the paper's main settings each model fell behind a baseline that samples each answer independently from the human marginals; every LLM also had a higher RCG than a Gaussian copula fitted to the human data. The authors argue existing evaluations of silicon sampling miss this gap because they only check individual-answer accuracy, per-question distributions and cross-question correlations.", "body_md": "arXiv:2610.30900v1 Announce Type: new \nAbstract: Silicon sampling generates survey respondents by having a large language model (LLM) answer a survey as a person with given demographic attributes, and the resulting synthetic populations are increasingly used in place of human ones. Existing evaluations focus on individual-answer accuracy, per-question answer distributions, and correlations across questions. None of these compares the respondents themselves, so a generator whose respondents have combinations of answers found in no human respondent can still score well if it reproduces every answer distribution. We propose the respondent coverage gap (RCG), which represents each respondent as a point defined by their complete answer profile and measures how far each human respondent is from the nearest generated one, relative to the distance between human samples. On two US social surveys, all six LLMs we evaluate leave human respondents uncovered, and under our main settings, they fall behind a baseline that samples each answer independently from the human marginals. Across the prompting and inference settings we test, every LLM has a higher RCG than a Gaussian copula fitted to the human data.", "url": "https://wpnews.pro/news/silicon-samples-match-the-answers-but-not-the-respondents", "canonical_source": "https://www.machinebrief.com/news/silicon-samples-match-the-answers-but-not-the-respondents-zkm8", "published_at": "2026-10-23 04:00:00+00:00", "updated_at": "2026-10-23 04:42:46.619549+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "machine-learning"], "entities": ["arXiv", "respondent coverage gap", "Gaussian copula"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/silicon-samples-match-the-answers-but-not-the-respondents", "markdown": "https://wpnews.pro/news/silicon-samples-match-the-answers-but-not-the-respondents.md", "text": "https://wpnews.pro/news/silicon-samples-match-the-answers-but-not-the-respondents.txt", "jsonld": "https://wpnews.pro/news/silicon-samples-match-the-answers-but-not-the-respondents.jsonld"}}