Different models get different questions correct
Questions where each synthetic sample achieved a question-level average absolute error of 5.0 percentage points or less, when compared with real ATP data
| GPT-5.1 | Both models | Claude Opus 4.6 |
|---|---|---|
| • Whether there are clear solutions to most big issues facing the country today | • Have an IRA, 401(k) or similar retirement account | • Satisfaction with the way things are going in this country today |
| • Whether 2026 will be better than or worse than 2025 | • Whether voting gives people like you some say about how government runs things | |
| • Preference for living in a community where houses are larger and farther apart or smaller and closer to each other | • Whether your side has been winning more often than losing in political issues that matter to you over the last few years | |
| • Concern about the cost of housing | • Whether the Trump administration’s tariff policies will have a positive or negative overall effect on the country | |
| • Participation in a political campaign, meeting, protest or rally in the last two years | • Favorability of suspending all asylum applications from people seeking to live in the U.S. to escape violence or danger | |
| • Whether it’s acceptable or not for federal immigration officers to wear face coverings that hide their identities while working | • Whether it matters or not which party wins control of Congress in the 2026 elections | |
| • Trump presidential approval | • Whether Trump will be a successful or unsuccessful president in the long run | |
| • Ease or difficulty of someone obtaining an abortion in the area where you live | • Whether Democratic congressional leaders this year should work with Trump to accomplish things or stand up to Trump on issues important to their voters | |
| • Whether police should be allowed to stop and search anyone who fits the general definition of a crime suspect | • Whether obtaining an abortion in the area where you live should be harder, easier or about the same as it is now” | |
| • Favor or oppose pausing visa applications from people in 75 countries seeking to legally immigrate to the U.S. | ||
| • Whether Trump’s economic policies have made economic conditions better, worse or have had not much of an effect” | ||
| • Whether you find talking about politics with people you disagree with interesting and informative or stressful and frustrating | ||
| • Confidence that Trump has the leadership skills needed to be president |
Note:
Source: Survey of U.S. adults and “digital twins” synthetic analysis using models set to low reasoning with extended profile information and expert reflection. Survey was conducted Jan. 20-26, 2026 (replicated March 2-3 with OpenAI GPT-5.1; and replicated March 9-12 and April 7-10 with Claude Opus 4.6).
Refer to the Wave 185 questionnairefor full question wording. Source: Survey of U.S. adults and “digital twins” synthetic analysis using models set to low reasoning with extended profile information and expert reflection. Survey was conducted Jan. 20-26, 2026 (replicated March 2-3 with OpenAI GPT-5.1; and replicated March 9-12 and April 7-10 with Claude Opus 4.6).
“Can AI Stand In for Human Survey-Takers? Not Really”
PEW RESEARCH CENTER
Different models get different questions correct
Questions where each synthetic sample achieved a question-level average absolute error of 5.0 percentage points or less, when compared with real ATP data
| Independent variable | n1 | n2 |
|---|---|---|
| Germany | 40 | 60 |
| Spain | 50 | 50 |
| France | 60 | 40 |
Note:
Source: Survey of U.S. adults and “digital twins” synthetic analysis using models set to low reasoning with extended profile information and expert reflection. Survey was conducted Jan. 20-26, 2026 (replicated March 2-3 with OpenAI GPT-5.1; and replicated March 9-12 and April 7-10 with Claude Opus 4.6).
Refer to the Wave 185 questionnairefor full question wording. Source: Survey of U.S. adults and “digital twins” synthetic analysis using models set to low reasoning with extended profile information and expert reflection. Survey was conducted Jan. 20-26, 2026 (replicated March 2-3 with OpenAI GPT-5.1; and replicated March 9-12 and April 7-10 with Claude Opus 4.6).
“Can AI Stand In for Human Survey-Takers? Not Really”
PEW RESEARCH CENTER