Pew just tested whether AI can replace survey takers
Imagine a vendor telling your People team it can skip the next employee pulse survey: feed an AI model your roster, let it predict how staff would answer, and save weeks of waiting on real responses. The Pew Research AI survey study, published September 30, 2026, tested that exact idea on real U.S. survey panelists.
The question behind it is one worth asking plainly: can AI replace survey takers without losing accuracy along the way?
Pew Research Center's conclusion in that same report is direct: AI models are not an adequate replacement for traditional polling on topics of broad public importance. That's a strong line from an organization that runs some of the most closely watched public-opinion panels in the country.
We're skeptical of any pitch that promises to replace real respondents with an AI-simulated panel, and Pew's test gives that skepticism some backing: confident-sounding numbers aren't the same as accurate ones.
Sources: Silicon Samples and Synthetic Surveys: Can AI Stand In for Human Respondents? | Pew Research Center
How the Pew Research AI survey study built AI stand-ins
To run the test, Pew gave an AI model each panelist's demographics and prior answers from its American Trends Panel, then asked the model to answer new survey questions as that person, an AI doppelganger standing in for the real respondent.
Pew tested three models this way: GPT-5.1 and GPT-5 nano from OpenAI, and Claude Opus 4.6 from Anthropic. Which model it used changed the results, worth remembering if a vendor ever claims an AI survey tool is accurate without naming the model behind it.
Sources: Methodology | Pew Research Center
AI answers missed real human answers by a wide margin
Pew's test is really about AI vs human survey data quality, and the Pew Research AI survey study doesn't flatter the AI side. Averaged across the three survey waves it replicated, AI-simulated answers differed from real human answers by about 12 percentage points.
On a question about AI chatbots survey accuracy, that gap is not a rounding error. It's enough to send a team toward the wrong decision entirely, especially on anything meant to track change over time.
The clearest failure was tracking a real-world shift. Pew's synthetic sample held presidential approval steady at 46% across both waves, while real respondents' approval fell from 37% to 34% over the same period, a shift the AI version never caught.
On a newly salient topic, data centers, real adults split their awareness roughly evenly, while the AI sample clustered heavily into one category. Pew reports that across the study, AI answers often erased much of the diversity found in real responses. On one international-relations question, the AI poll overstated a view by 56 percentage points.
| Topic | Real human result | AI synthetic result |
|---|---|---|
| Presidential approval, wave 1 to 2 | Fell from 37% to 34% | Held steady at 46% in both waves |
| Awareness of data centers | Views split, roughly even | Clustered heavily into one answer |
| An international-relations question | Lower share held this view | Held by 56 points more adults than really did |
Sources: AI survey samples poorly replicate human public opinion | Pew Research Center, AI surveys struggle with questions about current events | Pew Research Center, AI surveys fail to capture the diversity of public opinion | Pew Research Center
Where AI's guesses were most wrong
The errors weren't spread evenly. Pew names four groups where its AI stand-ins missed most often:
- Republicans and Republican-leaning adults
- Black adults
- Adults without a college degree
- Infrequent internet users
That pattern matters beyond politics: when an AI model answers as a group it represents poorly, the error tends to concentrate exactly where you'd most want accuracy, not around some harmless average.
Sources: AI survey samples poorly replicate human public opinion | Pew Research Center
What this means for your AI-adoption surveys
The Pew Research AI survey study focuses on U.S. public-opinion polling on Pew's own panel; it does not address internal or employee surveys, and doesn't say whether its findings would carry over. Pew's vice president of methods and innovation, Courtney Kennedy, had already framed the organization's position earlier in 2026:
“We don’t use AI to tell us what the public thinks.”
— Courtney Kennedy, Pew Research Center's vice president of methods and innovation · Source: Do AI and bogus respondents threaten polling’s future? | Pew Research Center
We think the subgroup finding is the one to sit with longest: if an AI model's guesses about Black adults or adults without a college degree were least reliable, that's reason enough to doubt any AI-simulated subgroup estimate inside your own company too.
Our own view is pragmatic: AI still has a place in survey work, categorizing open-text comments or writing analysis scripts, but not in generating the answers themselves.
Since Pew's synthetic sample missed a real shift in opinion between two waves, treat any quarter's AI-adoption numbers as a snapshot employees gave you, not a constant you can assume still holds.
Sources: Silicon Samples and Synthetic Surveys: Can AI Stand In for Human Respondents? | Pew Research Center, AI surveys struggle with questions about current events | Pew Research Center, Do AI and bogus respondents threaten polling’s future? | Pew Research Center
Keep real employees answering, not AI guessing
None of this means AI has no role near your AI-adoption survey. It does mean the actual answers should keep coming from your own people. A few moves worth making now, as our own suggestion rather than anything Pew's report instructs:
What Pew's report doesn't tell you is whether these error rates would look the same on narrower, workplace-specific questions, like how often someone on a sales team actually opens an AI assistant. The report doesn't test that, and we won't pretend otherwise.
If you're measuring AI adoption through employee surveys, the lesson holds either way: answers are only as good as the real people who gave them. AI Adoption Score's surveys are built to capture skills, habits and culture directly from employees, self-reported by design rather than simulated by a model.
Frequently asked questions
Does Pew's finding rule out using AI anywhere in survey work?
No. Pew's statement rules out using AI to generate respondent answers, not every use of AI in survey work. Pew states it has no current or future plans to use AI models to generate its own survey results, a policy specific to answer generation rather than other processing steps.
Which AI models did Pew test, and did it test the whole market?
No. Pew tested three models: GPT-5.1, GPT-5 nano and Claude Opus 4.6, not every model on the market. Because Claude Opus 4.6 cost more to run, Pew used it on a smaller panelist subsample, and found that switching models changed the resulting picture of public opinion.
Does Pew's report name or evaluate any AI survey vendor or platform?
No. Pew's report describes its own internal testing process comparing AI-simulated responses with its American Trends Panel; it doesn't name, endorse or evaluate any commercial AI survey vendor or platform. Readers assessing a vendor's claims would need to request that vendor's own accuracy data separately.
How often should we re-survey employees on AI adoption?
Pew's report doesn't set a cadence, since it isn't about employee surveys. Beyond a regular cycle, consider re-surveying right after a training rollout or a major change in available AI tools, so you catch a shift in response rather than waiting for the next scheduled check.
Sources: Silicon Samples and Synthetic Surveys: Can AI Stand In for Human Respondents? | Pew Research Center, Methodology | Pew Research Center
