5 Things Not To Expect From Synthetic Audience
The potential of synthetic users is exciting—on-demand consumer feedback, instant insights, scalable testing across hundreds of personas.
But to use them effectively, it's just as important to understand what they can't do as it is to know what they can.
This chapter is about setting realistic expectations. Synthetic users are powerful tools for certain tasks, but they're not magic.
They have clear boundaries, and understanding those boundaries will help you avoid disappointment and make smarter decisions.
The Core Limitations You Need to Know
1. Emotional Depth and Cultural Nuance
Synthetic users can simulate responses, but they don't feel. They can't replicate the full spectrum of human emotions—the subtle intensity, the cultural context, the lived experiences that shape how people truly react to messages and products.
Research from the Nielsen Norman Group found that when asked about online course completion, synthetic users reported 100% completion rates—an unrealistically optimistic picture.
Real users, by contrast, admitted dropping out after the third course due to work demands and competing priorities.
2. Ethnographic and Experiential Context
Synthetic users excel at pattern-matching based on existing data. But they struggle—or fail entirely—with sensory and experiential research.
They can't taste your new beverage, feel the texture of your fabric, or experience the frustration of a confusing checkout flow.
According to AI research experts, synthetic users cannot produce behavioral data. AI can't actually use a product like a human does, which means it can't have the specific, lived experiences that drive real consumer decisions.
What this means for you: If your research involves physical products, in-store experiences, or tactile interactions, synthetic users won't give you valid feedback.
They're not a replacement for usability testing, sensory panels, or ethnographic observation.
3. Bias Inheritance and Propagation
Here's a critical limitation that deserves your attention: synthetic users can inherit and amplify biases present in their training data.
A 2024 study on AI-generated personas revealed a strong "positivity bias"—LLMs tend to create persona descriptions that are more successful, well-adjusted, and socially conscious than realistic population distributions would suggest.
Other research from Columbia and Stanford found that AI models express opinions more typical of people who are liberal, well-educated, and younger, while underrepresenting perspectives from older, more religious, or more conservative populations.
What this means for your brand: If you're testing concepts with synthetic users representing diverse demographics, be aware that the outputs may skew toward certain worldviews and miss the nuances of how different segments actually think and behave.
4. Limited Creative Originality
Synthetic users are excellent at mirroring patterns from existing data—predicting how consumers might respond based on past behaviors. But they're less effective at generating radically new ideas or anticipating unexpected consumer reactions.
They're pattern-followers, not true innovators. If you're looking for breakthrough insights, contrarian perspectives, or fresh creative directions, human research will always outperform synthetic simulation.
5. No True "Lived Experience"
Perhaps the most fundamental limitation: synthetic users don't live. They don't have childhoods, families, aspirations, or regrets.
They pattern-match responses based on data, but they lack the messy, complex, contradictory nature of actual human experience.
When the Nielsen Norman Group tested synthetic users on concept evaluation, they found that AI provided "favorable, vague feedback to nearly all ideas," including completely implausible solutions. Real users, by contrast, bring skepticism, practical constraints, and life experience to their feedback.
Conclusion
Synthetic users extend your reach—they don't replace your intuition or eliminate the need for human connection. The most effective research strategies combine synthetic speed with human depth.
Think of synthetic users as a powerful first layer in your research stack. They help you ask better questions, test more ideas, and arrive at human research with sharper hypotheses.
But they're not the finish line—they're the starting point that makes everything else smarter and more efficient.