Opinion

How the research industry needs to change

Lyndall Spooner takes a hard look at the research industry and its clients, and finds that many businesses are less interested in “why” and the average consumer just doesn’t exist anymore. That means it’s time to change.

LYNDALL SPOONER

February 9, 2026 2:59

How the research industry needs to change

Lyndall Spooner is the founder and CEO of research agency 5D

The research industry is at a turning point. Not because it’s failing, but because the world around it has transformed and what many clients want has shifted, but too many of the industry’s practices haven’t changed.

There is a growing divide between how many research companies operate and how businesses use research. In a world going through significant cultural, technological and psychological change, too many researchers are reluctant to change “their craft”. But businesses want faster and cheaper answers, often without the depth of understanding.

Researchers are obsessed with finding out the “why” that drives behaviour, but many businesses are less interested in understanding the “why” today than in the past. In a lot of cases, insights have been distilled down to performance metrics and dashboards, and companies are only interested in meeting their KPIs, without really understanding what the metrics are actually measuring. At the same time, and despite decades of research practice proving that you can’t always ask people direct questions, a lot of research is being reduced to one question and the answers taken as valid.

Now, for the first time in decades, the research industry has permission to question everything. AI is transforming workflows, cultural shifts are rewriting consumer behaviour, and traditional research models are failing to explain what we’re seeing. Clinging to established practices is, in many cases, riskier than experimenting with new ones.

It’s not just the tools that need questioning, but the assumptions underneath them. If we have a blank sheet of paper to redesign the research industry on, what would we keep and what would we abandon?

The basics don’t change

The fundamentals would survive. Well-designed hypotheses. Rigorous analysis. These aren’t negotiable. How we apply them is where the opportunity lies.

The real danger isn’t change, it’s lazy change or change that isn’t well thought through such as new research methods that sacrifice accuracy for speed – delivering answers 30% faster but 40% less reliable – or flashy AI tools and synthetic data sources that are deployed without questioning how reliable and accurate they are.

Every researcher knows the difference between knowing what people do and understanding why they do it. The “what” is trackable: click patterns, purchase data, survey responses, and so on. The “why” is very different. It demands guiding people beyond their surface explanations into territory they haven’t fully explored themselves. That’s where real insight lives.

That work matters more now than ever. These days, consumers don’t fit into neat categories. Identity shifts constantly. Cultural reference points multiply across platforms and communities. Traditional demographics explain less with each passing year. The “average” consumer doesn’t exist.

The average consumer is a fiction (Midjourney)

The world has always been messy. Technology now gives us the ability to embrace that messiness, if we approach it with imagination and genuine curiosity rather than fear or a resistance to change.

Technology alone won’t transform the research industry. AI can process patterns faster than any human. It can spot correlations people would overlook. Technology can open up a range of new ways to understand the “why”, but fully understanding it still requires human judgment. It requires empathy and the intuition to recognise what questions need asking before any algorithm can find answers.

Beginning again

So, what would the industry look like if we rebuilt it from scratch? Job titles would sound different. The required skills would shift toward strategic thinking and creative problem-solving. Some traditional roles within agencies would disappear entirely. But the core purpose wouldn’t change: helping people understand other people in ways that drive meaningful action and generate business outcomes.

That purpose becomes more valuable as the world becomes more complex. Business leaders don’t need more confusions, they need clarity. They need insights that cut through noise. They need researchers who can translate human complexity into strategic direction without oversimplifying it into uselessness. That’s a rare and increasingly valuable skill, and one that demands less attachment to how things have always been done and more willingness to experiment and occasionally fail. It demands the confidence to say “this new method isn’t good enough yet” even when it’s faster and cheaper.

It also requires honesty about what technology can and cannot do. AI excels at pattern recognition and processing massive datasets. But it can struggle with context and nuance, and it misses the weight of human experience and emotion that shapes decision-making. The challenge is to use technology intentionally for what it does well while focusing human intelligence on synthesis, interpretation and the leap from data to meaning.

Great researchers need a genuine interest in understanding people, curiosity, rigorous thinking, and the ability to observe without bias. These skills don’t age. What has changed is the landscape: more data, more complexity, more tools promising easy answers.

The choice facing the industry isn’t between tradition and innovation. It’s between thoughtful evolution and reactive chaos, between using new capabilities to deepen understanding or letting them dilute it.

Lyndall Spooner is the founder and CEO of 5D.