A couple of weeks ago I spoke (at the kind invitation of Audrey Carr) at InteractionCamp. This was a BarCamp-style open unconference, and it was excellent. The theme was designing interactive user experiences, and there was a great mix of speakers including Jon Lax, Sam Ladner, and Audrey herself. I spoke on the topic of why consumer research isn't always helpful in creative development.
Like most open-source events, one of the conditions of speaking at InteractionCamp was that we post our presentations online for everyone to access. I've been sorely remiss on that point. So here goes.
The presentation I gave is a collection of anecdotes about research not doing what it was supposed to do: help us get to better business decisions. These stories come from various places - from Malcolm Gladwell in Blink, from my own experience, from popular culture. Things like New Coke (with its 100,000 taste tests), the Aeron Chair, research on tennis players by Vic Braden, reactions to the Friends pilot, the early reviews of the iPod, and all of the great work done by Tim Wilson at UVA.
The point of the presentation is simple: people can't always tell you what they want, what they like, or give their true opinion on something. I don't think this point is seriously debated anymore, but still an awful lot of research continues to ask people, point blank, to state a preference regarding their favourite product/package/ ad/etc, or to explain their decision-making process. This is a bad idea for three reasons.
- People don't have consistent preferences in what they like: even simple things like taste tests are remarkably inconsistent. Our likes and dislikes, which we think of as stable and fixed, are actually very malleable and context dependent.
- People are bad at explaining their behaviour. Many decisions are unconscious. And when we don't know why or how we've done something, we often just make up stories (without realizing it). But even though they're untrue, these stories are plausible and believable (even to ourselves), so they're hard to dismiss or refute.
- Asking people to reflect on their preferences or behaviour actually can change their answers away from the truth, towards the conservative and the familiar, because they lack the language to express their real feelings. This is a problem referred to as the perils of introspection.
The slides are here:
I should mention - given the recent comments on research I've been making around here - that none of this is meant as anti-research. On the contrary, I want people to ask better questions, to get to better research methods and techniques, and to do smart research for the right reasons. But right now too much research is done out of habit, on auto-pilot, for playing it safe, or for political reasons rather than for true insight and illumination.