Tilly and Focus Groups

I received a number of interesting responses to the piece I wrote for the New Yorker last week on Charles Tilly's new book "Why."

www.gladwell.com/2006/2006_04_10_a_why.html

For those who haven't read it, Tilly provides a taxonomy of reason-giving. We emply four kinds of explanations, he says: conventions (social formulae), stories (common sense narratives), codes (legal formulae) and technical accounts (specialized stories). And we get into trouble when we use one kind of reason in a context where another is necessary.  What's  fun about Tilly's argument is that it provides a way of understanding all kinds of problematic social interactions. In the piece, I talk about the difficulty children have in understanding the admonition "don't be a tattle-tale" and the Dick Cheney shooting accident and restorative justice—among other things. Here is a comment from a reader, Jason Oke, who is the senior planner at Leo Burnett in Toronto. It's an elegant extension of the arguments I made against focus groups in "Blink," this time using Tilly's taxonomy: 

I think, as I gather you do, that how we feel about a brand, and which products and services we choose, is usually explained by a fantastically complex set of factors: the brands our parents used, the brands we see people around us use, the image of the brand, our personal experience with it, a sale, a half-remembered ad from 10 years ago, and so on.  This is probably best explained as a story - we may both buy Tide, but there's a different narrative that brought each of us to pick it up. 

But in market research, the answers people give sound more like conventions: "It's a good value", "my family likes it", "it tastes good."  And it seems that because of the artificiality of the situation, the perils of introspection, etc, most market research actually encourages people to answer in conventions, and doesn't encourage the telling of stories.  Many of these stories are probably complex and deeply buried such that they are hard to consciously access anyway.

Comments

[Dane]

The best approach to advertising campaign,therefore,is to combine two types of reasongiving:stories (common sense narratives)and conventions (social formulae).Let sampled customers start with a narrative with strong personal touches,then have him or her come,almost naturally,to a conventional conclusion:They're really great,blah blah...

[Adam in Racine WI]

I am a market research professional, and I have moderated a few focus groups. What Jason from Leo Burnett is saying, I find to be more true of quantitative research (ie paper surveys, scripted telephone surveys) but not so much qualitative research (focus groups, open ended one-on-one interviews).

A good focus group will be full of stories and (if you recruited well) technical accounts. I think people rely on what Tilly calls conventions when they are unable to articulate their behaviors and motivations through stories. In my experience, if you hear an excess of conventions and a shortage of stories in a focus group, then the moderator is not asking the right questions!

Quantitative research such as scripted telephone interviews, on the other hand, are BUILT on conventions! Due to limitations of time and in the interest of having an objective metric for all surveys, it's really all you can do.

Whether or not focus groups or telephone interviews are reliable sources of consumer input is a debate for another day. There are a lot of biases that come into play doing any kind of market research, as Jason alluded to.

Just thought I would toss my two cents in there about conventions and focus groups.

[Cobb]

Very interesting. I've just been thinking about decision heuristics along market terms for the politics of 'voting with your feet' and multiple intelligences.

http://cobb.typepad.com/cobb/2006/04/on_manufactured.html

[tom Bamberger]

What about Hank Aaron?

He hit over 40 homers when he was 23, 26, 28, 29, 32, 35, 36, 39.

[Arnie McKinnis]

Market research is used to provide lots of CYA coverage. Most companies can not make a decision (strategic or tactical) without reams of data. How big is the market? Who’s the target customer and what is their share of wallet? What are the budget estimates for such-n-so? Decisions, that affect people’s lives are based upon these might tomes of knowledge about a particular subject.

And the sad truth is that most of it is bunk. I’m not saying anyone is unethical in creation of the data – researchers are very meticulous with how questions are worded, in what order, influences of the interview, all sorts of things. And the people answering the questions are usually honest in their answers. But I going to expose a major flaw…

Sometimes either the questions are wrong or people answering those questions are not the right people. And there is not way to compensate for these errors within such a small sampling of people. You just have to go with answers and everyone involved just turns a blind eye to these flaws. Because, at the end of the day – I just need something or someone to point a finger at.

BTW, great article.

[MT]

Conventional explanations are what you often get from children, and you get them--"from the mouths of babes"--before they understand any more than that they count as explanations--meaning they placate interlocutors. I'm not sure how, when and/or to what extent we ever learn to give an organically grown explanation for why we do what we do. Anyway, I'm of the school that even at our reflective best we're likely rationalizing. Would an emotionally aware, psychologically astute explanation be useful as an explanation to anybody but the subject?

[ana]

I am a firm believer that we are seduced by certain elements in a marketing campagne or advertising; the arguments are what we make up to convince ourselves that it is a good idea to buy. Other concrete elements are just there to help us form these self-convincing arguments.

[spiiderweb]

I believe why we buy also changes over time. I loved Fords because my father did and, as he was far more mechanical than I, he must have been right. When I got older I realised Toyotas were superior and never bought another Ford.

Another example is GE. I noticed over a few years that every single item I bought from GE had failed. During that time I bought from other suppliers and never had a failure. So experience taught me to avoid GE.

[Paul Bowman]

Interesting thoughts. Perhaps this is one reason why the best marketing is that which is able to tap into the deeper consciousness of potential consumers.

[Arnie McKinnis]

We can never know exactly why someone buys - sometimes they don't eve know (myself included) - and that's because each "customer" enters into the buying experience with their own unique "context". That's why market research is sometimes wrong, because they assume they are seeing one pattern (or a dominant trend) and that is true for the whole universe of purchasers. It's funny how in some of the largest ticket sales, it really comes down to how the purchaser and seller related to each other (or in the case of a lost sale, don't relate) - and that's has more to do with context than anything else.

[Chris]

Tilly's theory, as you describe it in the review, doesn't sound so bad. But the example of abortion that you used to illustrate the theory struck me as absolutely terrible: http://www.explananda.com/archives/001575.html

[Anne]

Probably stating the obvious here but the Web is an increasingly robust component of consumer decision-making, IMO. It's a combination of word-of-mouth and comparative research, all readily available via search engine. Through data and anecdotes, sites like Consumerist.com help create perceptions of good and bad service, product reliability, and new-product desirability that infect our consciousness with the speed of a contagious virus. Savyy marketers are already courting such blogs and sites with freebies.

[Overworm]

I liked the article and found all the examples compelling. That is, until I got to the abortion example. The description of the type of reasoning used doesn't jibe with my anectdotal experience.

I've heard pro-choice people utilize stories ("I had a baby when I was 16 because my parents wouldn't let me get an abortion, then they made me drop out of high school to take care of him. I love my baby, but I often think about how my life might be different.") And, I have often seen anti-abortion people fall back to the convention of "life begins at conception."

The example that most caught my attention was the one of the criminal/victim conversation. A few years ago, I was on a jury in a case of a guy who was being tried as a habitual felon. That meant that a guilty verdict would result in a sentence of nearly 30 years.

The man's first two felony convictions were (1) for breaking in an unoccupied home and stealing a VCR and less than $3 in change, and (2) selling fake crack cocaine. His current court case was for breaking in a neighbors tool shed and stealing a mower and other lawn tools.

I was having a hard time reconciling the fact that a man might have to spend 30 years in jail for such a pathetic assembly of crimes. Others in the jury took the opposite opinion and justified it to the group with conventional reasoning.

The problem for me was that a discussion concerning imprisoning a man for 30 years of his life demanded more than a rote decision. In fact, as things stood, I had entrenched my mind in a bunker that would have never allowed me to vote for a guilty verdict, solely on the fact that I didn't think the jury was giving the defendent the benefit of their full concentration.

I introduced story-form reasoning to the discussion, broadening the topic from the man's crime to what we had learned of the man himself from the testimonies. Immediately, a jury room that had been detatched and stale became electrified. After me, other jurists chimed in, some for conviction, some not quite, but they all had a story to explain their reasoning.

In the end, we found the man guilty and he was sentenced to 27 years in prison. The bottom line was that, while had never committed a violent crime, he had a very lengthy record that stretched nearly sixteen years, about half of his life. He'd already spent a third of his life in jail, and it seemed he spent all of his time as a free man committing crimes.

But he would have never been found guilty using only conventional reasoning. Once we moved from conventional reasoning to story reasoning, we resolved our differences were then able to move on to codes reasoning and apply the legal law.

That was an interesting article, and will cause me to think a little more about how I communicate with various people and in various situations.

[Ed Cone]

Enjoying your blog.

Please consider including more hyperlinks in your posts.

That way you don't have to break up the text to post a clunky URL.

Linking is like writing in 3-D, and it's an important medium of exchange for many bloggers.

Thanks for this interesting site.

[communicatrix]

Seconding Arnie's thoughts on CYA coverage in qualitative research. While I think there are (rare) cases where the truth can be ferreted out of a focus group, I think most times the agendas of the people on the other side of the glass preclude it.

Also seconding Ed's longing for hyperlinks. Oh, we love the linkage...

[Arnie McKinnis]

Commnicatrix - thanks!! I agree with you also, I've been on the "other side of the glass" several times in my career, and in most cases, I heard what I wanted to hear to justify either my position or my bosses. Also, thank you Overworm, great comment and real life example.

Malcom - love the blog, your article and the books -- keep 'em coming!

[Shawn Callahan]

Your restorative justice example is a powerful one. In my experience as a corporate anthropologist focus groups and interviews create the wrong environment to elicit stories because the participants want to authorative and learned. Getting a group to retell their stories and avoid judgement and option creates an entirely different dynamic.

My word of warning is to be careful how you use the term 'storytelling'. I find people interpret this as something that is made up, crafted. It is better to ask for examples, illustrations or just say 'What happened?' One story sparks the next and new insights emerge.

You might like this paper we wrote describing a technique we call anecdote circles which is all about eliciting workplace stories: http://www.anecdote.com.au/whitepapers/wp1.php

[Overworm]

You're welcome, Arnie. Thanks for the thanks.

[Steve Denning]

Tilly's book and your review are very interesting.

Where I have a quibble with the book is Tilly's taxonomy of explanations, which sorts them into four categories:
1. Conventions
2. Stories
3. Codes
4. Technical accounts.

What's the difference between conventions and codes? It turns out that "codes" are "high-level conventions", i.e. fancy conventions, but still conventions.

And what's the difference between stories and technical accounts? It turns out that "technical accounts" are "stories informed by specialized knowledge and authority," i.e. fancy stories, but still stories.

It seems that what we really have are two basic categories: (a) abstract conventions/codes and (b) stories. The big split - and the big merit of the book - is the distinction beween abstract explanations and narrative explanations.

Having four categories just confuses things, as there is no clear distinction between codes and conventions, or between stories and technical accounts.

Apparently Tilly couldn't resist creating one of those two by two matrices.

With this clarification in mind, the book makes a very useful contribution to an understanding of explanations.

[Dane]

Steve's view is well-taken.

[Thomas]

Malcolm, what is the scoop on Blink being made into a movie?

[Overworm]

If that's true, I want to play the part of Malcolm! I could grow back my afro and look close enough to the part. ;-)

[Todd Ruth]

Great blog!

[Phil Adikes]

A similar illustration of this comes to mind. When I was growing up in the NYC area, people would say "I am never buying a stereo from the Crazy Eddie guy because his ads are so obnoxios." The mention of him illustrated his top of mind association. His "lousy ads" gained him national aclaim and even made him the subject of satire on Saturday Night Live.

[Allison Hunt at HATCH Research Intelligence Inc.]

If a focus group participant is coughing up non-responses such as "it's good value" or "it's convenient" and the facilitator is accepting it, then you've got an excellent example of an awful focus group.

Having moderated over a thousand focus groups I assure you there are just two key ingredients to successful, productive qualitative market research.

First is a tireless facilitator who loves the challenge and will always dig deeper to get to the true heart of the matter.

Participants aren't necessarily that hard-working and don't freely give of themselves. However, they typically respond extremely well, and actually enjoy themselves, when provoked into deeper thinking. (Enjoying the session drives engagement. And it isn't about eating free muffins, it's about feeling like you're actually contributing in a meaningful way.)

Second are backroom observers who are watching the whole show and not just listening for what they want to hear to support what they already believe. If you're not listening with an open mind you might as well skip the bad chicken dinner and go home.

Finally, use the right tool for the job. Qualitative market research isn't going to make an ill-conceived strategy smart, or turn a bad advertising idea into strong communication.