For Quality Public Opinion Research, Qualitative Research Comes First

Discusses the importance of focus groups in developing a feel for how people are thinking about an issue.

People often ask what makes Public Agenda’s opinion studies stand out. One way we break new ground is by reaching back to an old research technique: the focus group. Interview Americans face-to-face, with a flexible line of questioning, giving them a chance to talk in their own way about the things that matter to them, and you will always find something new and fascinating. And invariably, the result is a far better opinion study.

Focus groups make questionnaires better because they suggest fresh ways to word questions and come closer to capturing how people really talk about an issue. That’s important when you’re a stranger trying to convince someone to stay on the phone with you for 20 minutes to answer questions that are often not immediately relevant to their day-to-day life – for free.

More importantly, focus groups will reveal the unexpected: concerns that are on the public’s mind but absent from the press coverage, the expert debate and the positions of the political parties.

A good example was Public Agenda’s 1996 study on welfare reform. At the time, many experts believed that people were upset with welfare for the wrong reasons – because they vastly over-estimated the costs, or because polls were using the word “welfare” instead of “public assistance,” or because of misleading stories about welfare queens – and even because they hated the poor. What we found was that welfare by any other name would still rile the public, that welfare offended people’s values more than their pocketbooks, that they relied on their personal experiences with work, charity and welfare to negotiate their way through the issue. Moreover, if it were up to the public, welfare recipients would have been offered child care and education to help them transition to the workforce – hardly an indication of hatred toward the poor. The survey confirmed all of these findings. But the insights were first picked up through the focus groups.

Focus groups are also excellent for revealing the intensity of the public’s feeling about an issue. Again, take welfare reform as an example. If you used Public Agenda’s focus group research – and the subsequent survey – as a guide, you would have correctly predicted that the issue would not wane until something was done. That’s how much sustained energy and emotion it tapped.

Take the opposite example – Public Agenda’s work on school vouchers. Even when the concept was carefully explained to parents in focus groups – and it required a lot of explaining – it never took off. The proposal simply did not tap passion. That may partly explain the initiative’s lack of traction thus far.

The focus groups conducted in preparation for our 1999 school voucher study also gave us fair warning that parents did not have much knowledge or familiarity with the issue. This was a bit startling given the political heat and partisan infighting that school vouchers were, and still are, generating among elites. But the focus groups guided the questionnaire – we knew the survey had to ask simple questions, explain and break down concepts and, most importantly, give people a chance to say that they didn’t know, that they needed more information. Otherwise, we would be portraying attitudes that were simply unreliable.

For many reasons, today’s opinion researchers often have minimal direct contact with the Americans they survey. The typical starting point for creating a survey is “how have polls asked questions on this issue before?” But too often, it’s also the end point. Breaking new ground requires going beyond the polls of the past. It also requires going beyond the agenda of the experts, beyond the agenda of the partisans, even beyond the agenda set by the press. It means getting on the ground, listening firsthand to the voices of real people and what they are saying.

Today it is entirely possible to become a recognized public opinion expert without ever sitting down for face-to-face interviews with ordinary citizens and developing a feel for how people are thinking about an issue and why they answer as they do. Focus groups are not the only qualitative tool, and they have their own pitfalls. But without qualitative work of some kind, the very profession responsible for keeping its finger on the pulse of the public is in danger of being disconnected from it.