Choosing the Best Method for Listening to the Customer

Once you decide what you want to know, from whom you want to know it, and what you will do with the data, you must carefully consider what method is best suited for gathering customer information for your Six Sigma project. Among the factors to consider when deciding on a research method are the length of time it will take to collect data, the type of data that will be collected, the cost of the collection method, and the advantages and disadvantages of each method.

Here is a table which summarizes these aspects of eight types of data collection. It is followed by a short discussion of each method.

Telephone Surveys

The results of a telephone survey of a randomly selected sample can usually be generalized to an entire population. For example, if a person wants to learn how satisfied customers are with a type of software, a survey of randomly selected individuals who have purchased that software will provide the information. A standard questionnaire must be developed, a sample selected, and interviewers must make the phone calls and record the data.

This type of data can produce valuable information but it is expensive. A 10-minute telephone interview with a random sample could cost in the neighborhood of $20,000 or more; the actual cost will, of course, depend on a number of factors such as how many people will be interviewed and how readily available an accurate sample of the target group is. Actual data collection usually takes a couple weeks although the preparation time needed to write the questionnaire can be several weeks.

The primary disadvantage of this method is that standardized, quantifiable telephone surveys are somewhat rigid. For example, all questions must be asked to all respondents in the same manner. Question wording should not vary. With some exceptions, all response categories must be the same and all inclusive. Also, no visual aids can be used.

Mail Surveys

A mail survey also can yield quantifiable data that can be generalized to an entire population and typically at a much lower cost than a telephone survey. However, the survey can take several months to complete and response rates are low. As a rule of thumb, the lower the response rate, the less reliable the data. For example, if a questionnaire is sent out to learn whether customers are satisfied with that same software, a small percentage – maybe 5 to 20 percent – will usually return the survey. Follow-up letters and questionnaires must be sent which adds to the cost and, more importantly, the time it takes to complete the study. After all of those efforts, it is not unusual to end up with a completion rate of 40 or 50 percent. Fifty percent is generally considered the lowest acceptable response rate.

Focus Groups: In-Person

The trade off in an in-person focus group – a discussion of 5 to15 people, usually for two hours, guided by a moderator – is that detailed information can be obtained but the information cannot be generalized to a population larger than the group itself. Using the same software example as above, a questionnaire may be able to determine how satisfied the customer is with the product and perhaps even areas of dissatisfaction. In a focus group, the moderator can probe about the reasons behind the dissatisfaction and perhaps discuss some possible solutions to the problems. The discussion can be quite in-depth. Visual aids can be provided. Much flexibility exists. To help mediate the fact that the results cannot be generalized to a larger population, focus groups and surveys are often conducted in partnership which provides both qualitative and quantitative data.

The cost of a focus group varies somewhat depending on location and availability of participants. However, a focus group typically costs about $5,000. Travel also can be expensive. Groups should be conducted in a variety of locations throughout the area in which the product or service is used. That often can require clients, moderators and viewers to travel to the different locations. Although a focus group takes only a couple of hours to complete, recruiting participants and preparation of guidelines usually takes at least a couple of weeks.

Another issue to consider is that many focus group facilities have a list of people they tap into to participate in focus groups. They can and often do use the same people over and over. Although research has yet to be done on the effects of using the same individuals frequently as focus group participants, one might conclude that these people could lack the spontaneity or freshness that is sought from focus group participants.

Focus Groups: Online

While in-person focus groups have been used by researchers since the 1940s, online focus groups are a recent development. Online groups consist of a dozen or so people who log onto an Internet chat site at the same time. A moderator leads the group. The moderator types in questions and participants respond and a dialogue among participants ensues.

The biggest disadvantage is that this type of research is suited only for younger age groups. Individuals age 40 and over generally are not as comfortable or accustomed to online chatting. Their input can be stifled by the technology. In contrast, younger participants are comfortable and this method can yield a lot of information.

The cost of this method can be as much as an in-person focus group. The primary savings is in the elimination of travel expenses. Viewers, clients and the moderator can all log on from different locations.

One-on-One Interviews

One-on-one interviews typically provide qualitative information that cannot be generalized to a larger population. As with focus groups, however, the interview allows for detailed information that cannot be obtained from a survey. One disadvantage is that the interaction or discussion that results from a group is eliminated. There is only a dialogue between the interviewer and interviewee.

One-on-one interviews can be particularly useful for people with limited availability such as doctors, CEOs and celebrities. Getting a group of these people together in a room at the same time for a two-hour focus group discussion can be difficult. One-on-one interviews can be planned more easily around a busy individual’s schedule. Regardless, interviews with such people can still be difficult to obtain because of the demands on their time. And often high-profile individuals expect some type of financial incentive, like compensation for their time or a donation to a favorite charity in their name.

An advantage of one-on-one interviews is the cost usually is low. Of course, possible compensation for interviewees and the cost of the interviewer must be considered. If the same interviewer is used – which is often a good approach – travel expenses may become a factor as the interviewer travels to various locations to meet with the individuals.

Intercepts

Intercepts consist of approaching an individual in a public location. For example, if information is needed from mothers of infants, the interviewer may go to a shopping mall and approach people who fit that description and ask for their input. Often these individuals are provided an incentive, perhaps $10 or $20.

This information is qualitative in most respects. Intercepts also can yield limited quantitative information if enough interviews are conducted. However, the population to which the data is being generalized must be clearly noted. Using the above example, if data is gathered from 200 mothers with a standardized questionnaire who were randomly selected (such as every fourth woman with an infant), the data can be generalized only to women at the mall on that specific day and time frame who passed the location of the interviewer and who had brought an infant.

Another example of intercept data is to approach customers who are leaving a movie theater to ask them about their experience at the theater or their reaction to the movie, advertisements or whatever else was shown. Again, the researcher must be careful in generalizing the data. Interviewers also must be trained because they tend to want to approach only people who look friendly even though data is needed from all types of people.

User Testing

This methodology requires asking individuals to use a product, often while they are being observed. Or, instead of being observed, the individuals could be asked to keep a diary. User testing can be extremely valuable in understanding how to make a product easier to use. Web sites often undertake user testing to see if their site is easy to navigate. It can be determined at what point and why, for example, people who have initiated a purchase on a web site suddenly exit and the sale is lost. This data is primarily qualitative.

The time frame for user testing varies. If a group of people can be observed while navigating a web site, the data can be collected in one evening. However, research may need to done over the course of several weeks. The cost depends on specific situations but it generally is comparable to a focus group.

Customer Complaints

Obtaining input from customers who complain can provide insight into problem areas but it also is qualitative data that cannot be generalized. For example, if customers write letters or phone with complaints, there has been no systematic way of collecting the data. And it is often the case that people who complain are simply those who are habitual complainers, who have had a particularly bad experience, or who have the time to register a complaint. These individuals do not provide an accurate reading of the experiences of all customers. Customer complaints can provide a sense of where problems exist, but the data cannot be generalized.

The cost of collecting customer complaints usually is low. The only effort required is for someone to monitor and tabulate the complaints as they come in. It usually is necessary to collect complaints for several months to provide insights into any but the most obvious problems.