Marketers Would Be Most Likely To Use A Focus Group In Which Stage Of New Product Development?

Tweet
Post
Share
Save
Get PDF
Buy Copies
Print

To every marketing executive, the necessity and value of test marketing are often murky issues. The problem is partly that new products aren't developed and put through their paces in a systematic enough way to let marketing men know when a test market is really in order. Compounding this difficulty is that the goals of test marketing are sometimes unclear and that the information, once gathered, is often improperly used. This article is an attempt to lay bare the bones of the issue. Beginning with an overview of sound new product development, it clarifies when a test market should be done, what its aims should be, and to what uses it should be put. Relying for many of their judgments on quoted first-person interview material with marketing executives, the authors finish with a postscript on how technological innovation can aid in test marketing.

The executive must weigh a myriad of information as he decides whether to go with a new product. To help him in his decision, he may consider test marketing to gather yet more information.

For the executive contemplating test marketing, we intend to focus first on three strategic questions:

When should you conduct a test market?

What can you learn from a test market?

How should you use information from a test market?

We will then explore recent developments in new product research—specifically, the use of simulation models and test markets in the laboratory environment.

The questions and answers that follow reflect test marketing experiences we gathered from 31 marketing research executives who had been involved in hundreds of new product introductions. (See the sidebar "Commentators" for the list of interviewees.) These products included packaged grocery items, health-care products, and consumer durables. The executives represent three orientations toward test markets—the manufacturer, the advertising agency, and the marketing research supplied.

Alvin A. Achenbaum, Principal, Canter & Achenbaum, Inc., New York

Valentine Appel, President, W.R. Simmons Company, New York

Joel P. Baumwoll, Senior Vice President, Research and Planning, J. Walter Thompson Company

Henry J. Claycamp, Vice President, Corporate Marketing, International Harvester Company, Chicago

Theodore F. Dunn, Senior Vice President, Director of Research Services, Benton & Bowles, New York

Robert J. Eggert, Staff Vice President, Economics and Marketing Research, RCA Corporation, New York

Gerald J. Eskin, Associate Professor of Business Administration, University of Iowa, Iowa City

R. Stephen Fountaine, Director of Marketing Research, Philip Morris, New York

Lawrence D. Gibson, Director of Marketing Research, General Mills, Inc., Minneapolis

Paul F. Hase, Vice President, Research 100, Princeton, New Jersey

Calvin Hodock, Vice President, Consumer Research, Carter Wallace, Inc., New York

Alfred A. Kuehn, President, Market Science Associates, New York

Robert J. Lavidge, President, Elrick & Lavidge, Inc., Chicago

Philip Levine, Senior Vice President, Director of Research, Ogilvy & Mather, New York

Lawrence M. Light, Research Director, Senior Vice President, BBD&O, New York

Benjamin Lipstein, Senior Vice President, Director of Research, SSC&B, New York

Kenneth Longman, President, Benson & Benson, Inc., Princeton, New Jersey

Richard Lysaker, Senior Vice President, Audits & Surveys, New York

John McMennamin, Vice President, Research and New Products Director, Norton Simon Communications, New York

Joseph Nearby, Director of Market Planning, McCall's Patterns, New York

Sheldon Newman, Director of Marketing Research, Chesebrough-Pond's, Inc., Greenwich, Connecticut

Robert Olins, Vice President and Director of Research, Young & Rubicam, Chicago

Alvin Ossip, Associate Director Marketing Research Department, General Foods Corporation, White Plains, New York

Arthur S. Pearson, Director of Marketing Services, Clairol, Inc., New York

Vincent Petrilli, Marketing Consultant, Chicago

Fred Posner, Associate Director, Marketing Research Department, General Foods Corporation, White Plains, New York

Robert W. Pratt, Jr., Manager, Business Research and Forecasting Operation, Major Appliance Business Group, General Electric Company

William Saint, Manager, Sales Forecasting and Sales Research, Chesebrough-Pond's, Inc., Greenwich

Raymond E. Schucker, Marketing Research, Consultant, Minneapolis

Michael von Gonten, Senior Vice President, Research Systems, Inc., Evansville, Indiana

William Weilbacher, Vice Chairman, Dancer Fitzgerald Sample, Inc., New York

Whether to test market is a compromise between collecting more information, at considerable direct and indirect cost, to reduce uncertainty, and introducing the product immediately to make money. As Posner points out, "Every month in test means giving up sales if the product turns out to be successful."

The direct costs of test marketing include a pilot plant to make the product, commercials, an advertising agency because media are not billed, point-of-sale material produced in small quantities, higher media expenses because of low volumes, couponing, sampling, and higher trade allowances to obtain distribution. For example, the cost for a typical two-city test market in 1975 was $250,000. Four tests were required to test two levels of ad weight. This meant a market testing cost of $500,000.

But there are also indirect costs. There is, for example, the cost of revealing a new product idea to a competitor. If he is better organized for new product introductions, he can beat you to the market. It is hard to estimate the cost of being second in the market, but marketing experience indicates that it means a lower market share and higher promotional costs.

Another indirect cost is that of exposure. "If you are promoting a family brand, you are exposing the name of your company as well as the brand," Clay-camp reminds.

"If you get a reputation for bombing, you may have trouble getting distribution the next time, or in enlisting the enthusiasm of the sales force," observes Kuehn. It is impossible to estimate these exposure costs.

Diversion of employee time and activities is an internal, out-of-pocket expense that is rarely quantified when the cost of test markets is estimated. "The internal costs for test marketing are incredible. They divert management attention and manpower from other activities," McMennamin warns.

Adds Newman, "Sales management objects to too many test markets because they take away from sales activities."

These high direct and indirect costs of test marketing provide a strong incentive for postponing the decision to test market until the marketing executive feels confident he has the three basic elements of the ideal product development plan secure: a successful product, a competitive marketing strategy, and a superior communication plan. Many researchers feel that test marketing should be the last check before rolling the product out into national distribution. "I would say the batting average ought to be somewhere between 65% and 75%," Pearson calculates.

Light maintains that "when you go into test market, you've made the decision to go. The test market can of course alter your plans by giving you a no-go. But in the absence of bad results, you continue."

The rising costs of test markets have led researchers to use less expensive methods to kill a product or advertising campaign early in the process of new product development. "You certainly don't want to use a test market to learn something that you could have learned earlier—test marketing is much too expensive," Longman insists.

Implicit in the comments of our marketing executives is a well-articulated new product development process. It is to this process that the executives tacitly appeal when they make their judgments on the costs of test marketing.

A conceptualization of this process appears in the exhibit, "An Idealized Process For New Product Development." Here we can see how a product and a communication plan should be refined before test marketing.

Exhibit An Idealized Process For New Product Development

When Should You Conduct a Test Market?

The first step in the new product sequence is an identification of opportunities—needs not being met adequately by products that are currently available. This phase of research frequently uses qualitative techniques borrowed from social psychology. One method often used is the focus-group or group-depth interview. This is a two-hour discussion in which 8 to 12 persons are asked to describe their experiences with present products or to recount how they handle specific problems.

For example, a pharmaceutical company may ask a group of young mothers how they treat a child's cold. Mothers' perceptions of the strengths and weaknesses of available products will come to light during these discussions. Such interviews provide concepts for new products or for extension of present products. These concepts are actually hypotheses that can then be tested with more quantitative methods such as the survey.

Elimination of all but a few of the concepts is naturally a subjective process. It may be based on the experience of marketing executives, the politics of the company, or the executives' definition of their business and its capacity. (Capacity should be defined to include not only production and financial capabilities, but also executives' ability to operate in the industry of the proposed product.)

The number of product concepts finally chosen depends on the resources of the company and its strategy for new products. Given limited resources, should a company select only one or two concepts and spend most of its resources on developing communication plans, or should it retain four or five product concepts and test only a few communication plans for each one? The answer to this question must reflect the history and strengths of the organization. If most of its resources are in product R&D, it will probably experiment more with concepts than if its strength is in marketing and communications.

The next step in our ideal new product development is creating a preliminary profit plan that estimates the length of the payout. Based largely on estimates by experienced marketing executives, this profit plan can eliminate product candidates that do not reach the minimum payout period set by management.

(Each stage in the exhibit replaces management estimates with data, thereby reducing the range of uncertainty in estimating the profit plan.)

The center section of the exhibit has three columns—product development, strategy development, and communication development. These columns emphasize a very important point: the steps contained in them are taken simultaneously, not serially. Advertising development, for example, does not wait until a product is refined before starting its work, but begins immediately on the problem of communicating the concept reflected in the item.

Anticipated availability of a product in part determines the methods for its testing. For instance, a minimum of facilities may be required to produce actual products for testing a product extension such as the addition of a lemon flavor to an existing brand. In contrast, a new home entertainment center requires a whole new production facility. In this latter case, the concept is pretested in the first stage of product testing and the product in the second stage. (See the exhibit.) At the conclusion of these concept and product tests, the profit plan can then be revised to take into account information derived from the research. These new data suggest levels of production costs and therefore of product acceptability. If a proposed product meets the minimum payout period, it can then go into extended product use tests.

Marketing strategy requires setting goals, pricing strategies, and distribution strategies for a new product. Rough estimates of price ranges may be part of product testing. If your company plans to use its present channels of distribution, the channel strategy may be that of determining how to get your channel to accept the new product. If the strategy includes the development of a new channel—such as was required by the successful Hanes strategy of distributing "L'eggs" hosiery through supermarkets—considerable work may be needed to estimate the costs of such a tactic.

Communication strategy includes the development of the package, the copy theme for advertisements, and the selection of the desired media mix. It is important to note that the communication plan feeds into the development of the marketing mix plans for strategy development. The estimated cost of the communication plan is input for the final estimated profit plan. The plan is then submitted to management for approval in order to proceed to the test market stage.

When, then, should a test market be done? The experience of our experts suggests strongly that, as Saint says, "You go into test market under the assumption that you have a successful product."

What Can You Learn?

But when product and communications research have already told you that you have a successful product and communication plan, why should you go to the expense of test marketing, adding to the costs of delayed revenues from a successful item?

Our survey indicates that the primary purpose of a test market is to provide, as Newman puts it, a "good dress rehearsal for testing all of the product elements together, not individually." Before the test market, the product and the communication plan have been tested separately. The test market combines both these elements. Furthermore, it tests the elements of the plan and their combinations in the real world. It is, Posner says, a "microcosm of what you will be doing and what will happen nationally."

A real-world test of the marketing plan will provide estimates of marketing plan productivity, suggestions for improving the plan's productivity, and a disaster check.

Productivity of the marketing plan

The test market provides measures of consumers' responses to those elements that have been pretested—the product, the price, and the communication plan. It also measures the trade's acceptance of the product, the strategy, and the communication plan. (In most cases, the company can rely on its experience with the trade to estimate future acceptance.)

Consumer response

By measuring levels of consumer awareness, product trial, repeat purchase, market share, and sales volume, the test market gives some indication of the productivity of the elements of the marketing plan. These measures are the basis for making the "go/no-go" decision. Such a decision can be made on the basis of rule-of-thumb—for example, drop the product if the trial rate is less than 60%. The decision is then a financial one, the data from the test markets having been inputs into the profit plan.

Test markets provide better estimates of consumer response than any pretesting. But, as Claycamp emphasizes, "About all you can expect from a test market is the answers to the questions that you are asking of it. In other words, one has to be very specific about what one wants from a test market. For this reason, they really are designed only for go/no-go decisions."

Pearson agrees: "Test marketing can tell you if you have a loser, a medium success, or a giant success. It cannot fine-tune a payout plan between a two share and a three share."

However, Ossip asserts that "we find test markets useful in telling us whether we will make our minimum."

Lavidge sees two reasons for a test market: "One reason to test market is to decide how much of the product is likely to sell so that you can decide whether to go national. The other reason to test market is to determine how to market the product."

Trade response

"How to market" includes the question of trade acceptance—which, in many cases, has not been tested in the process of new product development. Saint describes the problem: "Suppose Chesebrough came out with an entirely new product in an unfamiliar product category. The Chesebrough sales force and Chesebrough as a company have built a fine reputation in the health and beauty-aid field. But in this case you would be taking the Chesebrough salesman and putting him in a new buyer's office. The soap buyer wouldn't know the salesman. So you would want to see the capacity of your sales force to sell an unfamiliar product to unfamiliar buyers." Thus acceptance of a new product is greatly dependent on experience and a successful track record.

Experience in the product category enables the researcher to make reliable estimates of trade acceptance. "If you don't know what kind of distribution your sales force is capable of getting, either you don't know much about your sales people or else they are very unreliable," Pearson contends.

Experienced researchers estimate that the effect of the extra emphasis given by the sales force in the test market gains a 15% to 17% higher sales level than would be achieved after having gone national with the product. Eggert affirms this: "In a test, you get more emphasis, since the sales manager visits the place two or three times. And just the sheer fact that he visits adds impetus to a product."

Being first with a new product is extremely important in the grocery-product category, where limited shelf space in a store constrains distribution. When Gillette successfully introduced "The Dry Look," a hair spray for men, several other companies tried to cash in on the success by introducing hair sprays of their own. Since hair sprays came in much larger containers than most hair preparations for men, more shelf space was required to stock the same number of competing brands. And because retailers tend to devote a fixed amount of shelf space to a particular product category, many of the companies with me-too products had difficulty getting trade acceptance for copied items.

When measuring trade acceptance is not a problem (as is often the case), a marketer may use a controlled store test, which greatly reduces the time required to run a test market.

A successful test market can improve the sales force's ability to obtain distribution during national introduction. A resounding success in, say, Albany is often useful to the salesman seeking trade acceptance in St. Louis. The salesman is equipped with real market results, not in-house test results or the results of a survey. The only question is whether the results in Albany are applicable to the situation in St. Louis.

How to improve marketing productivity

As we discussed earlier, you should run a test market only after extensive pretesting has shown that your new product will be a winner. And even though it is useful to get better estimates of consumer and trade acceptance, some marketers design their test markets to provide these better estimates and to provide information that will enable them to improve the productivity of the plan. As Schucker says, "Before test marketing, you are convinced at least that the business is there. So the question becomes one of how high you can go, how much leverage through your marketing variables you can apply to the business to make it bigger."

And von Gonten agrees: "I don't want to go to test market with anything that I don't know will be a success. And I will use the test market to derive a means to make my product an even bigger success."

To use the test market to improve the productivity of the marketing plan requires a research design that tests variations in the plan. "There is more to be learned from test marketing than an estimate of trial and repeat. There are the effects of sampling to be learned. There are also the effects of varying levels of promotion to be learned," Newman explains.

We conducted an informal survey of 25 marketing research executives attending a seminar on test marketing at the University of North Carolina at Chapel Hill to learn how they recommend using test markets. These are the results:

Saint's experience with "Vaseline Intensive Care" hand lotion illustrates how test markets can improve the productivity of the marketing mix. "We tested in three markets. In two of the markets we went with straight advertising, and in the third market we used a combination of advertising and sampling. The success we found in that third market led us to a national program which used a combination of sampling and advertising."

However, research designs can be complicated when you are testing several variables in the marketing mix. For example, the famous DuPont test of advertising for cookware coated with "Teflon" used 13 cities to measure the direct and cumulative effects of three levels of television advertising.¹ As the number of elements or levels to be tested increases, the number of cities, and therefore the costs of test marketing, increase.

Newman feels that "ideally, you should have a minimum of four cities on each variable," while Schucker asserts that "you need at least two markets per variable." Included in these increased costs are the expense of getting distribution in an increased number of cities across the country. Similar problems arise in purchasing advertising.

Marketers who want to use test marketing to test several elements in the marketing mix may want to consider an approach used by Michael von Gonten. This approach uses a single large city as the test site and treats neighborhoods as separate markets. The logistical problems associated with multiple-city tests are therefore greatly reduced. A single-city test may involve only one sales district. Shipping and warehousing costs are thus reduced. Media purchases are simplified.

Another feature of this design is a competitive one. Testing in a single city reduces exposure to competitors and may make it difficult for them to know the nature of the test. Variables under the control of the marketer—such as sampling, couponing, and in-store variations—may work better in this design than media tests because few print and electronic media have split runs within cities.

It is naturally possible to design a test that is too complex. As Fountaine says, "The further you get from reality, the more likely you are to make an error."

Pearson avers: "You better not do too many different things in test market because, if you do, you are going to confound your purposes."

How to avoid disaster

Murphy's Law is familiar to most businessmen: "If anything can go wrong, it will." This law is particularly descriptive of the new product development process. Disasters are common in the development and introduction of new products, and test markets are one of the ways to avoid as many disasters as possible.

The following is a partial list of the variety of things that have gone wrong in the development of a new product:

Because packages would not stack, the scouring pads fell off the shelf.

A dog food discolored on the store shelves.

In cold weather, baby food separated into a clear liquid and a sludge.

In hot weather, cigarettes in a new package dried out.

A pet food gave the test animals diarrhea.

When it was combined with a price reduction, a product change in a liquid detergent was thought by consumers to be dilution with water.

Because of insufficient glue, over half of the packages came apart during transit.

Excessive settling in a box of paper tissues caused the box to be one-third empty at purchase.

A large proportion of samples was not delivered until long after the test.

It is difficult, however, to justify a test market solely as a disaster check. As Pearson points out, "Some people put a lot of emphasis on the fact that a test market is a dry run. My feeling is that it is an awfully expensive dry run."

How Should You Use Test Data?

Test marketing's role is the evaluation of marketing plans. Many researchers caution against two major misuses.

1. Evaluation, not generation or development

Test marketing is the last step in a process whose goal is a successful product. The "ideal" process begins with the generation of new product ideas and ends with the evaluation of a complete marketing plan. The beginning of this process, as we showed in the exhibit, is designed to generate many ideas with a minimum of evaluation. These ideas are then subjected to a series of screenings using criteria established by corporate policy.

For instance, the first screen in the exhibit is a financial one—a rough profit plan that compares the estimated payout period with the maximum period allowed by corporate policy. Concepts that pass through this screen receive further evaluation as they pass through successive screens during product and communication development. A concept that fails to pass through a screen may be killed, or it may be reevaluated and revised.

The outputs of the product and communication developments should be combined with the marketing strategy to form a complete marketing plan. From this point, the process should be concerned with the evaluation of the plan and some fine-tuning, not with generation or development.

If idea generation and concept development appear at the test market stage, these reflect an unwillingness on the part of management to kill products earlier. A low "kill ratio" early in the screening process indicates that criteria are not stringent enough or that management is overriding them.

Gibson warns: "Test marketing is a big, expensive thing. If your process lets everything get through to test market, you are, by definition, not going to work on many ideas and you are not going to have many successes unless you just happen to have blind, dumb luck. You have to have a process that permits lots of ideas at the beginning that gradually get weeded out."

The process for product development is not a mechanical one. It is open to inputs from executives with creative spirit. And a creative suggestion may, of course, override a criterion that has been established for a new product policy. It is important to remember, however, that what is needed first is, according to Pearson, a policy that clearly says:

"No concept that scores below a certain level will proceed to the next step. Just because you have such a policy naturally doesn't mean that you shouldn't sometimes bypass it. But at least you should spell out the general rules so you know what you're bypassing.

"We should articulate a policy that says, 'Well, sure, it is all right to bypass the system, but let's be sure everybody knows what the rules are and accepts the fact that we have made a change. Because somebody has a great insight, a great solution to a great marketing need, we are going to go full steam ahead to test market or even national markets.' You then preserve your flexibility, but at the same time you don't let it confuse your system. If most of your ideas go through a systematic process of development and evaluation, then at least you know that you have the odds working for you."

Why are criteria overridden? The answers seem to be related to differences in reward systems for researchers and product managers.

2. Evaluation of plans, not people

Product managers tend to say "go" if a product concept does not look bad; researchers want to say "stop" unless it looks good. This difference in philosophy is the result of differences in evaluating product managers and researchers: product managers succeed by having successful products; researchers are expected to prevent costly failures.

McMennamin observes that "because of the pressure to get something into test market, it's amazing how unconcerned product managers can be going into one. But when negative results start coming in, they get very detailed in their explanations of what's happening."

These differences in philosophy come into conflict in the new product development process during the establishment of criteria for evaluation. Researchers would like higher levels for earlier kill. Product managers would like lower levels so that a larger number of products could reach test market. The marketing executive must reconcile these differences by establishing screening criteria consistent with corporate objectives, resources, and opportunities.

In addition to establishing these criteria, marketing executives must establish the policies for overriding them. Researchers agree with the need to override criteria, but they argue that an override should be noted as an exception to the system. This notation protects the researcher from unfair criticism without constraining the product manager's creative spirit.

In conjunction with other information

To reiterate a point made earlier, "Test marketing," says Levine, "should not be used for trying to obtain sales-point estimates in a national market."

Posner concurs: "Management has regarded test marketing results as point estimates and has not accepted their imprecision. The results should be considered in economic-risk terms."

"Researchers bring management information which is, at best, qualitative. Management puts it with information it already has and then makes a decision," concludes Dunn. Management, like navigators, should use more than one source of information when sailing in dangerous waters.

The reality and integrative nature of a test market make it more credible to the executive than any other test procedure. This partially explains why executives focus primarily on test market results without integrating other data. A second reason for relying only on test markets is that it is difficult to assimilate the variety of information and judgments available to the marketing executive. But recently developed simulation models can help the marketing executive in combining information to make new product decisions.

Recent Developments

The high costs of test marketing—indirect and direct—provide an incentive for developing methods that circumvent this procedure. Laboratory and mathematical simulation models are two developments.

Laboratory simulation

In this procedure, a sample of consumers is exposed to test commercials and then allowed to shop in a simulated supermarket environment. Demographic, economic, and brand preference data can be obtained prior to the test. Followup interviews after home use measure product satisfaction and intentions to repurchase. These data can then be fed into mathematical models that predict market share.

There are three possible alternatives after the laboratory simulation. First, the product may be killed outright. Second, favorable laboratory simulation results may encourage the marketer to go directly to a regional rollout. Third, the laboratory simulation may be followed by a test market to measure the productivity of the marketing plan, including trade acceptance, to fine-tune the plan, and to serve as a disaster check.

Mathematical market simulation

Marketing researchers have established relationships among promotional expenditures, awareness, trial, and repeat purchase that are used in mathematical simulations to predict market share. Market share predictions become inputs to the profit plan that estimates the payout period.

Simulation models are used first to develop an initial payout estimate. At this stage in the process, the inputs to the model are executive judgments. Product-test data replace executive judgment in the next estimate of the payout period. The estimate can then be revised again by using the simulation model with data from the extended product-use test and communication tests.

Mathematical simulations complement test marketing in a variety of ways. "They force people to consider the variables with which they are dealing before they go into test market," says McMennamin. Models can be used to project test market results so that tests can be stopped sooner and they can be used for diagnostics after a test market is completed.

Final Note

The decision to test market should be made only after the executive has received a complete marketing plan based on comprehensive product and communication tests. Test markets should be conducted to learn about the productivity of the marketing plan in terms of both consumer and trade response, to learn ways to improve the productivity of the plan, and to avoid potential disasters. The results of the test market should be used to evaluate the marketing plan, not to generate additional ideas, and not for the evaluation of people.

Executives play a major role in the new product development process by establishing levels of criteria for evaluation and policies for overriding these levels. These levels and policies are part of the mechanism by which executives control the new product development process.

1. James C. Becknell, Jr. and Robert W. McIsaac, "Test Marketing Cookware Coated with 'Teflon,' " Journal of Advertising Research, September 1963, p. 2.

A version of this article appeared in the May 1976 issue of Harvard Business Review.