Importance of Product Testing Throughout the Product Lifecycle

Product testing should start before your development team even has a product concept and continue throughout the product lifecycle. Testing opens doors to many ideas, allows you to make sure that you have a good product concept, and helps you improve on the product even after it s in the market. In this chapter, we give you an outline of tests that follow the flow of the product development process. You can read this chapter top to bottom to get an idea of the sequence of tests product developers use, or you can just skip through and pick out the tests you need for your current projects.

To achieve success with many of the tests we include in this chapter, you need the assistance of experts: marketers, engineers, facilitators, and so on. Some larger companies have a lot of expertise in-house. But if you need to hire it in, you can find these experts in many different ways: You can do an Internet search for the term Product Testing (you get around 91 million results), select a few firms, and talk with people to get ideas on narrowing down your search for where to find the help you need.

You can do an Internet search for the term Product Testing (you get around 91 million results), select a few firms, and talk with people to get ideas on narrowing down your search for where to find the help you need. You can network with a local professional group, like a Product Development and Management Association (PDMA) chapter, to ask for references to firms that can address your particular need. You can ask your marketing department to recommend firms it has used in the past. You can ask friends in other companies how they found someone to help them with their product testing.

Testing expertise is available at many cost levels. The size of your testing program should correlate to the size of the firm you work with, and to the size of the project. Don t make the mistake of paying more for the test than you expect to gain from the test results (yes, we ve seen people do this!). In all cases, you should budget your new product development project to cover the necessary testing.

Long before your new product development team has anything you can call a product, you can and should start testing. You have two ways to confirm that you have a good product concept: qualitative market research and quanti-tative market research. The difference between these methods is that qualita- tive research tends to be exploratory it addresses the question What could we do? while quantitative research confirms, in fact, that you should push ahead with a project, modify it, or toss it into the scrap heap.

You should conduct qualitative research with customers way before you even have a product concept. Early research helps your team understand customers needs so that you can come up with ideas for products that meet those needs. When you ask What could we do? qualitative research (in effect, the customer) answers the question. Customer visits, interviews with customers, and observation of customer environments and work/play processes reveal what they need and want. (Take a look at Chapter 4 for details on how to conduct this early research.)

After you do come up with a concept (see Chapter 5), you can use a number of other qualitative methods to zero in on the customer s reactions to differ- ent possible functions and features of proposed products, for example using con-joint analysis or simple preference statements and surveys. Qualitative research can guide your product development efforts, but what you discover in a qualitative study can t tell you how many people want a certain feature or how many customers might buy a product at a certain price. In other words, qualitative research may lead your team to develop a great product, but you can t count on it to support your business case. For that, you need quantitative research.

Quantitative research answers the how much and how many questions your development team needs to answer to develop the business case (see Chapter 12). In other words, you take something that represents the product or the product experience to customers you believe would want that prod- uct, and in so many words, ask Would you buy this? How much would you pay for it? And, When would you like to buy it? Quantitative research also helps establish the relative importance of different features. For example, your team may think that the blue whirly-gig on the top of the product is the coolest thing since sliced bread, but it may leave most of your target cus- tomers cold.

Quantitative research helps you come up with statistically significant conclusions about the relative importance of the following: The customer needs you identify through qualitative research What difference adding or subtracting a feature could make The number of people who are likely to buy the product you re developing; in other words, how large the total market for a product may be

The usefulness and validity of quantitative research depends on how well your product development team designs the research. If you ask the wrong customers whether they want a product or how much a product is worth to them, you could kill a potentially successful product or go to market with a product that your target customers don t really want (in terms of features, price point, quantity, and so on). Start with qualitative research so you can identify important questions in your customers minds, and follow that work with quantitative research to establish the significance of what you discover.

W When your new product development team moves into the design and devel- opment phases of the NPD process (see Chapter 9), you ll encounter a time when you need to build a prototype a physical representation of the prod- uct or product features that has previously existed only in someone s mind, or as words on a page or sketches in a software program. People interpretations of words, but seeing something in physical form greatly reduces potential confusion over the look, feel, and actions of a product idea or feature. Prototypes differ, depending on the industry and the kind of prod- uct. often have different

Theyre hugely important in making sure that the products you design and build actually perform as your test audiences expect. In the following sections, you discover the differences between fit, form, and function prototypes; you see what kind of prototypes work best in different situations; and you find out how to give rapid prototyping a try.

Building a prototype that includes all the parts of the final product (fit), looks just like you expect the final product to look (form), and does everything you want the finished product to do (function) takes much longer than creating a prototype that meets only one or two of these criteria. A prototype should represent the full fit, form, and function of the product as much as possible, but we all know that budget constraints sometimes require compromises. When remember what information you re hoping to get from your prototype test. A service product is more function than any- thing else, for example. A new paint color is more form (color, texture) and function (coverage) than it is fit. you have to compromise,

When your NPD team is building a prototype, make sure that all your mem- bers are totally clear about the purpose of the prototype: A fit prototype gives the team a chance to see how all the parts go together. Different members of the team may be working on different parts of the product. If you don t track how the parts fit, you may find the lens occupying the space that belongs to the shutter, or that the front seat of a car isn t designed in a way that allows the driver to access the glove compartment. A form prototype makes sure that the beautiful product you designed will be as beautiful in finished form. A 3-D version of the product may show flaws that the initial sketches hid. A form prototype also can gen- erate very interesting discussions with a focus group

A function prototype shows how the product will work. In the early stages, you shouldn t mind if wires are running all over the place or if the back of the product is made of cardboard. The point you want to prove with a function prototype is that when you push the button, the light really does go on and the bells really do ring.

Different businesses use different prototypes during the product develop- ment process. The Product Development and Management Association Glossary defines prototype as a physical model of the new product con- cept. However, you shouldn t take the word physical too literally. Software developers make prototypes that consist of code. Service developers make prototypes that consist of people walking through routines. And financial developers make prototypes that describe financial transactions in detail.

You have many different kinds of prototypes to choose from, depending on your business and product lines. We cover the basics in the following list: Models of physical products: When you hear the word prototype, you probably think of a physical model of a physical product, such as an air- plane model in a wind tunnel. A company uses physical prototypes as early in the design/development process as possible so product devel- opers can see and feel what they re working on. Product developers also show physical prototypes to customers to get feedback. Examples of set- tings where product developers might use prototypes to get customer feedback include focus groups and early Alpha Tests.

Software prototypes: Software is a virtual product whose functions can be represented with a series of screens that represent actual functions. The software product is all the code that users never see that s behind the user interface that is, the screens the users see or the sounds they hear. Service prototypes: To prototype a service, you really have to create a life-size model and test it with your customers. For example, if a hotel chain wants to offer a new way to clean rooms for long-term guests, it has to implement the change in one hotel to see whether it really works and to gauge guest reactions. You can avoid plenty of costly problems by testing your services before you send them out to market.

More and more product developers are focusing on solutions rather than products or services alone (see Chapter 5 for more on this topic). If your organization s product includes a service component, don t forget to proto- type and test the service part of it. For example, a new robot lawn mower may have to include training over the Internet or via CD- ROM to show purchasers how to set up the robot to work on their lawns. Testing the training module, as well as the lawn mower, ensures that purchasers of the lawn mowing solu- tion actually have a working product.

Rapid prototypes are models or full-size representations of physical products built from successive layers of liquid or powdered material that will harden to become the shapes that represent the product ideas. You use a machine to build up the representation from a physical or digital drawing or from a set of instructions from a computer aided design (CAD) system. Not so long ago, many companies switched to CAD because it s faster than paper and pencil and has the ability to generate dimensions and other data that paper models can t. CAD also enables teams that work in different places to share their work.

Product developers can look at and feel these rapid prototypes, turn them over, and share them with customers in idea sessions or focus groups. With their physical information and feedback, developers can revise, correct, and add to their original ideas quickly and painlessly. (In Chapter 14 you can find out how computer aided design and rapid prototyping connects to the digital revolution in product development.) .

For example, a CAD machine may have been used to develop a drawing of the Apple iPhone. A rapid prototype machine would use data to produce a physi- cal model of the phone that developers could use to test how users like the size and feel of the phone. This provides more valuable testing than a draw- ing alone could accomplish. The instructions that drive the rapid prototyping machine also can drive manufacturing robots that produce the case.

When your NPD team has sailed around a few of the bends in the river of development when you ve designed your product, have the ability to produce at least a small number, and have brought in customers to react to how the product looks and functions the time has come to put some of your product directly into your customers hands to see how they get along. Whether your team has made a sippy cup for babies or a sophisticated software product for bankers, you take a huge chance if you put your product into the market without seeing how your customers will respond to it. When your product hits the market, your team can t tell a baby how to grab the neat handles you designed, and you can t tell a banker how to find certain features. The customers will have to dance without your help, and your prod- uct has to be the teacher.

Product developers often talk about three stages of product use testing: Alpha, Beta, and Gamma (this last one is mostly used in software development). They also use Market Tests. This section discusses each of these tests.

The Alpha Test is the first time you put your real product into customers hands. It isn t a prototype test. You ve manufactured or produced your prod- uct the way you ll produce subsequent real products. Your finished product may have bugs, which leads to the point of the Alpha Test: to find out about the bugs now, not after you ve manufactured and distributed hundreds or thousands of products.

The Alpha Test is the first time you put your real product into customers hands. It isn t a prototype test. You ve manufactured or produced your product the way you ll produce subsequent real products. Your finished product may have bugs, which leads to the point of the Alpha Test: to find out about the bugs now, not after you ve manufactured and distributed hundreds or thousands of products.

Heres what you need for Alpha Tests: A clear understanding, from the product definition (see Chapter 9), of how the product should perform when in your customers hands Enough of the product, in its final or near-final form, to run the test Customers who will use the product Whenever possible, conduct Alpha Tests with people who work for your company or with people who are very friendly and won t spread the word if your new plastic bags break and they get spaghetti sauce all over them. Researchers who will observe and/or interview the customers Continuous reporting to assess how the product works for the customers

Beta Tests are field tests of products. By using a Beta Test, you get to see how your customers will use your product, allowing you to make modifications if necessary (although it s unusual and upsetting! to have to make major changes during Beta Testing). During Beta Tests, you can discover plenty about how to install your product, how to instruct customers to use it, and what kinds of questions customers may have. You may even pick up valuable tips about how to market your product and support it when it hits its market.

Heres what you need to conduct Beta Tests: Products that you ve debugged during Alpha Testing Products whose production mirrors the production you intend for your launch products A clear definition of your expectations for the products Customers who will use the products and participate in the research Work with customers who seem friendly, but make sure that their desire to please you doesn t prevent them from telling you what s wrong with the products. Researchers who will observe, test, and report on the progress A plan for making the changes to the product suggested by the test results

A Gamma Test is the final test phase when a product is in general release. It assesses how well your product performs, whether it addresses the customer needs it should address, and how well it satisfies customers. Often, what characterizes this test phase is the fact that a company will maintain extra staff to deal with problems that come up after a product s general release exposes it to unanticipated user conditions.

Teams in the software industry are more likely to use Gamma Tests. Most software products are so complex and have to operate in so many different technical environments that they re almost never completely bug-free. Often we re sure you know this from your own experience! -- a software firm will release a product to the open market knowing that while it doesn t have fatal bugs, it may have some relatively minor bugs. Finding those bugs, and sup- porting customers who run into them, is the essence of a Gamma Test. Some people say software never gets out of Gamma Testing.

Market Testing occurs when the new product and its marketing plan are tested together. It gives you a chance to find out how well your whole package works. It also helps you put together a more accurate forecast of how much and how consistently you can sell so you can update your business plan and prepare for launch (see Chapter 13). A Market Test should be carried out toward the end of the product development process.

Heres what you need to carry out a Market Test: Enough product in its final form complete with packaging, instructional guides, and any other materials that will allow you to put the product in a statistically valid number of users hands. The sample size is driven by the variability of the target market. If users in the target market are essentially the same, you can get away with a smaller amount of test product; if the users have many differences, you need enough product for a large sample. Advertising and other promotional materials

Advertising and other promotional materials A site, or several sites, where you can carry out the tests Researchers who will assess the test results A plan for responding to the test results Potential customers who represent the conditions in the market you built the product for. Food companies, for example, have to test every region of the target market that has different taste preferences. Software companies have to represent the variety of technical skills and operating systems found in their defined target market.