CONCEPTPROJECT EVALUATION - PowerPoint PPT Presentation

Title: CONCEPTPROJECT EVALUATION


1
CONCEPT/PROJECT EVALUATION

• THE CONCEPT EVALUATION SYSTEM

2
The Evaluation System
3
Cumulative Expenditures Curve
of expenditures
Many high-tech products
Many consumer products
Time
Launch
4
Risk/Payoff Matrix at Each Evaluation

• Cells AA and BB are correct decisions.
• Cells BA and AB are errors, but they have
  different cost and probability dimensions.

5
Planning the Evaluation System Four Concepts

• Rolling Evaluation (tentative nature of new
  products process)
• Potholes
• People
• Surrogates

6
Rolling Evaluation (or, "Everything is Tentative")

• Project is assessed continuously (rather than a
  single Go/No Go decision)
• Financial analysis also needs to be built up
  continuously
• Not enough data early on for complex financial
  analyses
• Run risk of killing off too many good ideas early
• Marketing begins early in the process
• Key new product participants avoid "good/bad"
  mindsets, avoid premature closure

7
Potholes

• Know what the really damaging problems are for
  your firm and focus on them when evaluating
  concepts.
• Example Campbell Soup focuses on
• 1. Manufacturing Cost
• 2. Taste

8
People

• Proposal may be hard to stop once there is buy-in
  on the concept.
• Need tough demanding hurdles, especially late in
  new products process.
• Personal risk associated with new product
  development.
• Need system that protects developers and offers
  reassurance (if warranted).

9
Surrogates

• Surrogate questions give clues to the real answer.

Real Question Surrogate
Question Will they prefer it? Did they
keep the prototype product we
gave them after the concept
test? Will cost be competitive? Does it match
our manufacturing skills? Will competition
leap in? What did they do last time? Will it
sell? Did it do well in field
testing?
10
An A-T-A-R Model of Innovation Diffusion

• Profits Units Sold x Profit Per Unit
• Units Sold Number of buying units
• x aware of product
• x who would try product if they
  can get it
• x to whom product is available
• x of triers who become repeat
  purchasers
• x Number of units repeaters buy in
  a year
• Profit Per Unit Revenue per unit - cost per unit

11
The A-T-A-R Model Definitions

• Buying Unit Purchase point (person or
  department/buying center).
• Aware Has heard about the new product with some
  characteristic that differentiates it.
• Available If the buyer wants to try the product,
  the effort to find it will be successful
  (expressed as a percentage).
• Trial Usually means a purchase or consumption of
  the product.
• Repeat The product is bought at least once more,
  or (for durables) recommended to others.

12
A-T-A-R Model Application

• 10 million Number of owners of Walkman-like
  CD players
• x 40 Percent awareness after one year
• x 20 Percent of "aware" owners who
  will try product
• x 70 Percent availability at
  electronics retailers
• x 20 Percent of triers who will buy a
  second unit
• x 50 Price per unit minus trade
  margins and discounts (100) minus unit cost at
  the intended volume (50)
• 5,600,000 Profits

13
Points to Note About A-T-A-R Model

• 1. Each factor is subject to estimation.
• Estimates improve with each step in the
  development phase.
• 2. Inadequate profit forecast can be improved by
  changing factors.
• If profit forecast is inadequate, look at
  each factor and see which can be improved, and at
  what cost.

14
Getting the Estimates for A-T-A-R Model

• xx Best source for that item.
• x Some knowledge gained.

15
CONCEPT/PROJECT EVALUATION

• CONCEPT TESTING

16
Many Ideas Are Eliminated Before Concept Testing

• PIC eliminates most new product ideas even before
  they are developed into concepts.
• Ideas of the following types are excluded
• Ideas requiring technologies the firm does not
  have.
• Ideas to be sold to customers about whom the firm
  has no close knowledge.
• Ideas that offer too much (or too little)
  innovativeness.
• Ideas wrong on other dimensions not low cost,
  too close to certain competitors, etc.

17
Market Analysis and Initial Reaction

• Market analysis in-depth study of market area
  that the PIC has selected for focus.
• Conducted immediately after PIC approval.
• Initial reaction preliminary, inexpensive
  assessment of concepts, which may be flowing very
  quickly at this point.
• Avoid bazooka effect (quickly blasting out
  concepts without forethought)
• Do not include idea source in initial reaction.
• Respect the fragility of ideas -- have more
  than a single person involved.
• Use more than pure intuition -- keep records and
  stay objective.

18
Suggested Questions for the Initial Reaction

• Market Worth what is the attractiveness of the
  new product to the targeted customer population?
• Firm Worth Is the new product project viewed
  positively by management? Does this new product
  project enhance the firms competencies?
• Competitive Insulation Can the products
  advantage be maintained against competitive
  retaliation?

19
Concept Testing Cautions and Concerns

• If the prime benefit is a personal sense (aroma,
  taste).
• If the concept involves new art and
  entertainment.
• If the concept embodies a new technology that
  users cannot visualize.
• If concept testing is mishandled by management,
  then blamed for product failure.
• If customers simply do not know what problems
  they have.

20
What Is a Product Concept Statement?

• A statement about anticipated product features
  (form or technology) that will yield selected
  benefits relative to other products or problem
  solutions already available.
• Example A new electric razor whose screen is so
  thin it can cut closer than any other electric
  razor on the market.

21
Purposes of Concept Testing

• To identify very poor concepts so that they can
  be eliminated.
• To estimate (at least crudely) the sales or trial
  rate the product would enjoy (buying intentions,
  early projection of market share).
• To help develop the idea (e.g. make tradeoffs
  among attributes).

22
Procedure for a Concept Test

• Prepare concept statement
• Clarify specific purposes
• Decide format(s)
• Select commercialization
• Determine price(s)
• Select respondent type(s)
• Select response situation
• Define the interview
• Conduct trial interviews
• Interview, tabulate, analyze

23
Some Key Issues in Concept Testing

• Concept statement narrative, drawing, model?
• Respondent group Lead users? Large users?
• Response situation Where? How?
• Interviewing sequence Believable? Important?
  Interesting? Would it work? What problems do they
  see? Would they buy?
• Test procedure, change and implement, study
  findings.

24
Mail Concept Test -- Verbal Description

• Here is a tasty, sparkling beverage that quenches
  thirst, refreshes, and makes the mouth tingle
  with a delightful flavor blend of orange, mint,
  and lime.
• It helps adults (and kids too) control weight by
  reducing the craving for sweets and between-meal
  snacks. And, best of all, it contains absolutely
  no calories.
• Comes in 12-ounce cans or bottles and costs 60
  cents each.
• 1. How different, if at all, do you think this
  diet soft drink would be from other available
  products now on the market that might be compared
  with it?
• Very different ( ) ( ) ( ) ( )
  Not at all different
• 2. Assuming you tried the product described above
  and liked it, about how often do you think you
  would buy it?
• More than once a week ( ) ( ) ( ) (
  ) ( ) ( ) Would never buy it

25
Mail Concept Test -- Sketch
26
Benefit Segmentation in Swimsuit Market
27
Joint Space Map Showing Ideal Points
3
2
Aqualine
Comfort
1
Molokai
Islands
Fashion
Splash
Sunflare
28
Joint Space Map Showing Ideal Vectors
Comfort
Benefit Segment 1
Aqualine
X
Overall
Islands
Molokai
Benefit Segment 2
Fashion
Sunflare
Splash
29
CONCEPT/PROJECT EVALUATION

• THE FULL SCREEN

30
The Full Screen

• A step often seen as a necessary evil, yet very
  powerful and with long-lasting effects.
• Forces pre-technical evaluation, and summarizes
  what must be done.
• Methods range from simple checklists to complex
  mathematical models.

31
Purposes of the Full Screen

• To decide whether technical resources should be
  devoted to the project.
• Feasibility of technical accomplishment -- can we
  do it?
• Feasibility of commercial accomplishment -- do we
  want to do it?
• To help manage the process.
• Recycle and rework concepts
• Rank order good concepts
• Track appraisals of failed concepts
• To encourage cross-functional communication.

32
Screening Alternatives

• Judgment/Managerial Opinion
• Concept Test followed by Sales Forecast
• (if only issue is whether consumers will like
  it)
• Scoring Models

33
A Simple Scoring Model
Answer Go boating.
34
Source of Scoring Factor Models
35
A Scoring Model for Full Screen

• Note this model only shows a few sample
  screening factors.
• Factor Score (1-5) Weight
  Weighted Score
• Technical Accomplishment
• Technical task difficulty
• Research skills required
• Rate of technological change
• Design superiority assurance
• Manufacturing equipment...
• Commercial Accomplishment
• Market volatility
• Probable market share
• Sales force requirements
• Competition to be faced
• Degree of unmet need...

36
The Scorers

• Scoring Team
• Major Functions (marketing, technical,
  operations, finance)
• New Products Managers
• Staff Specialists (IT, distribution,
  procurement, PR, HR)
• Problems with Scorers
• May be always optimistic/pessimistic
• May be "moody" (alternately optimistic and
  pessimistic)
• May always score neutral
• May be less reliable or accurate
• May be easily swayed by the group
• May be erratic

37
IRI Scoring Model

• Technical success factors
• Proprietary Position
• Competencies/Skills
• Technical Complexity
• Access to and Effective Use of External
  Technology
• Manufacturing Capability

• Commercial success factors
• Customer/Market Need
• Market/Brand Recognition
• Channels to Market
• Customer Strength
• Raw Materials/Components Supply
• Safety, Health and Environmental Risks

Source John Davis, Alan Fusfield, Eric Scriven,
and Gary Tritle, Determining a Projects
Probability of Success, Research-Technology
Management, May-June 2001, pp. 51-57.
38
Alternatives to the Full Screen

• Profile Sheet
• Empirical Model
• Expert Systems
• Analytic Hierarchy Process

39
A Profile Sheet
40
Empirical Screening Model

• (This example is based on Project NewProd
  database.)
• Eight Significant Factors
• Product superiority
• Overall firm/resource compatibility
• Market need, growth, and size
• Economic advantage of product to end user
• Technological resource compatibility
• Product scope (mass vs. narrow specialty)
• Market competitiveness (-)
• Newness to the firm (-)

41
Items Constituting the First Factor

• Factor One Product Superiority
• 1. Product is superior.
• 2. Product has unique feature.
• 3. Product is higher quality.
• 4. Product does unique task.
• 5. Product cuts user's costs.
• 6. Product is first of kind.
• (There are about six items constituting each of
  the other factors as well.)

42
Sample Items on Other Factors

• Factor Two Overall Company Project Fit
• Good fit in terms of managerial, marketing,
  engineering skills financial, RD, production
  resources
• Factor Three Market Need, Growth and Size
• High need level by customers for this product
  class
• Large, fast-growing market
• Factor Four Economic Advantage to User
• Product reduces customers costs
• Product is priced lower than competitors

43
Sample Items on Other Factors

• Factor Five Newness to the Firm
• New product class, customer need served,
  technology, production process, sales force or
  distribution
• Factor Six Technological Capability
• Good fit in terms of RD and engineering
  resources
• Factor Seven Market Competitiveness
• Intense price competition, many competitors, many
  new product introductions, changing user needs
• Factor Eight Product Scope
• Market-derived new product idea, not a custom
  product (has mass appeal), mass market exists for
  product

44
Sample Application of NewProd Screening Model

• Factor Mean Evaluation Impact
• Project Superiority 1.19 POSITIVE
• Economic Advantage -0.49 negative
• Company-Project Fit -0.16 marginal (-)
• Tech. Compatibility -0.19 marginal (-)
• Newness to Firm -0.24 marginal ()
• Market Need/Growth/Size 0.88 POSITIVE
• Market Competitiveness -1.82 positive
• Product Scope 0.90 marginal ()

45
Pros and Cons of Project

• Pros
• 1. Product Superiority/Quality
• 6. Market Need/Growth/Size
• 7. Market Competitiveness
• Cons
• 2. Economic Advantage to User
• Marginals
• 8. Product Scope
• 5. Newness to Firm
• 4. Technology Compatibility
• 3. Overall Company-Project Fit

46
Analytic Hierarchy Process (AHP)
47
Partial Input to AHP
48
Abbreviated Output from AHP