Ver. 1.21

1
TPI at Capital One
Tom Anderson Thomas.Anderson_at_capitalone.com 804-96
7-1688
Lamar Barnett Lamar.Barnett_at_capitalone.com 804-934
-8860

• CVSQAA Meeting
• 11/18/03

2
What is TPI?

• TPI is the Test Process Improvement model for
  assessing the maturity of test processes
• One of several models generally accepted by
  testing profession
• Prescriptive model Tells you how to improve, as
  well as describing current state
• 4 Levels, A, B, C D, where D is better than A
• 20 different Key Process areas, with levels in
  each
• Assessment is a point-in-time, a basis for
  improvement

For more info, read Test Process Improvement A
Practical Step-by-Step Guide to Structured
Testing by Tim Koomen and Martin Pol, ISBN
0201596245
3
TPI Model Attributes

• Pinpoints strong and weak points of an
  organization
• Helps formulate improvement actions
• Factors in life cycle, organization,
  infrastructure and tools, and techniques
• Independent of development methodology

4
TPI Key Areas
Key Areas
Test Maturity Matrix
Levels
Improvement Suggestions
Checkpoints
5
Types of Assessments

• Self-assessment
• Some or all of the people doing the assessment
  are performers of the work being assessed
• Internal assessment
• Assessors are from the same organization as the
  performers, but are not normally involved in the
  same projects
• Formal Assessment
• Assessment is performed by an independent firm
• Assessors usually certified in assessment model
  and techniques

In 4Q 2003, QS Enterprise Systems Test Teams will
perform Internal Assessments on each other
6
TPI at Capital One

• Quality Services adopted TPI as the model for
  assessing and improving test process maturity in
  2002
• Many test teams were assessed by Sogeti in late
  2002
• Used TPI Assessment on OneView program in
  mid-2003 as a basis to recommend improvements
• Test managers are now being trained to conduct
  TPI assessments internally
• Goal is to do Internal Assessments on all teams
  by 1/31/04, use as basis for improvement in 2004

Internal Assessments Complete
Internal Assessor Training
Improvement Plans
Sogeti Assessments
Re-assess
Improvements Based on 1Q04 Assessments
Model Selected
OneView Assessment
2003
2004
2005
7
My Experience in Process Audit

• ISO 9000 training TickIT Lead Auditor
• Duval County School Board HR Dept
• CIS, HTE, London Bridge process audits and
  improvements
• parts.com

8
Assessing Auditing

• How can I guarantee a successful assessment?
• Commando audit techniques
• Intelligence work
• Reporting Findings
• Follow up
• OneView example

9
TPI as an Audit Model

• Audit for Compliance Findings benchmarking, not
  strict adherence to regulations policies
• Make practical recommendations to improve work
  life

10
Audit Interviews?

• Why should internal audits involve interviews?
• One viewpoint is not as robust.
• Reading between the lines.

Adherence to achieve compliance is through
consistent, uniform practice!
11
What does a successfulaudit interview look like?
?
12
The Steve Irwin Model

• In search of the phenomenon.

13
The Stephen Covey Model

• Begin with the End in Mind.
• First Things First.
• Think Win-Win.

14
The Tom Brokaw Model

• Separate Fact From Perception Feelings.

15
Interview Techniques
16
Interviews Etiquette Personalities

• Pre-interview Provide an agenda background
  material.
• Dress for success.
• Be punctual.
• Address with respect.
• Frame open-ended questions with charity.
• Avoid confrontational questioning.

17
Interviews The Offensive Game

• The Red Herring
• Obfuscation
• Stonewalling
• Disorganized / Cant Find It
• Interruptions
• Schedule conflicts
• Vacillation, uncertainty, indecisiveness

18
Interviews The Defensive Game

• Prevent defensive game by preparing.
• Remain calm.
• Remember you can leave.

19
Interview Difficulties

• Misunderstandings sometimes occur.
• Listen carefully actively.
• Address concerns fully repeat for feedback.
• Stay grounded in facts.

20
Validation of Responses

• Review and complete your notes soon after the
  interview.
• Refer to documentation.
• Chat informally with others in the group.
• Set up additional interviews.
• Phone or stop by for follow-up questions.

21
Pacing the Assessment

• Two weeks includes interviews, research, and
  reporting.
• Total effort should be less than 40 hours.

Dont forget scheduling delay! Schedule all
interviews and the end report up front!
22
Tracking Results

• Develop working hypotheses.
• Prove right or wrong.
• Follow-up with questions calls.

23
Scoring the Assessment
24
Develop the Score
25
Develop the Score

• KA 1.A.1 A motivated consideration of the
  product risks takes place
• Question What is your test strategy, and how
  did you formulate it?

26
Develop the Score
27
Develop the Score
28
Develop the Score
29
Finalizing the Presentation

• Remember your audience.
• Should be a mere formality.
• Tell them more than they know.
• Provide actionable plan

30
The Report Presentation
31
Reporting Findings

• Management team receives report
• Written as well as verbal presentation
• No surprises

32
Leveraging TPI for OneView

• The 20 TPI Key Areas provided concrete
  suggestions for improvement
• Treat testing as a project, dedicate a manager
• Consolidate defect reporting with a DB tool
• Dedicate professional testing resources
• Involve test design resources early

33
Postscriptum
34
The Postscriptum

• After the presentation, follow up will be
  required. Share the formal document with
  participants.
• Schedule any f/u meetings.

35
The Take-Home

• Whats the take-home?

?