Test and Evaluation Master Plan TEMP OVERVIEW

1
Test and Evaluation Master Plan (TEMP)OVERVIEW
As of 18 April 01

• US Army Test and Evaluation Management Agency
• 200 Army Pentagon
• Washington, DC
• www.hqda.army.mil/tema

2
What is a TEMP ?

• Executive level strategy and Basic planning
  document for all life cycle TE for a particular
  system acquisition
• Overall TE structure, major elements, and
  objectives
• Consistent with Acquisition Strategy
• Sufficient detail to permit planning for timely
  availability of
• test resources required to support the TE
  program
• Road Map for integrated
• simulation, test, and evaluation plans
• resource requirements
• schedules

3
Policy/Guidance

• DOD 5000.2-R Mandatory Procedures for MDAP/MAIS
  Programs
• TEMP Mandatory Procedures and Format
• Integrated Product and Process Development (IPPD)
  philosophy
• AR 73-1 Test and Evaluation Policy
• Chapter 7 - TE Review and Reporting Requirements
• Para. 7-4 - Test and Evaluation Master Plan
• Supplemented by 2 March 2000 and 16 March 1996
  Policy Guidance Memorandums TE WIPT and DA
  Level TEMP Process Guide
• Sep 2000 Version of AR 73-1 (Subject to USAPA
  Publication)
• Chapter 8 TE WIPT
• Para. 10-2 - Test and Evaluation Master Plan
• DA PAM 73-2 TEMP Procedures and Guidelines

4
Applicable Programs
Every Army acquisition program will have a
TEMP. Only exceptions are - investigational
drugs and biologicals - devices in humans
Includes Materiel and Information
Technology Programs
5
Preparation

• Responsibility Program/Project Manager (PM)
• Execution TE WIPT

6
Specific Responsibilities

• Program/Project Manager (PM)
• Overall responsibility
• Part I - System Introduction/Critical Technical
  Parameters
• Part II - Integrated Test Program Summary
• Part III - Developmental TE Outline
• Contractor testing
• Tech Base Activities (e.g., ATD/ACTD)
• Part V - TE Resource Summary

7
Specific Responsibilities

• Combat Developer/Training Developer/Functional
  Proponent
• Part I
• Mission Description
• Includes reference to System ORD
• Measures of Effectiveness and Suitability
• Part IV
• Critical Operational Issues and Criteria (COIC)
  
• Tactical Context Input
• Warfighting Experimentation Input
• DTLOS Test Needs Input
• TRADOC Force Development Test or Experimentation
  (FDT/E)
• Assists in Identifying Follow-on Operational
  TE (FOTE) Requirements
• Part V
• Training Requirements
• Attachment 1
• MNS and ORD Relationship Linkage
• AoA Linkage

8
Specific Responsibilities

• Army Test Evaluation Command (ATEC) System Team
  (AST)
• Part II - Integrated Test Program Summary
  (contribution)
• Part III - DTE Outline
• Part IV - OTE Outline
• Part V - TE Resource Summary

Recommend that these now be combined as feasible
9
Format

• Specified in DOD 5000.2-R - Appendix B
• Detailed in DA PAM 73-2
• Level of detail directly related to TE strategy
  and complexity of TE effort - not size of the
  program
• Guideline length is 30 pages (excluding annexes
  and attachments)

10
Content

• Signature page
• Coordination sheet
• Part I - System Introduction
• Part II - Integrated Test Program Summary
• Part III - Developmental Test Outline
• Part IV - Operational Test Outline
• Part V - TE Resource Summary
• Annex A - Bibliography
• Annex B - Acronyms
• Annex C - Points of Contact
• Attachment 1 - Requirements/Test Crosswalk Matrix

Or combined Integrated TE Outline
11
Content Detail

• Part I - System Introduction
• a. Mission Description
• b. System Description
• c. System Threat Assessment
• d. Measures of Effectiveness and Suitability
• e. Critical Technical Parameters

2 pages suggested -- refer to annexes
12
Content Detail

• Part II - Integrated Test Program Summary
• a. Integrated Test Program Schedule
• b. Management

2 pages suggested
13
Content Detail(Can be combined with Part IV)

• Part III - Developmental Test Outline
• a. Developmental Test Overview
• b. Developmental Test to Date (if necessary)
• c. Future Developmental Test

10 pages suggested
14
Content Detail(Can be combined with Part III)

• Part IV - Operational Test and Evaluation Outline
  
• a. Operational Test and Evaluation Overview
• b. Critical Operational Issues
• c. Operational Test to Date (if necessary)
• d. Future Operational Test
• e. Live Fire Test and Evaluation

10 pages suggested
15
Content Detail

• Part V - Test and Evaluation Resource Summary
• a. Test Articles
• b. Test Sites and Instrumentation
• c. Test Support Equipment
• d. Threat Representation
• e. Test Targets and Expendables
• f. Operational Force Test Support
• g. Simulations, Models and Testbeds
• h. Special Requirements
• i. TE Funding Requirements
• j. Manpower/Personnel Training

6 pages suggested
16
Content Detail

• Annex A Bibliography
• Annex B Acronyms
• Annex C Points of Contact
• Attachments
• Attachment 1 Test-Requirements Crosswalk Matrix
• Other Attachments (as appropriate)

17
TEMP vs SEP
TEMP
SEP

• TE WIPT produces
• WHAT TE Contract
• Issue level of detail
• Can include all events

• ATEC System Team (AST) produces
• HOW TE will be done
• Measure level of detail
• Only events necessary by ATEC
• Addresses Event Design Plan(s)

18
Narrative TEMP Approval Process(TEMPs requiring
DA and/or OSD approval)

• TE WIPT develops TEMP all members empowered
  includes DA Staff
• Final TE WIPT meeting - all members sign TEMP
  coordination sheet
• PM signs TEMP approval Signature Sheet and
  obtains PEO, TRADOC ATEC Concurrence
  Simultaneously (Goal Complete at TE WIPT via
  empowered members)
• PM delivers TEMP with all Concurrence signatures
  to TEMA within 20 days after PM signature
• TEMA staffs TEMP delivers to DUSA(OR) for Army
  approval
• DUSA(OR) provides Approval signature within 10
  days
• TEMA sends TEMP to OSD - Army goal is within 30
  calendar days from PM signature
• OSD staffing and approval objective - 30 calendar
  days from receipt to return to TEMA
• Note
• A new or updated TEMP must be completed in
  conjunction with approaching a milestone, a
  program rebaseline (breach), when the
  associated ORD or Command, Control,
  Communications, Computers and Intelligence
  Support Plan (C4ISP) changes, or after a
  significant change to the program

19
ARMY TE WIPT TEMP APPROVAL PROCESS
TE Manager
TEMP
PM
PM
0 -20 Days
OSD Action Officers
0 Days
PEO
Combat Developer
TE WIPT
Training Developer
Coordinated TEMP
0 days
HQDA STAFF DCSPER DSCLOG DSCINT ASA(ALT) DSCOPS D
ISC4 ODUSA(OR)
Other Agencies
ATEC AST
OTC
Simultaneous Staffing
AEC
DTC
ATEC
TEMP
DUSA(OR) Signs
TEMA
OSD
0-20 days
USER Rep (TRADOC/Others)
5-20 days
lt30Days
TEMA finishes HQDA staffing (if incomplete) for
DUSA(OR)
PM
TEMA

20
TEMP Staffing and Approval Process
Army Programs



PEO or Developing
Program
Yes
Agency Signs
Manager
or


TRADOC Signs
Signs

ATEC Signs
DUSA(OR)

Goal Simultaneous Staffing at TE WIPT
Signs
No
Non-OSD TE Oversight
Program Manager and TE WIPT
OSD TE Oversight
MDA
TEMA Distributes
or designated TEMP
approval authority
for ACAT III and IV
OSD Signs
TIMELINES
- DTE OUSA(ATL)
PM Signature to DUSA(OR) Signature - 30 Calendar
Days or less
- DOTE
OSD Objective - 30 Calendar Days or less
21
Staffing and Approval Process Army Lead
Multi-Service Programs (ACAT I/II and OSD
Oversight)
Signatures PEO or Developing Agency TRADOC
ATEC Participating Service OTA PEO or
Dev Agency User rep
TE WIPT Army Staff Coordinated TEMP
Program Manager Signs
TEMA Staffs
DUSA(OR) or MDA Signs
ACAT I or OSD TE Oversight
Other Service Approval
ACAT II (no oversight)
Goal Simultaneous Staffing at TEWIPT
Yes
TEMA Forwards
TEMA Forwards
OSD Signs - DTE OUSA(ATL) - DOTE
TEMA Distributes
Program Manager
TIMELINES PM Signature to DUSA(OR) Signature - 30
Calendar Days or less OSD Objective - 30 Calendar
Days or less Select ACAT II Programs Have
Approval Authority Delegated from DUSA(OR) to an
MDA
TRADOC ATEC TE WIPT Other Services
22
TEMP Staffing and Approval Process TEMP Staffing
and Approval Process BMDO Programs
23
Staffing and Approval Process Army Participating
Multi-Service Programs (ACAT I/IIOSD Oversight)

• Army PM, PEO or Developing Agency Signs
• TRADOC Signs
• ATEC Signs

TE WIPT Equivalent Coordinated TEMP
Lead Service Acquisition Executive
TEMA Staffs
DUSA(OR) Signs

• Simultaneous Staffing

ACAT I or OSD TE Oversight
HQDA Offices
No
ACAT II Program
TEMA Distributes
Army PM, PEO or Developing Agency
TRADOC ATEC TE WIPT Other Services
OSD Signs - DTE OUSA(ATL) - DOTE
Lead Service Acquisition Executive
TEMA Distributes
TIMELINES TEMA Receipt to DUSA(OR) Signature - 20
Calendar Days or less OSD Objective - 30 Calendar
Days or less
24
Ten TEMP DOs and Donts

• 1. Dont dismiss TE WIPT meetings as not
  important
• 2. Dont Concur with Comment on TE WIPT
  Coordination Sheet.
• 3. Dont have TEMP simply refer to SEP.
• 4. Do invite HQDA offices to participate in
  the TE WIPT.
• 5. Do know TE WIPT issues be empowered.
• 6. Do have Part V Resources based upon Parts
  III and IV.
• 7. Do have Requirements/Test Crosswalk Matrix
  as Attachment 1.
• 8. Do have cognizant OIPT Chair, depending on
  type of program, and DOTE Co-approve.
• 9. Do hold program to the rigorous TE
  process.
• 10. Do treat the TEMP as a Contract among all
  Stakeholders.

25
Seven Deadly Sins Affecting TEMP Approval

• 1. Size. Significantly exceeds 30 pages,
  excluding annexes.
• 2. Annexes. Not kept to a minimum.
• 3. Not complete. Classified data not
  integrated submitted separately too many TBDs
  in Part V.
• 4. Approval Page Coordination Sheet.
• Not present, not properly signed, out of
  date, or otherwise altered.
• 5. COIC. Not approved and incorporated.
• 6. Reference Documents. MNS, ORD, STAR
  unavailable.
• 7. AoA - COIC Linkage. When applicable, TE
  measures not discussed.

26
OSD TE Oversight Programs TEMP (DTE) Signature
Process
As of 1 Jan 00
SECDEF
C3I
ATL
D, DOTE (TEMP APPROVAL)
ADVICE
OIPT CHAIR (STS) DR. SCHNEITER (DT TEMP Approval)
OIPT CHAIR (C3I) MR. LANDON (DT TEMP Approval)
OIPT CHAIR (MAIS) MR. CURTIS (DT TEMP Approval)
DOTE AO
DTE MR. LOCKHART
DTE AO (DoD TEMP Coordination POC)
NOTE OIPT Chairmen sign for DT - C3I
approves C3I and Space Programs - MAIS
chair approves MAIS Programs - STS
approves all other programs.
27
OSD TEMP Process
28
OSD TEMP Process (1 of 4)

• Develop the TEMP through the TE IPT process
• Add the DTE/DOTE action officers to your TE
  IPT
• Maintain informal dialog with the DTE and DOTE
  action officers -- and our contractor support
• Use Appendix B, DODR 5000.2-R, to prepare your
  TEMP!
• Straightforward cookbook OSDs checklist for
  TEMPs
• Provide sufficient details about your future DTE
  activities (Part III).
• Be specific, but eliminate the fluff.
• A TEMP is a master strategy document, not a
  detailed test plan. This is the section that
  answers the how, not the why.
• The ATPS interactive software is a good tool to
  use in building TEMPS, and can be downloaded from
  the Deskbook.

29
OSD TEMP Process (2 of 4)

• Work with the Defense Intelligence Agency action
  officers
• Plan to test the system against the threats in
  the System Threat Assessment
• To reduce the size of TEMPs, use pointers to
  other documents where more details are present
• i.e., software development plans, modeling and
  simulation plans, and logistic support plans.
• Include Modeling Simulation in your TE
  planning.
• Reduces cycle time costs

30
OSD TEMP Process (3 of 4)

• Work to combine DT with OT to achieve mutual
  objectives at reduced costs
• Successfully completing independent OTE (Title
  10 statute requirement) is the final exam in TE
  planning
• Be careful to not give up PM control of your
  development test and evaluation plan to the
  operational test and evaluation side of the
  house.
• There will still be a requirement for a dedicated
  OTE event that the OTE folks will control.
• The DTE is part of your risk reduction measures
  that are needed to ensure the system is ready for
  operational testing.
• If the size and scope of the dedicated OTE can
  be reduced without jeopardizing your DTE
  objectives, then combined DT and OT makes sense

31
OSD TEMP Process (4 of 4)

• Submit electronic version of draft to OSD for
  staff review
• Identifies showstoppers before TEMP enters the
  Service signature chain
• Let DTE/DOTE know early if you need an
  extension to your TEMP due date
• Final staffing at OSD
• 30 days window for USD(AT) and DOTE signature
• STS/DTE TEMP signature process
• OIPT Chairman (i.e., STS, C3I, IT) will now sign
  TEMP for USD(AT)
• Service TEMP approval timeline is too long
• Especially if all the issues have been worked in
  the IPT process
• Challenges to reduce the cycle time to be posed