Twelfth Period Award Fee Evaluation International Space Station Contract NAS 1510000

1
Twelfth Period Award Fee EvaluationInternational
Space StationContract NAS 15-10000

• Executive Summary
• Boeing continues to provide excellent
  technical performance in the deployment of
  hardware and software and in support of the
  on-orbit vehicle. This is apparent in the
  continued outstanding success of the on-orbit
  vehicle.
• Progress was made this period in dealing with
  significant management issues highlighted in
  the mid-term evaluation.
• NASA remains concerned with the contractors
  ability to effectively plan and manage to
  metrics, quality escapes, accurate reporting and
  forecasting of financial data, and the resulting
  impact and ultimate cost.

2

• Twelfth Period Final Evaluation
• International Space Station
• Contract NAS 15-10000
• April 2002

3
Twelfth Period Award Fee Evaluation International
Space Station Contract NAS 15-10000

• A. Program Management
• Summary
• The contractor has implemented a process for
  managing and tracking the IO activity and
  continues to implement improvements to the
  process. The contractor has been aggressively
  managing work package adjustments as well as
  expedited approval of certain urgent change
  proposals. Through this period the truss
  activity and, in general, proactive truss
  schedule management has been a significant
  weakness. The contractor has recognized this and
  has updated the S1/P1 schedules and reorganized
  the element teams. Additionally, cost reporting
  has been a serious weakness this period.
• Strengths
• The contractor developed and implemented an IO
  management process (WPRR) and continues to
  develop improved processes for increased insight
  into the day to day implementation of IO work
  content and for increased efficiencies in
  management of IO content.
• An outstanding scheduling process was put in
  place at KSC that accurately portrays work
  content for a given day, tracks progress daily
  and incorporates problems encountered. It is
  used for analyzing the critical path and deriving
  metrics for management review. Implementation of
  the scheduling process has resulted in
  significant progress toward effective and
  efficient planning of work activities and
  resource utilization on S1/P1.
• Realignment and augmentation of the management
  teams at KSC for both the inboard and outboard
  truss teams has resulted in improved focus and
  insight into resolution of processing challenges
  and problems. Additionally, the staffing of
  distinct truss teams, down to the QA / technician
  level, has resulted in improved continuity and
  overall ownership by the teams.
• The Avionics and Software team has been very
  proactive in preparing for consolidation of
  tasks, such as in aggressive hiring of
  critical skills and transfer of contracts. This
  is expected to result in a smooth transition of
  tasks from HB and CP to Houston.

A - 1
4
Twelfth Period Award Fee Evaluation International
Space Station Contract NAS 15-10000

• A. Program Management
• Strengths (cont.)
• Boeings continued proactive participation in the
  Integrated Program Schedule Panel (IPSP) has been
  exceptional as a result of experienced personnel
  assignments in the software scheduling area. The
  Software Plan to launch, which is now being
  statused weekly at the IPSP, is an example of the
  result of these assignments.
• Boeing developed a new product to facilitate
  accurate and timely annual property NF-1018
  reporting by incorporating their own internal
  form called a BF-1018. The BF-1018 has more
  specific property reporting instructions, which
  resulted in a more accurate and timely NF-1018
  submittal. Boeing's submittal of the NF-1018 for
  '01 has been the most accurate property report
  under the contract to date, as evidenced by
  feedback from JSC's financial management office.
  In addition, the first on-orbit property report
  was outstanding.
• Boeing contracts did an outstanding job of
  turning around urgent proposals when Authority to
  Proceed was needed quickly. Ninety days are
  normally allowed to build a proposal, however,
  SSCNs 4407, 5936, 5949 and 4823 were developed in
  significantly less time. In addition, the
  contractor was instrumental in finalizing
  paperwork and negotiations to meet the short time
  constraints.
• Weakness
• The contractor has not been able to adequately
  portray the financial posture of the baseline
  contract. Presentations at the Space Station
  Program Control Board (SSPCB) , Cost, Schedule,
  Technical Reviews (CSTRs), and Quarterly Program
  Manager Reviews do not clearly define key
  management metrics, such as fiscal year reserves
  and performance issues, and provide no summary
  perspective or conclusions regarding the fiscal
  posture. In numerous instances, the contractor
  could not explain the data being presented and
  actual data and information were not in sync with
  forecasts. Additionally, the quality and
  consistency in detailed presentations vary widely
  across Super Cost Account Managers

A-2
5
Twelfth Period Award Fee EvaluationInternational
Space StationContract NAS 15-10000

• Program Management (cont.)
• Weaknesses (cont.)
• Boeing technical offices are not adequately
  supporting the change proposal process. Examples
  include
• Recent LKP data delivery dates (I.e. 5046 5811)
  continue to slip.
• In some cases, NASA has begun negotiations based
  on proposal data, only to be informed that actual
  work costs are already in excess of proposal data
  before negotiations even begin.
• Planning and scheduling of required work to
  support definitization by the required
  authorization to proceed (ATP) date particularly
  in the area of external carriers.
• In a formal audit, DCAA found Boeings estimating
  system to be inadequate in part in the area of
  subcontract and material costs. Although the
  contractors response previously indicated that
  all corrective actions had been completed, NASA,
  DCAA, and the DCMA were not satisfied that
  adequate training had been provided. Since that
  time, Boeing has established an adequate
  corrective action plan. DCAA intends to monitor
  the effectiveness of the training by auditing
  proposals over the next few months.
• In a formal audit, DCAA found Boeings Billing
  System to be inadequate in part. Furthermore,
  NASA requested a supplemental audit because of an
  over billing of 46M in November 2001. NASA
  requested that DCAA perform quarterly voucher
  reviews as a result. Boeing is cooperating with
  NASA to resolve this issue and has an adequate
  corrective action plan in place.
• The contractor did not manage the S0 truss
  activity effectively. Significant management and
  performance issues on S0 impacted S0 and
  downstream truss schedules and resolution was not
  proactive or timely and resulted in additional
  Program cost.

A-3
6
Twelfth Period Final Evaluation International
Space Station Contract NAS 15-10000

• B. 1 Vehicle Hardware Production
• Summary
• The contractor has exhibited improved
  performance in development and final assembly of
  the truss elements through incorporation of
  several process improvements. Additionally,
  preparation and support of FCAs and ARBs
  continues to be a strength through this period.
  However, slow closure of engineering paper
  against the elements continues to be a
  significant concern relative to the ability to
  maintain a full understanding of the effort to
  complete each element.
• Strengths
• Boeing support of FCA and ARB milestone
  activities continues to be a strength. During
  this period the P3/S3 and P5/S5 FCAs were
  completed. Also, P4/S4, P5, and P1/S1 ARBs were
  completed and the elements DD250d.
• The contractor has done an outstanding job of
  applying lessons learned from earlier missions to
  assembly tasks on 9A, 11A, 12A and 13A resulting
  in improved schedule performance and fewer PRs
• Added assembly drawing notes to streamline the
  resolution of dents/dings/scratches
• Revised the shroud fit-check process to include
  onsite thermal support and buy-in
• P4 MT rail fit-up and match drilling
• Solar array wing buildup
• P3/S3 MLI shroud fit checks
• The Contractor has implemented process
  improvements to streamline flight preparation of
  truss elements
• Significantly streamlined ground processing
  activities through the implementation of the
  Discrepancy Report system to expedite minor
  repairs and return-to-print anomalies.
• Implemented a new process by which shroud
  modifications can be performed without class 1
  control or Configuration Management tracking.
• Developed a method to install the S1 and P1 Pump
  Module Assemblies (PMA) and Crew Equipment
  Translation Aid (CETA) carts in the Space Station
  Processing Facility (SSPF), thereby eliminating
  the need to return to the Operations and Checkout
  (OC) building.

B.1-1
7
Twelfth Period Final Evaluation International
Space Station Contract NAS 15-10000

• B. 1 Vehicle Hardware Production
• Strengths (continued)
• Boeing EVR team did a notable job in the DDTE of
  the Camera Berthing and Centerline System. The
  technical products met the aggressive schedule to
  ensure that only minor schedule impacts were seen
  in the MBS ground processing at KSC.
• The Structure and Mechanisms team has done an
  outstanding job of completing the work on MCASS,
  PAS, CAS and the test tool PCAS. The completion
  of the work on MCASS was schedule critical and
  the team met critical milestones.
• The contractor has done an outstanding job of
  accommodating new or unplanned work on S0 and the
  Mobile Transporter and still maintain Program
  milestones. These new or unplanned items were
  due to several factors that included requirement
  changes, problem reports taken during processing,
  and hardware design or manufacturing issues.
  Some examples include reworking of the Mobile
  Transporter captive nut retainers with no
  significant impact to the Ready for Shuttle
  Integration date and reworking of the fluid
  umbilical multilayer insulation blankets without
  any impact to the Installation into the Canister
  date.
• The contractor has done a notable job of
  performing planned work earlier than scheduled to
  ensure the work does not affect the critical
  path. Some examples of this include the SVS
  target installations and surveys, handrail label
  installations, flight cover removals in the OC
  building versus at the Pad, and completion of 13
  S1 QD leak checks to support S0 problem
  resolution.
• The contractor did an outstanding job
  coordinating and executing the Passive Common
  Attach Mechanism interface test with the Mobile
  Base System (MBS) Common Attach System (MCAS).
  The hardware arrived early at KSC and the test
  was completed on time. No anomalies were noted
  during the test. No retesting had to be
  performed. The management of the test was
  outstanding.

B.1-2
8
Twelfth Period Final Evaluation International
Space Station Contract NAS 15-10000

• B. 1 Vehicle Hardware Production
• Strengths (continued)
• The Boeing Outboard Truss test team continues to
  perform exceptionally well. Their methodology
  for team make-up, procedural/test script
  development, dry-running procedures with
  high-fidelity test equipment, effective use of
  lessons learned, and test execution is
  outstanding. During this period the Outboard
  Truss test team successfully completed the P4 and
  S4 command and response tests and the functional
  checkout of the two 12A flight wings ahead of
  schedule and without ANY Problem Reports (PR)
  taken or procedural deviations required.
• Weaknesses
• The contractor continues to defer paper closure
  until late in the processing flow, causing a
  large commitment of resources at that time and
  increasing the risk that open issues will not be
  identified early enough to minimize impacts to
  element flow.
• Inadequate performance associated with open paper
  closure forced rescheduling of the Flight 8A LPA
  from 12/18/01 to 01/29/02.
• Truss elements S1 and P1 (Flights 9A, 11A) have a
  combined 1369 post-ARB open items documented in
  OPMT as of 3/18/02, including approximately 606
  KSC items.
• Significant amount of S0 open paper encountered
  at both LPA and SORR, requiring significant
  efforts at end of process which increases both
  risk and cost to the Program.
• KSC build books and open paper closures were not
  maintained as current, resulting in closure
  inefficiencies which cost the Program.

B.1-3
9
Twelfth Period Final Evaluation International
Space Station Contract NAS 15-10000

• B. 1 Vehicle Hardware Production
• Weaknesses (cont.)
• The contractor needs to focus more attention on
  ensuring quality procedures, training, attention
  to detail and workmanship in the development of
  flight hardware. Examples of problems associated
  with these items include
• S0 problems (1) Improper 1-g Umbilical
  installation, (2) Inadvertent loading of MT by
  Zero-g Support Equipment (ZSE) during
  MT-to-S0 mate, (3) Use of silver-plated
  fasteners, (4) Improper installation of umbilical
  thermal blankets, (5) Click Bond installation
  failures.
• During the removal of quick disconnect (QD) 164
  from the P1 truss, the contractor failed to
  follow approved procedure and established flight
  cleanliness requirements by using a red hacksaw
  to remove the QD. The recovery effort associated
  with this failure has been very costly.
• The contractor failed to implement established
  program cleanliness practices and quality control
  in manufacture of the Pump Module Assembly (PMA)
  test stands for P1 and S1 in MEIT. A foam plug
  intended to prevent contamination during welding
  procedures was left in a line resulting in foam
  particles throughout much of the plumbing. In
  addition, the heat exchanger used during the test
  was not properly passivated, resulting in
  corrosion and subsequent contamination of the
  loop.
• Boeing failed to direct their contractor
  (Parker-Symetrics) to update the Internal Self
  Sealing Quick Disconnect Acceptance Test
  Procedure (ATP) which was mandated by SSCM 4879.
  Boeings failure to direct Parker-Symetrics
  resulted in several hundred QDs delivered to
  NASA with inadequate leak testing.
• The contractor is still lacking in the planning
  and coordination of Cargo Element envelope
  checks. 8A DPA measurements remaining to be
  taken, as well as the results of those which have
  been, are not being reported to Boeing KSC.

B.1-4
10
Twelfth Period Final Evaluation International
Space Station Contract NAS 15-10000

• B. 1 Vehicle Hardware Production
• Weaknesses (cont.)
• Although improving, Boeing continues to be slow
  to identify and/or resolve significant hardware
  issues. Examples include
• An on-orbit failure of the Launch to Activation
  (LTA) heater thermostats during the 5A mission in
  early 2001 remained an open issue for S0 late
  into the 8A Processing Flow.
• The contractor missed the latching motor valve
  from the QPIP 1459 corrective action
  investigation and forced last minute engineering
  analysis to fly as is with accepted risk.
• Late identification of Mated QD Hydraulic Lockup
  issue resulted in impacts to 8A LPA, turnover to
  Shuttle, and SORR.
• In the area of Inboard Truss testing, The
  contractor needs to focus more attention on
  ensuring quality procedures, training, and
  attention to detail in preparing for and
  implementing testing. Examples include
• Revision C of the P1 TRD had been released and no
  updates or hand writes to the P1 P1000 or P1100
  procedures written to Revision B requirements had
  been incorporated into the procedure.
• AT P1000 and P1100 Test Readiness Reviews,
  schedules were not provided and test
  configuration specific exceptions to the
  referenced drawing were not documented.
• Boeing missed testing the 8A MBSU S0-1A/1B RBI 11
  output copper path which provides primary power
  to the Node 2 DDCU (N2NAD-1B) even though it was
  in the procedure. Boeing has reviewed part of
  the root cause and has taken action to resolve.

B.1-5
11
Twelfth Period Final Evaluation International
Space Station Contract NAS 15-10000

• B.2 Technical Performance - Avionics and
  Software Production
• Summary
• The contractors performance in the avionics
  and software areas has been solid this period.
  Major strengths included SSMMU delivery, 8A
  transition, and improved PR cycle times.
  Although improving, ECZ and CCS software schedule
  performance remains a major problem.
• Strengths
• The Boeing support to the CDH Hardware Team has
  continued to be outstanding. All SSMMU sets
  were delivered eight months ahead of the
  pre-6A implementation schedule and all on-board
  Command and Control MDMs and Payload MDMs were
  upgraded with the SSMMU. This completes the
  on-board mass memory upgrade effort.
• Boeings support for the successful 8A software
  on-orbit transition was outstanding, adding
  functionality to CC, GNC, PEP, and ECZ. A very
  effective Software Transition Readiness Review
  preceded this milestone.
• The prime instituted several valuable process
  improvements
• The average flow time for PRs in analysis was
  reduced.
• Instituted an Avionics Equipment Board to more
  efficiently manage avionics hardware.
• Improved SMC processes for Caution Warning
  (including CW impacts of PRs), FCA/PCA, software
  ICDs, Stage Test DVOs, and SPNs. Also notable
  support for assessment of PCS development
  programs.
• EME expertise continues to be of top-notch
  quality and has provided notable support of
  testing, design reviews, IP integration, and
  anomaly resolution teams.

B.2-1
12
Twelfth Period Final Evaluation International
Space Station Contract NAS 15-10000

• B.2 Technical Performance - Avionics and
  Software Production (cont.)
• Strengths (Continued)
• The Boeing GNC team did an outstanding job of
  supporting the 8A GNC activation and checkout
  TIM. This led to a well-planned and systematic
  activation and checkout of the GNC system for 8A.
  
• The contractor has performed beyond expectations
  in supporting testing, in such areas as 8A stage
  testing, NASA/NASDA JEM CDH Joint Test 2,
  planning for APM Joint Test 3A, supporting the
  MEIT-3 CDH reassessment study, and test support
  at KSC (e.g. S1 S-Band re-acceptance, video and
  CDH support for S0 Combined Functional Test,
  GNC support to Regression test, and overall EME
  support).
• Software teams have shown notable flexibility to
  accommodate late requests for new PPLs and
  patches, e.g. the Ku-band patch which was
  accelerated from CCS R4 to R3.
• The contractor has aggressively instituted fault
  trees to resolve Avionics PRACAs.
• The contractor provided notable facility support
  in developing requirements for ISIL enhancements,
  multiple test rig releases and upgrades, and
  planning for software management on the new Sun
  6800 in the MBF.
• Superior initiative was shown by the DIT, e.g.
  trying to reduce the number of Standard Out
  products each cycle, helping the IPs integrate
  CGS tools, and ensuring that the customer had the
  best RCL possible for upcoming Timeliner testing.
  
• The contractor was highly effective in isolating
  cable harness wiring problems of the Integrated
  Truss Segment S0.

B.2-2
13
Twelfth Period Final Evaluation International
Space Station Contract NAS 15-10000

• B.2 Technical Performance - Avionics and
  Software Production (cont.)
• Strengths (Continued)
• The contractor has successfully supported
  unplanned tasks without interrupting avionics and
  software work, e.g. supporting the CARD and Node
  3 relocation study.
• Boeing has demonstrated exemplary performance in
  the delivery, test and integration of software
  products for quick turnaround (i.e. 24 hours) to
  support MOD/MCC-H real-time operations and
  testing.
• Weaknesses
• The contractor continues to have problems meeting
  ECZ and CCS software schedules.
• Although the development cycle time for large
  IFLs was reduced, the contractor continues to
  commit to unrealistic schedules for Build
  Specification Reports (BSRs) and IFLs resulting
  in delivery dates not being met. Also, customers
  continued to find BSR errors.

B.2-3
14
Twelfth Period Midterm Evaluation Exhibit
A International Space Station Contract NAS
15-10000

• B.3 Technical Performance Cargo Integration
  Hardware
• Summary
• Overall, the contractor has shown improvement
  for this period. The contractor has made some
  progress in planning, coordinating, and
  integrating across the various projects.
  However, the contractors management needs to be
  more proactive and involved on a daily basis.
  This daily involvement is needed to help
  facilitate timely communication, integration
  across multiple projects, identification of
  issues, and delivery of hardware per the agreed
  to schedule.
• Strengths
• Strong technical support was provided for
  negotiating FRAM connector design with
  ESA/Columbus. Quick turnaround on issue
  resolution and actions. Contractor demonstrated
  flexibility and desire to work with IP
  limitations.
• The contractor has made significant improvement
  in schedule management and tracking for External
  Carriers projects. Using these improved tools,
  performance to plan has improved.
• The contractors performance on recent ESP2 and
  FSE design reviews has been outstanding. The most
  significant example is the recent completion of
  ESP2 IDR2. The presentations and data package
  were well done and led to meaningful discussions
  with the participants. All major issues were
  identified and documented for follow-up.
• The contractor has made progress in day to day
  project management. Specifically, the contractor
  has a much better set of project schedules in
  place.
• The contractor has improved in coordinating their
  ground activities with the KSC Payload Ground
  Operations Contractor (PGOC) for External
  Carriers FSE support. An example of this is the
  work the contractor did on the UF1 LMC FRAM
  integration.

B.3-1
15
Twelfth Period Midterm Evaluation International
Space Station Contract NAS 15-10000

• B. 3 Technical Performance Cargo Integration
  Hardware
• Strengths (cont.)
• The contractor has made substantial progress in
  their strategic planning. Specific examples
  include the updated version of the External
  Carriers Office Hardware Development Strategic
  and Tactical Plans and development of a plan to
  react to failures during assembly and a steady
  state plan post-assembly.
•  Weaknesses
• The contractor continues to have significant
  lapses indicating a reactive rather than
  proactive management style. A specific example
  is the recent attempt to deliver the PCU FSE, in
  spite of the fact that required structural
  analysis for carrier types identified in the FSE
  specification were not complete. When questioned,
  the contractor stated the subject requirements
  were incorrect. While this assessment may be
  true, identification of this issue AT DELIVERY is
  unsatisfactory. This led to an avoidable delay in
  acceptance (FCA/PCA) of this hardware.
• The contractor continues to lack technical
  communication and integration across different,
  related External Carriers projects. An example
  of this would be the FRAM and FRAM connector
  projects. The contractor has been slow to
  integrate these two projects, where connector
  delivery and integration into the FRAM assemblies
  is essential to assure FSE deliveries to ULF1 and
  subsequent missions. Another example of this
  would be the FRAM Connector (K) CDR. In this
  instance, the connector design review meeting
  started, without data packages being distributed
  in advance to FRAM personnel ( others).
• The contractor needs to improve on assuring
  rigorous manufacturing standards for External
  Carriers hardware. Examples of this include the
  rework of the passive FRAMs and the UTAS, where
  easily avoided manufacturing process faults
  resulted in needless cost and schedule impacts.
•  

B.3-2
16
Twelfth Period Midterm EvaluationInternational
Space StationContract NAS 15-10000
B. 3 Technical Performance Cargo Integration
Hardware

• Weakness (continued)
• The contractor needs to improve on the timeliness
  associated with identifying long lead items, and
  then following through on the procurement.
  Examples of this include the procurements
  associated with the FHRC and UTA FSE.
• The contractors management needs to be more
  pro-active and involved on a daily basis. It is
  unclear if the management is involved in the
  daily implementation, across the various
  projects. This daily involvement is needed to
  help facilitate communication, integration across
  multiple projects, and delivery of hardware per
  the agreed to schedule. In addition, this daily
  involvement will allow the contractor to better
  understand the customers concerns/issues.

B.3-3
17
Twelfth Period Final Evaluation International
Space Station Contract NAS 15-10000

• B.4 Technical Performance Launch Package
  Integration
• Summary
• The contractor has improved performance
  during this period and demonstrated superior
  initiative and provided quality support to the
  Launch Package and Manifesting teams.
• Strengths
• The Boeing Mission Managers and their supporting
  organizations have improved their inputs of the
  hardware handling and processing requirements
  to the Launch Package Teams, Stowage and Packing
  organizations.
• Boeing demonstrated superior initiative in
  developing an ISS Manifest Analysis Tool (IMAT).
  The IMAT has resulted in significant process
  improvements and has greatly improved ISS ability
  to respond quickly and accurately to changing
  requirements.
• The contractor has provided notable support in
  assuring that Boeing hardware and products are
  delivered per schedule to support UF-2.
• Boeing has instituted a weekly telecon to status
  the Boeing hardware manifested for a specific
  flight. These meetings are used to help identify
  and resolve any hardware processing/delivery
  issues and have helped to reduce the number of
  surprises regarding late hardware deliveries.
  They have become instrumental in helping to
  involve the appropriate parties, including
  management, necessary to make critical decisions
  and establish recovery plans for late deliveries.
  
• Weaknesses
• Although improving, identification of shortages
  and/or late deliveries continue to be identified
  too late in the process (ie shortly before the
  required hardware on-dock date).

B.4-1
18
Twelfth Period Midterm Evaluation International
Space Station Contract NAS 15-10000

• B.5 Technical Performance Preparation for
  Flight
• Summary
• The Boeing VIPeR team has performed exceptionally
  well this period, providing analysis supporting
  ISS Assembly Flights and ISS on-orbit
  operations. Boeing continues to provide notable
  support in the conduct of CoFR reviews and Stage
  Integration Reviews and in providing the products
  that support these reviews. The Boeing EBIT Team
  has shown weakness in overall management with the
  absence of detailed schedules and late delivery
  of flight critical analysis.
• Strengths
• The VIPER team continues to perform in an
  outstanding manner. Specific examples include
• The integration and development of the ISS
  external stowage configuration report through
  flight 4R.
• Continued improvement of tool development for
  on-orbit data correlation to improve quality of
  data inputs and outputs and improved means of
  delivering quality products to customers. These
  include initial development and peer review of
  the SETI power/thermal mission support tool for
  targeted use during the 8A mission, development
  of an automatic on-orbit data download
  capability, and process improvements in the
  delivery of the Semi-Annual Performance
  Assessment Report (SAPAR).
• Exceptional and timely response in developing a
  SSODB repository for flight attitudes for the ISS
  Program.
• Leading an effort to develop a process for
  including the international partners
  capabilities into the SSODB.
• Supporting NASAs assessments to maintain 3
  versus 6 crew.

B.5-1
19
Twelfth Period Midterm Evaluation International
Space Station Contract NAS 15-10000

• B.5 Technical Performance Preparation for
  Flight
• Strengths
• Boeing has performed exceptionally well in
  maintaining and closing issues associated with
  Stage Integration Review 7 (SIR 7, ULF-1, 12A,
  12A.1) and in preparing for SIR 8 (13A, 13A.1,
  and 15A).
• Boeing has done an outstanding job preparing for
  and conducting the Launch Package Assessments
  (LPA) and the CoFR Stage Operations Readiness
  Review (SORR).
• The contractor Incorporated stage verification
  compliance milestones into the OPMT to insure
  time visibility and action for the verification
  products that support CoFR.
• The Boeing KSC Imagery Group has established new
  procedures for providing imagery status of the
  different flights. The report is updated monthly
  and contain easy to understand graphics depicting
  acquired imagery vs. total number of images to be
  acquired, schedule, and budget data. Boeings
  outstanding management tools applies for tracking
  imagery have resulted in timely delivery of Pre
  Flight Imagery Plans and actual images.
• Boeing performance has greatly improved in the
  on-orbit installation drawing task. They have
  recently implemented the Operations Sequence
  Report, which ensures better coordination with
  MOD and less rework for both teams in support of
  SORR.
• Weaknesses
• Boeing has not provided a detailed schedule from
  which to manage the EBIT process. There is a
  lack of efficiency in the management of the
  process and products which result in the late
  delivery of flight critical analysis.

B.5-2
20
Twelfth Period Final Evaluation International
Space Station Contract NAS 15-10000

• B.6 Technical Performance Assemble, Sustain and
  Operate On-Orbit Vehicle
• Summary
• The contractor continues to provide
  exceptional technical leadership and support to
  the on-orbit vehicle, resolving issues in a
  timely manner and ensuring the proper resources
  and skills are available when required in the
  majority of cases.
• Strengths
• The contractor continues to provide superior
  technical leadership and support to on-orbit ISS
  operations. Some examples of exceptional
  support include
• Extensive fault tree and root cause investigation
  for the BGA anomaly, identifying potential flight
  attitude solutions and identifying a potential
  fix (thermal blanket) and manufacturing/flying it
  within one month.
• ISS environments and contamination team showed
  outstanding progress on development of
  understanding of the ISS plasma charging hazard
  resulting from PV array driven charging for
  vehicle configurations prior to flight 12A.
• Boeing's CDH MER support has been very
  responsive to the flight control team. They have
  supported the CDH flight controllers positively
  with timely results. On multiple occasions, the
  CDH flight controllers have been able to quickly
  correct problems while preserving the appropriate
  data because of the quick turnaround of requests
  to the MER CDH personnel.
• The SM Loads and Dynamics team has provided
  timely mission support with prompt VAC analyses,
  IP support, and very critical analysis/assessment
  of loading events associated with hardware
  malfunction off-nominal performance of ISS
  hardware.

B.6 -1
21
Twelfth Period Final Evaluation International
Space Station Contract NAS 15-10000

• B.6 Technical Performance Assemble, Sustain
  and Operate On-Orbit Vehicle (cont.)
• Strengths
• The majority of the contractors MER systems
  consoles continue to maintain strong
  relationships with the Flight Control Team
  counterparts, resulting in the ability to address
  significant on-orbit issues in a timely manner.
• Boeing management has provided valuable
  suggestions to training content restructuring
  that has helped reduce future cost of training.
  In addition, they are working to ensure a smooth
  transition between JSC and Marshall for payload
  training responsibilities.
• Weaknesses
• None Identified

B.6-2
22
Twelfth Period Midterm EvaluationInternational
Space StationContract NAS 15-10000

• B. 7 Utilization
• Summary
• The contractor continues to provide
  exceptional hardware and software technical
  expertise in supporting the integration of
  payloads with the ISS. During this period of
  performance, Utilization has met all requirements
  in support of integration of payloads into
  Destiny. Management of PaRIS interfaces with the
  Habitat Holding Racks (HHRs) has been inadequate,
  resulting in cost and schedule impacts.
• Strengths
• The contractor has continued to excel in the area
  of payload flight SW deliveries and related
  documentation. All products were delivered on
  schedule and within budget even though numerous
  payload manifest changes continue to occur as
  well as late payload developer submittals.
• Extensive support was provided for the MELFI
  verification review prior to MELFI being shipped
  to KSC from Europe. This support helps abate the
  risk of late verification issues arising at KSC
  which could impact the ULF-1 launch readiness.
• Completed and delivered all Payload Rack Checkout
  Units (PRCU) and Suitcase Test Environment for
  Payloads (STEP) units. Timely delivery and
  acceptance of PRCU unit 6 allowed the MSG project
  to conduct sub-rack verification for the UF-2
  mission. Completed and delivered all Payload Rack
  Checkout Units (PRCU) and Suitcase Test
  Environment for Payloads (STEP) units. Timely
  delivery and acceptance of PRCU unit 6 allowed
  the MSG project to conduct subrack verification
  for the UF-2 mission. The contractors
  proactiveness and ingenuity in the delivery
  process prevented delays in the delivery of these
  units. In addition, the Boeing team has also
  provided a well thought out and thorough forward
  plan for transitioning into sustaining tasks.
• In a mere 3 1/2 working months, the contractor
  designed, developed, fabricated, tested, and
  delivered the Passive Common Attach System
  Simulator (PCASS) (SSCN 5811). Not only was the
  PCASS delivered ahead of schedule, but it far
  exceeded its performance requirements. This
  early delivery allowed the completion of critical
  path testing of the MBS ahead of schedule,
  earning recognition from senior ISS management.

B.7 -1
23
Twelfth Period Midterm EvaluationInternational
Space StationContract NAS 15-10000

• B. 7 Utilization
• Strengths (continued)
• Completed the S3, MCAS, and keel revised envelope
  analyses, completing the process for S3 sites.
  Results are captured in PIRNs scheduled for final
  End to End Berthing Integration Team (EBIT)
  approval on March 6 and the Payload Control
  Boards formal CR to update the document. The
  contractor was able to perform this task which
  required multi ISS and SSP discipline integration
  when it was not carried as a high priority by
  those teams.
• The administrative services being provided for
  the PSCP have been exemplary. Meeting minutes
  have been generated on average in two days time.
  In addition, the minutes have been accurate,
  requiring very minimal if any modifications.
  This quick turnaround of minutes allows PSCP
  decisions and direction to get out to the payload
  SW community quickly allowing them to respond and
  implement in a timely fashion.
• The contractor has processed a significant amount
  of PIRNs which resulted in a better defined
  interface for ISPRs. In addition, a new
  structured process has been set up to handle PIRN
  traffic related to core systems interface
  documentation.
• Weaknesses
• Inadequate coordination with Boeing HHR Project
  on the PaRIS ICD resulted in a HHR schedule delay
  and cost impact.

B.7-2
24
Twelfth Period Final Evaluation
International Space Station
Contract NAS 15-10000

• B. 8 Logistics Maintenance
• Summary
• The contractor has performed exceptionally well
  during this period in all areas of logistics
  support. Of note is their development of
  on-orbit maintenance planning, the support to the
  S5 flight article move, a new management plan on
  new spares procurements, the tracking and
  management of the depot certifications, and the
  support to logistics data systems. In addition,
  the contractor has shown marked improvement in
  the delivery of logistics data products. The
  contractor needs to continue to work on ensuring
  that data required by other organizations to
  support development of logistics products, are
  provided in a timely manner so that product
  deliverables arent impacted.
• Strengths
• The contractor has done an exceptional job
  developing on-orbit maintenance plans. Examples
  include
• Performing a superior integrated analysis
  validating the delivery plans for Nitrogen and
  Ammonia tanks during assembly, and providing new
  insight into post assembly plans.
• Developing a comprehensive plan for addressing
  potential failures of cable boxes on-orbit that
  allowed the Program to avoid potential costs of
  additional spares procurement and FSE development
  beyond the baseline.
• Developing and coordinating an EVA scenario for
  removal and replacement of a Utility Transfer
  Assembly within a single EVA sortie. By providing
  this quality product at the right time, the
  Program will be able to defer FY02 and 03 cost of
  FSE.
• The contractor has provided outstanding support
  in the development and implementation of a
  Common Spares Pool approach with the
  International Partners.

B.8-1
25
Twelfth Period Final EvaluationInternational
Space Station Contract NAS 15-10000

• B. 8 Logistics Maintenance (cont.)
• Strengths (continued)
• The contractor has implemented several corrective
  actions to improve forecasting and management of
  the spares schedules. These include
• Implementing a monthly Super CAM meeting to
  review the cost, schedule and performance for
  spares hardware.
• Streamlining the weekly spares VTC to more
  clearly focus on specific cost and schedule
  milestones.
• Initiating a review team to closely monitor PIO
  proposal implementation process from inception to
  definitization. This has successfully reduced
  the total cycle time to implement new
  procurements.
• The contractor support of logistics data systems
  has been exceptional. During this period the
  contractor completed the KSC PGOC transition from
  KIMS to GOLD on December 01, 2001 as planned. To
  meet this date, the contractor overcame
  significant technical challenges to accomplish
  this task on time including resolving issues with
  the KSC to JSC server interfaces. In addition,
  the contractor identified several problems with
  the Enhanced Automated Graphical Logistics
  Environment (EAGLE) Software and quickly worked
  with the vendor to develop user friendly
  solutions.
• The contractor delivery of logistics data
  products has been greatly improved. Examples of
  this include
• Exceeding the minimum provisioning data
  requirement by five ORUs in support of the
  Provisioning Technical Documentation Meeting.
• Converting the Lockheed-Martin Mil-Std-1388 2A
  data to Mil-Std-1388 2B data and training
  engineering personnel on how to use the EAGLE 2B
  Software System.
• Converting CSA Logistics Support Analysis 2A data
  to 2B data .

B.8-2
26
Twelfth Period Final EvaluationInternational
Space Station Contract NAS 15-10000

• B. 8 Logistics Maintenance (cont.)
• Weaknesses
• The contractor has not provided detailed
  schedules of their development plans for the
  end-to-end external maintenance scenarios. This
  lack of detailed schedules has impacted the
  ability of institutional organizations, such as
  MOD, Crew Office and Engineering, to support this
  effort.
• The electronic delivery of the OP04 (maintenance
  procedure) and OP05 (illustrated parts breakdown)
  needs improvement. The contractor was late in
  providing an essential cross reference table
  which caused the LSAR Review of the data to slip
  one (1) week.
• The contractor did not adequately monitor the
  schedule and cost performance of the Lockheed
  Martin Sunnyvale spares.

B.8-3
27
Twelfth Period Final Evaluation International
Space Station Contract NAS 15-10000

• B. 9 Technical Performance - Configuration
  Management and Data Management/Integration
• Summary
• Boeing Configuration Management and Data
  Management/Integration has demonstrated overall
  highly effective performance during this
  evaluation period. Boeing continues to provide
  outstanding support on the on-orbit mission
  configuration status accounting activities. The
  performance of the SW CM team in configuring the
  IFLs, patches and PPLs has been outstanding.
  Improvement is needed in coordination and
  planning with engineering, quality and CM to
  enable the closure of open paper in time to
  support the COFR reviews.
• Strengths
• Boeing KSC continues to provide outstanding
  support of the on orbit configuration
  reconciliation efforts with on orbit stage
  reconciliation 99.5 complete.
• Boeing has made notable progress in implementing
  change process improvement initiatives
  (streamlined SSCMs, CR reduction) and has
  aggressively worked the NASA initiated Open CR
  Performance Improvement Plan.
• Boeing Software CM has improved the quality of SW
  status accounting by consolidating site data into
  PVCS.
• Boeing Data Management continues to provide
  outstanding support to the Program Data
  Management and Integration Team (PDIT) in the
  development and implementation of a streamlined
  process for receiving and tracking Boeing data
  deliverables.
• The Boeing VDI team did a notable job resolving
  issues and meeting the October and January
  milestone VMDB data deliveries. Outstanding
  support in defining Application Support Data Base
  (ASDB) requirements and helping define VMDB
  Requirements Database (VRD) data for conversion
  was also provided.

B.9-1
28
Twelfth Period Final Evaluation International
Space Station Contract NAS 15-10000

• B. 9 Technical Performance - Configuration
  Management and Data Management/Integration
• Weaknesses
• Boeing was delinquent in providing completed
  build paper and engineering rationale for
  subsequent as-designed vs. as-built discrepancies
  on the S0 truss in time for the 8A SORR. This
  resulted in a CoFR exception which required
  extraordinary effort by engineering and CM to
  complete last minute waivers prior to the FRR.
  Late waivers were also processed on the 7A.1 and
  UF-1 missions.

B.9-2
29
Twelfth Period Final Evaluation International
Space Station Contract NAS 15-10000

• B. 10 Technical Performance International and
  GFE Integration
• Summary
• The contractor has provided outstanding
  support to the MSFC Node Projects Office, ESA SW
  and Systems Engineering Integration. The
  contractor has provided notable support and
  coordination regarding FGB stowage
  enclosures,RSC-E integration activities for UF-2,
  Increment 5, and for the 7P and 4S Stage
  Operations Readiness Reviews. The contractor has
  not performed well in the areas of Columbus
  Avionics and Software schedule maintenance,
  action item and issue resolution.
• Strengths
• Boeing provided outstanding support to the the
  Node 2/3 Projects for the following Node
  activities
• Working with NASA and Alenia to coordinate the
  planning, installation, and testing of CBM
  concurrently with Alenia activities.
• Supporting the Node 3 PIDS baselining effort.
• Providing lessons learned from other elements and
  an independent assessment of the Node 2 human
  factors and logistics verification program during
  the Node 2 Verification TIM.
• The Contractors effort in defining SW
  interfaces, as well as systems engineering
  technical support associated with ESA/Columbus
  integration continues to be a strength. The
  contractor assumed responsibility for the
  development and maintenance of the ESA/Columbus
  SW ICD.
• Boeing provided notable support and coordination
  in regards to the FGB stowage enclosures. Boeing
  has kept the task on schedule and provided status
  on a regular basis. Boeing support and effort
  helped to integrate the MPLM stowage of the
  enclosures for Flight UF-2. Additionally, Boeing
  coordinated the enclosure installation crew
  training in Moscow.

B.10-1
30
Twelfth Period Final Evaluation International
Space Station Contract NAS 15-10000

• B. 10 Technical Performance International and
  GFE Integration
• Strengths (cont.)
• Boeing continues to effectively coordinate with
  RSC-E, the Service Module Debris Panel (SMDP) and
  TDK (Transportation Device for Debris Panels)
  integration activities for Flight UF-2. Boeing
  has provided the technical data (interface and
  thermal loads data) needed by NASA and RSC-E for
  integration analysis and certification of this
  payload. Boeing coordinated design of the
  stowage adapter for the temporary stowage of the
  SMDP on Pressurized Mating Adapter-1.
  Additionally, Boeing provided notable support in
  organizing UF-2 and Increment 5 crew training in
  Moscow.
• Boeing support was instrumental to the successful
  7P and 4S Stage Operations Readiness Reviews.
• Weaknesses
• IP Integration support with regard to schedule
  maintenance, tracking and closure of action
  items, and meeting minutes continues to be weak.
• IP integration support to negotiations of issue
  resolution for the ESA Columbus hardware
  interfaces and test 3A has been ineffective.
• The contractor was delinquent in providing
  interface information on the TVCIC/PTU interface
  to allow the Floating Potential Measurement Unit
  (FPMU) to be integrated in place of the PTU. This
  information was requested in November however
  the complete, technically correct information was
  not provided until April 3rd. By this time, the
  design was so far along that design changes of
  the qual FPMU hardware were required, resulting
  in cost and schedule impacts.

B.10-2
31
Twelfth Period Final Evaluation International
Space Station Contract NAS 15-10000

• B11. Technical Performance Safety and Mission
  Assurance
• Summary
• The contractor actively participated in ISS
  flight operations, stage operations, and
  pre-flight and post-flight activities and was
  proactively engaged in the resolution of
  technical issues and in-flight anomalies
  throughout the reporting period. The submittal of
  FMEA/CIL worksheets has occurred in a timely
  manner. However, Critical Item presentations are
  not provided to ISS management on schedule,
  resulting in late identification of risk to
  management.
• Strengths
• The contractor proactively supported the 8A SSRMS
  brakes-on berthing loads issue. The contractor
  accurately characterized the risks and provided
  options to minimize the risk.
• The contractor successfully backfilled
  nonconformance reports from major Boeing sites
  into the PRACA system, ahead of the agreed-to
  schedule.
• Software Safety captured hazard analysis results
  for Computer Based Control Systems into a
  database structure. The delivery of the hazard
  data in this format enables JSC to use computer
  based tools to produce hazard traceability and
  command restrictions for use in Operations
  products. This product is a significant value to
  the safety review process.

B.11 -1
32
Twelfth Period Final Evaluation International
Space Station Contract NAS 15-10000

• B11. Technical Performance Safety and Mission
  Assurance
• Weaknesses
• During this period, twelve (12) mishaps were
  recorded where ISS personnel were injured on duty
  or ISS hardware was damaged. Although
  Boeing-ISS/GO management is placing emphasis on
  safe work practices and safety awareness,
  continued improvement in safety awareness is
  needed.
• The contractor is not actively managing resources
  at the HB and Huntsville sites. This is
  evidenced by cost overruns in quality assurance
  that are not being dealt with in a timely manner.
  
• The contractor did not adequately coordinate and
  integrate SMA issues prior to, and during,
  several ISS flight reviews (7P and 8A SORR, 8A
  delta SORR).
• The contractor continued to experience problems
  in providing necessary audit material and action
  item closure for PCAs.
• During the S1/P1 PCA, the contractor was unable
  to provide build paper and draft ADPs for
  review/audit, making it difficult to accomplish
  the objective of the PCA and led to large numbers
  of actions. The audit also revealed a general
  lack of quality discipline in the manufacturing
  and test documentation.
• The contractor is slow in closing action items
  assigned at the PCA/AR and the closure dates are
  routinely pushed to the right against the launch.
  
• First time quality of Reliability Block Diagram
  models is not at the desired level, which
  requires additional reviews and increased
  man-hours to ensure the model is accurate and
  complete.
• The contractor failed to review the
  implementation of design changes or operational
  controls to eliminate the hazards associated with
  the TCS braid flex hose sharp issue. The
  Contractor also failed to initiate a SCAN report
  to notify other US Hardware Providers and
  International Partners of the issue. Only after
  NASA Management involvement was a SCAN initiated.
  

B.11-2
33
Twelfth Period Award Fee Evaluation International
Space Station Contract NAS 15-10000

• Small and Disadvantaged Business (SDB) (15)
• Summary
• The contractor continues to exceed the NASA
  SDB goal of 8. The contractors current overall
  contract SDB percentage is 8.2.
• Strengths
• Boeing set in place their infrastructure for the
  addition of the HUB/Veterans/HBCU-MI goals by
  performing an information systems upgrade that
  captures input of these new supplier diversity
  goals.
• Boeings supplier, Muniz Engineering, certified
  as a Service Disabled Veteran business. This
  contract is valued at approximately 2.98M, which
  equates to about 1.5.
• Boeing's Protege', Cimarron, completed and
  received ISO certification.
• Boeing set in place an HBCU/MI initiative by
  implementing the 3rd year teacher in residence at
  Prairie View AM University.
• Weaknesses
• None

C.1 -1
34
Twelfth Period Final Evaluation International
Space Station Contract NAS 15-10000

• COST POOL
• Cost Performance
• This factor measures contractor cost performance
  in support of executing the program baseline
  plan.
• Summary
• The variance to the contract level funding
  baseline, as evaluated through the March 2002
  financial reports, was a 6.4 under-spend, which
  when adjusted for performance, showed a potential
  range of actual performance from the 6.4
  under-run to a 3.6 overrun. There are
  indications of a potential budget impact to
  future years due to performance within the period
  (estimates up to 10.3M). The figures are
  estimates which indicate nominal performance.
  See the cost assessment summary attachment.

Cost -1
35
Twelfth Period Final Evaluation International
Space Station Contract NAS 15-10000
Cost Performance SEE EXCEL SHEET
Cost-2
36
Twelfth Period Final Evaluation International
Space Station Contract NAS 15-10000

• Cost Management
• This factor measures contractors effectiveness
  in implementing the FY 2002 program baseline.
• Summary
• The contractor expended significant effort on
  the development of new financial reporting
  systems (OPP/Cobra) and the implementation of the
  new NASA Form 533. There continues to be
  significant problems in the cost management area.
  Financial reports were late and contained
  excessive errors, the earned value management
  system has degraded and the risk and opportunity
  system is not adequately portraying risks.
• Strengths
• The customer reporting team was very responsive
  in providing detailed variance information and
  supporting data, and resolving discrepancies in
  the reports. A particular example was the
  ability to quickly provide extensive historical
  data to the independent cost estimating teams
  supporting the program.
• Boeing implemented a NASA Form 533 M/Q during the
  period. The transition to this comprehensive
  financial report allowed for the discontinuation
  of five existing reports. The new document
  provides detailed total cost, a six-month
  forecast, fiscal year forecast, and latest
  revised estimate down to the 4th level Work
  Breakdown Structure (WBS). In addition, the
  program now has visibility down to the 4th level
  on total cost and EPs and at the 2nd level WBS
  by cost element. Additional visibility is
  provided for spares and subcontractors.

Cost-3
37
Twelfth Period Final Evaluation International
Space Station Contract NAS 15-10000

• Weaknesses
• Continued significant issues with financial
  reporting
• Reconciliation issues between the financial
  reports and management presentations remained a
  significant concern. Financial reports did not
  reconcile with management presentations or the
  SuperCAM database until March.
• Boeing financial reports continue to contain
  excessive errors and unexplained problems.
  Boeings deficiencies in the Forecast Report/533
  and the primary earned value report reduced the
  governments confidence in the Boeing financial
  systems and processes.
• Deliveries continue to be late. The forecast
  report/533 (PC27) was delivered late 3 out of 5
  occurrences. It was 4 days late in December, 5
  days late in January and 11 days late in
  February.
• A complete SuperCAM database, which includes
  detailed earned value data, has not been
  available since December.

Cost - 4
38
Twelfth Period Final Evaluation International
Space Station Contract NAS 15-10000

• Weaknesses (cont.)
• The Earned Value Management System (EVMS) has
  degraded examples include
• The contractor reported a 95M increase to the
  Variance at Completion for the DDTE work due to
  past period reconciliation. This action
  indicates a significant failure in the primary
  early warning tool for the contract. NASA has
  not received adequate explanations of this
  performance.
• The Defense Contract Audit Agency found
  discrepancies in data between the EVMS and
  accounting systems for 1999.
• The inability to properly measure performance on
  new work (OPD), for example the External
  Carriers, indicates this tool is unreliable.
• Risk Opportunities Management System
• Boeing failed to meet NASAs requirement to phase
  data by fiscal year.
• The system failed to consistently capture rate
  issues.

Cost - 5