ValueBased Processes for COTS Based Development CommercialOffTheShelf

1
Value-Based Processes for COTS Based
Development(Commercial-Off-The-Shelf)
COTS The Future is Here

• Jesal Bhuta
• jesal_at_usc.edu

2
Outline

• Introduction
• COTS Process Drivers
• Processes for COTS-Based Development CBA
  Decision Framework
• Case Study
• Study CBA Decision Framework
• Evaluation for COTS Integration
• Principles for COTS Process Models

3
What is COTS?

• COTS definition by SEI a product that is
• Sold, leased or licensed to the general public
• Offered by a vendor trying to profit from it
• Supported and evolved by the vendor, who retains
  the intellectual property rights
• Available in multiple copies
• Used without internal modification by a consumer

4
Example COTS Use from CSCI 577 Projects
5
COTS Systems Definitions

• COTS Based Systems (CBS)
• Any system that uses COTS
• COTS Based Applications (CBA)
• A system for which
• At least 30 of the end-user functionality is
  provided by COTS products and
• At least 10 of the project effort is devoted to
  COTS related activities
• The numbers 10 and 30 are approximate
  behavioral CBA boundaries observed in the USC
  e-services projects
• Paradigm shift from custom dev. to COTS-based
  dev.
• USC e-services projects CBA projects increase
  from 28 in 1997 to 70 in 2002

6
CBA Increasing Trend

• CBA Growth Trends

7
Why use COTS?

• Change in software development practice over the
  past 20 years
• Build system with pre-existing software to reduce
  development and maintenance costs
• One such source COTS
• COTS Based Systems
• Involve less development time and lower
  development cost by taking advantage of existing,
  market proven, vendor supported products.

8
Using COTS - Trade Offs

• Two main characteristics of COTS
• source code not available to developer
• evolution not under control of developer
• Results in trade-off
• development time can be reduced, but often at
  cost of increased software component integration
  work

9
COTS Pros and Cons

• Pros
• Available now, earlier payback
• Avoids expensive development maintenance
• Predictable license costs performance
• Rich in functionality
• Broadly used, mature technology
• Frequent upgrades often anticipate organizations
  needs
• Dedicated support organization
• Dedicated support organization
• Hardware/software independence
• Tracks technology trends

• Cons
• Licensing and procurement delays
• Up front license fees
• Recurring maintenance fees
• Reliability often unknown/ inadequate
• Unnecessary features compromise usability,
  performance
• Functionality, efficiency constraints
• No control over upgrades/maintenance
• Dependency on vendor
• Integration not always trivial incompatibilities
  among different COTS
• Synchronizing multiple-vendor upgrades

10
When is COTS right for you

• When they lie at the intersection of the three
  determinants of feasibility, and do so
  demonstrably better than could original code
• Technical,
• Economic
• Strategic constraints

11
When is COTS right for you

• Technical constraint
• Ability supply the desired functionality at the
  required level of reliability
• Economic constraint
• Ability to be incorporated and maintained in the
  new system within the available budget and
  schedule
• Strategic constraint
• Ability to meet needs of the system operating
  environment--including technical, political, and
  legal considerations--now, and as environment is
  expected to evolve in the future

12
COTS is not a Silver Bullet

• However, COTS is not a Silver Bullet
• Involving short-term long-term cost, evolution
  and associated risks
• Requiring different processes w.r.t. new skill,
  knowledge, and abilities
• If not handled well, resulting in difficulties to
  meet expected economic objectives, even causing
  tremendous cost and schedule overruns
• Need for COTS-Oriented Processes

13
Outline

• Introduction
• COTS Process Drivers
• Processes for COTS-Based Development CBA
  Decision Framework
• Case Study
• Study CBA Decision Framework
• Evaluation for COTS Integration
• Principles for COTS Process Models

14
COTS Process Drivers

• Instant capabilities vendor-maintained
• Economic attractiveness to adopt
• Capabilities determined by vendor
• Need to rethink requirements engineering
• Content driven by need for competitive
  differentiation
• General lack of interoperability
• Need for complex architecting, glue code
• Source code generally not available
• Complications testing and debugging
• New releases average every 10 months
• Adaptive maintenance starts early and
  proliferates
• Vendor support duration averages 3 releases
• Contract for refreshed COTS when outsourcing

15
Software Development Phase Activities Custom
COTS
16
Outline

• Introduction
• COTS Process Drivers
• Processes for COTS-Based Development CBA
  Decision Framework
• Case Study
• Study CBA Decision Framework
• Evaluation for COTS Integration
• Principles for COTS Process Models

17
Value-Based CBA Process Framework Interfaces
18
Value-Based CBA Process Framework Features

• Adapts to frequent go-backs with a recursive, and
  reentrant Spiral framework
• Reflecting evolving stakeholder value
  propositions and risk mitigation strategies
• Defines process navigation options and three
  composable process elements to accommodate
  process flexibility
• Assessment
• Tailoring
• Glue code development and integration
• Applicable in each win win Spiral cycle
• Facilitates development effort/resource
  allocation based on COCOTS estimates
• Answers two questions
• What to do next?
• How much of it to do?

19
Value-Based CBA Process Framework
20
Assessment Process Element
21
A4 Detailed Assessment
Effort Reporting!!!
Effort Reporting

• Prototyping
• Tailoring to evaluate COTS tailorability/usability
  
• Glue code to test the interoperability
• Business Case Analysis
• COCOTS model for effort and schedule estimation
• Assessment, glue code, tailoring effort tradeoffs
• Return on Investment analysis
• Value, risk, cost tradeoffs
• COTS specific risks and mitigation costs
• Vendor volatility, upgrade lifecycle, etc.

22
Exits from Assessment

• Single full COTS solution is the best
• which means there is a single COTS product or a
  combination of COTS products covering desired
  OCPs
• Partial COTS solution is the best
• which means that COTS product(s) only cover part
  of the OCPs, and custom development is needed
  to meet the gap between COTS and OCPs
• No COTS products are acceptable
• which means that pure custom development is the
  optimal solution, unless the stakeholders are
  willing to adjust unsatisfied OCPs

23
COTS Assessment Guidelines

• COTS Assessment Background (CAB)
• Provides the minimum essential set of
  organization objectives, constraints, priorities
  (OCPs), and situation background needed to
  perform a COTS/NDI assessment
• COTS Assessment Plan (CAP)
• Covers the minimum essential why/whereas,
  what/when, who/where, how, and how much aspects
  of the activity being planned
• COTS Assessment Report (CAR)
• Covers the COTS assessment objectives, context,
  approach, results, conclusions, recommendations,
  and supporting data, plus other significant
  topics if any

24
Resources How much?

• Is a function of of COTS candidate products
• COCOTS Assessment sub-model
• Assessment Effort is estimated based on
• of COTS classes
• of COTS candidates in each class

COCOTS Estimation Model
25
Example USCCS Evaluation Criteria
26
Example USCCS Feature Checklist
27
Example USCCS Evaluation Results
28
Tailoring Process Elements

• Tailoring options
• GUI Based
• Parameter Based
• Programmable

29
Trade-Offs for Tailoring

• Tailoring effort can vary significantly depending
  on COTS package used
• Automated tailoring tools
• E.g. Microsoft Excel macro recorder
• Extensive tailoring can cause much rework during
  COTS refresh cycles
• Oracle Use our Business Processes
• Tailoring effort v/s functionality tradeoff
• Minimum tailoring effort to obtain maximum
  possible functionality
• Tailoring easy to redo during COTS refresh
  cycles

30
COCOTS to Estimate Tailoring Effort

• Tool calibrated by the class of product being
  tailored
• Tailoring Effort S ( of COTS tailored in
  class)(average tailoring effort for class and
  complexity) over all classes, by project domain
• Complexity of tailoring task rated over five
  levels (very low to very high) for
• Parameter specification
• Script writing
• I/O report layout
• GUI Screen specification
• Security/Access protocol initialization and setup
• Availability of COTS tailoring tools

31
Glue Code Process Element

• Architecture considerations
• Determine interconnection topology options
• Minimize the complexity of interactions
• Select connectors w.r.t. COTS interfaces
• Identify potential architectural mismatches

32
Architectural Considerations

• Selection of Connectors
• Communication (Data)
• Coordination (Control)
• Facilitation (Control)
• Conversion (DATA)
• Available COTS interfaces and their support
  connectors
• E.g. Eclipse supports Invocation, Data sharing,
  Application Programming Interfaces, User
  Interface Integration
• Architectural mismatches between COTS packages

33
COCOTS to estimate Glue-Code Effort

• Glue code effort Asize(1CREVOL)B.p(effort
  multipliers)
• Where
• A linear scaling constant
• Size glue code in source lines of code or
  function points
• CREVOL percentage of rework of glue code due to
  requirements change or volatility in the COTS
  products
• B an architectural nonlinear scaling factor
• Effort multipliers 13 multiplicative effort
  adjustment factors with ratings from very low to
  very high

34
CBA Process Plan Template
35
Elaborated Win-Win Spiral Model
36
Outline

• Introduction
• COTS Process Drivers
• Processes for COTS-Based Development CBA
  Decision Framework
• Case Study
• Study CBA Decision Framework
• Evaluation for COTS Integration
• Principles for COTS Process Models

37
Example CBA Oversize Image Viewer

• Client
• A librarian at USC
• Developers
• A development team of 5 graduates at USC
• Objectives
• Development of a viewing capability for oversized
  images, including features such as
• image navigation, cataloging, search, archive,
  and access administration

38
Applying the CBA Decision Framework

• First three spiral cycles
• Each cycle description begins with its use of the
  Win Win Spiral Model, as the primary sequencing
  of tasks is driven by the success-critical
  stakeholders win conditions and the projects
  major risk items

39
Framework Application to Oversize Image Viewer (1)
40
Framework Application to Oversize Image Viewer (1)
Assessment element
Tailoring element
New Objective
41
Framework Application Spiral Cycle I
42
Framework Application Spiral 1 Assessment
Process
43
Framework Application to Oversize Image Viewer (2)
44
Framework Application to Oversize Image Viewer (2)
45
Framework Application to Oversize Image Viewer (2)
Assessment element
Assessment element
46
Framework Application Spiral 2
47
Framework Application to Oversize Image Viewer (3)
48
Framework Application to Oversize Image Viewer (3)
Assessment element
Tailoring element
Glue code element
49
Framework Application Spiral 3 and Further
50
Spiral 3 Glue Code Process Element
51
Spiral 3 Tailoring Process Element
52
Use of CBA Decision Framework Spiral Cycle 1

• Entrance
• The first two steps establish the preconditions
  for entering the CBA Assessment decision
  framework
• Assessment Process Element
• Detailed assessment involved Tailoring to
  accommodate the newspaper image files, and
  identified ER Mapper as the best OIV solution.
• Exit
• In the Partial COTS solution best direction.

53
Use of CBA Decision Framework Spiral Cycle 2

• Go-back
• The new stakeholders and OCPs in cycle 2
  required the project to backtrack to the
  beginning of the Assessment process element.
• Assessment Process Element
• ER Mapper was filtered out when it declined to
  guarantee early Unix and Mac versions.
• Tailoring Process Element
• Tailoring was required to verify that Mr. SID
  performed satisfactorily on Unix and Mac
  platforms.
• Assessment Process Element
• Concurrently, Assessment filtering and evaluation
  tasks were being performed for the cataloguing
  and GUI functions.

54
Use of CBA Decision Framework Spiral Cycle 3

• Assessment Process Element
• The Assessment of detailed interoperability
  characteristics of Mr. SID, MySQL, and the GUI
  software on the Windows, Unix, and Mac platforms.
• The interoperability assessment involved
• Tailoring Process Element
• Tailoring was required to verify that Mr. SID
  performed satisfactorily on Unix and Mac
  platforms.
• Glue Code Process Element
• To enable interoperability

55
Use of CBA Decision Framework Summary

• Subsequent spiral cycles to develop the core
  capability and the IOC did not involve further
  Assessment, but involved concurrent use of the
  Tailoring, Glue Code, and custom development
  processes.
• The use of the CBA decision framework during the
  three spiral system definition cycles and the
  subsequent development activity can be summarized
  by the sequence A, T (ATA) A, (TG) (TGC).

56
Summary of CBA Decision Framework Use

• Subsequent spiral cycles to develop the core
  capability and the IOC did not involve further
  Assessment, but involved concurrent use of the
  Tailoring, Glue Code, and custom development
  processes.
• The use of the CBA decision framework during the
  three spiral system definition cycles and the
  subsequent development activity can be summarized
  by the sequence A, T (ATA) A, (TG) (TGC).

57
Outline

• Introduction
• COTS Process Drivers
• Processes for COTS-Based Development CBA
  Decision Framework
• Case Study
• Study CBA Decision Framework
• Evaluation for COTS Integration
• Principles for COTS Process Models

58
CBA Framework Study

• 2. During the semester
• Individual weekly effort reporting
• Team weekly progress report

• 1. Beginning of semester
• COTS lecture
• COTS Experience Survey

• 3. End of semester
• COCOTS estimation report
• COTS process usage feedback survey

59
Outline

• Introduction
• COTS Process Drivers
• Processes for COTS-Based Development CBA
  Decision Framework
• Case Study
• Study CBA Decision Framework
• Evaluation for COTS Integration
• Principles for COTS Process Models

60
COTS Integration Issues

• We have licensed COTS packages A and B, but they
  do not to work together
• We selected a programming language X to build our
  system, but it does not communicate well with
  COTS A
• Our system worked well with JRE 1.4.0, but when
  we upgraded the JRE to 1.4.2 it fails
• The connector we selected does not support the
  conversion of a specific data-type
• The system requires an overall response time of 4
  seconds, but the single interaction between
  components A and B itself takes 5 seconds

61
COTS Interoperability Evaluation

• Identify architecture choices
• FRD 7.2 System Structure
• Identify COTS, Custom, and Legacy components and
  connectors and their characteristics
• FRD 7.3 COTS, Legacy, and Custom Components and
  Connectors Description
• Identify component, granularity, packaging,
  inputs, outputs (only the ones required),
  binding, information flow, dependencies,
• Perform theoretical analysis to rule out
  combinations (LCO)
• Prototype interoperability for the rest of the
  combinations (LCA)

62
Example Digital Imaging System
63
Example Feasible Combination

• Combination
• Database MySQL
• Database Connector Java-MySQL Connector
• Business Logic language Java
• Image manipulation toolkit Java Image
  Manipulation toolkit
• FTP Server Windows, Linux, Unix
• Browser Internet Explorer, Mozilla Firefox,
  Safari
• Evaluation
• Available Java image manipulation toolkit,
• Java libraries support FTP to store and retrieve
  images,
• Java-MySQL connector successfully transfers data
  types that are required (string, and integer)
• MySql response time to a query for a 1 GB
  database less than 2 seconds

64
Example Infeasible Combination

• Combination
• Database MS Access
• Database Connector ADODB Connection
• Business Logic language Visual Basic, C, Visual
  J
• Image manipulation toolkit MS Image Manipulation
  Classes
• FTP Server Windows, Linux, Unix
• Browser Internet Explorer, Mozilla Firefox,
  Safari
• Evaluation
• MySql response time to a query for a 1 GB
  database more than 2 seconds

65
Outline

• Introduction
• COTS Process Drivers
• Processes for COTS-Based Development CBA
  Decision Framework
• Case Study
• Study CBA Decision Framework
• Evaluation for COTS Integration
• Principles for COTS Process Models

66
Principles for CBA Process Models

• Process happens where the effort happens
• Dont start with Requirements
• Avoid premature commitments -- but have and use a
  plan
• Buy information early to reduce risk and rework
• Prepare for COTS changes

67
1. Process Happens Where the Effort Happens

• Effort sources
• COTS Assessment, Tailoring, Glue Code/Integration

68
Activity Sequence Patterns

• COTS activity sequences

69
2. Dont Start With Requirements

• Definition of Requirement Webster
• Something required claimed or asked for by right
  and authority
• Semantics ingrained into customer and user
  cultures
• Pitfalls unmanageable expectations, severe COTS
  constraints
• Process examples Waterfall variants
• Counterexample large TRW-Government project
• Hazard Many corporate, Government policies
  enforce Waterfall

70
Waterfall Variant UMD CBA Process
71
Waterfall Processes Over-constraint COTS Options
72
Hazard Avoidance Waterfall Policies

• Interpret as risk-driven waterfall
• Defer Requirement Review until risks are resolved
  or covered by risk management plans
• Concurrently engineer requirements, architecture,
  plans, COTS
• Use anchor point milestone definitions
• With Pass/Fail Feasibility Rationales

73
3. Avoid Premature Commitments- But have and use
a plan

• Pitfalls study-wait-study cycle lack of
  progress metrics
• Process examples convergent spheres of concern
• Counterexample software environment S-W-S cycle
• Hazard Avoiding premature commitments by
  avoiding commitments
• Hazard avoidance
• Use a tailorable planning framework
• Use goal-oriented adaptive control process to
  monitor and adjust plans

74
Converging Spheres SEI Epic Process
75
4. Buy Information to Reduce Risk

• Pitfall COTS interoperability, scalability,
  culture match
• Hazard pressure to be decisive, commit early
• Hazard avoidance Evaluate COTS risk mitigation
  options use CBA-specialized spiral process
  framework

76
5. Prepare for COTS Changes

• New releases every 10 months (GSAW 2000-03)
• Releases unsupported after 3 newer releases
• Releases likely to diverge
• Vendor need product differentiation
• Change interactions scale as high as square of
  COTS products
• Large outsourced applications may have
  unsupported COTS
• Example 3-year project 120 COTS 55 unsupported
  (46)

77
Coping with COTS Changes

• Win-win relationships with COTS vendors
• Pro-active market watch
• Reduce number of COTS products and vendors
• Reduce inter-COTS coupling
• Wrappers, standards, mediators
• Develop, evolve COTS refresh strategy
• Contract for delivery of refreshed COTS

78
Questions