USCCSE Annual Research Review

1
USC-CSE Annual Research Review

• Los Angeles, CA
• March 17, 2004

2
Outline

• Background
• Impact of 2003 Workshop on book
• Definitions and characterizations
• Home grounds and the five critical dimensions
• A risk-based approach to balancing agility and
  discipline
• Observations

3
Background (Rationale for book)

• Two approaches to software development
• Plan-driven (SW-CMM, document-based, heavy
  process)
• Agile (XP, tacit knowledge, light process)
• Both have strengths and weaknesses
• Agile and plan-driven proponents are believers
• Many of us are perplexed
• We still perceive perplexity
• More voices being heard
• Larman
• Anderson
• Rosenberg/Stevens
• Poppendieck
• Others???

4
Impact of 2003 Workshop - 1

• Disciplined changed to plan-driven
• Driven by process and product plans
• Both agile and plan-driven methods need to
  balance agility and discipline
• XP 12 highly-disciplined practices
• RUP Flexible mix of requirements, design, code

5
Impact of 2003 Workshop - 2

• More examples of hybrid methods
• AgileTek Agile with top-level plans,
  architecture
• Lightweight versions of RUP and TSP
• XP variants
• Part-time customers
• Longer workweeks
• Less collective ownership and metaphor
• More examples of empirical data
• Balancing illustration and need for transition
  guidance
• Collins Good to Great

6
Collins Good to Great
7
Trends since last year

• Increasing demands for agility, velocity, quality
• Alternative manifestos
• Push toward evolutionary acquisition in
  traditional communities
• More agile approaches to CMMI improvement,
  assessment
• More use of agile methods in large companies
• More hybrid agile/plan-driven approaches

8
Outline

• Background
• Impact of 2003 Workshop on book
• Definitions and characterizations
• Home grounds and the five critical dimensions
• A risk-based approach to balancing agility and
  discipline
• Observations

9
Sources of Perplexity

• Distinguishing method use from method misuse
• Claiming XP use when simply not documenting
• CMM Level 4 Memorial Library of 99 2-inch
  binders
• Overgeneralization based on the most visible
  instances
• XP is Agile CMM is Plan-driven
• Multiple definitions
• Quality customer satisfaction or compliance?
• Claims of universality
• The pace of IT change is accelerating and Agile
  methods adapt to change better than disciplined
  methods therefore Agile methods will take over
  the IT world.
• Software development is uncertain and the SW-CMM
  improves predictability therefore all software
  developers should use the SW-CMM

10
Sources of Perplexity (2)

• Early success stories
• Chrysler project that successfully birthed XP was
  later cancelled
• Cleanroom has never made it into the mainstream
• Purist interpretations (and internal
  disagreements)
• Don't start by incrementally adopting parts of
  XP. Its pieces fit together like a fine Swiss
  watch
• "An advantage of agile methods is that you can
  apply them selectively and generatively."
• "If you aren't 100 compliant with SW CMM Level
  3, don't bother to bid"
• "With the CMMI continuous interpretation, you can
  improve your processes in any order you feel is
  best."

11
Key Definitions

• Agile method
• one which fully adopts the four value
  propositions and twelve principles in the Agile
  Manifesto.
• Discipline (per Webster)
• control gained by enforcing obedience or order
• orderly or prescribed conduct or pattern of
  behavior
• self-control.
• Plan-driven
• Processes driven by documented plans (order is
  often defined in plans)
• Plan (per Webster)
• a method for achieving an end (a process plan)
• an orderly arrangements of parts of an overall
  design (a product plan).
• We assume that such plans are documented.

12
Finding Middle Ground

• A pragmatic view
• Both approaches have home grounds
• A broad range of environments and needs
• Trends require better handling of rapid change
• Processes should be the right weight for the job
• We believe risk-based analysis is the key to
  finding middle ground

13
Contrasts and Home Grounds

• Comparison is difficult, but we found 4 areas
  where there are clear differences
• Application
• Project goals, size, environment velocity
• Management
• Customer relations, planning and control,
  communications
• Technical
• How the software is developed
• Personnel
• The type and competency of developers and
  stakeholders

14
Application Characteristics

• Primary goals
• A Rapid value, responsiveness to change
• P Predictability, stability, high assurance
• Size
• A Small to medium
• P Large
• Environment
• A Turbulent, high change
• P Stable, requirements defined
• Project velocity
• A Code, learn, and fix
• P Architect for parallel, incremental development

15
Management Characteristics

• Customer Relations
• A Dedicated, collocated where feasible
• P Contractual increasingly evolutionary
• A Trust through working software and
  participation
• P Trust through process maturity evaluations
• Planning and Control
• A Means to an end
• P Anchor processes, communication
• Project Communications
• A Tacit, interpersonal knowledge
• P Explicit, documented knowledge

16
Technical Characteristics

• Requirements
• A Adjustable, informal stories
• P Formally baselined, complete, consistent
  specifications
• Development
• A Simple Design
• P Architecture-based design
• Testing
• A Automated, test-driven
• P Planned, requirements-driven

17
Technical Characteristics (2)Cost of Change
Beck, Li
Beck
Li
Li
18
Technical Characteristics (3) Cost of change
Poppendieck

• At least two cost-escalation curves
• Lean development Architect to defer high-stakes
  decisions e.g. COTS
• Easier to change an unmade decision

19
Technical Characteristics (4)Cost of Change TRW
Beck
Li
Li
20
Technical Characteristics (5)Cost of Change
CCPDS-R
Beck
21
Technical Characteristics (6)How Much
Architecting?
Beck
10,000 KSLOC
Sweet spot
100 KSLOC
Liu
Liu
10 KSLOC
22
Personnel Characteristics

• Customers
• A CRACK customers throughout development
• P CRACK customers early
• CRACK Collaborative, Representative, Authorized,
  Committed, and Knowledgeable
• Developers
• A Heavy mix of high caliber throughout
• P Heavy mix early with lower capability later
• Culture
• A Many degrees of freedom (Thrives on chaos)
• P Clear policies and procedures (Thrives on
  order)
• Collocated customers are difficult to
  find/keep/keep current on either side

23
Personnel Characteristics (2)Modified Cockburn
Levels
Level Characteristics 3 Able to revise a method
(break its rules) to fit an unprecedented new
situation 2 Able to tailor a method to fit a
precedented new situation 1A With training, able
to perform discretionary method steps (e.g.,
sizing stories to fit increments, composing
patterns, compound refactoring, complex COTS
integration). With experience can become Level
2. 1B With training, able to perform procedural
method steps (e.g. coding a simple method, simple
refactoring, following coding standards and CM
procedures, running tests). With experience can
master some Level 1A skills. -1 May have
technical skills, but unable or unwilling to
collaborate or follow shared methods.
24
Outline

• Background
• Impact of 2003 Workshop on book
• Definitions and characterizations
• Home grounds and the five critical dimensions
• A risk-based approach to balancing agility and
  discipline
• Observations

25
Summary of Home Grounds
26
Five Critical Decision Factors

• Represent five dimensions
• Size, Criticality, Dynamism, Personnel, Culture

27
Outline

• Background
• Impact of 2003 Workshop on book
• Definitions and characterizations
• Home grounds and the five critical dimensions
• A risk-based approach to balancing agility and
  discipline
• Observations

28
Using Risk to Balance Discipline and Agility -
Overview
29
Risks Used in Risk Comparison

• Environmental risks
• Technology uncertainties
• Many stakeholders
• Complex system-of-systems
• Legacy compatibility
• Agility risks
• Scalability
• Use of simple design
• Personnel turnover
• Too-frequent releases
• Not enough agile-capable people
• Plan-driven risks
• Rapid change
• Need for rapid results
• Emergent requirements
• Not enough discipline-capable people

30
Three Examples

• Agent-based systems
• Small Event Managers application
• Medium SupplyChain.com application
• Large National Information System for Crisis
  Management (NISCM) application
• Each example results in a development strategy
  based on risk analyses

31
Event Managers Project

• Small startup company
• Diverse set of smaller agent-based planning
  applications
• This project support for managing the
  infrastructure and operations of conferences and
  conventions
• Widening variety of options and
  interdependencies, makes an agent-based approach
  highly attractive

32
Event Managers Profile
33
Event Managers Strategy
34
SupplyChain.com Profile

• Turnkey agent-based supply chain management
  systems
• Distributed, multi-organization teams of about 50
  people
• Parts of applications are relatively stable,
  while others are highly volatile
• Architectures are driven by a few key COTS
  packages that are also continually evolving

35
SupplyChain.com Profile
36
SupplyChain.com Strategy
37
NISCM Profile

• Broad coalition of government agencies and
  critical private-sector organizations
• Support cross-agency and public-private sector
  coordination of crisis management activities
• Adaptive mobile network, virtual collaboration,
  information assurance, and information
  integration technology
• Private-sector system-of-systems integration
  contractor

38
NISCM Profile
39
NISCM Strategy
40
Risk Profiles Across Examples
41
Outline

• Background
• Impact of 2003 Workshop on book
• Definitions and characterizations
• Home grounds and the five critical dimensions
• A risk-based approach to balancing agility and
  discipline
• Observations

42
Summary of Observations

• No silver bullet
• Clear agile and plan-driven home grounds
• Increased demands for agility, velocity, quality
• Balanced methods are emerging

• Build methods up vs. tailor down
• Methods are important people more so
• Values
• Communications
• Expectations management

43
No Silver Bullet

• Brooks werewolf concerns
• Complexity, conformity, changeability,
  invisibility
• Agile methods
• Handle changeability and invisibility
• Miss on complexity and conformity
• Plan-driven methods
• Handle conformity and invisibility
• Miss on changeability and complexity

44
Future Applications Need Both

• Historically
• Many small, non-critical, well-skilled, agile
  culture, rapidly evolving projects
• Many large, critical, mixed-skill, ordered
  culture, stable projects
• In the future
• Large projects are no longer stable
• Maintenance of extensive process and product
  plans will become too expensive
• Complexity and conformity werewolves are waiting
  for agile projects
• Attributes of both approaches will be needed

45
Balanced Methods are Emerging

• Agile methods
• Crystal Orange
• DSDM
• FDD
• Lean Development
• Plan-Driven methods
• Rational Unified Process
• CMMI
• Hybrid
• Boehm-Turner Risk-based
• Code Science/Agile Plus

46
Build up Dont Tailor Down

• Plan-driven methods traditionally
• Define heavyweight process guidelines
• Advocate tailoring
• Treat mass of processes as security blanket
• Agilists traditionally
• Begin with the minimum
• Add as needed (and justified by cost-benefit)
• Start small extend only where necessary

47
Maybe its not the methods!

• Agile movement has echoed a long line of warning
  calls
• Success of agile may be due as much to people
  factors as to technology
• Valuing people over processes is the most
  important factor in the manifesto

I know I saw something about that in the process
somewhere
48
People

• Of the people, by the people, for the people
• Separations of concerns is increasingly harmful
• A few quotes
• The notion of user cannot be precisely defined,
  and therefore it has no place in computer science
  or software engineering. Dijkstra
• Analysis and allocation of the system
  requirements is not the responsibility of the
  software engineering group, but it is a
  prerequisite for their work. The Capability
  Maturity Model for Software.
• Software engineering is not project management.
  Tucker

49
Values

• Reconciling values is a critical people-oriented
  task
• Stakeholders value different things
• Software engineering is usually value-neutral
• Every requirement, object, test case, defect
  equally important
• Process improvement and plan-driven methods are
  inwardly-focused
• Aimed at productivity improvement
• Not higher value to customer
• Agilists attention to prioritization and
  negotiation are promising

50
Communications

• Face it, most engineers cant talk, much less
  write
• Need to bridge the customer-developer
  communications gap
• IKIWISI reigns
• Rapid change increases need for solid
  communication
• Few sources of guidance
• Alistairs Agile Software Development is a good
  starting place

Ill know It When I See It
51
Expectations Management

• Differences between successful and troubled
  projects is often expectations management
• SW developers have problems with EM
• Strong desire to please
• Little confidence in prediction
• Over confidence in abilities
• Most significant common factor in highly
  effective plan-driven or agile teams
• EM means not setting up unrealistic expectations
• Process mastery
• Preparation
• Courage

Developers seem to like Sisyphean tasks
52
Conclusions

• Plan-driven and agile methods both aim to
• Satisfy customers
• Meet cost and schedule arameters
• Home grounds exist, but the opportunity for
  integration is expanding
• We recommend a self-assessment of your
  organization and business
• Locate your place in the home ground space
• Identify areas of change and how they might move
  your location
• Look for ways to integrate method to better
  repond to change