Overview%20of%20Software%20Engineering

1
Overview of Software Engineering

• CS 330
• Spring 2007

2
Key Ingredients in successful organizations
Process
People
Customer
Technology
3
A better viewProcess and Technology supporting
people
People
Solution
Processes
Technology
4
What is software?

• Computer programs and associated documentation
• Software products may be developed for a
  particular customer or may be developed for a
  general market
• Software products may be
• Generic/COTS - developed to be sold to a range of
  different customers
• Custom- developed for a customer according to
  their specification

5
Engineering

• Engineering is
• The application of scientific principles and
  methods to the construction of useful structures
  machines
• Examples
• Mechanical engineering
• Computer engineering
• Civil engineering
• Chemical engineering
• Electrical engineering
• Nuclear engineering
• Aeronautical engineering

6
Software Engineering

• The term is 35 years old NATO Conferences
• Garmisch, Germany, October 7-11, 1968
• Rome, Italy, October 27-31, 1969
• The reality is it is finally beginning to arrive
• Computer science one the scientific basis
• Years of studies/experience/statistics provide
  basis too
• Many aspects have been made systematic
• Methods/methodologies/techniques
• Languages
• Tools
• Processes

7
Why Engineer Software ?

• The problem is complexity
• Many sources, but size is a key
• Mozilla contains 3 Million lines of code
• UNIX contains 4 million lines of code
• Windows 2000 contains 108 lines of code
• Second is role and combinatorics of state
• Third is uncertainty of inputs and their timing
• Fourth is the continuing changing environment
  and demands.
• Software engineering is about managing all the
  sources of complexity to produce effective
  software.

8
Software Engineering in a Nutshell

• Development of software systems whose
  size/complexity warrants team(s) of engineers
• multi-person construction of multi-version
  software Parnas 1987
• Scope
• study of software process, development/management
  principles, techniques, tools and notations
• Goal
• production of quality software, delivered on
  time, within budget, satisfying customers
  requirements and users needs

9
What does a software engineer do?

• Software engineers should
• adopt a systematic and organised approach to all
  aspects of software development.
• use appropriate tools and techniques depending on
  
• the problem to be solved,
• the development constraints and
• the resources available
• Understand and communicate processes for improved
  software development within their organization
• Be effective team members and/or leaders.
• Can be very technical or more managerial
  depending on organizational need.

10
What is the difference between software
engineering and computer science?
Computer Science
Software Engineering
is concerned with
Computer science theories are currently
insufficient to act as a complete underpinning
for software engineering, BUT it is a foundation
for practical aspects of software engineering
11
What is the difference between software
engineering and system engineering?

• Software engineering is part of System
  engineering
• System engineering is concerned with all aspects
  of computer-based systems development including
• hardware,
• software and
• process engineering
• System engineers are involved in
• system specification,
• architectural design,
• integration and deployment

12
Difficulties?

• SE is a unique brand of engineering
• Software is malleable
• Software construction is human-intensive
• Software is intangible and generally invisible
• Software problems are unprecedentedly complex
• Software directly depends upon the hardware
• It is at the top of the system engineering food
  chain
• Software solutions require unusual rigor
• Software state means behaviors can depend on
  history.
• Software has discontinuous operational nature

13
Software Engineering ? Software Programming

• Software programming
• Single developer
• Toy applications
• Short lifespan
• Single or few stakeholders
• Architect Developer Manager Tester
  Customer User
• One-of-a-kind systems
• Built from scratch
• Minimal maintenance

14
Software Engineering ? Software Programming

• Software engineering
• Teams of developers with multiple roles
• Complex systems
• Indefinite lifespan
• Numerous stakeholders
• Architect ? Developer ? Manager ? Tester ?
  Customer ? User
• System families
• Reuse to amortize costs
• Maintenance accounts for 60-80 of overall
  development costs

15
Economic and Management Aspects of SE

• Software Engineering is about improved ROI (can
  be Capital and/or Social ROI)
• Software production development maintenance
• Maintenance costs 60-80 of all (successful)
  development costs
• 20 corrective (12-16 total costs)
• 30 adaptive (18-24 total costs)
• 50 perfective (30-40 total costs)
• Quicker development is not always preferable
• higher up-front costs may defray downstream costs
• poorly designed/implemented software is a
  critical cost factor in system cost and delays

16
Relative Costs of Fixing Software Faults
17
Mythical Man-Monthby Fred Brooks

• Published in 1975, republished in 1995
• Experience managing development of OS/360 in
  1964-65
• Central argument
• Large projects suffer management problems
  different in kind than small ones, due to
  division in labor
• Critical need is the preservation of the
  conceptual integrity of the product itself
• Central conclusions
• Conceptual integrity achieved through chief
  architect
• Implementation achieved through well-managed
  effort
• software developers are not interchangeable
  work units.
• Brooks Law
• Adding personnel to a late project makes it later

18
Software EngineeringFrom Principles to Tools
19
Software Qualities

• Qualities are goals in the practice of software
  engineering, and directly relate to many of the
  guiding principles.
• External vs. Internal qualities
• Product vs. Process qualities

20
Software Qualities

• Critical Quality Attributes
• Correctness
• Maintainability
• Dependability
• Usability
• Reliability

• Other Attributes
• Completeness
• Compatibility
• Portability
• Internationalization
• Understandability
• Scalability
• Robustness
• Testability
• Reusability
• Customizability
• Efficiency

21
External vs. Internal Qualities

• External qualities are visible to the user
• reliability, usability, efficiency (maybe),
  robustness, scalability
• Internal qualities are the concern of developers
• they help developers achieve external qualities
• verifiability, maintainability, extensibility,
  evolvability, adaptability, portability,
  testability, reusability

22
Product vs. Process Qualities

• Product qualities concern the developed artifacts
• maintainability, performance, understandability,
• Process qualities deal with the development
  activity
• products are developed through process
• maintainability, productivity, predictability

23
Some Software Qualities

• Correctness
• ideal quality
• established w.r.t. the requirements/specification
• absolute
• Reliability
• statistical property
• probability that software will operate as
  expected over a given period of time/inputs
• relative

24
Some Software Qualities (cont.)

• Robustness
• reasonable behavior in unforeseen circumstances
• subjective
• a specified requirement is an issue of
  correctnessan unspecified requirement is an
  issue of robustness
• Usability
• ability of end-users to easily use software
• extremely subjective

25
Some Software Qualities (cont.)

• Understandability
• ability of developers to easily understand
  produced artifacts
• internal product quality
• subjective
• Verifiability
• ease of establishing desired properties
• performed by formal analysis or testing
• internal quality

26
Some Software Qualities (cont.)

• Performance
• equated with efficiency
• assessable by measurement, analysis, and
  simulation
• Evolvability
• ability to add or modify functionality
• addresses adaptive and perfective maintenance
• problem evolution of implementation is too easy
• evolution should start at requirements or design

27
Some Software Qualities (cont.)

• Reusability
• ability to construct new software from existing
  pieces
• must be planned for
• occurs at all levels from people to process,
  from requirements to code
• Interoperability
• ability of software (sub)systems to cooperate
  with others
• easily integratable into larger systems
• common techniques include APIs, distributed
  programming interfaces (CORBA, DCOM), plug-in
  protocols, etc.

28
Some Software Qualities (cont.)

• Scalability
• ability of a software system to grow in size
  while maintaining its properties and qualities
• assumes maintainability and evolvability
• goal of component-based development

29
Process Principles

• Prescribes all major activities
• Uses resources, within a set of constraints, to
  produce intermediate and final products
• May be composed of sub-processes
• Each activity has entry and exit criteria
• Activities are organized in a sequence
• Has a set of guiding principles to explain goals
• Constraints may apply to activity, resource or
  product

30
Software Development Stages

• Requirements Analysis Specification
• Conceptual/System/Architectural Design
• Detailed/Program Design
• Implementation/Coding
• Unit Integration Testing
• System Testing/Validation
• System Delivery/Deployment
• Maintenance
• Note there are many variations on the names.
  You are responsible for the main categories above
  (an on the next pages)..

31
Software Lifecycle Models

• Waterfall Model
• V Model
• Phased Development Model
• Incremental Model
• Prototyping Model
• Spiral Model

32
Software Development LifecycleWaterfall Model
Requirements
Design
Implementation
Integration
Validation
Deployment
33
V Model
Pfleeger 98
34
Phased Development Model
Pfleeger 98
35
Software Development Lifecycle Incremental Model
Version 1 Complete General Design
Version 2 Design/Implement first set of planned
new features. Note overlap with V1 schedule
Version 3 Design/Implement second set of
planned new features
36
Prototyping Model
Listen to Customer
Build/Revise Mock-Up
Customer Test-drives Mock-up
Pressman 97
37
Prototyping Model
Pfleeger 98
38
Spiral development

• Process is represented as a spiral rather than as
  a sequence of activities with backtracking.
• Each loop in the spiral represents a phase in the
  process.
• No fixed phases such as specification or design -
  loops in the spiral are chosen depending on what
  is required.
• Risks are explicitly assessed and resolved
  throughout the process.

39
Spiral model of the software process
40
Spiral model sectors

• Objective setting
• Specific objectives for the phase are identified.
• Risk assessment and reduction
• Risks are assessed and activities put in place to
  reduce the key risks.
• Development and validation
• A development model for the system is chosen
  which can be any of the generic models.
• Planning
• The project is reviewed and the next phase of the
  spiral is planned.

41
Evolutionary development

• Exploratory development
• Objective is to work with customers and to evolve
  a final system from an initial outline
  specification. Should start with well-understood
  requirements and add new features as proposed by
  the customer.
• Throw-away prototyping
• Objective is to understand the system
  requirements. Should start with poorly understood
  requirements to clarify what is really needed.

42
Evolutionary development
43
Evolutionary development

• Problems
• Lack of process visibility
• Systems are often poorly structured
• Special skills (e.g. in languages for rapid
  prototyping) may be required.
• Applicability
• For small or medium-size interactive systems
• For parts of large systems (e.g. the user
  interface)
• For short-lifetime systems.

44
Component-based software engineering

• Based on systematic reuse where systems are
  integrated from existing components or COTS
  (Commercial-off-the-shelf) systems.
• Process stages
• Component analysis
• Requirements modification
• System design with reuse
• Development and integration.
• This approach is becoming increasingly used as
  component standards have emerged.

45
Reuse-oriented development
46
Component-Based Development

• Develop generally applicable components of a
  reasonable size and reuse them across systems
• Make sure they are adaptable to varying contexts
• Extend the idea beyond code to other development
  artifacts
• Question what comes first?
• Integration, then deployment
• Deployment, then integration

47
Different Flavors of Components

• Third-party software pieces
• Plug-ins / add-ins
• Applets
• Frameworks
• Open Systems
• Distributed object infrastructures
• Compound documents
• Legacy systems

48
Process iteration

• System requirements ALWAYS evolve in the course
  of a project so process iteration where earlier
  stages are reworked is always part of the process
  for large systems.
• Iteration can be applied to any of the generic
  process models.
• Two (related) approaches
• Incremental delivery
• Spiral development.

49
Incremental delivery

• Rather than deliver the system as a single
  delivery, the development and delivery is broken
  down into increments with each increment
  delivering part of the required functionality.
• User requirements are prioritised and the highest
  priority requirements are included in early
  increments.
• Once the development of an increment is started,
  the requirements are frozen though requirements
  for later increments can continue to evolve.

50
Incremental development
51
Incremental development advantages

• Customer value can be delivered with each
  increment so system functionality is available
  earlier.
• Early increments act as a prototype to help
  elicit requirements for later increments.
• Lower risk of overall project failure.
• The highest priority system services tend to
  receive the most testing.

52
Extreme programming

• An approach to development based on the
  development and delivery of very small increments
  of functionality.
• Relies on constant code improvement, user
  involvement in the development team and pairwise
  programming.
• Covered in Chapter 17

53
Software Development LifecycleWaterfall Model
Requirements
Design
Implementation
Integration
Validation
Deployment
54
Software specification

• The process of establishing what services are
  required and the constraints on the systems
  operation and development.
• Requirements engineering process
• Feasibility study
• Requirements elicitation and analysis
• Requirements specification
• Requirements validation.

55
Requirements

• Problem Definition ? Requirements/Specification
• determine exactly what the customer and user need
  (maybe want)
• Requirements develop a contract with the customer
• Specification say what the software product is to
  do
• Difficulties
• client is computer/software illiterate (no idea
  what is doable)
• client asks for wrong product (want vs need)
• client is computer/software literate (specifies
  solution not need)
• specifications are ambiguous, inconsistent,
  incomplete
• Studies have shown that the percentage of defects
  originating during requirements engineering is
  estimated at more than 50 percent. The total
  percentage of project budget due to requirements
  defects is 25 to 40 percent.

56
The requirements engineering process
57
Software design and implementation

• The process of converting the system
  specification into an executable system.
• Software design
• Design a software structure that realises the
  specification
• Implementation
• Translate this structure into an executable
  program
• The activities of design and implementation are
  closely related and may be inter-leaved.

58
Design process activities

• Architectural design
• Abstract specification
• Interface design
• Component design
• Data structure design
• Algorithm design

59
The software design process
60
Structured methods

• Systematic approaches to developing a software
  design.
• The design is usually documented as a set of
  graphical models.
• Possible models
• Object model
• Sequence model
• State transition model
• Structural model
• Data-flow model.

61
Architecture vs. DesignPerry Wolf 1992

• Architecture is concerned with the selection of
  architectural elements, their interactions, and
  the constraints on those elements and their
  interactions necessary to provide a framework in
  which to satisfy the requirements and serve as a
  basis for the design.
• Design is concerned with the modularization and
  detailed interfaces of the design elements, their
  algorithms and procedures, and the data types
  needed to support the architecture and to satisfy
  the requirements.

62
Architecture/Design

• Requirements/Specification ? Architecture/Design
• architecture decompose software into
  modules/objects/components with interfaces
• design develop module/object/component
  specifications (algorithms, data types) and
  communication details
• maintain a record of design decisions and
  traceability
• specifies how the software product is to do its
  tasks
• Difficulties
• miscommunication between module designers
• design may be inconsistent, incomplete, ambiguous
• How to achieve a requirement may be unknown

63
Planning/Scheduling

• Before undertaking cost of development, need to
  estimate the costs/sizes of various steps
• Estimate Code size
• Estimate tools needed
• Estimate personnel
• Often Done after Architecture and before rest of
  design, but revised again after full design.
• Develop schedule for aspects of project lifecycle
• If doing predictive/quantitative SE, build on
  past experience, considering how to improve
  process.

64
Implementation Integration

• Design ? Implementation
• implement modules verify that they meet their
  specifications
• combine modules according to the design
• specifies how the software design is realized
• Difficulties
• module interaction errors
• order of integration may influence quality and
  productivity

65
Programming and debugging

• Translating a design into a program and removing
  errors from that program.
• Programming is a personal activity - there is no
  generic programming process.
• Programmers carry out some program testing to
  discover faults in the program and remove these
  faults in the debugging process.

66
The debugging process
67
Software validation

• Verification and validation (V V) is intended
  to show that a system conforms to its
  specification and meets the requirements of the
  system customer.
• Involves checking and review processes and system
  testing.
• System testing involves executing the system with
  test cases that are derived from the
  specification of the real data to be processed by
  the system.

68
Verification and Validation

• Analysis
• Static
• Science
• Formal verification
• Informal reviews and walkthroughs
• Testing
• Dynamic
• Engineering
• White box vs. black box
• Structural vs. behavioral
• Issues of test adequacy

69
The testing process
70
Testing stages

• Component or unit testing
• Individual components are tested independently
• Components may be functions or objects or
  coherent groupings of these entities.
• System testing
• Testing of the system as a whole. Testing of
  emergent properties is particularly important.
• Acceptance testing
• Testing with customer data to check that the
  system meets the customers needs.

71
Testing phases
72
Quality Assurance

• Done as part of each step
• Reduce costs by catching errors early.
• Help determine ambiguities/inconsistencies
• Help ensure quality product.

73
Deployment

• Completed End-User Documentation
• Separate from Developer documentation
• Installation Process(es)
• Customer test procedures
• Support Processes (help desk, etc)
• Trouble Tracking
• Repair/rework to address bugs
• Regression testing (as bugs are fixed)

74
Maintenance Evolution

• Operation ? Change
• maintain software during/after user operation
• determine whether the product still functions
  correctly
• Difficulties
• Rigid or fragile designs
• lack of documentation
• personnel turnover

75
Software evolution

• Software is inherently flexible and can change.
• As requirements change through changing business
  circumstances, the software that supports the
  business must also evolve and change.
• Although there has been a demarcation between
  development and evolution (maintenance) this is
  increasingly irrelevant as fewer and fewer
  systems are completely new.

76
System evolution
77
Why I include CASE Tools

• Computer Aides Software Engineering tools support
  good SE processes (e.g. UML)
• Some tools absolute requirement for scaling e.g.
  build and configuration management.
• Integrated CASE (ICASE) tools embody good
  processes and improve productivity (E.g. Rational
  tool set)
• Some tools (e.g. debuggers, Purify) do almost
  impossible for humans.

• But.. Tools change
• No SE tools from my first 3 jobs exist (except
  Fortran/C languages)
• I use regularly use 3 SE tools from my next set
  of jobs.
• Other tools I learned have been replaced with
  similar but expanded concepts.. Understanding
  todays tools gives a basis for learning future
  ones.

78
ICASE Design Tools

• Rational Rose and Rational Unified Development.
• From UML drawing to code and back.
• Generates stubs and eventually testing code.
• Supports multiple languages

79
Configuration Management

• CM is a discipline whose goal is to control
  changes to large software through the functions
  of
• Component identification
• Change tracking
• Version selection and baselining
• Managing simultaneous updates (team work)
• Build processes with automated regression testing
• Software manufacture

80
CM in Action
1.0
81
Build Tools

• Necessary for large projects. Keep track of what
  depends upon on what, and what needs recompiled
  or regenerated when things change.
• Important even for small 1-person projects as
  soon as you have multiple files.
• Can do much more than just compile, can
  generate document (if using code-based docs),
  generate manufactured code (e.g. SOAP
  interfaces), even send emails or suggest
  alternatives.
• E.g. in our IUE project, edit some files
  compile was one in seconds, edit another and a
  rebuild taking days would be needed. If more than
  30 files impacted, our make process recommend a
  new branch to avoid conflicts!

82
Debugging Tools

• How do you see what the code is really doing (not
  what it seems it should do)?
• How to you see what happened to code during
  compiler optimization?
• How do you find/track down the cause of
  Segfault/GFP in code youve never seen before?
• How can you test various possibilities without
  generating special code or recompiling.
• How do you track down a memory leak?

83
Tools, workbenches, environments
84
The Rational Unified Process

• A modern process model derived from the work on
  the UML and associated process.
• Normally described from 3 perspectives
• A dynamic perspective that shows phases over
  time
• A static perspective that shows process
  activities
• A practive perspective that suggests good
  practice.

85
RUP phase model
86
RUP phases

• Inception
• Establish the business case for the system.
• Elaboration
• Develop an understanding of the problem domain
  and the system architecture.
• Construction
• System design, programming and testing.
• Transition
• Deploy the system in its operating environment.

87
RUP good practice

• Develop software iteratively
• Manage requirements
• Use component-based architectures
• Visually model software
• Verify software quality
• Control changes to software

88
Static workflows
89
Computer-aided software engineering

• Computer-aided software engineering (CASE) is
  software to support software development and
  evolution processes.
• Activity automation
• Graphical editors for system model development
• Data dictionary to manage design entities
• Graphical UI builder for user interface
  construction
• Debuggers to support program fault finding
• Automated translators to generate new versions of
  a program.

90
Case technology

• Case technology has led to significant
  improvements in the software process. However,
  these are not the order of magnitude improvements
  that were once predicted
• Software engineering requires creative thought -
  this is not readily automated
• Software engineering is a team activity and, for
  large projects, much time is spent in team
  interactions. CASE technology does not really
  support these.

91
CASE classification

• Classification helps us understand the different
  types of CASE tools and their support for process
  activities.
• Functional perspective
• Tools are classified according to their specific
  function.
• Process perspective
• Tools are classified according to process
  activities that are supported.
• Integration perspective
• Tools are classified according to their
  organisation into integrated units.

92
Functional tool classification
93
Activity-based tool classification
94
CASE integration

• Tools
• Support individual process tasks such as design
  consistency checking, text editing, etc.
• Workbenches
• Support a process phase such as specification or
  design, Normally include a number of integrated
  tools.
• Environments
• Support all or a substantial part of an entire
  software process. Normally include several
  integrated workbenches.

95
Boults view of SE

• SE must balance risks in software development
  process
• Risks of error in
• requirements
• specification,
• design,
• implementation,
• and integration
• Risks of exceeding available resources
• Risks of being late on delivery or missing the
  market
• Dont let push for formality dominate your
  process.
• Dont let push for expedience destroy your
  process.

96
Software Process Qualities

• Process is reliable if it consistently leads to
  high-quality products
• Process is robust if it can accommodate
  unanticipated changes in tools and environments
• Process performance is productivity
• Process is evolvable if it can accommodate new
  management and organizational techniques
• Process is reusable if it can be applied across
  projects and organizations

97
Assessing Software Qualities

• Qualities must be measurable/quantifiable
• Measurement requires that qualities be precisely
  defined
• Improvement requires accurate and consistent
  measurements
• For most SD groups, qualities are informally
  defined and are difficult to assess

98
Software Engineering Axioms

• Adding developers to a project will likely result
  in further delays and accumulated costs
• The longer a fault exists in software
• the more costly it is to detect and correct
• the less likely it is to be properly corrected
• Up to 70 of all faults detected in large-scale
  software projects are introduced in requirements
  and design
• detecting the causes of those faults early may
  reduce their resulting costs by a factor of 200
  or more
• Basic tension of software engineering
• better, cheaper, faster pick any two!
• functionality, scalability, performance pick
  any two!
• Want/Need Managements buy in to formal SE
  process.
• If you dont document your process, you dont
  have one!

99
Boehms Spiral Model
EVALUATE ALTERNATIVES
DETERMINE GOALS,
AND RISKS
ALTERNATIVES,
CONSTRAINTS
4
Constraints
Risk analysis
4
Risk analysis
3
Constraints
4
3
Alternatives
3
2
Constraints
Risk analysis
2
Alternatives
2
Alternatives
Risk analysis
Constraints
Proto
-
Proto
-
Proto
-
1
Alternatives
Budget
type
type
type
Budget
Budget
Prototype
4
2
3
4
Budget
1
2
3
1
1
1
start
Requirements,
Concept of
Detailed
life-cycle plan
operation
Software
design
design
Software
requirements
Development
Integration
Code
Validated
plan
and test plan
requirements
Validated,
verified design
Unit test
System
test
Acceptance
Implementation
test
plan
PLAN
DEVELOP AND TEST
100
Key points

• Software processes are the activities involved in
  producing and evolving a software system.
• Software process models are abstract
  representations of these processes.
• General activities are specification, design and
  implementation, validation and evolution.
• Generic process models describe the organisation
  of software processes. Examples include the
  waterfall model, evolutionary development and
  component-based software engineering.
• Iterative process models describe the software
  process as a cycle of activities.

101
Key points

• Requirements engineering is the process of
  developing a software specification.
• Design and implementation processes transform the
  specification to an executable program.
• Validation involves checking that the system
  meets to its specification and user needs.
• Evolution is concerned with modifying the system
  after it is in use.
• The Rational Unified Process is a generic process
  model that separates activities from phases.
• CASE technology supports software process
  activities.