CIT 595 Project Presentation

1
CIT 595Project Presentation

• Software Quality Assurance and Testing
• Ben Kim

2
Importance of SQA / Testing

• NASA / DoD / DoE
• If any of things below fails, the consequence can
  be disastrous

3
Role of Software in a Satellite Development

• Unified Modeling Language (UML) used for
  high-level software modeling and functional
  analysis (i.e. use cases, sequence diagrams)
  Software Architecture and Planning
• Software development and configuration management
  for onboard software (bus/payload) and ground
  segment (mission planning, mission operations,
  etc.) Software Coding and Detailed Design
• Testing of software and integration with
  satellite hardware and ground segment Software
  Test and Integration

4
Concerns about Quality

• General Accounting Office reported in 1979 that
  of the government software development projects
  studied
• more than 50 had cost overruns
• more than 60 had schedule overruns
• more than 45 of the delivered software could not
  be used
• more than 29 of the software contracted for was
  never delivered
• more than 19 of the delivered software had to be
  reworked.
• Hence, the term software crisis was coined!

5
More Concerns about Quality Today

• Programmers spend 90 of their time on the first
  80 of a project and 90 of their time the final
  20 May 11, 1990 on WSJ
• 15 of all software projects never deliver
  anything.
• Overruns of 100-200 are common.
• Software projects are considered successful when
  overruns are held to 30

6
Key Definitions

• Software Quality Assurance (SQA) is a set of
  systematic activities providing evidence of the
  ability of the software process to produce a
  software product that is fit to use The SQA
  Handbook
• Software Testing is the process used to help
  identify the correctness, completeness, security,
  and quality of developed computer software (i.e.
  enabler for SQA)

7
ISO 9126

• An International standard for evaluation of
• software quality and specifies the attributes
  that
• enables high-quality software
• Functionality are required functions available
  in the software?
• Reliability how reliable is the software?
• Usability is the software easy to use?
• Efficiency how efficient is the software?
• Maintainability how easy is it to modify the
  software?
• Portability how easy is it to transfer the
  software to another environment

8
CMMI

• Capability Maturity Model Integrated (CMMI) from
  Software Engineering Institute (SEI) at Carnegie
  Mellon University.
• Five-level framework assessing the maturity of an
  organizations software process, based on
  specific key process areas.
• Uses Standard CMMI Appraisal Method for Process
  Improvement (SCAMPI) to appraise and rate an
  organization (levels 1 through 5)
• Level 1 Initial
• Level 2 Repeatable
• Level 3 Defined
• Level 4 Managed
• Level 5 - Optimized

9
Importance of Quality Metrics

• When you can measure what you are speaking
  about, and express it in numbers, you know
  something about it when you cannot express it in
  numbers, your knowledge is of a meager and
  unsatisfactory kind
• - Lord Kelvin (1899)

10
Quality Metrics Examples

• Software Line of Codes SLOCs
• Software Reuse
• Requirement Stability
• Defect Reports (DR) / Defects Density Profile
  (DDP)
• McCabe Cyclometic Complexity broad measure of
  soundness and confidence for a program the
  metrics measure the number of linearly
  independent paths through a program module.

11
Software Testing
12
Black-Box Testing

• Treats software as a black-box without any
  understanding internal behavior (tester knows
  input/output only).

Whats happening in here?
Legal Input
Expected Output
13
Black-Box Test Examples

• Functional Testing test for functional
  requirements
• Stress Testing test against more stringent
  criteria than required
• Scenario Testing test against realistic
  scenarios / threads.
• Regression Testing retesting after changes are
  made
• User Acceptance (Beta Testing) tests to see if
  software meets user requirements at the handoff
  to the user

14
White-Box Testing

• Allows one to peek inside the box and focuses
  specifically on using internal knowledge of
  software to guide selection of data

include ltstdio.hgt int main() int claritin
printf(I can see the box!)
Legal Input
Expected Output
15
White-Box Test Examples

• Unit Testing check to see the particular
  module/unit of code is working fine
• Static and Dynamic Analysis going thru code to
  find out any defects / executing code and
  analyzing output
• Security Testing test to find out how well
  system can protect itself from unauthorized
  access, hacking, etc.
• Code Coverage See example next

16
Code Coverage Example (White Box)

• void foo(int bar)
• printf("This is ")
• if (bar lt 0)
• printf("not ")
• printf("a positive integer.\n")
• return

void foo(int bar) printf("This is ") if
(bar lt 0) printf("not ")
printf("a positive integer.\n") return
void foo(int bar) printf("This is ") if
(bar lt 0) printf("not ")
printf("a positive integer.\n") return
bar -1 Achieve statement coverage, but does
not achieve conditional coverage
bar 1 Does not achieve statement coverage, and
does not achieve conditional coverage
17
Testing Hierarchy SQA

• Automated Script Test / Scaffolds
• Test Plans / SW Builds via CM tool
• SW / HW Physical and Functional Configuration
  Audits

• Unit-Test
• Integration Testing
• System Testing
• System Integration Testing

More lines of code
18
JUnit Test Case
// Test Scenario public void testExecTReventDiapa
tcher() proc.setTime(0) //set clock to 0
sec for (i1ilt100i) //generates a random
track Track obj genTrack() //send track to
ProcessTracks proc.putTrack(obj) //advance clock
by 1 sec proc.incrTime(1) this.assertTrue(proc.
isSuccess()) Track genTrack() // code for
generating a random track
import junit.framework. public class
TestProcessTracks1 extends TestCase private
ProcessTracks proc null public
TestProcessTracks1(String name)
super(name) protected void setUp() throws
Exception super.setUp() proc new
ProcessTracks () protected void tearDown()
throws Exception proc null super.tearDown()

19
Test Driven Development (TDD)

• A software development practice in which unit
  test cases are incrementally written prior to
  code implementation.
• According to one industry study (ACM Press,
  2005), TDD produces code that passed between 18
  and 50 percent more external test cases than code
  produced by control groups not using TDD.
• Another case study showed defect rate decrease of
  62 and 39 in two successive quarters after TDD
  approach was utilized.

20
Role of Software Quality Engineer

• Process Assurance (ISO9000, CMMI, etc.)
• Product Assurance
• Concept Phase (quality, plans review)
• Requirement Phase (review/analyze requirements)
• Design Phase (design review and metrics)
• Implementation Phase (change control)
• Test Phase (problem reports / witness and signoff
  on tests)
• Operations and Maintenance Phase (lessons learned)

21
Questions?