T-76.4115/5115 Software Development Project I/II Experience Exchange Session: QA Managers

1
T-76.4115/5115Software Development Project
I/IIExperience Exchange SessionQA Managers

• Jari Vanhanen
• Ohjelmistoliiketoiminnan ja tuotannon
  laboratorio
• Software Business and Engineering Institute
• (SoberIT)

2
Agenda

• Esittäytyminen
• nimi, projekti
• riittävä laatutaso kahdella lauseella
• Laatutavoitteiden asettaminen
• Laatukäytännöt laatutavoitteiden saavuttamiseksi
• Seuranta ja raportointi

3
Laatutavoitteiden asettaminen

• Projektin laatutavoitteet
• Osa projektin tavoitteita
• Asiakkaan näkökulma
• Asiakkaan suorat laatuun liittyvät vaatimukset
• Asiakas ei aina koe laatutavoitteita tärkeiksi
  tällaisessa projektissa
• Ryhmän täytyy silloin asettaa laatutavoitteet
• Yrittää selvittää asiakkaan implisiittiset
  odotukset laadun suhteen
• Asettaa itse sellaiset laatutavoitteet, jotka
  ovat tarpeen asiakkaan asettamien muiden
  tavoitteiden saavuttamiseksi
• Ryhmän näkökulma
• Projektin hallinnan ja sujuvan etenemisen
  varmistaminen
• Liian suurten yllätysten välttäminen
• Kurssin vaatimusten ja ryhmän arvosanatavoitteiden
  huomioiminen

4
Quality goals point out what is important

• What are the most important quality
  characteristics for this project and product
• Quality goals show where to focus
• With clearly stated goals it is easier to select
  good practices and methods
• Quality goals steer every activity of the project
  into the right direction
• Helps decision making
• Goals reflect the different qualities for
  different parts of the system

5
How to measure the quality goals

• Some quality characteristics can be measured
  quantitatively
• Hard metrics
• Set exact goals and measure
• e.g. defect counts, performance
• Some can be assessed subjectively
• Subjective evaluation
• Try to quantify, track progress regularly
• e.g. usability, conformance to a standard
• Some cannot be measured at all (in a practical
  situation)
• Indirect metrics
• Select practices or guidelines and track
  conformance
• e.g. maintainability, reusability

6
Discussion on quality goals

• What kinds of quality goals do you have in your
  projects?
• What problems and challenges you have encountered
  in quality goal definition?
• Questions, experiences, ideas?

7
Quality Practices
8
A Project Plan
9
A Project Plan
The Reality
10
What quality practices can do for my project?

• Provide headlights
• Quality information
• where are we
• and what lies ahead
• Enable changing our plan
• We can get almost there and stick to our schedule
• We can get there, but later than planned
• But if we run full steam ahead according to our
  plan we will definitely crash

11
Quality Practices

• Plan which practices are used to achieve and
  measure each of the quality goals
• Select existing practices in use
• Find good best practices from outside
• Invent new practices
• The goals give guidance for performing the
  practices
• Many practices strive for better quality
• Testing
• Reviews, inspections
• Conventions, guidelines
• Documenting
• Design and development methods techniques

12
Mapping the goals and practices
Practices Quality goals Quality goals Quality goals Quality goals Risks Risks
Practices Functional correctness Conform-ance to standard X Usability Change-ability DB changes break existing client sw One developer has no experience in J2EE
Functional testing X X
GUI prototyping X
Code reviews X

• Documents how quality goals are to be achieved
  and risks handled
• Focuses activities into right issues
• Reveals and communicates quality risks
• explicitly identified risks
• risks of quality goals with weak practices

13
Mapping the goals and practices
Practices Quality goals Quality goals Quality goals Quality goals Risks Risks
Practices Functional correctness Conform-ance to standard X Usability Change-ability DB changes break existing client sw One developer has no experience in J2EE
Functional testing X X
GUI prototyping X
Code reviews X X X
Regression testing client sw X X

• Can be used to emphasize different goals for
  different parts of the system
• Can be used to focus quality practices into right
  issues
• E.g. provides the exact goals of some review and
  testing practices

14
Who is responsible for quality?

• Make sure that quality is everybodys
  responsibility
• and it shows in the project plan
• It is good to have a project QA manager role
• Responsibility to actively track quality and help
  project manager to make QA activities to happen
• Reports projects progress from the quality point
  of view in the project meetings
• Tracking quality goals
• Tries to be less eager sacrificing quality to the
  new fancy features
• Does not mean personal responsibility to create
  the quality or to do all quality assurance
  activities

15
Recognize heartbeat and iteration quality
practices
Iteration
Heartbeat

• Heartbeat quality practices
• Quality practices are part of each development
  task
• In-synch quality practices that help building
  in quality during design and implementation
• E.g.
• Automated unit testing
• Continuous integration and nightly builds
• Peer reviews
• Continuous exploratory testing
• Iterative function testing
• Iteration quality practices
• Assuring the quality of each increment
• E.g.
• Systematic test rounds
• System tests for non-functional qualities
• Re-testing bug fixes and changes
• Formal documentation reviews and acceptance
• Acceptance testing

Release
t
Blue QA practices
16
Discussion on quality practices

• What kinds of quality practices you have planned
  in your projects?
• What problems and challenges you have encountered
  in defining the quality practices?
• Questions, experiences, ideas?

17
Test-Case Based and Exploratory Testing
Approaches

• Test case based approach
• Test cases are documented
• Inputs, data
• Execution steps
• Expected outputs
• Documented test cases can be reused later
• Perhaps automated
• Exploratory testing approach
• Exact test cases are not designed and documented
• Based on knowledge and skills of the tester
• Focuses on revealing defects
• Only higher level instructions are planned
• Can be structured by functionality, components,
  main GUI areas, scenarios, use cases, etc.
• Must be planned, tracked and reported

18
Plan how do you design your test cases

• On what levels do you test, and why?
• Unit, integration, system, acceptance
• What is the goal of your test cases?
• What your test cases are based on?
• Use cases
• Technical specification
• Requirements document, functional specification
• User interface functions
• Some other interface
• What is the level of detail in test case design?
• Cover all GUI components
• Set of test for each feature/function
• One test case per use case
• How do you document and track your test cases?
• Do you automate your tests?
• What tests, why?
• How?
• How much does it cost?

19
Discussion on testing and test case design

• What kind of testing have you planned in your
  projects?
• Questions, experiences, ideas?

20
Discussion on QA Reporting

• How to report the results of quality practices?
• How to report the quality status of the system?
• Questions, experiences, ideas?