Imagination is more important than knowledge

1
Imagination is more important than knowledge

• Software testing in a complex, networked world
• Kasper Hanselman

2
Who am I?

• My views
• My experience

3
Why this talk?

• Why Software Fails
• 1 . Unrealistic or unarticulated project goals
• 2 . Inaccurate estimates of needed resources
• 3 . Badly defined system requirements
• 4 . Poor reporting of the project's status
• 5 . Unmanaged risks
• 6 . Poor communication among customers,
  developers, and users
• 7 . Use of immature technology
• 8 . Inability to handle the project's complexity
• 9 . Sloppy development practices
• 10. Poor project management

• What we test
• Functionality
• Performance
• Maintainability
• (Security?)?

(Robert N. Charette)?
4
What is it all about?

• Leading testing methodologies focus on structured
  functional testing
• Testers perform their craft as if they are
  testing stand alone or client-server software
  against clear and unambiguous requirements
• The world has changed software testing needs to
  adapt

5
The world today

• Software is everywhere
• This trend will continue
• IT services become more integrated and are made
  available in networked environments
• Users, both end-users and administrators, become
  more creative in linking applications together
• The speed in which our environment changes is
  increasing, it seems at an exponential rate

6
Testing today

• Mostly focused on functionality and performance
• Often set apart from development and software
  management
• Limited technical skills of testers
• Security and networking are not systematically
  tested
• No overall testing and quality approach

7
Problems with this

• Software projects fail
• Unauthorized use of systems
• Malware, viruses, trojans
• Spam
• Cross site scripting
• Issues with non web applications

8

• We Are Not Alone!

9
Testing

• Testing is the process of establishing confidence
  that a program or system does what it is supposed
  to do (Hetzel, 1983)?
• Testing is a process of planning, preparing,
  executing and analysing, aimed at establishing
  the characteristics of an information system, and
  demonstrating the difference between the actual
  status and the required status (Pol, Teunissen,
  van Veenendaal, 1995)?
• Testing is demonstrating that a system is fit for
  purpose (Evans, Mills, Warden, 1996)?

10
Testing a definition

• Software testing is the process of establishing
  whether an application or system of connected
  software services performs the purpose for which
  it is designed against agreed upon quality
  attributes and in its context and its technical
  environment

11
Testers

• We live in a complex networked world, software
  testing needs to adapt
• Testers need to keep up with technical changes
• Testers need to specialize

12
Be creative!

• Become an expert in the field or domain you are
  working in
• Use methods, don't be confined by them
• Look at software from a different perspective

13
Specialize

• Usability testing is not functionality testing is
  not security testing
• It can not use the same test techniques
• We need to specialize

14
Test Management

• Focus on the process
• Be the spokesperson for your testers
• Use the same tools and methods as the project
  manager
• Know your testers

15
Know your clients

• What is the core business
• What is the business purpose of the software
• What quality is needed

16
What to test?

• Test to make sure the software will not fail
• Factor in the business context
• Be flexible, investigate issues and adapt your
  testing strategy as needed

17
Planning and controlling test efforts

• Use the same terminology as the project manager
• Use language project managers, business managers
  and end users understand
• Use risk based testing
• Use Time boxing
• Cope with changes in a structured way

18
Deliver the goods!

• The product is a solution. If the problem isn't
  solved, the product doesn't work. (The Seven
  Basic Principles of context driven testing)?
• Keep this in mind at all times. You are
  responsible for reporting whether the software
  works.You are not responsible for making it
  work.

19
The Future

• Questions / Discussion