November 2004

1
Popkin SoftwareSoftware Quality Assurance

• November 2004
• Mariya Shapran

2
Content Table

• Popkin Software
• who we are
• what we do
• Our Development Life Cycle
• what, when, why, who and where..
• Quality Assurance vs. Quality Control
• Quality assurance as part of the product
  development lifecycle
• Standards
• Test Life Cycle
• SA Metamodel
• IDEF0/Black Box/White Box Test Approaches
• Best practices
• Test Approaches
• Merging industry standards into our test
  methodology
• Defects Life Cycle
• Critical path
• Non-critical path
• Questions and Answers

3
Popkin Software

• The Enterprise Architecture Company

4
Popkin Software Overview

• The Enterprise Architecture Company
• Founded in 1986
• Leading Provider of Enterprise Architecture Tools
• Flagship Product is System Architect
• Vision
• We develop, globally market, sell, and support a
  diverse but integrated set of Enterprise
  Architecture, Business Analysis, and Systems
  Design tools.
• Our tools enable clients to visualize, analyze,
  implement, and communicate their Enterprise
  Architecture. This enables clients to maximize
  the efficiency of their organization and improve
  their strategic decision making capabilities.
• World Wide Presence
• Corporate Headquarters in New York City
• European Headquarters in United Kingdom
• Other Major Offices in Washington DC and The
  Netherlands
• Distributors throughout the World

5
Company Overview

• Products
• System Architect Flagship Product
• System Architect DoDAF
• iRMA
• SA Information Web Publisher
• SA Compare
• SA Simulator
• Training
• Consulting
• Users
• More than 80,000 users and 40,000 licenses at
  7,000 sites worldwide
• User groups in the US, UK, and Australia

6
Partial List of Clients

• IBM
• Lockheed Martin
• Publix
• Paychex
• McGraw Hill
• Scholastic
• CSC
• JPMorganChase
• Virgin Mobile
• DuPont Corporation
• Borders
• Vodafone
• JB Hunt
• Siemens

• British Airways
• Samsung
• Verizon
• ATT
• Glaxo-Smithkline
• MCI Worldcom
• Bell Canada
• Time Warner Telecom
• BT
• Ericsson
• Nokia
• T-Mobile
• Telia Sonera

• Department of Defense
• US Air Force
• Department of Treasury
• Department of Agriculture
• Veterans Administration
• GSA
• Department of Homeland Security
• Intelligence Agencies

7
System Architect
8
System Architect

• Leading Enterprise Architecture Tool on Market
• Integrated Support for Modeling Disciplines
• Business Modeling
• Latest Standards BPMN, BPEL4WS, etc
• Simulation of Process Models
• UML for Systems Modeling
• Synchronization with code such as Java, VB.NET,
  C, etc
• Relational Data Modeling
• Implement/Reverse Engineer DBMSs such as Oracle,
  DB2, and SQL Server
• XML Design
• Network Modeling
• Communicate Models by Publishing to Websites
• Repository based with Networking support

9
System Architect V10
Framework Manager
SA Guidebooks
Shared Definitions
New

• Analyze Business
• Enterprise Direction
• Processes (BPMN)
• Organization
• Functions
• Technology Infra
• IDEF0/IDEF3
• Simulation (IDEF3, BPMN, Process Charts)

• UML
• Design Applications
• Use Cases
• Object Interactions
• Classes
• Components
• State Machines
• Java
• VB .NET

• Design Databases
• ERD Model
• Physical Model
• IDEF1X
• DB Synchronize
• SQL Server
• Oracle 8, 9i

• Analyze Legacy Systems
• Gane/Sarson
• Ward/Mellor
• Yourdon/DeMarco
• SSADM

• Design XML Schemas
• DTDs
• BizTalk
• Instance Docs
• Test Data
• UML Integration
• Data Modeling Integration

XML Design
Structured Methods
Data Modeling
Business Modeling
New
Explorer Diagram
New
New
New
Repository
New
Customizable Repository Metamodel
SA Compare
Microsoft SQL Server
New
Reporting/Publishing
Browsing
Interfaces
Matrices
MS Office
HTML
XML
SVG
HTML
OLE Automation
CSV
XML
HTML
MS Office
XML
Improved
SA InfoWeb Pub
XML
New
10
System Architect V10
FunctionsUsers Encyclopedias NT Authorization
Role-BasedAccess Control
New
Catalog
Configuration Management
SA Compare
Backup
Reporting
Merge/Extract
11
Popkins View of Standards
12
Enterprise Architecture
HTML/Web
Information Publisher
Automated Reports on Information in Repository
XSLT
Enterprise view of data
SA Repository
16 years of enterprise architecture and modeling
ICAMS
System Architect
Multiple Method View of Information
Industry Standards BPMN XML Schema Entity /
Relation UML Frameworks C4ISR DoDAF
Methods
E/R Data
UML
BPM
Frameworks
13
Evolution of System Architect
Business Process Modeling (IDEF)
First Windows-Based Modeling Tool Introduced
C4ISR Framework Support
Structured Analysis and Design
TOGAF Support
UML Support
88 96 97 98
99 00 01 02 03
Object- oriented
Data Modeling
Simulation, XML Support
SQL Server As Underlying Repository
Zachman Framework Support
Multiple Methods
14
Product Development Life Cycle
15
Quality Assurance vs. Quality Control

• Quality Assurance
• Quality Control
• Whats the difference?

16
Quality Assurance

• System Development Methodologies
• Estimation Processes
• System Maintenance Processes
• Requirements Definition Processes
• Testing Processes and Standards
• Measurement Programs to evaluate processes
• Processes gap analysis
• Concerned with all of the products that will ever
  will be produced by a process

17
Quality Control

• Relates and concern with a specific product or
  service
• Verifies whether specific attributes are in, or
  are not in a specific product or service
• Identifies defects for the primary purpose of
  correcting defects

18
Test Life Cycle
19
Quality Factors
Correctness Extend to which a program satisfies its specifications and fulfills the users mission objectives
Reliability Extend to which a program can be expected to perform its intended function with required precision
Efficiency The amount of computing resources and code required by a program to perform a function
Integrity Extend to which access to software or data by unauthorized persons can be controlled
Usability Effort required to learn, operate, prepare input, and interpret output of a program
Maintainability Effort required to locate and fix an error in an operational program
Testability Effort required to test a program to ensure that it performs its intended function
Flexibility Effort required to modify an operation program
Reusability Extend to which program can be used in other applications related to the packaging and scope of the functions that programs perform
Interoperability Effort required to couple one system with another
20
The System Architect Metamodel
21
IDEF0 Integration Definition for Function
Modeling
22
Black Box Test Approach
23
White Box Test Approach
24
Test Approach

• Black Box Approach
• Boundary Value Analysis (internal code)
• Regression Testing (Test Partner and Loadgen)
• Data Integrity Testing (Internal Tools)
• Stress (Volume) Testing (Loadgen)
• Backup and Recoverability (Internal Tools)
• Functional Testing (Test Partner and Manual
  Testing)
• Security Testing
• Configuration/Compatibility Testing (VMWare
  images)
• User Acceptance Testing (End Users)
• Benchmark Testing (Loadgen)
• Operational Readiness Testing (Loadgen and Test
  Partner)
• Usability Testing (End Users/508 Compliance
  Guidelines)
• Alpha/Beta Testing (Loadgen)
• White Box Approach
• Memory Usage (BounceChecker)
• General Resource Usage
• Error Handling

25
Boundary Value Analysis Example

• Problem
• Need to understand what will take place at the
  boundaries of the behavior for each component.
• Example
• Field Name for the Diagram is required to accept
  unique name for the diagram consistent of
  alpha-numeric characters up to 105 characters
  (with exception for special characters)
• Boundary Value Analysis
• Absence of the name rejected
• Special Characters rejected (this is on the
  boundary)
• null rejected
• 2alpha5 numeric characters accepted
• All alpha characters accepted
• All numeric characters accepted
• 106 characters rejected

26
Best Practices

• Peer Reviews
• Defined Requirements Standards
• Defined Quality Standards
• Constant improvements to Quality Processes
• Implementation of Automation Testing into
  Regression and Performance Test Phases

27
Merging Industry Standards into our Test
Methodology

• 508 Compliance
• 508 Compliance Checklist
• Methodologies IDEF0, IDEF1, IDEF3
• IDEF0/IDEF1/IDEF3 Compliance Checklist
• IEEE Standards
• Used as guideline in defining of Quality
  Assurance Processes
• ISO Standards
• Used as guideline in defining of Quality
  Assurance Processes

28
508 Compliance Check List

• Menu items have a unique, functioning access key.
• Shortcut or popup menu items have a unique,
  functioning access key.
• Dialog controls have a unique, functioning access
  key.
• Dialogs can be closed by pressing the ESC key.
• Shortcut or popup menus can be opened via the
  keyboard. Most keyboards have what's called an
  application key. Pressing this key displays the
  shortcut menu for the selected object.
• Drag and drop functionality can also be executed
  via the keyboard.
• Dialogs have a logical and consistent tab order
  (left to right and top to bottom).
• Color alone should not be used to indicate
  different states. For example, in access control
  red and green colored arrows were used to
  represent checked in and checked out items.
• The tabs on a tabbed control can be navigated by
  pressing CTRLTAB.
• Tree controls can be navigated by pressing the
  arrow keys.
• Interfaces containing multiple windows and/or
  frames allow for focus to be shifted via the
  keyboard.
• Focus is always placed on the most recently
  opened or accessed window.
• The applications active window should display a
  visual focus indicator at all times so that users
  can anticipate the effects of their keystrokes.
• The application can read, use, and preserve
  system-wide user interface (UI) settings when
  displaying customized controls or window content.
  The system-wide settings adopted by a user
  enhance the products accessibility the
  application should use them where possible, and
  not disable or disregard them. An example of a
  system-wide setting is the ToolTip size.
• The application does not convey information
  exclusively by sound. If sound is the default
  method for conveying information, the application
  should provide other options to express this
  information.

29
IEEE Software Engineering Standards
610.12.1990 IEEE Standard Glossary of Software Engineering Terminology
730.1998 IEEE Standard for Software Quality Assurance Plans
730.1 1995 IEEE Guide for Software Quality Assurance Plans (ANSI)
828 1998 IEEE Standard for Software Configuration Management Plans
829 1998 IEEE Standard for Software Test Documentation
830 1998 IEEE Recommended Practice for Software Requirements Specifications
1008 1987 (R1993) IEEE Standard for Software Unit Testing (ANSI)
1012 1998 IEEE Standard for Software Verification and Validation
1012a - 1998 IEEE Standard for Software Verification and Validation Supplement to 1012-1998. Content Map to IEEE 12207.1
1028 1997 IEEE Standard for Software Reviews
1058 1998 IEEE Standard for Software Project Management Plans
1058.1 1987 (R1993) IEEE Standard for Software Project Management Plans (ANSI)
30
Defect Life Cycle
31
Glossary

• Organizations
• ARTS Retail Tech Standards with Industry Data
  Model
• www.nrf-arts.org
• BPMI Business Process Management Standards body
• www.bpmi.org
• DAMA Data Management Association
• www.dama.org
• DSDM Dynamic Systems Development Method
• www.dsdm.org
• NASCIO National Association of State Chief
  Information Officers
• www.nascio.org
• OMG Object and XMI Tech Standards Body
• www.omg.org
• Supply Chain Management standards with SCM data
  model
• www.supply-chain.org
• Telecoms Tech Standards with Industry Process
  Mode
• www.tmforum.org
• The Open Group
• www.opengroup.org

32
Glossary

• FEA
• Federal Enterprise Architecture
• BRM - Business Reference Model
• SRM - System Component Reference Model
• PRM - Performance Reference Model
• DRM - Data Reference Model
• TRM - Technical Reference Model
• BPMN
• Business Process Modeling Notation
• IDEF
• Integrated Definition Language, a systems
  modeling technique using a specific graphical
  structure
• ORM
• Object Role Modeling
• E/R
• Entity Relation
• BPEL4WS
• Business Process Execution Language for Web
  Services
• UML
• Unified Modeling Language, an object-oriented
  design language

33

POPKIN SOFTWARE
Popkin Software 2 Rector Street New York, NY
10006 Tel [phone] Fax [phone]