A%20Embedded%20software%20component%20quality%20framework

1
A Embedded software component quality framework

• Fernando Ferreira de Carvalho
• Advisor Silvio Romero de Lemos Meira
• Informatics Center - Federal University of
  Pernambuco
• C.E.S.A.R. - Recife Center for Advanced Studies
  and Systems
• ffc_at_cin.ufpe.br
• 12-Junho-2008

2
Introduction / Motivation
A Embedded software component quality framework

• Embedded system is at the heart of many systems
• So, embedded system industry needs,
• Low production cost
• Short time to market
• High Quality
• to be more efficient and competitive (Brown,
  2000)
• The CBD with reuse technique had been a nice
  direction to reach this objectives
• But, Component reuse without quality assurance
  give catastrophic results ariane.

http//www.rise.com.br Fernando Carvalho, PhD
Candidate
3
Introduction / Motivation
A Embedded software component quality framework

• The CBD technologies do not take into account
  the specifics needs of embedded-systems
  development Timing, Memory, power, hardware
  constraints and others.

SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
4
Introduction /
A Embedded software component quality framework

• Solution
• Certification is the future of software
  components Wallnau, Heineman, Councill, Shaw
• According to Weber et al. (Weber et al., 2002),
  the need for quality assurance in software
  development has exponentially increased in the
  past few years

SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
5
Introduction / Motivation
A Embedded software component quality framework

• Benefits
• More reliability
• Short time-to-market
• More product quality
• higher quality levels,
• reduced maintenance time,
• investment return,
• reduced time-to-market, among others
• Wide used, where certification is mandatory
  (aircraft)

SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
6
Rise Framework
A Embedded software component quality framework
SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
7
Component Certification Process
A Embedded software component quality framework
SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
8
Component Certification Processfocused in
embedded systems
A Embedded software component quality framework
An Embedded Computer System A computer system
that is part of a larger system and performs some
of the requirements of that system for example,
a computer system used in an aircraft or rapid
transit system. (IEEE,1992).
SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
9
Component Certification Processfocused in
embedded systems
A Embedded software component quality framework

• Embedded systems
• Used for specifics propose
• Used to control systems (ex mechanical machines)
• Ultra small devices with simple specific
  functionality
• Small systems with sophisticated functions
• Produced in large scale

SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
10
Component Certification Processfocused in
embedded systems
A Embedded software component quality framework
Embedded systems has a specific requirements -
real-time - hi reliability - low power
consumption - low data and code memory - low
resources - low CPU capabilities - others
SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
11
Problem Formulation
A Embedded software component quality framework
The CBD is an efficient and effective way for
design of simple and complex embedded systems.
However, quality assurance of components is must
be done to take advantage of CBD.
The Software industry still far to reach the
maturity level the hardware industry which it
have catalogues and datasheets available for its
components.
For this reason, a well-defined and consistent
embedded software component quality assurance is
essential for CBD and reuse adoption.
SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
12
Proposed solution
A Embedded software component quality framework
An Embedded Software Component Quality
Verification Framework
It is composed of four inter-relation module,
based on a set of activities, metrics and
guidelines.

• Embedded software component Quality Model (EQM)
• Maturity Level evaluation Techniques
• Metrics Approach
• Component Certification

based on a set of activities, metrics and
guidelines.
SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
13
Proposed solution
A Embedded software component quality framework
This Framework is based in the standards

• ISO/IEC 9126, 2001 - Quality Model for Software
  Product
• ISO/IEC 14598, 1998 - Software Product Evaluation
  Process

This two standards converged to

• ISO/IEC 25010, 2005 - Software product quality
  - requirements and evaluation

the Framework adapted the quality model and
evaluation to component context and embedded
domain.
SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
14
Out of scope
A Embedded software component quality framework
This Framework is part of broad context, some
aspects were expected since initial definition.
Nevertheless, other process can be added in the
future.

• Cost Model
• Formal Proof
• Prediction of the component assembly

SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
15
2 Embedded System Design
A Embedded software component quality framework
Embedded system design comprise

• Ultra-small device x simple functionality
• Small system x sophisticated functions
• Large systems and distributed systems
• Systems produced in large quantities x low
  production cost
• Systems produced in low volume x important
  features

SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
16
2 Embedded System Design
A Embedded software component quality framework
The different requirements of embedded systems
have a impact on feasibility, on use of CBD for
it.
A common characteristic in different area of
embedded domain is increasing importance of
software Crnkovic, 2003.

• Example, the software cost in embedded systems
• in industrial robots constitute about 75 of
  total cots
• in car industry it is about 30
• Fifteen year ago
• 25 of total cots in industrial robots
• Negligible for cars

SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
17
2 Embedded System Design
A Embedded software component quality framework
Properties that involves embedded software
component is divided in

• Functional property (component interface)
• Non-functional or Extra-functional property, so
  called Quality attributes, fox example
• Timing
• Performance
• Consumption
• Resource Behavior, and others.

This properties can be classified in run-time and
life-cycle.
SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
18
2 Specific Requirements for Embedded System
A Embedded software component quality framework
In the most of case, embedded system is real-time
with limited resource. So, it has specifics
characteristics which depends on domain
application, but it have strong implication on
requirements.
The REQUIREMENTS are related Extra-functional
property or Quality attributes, and its priority
depends on the domain application.

SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
19
2 Specific Requirements for Embedded System
A Embedded software component quality framework
There has been developed a research in order to
find the most important characteristics in
different areas in embedded domain.

• Industrial Automation
• Automotive
• Medical
• Electronic consumer
• Other domain

SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
20
2 Specific Requirements for Embedded System
Industrial Automation
A Embedded software component quality framework
Industrial Automation was classified by
researchs Larsson, Larsson, 2002
The most important characteristics, following the
research

• At low level
• Availability
• Timeliness
• Reliability

• At high level
• Performance
• Usability
• Integrability

SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
21
2 Specific Requirements for Embedded System -
Automotive
A Embedded software component quality framework
Akerholm Akerholm, 2005 executed a research in
vehicle industry.
The resulting list of characteristics is
presented below

1. Safety
2. Reliability
3. Predictability
4. Usability
5. Extendibility
6. Maintainability
7. Efficiency
8. Testability
9. Security
10. Flexibility

SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
22
2 Specific Requirements for Embedded System -
Medical
A Embedded software component quality framework
Wijnstra Wijnstra, 2001 describe their
experience with characteristics in the
development of medical imaging family.
The resulting list of characteristics is
presented below

• Reliability
• Safety
• Functionality
• Portability
• Modifiability
• Configurability
• Extensibility and Evolvability
• Security
• Serviceability

SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
23
2 Specific Requirements for Embedded System
Others Domain
A Embedded software component quality framework
Characteristics Sub-characteristics
Real-time properties Response time or latency
execution time
worst case execution time
Deadline
Dependability Reliability
Availability
integrity
confidentiality
safety
Resource consumption Power consumption
computation (CPU) power
memory Consumption
execution (CPU) time,
Life cycle properties maintainability
Crnkovic Crnkovic, 2003 summarized the main
characteristics and sub-characteristics in the
CBD apply embedded system.
The table show the results.
SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
24
A Embedded software component quality framework
Component Certification Processfocused in
embedded systems
But the specifics requirements depends on the
application (domain) - Industrial Automation -
Automotive - Medical - Consumer electronics -
Aircrafts - others
SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
25
A Embedded software component quality framework
Component Certification Processfocused in
embedded systems

• So, embedded component certification process must
  be different that general propose component,
  because the component evaluation is realized
  focused in other requirements
• We divided in two groups
• General propose sw component certification
  process
• desktops, servers, x86 architecture
• Specific propose sw component certification
  process (embedded systems, others architectures)

SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
26
A Embedded software component quality framework
Component Certification Processfocused in
embedded systems (requiriments)

• Industrial Automation (Crnkovic et al., 2005)
• In the lowest levels
• availability,
• timeliness, and
• reliability
• At higher levels
• performance,
• usability, and
• integrability

SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
27
A Embedded software component quality framework
Component Certification Processfocused in
embedded systems (requirements)

• Automotive (Åkerholm e.t. al, 2004)
• Extendibility
• Maintainability
• Usability
• Predictability
• Security
• Safety
• Reliability
• Testability
• Flexibility
• Efficiency

SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
28
A Embedded software component quality framework
Component Certification Processfocused in
embedded systems (requirements)

• Medical (Wijnstra e.t. al,2001)
• Reliability
• Safety
• Functionality
• Portability
• Modifiability
• Configurability
• Extensibility and Evolvability
• Testability
• Serviceability

SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
29
Embedded Software Component Quality Process
A Embedded software component quality framework

• Certification is the future of software
  components Wallnau, Heineman, Councill, Shaw
• RiSE Approach
• Embedded Component Quality Model (ECQM)
• Embedded Software Component Maturity Model
  (ESCMM)
• Certification Techniques Framework
• Defining techniques for evaluate quality
  attributes of ECQM
• Metrics Framework
• Track the properties of the ECQM, certification
  techniques and process.
• Embedded Software Component Certification Process
• Defining the steps for certify a component

SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
30
Embedded Component Certification Process
A Embedded software component quality framework
Changes in the Proposed Component Quality Model,
in relation to ISO/IEC 25010

• Embedded Component Quality Model (ECQM)
• Embedded Software Component Maturity Model
  (ESCMM)
• Based ISO/IEC 25010 () standard, with some news
  and extended characteristics.

Sub-Characteristics
Characteristics
Suitability Accuracy Interoperability Security Com
pliance Self-contained
Functionality
Maturity Recoverability Fault Tolerance
Reliability
Understandability Configurability Learnability Ope
rability
Usability
Time Behavior Resource behavior Scalability
Efficiency
Analyzability Stability Changeability Testability
Maintainability
Deployability Replaceability Adaptability Reusabil
ity
Portability
ISO/IEC 9126 - Software Products Quality
ISO/IEC 14598 - Evaluation software product
Development time Cost Time to market Targeted
market Affordability Licensing
Marketability
SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
31
Embedded Component Certification Process
A Embedded software component quality framework
Sub-Characteristics
Characteristics

1. Embedded Component Quality Model (ECQM)

Suitability Accuracy Interoperability Security Com
pliance Self-contained
Functionality
Changes in the Proposed Component Qualit Model,
in relation to ISO/IEC 25010
Maturity Recoverability Fault Tolerance
Reliability
Understandability Configurability Learnability Ope
rability
Usability
Time Behavior Resource behavior Scalability
Efficiency
Analyzability Stability Changeability Testability
Maintainability
Deployability Replaceability Adaptability Reusabil
ity
Portability
Development time Cost Time to market Targeted
market Affordability Licensing
Marketability
SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
32
A Embedded software component quality framework

• 2. Certification Techniques Framework
• Defining techniques for evaluate quality
  attributes of ECQM

Guidelines for selecting evaluation level.
Economic
Safety/Security
Environment
Level
Negligible economic loss
Few material damage No specific risk
No damage
ESCMM I
Few economic loss
Few people disabled
Small/Medium damage properly
ESCMM II
Significant economic loss
Large number of people disabled
Damage properly
ESCMM III
Large economic gross
Threat to human lives
Recoverable environment damage
ESCMM IV
Financial disaster
Many people killed
Unrecoverable environmental damage
ESCMM V
SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
33
A Embedded software component quality framework

• 2. Certification Techniques Framework
• Defining techniques for evaluate quality
  attributes of ECQM

Guidelines for selecting evaluation level.
ESCMM II
ESCMM III
ESCMM V
ESCMM IV
ESCMM I
Level
Programming Language Facilities
(Best Practices) Maturity analysis
Fault tolerance analysis Error
Manipulation analysis
Formal Proof
Reliability growth model
Suitability analysis
Reliability
SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
34
A Embedded software component quality framework

• 3. Metrics Framework
• Track the properties of the ECQM, certification
  techniques and process.

Functionality
Accuracy
Sub-Characteristic
Correctness
Quality Attribute
Evaluates the percentage of the results that were
obtained with precision
Goal
Based on the amount of tests executed, how much
test results return with precision?
Question
Precision on results / Amount of tests
Metric
0 lt x lt 1 which closer to 1 is better
Interpretation
SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
35
A Embedded software component quality framework

• 4. Embedded Software Component Certification
  Process
• Defining the steps for certify a component
• 4.1 Establish Evaluation Requirements activity
• 4.2 Specify the Evaluation activity
• 4.3 Design the Evaluation activity
• 4.4 Execute the Evaluation activity

SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
36
4. Embedded Software Component Certification
Process
A Embedded software component quality framework
SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
37
A Embedded software component quality framework
4.1 Establish Evaluation Requirements activity
SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
38
A Embedded software component quality framework

• 4.2 Specify the Evaluation activity

SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
39
A Embedded software component quality framework

• 4.3 Design the Evaluation activity

SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
40
A Embedded software component quality framework

• 4.4 Execute the Evaluation activity

SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
41
A Embedded software component quality framework
Submissão de artigo dia 05/06/2008 - SBCARS
SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
42
A Embedded software component quality framework
SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
43
Thank you !
A Embedded software component quality framework
Questions ?

• Fernando F. de Carvalho
• ffc_at_cin.ufpe.br

SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate
44
Embedded Component Certification Timeline
A Embedded software component quality framework
SAAP 2008.1
http//www.rise.com.br Fernando Carvalho, PhD
Candidate