Productline architecture: New issues for evaluation

1
Product-line architecture New issues for
evaluation

• Leire Etxeberria Elorza
• University of Mondragon

2
Table of contents

• Software Architecture Evaluation
• Software product lines
• Product-Line Architecture Evaluation New
  Challenges
• Two different architecture types to evaluate
• Different strategies
• What to evaluate in each level?
• New moments to evaluate
• Architecture evaluation methods
• Conclusions
• Future research
• References

3
Software Architecture Evaluation
Requirements
Functional requirements
Quality attributes
Operational attributes performance, security,
availability, usability
Development attributes modifiability,
portability, reusability, integrability,
testability
A great influence
Software Architecture
Analyse or evaluate the potential of a software
architecture to reach the required quality
levels. Why? To find the problems early in the
life cycle, when they are easier and less
expensive to correct
4
Software Product lines

• Software Product Line (SPL) is a set of
  software-intensive systems that share a common,
  managed set of features satisfying the specific
  needs of a particular market segment or mission
  and that are developed from a common set of core
  assets in a prescribed way Clements and
  Northrop, 2002
• The key core asset is the Product Line
  Architecture (PLA) since it is the generic
  structure that captures all the communality and
  the variability of the line. Product
  architectures are derived from this architecture.

Product-line Architecture (PLA)
Product Arch 1
Product Arch 2
5
PLA evaluation New challenges

• Research Question How change architecture
  evaluation in a product-line context?
• For evaluating architectures in the context of a
  product line new challenges and aspects arise
• Two level of architectural abstraction or two
  architecture types and different requirements to
  evaluate in each one
• It is vital to assure that product line
  architecture offers the required variability and
  extensibility to cover all the products of the
  envisioned scope as well as support new
  characteristics and products in the future
• New evaluation moments arise before designing
  the PLA, during derivation
• More stakeholders are involved and they may be
  far away from each other

6
Two architecture types to evaluate
Ensure that the PLA is flexible enough to support
different products and allow evolution.
Product-line Architecture
Evolution
Two level of architectural abstraction

Product arch 1
Product arch 2
Ensure that each product architecture fulfils the
required quality attributes
7
What to evaluate in each level?
Product Line Architecture

• Product Line quality attributes
• Domain-relevant quality attributes
• Common behaviour

Instantiation

• Instanted quality attributes
• Specific behaviour

Concrete architectures of products
8
What to evaluate in each level?
Product Line Architecture

• Product Line quality attributes To be the basis
  for a set of related products as well as future
  new products.
• Domain-relevant quality attributes
• Common behaviour

Modifiability (Variation over time)
Extensibility, Portability, Scalability...
Variability or Flexibility
Configurability (Variation over space)
Reusability, composability, interoperability...
Instantiation
Concrete architectures of products
9
What to evaluate in each level?
Product Line Architecture

• Product Line quality attributes
• Domain-relevant quality attributes Important
  quality attributes for the specific domain
• Common behaviour

Performance, Safety, Security, Reliability,
Availability, Usability ...
Instantiation
Concrete architectures of products
10
What to evaluate in each level?
Product Line Architecture

• Product Line quality attributes
• Domain-relevant quality attributes
• Common behaviour the behaviour viewed as common
  across product line members

Instantiation
Concrete architectures of products
11
What to evaluate in each level?
Product Line Architecture

• Product Line quality attributes
• Domain-relevant quality attributes
• Common behaviour

Instantiation
Instated Variability

• Instated quality attributes
• Specific behaviour

Concrete architectures of products
12
Different strategies

• To evaluate all the products is not
  cost-effective
• The product architecture evaluation is a
  variation of the product line architecture
  evaluation, just as the product architecture is a
  variation of the product line architecture
  Clements and Northrop, 2002
• Product architecture evaluations can be shortened
  because many issues have been previously dealt
  with in the evaluation of the product line
  architecture.
• There are different strategies to shorten the
  product evaluations
• Evaluate only a set of products
• A reference product
• Extreme products
• Automate the evaluation
• Reuse evaluation assets (checklists, scenarios)

13
Evaluation techniques

• For quality evaluation
• Questioning techniques
• Scenarios, questionnaires, checklists
• Measuring techniques
• Simulations, prototypes, experiments,
  mathematical models (metrics, RMA)
• For Functional requirement evaluation
• Model Checking
• Theorem Proving
• In some cases, these techniques have to be
  adapted to use them in a product-line context

14
New evaluation moments
Product Line Architecture
Assures reference architecture compliance to
product-line quality attributes, domain relevant
qualities and common behaviour.

PLA Evaluation
Variability
DERIVATION
PA Evaluation
Assures instantiated quality attributes and
product specific behaviour.
PA Evaluation
-
PA Evaluation
Concrete architectures
15
New evaluation moments
Analyse or compare existing product architectures
to use them as a basis for the product line.
Existing PA Evaluation
Product Line Architecture

PLA Evaluation
Variability
DERIVATION
PA Evaluation
PA Evaluation
-
PA Evaluation
Concrete architectures
16
New evaluation moments
Existing PA Evaluation
Product Line Architecture

PLA Evaluation
Variability
Assess the impact of variants on quality
attributes and help to select between variants or
variant versions
Evaluation during Derivation
DERIVATION
PA Evaluation
PA Evaluation
-
PA Evaluation
Concrete architectures
17
New evaluation moments

• Determine the ability of the product line to
  support a new product, a new requirement
• Assure that the architecture continue meeting
  its quality goals
• Architectural conformance to determine if an
  existing product can be part of the product line

Existing PA Evaluation
Product Line Architecture
Evolution related PLA evaluation

PLA Evaluation
t
Variability
Evaluation during Derivation
DERIVATION

• Assess the conformance of the instated
  architecture to the reference architecture
• Assure that the architecture continue meeting
  its quality goals

Evolution related PA Evaluation
PA Evaluation
Evolution related PA Evaluation
PA Evaluation
-
Evolution related PA Evaluation
PA Evaluation
Concrete architectures
t
18
Architecture evaluation methods

• For evaluating existing product architectures
• Pinzgers architectural recovery (Pinzger et al,
  2004)
• MAP (Mining Architectures for Product lines)
  (Stoermer and OBrien, 2001)
• SACAM (Software Architecture Comparison Analysis
  Method) (Stoermer et al, 2003)
• Korhonens approach (Korhonen and Mikkonen, 2001)
• For evaluating product line architectures
• FAAM (Family Architecture Assessment Method)
  (Dolan, 2002)
• AQA (Architecture Quality Analysis) (Dobrica and
  Niemela, 2000)
• REDA (Reliability Evaluation of Domain
  Architectures) (Auerswald et al, 2001)
• D-SAAM (Distributed SAAM) (Graaf et al, 2005)
• Gannods approach (Gannod and Lutz, 2000)
• Maccaris approach (Maccari, 2002)
• Rivas approach (Riva and Del Rosso, 2003)
• SBA (Scenario-Based Architecting) (America et al,
  2004)
• Wijnstras approach (Wijnstra, 2004)
• COSVAM (The COVAMOF Software Variability
  Assessment Method) (Deelstra et al, 2004)

19
Architecture evaluation methods

• For evaluating derived product architectures
• TPA (Timing Property Assessment) (Alonso et al,
  1998)
• Zhangs method (Zhang et al, 2003)
• Use service utilization metrics (Van der Hoek et
  al, 2001)
• Rahmans metrics (Rahman, 2004)

Single system architecture evaluation methods
SAAM, ATAM
20
Conclusions

• Product line architecture evaluation becomes more
  important than single system architecture
  evaluation because an error in the PLA can be
  spread to a lot of products
• All the methods focus on a specific type of
  architecture existing product architecture,
  product line architecture or derived product
  architecture, and they do not take advantage of
  the relation between these evaluations.
• There are few methods that reuse assets (to
  evaluate derived architectures)
• Single-product architecture evaluation is quite a
  mature field so the methods and techniques used
  in single product architecture evaluation should
  be adapted to use them into product-line context

21
Future research

• Some ideas
• New strategies and techniques to shorten product
  evaluations
• To develop a framework to encourage reuse among
  evaluations
• Reuse the knowledge got during evaluation, during
  others activities (evolution, derivation).
• Open issues
• Adaptation of techniques
• Measuring quality
• Organizational aspects

22
Future Research Encourage reuse

• An open framework for evaluations in a
  product-line context
• Interconnect evaluations
• Encourage the reuse of evaluation assets among
  different evaluations
• Provide feedback between evaluation types. This
  help to get a complete vision of the quality of
  the line for instance, the feedback from product
  evaluations give information about the PLA, if it
  is sufficiently flexible
• An open framework where different techniques and
  methods can fit in.
• Benefits
• Make easier the evaluation process through
  reusing
• Increase the evaluation quality
• Provide a global view of the product line quality

23
Future Research Encourage reuse
Architectural Evaluation Framework for
Product-Lines (AEF)
Existing PA Evaluation
Product Line Architecture
Reuse
Evolution related PLA evaluation
PLA Evaluation

t
Evaluation during Derivation
Variability
DERIVATION
Feedback
Evolution related PA Evaluation
PA Evaluation
Evolution related PA Evaluation
PA Evaluation
-
Evolution related PA Evaluation
PA Evaluation
Concrete architectures
t
24
Future Research Reuse knowledge

• All the information and knowledge obtained in
  evaluations can be gathered and reused during
  other activities such as derivation or evolution
• During derivation, variants are selected and
  these variants may influence quality attributes,
  so information about how they influence qualities
  can be very useful.

Product Line Architecture

PLA Evaluation
Results
Variability
Get Knowledge
DERIVATION
PA Evaluation
Results
PA Evaluation
-
PA Evaluation
Concrete architectures
25
References

• Alejandro Alonso, Marisol García-Valls, Juan A.
  de la Puente, Assessment of Timing Properties of
  Family Products, In Frank van der Linden (Ed.)
  Development and Evolution of Software
  Architectures for Product Families, Second
  International ESPRIT ARES Workshop, Las Palmas de
  Gran Canaria, Spain, 1998, Proceedings. Lecture
  Notes in Computer Science 1429 Springer 1998
• Pierre America, Dieter Hammer, Mugurel T. Ionita,
  Henk Obbink and Eelco Rommes, Scenario-Based
  Decision Making for Architectural Variability in
  Product Families, In Robert L. Nord, editor,
  Software Product lines, Third International
  Conference, SPLC 3, Boston, USA, 2004,
  Proceeedings, volume 3154 of Lecture Notes in
  Computer Science, Springer, 2004
• Marko Auerswald, Martin Herrmann, Stefan
  Kowalewski, Vincent Schulte-Coerne,
  Reliability-Oriented Product Line Engineering of
  Embedded Systems, In Frank van der Linden (Ed.)
  Software Product-Family Engineering, 4th
  International Workshop, PFE 2001, Bilbao, Spain,
  2001, Revised Papers. Lecture Notes in Computer
  Science 2290, Springer 2002, pp 83-100
• Paul Clements, Linda Northrop, Software Product
  Lines Practices and Patterns, Addison Wesley,
  2002
• Sybren Deelstra, Jos Nijhuis, Jan Bosch, Marco
  Sinnema, The COVAMOF Software Variability
  Assessment Method (COSVAM), 2nd Groningen
  Workshop on Software Variability Management, 2004
• Liliana Dobrica, Elia Niemelä, A strategy for
  analyzing product line architectures, VTT
  Publications, 2000
• Thomas J. Dolan, Ph.D. Thesis, Architecture
  Assessment of Information-Systems Families,
  Department of Technology Management, Eindhoven
  University of Technology, February 2002
• Gerald C. Gannod, Robyn R. Lutz, An Approach to
  Architectural Analysis of Product Lines, ICSE
  2000, Proceedings of the 22nd International
  Conference on Software Engineering, June 4-11,
  2000, Limerick Ireland. ACM, 2000, pp 548 557
• Bas Graaf, Hylke van Kijk, Arie van Deursen,
  Evaluating an Embedded Software Reference
  Architecture Industrial Experience Report-
  Accepted for CSMR 2005
• Mika Korhonen, Tommi Mikkonen, Assessing Systems
  Adaptability to a Product Family, In Proceedings
  of the International Conference on Software
  Engineering Research and Practice (SERP03), pp
  135-141, Las Vegas, 2003
• Alessandro Maccari, Experiences in assessing
  product family software architecture for
  evolution, Proceedings of the 22nd International
  Conference on Software Engineering, ICSE 2002,
  Orlando, Florida, USA. ACM 2002, pp 585-592
• Martin Pinzger, Harald Gall, Jean-Francois
  Girard, Jens Knodel, Claudio Riva, Wim Pasman,
  Chris Broerse, Jan Gerben Wijnstra, Architectural
  Recovery for Product Families, In Frank van
  Linden, editor, Software Product-Family
  Engineering, 5th International Workshop, PFE 5,
  Siena, Italy, November 2003, Revised Papers,
  volume 3014 of Lecture Notes in Computer Science,
  Springer, 2004
• Asim Rahman, Metrics for the Structural
  Assessment of Product Line Architecture, Master
  Thesis Software Engineering, Thesis nº
  MSE-200424, 2004
• Claudio Riva, Christian Del Rosso, Experiences
  with Software Product Family Evolution, 6th
  International Workshop on Principles of Software
  Evolution (IWPSE 2003), Helsinki, Finland. IEEE
  Computer Society 2003, pp 161-169
• Christoph Stoermer, Liam OBrien, MAP Mining
  Architectures for Product Line Evaluations,
  Proceedings of the Third Working IFIP Conference
  on Software Architectures (WICSA 01), Amsterdam,
  2001
• Christoph Stoermer, Felix Bachman, Chris Verhoef,
  SACAM The Software Architecture Comparison
  Analysis Method, Technical Report,
  CMU/SEI-2003-TR-006, 2003
• André van der Hoek, Ebru Dincel, Nenad
  Medvidovic, Using Service Utilization Metrics to
  Assess and Improve Product Line Architectures,
  9th IEEE International Software Metrics Symposium
  (METRICS 2003), Sydney, Australia. IEEE Computer
  Society 2003, pp 298-308
• Jan Gerben Wijnstra, Evolving a Product Family in
  a Changing Context, In Frank van Linden, editor,
  Software Product-Family Engineering, 5th
  International Workshop, PFE 5, Siena, Italy,
  November 2003, Revised Papers, volume 3014 of
  Lecture Notes in Computer Science, Springer, 2004
• Hongyu Zhang, Stan Jarzabek, Bo Yang, Quality
  Prediction and Assessment for Product Lines, In
  Johann Eder, Michele Missikoff (Eds.) Advanced
  Information Systems Engineering, 15th
  International Conference, CAiSE 2003, Klagenfurt,
  Austria, Proceedings. Lecture Notes in Computer
  Science 2681 Springer 2003, pp 681-695

26
Questions