Enhancing Data Warehouse Design with the NFR Framework

1
Enhancing Data Warehouse Design with the NFR
Framework

• Fábio Rilston Silva Paim
• Jaelson Brelaz de Castro

Universidade Federal de Pernambuco (UFPE) Centro
de Informática Recife Pernambuco Brazil
V INTERNATIONAL WORKSHOP ON SOFTWARE ENGINEERING
2
Outline

• Data Warehouse Systems
• Requirements Engineering in Data Warehouses
• Methodological Approach
• Motivation for NFR Vision
• DW Requirements Process
• Data Warehouse-Extended NFR Framework
• The NFR Framework
• Data Warehouse NFR Types
• Main Data Warehouse NFRs
• Operationalizing Catalogues
• Study Case
• Future Works

3
Data Warehousing
Inmon 1996

• Defined process that entails data
• Extraction from distributed operational sources.
• Transformation to enable data integration
  according to strategic requirements.
• Organization in a dimensional model, more
  appropriate to analytical querying.
• Access in an efficient, flexible way.

4
Data Warehouse Process
DATA SOURCES STAGING AREA
DATA WAREHOUSE
DECISION SUPPORT
Application Databases
__________________________________________________
____
Reports
Packaged application/ERP Data
DATA MARTS
INCOME ANNUAL REPORT ___ ___ ____ _____ ___ __
___ ___ ____ _____ ___ __ ___ ___ ____ _____
___ __
EXTRACTION TRANSFORMING CLEANING AGGREGATION
DATA WAREHOUSE
EIS
Desktop Data
OLAP
External Data
Statistical Financial Analysis
Web-based Data
Adapted from SunExpert Magazine, October 1998.
5
Data Warehouse Basic Elements

• Multidimensionality
• Separates information into facts and dimensions.
• Facts
• Numeric or factual data that represents a
  specific business activity (ex. Monthly Sales).
• Dimensions
• Single perspective on the data (ex. Time,
  Product).
• Attribute
• Descriptive items that compound a dimension.
• Hierarchy
• Single aggregation path within a dimension.

6
Dimensional Model (Star)

• A fact table surrounded by dimension tables.

Hierarchy
ProductID Description Make Category
Product
Fact Table
Sales No. ClientID ProductID Issue Date Amount
Sales
ClientID Name Category
Client
Day Month Year
Time
Measure
Dimension Table
Attribute
7
OLAP (OnLine Analytical Processing)

• Data seen from different angles and on different
  aggregational levels.

• OLAP operations work on
• Data Cubes.

Measure
Cell
Roll-up Drill-down Pivot
Dimension 1
Hierarchy
Dimension 2
8
Conventional versus DW Design
9
Outline

• Data Warehouse Systems
• Requirements Engineering in Data Warehouses
• Methodological Approach
• Motivation for NFR Vision
• DW Requirements Process
• Data Warehouse-Extended NFR Framework
• The NFR Framework
• Data Warehouse NFR Types
• Main Data Warehouse NFRs
• Operationalizing Catalogues
• Study Case
• Future Works

10
Requirements Approach for DW
Paim et al. 2002
Data Warehouse Requirements
Data Mart Users Needs
Development Cycle
Data Warehouse Requirements Updated
Early DataMart Requirements
Requirements Specification
Business Domain
Project Guidelines
Requirements Management Planning
New Baseline
Accorded Changes
User Needs
Management Plan
Requirements Validation
DataMart Requirements Release
Requirements Management Control
11
General DW Design Requirements

• Multidimensionality.
• Summarizability assurance.
• Correctly represent integration with data
  sources.
• Query performance.
• Conformance between common aspects.
• Timely, Precise background process.
• (Others requirements )

NFR
12
IMPACT
13
DW Requirements Process
User Needs
Data Mart Requirements Release
Update DW Model
RequirementsManagement Planning
Set DW Scope
Elicitation
Analysis
Correlate Requirements
Documen tation
Confor mance
Set Data Mart Scope
Business Domain
Specify Functional Requirem.
Identify Multidim. Requirem.
Requirements Management Control
Methodological Phases
Data Warehouse Requirements Process
14
Outline

• Data Warehouse Systems
• Requirements Engineering in Data Warehouses
• Methodological Approach
• Motivation for NFR Vision
• DW Requirements Process
• Data Warehouse-Extended NFR Framework
• The NFR Framework
• Data Warehouse NFR Types
• Main Data Warehouse NFRs
• Operationalizing Catalogues
• Study Case
• Future Works

15
NFR Framework
CHUNG et al. 2000
User-Friendly Access accounts
Secure accounts
Good Performance accounts
?
Integrity accounts
Availability accounts
?
Confidentiality accounts
Accuracy accounts
Space accounts
Response Time accounts
-
?
X
?
!

Authorize access to information accounts
Authenticate user access
Completeness accounts
?
-


?

-
?
?
X
?
Use Indexing accounts
Identify users
?
Use PIN
Require additional ID
Use uncompressed format accounts
Claim Optimized validation will not hurt
Response too much
?
Validate access against eligibility rules
?
Compare signature
Softgoals
Interdependency Implicity
Explicity
Operationalizing Method
Strongly positive satisficing Positive
satisficing - Negative satisficing --
Strongly Negative satisficing
! Critical ?Accepted X Rejected
NFR Softgoal
Claim
16
Data Warehouse NFRs
Data Warehouse NFR-Type Hierarchical Tree
Security
User-Friendliness
Performance
Multidimensionality
Availability
Integrability
Integrity
Operability
Space
Time
Accessibility
Confidentiality
Interpretability
Flexibility
Timely
Correctness
Processing Time
Completeness
Accuracy
Main Memory
Learnability
Distributivity
Consistency
Minimality
Traceability
Response Time
Secondary Memory
Data Interpretability
Documentation Readability
Domain Compliance
Reliability
Summarizability
17
Performance Catalogue
18
Multidimensionality Catalogue
19
Integrity Catalogue
20
Outline

• Data Warehouse Systems
• Requirements Engineering in Data Warehouses
• Methodological Approach
• Motivation for NFR Vision
• DW Requirements Process
• Data Warehouse-Extended NFR Framework
• The NFR Framework
• Data Warehouse NFR Types
• Main Data Warehouse NFRs
• Operationalizing Catalogues
• Study Case
• Future Works

21
Study Case

• S.A.F.E.
• Federal Revenues Secretariat Strategic System.
• Subject-oriented complex design.
• Users main quality goals
• Performance
• Multidimensionality Mapping
• Some user non-functional requirements

R1 - Query response time must not exceed 1
minute R2 - The system should present the most
up-to-date information as possible R3 - The
system must allow the user to drill across data
from different subjects
22
Design Problem
which design alternative best fits the three
conditions ?
?
23
Analysing NFR Requirements

• From R1
• Time Performance is a critical issue.
• From R1 and R2
• Response Time prevails against processing time.
• From R3
• Both regular and complex queries require high
  performance.
• Drill-Across OLAP Operations are demanded.
• Multidimensional Model must support joins between
  multiple fact tables.

24
Multidimensionality data
Performance queries
Applying the Data Warehouse Extended Framework
 
 
Space queries
Time queries
Integrability data
Acessibility data
X
?
?
!
?
Claim Response Time is critical for the system
-
Response Time queries
Timely data
Data Access data

Processing Time queries
X
!!
!
?
?
?
-
Critical path to best Design Solution
?



?
X  
Optimize Time queries
Static Access
X
?
?
Dinamic Access
Daily Loading
Periodic Loading


Critical NFR
?
X
?
OLAP
?
--
Join Techniques
Denormalization
Monthly Loading
?
OLAP Modeling
OLAP Operations
X
Indexing queries
?
Parallelism
Claim There exists no time window to process
daily
--
?
?
--


?
?
MOLAP
ROLAP
Indexing RegularQueries
Indexing ComplexQueries
?
X
?
?
?
OLAP Operations AdvancedQuerying
Claim Out of design scope




OLAP Operations BasicQuerying

?
?
X
?
?
StarSchema
B-Tree Indexes
Bitmap Indexes
Snowflake

Drill Across

?
Constellation
25
Derived Design Solution
The data warehouse architecture will be built
around a constellation schema, with interlinked
fact tables to enable drill across operations.
Data denormalization, together with B-Tree and
Bitmap indexes will be used to improve query
speed.
26
Future Works

• Investigate intertwined relationship between DW
  NFR Types.
• Use the NFR-Assistant Tool (Tran, Chung 1999)
  to
• Support more complex design analysis in our
  ongoing project.
• Investigate the influence of intertwined
  relationship between ilities when analyzing
  optimized design solutions.
• Investigate weaknesses and strengths of
  alternative architecture configurations.

27
References

• Paim et al. 2002 Paim, F., Carvalho, A.,
  Castro, J. Towards a Methodology for
  Requirements Analysis of Data Warehouse Systems".
  In Proceedings of XVI Simpósio Brasileiro de
  Engenharia de Software (SBES2002), Gramado, Rio
  Grande do Sul, Brazil 2002.
• Chung et al. 2000 Chung, L., Nixon, B., Yu, E.,
  Mylopoulos, J. Non-Functional Requirements in
  Software Engineering, Kluwer Publishing, 2000.
• Inmon 1996 Inmon, W. H. Building the Data
  Warehouse, John Wiley Sons, 2nd edition, 1996.
• Tran, Chung 1999 Tran, Q., Chung, L.
  "NFR-Assistant Tool Support for Achieving
  Quality". In Proc. of IEEE Symposium on
  Application-Specific Systems and Software
  Engineering and Technology (ASSET'99),
  Richardson, Texas, March 24 - 27, 1999.

28
Contact

• FÁBIO RILSTON
• frsp_at_cin.ufpe.br
• JAELSON CASTRO
• jbc_at_cin.ufpe.br