Quality assessment technique for ubiquitous computing

1
Quality assessment technique for ubiquitous
computing

• Hokyoung Ryu, Soonil Cha
• Institute of Information and Mathematical
  Sciences, Massey University
• Telecommunications Technology Association
• 28.01. 2005

2
Why we have to do this

• Personal computing environment
• Stand-alone, or one-to-one interaction
• e.g., Desk-top environment
• Ubiquitous computing environment
• Mobile, or one-to-many interactions
• e.g., Sensor network in Smart house project
• Research issue
• The same guidelines? ? whether we can use the
  same quality metrics for the ubiquitous computing
  environment with the personal computing
  environment
• If not, how we can evaluate (or test) it

3
Research approaches

• Technology-in-use evaluation
• See how the ubiquitous systems are using in the
  real life
• The same methods with the HCI evaluation approach
• Walkthroughs
• Usability testing
• No evaluation guidelines available
• Component-based evaluation
• See how each component is working in the whole
  systems
• Each metrics of each components
• Users
• Services
• OS/Middleware/Network System software
  technology
• I/O artefacts (or devices)
• A framework to integrate them is necessary

4
Research framework(Ubiquitous computing)
5
Component-Users

• Activities
• Predictability
• Predicted action/effect/goal reorganisation
• 5 questions and 7 metrics
• Accountability
• Automatic process
• 3 questions and 3 metrics
• Simplicity
• Activity complexity
• 4 questions and 4 metrics
• Evaluation
• Reversibility
• 3 questions, 3 subquestions and 4 metrics
• Acknowledgement
• 6 questions and 3 metrics

6
Component-I/O artefacts

• Reception
• Quality of reception
• Reliability, Accuracy, Falut tolerance
• 13 questions and 11 metrics
• Mobility
• 2 questions and 1 metrics
• Integration
• Ubiquitous connectivity and integrity
• 3 questions and 2 metrics
• Expression
• Transparency
• 2 questions and 3 metrics
• Unobtrusiveness
• 6 questions and 7 metrics
• Awareness support
• 3 questions and 6 metrics
• Controllability
• 6 questions and 3 metrics

7
Component - Services

• Interpretation
• Functionality
• Compliance, Suitability, Accuracy
• 8 questions and 8 metrics
• Service quality
• Efficiency, Resource availability, Reliability,
  Fault tolerance, Fault recovery, Maintainability,
  Adaptability
• 21 questions and 27 metrics
• Contextualisation
• Location awareness
• 2 questions and 2 metrics
• Environment awareness
• 3 questions and 2 metrics
• Situation awareness
• 3 questions and 2 metrics
• User recognition
• 1 question and 1 metric
• Personalisation
• 1 question and 1 metric
• Security

8
Component System software

• Configuration
• Scalability
• 6 questions, 11 measure, and 5 metrics
• Security
• 5 questions, 12 measures, and 5 metrics
• Network security
• 4 questions, 8 measures, and 3 metrics
• Extensibility
• 5 questions, 9 measures, and 5 metrics
• Coordination
• Resource coordination and optimisation
• 1 question, 5 measures, and 1 metric
• Mobility
• 2 questions, 4 measures, and 2 metrics
• Reliability
• 2 questions, 7 measures, and 6 metrics
• Distributed systems support
• 2 questions, 3 measures, and 2 metrics

9
Case study (I) Portable Help Desk
10
Case study (I) Portable Help Desk

• A campus tour guide
• Tracks user location
• Provide information about surroundings on PDA
• The devices were distracting, Users looked at the
  devices, not local surroundings
• Expression Awareness support, Unobtrusiveness
  (Expression of I/O artefacts)
• Users wanted more functionality
• Functionality (Services)
• Controllability (Expression of I/O artefacts)
• Did not sense location correctly
• Quality of reception (Reception of I/O artefacts)
• Location awareness (Contextualisation of
  Services)

11
Case study (II) Vodafone location awareness
service
12
Case study (II) Vodafone location awareness
service

• Vodafone Germany (2000), UK (2004)
• First location awareness service
• Traffic, shopping information
• The user has to explicitly request (pull service)
• Simplicity (Users activity)
• System is monitoring the user
• Transparency (Expression of I/O devices)
• Privacy (Service)
• Acknowledgement (User evaluation)
• Controllability (Expression of I/O devices)
• Accurately locating ?
• Quality of reception (Reception of I/O devices)
• Location awareness (Contextualisation of
  Services)
• Functionality (Services)

13
Case study (III) NZ mobile banking services
14
Case study (III) NZ mobile banking services

• Mobile banking service (2003)
• Bush areas
• Quality of reception (Reception of I/O devices)
• How secure or reliable
• Security (Service and Configuration of System
  software)
• Reliability (Service and Configuration of System
  software)
• Interface
• Simplicity (Users activity)
• Controllability (Expression of I/O devices)
• Service resources
• Scalability (Configuration of System software)
• Reliability (Service), Help support
  (Accountability users activity)

15
Further works

• Case studies
• Clarify the quality metrics
• External vs. Internal
• Collecting some case studies
• Designing a ubiquitous system
• Running tests
• Validation studies
• Checking the metrics proposed
• e.g., ubiquitous building
• New possibilities