Magic Book: Enhancing Natural Feature Tracking with the User

1
Magic Book Enhancing Natural Feature Tracking
with the Users Movement Context
Felix Loew Master Thesis (TU München)
21.01.2005
2
Outline

• Context-Awareness and Augmented Reality
• Vision Based Tracking Natural Feature Tracking
• A Magic Book User Study
• Software Architecture (DWARF )
• Expected results and conclusions

3
Context-Awareness and Augmented Reality

• A system has to adopt to the users behaviour!
• A system has to learn how it is used!
• New intuitive methods of interaction are
  required!
• The system has to disappear (Marc Weiser)!

4
The Magic Book in this context

• User has to learn how it is used
• Very error-prone to movement
• Movement of the handheld device is not considered
  in the tracking routine
• Vision based tracking loses orientation
• User has to reinitialize the tracking (looking on
  the marker again)
• Immersion decreases!

5
My Idea Enhancing the Magic Book
Movement
6
Outline

• Context-Awareness and Augmented Reality
• Vision Based Tracking Natural Feature Tracking
• A Magic Book User Study
• Software Architecture (DWARF )
• Summary, Conclusions Limitations

7
Basics Natural Feature Tracking
Frame 1
Frame 2
Frame 4
Frame 3
8
Basics Natural Feature Tracking

• NFTARToolkit (Kato) 2D planar scenes
• Estimation of feature position in next frame
• It is not realistic that no movement occurs
• Linear estimation
• Kalman Filter
• Within a window the best match with the
  template is searched (Template Matching)
• For every pixel within window the similarity with
  the template is calculated (Normalized Cross
  Correlation)

9
Template Matching
10
Normalized Cross Correlation

• Calculates similarity between template and area
  around search point
• values -1..1
• Best match is considered as feature point in
  current frame (threshold 0.7)!
• Computational expensive!
• O (templateSize2 searchSize2)

11
My approach

• Consider movement as well!
• Use additional tracking device (Inertial Tracker
  / 3DOF)
• Naive approach
• much movement large search window
• low movement small search window
• Dynamic configuration during runtime

12
My approach
What is much movement? Which movements actually
occur? How does a user use such an application?
USER STUDY ON MAGIC BOOK
13
Outline

• Context-Awareness and Augmented Reality
• Vision Based Tracking Natural Feature Tracking
• A Magic Book User Study
• Software Architecture (DWARF )
• Expected results and conclusions

14
User Study Setup

• Goals
• Detecting specific movement patterns
• Derive Movement model
• Estimation of a correlation between changes in
  orientation and feature points coordinates

Orientation (Handheld, 3D)
Position (FP-Coords, 2D)
15
User Study Setup

• Tracking Orientation and Position (6DOF)
• Magnetic Tracker Flock of Birds
• Bird1 Magic Book
• Bird2 Handheld device
• Calibration to Magic Book coordinate system
  (Bird1)

16
Logging

• Magnetic Tracker
• Timestamp
• Position P (px,py,pz)
• Orientation (quaternion) Q (v,s)
• NFT
• Timestamp
• 4 feature points per frame
• Id
• Screen-Position S (sx,sy)

17
Logging

• Magnetic Tracker
• pics\2_20_1.Case2.Flock.xls
• NFT
• pics\2_20_1.Case2.Coords.xls

18
User Tasks

• Comparable Tasks (Navigation Tasks)
• User has to answer certain questions (Overview,
  Focus and Search Tasks)
• What is the girls hair color?
• How many people do you see?
• Free Tasks
• User can use Magic Book freely

19
Evaluation

• Ideas for evaluation (Correlation)
• Which angular offset (FOB) causes which
  coordinate offset (NFT)?
• Which angular velocities occur?
• Which rotations occur?
• Which changes of position occur?
• Is there a common movement profile or are there
  serious differences between different users?

20
Evaluation

• Find mapping between
• Configure Software Architecture with this
  information

orientation
search window
21
Outline

• Context-Awareness and Augmented Reality
• Vision Based Tracking Natural Feature Tracking
• A Magic Book User Study
• Software Architecture (DWARF)
• Expected results and conclusions

22
Software Architecture (DWARF)
NFT ARToolkit
23
Dynamic Configuration
24
Outline

• Context-Awareness and Augmented Reality
• Vision Based Tracking Natural Feature Tracking
• A Magic Book User Study
• Software Architecture (DWARF)
• Expected results and conclusions

25
Conclusions Summary

• Hybrid Tracking approach to consider context
  information as well
• Movement context
• 2DO What about user context?
• User Study on Magic Book (work in progress)
• Expected conclusions on usage of Magic Book
• Find correlation between movement and NFT
  properties
• Process a configuration out of the study
• Derive Movement model of the Magic Book
• Limitation Very task specific (features are not
  moving, only 2D)!
• Software Architecture

26
Limitations 2DOs

• Heaps of work
• I wont cover everything I want to do
• Other sensor-fusion approaches might be more
  suited (Kalman-Filtering)
• Evaluation of User Study will be hard
• Is there an abstraction for NFT based
  applications?

27
Thanks

• I would be glad if some of you want to join the
  study!