Outdoors Augmented Reality on Mobile Phone using LoxelBased Visual Feature Organization

1
Outdoors Augmented Reality on Mobile Phone using
Loxel-Based Visual Feature Organization
Gabriel Takacs, Vijay Chandrasekhar, Thanos
Bismpigiannis, Bernd Girod Stanford University

• Radek Grzeszczuk, Natasha Gelfand,
• Wei-Chao Chen, Yingen Xiong, Kari Pulli
• Nokia Research Center, Palo Alto

2
Video Demonstration
3
Outline

• System Overview
• Image matching on the cell phone
• Data Organization
• Server groups images by location
• Data Reduction
• Server clusters, prunes and compresses
  descriptors
• Image Matching Results
• Qualitative and quantitative
• System Timing
• Low latency image matching on the cell phone

4
System Overview
5
Image Matching
Query Image
SURF-64 Descriptors
Ratio Test Matching
Affine RANSAC
Database Images
6
Data Organization
Loxel
Kernel
7
System Block Diagram
Geo-TaggedImages
Group Imagesby Loxel
ExtractFeatures
GeometricConsistency Check
Match Images
Cluster Features
Prune Features
Compress Descriptors
Loxel-Based Feature Store
Network
ExtractFeatures
ComputeFeature Matches
DeviceLocation
CameraImage
GeometricConsistency Check
Display Info forTop Ranked Image
8
Feature Descriptor Clustering

• Match all images in loxel
• Form graph on features
• Cut graph into clusters
• Create representative meta-features

meta-feature
9
Feature Descriptor Clustering
Images of the same landmark
10
Database Feature Pruning

• Rank images by number of meta-features
• Allocate equal budget for each landmark
• Fill budget with meta-features by rank
• Fill any remaining budget with single features

11
Feature Descriptor Pruning
200
100
All
500
Budget
12
Feature Compression

• Quantization
• Uniform, equal step-size
• 6 bits per dimension
• Entropy coding
• Huffman tables
• 12 different tables
• 64-dimensional SURF
• 256 bytes uncompressed
• 37 bytes with compression

Original Feature
Compressed Feature
Quantization
Entropy Coding
S dx S dy Sdx Sdy
13
ZuBuD Dataset (1000 Images)
14
Stanford Dataset (2500 Images)
15
Matching Results
Query
Rank 1
Rank 2
Rank 3
Rank 4
16
Matching Performance
True Matches False Matches
17
Timing Analysis
Nokia N95 332 MHz ARM 64 MB RAM
100 KByte JPEG over 60 Kbps Uplink
Downloads
Upload
Upload
Geometric Consistency
Extract Features
Extract Features
Feature Matching
All on Server
Extract Features on Phone
All on Phone
18
Conclusions

• Image matching on mobile phone
• Use loxels to reduce search space
• 27x reduction in data sent to phone
• Clustering
• Pruning
• Compression
• 3 seconds for image matching on N95

19
Questions