Analysis of Recommendation Algorithms for E-Commerce

1
Analysis of Recommendation Algorithms for
E-Commerce

• Badrul M. Sarwar, George Karypis,
  Joseph A. Konstan, and John T. Riedl
• GroupLens Research/Army HPCRC
• Department of Computer Science and Engineering
• University of Minnesota

2
Talk Outline

• Recommender Systems for E-Commerce
• Quality and Performance Challenges
• Synopsis of Recommendation Process
• Experimental Setup
• Result Highlights
• Conclusion

3
Recommender Systems

• Problem
• Information and commerce overload
• Solution
• Knowledge Discovery in Database (KDD)
• Recommender Systems (RS)
• Collaborative Filtering

4
Collaborative Filtering

• Adds human judgement to the filtering process

5
Collaborative Filtering (contd.)

• Major Tasks
• Representation of input data
• Customer-product rating matrix
• Neighborhood formation
• Output
• Prediction
• Top-N Recommendation

6
(No Transcript)
7
Challenges of RS

• Sparsity
• Enormous size of customer-product matrix
• Affects neighborhood formation
• Results in poor quality and reduced coverage
• Scalability
• Lots of customers and products
• Affects neighborhood and output
• Results in high response time

8
Challenges of RS

• Synonymy
• Similar products treated differently
• Increases sparsity, loss of transitivity
• Results in poor quality

9
Use of SVD for Collaborative Filtering

• 1. Low dimensional representation
• O(mn) storage requirement

k x n
m x k
2. Direct Prediction
.
m x n
m x m similarity

• Top-N Recommendation
• Prediction (CF algorithm)

3. Neighborhood Formation
10
Experimental Setup

• Data sets
• MovieLens Data (www.movielens.umn.edu)
• Size 943 x 1,682
• 100,000 ratings entry
• Ratings are from 1-5
• Used for Prediction and Neighborhood experiments
• E-Commerce Data
• Size 6,502 x 23,554
• 97,045 purchase entry
• Purchase entries are dollar amounts
• Used for Neighborhood experiment
• Train and Test Portions
• Percentage of Training data, x

11
Experimental Setup

• Benchmark Systems
• CF-Predict
• CF-Recommend
• Metrics
• Prediction
• Mean Absolute Error (MAE)
• Top-N Recommendation
• Recall and Precision
• Combined score F1

12
Results Prediction Experiment
13
Results Neighborhood Formation

• Movie Dataset

14
Results Neighborhood Formation

• E-Commerce Dataset

15
Conclusion

• SVD results are promising
• Provides better Recommendation for Movie data
• Provides better Prediction for xlt0.5
• Not as good for the E-Commerce data
• We only tried upto 400 dimensions
• SVD provides better online performance
• SVD is capable of meeting RS challenges
• Sparsity
• Scalability
• Synonymy

16
(No Transcript)
17
(No Transcript)
18
Acknowledgements

• National Science Foundation under grants IIS
  9613960, IIS 9734442, IIS 9978717, CCR 9972519,
  EIA 9986042, ACI 9982274.
• Army Research Office DAAG-55-98-1-0441, DOE ASCI
  program. Army High Performance Computing Research
  Center grant DAAH-04-95-C-0008
• Thanks to Netperceptions Inc. for additional
  support.
• Thanks to Fingerhut Inc. for the EC dataset.