The Effect of Dimensionality Reduction in Recommendation Systems

1
The Effect of Dimensionality Reduction in
Recommendation Systems

• Juntae Kim
• Department of Computer Engineering
• Dongguk University

2
Contents

• Introduction
• Collaborative Recommendation
• Data Sparseness Problem
• Dimensionality Reduction by using SVD
• An Example
• Experiments
• Conclusion

3
Introduction

• e-CRM
• Provides personalized service
• Enhance sales by
• Product recommendation, target advertisement,
  etc.
• Recommendation System

Demographic features
Recommend items
Item features
Customer
Purchase history
Sales history
4
Introduction

• Use item-to-item similarity content-based
• Use item-to-item similarity association

5
Introduction

• Use people-to-people similarity demographic
• Use people-to-people similarity collaborative

like
A
B
high correlation
Recommend
C
A
B
like
6
Collaborative Method

• Advantages
• No needs of contents analysis
• Items that are difficult to analyze contents can
  be recommended
• Exgt Movie, music,
• No needs of user information
• High precision
• Method
• Find out similar users
• Predict preferences based on similar users
  preferences

7
Collaborative Method

• Computing similarity
• Pearson correlation coefficient ( -1, 1 )
• Rating of user a to item i
• Example
• User a (1, 8, 9) ? (-5, 2, 3)
• User b (2, 9, 7) ? (-4, 3, 1)
  User a is similar to b
• User c (9, 3, 3) ? (4, -2, -2)

8
Collaborative Method

• Prediction of preferences
• Weighted sum of similar users preferences
• ??? a? u? ???
• Example
• Average rating of user a 5 Preferences of user a
• User b (2, 8, 8), wa,b 0.5 (5, 5, 5) (-4,
  2, 2)0.5
• User c (4, 4, 7), wa,c 0.1 (-1, -1,
  2)0.1
• (2.9, 5.9, 6.2)

9
Data Sparseness Problem

• Example data

10
Data Sparseness Problem

• Explicit ratings are not usually available
• Available data ? purchase, click, etc.
• ? 0 or 1
• Computing correlation is not appropriate
• (no negative preference information)
• use cosine similarity

11
Data Sparseness Problem

• Available data are usually very sparse
• Buy 23 items among thousands of items
• Cosine similarity can not be computed
• Reduce dimension

12
Dimensionality Reduction

• Using category information
• Represent user preference vector with item
  categories
• Monster Co., Lion King, Pocahontas ? animation
• Holloween, Scream ? horror

13
Dimensionality Reduction

• Singular Value Decomposition (SVD)
• Decompose the user-item matrix Amn
• Amn Umm Smn (Vnn)T
• S Diagonal matrix that contains the singular
  values of A in descending order
• U, V Orthogonal matrices
• Rotating the axes of the n-dimensional space
• 1st axis runs along the direction of largest
  variation

14
Dimensionality Reduction

• SVD example

15
Dimensionality Reduction

• Approximation of A
• Select largest k singular values
• Amn Umk Skk (Vnk)T
• Computing user similarity
• AAT USVT(USVT)T
• USVTVSTUT
• (US)(US)T
• Projection of A into k dimension
• Amn Vnk Umk Skk

16
An Example

• User-item matrix

17
An Example

• Reduction, k 2

18
An Example

• User-user similarity

19
An Example

• User vectors in 2-D space

u6
u4
u5
u1
u2
u3
20
Experiments

• Dataset MovieLens
• 943 users, 1628 movies, 15 rating, 6.4 rated
• Change ratings to 0/1 ? 3.6 rated
• Experiments
• Compare performance of plain collaborative(CF)
  and reduced dimension(SVD) recommendation
• CF 60 neighbor
• SVD rank 20
• Change sparseness to 2.0, 1.0, 0.5

21
Experiments

• Metric
• Hit ratio
• Remove 1 rating from each user ? test data
• Recommend 10 items for each user
• If the test data is in the recommended item ? hit
• Total of hit
• Total of test data
• Result
• Sparseness 3.6 ? SVD improves hit ratio by x
• Sparseness 0.5 ? SVD improves hit ratio by x

Hit ratio
22
Experiments

• Results

23
Conclusion

• Solve data sparseness problem
• Reduce dimension heuristics
• Reduce dimension SVD
• Experimental results
• SVD shows more performance improvement in sparser
  data
• Future research
• Statistical analysis
• Combined methods

24
References

• Basu, C, Hirsh, H., Cohen, W., Recommeder
  Systems. Recommedation As Classification Using
  Social And Conent-Based Information, Proceedings
  of the Workshop on Recommendation system. AAAI
  Press, Menlo Park California, 1998.
• Billsus, D., Pazzani, M. j., Learning
  Collaborative Information Filters, Proceedings
  of workshop on recommender system, 1998.
• Berry, M. W., Dumais, S. T., and OBrain, G. W.
  Using Linear Algebra for Intelligent Information
  Retrieval, SIAM Review, 37(4), pp. 573-595,
  1995.
• Breese, J. S., Heckerman, D., and Kadie, C.,
  Empirical Analysis of Predictive Algorithm for
  Collaborative Filtering,Proceeding of the
  Fourteenth Conference UAI, July 1998.
• Goldberg, k., Roeder, T., Gupta, D., and Perkins,
  C., Eigentaste A Constant Time Collaborative
  Filtering Algorithm, Technical Report M00/41.
  Electronics Research Laborotary, University of
  California, Berkeley, 2000.
• Herlocker, J., Konstan, J., Borchers, A., Riedl,
  J., An Algorithmic Framework for Performing
  Collaborative Filtering,Proceedings of the 1999
  Conference on Research and Development in
  Information Retrieval, Aug. 1999.
• Sarwar, B. M. Sparsity, Scalability, and
  Distribution in Recommender Systems, Ph.D.
  Thesis, Computer Science Dept., University of
  Minnesota, 2001.
• Sarwar, B. M., Karypis, G., Konstan, J. A., and
  Riedl, J., Application of Dimensionality
  Reduction in Recommender System-A Case
  Study,WebKDD 00-Web-mining for E-Commerce
  Workshop, 2000.
• Schafer, J. B., Konstan, J., and Riedl, J.,
  Recommender Systems in E-Commerce, Proceedings
  of the ACM Conference on Electronic Commerce,
  November 1999.
• Shardanand, U., "Social information filtering for
  music recommendation," Technical Report MA95, MIT
  Media Laboratory, 1995.