Hybrid Web Recommender Systems

1
Hybrid Web Recommender Systems

• Robin Burke
• Presentation by Jae-wook Ahn
• 10/04/05

2
References

• Entrée system dataset
• Burke, R. (2002). Semantic ratings and heuristic
  similarity for collaborative filtering. AAAI
  Workshop on Knowledge-based Electronic Markets
  2000.
• Feature augmentation, mixed hybrid example
• Torres, R., McNee, S., Abel, M., Konstan J.,
  Riedl J. (2004). Enhancing Digital Libraries with
  TechLens. Proceedings of the 2004 Joint ACM/IEEE
  Conference on Digital Libraries.
• Hybrid recommender system UI issue
• Schafer, J. (2005). DynamicLens A Dynamic
  User-Interface for a Meta-Recommendation System.
  Workshop Beyond Personalization 2005, IUI05.
• Collaborative filtering algorithm
• Sarwar, B., Karypis, G., Konstan, J., Riedl, J.
  (2001). Item-based collaborative filtering
  recommendation algorithms. In Proceedings of the
  10th international conference on World Wide Web.

3
Concepts and Techniques
4
Hybrid Recommender Systems

• Mix of recommender systems
• Recommender system classification knowledge
  source
• Collaborative (CF)
• Users ratings only
• Content-based (CN)
• Product features, users ratings
• Classifications of users likes/dislikes
• Demographic
• Users ratings, users demographics
• Knowledge-based (KB)
• Domain knowledge, product features, users
  need/query
• Inferences about a uses needs and preferences

5
CF vs. CN

• User-based CF
• Searches for similar users in user-item rating
  matrix
• Item-based CF
• Searches for similar items in user-item rating
  matrix
• CN
• Searches for similar items in item-feature matrix
• Example TFIDF term weight vector for news
  recommendation

Items
Ratings
Users
6
Recommender System Problems

• Cold-start problem
• Learning based techniques
• Collaborative, content-based, demographic
• ? Hybrid techniques
• Stability vs. plasticity problem
• Difficulty to change established users profile
• Temporal discount older rating with less
  influence
• KB fewer cold start problem (no need of
  historical data)
• CF/Demographic cross-genre niches, jump outside
  of the familiar (novelty, serendipity)

7
Strategies for Hybrid Recommendation

• Combination of multiple recommendation techniques
  together for producing output
• Different techniques of different types
• Most common implementations
• Most promise to resolve cold-start problem
• Different techniques of the same type
• Ex) NewsDude naïve Bayes kNN

8
Seven Types of Recommender Systems

• Taxonomy by Burke (2002)
• Weighted
• Switching
• Mixed
• Feature combination
• Feature augmentation
• Cascade
• Meta-level

9
Weighted Hybrid

• Concept
• Each component of the hybrid scores a given item
  and the scores are combined using a linear
  formula
• When recommenders have consistent relative
  accuracy across the product space
• Uniform performance among recommenders (otherwise
  ? other hybrids)

10
Weighted Hybrid Procedure

• Training
• Joint rating
• Intersection candidates shared between the
  candidates
• Union case with no possible rating ? neutral
  score (neither liked nor disliked)
• Linear combination

11
Mixed Hybrid

• Concepts
• Presentation of different components side-by-side
  in a combined list
• If lists are to be combined, how are rankings to
  be integrated?
• Merging based on predicted rating or on
  recommender confidence
• Not fit with retrospective data
• Cannot use actual ratings to test if right items
  ranked highly
• Example
• CF_rank(3) CN_rank(2) ? Mixed_rank(5)

12
Mixed Hybrid Procedure

• Candidate generation
• Multiple ranked lists
• Combined display

13
Switching Hybrid

• Concepts
• Selects a single recommender among components
  based on recommendation situation
• Different profile ? different recommendation
• Components with different performance for some
  types of users
• Existence of criterion for switching decision
• Ex) confidence value, external criteria

14
Switching Hybrid Procedure

• Switching decision
• Candidate generation
• Scoring
• No role for unchosen recommender

15
Feature Combination Hybrid

• Concepts
• Inject features of one source into a different
  source for processing different data
• Features of contributing recommender are used
  as a part of the actual recommender
• Adding new features into the mix
• Not combining components, just combining
  knowledge source

16
Feature Combination Hybrid Procedure

• Feature combination
• ? In training stage
• Candidate generation
• Scoring

17
Feature Augmentation Hybrid

• Concepts
• Similar to Feature Combination
• Generates new features for each item by
  contributing domain
• Augmentation/combination done offline
• Comparison with Feature Combination
• Not raw features (FC), but the result of
  computation from contribution (FA)
• More flexible to apply
• Adds smaller dimension

18
Feature Augmentation Hybrid Procedure
19
Cascade Hybrid

• Concepts
• Tie breaker
• Secondary recommender
• Just tie breaker
• Do refinements
• Primary recommender
• Integer-valued scores higher probability for
  ties
• Real-valued scores low probability for ties
• Precision reduction
• Score 0.8348694 ? 0.83

20
Cascade Hybrid Procedure

• Procedure
• Primary recommender
• Ranks
• Break ties by secondary recommender

21
Meta-level Hybrid

• Concepts
• A model learned by contributing recommender
• ? input for actual recommender
• Contributing recommender completely replaces the
  original knowledge source with a learned model
• Not all recommenders can produce the intermediary
  model

22
Meta-level Hybrid Procedure

• Procedure
• Contributing recommender
• ? Learned model
• Knowledge Source Replacement
• Actual Recommender

23
Experiments
24
Testbed Entrée Restaurant Recommender

• Entrée System
• Case-based reasoning
• Interactive critiquing dialog
• Ex) Entry ? Candidates ? Cheaper ? Candidates ?
  Nicer ? Candidates ? Exit
• Not narrowing the search by adding constrains,
  but changing the focus in the feature space

25
Testbed Entrée Restaurant Recommender (contd)

• Entrée Dataset
• Rating
• Entry, ending point positive rating
• Critiques negative rating
• Mostly negative ratings
• Validity test for positive ending point
  assumption strong correlation between original
  vs. modified (entry points with positive ratings)
  
• Small in size

26
Evaluation Methodology

• Measures
• ARC (Average Rank of the Correct recommendations)
• Accuracy of retrieval
• At different size retrieval set
• Fraction of the candidate set (0 1.0)
• Training Test set
• 5 fold cross validation random partition of
  training/test set
• Leave one out methodology randomly remove one
  item and check whether the system can recommend
  it
• Sessions Sizes
• Single visit profiles 5S, 10S, 15S
• Multiple visit profiles 10M, 20M, 30M

27
Baseline Algorithms

• Collaborative Pearson (CFP)
• Pearsons correlation coefficient for similarity
• Collaborative Heuristic (CFH)
• Heuristics for calculating distances between
  critiques
• nicer and cheaper ? dissimilar
• nicer quieter ? similar
• Content-based (CN)
• Naïve Bayes algorithm compute probability that
  a item is liked / disliked
• Too few liked items ? modified candidate
  generation
• Retrieve items with common features with the
  liked vector of the naïve Bayes profile
• Knowledge-based (KB)
• Knowledge-based comparison metrics of Entrée
• Nationality, price, atmosphere, etc.

28
Baseline Evaluations

• Techniques vary in performance on the Entrée data
• Content-based (CN) weak
• Knowledge-based (KB) better on single-session
  than multi-session
• Heuristic collaborative (CFH) better than
  correlation-based (CFP) for short profiles
• Room for improvement
• Multi-session profiles

29
Baseline Evaluations
30
Hybrid Comparative Study

• Missing components
• Mixed hybrid
• Not possible with retrospective data
• Demographic recommender
• No demographic data

31
Results Weighted

• Hybrid performance better in only 10 of 30
• CN/CFP consistent synergy (5 of 6)
• Lacks uniform performance
• KB, CFH
• Linear weighting scheme assumption fault

32
Results Switching

• KB hybrids best switching hybrids

33
Results Feature Combination

• CN/CFH, CN/CFP
• Contributing CN
• Identical to CFH, CFP
• CFH maintains accuracy with reduced dataset
• CF/CN Winnow modest improvement

34
Results Feature Augmentation

• Best performance so far
• Particularly CN/CF
• Good for multi-session profiles

35
Results Cascade

• CFP/KB, CFP/CN
• Great improvement
• Also good for multi-profile sessions

36
Results Meta-level Hybrids

• CN/CF, CN/KB, CF/KB, CF/CN
• Not effective
• No synergy
• Weakness of KB/CN in Entrée dataset
• Both components should be strong

37
Discussion

• Dominance of the hybrids over basic recommenders
• Synergy was found under
• Smaller profile size
• Sparse recommendation density
• ? hybridization conquers cold start problem

38
Discussion (contd)

• Best hybrids
• Feature augmentation, cascade
• FA allows a contributing recommender to make a
  positive impact
• without interfering with the performance of the
  better algorithm

39
Conclusions

• Knowledge-based recommendation is not limited
• Numerously combined to build hybrids
• Good for secondary or contributing components
• Cascade hybrids are effective
• Though rare in literatures
• Effective for combining recommender with
  different strengths
• Different performance characteristics
• Six hybridization techniques
• Relative accuracy consistency of hybrid
  components

40
System Example Related Issues
41
System Example TechLens

• Hybrid recommender system
• Recommenders CF, CN
• Hybrid algorithms CF/CN FA, CN/CF FA, Fusion
  (Mixed)
• Corpus
• CiteSeer
• Title, abstract (CN), citations (CF)
• Methodology
• Offline experiment, Online user study with
  questionnaire (by asking satisfaction on the
  recommendation)
• Results
• Fusion was the best
• Some FA were not good due the their sequential
  natures
• Different algorithms should be used for
  recommending different papers
• Users with different levels of experiences
  perceive recommendations differently

42
Meta-recommender DynamicLens

• Can user provided information improve hybrid
  recommender system output?
• Meta-recommender
• Provide users with personalized control over the
  generation of a recommendation list from hybrid
  recommender system
• MetaLens
• IF (Information Filtering), CF

43
Meta-recommender DynamicLens (contd)

• Dynamic query
• Merges preference recommendation interfaces
• Immediate feedback
• Discover why a given set of ranking
  recommendations were made

44
Questions Comments