NewsMe:

1
NewsMe

• A Case Study for Adaptive News Systems with Open
  User Model

Preliminary Examination Paper 2007 Chirayu
Wongchokprasitti IS PhD Student School of
Information Sciences
2
NewsMe
3
NewsMe Overview

• Personalized News Access System
• Feed the news that response to the users
  interest
• 82 RSS news feeds, 21 sources
• 8 News Topics
• Ranking the news
• Open User Model based system

4
NewsMe Interface

• 4 News Sections
• Recent News
• Recommended News
• My Profile
• News History

5
User Feedback Method

• Add a news item to Tracked News
• Add a news item to Blacklist

6
User Model Manipulation

• Update rating of news in user model

7
User Model Manipulation (Cont)

• List all history of viewed news
• Update rating of news in user model

8
Learning User Models for News Access

• The system uses a machine learning approach to
  build a simple model of each users interests.
• A similarity-based method achieves the balance of
  learning and adapts quickly to change interests
  while avoiding brittleness.

9
Learning User Models for News Access (cont.)

• The purpose of the user model
• First, it should contain information about
  recently read events, so that stories which
  belong to the same thread can be identified.
• To allow for identification of news that user
  already knows.
• The k-nearest-neighbor algorithm (kNN) is used to
  achieve the desired functionality.
• Convert news contents to tf-idf vectors
  (term-frequency/inverse-document-frequency).
• Use the cosine similarity measure to quantify the
  similarity of two vectors.

10
Learning User Models for News Access (cont.)

• Decay Function
• Freshness of news content is our issue.
• Freshness should decay exponentially day by day.
• Freshness of news remains a half after fed 7
  days.
• is the initial freshness of news content.
• is a decay instance, which its value is
  around 0.099.

11
Study Design

• 20 Participants
• Assign to be Information Analysts
• 2 News Topic US and Business
• 2 Sessions, 3 stages per session
• Retrieved News Nov 28th Dec 12th, 2006
• Google Notebook extension (http//www.google.com/n
  otebook)

12
Implicit VS Explicit Feedback

• Implicit feedback
• Assuming every news user read is a tracked news
• Explicit feedback
• Users add news items to their user model
• Tracked news as Positive sample
• Blacklist News as Negative sample

13
Hypotheses

• Performance hypotheses are
• H1 The open model system with user profile
  manipulation by users performs better than the
  open model system without them,
• H1.1 The open model system with explicit
  feedback generates results with better
  performance, and,
• H1.2 Users with explicit feedback system
  demonstrate higher task performance.

14
Hypotheses (Cont)

• User Perspective hypotheses are
• H2 Users prefer the user profile manipulation
  features in the open model system,
• H2.1 Users appreciate better in the system with
  explicit feedback, and,
• H2.2 Users appreciate the ability to control
  their profiles.

15
Preliminary Results

• The Ground Truth
• System Performance Analysis
• User Performance Analysis
• User Feedback Analysis

16
The Ground Truth

• F-measure defines as follows
• Summary of news items in the study

17
System Performance Analysis
18
System Precision _at_ First Screen
19
System Precision _at_ 60
20
System Precision _at_ 100
21
News Items Manipulation vs. System Performance
(Stage 2)
22
Tracked News ?Blacklist (Stage 2)
23
Tracked News ?History (Stage 2)
24
Blacklist ?Tracked News (Stage 2)
25
News Items Manipulation vs. System Performance
(Stage 3)
26
Tracked News ?Blacklist (Stage 3)
27
Tracked News ?History (Stage 3)
28
Blacklist ?Tracked News (Stage 3)
29
User Performance Analysis
30
User Precision
31
User Avg. Rank of Selected Items
32
User Feedback Analysis

• A two-way ANOVA was performed on the
  questionnaire data to examine significant
  differences in user answers by system and by
  stage.
• On the question 3, subjects indicated they
  trusted in systems ability to find useful
  information for the US topic versus the Business
  topic in overall (p-value 0.017).
• On the question 7, subjects indicated My Profile
  helps them to understand how the system finds
  useful news items for the US topic versus the
  Business topic in overall (p-value 0.013).

33
Future Work

• Open Model with explicit feedback did not
  outperform the baseline.
• The experiment indicates that without caution,
  user model manipulation not only benefit the
  performance but lower the output.
• Binary rating might not be a suitable way.
• Fuzzy rating is a good way to study further.

34
Q A