Collaborative Filtering: Possibilities for Digital Libraries

1
Collaborative FilteringPossibilities for
Digital Libraries

• Jon Herlocker
• Janet Webster
• Seikyung Jung
• Oregon State University

2

• Current search engines are insufficient.

3
Two important search engine problems

• They dont understand
• Quality
• Context

4
But First Our Context

• Why are we standing up here?
• We think we can improve the digital library
  experience.

5
Todays Context

1. Research questions hypotheses
2. Collaborative filtering
3. Our approach to CF in the Library
4. Challenges of collaborative filtering for library
   search
5. Initial lessons learned

6
The Librarians Questions

• As electronic information increases in amount and
  value, how to provide access to it?
• How to change digital libraries from disconnected
  collections to integrated systems?
• How to integrate the expertise of librarians into
  the development process?
• How to adapt traditional library values to new
  opportunities?

7
The Computer Scientists Questions

• What is the next big leap in document search
  technology?
• How to overcome the limitations of softwares
  ability to understand language?
• How can we build a search engine that learns by
  observing searchers?

8
Our Research Hypotheses

• Enabling the entire community to participate in
  organizing and recommending information will add
  value to the digital library
•  In other words Collaborative Filtering will
  increase the value of a digital library

9
What is Collaborative Filtering?

• Communities of people sharing their evaluations
  of content
• Recommendations are transferred between people of
  like interest
• Examples
• MovieLens.org
• Epinions.com
• Launchcast (launch.yahoo.com)
• Amazon.com

10
CF and Libraries

• Search is central to user experience of digital
  library
• Collaborative Filtering
• Could overcome the limitations of current search
  technology
• CF already exists in libraries.
• Not search, but cataloguing (OCLC)
• Adapting CF for document searching is not
  trivial.
• Information needs are dynamic.

11
Our Approach

• OSU Libraries Recommender System
• Perform at CF at query level
• Match similar queries in addition to similar
  users
• Generate results based on past user
  recommendations
• Infer recommendations from user behavior
• Integrate with existing library systems and
  traditions

12
(No Transcript)
13
(No Transcript)
14
(No Transcript)
15
The Benefits of CF

• Quality is considered.
• Recommendations are based on human evaluations.
• Context is considered.
• The system gets better as its used.
• Doesnt require significant, centralized human
  resources

16
CS Challenges

• How to collect evaluations?
• How to identify the useful element of
  recommendations?
• How to represent the information needs of
  searchers?
• How to rank results?
• How to design the interface?

17
Library Challenges

• How to balance privacy with personalization
  involvement?
• How to maintain authority of recommended
  information?
• How to deal with timeliness of information?
• How to integrate with existing library systems?
• How to fund research in the library setting?

18
What Weve Learned

• Weakness of old search technology affects
  perception of new
• Wrapper technology minimizes IT commitment
• Existing internal data can be used to jumpstart
  system
• Controlled experiments show
• Increased performance
• Increased perception of non-tangibles

19
CF and Digital Libraries

• Helps handle more electronic information
• Improve search results
• Shapes direction of digital libraries
• Supports collaboration on many levels
• Nothing ventured, nothing gained.

20
Funding

• OSU Libraries Gray Chair for Innovative
  Technologies
• National Partnership for Advanced Computing
  Infrastructure (NSF)
• Georgia Pacific HMSC internship

21
More information

• Silence of the Sleeper
• Malcom Gladwell, The New Yorker, October 4th,
  1999 (gladwell.com)
• System for Electronic Recommendation Filtering
  Prototype (SERF) for OSU Libraries
• http//dl.nacse.org/osu

22
Contacts

• Janet Webster
• Oregon State University Libraries, Hatfield
  Marine Science Center
• janet.webster_at_oregonstate.edu
• Jon Herlocker
• Oregon State University, School of Electrical
  Engineering Computer Science
• herlock_at_eecs.oregonstate.edu