Methodologies for Understanding Web Use with Logging in Context

1
Methodologies for Understanding Web Use with
Logging in Context

• Don Turnbull
• University of Texas at Austin
• School of Information

2
A set of methods

• Studying users with browser logs, interviews
  survey questionnaire
• Studying Web usage by collecting client browser
  trace logs, intranet server firewall or proxy
  logs
• A system to collect and analyze Web use via proxy
  logs that classify Web pages by content
• Each one offers different insights features
  customized log collection to verify accuracy

3
Contextual Methodology

• Goal to understanding Information Seeking
• 1. Develop a behavioral model of information
  seeking on the Web.
• 2. Design an operational methodology for
  measuring information seeking on the Web.
• 3. Learn about perceptions of Web resources.
• 4. Discover metrics of Web use.
• 5. Use field data.
• 6. Build upon study results for augmenting Web
  use.

4
Contextual Methodology 2

• Survey Questionnaire
• Web Use Data
• WebTracker
• History Files Server Logs
• Bookmarks Printouts when provided
• Follow up Interviews

5
Issues Collecting Web Client Data

• Modified client
• Total Control
• Non-native to Users (spyware)
• Bookmarks, Tagged sites, Search tools
• Chosen Web sites - personal information space
• Most valuable data file on users system
• Automatically organizing bookmarks
• Browser-native logs
• History files or other caching mechanism

6
WebTracker Expanded Window
7
Data Analysis

• Log files tabulated into spreadsheets
• Examined for clusters or patterns of behavior
• Selection of episodes of Information Seeking
  behavior
• a highlighting of the episode by the participant
  during the personal interview
• evidence of the episode having consumed a
  relatively substantial amount of time and effort
• evidence that the episode was a recurrent
  activity.
• Determined the modes of scanning moves
  exercised by the participants

8
Log Validation Methodology

• Tweaked Firewall Logs
• Intranet Web Server access.logs
• Validating log quality by referring to the other
  logs the type of use they reveal
• Larger datasets possible
• One Organization, Longer Duration
• Open-ended Interviews IT Survey
• More Quantitative Modeling
• Glassman (1994)
• Catledge Pitkow (1995)
• Tauscher Greenberg (1997a, 1997b)
• Huberman, Pirolli, Pitkow, Lukose (1998)

9
Proxy Logs for Topic Identification

• Proxies as filtering systems can provide insights
  into Web user interest when combined with Web
  page classification topic detection
• Historical record (archive) of Web use
• OpenChoice Web Page Classification System
• Use as internet content filter by analyzing usage
  logs classifying content
• Matching to known categories or taxonomies
• Using blacklists whitelists
• A different kind of insight into Web use

10
Standards for Logging Analysis

• We need some standard methods for Web user
  studies
• Often difficult or impossible to compare data
  results
• Common tools
• The Wrapper
• Firewall Proxy systems ( config files)
• Statistical analysis scripts
• R, SPSS, RDBMS
• Graphical display
• GNUPlot,

11
Questions Comments?

• Thank you
• Don Turnbull
• donturn_at_ischool.utexas.edu
• http//www.ischool.utexas.edu/donturn

12
A New Kind of (Empirical) Science?

• When you combine technology and the network
  effect there is a new kind of empiricism
• Relying on or derived from observation or
  experiment empirical results that supported the
  hypothesis.
• Verifiable or provable by means of observation or
  experiment empirical laws.
• Quant the Network Effect
• Bob Metcalfe and the value of fax machines
• Stanley Milgram and Six Degrees of Separation
• Duncan Watts () and Social Networks
• Data collection scales up
• Different types of data
• Statistics Algorithms to describe behavior
• New measure of precision and significance

13
Quantitative Research

• Bibliometrics
• Webometrics
• Web Use studies
• Characterization
• Behavioral
• Application (use)
• Usability but, not observational (the User Zoo)
• The scale of digital environments transform
  quantitative data into qualitative data

14
WebKDD Study

• Initially used WebTracker software to gather
  insight into Web use
• Network data collection of Web use
• 3000 more data about Web Use
• Longer study period, lots more users
• Many orders of magnitude of data to analyze means
  subtle patterns may be discovered
• Substantive evidence of patterns of behavior
• Larger than all previous studies of
  organizational Web use combined