Title: Exploring ClientSide Instrumentation for Personalized Search Intent Inference: Preliminary Experimen
1Exploring Client-Side Instrumentation for
Personalized Search Intent InferencePreliminary
Experiments
- Qi Guo and Eugene Agichtein
- Intelligent Information Access Lab
- Mathematics Computer Science
2Outline
- Introduction
- CSIP Client-side Intent Predictor
- Evaluation of CSIP
3Outline
- Introduction
- CSIP Client-side Intent Predictor
- Evaluation of CSIP
4Introduction
- Recovering user intent is an important yet
difficult problem - Traditional methods typically model a single
intent for the same query - Navigational/Informational/Transactional
- However user goals can vary a great deal
5An example, query obama
- Informational People may search to know more
about Barak Obama - Navigational visit his official website
- Transactional perhaps the user goal is to donate
money online to support Mr. Obamas campaign
6Introduction (cont.)
- Incorrect to classify the query into a single
intent - What really necessary is to classify user goals
for each query instance
7Goals
- Infer Personalized intent for each query instance
using Client-side instrumentation - Therefore, provide tailored user experience
- Focus
- mouse movements
- Query intent classification into
navigational/informational/transactional
8Outline
- Introduction
- CSIP Client-side Intent Predictor
- Evaluation of CSIP
9Outline
- Introduction
- CSIP Client-side Intent Predictor
- Evaluation of CSIP
10CSIP Client-side Intent Predictor
- Capture as much information as possible
- Model implicit user feedback based on the
real-time interactions - Keep light-weight and scalable
11CSIP Client-side instrumentation
- Implementation within the LibX Toolbar
(http//www.libx.org) - JavaScript code to track real-time interactions
(eg. mouse movements) - Installed on public-use shared machines in Emory
University Libraries - All participated users opted in, and no directly
identifiable information was stored
12Our approach Learning to recover intent
- Represent full client-side interactions as
feature vectors - Apply standard machine learning classification
methods
13CSIP System Overview
Figure1. Overview of CSIP
14CSIP Query text
- Traditional feature for inferring user intent
- Query length
15CSIP Other User/Server-Side Clickthrough
Features
- Click distribution
- Average deliberation time
- Similarity between a clicked search result URL
and the query
16CSIP Real-Time Interaction/Client-side
instrumentation
- Focus on the mouse movements
- 1. CS Client Simple
- 2. CF Client Full
17CS Client Simple
Horizontal range
- First representation
- Trajectory length
- Horizontal range
- Vertical range
Trajectory length
Vertical range
18CF Client Full
- Second representation
- 5 segments
- initial, early, middle, late, and end
- Each segment
- speed, acceleration, rotation, slope, etc.
1
2
3
4
5
19Outline
- Introduction
- CSIP Client-side Intent Predictor
- Evaluation of CSIP
20Outline
- Introduction
- CSIP Client-side Intent Predictor
- Evaluation of CSIP
21Experimental Setup
- Dataset
- Gathered from mid-January 2008 until mid-March
2008 from the public-used machines in Emory
University libraries. - Consist of 1500 initial query instances/search
sessions - Randomly sample 300 initial query instances
- Behavioral pattern for follow-up queries might be
different
22Creating Truth Labels
- Difficulty no identifiable user information
- How to recover the Truth?
- Use our best guess based on clues
- Query terms
- Next URL (eg. clicked result)
- How user behaves before click/exit
23Navigational query facebook
24Informational query spanish wine
25Transactional query integrator
26Intent Statistics in Labeled Sample
27Task 1 Classify a query instance into
Navigational / Informational / Transactional
CSIP gt CF gtgt CS gt S
28Task 2 same, but not distinguish
betweenTransactional and Navigational queries
All improved. Still, CSIP gt CF gtgt CS gt S
29Most Important CSIP features
30Error Analysis
- CSIP can help identify
- Relatively rare navigational queries (re-finding
queries or queries for obscure websites) - Informational queries that resemble navigational
queries (coincides with a name of a website)
31Summary
- Presented CSIP, a practical lightweight
client-side instrumentation for web search - Demonstrated the feasibility by the experiments
with real user interactions - Conducted preliminary result analysis exploring
the benefits of client-side vs. server-side
instrumentation
32Future Work
- Incorporate user history modeling
- Develop tailored machine-learning algorithms
- Apply our methods to other tasks such as
predicting user satisfaction or query performance
33Thank you!
- More information
- http//ir.mathcs.emory.edu/intent/
34Related Work
- The origins of user modeling research can be
traced to library and information science
research of the 1980s. - Previous research on user behavior modeling for
web search focused on aggregated behavior of
users to improve web search
35Related Work (cont.)
- Previous studies have primarily focused on
indicators such as clickthrough to disambiguate
queries and recover intent and model user goals. - Recently, eye tracking has started to emerge as a
useful technology for understanding some of the
mechanisms behind user behavior - Correlation between eye movements and mouse
movements