Discovering Relevant Concepts for Web Image Search

1
Discovering Relevant Concepts for Web Image Search

• Ruey-Cheng Chen1, Pu-Jen Cheng2
• 1 Institute of Information Science, Academia
  Sinica
• 2 Department of Computer Science and Information
  Engineering, National Taiwan University

2
Outline

• Motivation
• LiveImage
• Goal, Idea, Problem
• Features
• Relevant concept generation
• Results
• Challenges
• Conclusion

3
Motivation

• The study of Web image queries
• Pu Pu03 found users tend to submit simple
  queries to search engines, and, normally, the
  users also fail to refine them.
• Why?
• Users may not have a clear idea of what they are
  looking for.
• Users may not expand their queries as
  effectively as that in textual search.

4
Motivation
Large images available on the Web
It is time to organize image search result to
help users find what they want.
5
Goal

• A meta search engine
• Organizes image search results in a
• conceptually meaningful way

6
Idea

• Rather than organize images directly
• The result is composed of images obtained for a
  querys relevant concepts

7
Problem to be solved

• Generate relevant concepts for a query

meta search
8
Features
building
9
Features
It provides users a convenient and simple way to
retrieve images from the Web
10
Generate Relevant Concepts

• From image query log
• Q?XQ
• Q query, X modifier
• building ? office building, tall building, ...
• We call this relevant concept elaboration
• Simple method, promising results
• But
• If Q is an infrequent query
• ? Hard to find XQ (or QX) in a query log
• Sparseness Problem

11
Handle Sparseness Problem

• An observation compare
• Taiwan
• United States
• Japan

12
Handle Sparseness Problem

• Assume Japan doesnt exist in the log
• We can borrow modifier used in Taiwan and
  United States
• Because they are in the same concept group (i.e.,
  country)
• We call it relevant concept inheritance
• Difficulties
• Find out conceptually similar terms
• Filter out improper inherited result
• e.g., Taipei Taiwan ? Taipei Japan (improper)

13
Preliminary Results

• Pros of LiveImage
• Help users understanding the concepts related to
  the query
• Concepts along with corresponding images help
  users refining the query
• Cons of LiveImage
• Hard to generate relevant concepts for
  long/infrequent queries and name entities
• Longer response time

14
Evaluation Settings

• Subject 10 volunteers (experienced in image/text
  search)
• Queries 30 predefined
• balanced based on unique/non-unique, seen/unseen
  (log file), and query type

15
Eval Task 1 Quality of Relevant Concept-Class
(Text)

• Score 1(least relevant) 5(most relevant) for a
  query and its relevant concept-class
• Example

16
Eval Task 1

• Non-unique gt unique
• Seen gt unseen

17
Eval Task 2 Relevant Images Finding

• Score 1 5 relevance for a query and its
  relevant images

18
Eval Task 2

• The same trend compare with Task 1, but higher
  average
• Users have higher tolerance for image relevance
  test
• Some modifier terms, e.g., wallpaper are too
  generic to help users organize query results

19
Eval Task 3 Image Search Engine Comparison

• LiveImage v.s. Google Images
• evaluating 3010 queries via questionnaire

20
Eval 3

• Google Images same
• LiveImage greatly improve browsing experience
  for non-unique queries

-NE unique query -Common non-unique
21
Eval 3

• Pros of GI response time, easy to use, rendering
  of search results
• Pros of LI image browsing, semantic organization

22
Difficulties of Evaluation

• How to evaluate user experience?
• whether users want a clustering-with-text-label
  image search service?
• Compare with Google is unfair, who to be
  compared?
• Unbiased questionnaire
• How to setup a convincing evaluating environment?

23
Challenges

• Effectively handle infrequent/long queries
• Find a better approach to deal with name entities
  (specific queries)
• Enhance ranking/filtering mechanism

24
Conclusion

• A new direction of image search
• Organize images in a conceptual meaningful way
• LiveImage provides users a convenient and simple
  way to retrieve images from the Web

25
Conclusion

• Key features of LiveImage

• Compose results by corresponding images of
  relevant concepts
• Juxtapose concept annotations and images
• Click-in search query refinement

26
References

• Hsiao-Tieh Pu, 2003. "An Analysis of Web Image
  Queries for Search." ASIST 2003, Long Beach,
  California, U.S.A., pp. 340-348.
• Shuo-peng Liao, Pu-Jen Cheng, Ruey-Cheng Chen and
  Lee-Feng Chien, "LiveImage Organizing Web Images
  by Relevant Concepts," In the Workshop on the
  Science of the Artificial 2005, pp. 210-220,
  Hualien, Taiwan, December 2005.

27
Thank you!

• http//wkd.iis.sinica.edu.tw/LiveImage