You want me to do what

1
You want me to do what?

• The Transition from Distributed Searchingto
  Unified Federated Searching
• A somewhat light-hearted view offinding a new
  model for accessto scholarly resources
• Library Council
• April 27, 2005

2
Weve come a long way

• We have come a long way since 1989 when we
  introduced our first system for searching
  indexes.
• Today we offer a vast array of electronic
  resources using many systems.
• The expectations of our users have changed as
  well.

3
Abstract Index Searching

• Today we offer students and faculty
• 200,000,000 citations
• 500 titles
• 350 are from 50 systems with multiple databases
• 150 are direct from the vendor
• Over 200 interfaces
• All of them are different!

4
You want me to do what?

• I can search a billion web pages with Google,
  and you expect me to learn touse 200 systems to
  search the library stuff?
• If you were a student today,wouldnt you have
  the sameresponses to what we expect?

Fear
Frustration
5
The Impossible Dream

• Our goal is to provide one place for students and
  faculty to go for all scholarly resources
• for many reasons this is The Impossible Dream
• but we are working toward this goal

"The Impossible Dream" from MAN OF LA MANCHA
(1972) music by Mitch Leigh and lyrics by Joe
Darion To dream the impossible dream To fight
the unbeatable foe To bear with unbearable
sorrow To run where the brave dare not go To
right the unrightable wrong To love pure and
chaste from afar To try when your arms are too
weary To reach the unreachable star.MP3 file
courtesy MGM
6
Distributed Searching
Many interfaces Many search engines Many
databases Many search results
7
Unified Searching

• Single interface
• Single search engine
• Many databases
• Single search result

8
Unified Searching

• helps turn these reactions into these

9
Unified SearchingGood theory but impossible to
accomplish

• Searching everything on one system
• is not a realistic goal
• No aggregator has everything
• Many database vendors wont allow local loading
• is what Scholars Portal Search aims to do
• it uses this model but not for everything
• at best it will have only
• 25 of our titles
• 50 of our citations
• so, what do we do about the rest?

10
Federated SearchingBringing databases into a
uniform framework

• Searching everything with a single interface
• Federated searching comes close to doing this
• What is federated searching?
• A system used to
• simultaneously execute searches across many
  different information sources
• collect and collate the responses
• present the results in a single list

11
Federated Searching

• Can help turn the grumpiest of users from this
  to this

12
Federated Searching
Single interface Search processor /
agent Multiple search engines Multiple
databases Single search result
Search onnative interface
13
Unified vs Federated Searching
Unified
Federated
Single database with a unified indexing structure
Many different databases each with its own
indexing
Database Indexing
There is a single result set for the database
There is a merged result set with the top ?
citations from each database
Search Results
All citations are ranked using the same
algorithm
Ranking is done within each database with the
ranking algorithm of the system being
searched Undisplayed results from one database
may be more relevant to the query than the
displayed results from another database
Relevancy Ranking



Deduplication of Records

Search Performance
14
Unified vs Federated Searching
Unified
Federated
Single database with a unified indexing structure
Many different databases each with its own
indexing
Database Indexing
There is a single result set for the database
There is a merged result set with the top ?
citations from each database
Search Results
All citations are ranked using the same
algorithm
Ranking is done within each database with the
ranking algorithm of the system being
searched Undisplayed results from one database
may be more relevant to the query than the
displayed results from another database
Relevancy Ranking


Deduplication of Records

Search Performance
15
Unified vs Federated Searching
Unified
Federated
Single database with a unified indexing structure
Many different databases each with its own
indexing
Database Indexing
There is a single result set for the database
There is a merged result set with the top ?
citations from each database
Search Results
All citations are ranked using the same
algorithm
Ranking is done within each database with the
ranking algorithm of the system being
searched Undisplayed results from one database
may be more relevant to the query than the
displayed results from another database
Relevancy Ranking

Because data is loaded in a single data
structure with the same database definition, the
masking of duplicate records is reliable
Because data is loaded on different systems with
different database definitions, the masking of
dupli- cate records less reliable
Deduplication of Records
Searching is done on a single system on which
performance can be managed
Searching is done on multiple systems on which
performance is managed by each target system
Search Performance
16
How Federated Searching Works
Target Database
User enters query
Target system searches database
Search agent sends query to target
Target returns top citations
Search agent collects results
Search agent merges results
Search agent sends results to user
Search agent ranks or orders results
17
Federated Search Products

• Product Vendor
  Based on
• AGent Auto-Graphics
• Chameleon VTLS Muse Global
• ERA Endeavor Muse Global
• Horizon Portal Dynix WebFeat
• MetaLib Ex Libris
• MultiSearch CSA Muse Global
• SingleSearch Sirsi Muse Global
• WebFeat 3 WebFeat
• ZPORTAL Fretwell-Downing

18
Muse Global

• Muse Metasearch
• provides a single action for simultaneous
  searching across a diverse set of content sources
• gives searchers a single sign-on for
  authenticated access to local and remote
  resources, hiding the complexity of searching
  multiple repositories
• returns results in a consistent, uniform format
  which can then be sorted, filtered and
  de-duplicated
• The Muse Global market is not libraries it is
  information system vendors

19
Our goal should be

• to help our users find their way through the
  maze of resourcesthat we offer
• and to make them happier with searching

20
One Size Does Not Fit All

• Federated searching offers our users convenience,
  
• but as effective as it is
• we must recognize that
• it is not a panacea
• it has many limitations
• some users will need native searching

21
It is all about the user

• From our users do we want
• or
• or do we want ?

22
Questions
23
Demonstration

• Things to keep in mind
• You will be seeing CSAs MultiSearch
• We will not be able to link
• to the full text of articles or
• to the native interfaces
• Disclaimer
• I just got the test account so I can show it
  only ata superficial level and may not be able
  to answer detailed questions

24
Any more questions?
25
Not embracing new solutions is todays equivalent
of burying our heads in the sand
26
Put yourself in the shoes of the user