Do Acronyms of Higher Educational Institutions in Hungary Identify Their Own Institutions on the Web?

1
Do Acronyms of Higher Educational Institutions in
Hungary Identify Their Own Institutions on the
Web?
Adrienn Skrop University of Veszprem skrop_at_dcs.vei
n.hu
WWW 2004 Workshop on Measuring Web Search
Effectiveness the User Perspective New York,
May 18, 2004
2
PROBLEM STATEMENT
Many people in Hungary use the Web to obtain
information from public institutions and
organizations. Users typically do not know the
URL of the desired institutions home page, they
use a Web search engine and the acronym of the
institution is used as query to get there. The
ability of the acronyms of Hungarian
institutions names to find the home page of
their own institutions when being used as queries
in Web searching is investigated.
3
(No Transcript)
4
MEASURES
The first ten hits returned are evaluated for
every acronym and search engine. Every hit is
assigned to exactly one of the following two
categories

• Category_1 link to the home page of the
  institution. This is the Web page that is desired
  to be retrieved when the institutions acronym is
  used as a query.
• Category_2 link to a page or site page (i.e., it
  is not the home page) that contains a site map or
  a navigational link to the home page.

5
Pseudo-precision
For every acronym a, a measure called
pseudoprecision, denoted by ?a, is defined as
follows

• N the total number of search engines used
• ra the number of search engines that return
  Category_1 links

6
Reciprocal Rank
For every acronym a and every search engine i, a
measure called reciprocal rank, denoted by RRia,
is defined first as follows

• ria the rank of the link in the hit list
• ? a penalty factor

7
Mean Reciprocal Rank
A mean reciprocal rank, denoted by MRRa, for
every acronym is defined as follows

• N the number of search engines used

8
Experimental Setting

• Experiments were carried out during March 2004.
• A number of 191 institutions that have acronyms
  and are present with their own Web site were
  identified.

9
Search Engines Used

• Heuréka http//www.heureka.hu (Hungarian)
• Altavizsla http//www.altavizsla.hu (Hungarian)
• Ariadnet http//www.ariednet.hu (Hungarian)
• Google http//www.google.com (general)
• Metacrawler http//www.metacrawler.com (general)
  
• AltaVista http//www.altavista.com (general)

10
Pseudo-precision histogram for Hungarian, for
general and for all search engines
Average ? of institutions Hungarian
search engines 0.3 General
search engines 0.36 All
search engines
0.33
11
Mean reciprocal rank histogram for Hungarian, for
general and for all search engines
12
Possible Causes of Low Performance, Conclusions

• Poor Web page design
• Content information
• Page-related factors
• Overall Web page design
• ...