Question Answering Systems

1
Question Answering Systems

• Their Most Famous Applications

2
Question Answering

• Question answering (QA) is a task where the
  information need of the user is formulated as a
  natural language question and where the answer is
  given in natural language as well.
• In general, the length of the answer varies from
  one word to a couple of sentences or a document,
  depending on the type of the question and the
  system it self. Also those situations where the
  system is not able to provide an answer should be
  detected.

3
QA Domains

• QA systems can be designed to work on a specific
  domain only (e.g. an aid for a company help desk)
  or they can be general purpose systems (e.g.
  AskJeeves).
• Resources
• In general, question answering systems use
  unstructured text documents as their database,
  but in addition, they can use lists of FAQs
  (Frequently Asked Questions) and structured
  databases. In open domain QA systems, the Web is
  a major source of information.

4
Cross-Language QA

• Cross-language question answering means that the
  question is expressed in another language than
  that in which the documents from which the answer
  is extracted are written.
• Cross-language QA is usually implemented either
  by first applying machine translation (maybe
  unavailable for all languages or unreliable due
  to questions info. altering) to the question and
  then passing it on to a monolingual QA system or
  by integrating cross-language processing into the
  QA system.

5
Understanding "Keyword" Queries

• In order to understand natural language
  queries, we need to have an idea about keyword
  queries. These are the words (or phrases) which,
  if found, are most likely to provide the
  information for which you are searching.
• One of the most popular keywords based search
  engines is Google. Google is a straightforward
  search engine which is very easy to use. It
  returns pages based on the number of sites
  linking to them and how often they are visited,
  indicating their popularity.

6
Things You Need to Know About Google

• When listing keywords, Google defaults to 'AND',
  meaning that if you list 3 keywords, it will
  assume you want pages which only contain all
  three words.
• Google only searches for pages that exactly match
  your search, so try arranging your terms in
  different ways.
• So your query cheapest airline tickets is
  interpreted as
• "Show only pages which include the words cheapest
  AND airline AND tickets

7
"Natural Language" queries

• In an attempt to help you feel at home with their
  site, some search sites encourage you to approach
  your search like you might if you went to a
  library and asked the reference librarian for
  information. You visit their site, and type your
  question into the search box.
• Why use 'Natural Language' for your queries?
• It's easier. You don't have to provide "keywords"
  . Instead, you type your question in your
  'natural language', the language you might use
  when asking a question of a knowledgeable adult.

8
How Does Ask Jeeves Work?

• Ask Jeeves technology is a four-step process that
  provides fast and accurate answers. It works like
  this
• Ask Jeeves attempts to understand the precise
  nature of the question by using a
  question-processing engine. The engine attempts
  to understand both the meaning of the words in
  the question and the meaning in the grammar of
  the question.
• The answer-processing engine offers a list of
  questions (which users see). This is a little
  like saying, "If you meant this...the answer is
  this OR If you meant this other, try this other
  option."
• ...cont

9
How Does Ask Jeeves Work?

• The Ask Jeeves knowledgebase contains links to
  more than 7 million answers, which contain
  information about the most frequently asked
  questions on the Internet. So, for example, if
  you ask "What is the population of San
  Francisco?", the knowledgebase probably already
  has the answer and can point you to it.
• Ask Jeeves also has partners who provide answers,
  to supplement the answers Jeeves finds. This
  guarantees that the user finds the best answer by
  including the top ten answers from other leading
  search engines. Ask Jeeves searches these answers
  as well, to eliminate duplication.

10
"Natural Language" queries Examples

• Here are some ways (you see, what's 'natural' to
  you might not be 'natural' to another!
• Where can I find information about restaurants in
  Amman, Jordan?
• Show me a list of restaurants in Amman, Jordan?
• How many restaurants are there in Amman, Jordan?
• What are the best 5 restaurants in Amman, Jordan?
• Amman Jordan restaurants you can ask sentence
  fragments too!

11
Will 'Natural Language' queries work on all
Search sites?

• No. . . Well, actually, you could type your query
  in natural language, but the result would be
  unproductive. Many search sites will look for
  every word in the question and produce matches.
  so you will typically be overwhelmed by the
  number of pages which are returned and frustrated
  that many of the pages have little or no
  relevance to what you are trying to find.
• Only use natural language queries when the search
  site specifically invites and/or recommends that
  you do so. Definitely keep Ask Jeeves in mind
  when you are looking for answers to questions
  which you would bet other Web users have also
  asked.

12
AnswerBus

• AnswerBus is an open-domain question answering
  system based on sentence level Web information
  retrieval. It accepts users' natural-language
  questions in English, German, French, Spanish,
  Italian and Portuguese and provides answers in
  English. Five search engines and directories
  (Google, Yahoo, WiseNut, AltaVista, and Yahoo
  News) are used to retrieve Web pages that are
  relevant to user questions. From the Web pages,
  AnswerBus extracts sentences that are determined
  to contain answers. The performance of AnswerBus
  in terms of accuracy and response time is better
  than other similar systems.

13
AnswerBus

• Figure 1 describes the working process of
  AnswerBus. A simple language recognition module
  will determine whether the question is in
  English, or any of the other five languages. If
  the question language is not English, The module
  will send the original question and language
  information to AltaVista's translation tool
  BabelFish, and obtain the question that has been
  translated into English.

14
Figure 1 Working process of AnswerBus
15
Answer Forms
16
Query-Sentence Matching Formula

• In any language, a sentence is not a simple
  combination of a set of words. It is a string of
  words with logical and lexical structures. Thus,
  oftentimes, in real life, a person does not need
  to hear every word in a sentence in order to
  effective capture its meaning. AnswerBus uses
  following experimental matching rule to determine
  whether a sentence is an answer to the question
• It is not necessary for a sentence to match every
  word in the question in order to be an answer
  candidate.

17

• 200 questions were used to evaluate AnswerBus's
  question answering performance and also compare
  its performance to that of four other similar
  systems. Table 2 presents the performance of
  AnswerBus and other four systems. It provides the
  numbers of correct answers in the systems' top
  five and top one answers, the standard NIST
  scores, the maximal, minimal and average response
  times measured in seconds, standard deviation of
  response times, and the average lengths of
  returned answers.

18
Table 2 Performance of Online Question Answering
Systems
19

• As a conclusion, in comparison to other Web-based
  QA systems that are currently accessible on the
  Web, AnswerBus demonstrates a higher accuracy of
  question answering and a faster speed. The
  enhanced performance can be attributed to mainly
  three features of the system.
• Every module of the system, excluding the
  language translation, has been written by the
  same author, with the same architecture and
  programming language. The system was designed to
  perform with high efficiency.
• AnswerBus adopted a number of new techniques,
  including, QA specific dictionary,
  sentence-question matching formula, light NLP
  components, and dynamic named entities
  extraction. These techniques were found to
  enhance the system's performance.
• AnswerBus also exerted effort to enhance the
  techniques that have been widely used in other QA
  systems, for example, question-type detection
  module, question-sentence matching, search engine
  selection, search engine specific query forming,
  pseudo-answer detection, and HTML sentence
  parsing.