Companions: Explorations in Machine Personality

1
Companions Explorations in Machine Personality

• Yorick Wilks
• Computer Science, University of Sheffield
• and
• Oxford Internet Institute, Balliol College

2
The talk will not start in quite the place you
may expect but this (serendipitous) quote will
give you an idea of where we are going
..claim the internet is killing their trade
because customersseem to prefer an electronic
serf with limitless memory and no conversation.
(Guardian 8.11.03)

• Kings College, London
• 11 December, 2003

3
What the talk contains

• Two natural language technologies I work within
• Information extraction from the web
• Human dialogue modelling
• The new vision of the Semantic Web and its
  relation to
• Ontologies
• Agents
• Conversational agents as essential for
• personalizing the web
• making it tractable
• Companions for the non-technical as a cosier kind
  of agent

4
The underlying technology Human Language
Processing by computer (HLT or NLP)

• Whose Research Issues are not solved, but..
• It is generally agreed what the issues are
• Information access issues (Semantic Web..)
• Dialogue and interface issues
• Information presentation issues
• We shall touch only on the first two
• Good working versions of these technologies exist
  now and are increasingly being deployed, though I
  cannot give any full evidence of that today.

5
(No Transcript)
6
(No Transcript)
7

• What then is Information Extraction?
• getting information from content of huge
  document collections by computer at high speed
• looking not for key words but information that
  fits some template pattern or scenario.
• delivery of information as a structured database
  of the template fillers (usually pieces of text)
• the technology has now moved on to one based on
  machine learning (ML) rather than people writing
  these patterns down out of their heads.
• it has fused with machine Question-Answering.
• it is a technology created since 1990 by the US
  Defense Department

8

• A new core text processing technology

9
IE a new core text processing technology

• The Web text world is huge and expanding (now
  maybe 300 bn. pages in English alone)
• Text will not be bypassed by computer speech as
  seemed possible a decade ago
• Critical data will continue to come as text, and
  all the PAST is text
• Think about processing Japanese company finance
  reports (all on the same day!) without IE.

10
The semantic web

• Unexpected outcome of the text tagging movement
  from the Humanities! SGML--HTML--XML,
  VRML,voxML, anythingML)
• Purpose is making the web comprehensible for
  software agents (Berners-Lees dentist example)
• See this as a process of inverting IE doing
  markup now at author time not analysis time.
• This is an old dream of AI/NLP revived that
  information be stored and accessed by CONTENT
  (not by text string or index).

11
The Semantic Web is now another name for
Knowledge Management

• The Semantic Web (SW) will allow the Internet to
  consist not only of documents but also
  machine-readable structured data. Key application
  areas are
• Business apps B2B and grid applications
• Personal assistants to permit effective
  management of personal data, activities and
  interests
• Agents/Personal assistants will display real
  power of the Semantic Web
• Some analysts now argue that up to 50 of the
  value of companies is contained in knowledge
  intangibles that will find their way into its
  Knowledge Management structure.

12
Agents Interacting

• Fundamental justification for the SW is for
  software agents to use machine-readable data in
  order to perform tasks
• Scenario (from Berners-Lee et al.2001) for
  establishing physical therapy appointments

13
Ontologies are the structures of knowledge that
SW agents will consult

• Each agent must have internal representation,
  i.e. ontology, knowledge base or data base
• For each new piece of info, knowledge update or
  maintenance must take place
• The system must determine the consequences of the
  change in state and this raises the Frame Problem
• For agents to interact, events or new information
  trigger communication, BUT who should an agent
  communicate the info to? All? None? Some? If
  some, how does the system decide?

14
Ontologies are really just an old AI idea back in
a new form

• A philosophical word pressed into duty to serve
  for what used to be called Knowledge
  Representation in AI, usually a form of applied
  logic.
• The interacting agents of the Semantic Web bring
  back another old AI chestnut, the Frame Problem
  (remember spotting the aliens with questions
  about turtles in  Blade Runner ??)

15
Much novelty in the SW concept is classic
Artificial Intelligence

• The logic representation and reasoning
• The ontological hierarchies of knowledge
• These are all part of the A I schoolman
  tradition remember they are not new and we still
  have no real proof they are NEEDED.
• Accept them only with suspicion and wariness
  look how old they are---------

16
An early local AI man

• John Wilkins (1614-1672)
• A founder of the Royal Society
• Book An Essay Towards a Real Character and
  Philosophical Language, London, 1668
• An Ontology
• A Thesaurus
• A system of logical shorthand

17
(No Transcript)
18
The Need

• Ontologies and Taxonomies are needed for
• Ontologies for the Semantic Web
• Central component for agent services over the
  Web
• Knowledge acquisition for knowledge management
• Minds of employees Intangible assets
• Ontologies act as index to memory of an
  organisation
• Many organisations have built or are building
  their own ontologies/taxonomies (e.g. BBC,
  British Council, Clifford Chance, etc.---the
  majority of major UK companies say yes, if asked
  if they are).

19
What is an ontology--in its current sense?

• Widely-held Assumption knowledge can be codified
  in an ontology
• Ontology formal explicit specification of a
  shared conceptualisation (Gruber)
• a document or file that formally defines the
  relations among terms (Berners-Lee)

20
The way out?

• May be to learn these representational objects
  from scratch, from text, like we do..
• the only escape from the schoolmen??
• But the labour to write them all, to describe all
  of medicine formally, or all the knowldge of
  Rolls Toyceis inconceivable
• How to get ontologies from texts?

21
Text and Knowledge

• We want to generate ontologies from text
• But if an ontology shared/background knowledge,
  then a writer assumes the ontology to generate
  the text

Ontology
22
Background Knowledge is elusive in an explicit
form

• If ontology background knowledge,
• and background knowledge is implicit,
• then the text(s) will not express the domain
  ontology
• Less true of introductory textbooks, manuals,
  glossaries etc.
• We expect to find specification of the
  ontological knowledge at the borders of a domain
• Remember the old (John McCarthy, Stanford) AI
  goal of formally expressing COMMON SENSE
  knowledge, not expertise
• But, NLP techniques can extract plausible
  ontologies from pedagogic texts.

23
Change of topic dialogue with machine agents

• Leaving, for the moment, the internet and the
  semantic web of text, whose content has been
  made more explicit
• By means of ontologies as knowledge
  representations, and
• Whose services will be provided by
  agents---simple software packages with a
  purpose, like find me the best camera under 500.

24
Machine dialogue problems with theory

• It can be vacuous dialogues are systems of
  turn-taking
• Speech act analysis initially has led to
  implausibly deep levels of reasoning.
• For some researchers, dialogue theory is still a
  question of how best to deploy logic

25
Two important historical systems have all the
modern traits and functionalities in miniature

• Colbys PARRY (Stanford, 1971)
• Perrault, Cohen, Allens speech act system
  (Toronto, 1979)

26
Colbys PARRY

• Perhaps best ever performance, many users,
  robust, but not a normal subject (I.e. a
  paranoid)
• primitive individual models, some control of
  dialogue process but it had lots to say!
• Primitive simulation of intentionality
• not syntax analysis but fast pattern matching
• Far better than ELIZA

27
PARRY conversation

• Have you been hospitalized before?
• THIS IS THE FIRST TIME
• How long have you been there?
• ABOUT TWO WEEKS
• Any headaches?
• MY HEALTH IS FINE
• Are you have memory difficulties?

28
More PARRY

• NO
• Then can you tell me who is the President of the
  US?
• NIXON IS PRESIDENT
• And what day is today?
• TODAY IS THURSDAY--CAN WE GET ON WITH THE
  INTERVIEW?

29
How is this achieved in PARRY?

• WHAT IS YOUR MAIN PROBLEM _____________________
  __________
• WHAT BE YOU MAIN PROBLEM
• BE YOU MAIN PROBLEM
• WHAT BE MAIN PROBLEM
• WHAT BE YOU PROBLEM ?
• WHAT BE YOU MAIN

30
Perrault, Cohen, Allen system

• Based on speech act reasoning
• User must have one of two goals, meeting or
  catching a train
• Passenger/User Do you know when the Windsor
  train arrives?
• This is labelled as a REQUEST not a
  REQUEST-INFORM (Y/N) because the system knows the
  user knows it knows!

31
Perrault et al. At Toronto

• System has domain knowledge and reasoning power
• was the first to assign speech act labels to
  dialogue items
• But speech act reasoning is often implausible
  Can you pass the salt?
• It has a simple rigid model of nested belief
• but virtually no performance

32
Fixed nested beliefs passengers view of
systems view of passengers beliefs.
passenger
system
passenger
33
1970s division of approaches to machine
conversation.

• Domain-dependent systems with coded world
  knowledge and some parsing and reasoning VERSUS
• Wide shallow systems with little knowledge and
  high performance
• Published AI academic systems all in first group
• Only the second group performed at all

34
Academic systems have moved towards performance

• Best is Traums TRAINS system--descendant of
  Allens work (Toronto-Rochester tradition)
• Semi-empiricist uses corpora but retains
  reasoning
• gone to the movies in California at USC!!
• Also TRINDIKIT at Gothenburg/Edinburgh uses a
  large rule base
• Pressure from the Loebner competition to perform?

35
TRAINS CORPUSDavid Traum (Rochester)

• utt1 s hello can I help you
• utt2 u yeah I want t- I want to determine the
  maximum number of boxcars of oranges by
  seven a.m. tomorrow morning
• utt3 so hm
• so I guess all the boxcars will have to go
  through oran- through Corning because
  thats where the orange juice orange
  factory is
• utt4 so from Corning to Bath how far is
  that
• utt5 s two hours
• utt6 u and its gonna take us also an hour to
  load boxcars right
• utt7 s right
• utt8 u okay so hm so every
  trip will take at least three hours
  then
• utt9 um
• utt10 s right we can unload any amount of
  cargo onto a train in one hour
• utt11 so we can so a maximum of
  three boxcars in an hour

36
AI people join the performersthe Loebner
Competition

• Well designed competition over the last ten
  years---essentially a modified Turing test.
• Domain dependent entries till 1997, then domain
  free.
• The game is for journalists to separate and rank
  human and machine partners
• Search Loebner on web for full transcripts

37
Loebner 1994

• NEW JUDGE
• 150
  Judge 1
• Do you think the country is headed generally in a
  more conservative or more liberal direction right
  now?
• 151
  Terminal
• Didnt you say would you discribe that old texas
  billionaire as a conservative?? Answer me one
  question why are liberals portrayed as donkees?
  Are you a caring person? Youre not really
  talking about me, are you?
• 152
  Judge 1
• Ive got the feeling Ive heard these answers
  before. Is our nation swinging leftward or
  rightward right now?
• 
  Terminal
• We are at the beginning of a strong and
  wonderful economic recovery, and we can thank the
  liberals!

38
Sheffield University/Intelligent Systems won in
1997!

• The CONVERSE program pretended to be Catherine, a
  34-year old British journalist living in New York
  (to explain any usage problems).
• She could talk about 80 appropriate topics
• (80domain free???) clothes, shopping, food,
  abortion, fitness, sex.etc.

39
Loebner Competition 1997---Sheffields winning
entry!

• PROGRAM101804Hi, my name is Catherine, whats
  yours?
• JUDGE04101832My name is Elliott.
• PROGRAM101834Well, its nice to meet you,
  Elliott.
• PROGRAM101848Did you see that story on CNN
  last night about the lesbian
• PROGRAM101904couple who came out at a White
  House party on Sunday?
• JUDGE04101955Yes, I did. I think it may all
  be a publicity stunt for Ellen.
• PROGRAM102001Ellen Degeneres was one of them
  she was kissing her lover
• PROGRAM102011in front of Bill Clinton. Do you
  think he was very
• PROGRAM102025embarrassed?
• JUDGE04102100I think he want the gay vote. He
  will do anything to get it and keep it.
• JUDGE04102115He does not have a conservative
  base.
• PROGRAM102121
• PROGRAM102121Are you still a Friend of Bill
  Clinton?

40
The CONVERSE prototype 1997

• Push-me-pull-you architecture
• strong driving top-down scripts (80) in a
  re-enterable network with complex output
  functions
• bottom-up parsing of user input adapted from
  statistical prose parser
• minimal models of individuals
• contained Wordnet and Collins PNs
• some learning from past Loebners BNC
• It owed something to PARRY, nothing to Toronto.

41
Why the dialogue task is still hard

•  Where am I  in the conversation what is
  being talked about now, what do they want?
• Does topic stereotopy help or are just
  Finite-State pairs enough (VoiceXML!)?
• How to gather the beliefs/knowledge required ,
  preferably from existing sources?
• Are there distinctive procedures for managing
  conversations?
• How to learn the structures we need--assuming we
  do---and how to get and annotate the data?
• Some of this is the general NLP empiricist
  problem.

42
Dimensions of conversation construction the
Sheffield view

• Resources to build/learn world knowledge
  structures and belief system representations
• Quasi-linguistic learnable models of dialogue
  structure, scripts, finite state transitions etc.
• Effective learnable surface pattern matchers to
  dialogue act functions (an IE approach to
  dialogue)
• A stack and network structure that can be trained
  by reinforcement.
• Ascription of belief procedures to give dialogue
  act reasoning functionality

43
VIEWGENa belief model that computes agents
states

• Not a static nested belief structure like that of
  Perrault and Allen.
• Computes other agents RELEVANT states at time of
  need
• Topic restricted search for relevant information
• Can represent and maintain conflicting agent
  attitudes

44
VIEWGEN as a knowledge basis for
reference/anaphora resolutionprocedures

• Not just pronouns but grounding of descriptive
  phrases in a knowledge basis
• Reconsider finding the ground of
  that old Texas billionaire as
  Ross Perot, against a background of what the
  hearer may assume the speaker knows when he says
  that.

45
(No Transcript)
46
What is the most structure that might be needed
and how much of it can be learned?

• Steve Young (Cambridge) says learn it all and no
  a priori structures (cf MT history and Jelinek at
  IBM)
• Availability of data (dialogue is unlike MT)?
• Learning to partition the data into structures.
• Learing the semantic speech act interpretation
  of inputs alone has now reached a (low) ceiling
  (75).

47
Youngs strategy not like Jelineks MT strategy
of 1989!

• Which was non/anti-linguistic with no
  intermediate representations hypothesised
• Young assumes rougly the same intermediate
  objects as we do but in very simplified forms.
• The aim to to obtain training data for all of
  them so the whole process becomes a single
  throughput Markov model.

48
(No Transcript)
49
There are now four not two competing approaches
to machine dialogue

• Logic-based systems with reasoning (Old and still
  unvalidated by performance)
• Extensions of speech engineering methods, machine
  learning and no structure (New)
• Simple handcoded finite state systems in VoiceXML
  (Chatbots and commercial systems)
• Rational hybrids based on structure and machine
  learning.

50
The Companions a new economic and social goal
for dialogue systems
51
An idea for integrating the dialogue research
agenda in a new style of application...

• That meets social and economic needs
• That is not simply a product but everyone will
  want one if it succeeds
• That cannot be done now but could in six years by
  a series of staged prototypes
• That modularises easily for large project
  management, and whose modules cover the research
  issues.
• Whose speech and language technology components
  are now basically available

52
A series of intelligent and sociable COMPANIONS

• The SeniorCompanion
• The EU will have more and more old people who
  find technological life hard to handle, but will
  have access to funds
• The SC will sit beside you on the sofa but be
  easy to carry about--like a furry handbag--not a
  robot
• It will explain the plots of TV programs and help
  choose them for you
• It will know you and what you like and dont
• It wills send your messages, make calls and
  summon emergency help
• It will debrief your life.

53
(No Transcript)
54
Other COMPANIONS

• The JuniorCompanion
• Teaches and advises, maybe from a backpack
• Warns of dangerous situations
• Helps with homework and web search
• Helps with languages
• Always knows where the child is
• Explains ambient signals and information
• Its what e-learning might really mean!

55
(No Transcript)
56
The Senior Companion is a major technical and
social challenge

• It could represent old people as their agents and
  help in difficult situations e.g. with landlords,
  or guess when to summon human assistance
• It could debrief an elderly user about events
  and memories in their lives
• It could aid them to organise their life-memories
  (this is now hard!)(see Lifelog and Memories for
  Life)
• It would be a repository for relatives later
• Has  Loebner chat aspects  as well as
  information--it is to divert, like a pet, not
  just inform
• It is a persistent and personal social agent
  interfacing with Semantic Web agents

57
Other issues for Companions we can hardly begin
to formulate

• Companion identity as an issue that can be
  settled many ways---
• like that of the owner--web identity now a hot
  issue?
• Responsibilities of Companion agents--who to?
• Communications between agents and our access to
  them
• Are simulations of emotional behaviour or
  politeness desirable in a Companion?
• Protection of the vulnerable (young and old here)
• What happens to your Companion when you are gone?

58
Companions and the Web

• A new kind of agent as the answer to a passive
  web
• The web/internet must become more personal to be
  tractable, as it gets bigger (and more structured
  or unstructured?)
• Personal agents will need to be autonomous and
  trusted (like space craft on missions)
• But also personal and persistent, particularly
  for large sections of populations now largely
  excluded from the web.
• The semantic web is a start to structure the web
  for comprehension and activity, but web agents
  are currently abstract and transitory.
• The old are a good group to start with (growing
  and with funds).

59
The technologies for a Companion are all there
already

• Remember Tamagochi?
• Quite intelligent people rushed home to feed one
  (and later Furby) even though they knew it was a
  simple empty mechnaism.
• And Tamaogochi could not even talk!
• People with pets live longer.
• Wouldnt you like a warm pet to remind you what
  happened in the last episode of Coronation
  Street?
• OK, but perhaps millions of your compatriots
  would?!