Computational Modelling of Linguistic Processes

1
Computational Modelling of Linguistic Processes

• Simon Kirbysimon_at_ling.ed.ac.uk
• www.ling.ed.ac.uk/simon

2
What have computers got to do with linguistics?

• Many industry analysts think natural-language
  interfaces are the next big thing in computing
• A lot of energy (i.e. money) goes into
• Speech synthesis
• Speech recognition
• Dialogue generation
• Natural language understanding
• Machine translation
• etc.

3
Other uses of computing in linguistics

• All these areas of research are topics from
  engineering
• The problem is to build better machines
• Language just happens to have something to do
  with it
• Linguistics may help, but not necessarily
• E.g. much speech recognition now largely to do
  with statistical analysis
• Can computers give something back to
  linguistics?

4
Computers as linguistics research tools

• Aside from using linguistics to build better
  computer interfaces, we use computers as tools
  for linguistics
• For example
• Speech analysis (e.g. spectrograms)
• Psycholinguistics (e.g. resynthesis of speech,
  displaying stimuli)
• Corpus analysis (e.g. counting words, discovering
  patterns)

5
Computers as platforms for modelling

• This course is not about engineering, or research
  toolkits
• Instead we will be looking at model building
• Key questions
• Why would we want to build models in linguistics?
• Why would computers help?
• What is a model anyway?

6
What is a model?

• One view
• We use models when we cant be sure what our
  theories predict
• Especially useful when dealing with complex
  systems

MODEL
PREDICTION
THEORY
OBSERVATION
7
A simple example

• Vowels exist in a space

• Only some patterns arise cross-linguistically
• E.g. vowel space seems to be symmetrically filled
• Most common 3-vowel system i, a, u etc.
• Why?

8
The need for theory

• Why might such universal patterns exist?
• This is where we need a theory
• Possible theory
• Vowels tend to avoid being close to each other in
  order to maintain perceptual distinctiveness.
• How can we tell our theory is correct?

9
The need for a model

• We need some way of generating predictions from
  the theory which can be compared with the real
  data.
• A physical model (Liljencrants and Lindblom
  1972) a tank of water, some corks with magnets
  attached.

• From this model, predicted patterns can be
  compared with cross-linguistic data

10
How close to reality should this model be?

• One view is that the model should leave nothing
  out.
• But is it sensible to build the real thing in
  miniature?
• Will we actually learn anything from this?
• If not, then where do we draw the line?
• Build your model to have as little extra in it
  that isnt part of your theory.

11
Other kinds of models

• A physical model is only one of many possible
  modelling approaches.
• For example
• Mathematical models in biology and epidemiology
• Thought experiments in physics
• Verbal arguments in philosophy??

12
A mathematical version of the tank

• Start with some vowels.
• Measure the vowel system energy
• rij is the perceptual distance between vowels i
  and j
• Move the vowels slightly
• If this reduces E, keep the new positions

13
What does using a computer add to all this?

• What if
• your theory is too difficult to understand simply
  through verbal argument, or introspection?
• or a physical model cannot be constructed simply?
• or a mathematical model is too difficult (or
  impossible) to construct?
• When might this be the case?
• Particularly difficult problems involve dynamic
  interactions
• people interacting in groups over hundreds of
  years
• a child responding to hearing thousands of verbs
• communicating animals evolving over millennia

14
Computational modelling is the solution

• Computers are very good for models of many
  interacting components
• Many linguistic theories are best cast in these
  sorts of terms
• Some problems we will be looking at
• How do we learn morphological rules?
• How are syntactic patterns acquired?
• Why is there a critical period for language
  acquisition?
• Why is language change s-shaped?
• Where do language universals come from?
• How did language evolve in the human species?

15
Learning, change and evolution

• Most computational modelling is about one of
  these big three
• Change and evolution both build on learning
• The next few lectures will be about modelling
  learning
• various approaches to machine learning
• we will be concentrating on connectionism or
  neural network modelling.

16
Structure of the course

• Introduction - what is a model?
• Neural networks
• Language acquisition and connectionism
• Language change
• Language evolution
• Modelling revisited

17
Readings

• Only half the course will be lectures. The other
  half will be discussion based round readings.
• Readings will be assigned one week before
  discussion session.
• Topics for discussion (e.g. questions to think
  about) will be given out in the lecture prior to
  discussion session.
• Mondays lectures and question sheet hand out
• Thursdays discussion and next weeks reading
  assigned

18
Assessment

• One essay (possibly involving practical work).
  About 3000 words, topic to be agreed upon later.
• Exam combining one essay question and a number of
  short-answer questions.

19
This weeks readings

• MUST BE READ BY NEXT THURSDAY!
• Discussion points will be given out on Monday.
• Bechtel, W. and Abrahamsen, A. (1991).
  Connectionism and the Mind. Chapter 1. Networks
  versus Symbol Systems Two approaches to modeling
  cognition pp. 1-20
• Elman, J. et. al. (1996). Rethinking Innateness.
  Chapter 2. Why connectionism? pp. 47-66.