Grammars, Automata, Complexity and Computability

1
Grammars, Automata, Complexity and Computability
Information Systems 3S?
October 2004
Dr. Colin Campbell Room 2.9 QB C.Campbell_at_bris.ac.
uk
2
Information systems

• Part 1 - Theory of discrete information and
  communication systems
• formal approach to information measurement and
  communication
• e.g. estimate how much information a channel can
  carry
• Part 2 - Grammars automata, computability,
  complexity
• formal approach to information representation and
  manipulation systems
• e.g. 1. information as natural language 2.
  predicting time it takes to solve a problem

3
Course Outline

• Grammars
• Formal models of grammar for use in modelling
  the structure of computer languages and natural
  language
• Grammars generate sentences
• Automata
• An introduction to finite automata as abstract
  machines
• An introduction to push-down automata and their
  applications
• Examine how automata relate to formal grammars
• Automata accept sentences

4
Course Outline II

• Complexity
• Complexity of algorithms and the Travelling
  Salesman problem
• Turing machine as the most general automaton, and
  RAMachines
• Computability
• The Church-Turing thesis
• Complexity classes P, NP and NP-complete
• Heuristics for the Travelling Salesman problem
• The Halting Problem

5
How are grammars, automata, complexity and
computability linked?
6
Relating components
channel
Information
Information
I
I
machine
machine
7
Introduction to Grammars

• Why use grammars?
• basis for
• natural language processing
• programming languages
• symbolic structures
• all of which represent information
• used in
• written text
• spoken language

8
Good ? and bad ? points

• ? sentences can be incomplete descriptions of
  what they are intended to convey
• e.g.
• My children have gone out to play
• Peter and Fred have gone out to play
• Peter and Fred have gone out to play football
• ? we can be as vague or precise as we like. We
  can leave out information if we think the other
  person already knows it.

9
Good ? and bad ? points

• ? Many different ways of saying things
• e.g.
• I was born on July 3
• Jonathans birthday is the third of July
• ? rules are natural if we have enough background
  information
• e.g.
• If person X was born on date D then Xs birthday
  is date D

10
Written languages

• Process using
• lexical (morphological) analysis,
• syntactic analysis,
• semantic analysis,
• and contextual knowledge

11
Lexical (morphological) analysis

• Check that each symbol in a sentence is valid
• e.g.
• I envy Freds computer ?
• I en vy Fre ds computer ?
• classify symbols into syntactic categories
• nouns Fred, I
• verbs to envy
• possesive s
• etc.

12
Syntactical analysis

• Convert flat list of words (or symbols) into a
  structure that is defined by the syntactic types
  of the words
• e.g.
• simple breakdown of sentence based on structure

13
Semantic analysis

• 1. Maps words onto objects in knowledge base
• e.g. in our context
• I maps to object Jonathan, a lecturer
• Freds computer maps to object Pentium 450MHz
• envy maps to a known mental state like
  jealousy, etc.
• 2. Creates internal structure among objects
  corresponding to how the words combine together
• e.g.
• links Jonathan to jealousy
• links jealousy to Pentium 450MHz, etc.
• Further contextual processing is now performed

14
Spoken language

• As for written language, but also
• knowledge of phonology
• handle ambiguities in speech
• accents,
• dialect,
• tonal inflection,
• etc.
• Too difficult for this course!
• We will stick to syntactical processing of
  written languages only

15
Formal Grammars

• Why have formal grammars?
• Language used for
• formal system used to maintain consistency

16
Consistency
17
Formal grammar features

• Formal models of grammar for use in modelling the
  structure of computer languages and natural
  language, including
• use of symbols for meaningful blocks such as
  phrases
• production rules for deriving well-formed
  expressions
• the idea of a language generated by a grammar
• parse trees for describing the structure of a
  sentence or expression
• the hierarchy of classes of grammar, with special
  reference to regular and context-free grammars
• the presence of ambiguity in languages

18
Language contexts

• Different contexts of languages
• natural language has structure through which we
  convey, often subtle, messages (intellectual and
  emotional)
• computer languages have similar, but simpler,
  structure - for exchange of clear and unambiguous
  messages between man and machine
• mathematical models of language structures
• (a) clarify what the messages are,
• (b) suggest efficient algorithms for processing

19
Language

• Language has structure (syntax) to support the
  transfer of meaning (semantics).
• Natural languages have evolved
• (English, French, Chinese, ... , American?)
• whereas computer languages are defined
• (C, Pascal, Fril, ... )
• Similar syntactic models apply to both, with
  applications in
• parsing and compiling of computer software
• natural language interfaces to computers,
  databases, web, . . .
• Sentences consist of strings of
  words Expressions consist of strings of symbols

20
Well-formed grammars

• Syntactic rules of grammars distinguish
  well-formed from ill-formed expressions
• What is a well-formed expression?

21
Well-formed expressions
22
Syntax and semantics

• Model syntax (structure) by, e.g. context-free
  grammar
• the boy kicked the red ball

23
Syntax and semantics II

• Model semantics by, e.g. Conceptual Graph
  relational model
• a man named Jeff is eating beans with his spoon
  in a restaurant

24
(No Transcript)