Soft Computing

1
Soft Computing

Lecture 22 Using of NN in NLP and speech
recognition
2
Agenda

• Introduction to NLP
• Using of recurrent NN for recognition of correct
  sentences
• Example of learning software for searching of
  documents by query in natural language

3
Programs based on NLP

• Question-Answering Systems
• Control by command in Natural Language
• Readers from text to speech
• Translators
• Search of information by query in Natural
  Language
• OCR Optical Characters Recognition
• Virtual Persons

4
Main areas of NLP

• Understanding of NL
• Generation of NL
• Analyzing and synthesis of speech

5
Levels of language

• Words, parts of words (lexical level, morphology)
• Structure of words
• Phrases, sentences (Syntax, syntactic level)
• Structure of phrases and sentences
• Sense, meaning of phrases (Semantics, semantic
  level)
• The meaning here is that associated with the
  sentential structure, the juxtaposition of the
  meanings of the individual words
• Sense, meaning of sentences (Semantics, discourse
  level)
• - Its domain is intersentenial, concerning the
  way sentences fit into the context of a dialog
  text
• Sense as goals, wishes, motivations and so on
  (Pragmatics)
• Deals with not just a particular linguist context
  but the whole realm of human experience

6
Example

• Following sentences are unacceptable on the basis
  of syntax, semantics, and pragmatics,
  respectively
• John water drink.
• John drinks dirt.
• John drinks gasoline.
• Note that the combination of "drink" and
  "gasoline" is not unacceptable, as in "People do
  not drink gasoline" or the metaphorical "Cars
  drink gasoline.
• It is traditional for linguists to study these
  levels separately and for computational linguists
  to implement them in natural language systems as
  separate components. Sequential processing is
  easier and more efficient but far less effective
  than an iterated approach.

7
Syntactic analyzing

• A natural language grammar specifies allowable
  sentence structures in terms of basic syntactic
  categories such as nouns and verbs, and allows us
  to determine the structure of the sentence. It is
  defined in a similar way to a grammar for a
  programming language, though tends to be more
  complex, and the notations used are somewhat
  different. Because of the complexity of natural
  language a given grammar is unlikely to cover all
  possible syntactically acceptable sentences.
• To parse correct sentences
• John ate the biscuit.
• The lion ate the schizophrenic.
• The lion kissed John.
• To exclude incorrect sentences
• Ate John biscuit the.
• Schizophrenic the lion the ate.
• Biscuit lion kissed.

8
Simple context free grammar for previous examples

• sentence --gt noun_phrase, verb_phrase.
• noun_phrase --gt proper_name.
• noun_phrase --gt determiner, noun.
• verb_phrase --gt verb, noun_phrase.
• proper_name --gt Mary.
• proper_name --gt John.
• noun --gt schizophrenic.
• noun --gt biscuit.
• verb --gt ate.
• verb --gt kissed.
• determiner --gt the.

9
Parsing
sentence
noun_phrase
verb_pharse
verb
noun_phrase
p_name
p_name
Mary
loves
John
10
Parts of speech
11
Examples of the part-of-speech tagging
Experiments showed that adding sub-categorization
to the bare category information improved the
performance of the models. For example, an
intransitive verb such as sleep would be placed
into a different class from the obligatorily
transitive verb hit. Similarly, verbs that take
sentential complements or double objects such as
seem, give or persuade would be representative of
other classes. Fleshing out the
sub-categorization requirements along these lines
for lexical items in the training set resulted in
9 classes for verbs, 4 for nouns and adjectives,
and 2 for prepositions.
12
Recurrent Elman network
13
Extraction of grammar (DFA) from learned
recurrent Elman network

• The algorithm we use for automata extraction
  works as follows after the network is trained
  (or even during training), we apply a procedure
  for extracting what the network has learnedi.e.,
  the networks current conception of what DFA it
  has learned.
• The DFA extraction process includes the following
  steps
• clustering of the recurrent network activation
  space, S, to form DFA states,
• constructing a transition diagram by connecting
  these states together with the alphabet labelled
  arcs,
• putting these transitions together to make the
  full digraph forming loops,
• reducing the digraph to a minimal representation.

14
Acoustic Waves

• Human speech generates a wave
• like a loudspeaker moving
• A wave for the words speech lab looks like

s p ee ch
l a b
l to a transition
Graphs from Simon Arnfields web tutorial on
speech, Sheffield http//lethe.leeds.ac.uk/resear
ch/cogn/speech/tutorial/
15
Acoustic Sampling

• 10 ms frame (ms millisecond 1/1000 second)
• 25 ms window around frame to smooth signal
  processing

25 ms
. . .
10ms
Result Acoustic Feature Vectors
a1 a2 a3
16
Acoustic Features Mel Scale Filterbank

• Derive Mel Scale Filterbank coefficients
• Mel scale
• models non-linearity of human audio perception
• mel(f) 2595 log10(1 f / 700)
• roughly linear to 1000Hz and then logarithmic
• Filterbank
• collapses large number of FFT parameters by
  filtering with 20 triangular filters spaced on
  mel scale

...
frequency

m1 m2 m3 m4 m5 m6

coefficients
17
Phoneme recognition system based onthe Elman
predictive neural networks.

• The phrases are available in segmented form with
  speech labeled into a total of 25 phonemes.
• Speech data was parametrisized into 12 liftered
  mel-frequency cepstral coeficients (MFCCs)
  without delta coeficients. The analysis window is
  25ms and the window shift 10ms.
• Each phoneme is modeled by one neural network.
  The architecture of the neural networks which is
  seen during recognition with the Viterbi
  algorithm (when the neural network models provide
  the prediction error as distortion measure)
  corresponds to a HMM with 3 states (supposing
  that the second state is modeling the speech
  signal, and the first and last state act as input
  and output states, respectively).
• Results of experiments Elman network provides
  best results of recognition on training set in
  comparison with HMM

18
Technology for building of learned system for
search of documents by sense
End User
Processing of query
Knowledge Base
Constant part
Variable part
Additioanl dictionary
Base Dictionary
Dialogue with teacher
Processing of documents
Teacher-administrator
Store of documents
19
Main principles in proposed technology

• Orientation to recognition of semantics with
  minimum usage of knowledge about syntax of the
  language,
• Creation of hierarchies from concepts with
  horizontal (associative) links between nodes of
  these hierarchies as result of processing of
  documents,
• Recognition of words and word collocations on
  maximum resembling with usage of neural
  algorithms.

20
Kinds of frames

• 1. frame coupled immediately to the word or the
  document (the frame-word or the frame-document)
• 2. frame, with which associates a word
  collocation (composite frame)
• 3. frame-concept including the links on several
  other frames, playing the defined role in this
  concept
• 4. frame-heading circumscribing concept, which is
  "exposition" of all concepts and documents
  coupled to this heading

21
Slots of frame

• Parent - link on the frame - parent or class
  (vertical links)
• Owner - list of links to frames - concepts or the
  composite frame, in which structure enters the
  given frame
• Obj - object participating in concept,
• Subject - subject (or main object), participating
  in concept
• Act - operation (action) participating in concept
• Prop - property participating in concept
• Equal - list of concepts - synonyms circumscribed
  in the given frame (horizontal links)
• UnEqual - list of concepts - antonyms
  circumscribed in the given frame
• Include - list of links to the frames switched on
  in the given concept constituent (vertical links)

22
Other main parameters of frame

• Level - level of the frame in hierarchy
• DocName - index of filename (path) of document
  coupled to the frame
• IndWord - index of a word in the dictionary
  coupled to the frame
• H - threshold of operation of the frame, as
  neuron
• Role - role of the frame in concept, which it
  enters or can enter (A-operation, O-object,
  S-subject, P-property, U-undefined or D - the
  operation at the analysis (by special procedure)
• NO - indication of inversion of the frame

23
Dictionaries

• Basic, in which the words with their roles
  (essence, operation or property, in other words -
  noun, verb or adjective are stored
• The supplemented (dynamic) dictionary including a
  words, not recognized in the base dictionary
• Dictionary of special words, associated with
  separators and analyzed as separators.

24
Steps of analyzing of sentence in context of
learning

• 1) selection of words (using signs of punctuation
  and spaces)
• 2) the recognition of words on maximum resembling
  with words in the dictionary, thus if the
  approaching word is not in the fundamental
  dictionary, then searching of this word in the
  supplemented dictionary, and in fail case this
  word adds in this dictionary
• 3) the creation of the frames of a level 0, the
  result of this stage is object-sentence
  representing list of the frames
• 4) replacement in this object of special words by
  signs-separators,
• 5) processing of the object-sentence by a
  procedure of recognition-creation of the frames
  of levels 1 and 2

25
Steps of analyzing of sentence in context of
processing of query

• 1) selection of words (using signs of
  punctuation and spaces),
• 2) the recognition of words on maximum resembling
  with words in the dictionaries. In case of
  unknown word system ask question "what is lt new
  word gt?". The answer of the user is processed in
  context of learning.
• 3) the creation of the frames of a level 0, the
  result of this stage is object-sentence
  representing list of the frames,
• 4) the recognition of the frames of a level 1 or
  2 - word collocations in the knowledge base
  maximum similar to recognized phrase (here is
  used neural algorithm, i.e. weighed addition of
  signals from words, entering into the frame, or
  frames and matching with a threshold),
• 5) the searching associatively coupled by the
  links Equal with the recognized phrases of the
  frames (level 0), coupled with documents,
• 6) the searching of frames-documents from the
  retrieved frames on connections such as include,
  act, obj, subject, prop from above downwards
• 7) the output of the retrieved names of
  documents or words which are included in
  structure of the retrieved frames.

26
Steps of learning of System

• Initial tutoring to recognition of structure of
  sentence by input of sentences as "word -
  _at_symbol. This step provides creation of
  dictionary of special words
• Initial tutoring. During this step the knowledge
  base is filling by fundamental concepts from
  everyday practice or data domain as sentences
  such as "money - means of payment", "morals -
  rule of behavior", kinds of business trade,
  production, service" etc.
• Base tutoring. In this step the explanatory
  dictionary of data domain is processed, where the
  concepts of any area are explained with use "-"
  or corresponding words.
• Information filling. In this step the real
  documents are processed.

27
(No Transcript)
28
(No Transcript)