How the electronic mind can emulate the human mind: some applications of Artificial Intelligence

1
How the electronic mind can emulate the human
mind some applications of Artificial
Intelligence

• 7th International Interdisciplinary Seminar

Luca Arcara, Federico Cassoli, Mattia
Ferrini Politecnico di Milano Campus of Como
2
Agenda

• Introduction
• Expert Systems
• Neural Networks
• A sample application
• Genetic Algorithms

3
Introduction

• What is Artificial Intelligence?

Systems that think like humans
Systems that act like humans
Systems that think rationally
Systems that act rationally
4
Acting humanly the Turing test

• Described in Computing machinery and
  intelligence Turing (1950)
• Can machines think? becomes Can machines
  behave intelligently?
• Operational test for intelligent behavior the
  Imitation Game.

5
Acting humanly the Turing test

• Suggested major components of AI
• Knowledge
• Reasoning
• Language understanding
• Learning
• Problem Turing test is not reproducible,
  constructive or apt to mathematical analysis

6
Subjects linked to A. I.
Philosophy Logic, methods of reasoning Mind as physical system Foundations of learning, language, rationality Mathematics Formal representation and proof Algorithms, computation, (un)decidability, (in)tractability Probability Psychology Adaptation Phenomena of perception and motor control Economics Formal theory of rational decisions Linguistics Knowledge representation Grammar Neuroscience Plastic physical substrate for mental activity Control theory Stability of systems Simple optimal agent designs
7
State of the art
Artificial Intelligence
Expert systems
Neural networks

Genetic algorithms
8
Expert systems

• Def software systems simulating expert-like
  decision making while keeping knowledge separate
  from the reasoning mechanism.
• Replace human expert decision making when not
  available
• Assist human expert when integrating various
  decisions
• Provides a user with
• an appropriate hypothesis
• methodology for knowledge storage and reuse.

9
Expert systems architecture
Lets the user change the k.b.
Experts knowledge facts and rules
Simplifies the users interaction
Uses the k.b. to infer new facts and produce
solutions
Tells the user the steps that produced the
solution
10
Rule-Based Systems

• Knowledge in the form of if condition then effect
  rules
• Example

here ? fine not here ? absent absent and not seen
? at home absent and seen ? in the building in
the building ? fine at home and not holiday ?
sick here and holiday ? sick
? here ? no ? seen ? no ? holiday ? no sick ?
here ? yes fine ? here ? yes ? holiday ?
yes sick
?
11
Expert systems vs. conventional programs
Aspect Expert systems Conventional programs
Paradigm Heuristical rules, exploration of the space of states Algorithms, explicit pre-defined steps
Approach Declarative Procedural
Data manipulated Knowledge, often rules Vectors and matrixes of data
Control system Inferential engine separated from the knowledge base Data and information integrated with programs
User interface Highly interactive, usually questions and answers No standard typology
Explanation capability It presents the steps that led to the proposed solution Not available
Learning capability Present Not available
12
Applications

• Interpretation
• Diagnosis
• Monitoring
• Planning and scheduling
• Forecasting
• Project and configuration

13
Neural Networks

• Def mathematical models that try to emulate the
  human nervous system.
• Final target of neural networks is to simulate
  the process of learning of the human brain, so
  that it can interact with the external
  environment without human help, except for the
  creation.
• The first models were developed by W. McCulloch
  and W. Pitts in 1943, with their manifest A
  logical calculus of the ideas immanent in nervous
  activity.

14
Brain and Neurons

• General Structure of a Neuron
• Learning

dendrites
axon
synapses
nucleus
15
The structure of neural networks

• Artificial neural networks are typically composed
  of interconnected units which serve as a model
  for neurons.
• The synapse is modelled by a modifiable weight
  associated with each particular connection.
• Each unit converts the pattern of incoming
  activities that it receives into a single
  outgoing activity that it sends to other units.
• First biased weighted sum
• Second transfer function

16
How do they work?
1
0.19 0.88 0 - 0.8 0.27
x
0.19
0.4 0 - 0.73 -0.33
0.88
1
x
x
0.4
1
0
0
0
x
-0.13
0
x
0
-0.13 0 - 0.82 -0.95
0

• Example of a two levels network

17
Learning

• After a neural network has been created it can be
  trained using one of the supervised learning
  algorithms (an example is back propagation),
  which uses the data to adjust the network's
  weights and thresholds so as to minimize the
  error in its predictions on the training set.

18
Why are they useful?

• If the network is properly trained, it can model
  the (unknown) function that relates the input
  variables to the output variables, and can
  subsequently be used to make predictions where
  the output is not known.
• They are based on the concept that often (not
  always), it is possible to teach to a
  mathematical system some laws that we did not
  know before, only by letting it analyze a lot of
  real cases.

19
Applications

• Common fields of application are when the
  statistical analysis of all the problems
  variables is too difficult or expensive (at the
  calculation level), but overall is not clear
  beforehand what kind of deterministic
  relationships there are between the different
  variables.
• OCR (Optical character recognition)
• Diagnosis
• Control of industrial productions quality
• Recognition of potentially dangerous molecules
  (using electronic noses)
• Engine management
• Control of robots

20
A sample application (1)

• Problem
• We want to create a neural network that is able
  to determine if one binary number with 4 figures
  is even.

21
A sample application (2)

• We use a network with
• 4 input nodes
• 2 hidden nodes
• 1 output node (1 if the number is even, 0 if it
  is odd).

22
A sample application (3)

• Training Data

23
A sample application (4)

• DEMO

To solve the problem we will use a program that
was produced at the Laboratory for Computational
Intelligence at the University of British
Columbia.
24
GAs Definition The idea
Genetic algorithms are based on a biological
metaphor they view learning as a competition
among a population of evolving candidate problem
solutions. So a GA is a probabilistic
optimization algorithm that makes use of a
population of test solutions which artificially
reproduce through operations analogous to gene
transfer in sexual reproduction.
25
History

• Genetic Algorithms (GAs) originated from the
  studies conducted by John H. Holland and his
  colleagues at the University of Michigan.
  Hollands book Adaptation in Natural and
  Artificial Systems, published in 1975, is
  generally acknowledged as the beginning of the
  research of genetic algorithms.

26
Definitions(1) Chromosome

• A chromosome, a collection of genes, represents a
  possible solution of the problem.
• ENCODING EXAMPLES
• Binary encoding A chromosome is a collection of
  bits

Tree encoding - In the tree encoding every
chromosome is a tree of some objects, such as
functions or commands in a programming language (
i.e. LISP )
27
Definitions(2) Fitness Function

• Fitness Function
• A fitness function evaluates the ability of a
  candidate solution to solve the given problem.

28
Definitions(3) Crossover

• Crossover operates on selected genes from
  parent chromosomes and creates new offspring.

EXAMPLE SINGLE POINT CROSSOVER WITH BINARY
ENCODING
101
FATHER - 101100
OFFSPRING
111
MOTHER - 010111
29
Definitions(4) Mutation

• The chromosome is randomly mutated to prevent
  premature convergence upon a local maximum.
• Its a further techique through wich a GA
  explores the solution space mutation gives an
  extra-probability to every possible solution of
  the problem out of the finite population of
  solutions generated by the GA.
• Mutation should not occur very often, because
  then GA will in fact change to random search.

30
Outline of a basic GA Flowchart
31
Parameters(1)

• Crossover probability ( Pc ) how often crossover
  will be performed. If there are too few
  chromosomes, GAs have few possibilities to
  perform crossover and only a small part of search
  space is explored. On the other hand, if there
  are too many chromosomes, GA slow down. Crossover
  probability is usually beetween 0.4 and 0.7.
• Mutation probability ( Pm ) how often parts of
  chromosome will be mutated. Mutation should not
  occur very often, because then GA will in fact
  change to random search. Tipical Pm is
  0.01-0.001.
• Population size how many chromosomes are in a
  population (in one generation). Good population
  size is reported to be about 20-100.

32
Parameters(2)

• Elitism Number Elitism is the name of the method
  that first copies the best chromosome (or few
  best chromosomes) to the new population. Elitism
  can rapidly increase the performance of GA,
  because it prevents a loss of the best found
  solution. Elitism number specifies how many
  chromosomes copy directly in the new population.

33
GAs vs. Ad-hoc approach
Genetic Approach is a rather brutal approach,
requiring large amounts of processing power, but
with the immense advantage of supplying solutions
to things we don't know how to solve, or don't
know how to solve quickly. In fact no knowledge
of how to solve the problem is needed BUT you
need to be able to encode the chromosome and
design the fitness function. This means
implementation relies on a problem-independent
"engine.
34
What are GAs used for ?

• GAs are excellent for all tasks requiring
  optimization and are highly effective in any
  situation where many inputs (variables) interact
  to produce a large number of possible outputs
  (solutions).
• Some example situations are
• Optimization such as data fitting, clustering,
  trend spotting, path finding, ordering.
• Management Distribution, scheduling, project
  management, courier routing, container packing,
  task assignment, time tables.
• Financial Portfolio balancing, budgeting,
  forecasting, investment analysis and payment
  scheduling.
• Data Mining
• .