AI Foundations

1
Artificial Intelligence Foundations
Machine Learning, Deep Learning
and Neural Networks
www.leewayhertz.com
2

• Contents
• Overview
• Machine Learning
• History of ML
• Deep Learning
• History of DL
• Neural Networks
• History of NN

www.leewayhertz.com
3
Overview

• Artificial Intelligence system consists of
• People
• Procedures
• Hardware
• Software
• Data
• And Knowledge
• Needed to develop computer systems and machines
  that demonstrate the characteristics of
  intelligence.

www.leewayhertz.com
4
Machine Learning
Machine Learning
Input Data Information ( Answers)
Output Optimum Model

• Relationships
• Patterns
• Dependencies
• Hidden Structures

www.leewayhertz.com
5
Training Phase
Machine Learning Algorithm
Labels
Feature Extractor
Images
Features
Prediction Phase
Trained Classifier
Feature Extractor
Images
Features
Label
Machine Learning Phases
Traditional ML Algorithm
Feature Extractor
Image
Features
Output
Traditional Machine Learning Flow
www.leewayhertz.com
6
History of ML

Decision Trees Quinlan, 1979 (ID3) Breiman, 1984
(CART)
Ensembles Breiman, 1994 (Bagging) Breiman, 2001
(Random Forests)
Boosting Schapire, 1989 (Boosting) Schapire, 1995
(Adaboost)
Interpretability
Support Vector Machines Vapnik, 1963 (ANN)

Corina Vapnik, 1995
Neural Networks Minsky, 1969
Deep Learning Fukushima, 1989 (ANN) Hinton,
2006
Perception Rosenblatt, 1957
-
1960
1950
1980
1970
2000
1990
2010
2020
www.leewayhertz.com
7
Deep Learning
Deep Learning is a set of algorithms in machine
learning that attempts to model high-level
abstractions in data by using architectures
composed of multiple non-linear transformations.
Image
Deep Learning Flow
Output
Deep Learning Flow
www.leewayhertz.com
8
History of DL
McCulloch/Pitts Neurons
Hebb's Organization of Behavior
Rosenblatt's Perceptions
Multi-Layer Perceptrons
Backpropagation
Hopfield Networks
Convolutional Neural Networks
Long Short-Term Memory (LSTM)
Deep Learning
Deep Learning with GPUs
1940 1950 1960
1970 1980 1990
2000 2010
www.leewayhertz.com
9
Neural Networks
A Neural Network is a system composed of many
simple processing elements operating in parallel
which can acquire, store, and utilize
experimental knowledge
www.leewayhertz.com
10
Preprocessing
Raw Data
Grayscale Image Size
Train RBM
Data Set
Extracted Features
Features Map
Reshape Extracted Features
Output
Train CNN
Neural Network Flow
www.leewayhertz.com
11
History of Neural Networks
LVQ, SOM 1981, 1982
RBF 1988
Discrete Hopfield 1982
Continuous Hopfield 1984
Boltzmann Machine 1984
Modified Backpropagating Perception 1986-1990
Backpropagating Perception 1974
Perception 1958
Perception 1958
1950 1960
1970 1980
1985 1990
www.leewayhertz.com
12
Want to know more about
AI, ML, and DL?
www.leewayhertz.com
13
Thank You!
www.leewayhertz.com