Heuristic Programming in Business Solutions ISECON

1
Heuristic Programming in Business
SolutionsISECON02

• Kurt F. Lauckner
• Department of Computer Science
• Eastern Michigan University
• csc_lauckner_at_online.emich.edu

2
What is Soft Computing

• Often called Heuristic programming
• Includes
• Expert Systems and Fuzzy Logic
• Neural Networks
• Evolutionary Systems
• Genetic Algorithms
• Genetic Programming
• Other Evolutionary Structures

3
Expert Systems

• Expert systems are also called Rule-based
  systems.
• Experts expertise is built into the program
  through a collection of rules.
• The desired program functions at the same level
  as the human expert.
• The rules are typically of the form
• If (some condition) then (some action)
• Example If (gas near empty AND going on long
  trip) then (stop at gas station AND fill the gas
  tank AND check the oil).
• EXCON An expert system used by DEC

4
Expert Systems

• Two major parts of an expert system
• The knowledge base The collection of rules that
  make up the expert system.
• The inference engine A program that uses the
  rules by making several passes over them.
• On each pass, the inference engine looks for all
  rules whose condition is satisfied (if part).
• It then takes the action (then part) and makes
  another pass over all the rules looking for
  matching condition.
• This goes on until no rules conditions are
  matched.
• The results are all those action parts left.

5
Expert Systems

• Inference engines can pass through the rules in
  different directions
• Forward chaining Going from a rules condition
  to a rules action and using the action as a new
  condition.
• Backward chaining Goes in the other direction.
• Example Medical doctors use both.
• Forward chaining Going to the doctor with
  symptoms (stomach pain). The doctor will come up
  with a diagnosis (ulcer).
• Backward chaining The doctor asks if patient has
  been eating green apples knowing green apples
  cause stomach aches.

6
Expert System -- Case Study

• Business Resource Software, Inc. (BRS) has a
  series of expert systems products that are a
  Knowledge Base of Business Experts. Their
  clientele include

American Express ATT Back-In-A-Flash Boeing Centu
ry 21 Dun Bradstreet E.I.DuPont Eastman
Kodak Ernst Young Federal Express Harvard
University IBM Corp.
J.C.Penny McDonalds Motorola New York
Life Olympus Pfizer Roadway Express Shell
Oil Texas Instruments Toyota Wells Fargo
Bank Xerox
7
BSR -- Expert System Products

• Plan Write Expert Edition --Evaluates business
  idea and documents business plan with an Expert
  Assistant.
• Plan Write for Marketing-- Produces a complete,
  fully documented marketing plan with defined
  objectives.
• Plan Write for Hi-Tech Marketing -- A special
  version designed to assist the high technology
  industries to produce a complete, fully
  documented marketing plan with defined objectives
  and tactical lists for implementation.

• Plan Write for Pricing -- Helps you to set the
  optimal price on any product or service.
• Insight for Sales Strategy --Analyzes complex
  sales, identifies roadblocks, rates the potential
  for closing the sale, provides a pipeline of all
  forecasted sales.
• Plan Write for Sales -- Develops and documents a
  plan for selling including strategy and tactics.

http//www.business-strategy-brs.com/customers.asp
8
Fuzzy Logic

• Fuzzy logic comprises fuzzy sets and approximate
  reasoning
• A fuzzy fact is any assertion or piece of
  information, and can have a degree of truth,
  usually a value between 0 and 1
• Fuzziness A type of imprecision which is
  associated with ... classes in which there is no
  sharp transition from membership to
  non-membership - Zadeh (1970)

9
Linguistic Variables

• Linguistic variable a variable whose values are
  words or sentences in a natural or artificial
  language. - Zadeh
• Linguistic variables translate ordinary language
  into logical or numerical statements
• Imprecision of linguistic variables makes them
  useful for reasoning

10
Linguistic Variables
In fuzzy set theory we can translate the body
temperatures of patients into linguistic values
with an appropriate degree of membership µ.
With the linguistic variable fever, fuzzy
membership functions can be defined --low
--normal
--raised --strong_fever
µ(x)
strong_fever
raised
normal
low
1
0
x
96F
98F
100F
102F
104F
106F
108F
110F
11
Fuzzy Logic -- Case Study

• Economic red flags in risk analysis and audit
  planning
• Touche Ross 1974, Coopers and Lybrand 1977,
  and Price Waterhouse 1985 released a list of
  red flags for auditors
• Deshmukh and Romine describe a fuzzy number based
  spreadsheet approach in assessing the risk of
  management fraud using red flags
  http//www.swcollege.com/acct/jac/jac12/jac12_arti
  cle1.html

12
Fuzzy Logic -- Case Study

• Uses FuziCalc which provides a spreadsheet based
  environment that incorporates fuzzy numbers in
  the calculations
• Define fuzzy sets
• Strong internal control
• Weak internal control
• Define that the internal control is effective if
  it detects or prevents 90 of the errors or
  irregularities

• 80 effective control may have a membership of
  0.9 in a fuzzy set of "strong internal control"
  and a membership of 0.1 in the fuzzy set "weak
  internal control
• These concepts are useful in measuring and
  quantifying red flags

13
Fuzzy Logic -- Case Study

• The Loebbecke, Eining, and Willingham 1989
  model stipulates that the probability of
  management fraud is a function of three factors
  condition, attitude, and motivation.
• An effort must be made to aggregate red flags
  for each factor and then combine these three
  factors to determine the probability of
  management fraud.
• There are no statistical formulas to do this.
• The fuzzy number based spreadsheet can help
  accomplish this by mathematically manipulating
  fuzzy numbers.
• If each red flag is considered equally important
  then the beliefs for all red flags are equally
  weighed and the belief for the Condition factor
  can be calculated

14
Neural Networks

• Neuron Basic building-block of the brain.
• There are several specialized types, but all have
  the same basic structure
• The basic structure of an animal neuron.

15
What is topology of NNs

• NNs consist of processing elements (PEs) and
  weighted connections

16
What NNs are and why they are useful

• Connection weights store the information
• Weight values usually determined by learning
  procedure
• Each PE acts independently of all others
• Each PE relies only on local information
• Connection pattern provides redundancy and
  facilitates fault tolerance
• The most common NN uses backpropagation

17
Processing Element
X1
Wj 1
Yj f(X,W)
X2
Wj 2
Wj 3
X3
qj
Yj
Wj n
Xn
Propagation Function (Linear Combiner)
Activation Function
Example (Sigmoid)
q f(X,W)
Example
18
Neural Network -- Case Study

• Real Estate Example--Home price estimator
• The 17 inputs include
• Number of bedrooms, bathrooms, other rooms.
• Garage type (attached, detached or none)
• Patio (yes or no)
• Air conditioning
• Specialized windows (high R factor, security)
• Area code
• One hidden layer with 8 nodes

19
Real Estate Case Study (Cont)

• Using 100 different homes
• With 17 inputs and selling price of the home
• Used EasyNN a Windows under 40 neural network
  program shell
• Trained network for several hours
• Results of homes not known to the neural network
  were estimated to within 2 of actual pricing

20
LVQ or Kohonen Networks

• LVQ (Learning Vector Quantizer) neural networks
  are often called Kohonen networks
• Probably second only to backpropagation in number
  of applications
• No rigorous mathematical derivation
• Seem to be more biologically oriented than most
  paradigms
• Reduce dimensionality of inputs
• LVQ-I Makes use of unsupervised learning
• Network learns from input patterns alone

21
Self-Organizing Feature Maps

• Primary features of input cause corresponding
  local responses in the output PE field.
• Are non-linear mappings of input space onto the
  output PE space (field).
• The mapping is referred to as a self-organizing
  feature maps
• Extensions of LVQ-I resemble the way certain
  areas of the brain are organized

22
General Comments about LVQ-I

• Main accomplishments of LVQ-I networks is they
  cluster input data patterns into classes, thus
  reducing dimensionality of data (e.g.,
  automatically parameterizes the data)
• Input vectors are usually normalized
• Weight vector normalization is done for all
  weights connected to a given output PE from all
  input PEs

23
Kohonen Neural Network -- Case Study
Marketing A campaign is being run to increase
sales of a product.  It is intended to do a
targeted  mailshot to individuals most likely to
be interested in the product.  There is
information available from a consumer survey
which includes information about current
purchasers of the product. An unsupervised net is
trained using information from the consumer
survey.  The groups formed are examined and all
individuals in the groups rich in existing
purchasers are used as targets for the
mailshot. Data Cleaning  Data being extracted
from a database is being analysed but
unfortunately the data is too inconsistent for
the analysis technique.  An unsupervised net is
used to create groups and the analysis is based
on the individual groups instead of the entire
database. http//www.neusciences.com/Technologies/
nn_unsupervised.htm
24
Evolution--Darwin
if variations useful to any organic being do
occur, assuredly individuals thus characterized
will have the best chance of being preserved in
the struggle for life and from the strong
principle of inheritance they will tend to
produce offspring similarly characterized. This
principle of preservation, I have called, for the
sake of brevity, Natural Selection.
--C. Darwin, 1859
25
Evolution--Darwin
Translating Darwins statement, the four
essential preconditions for the occurrence of
evolution by natural selection are

• Reproduction of individuals in the population
• Variation that affects the likelihood of
  survival of individuals
• Heredity in reproduction (e.g.,like begets like)
• Competition caused by finite resources.

--Banzhaf
26
Evolutionary Algorithms
Basic Ingredients of EAs

• Populations of solutions
• Innovation operations (e.g., mutation)
• Conservation operations (e.g.,crossover)
• Quality differentials (e.g.,graded fitness
  function, richness of differences)
• Selection (e.g., differences in population are
  used to select fittest.

27
Genetic Algorithm--Holland

• Uses binary genome of fixed length
• Crossover most common and usually of the is
  one-point crossover variety
• Mutation used very sparingly
• Classifier systems used to select fittest
  consist of three parts
• Rule-based programming language
• Simulated market economy
• Genetic algorithm

1
0
0
1
1
1
0
1
0
1
0
0
1
1
1
0
1
0
1
0
0
1
1
1
1
1
1
Parents
Offspring
1
1
0
1
0
1
1
1
1
1
1
0
1
0
1
0
1
0
28
Genetic Algorithm -- Case Study

• Improve model for tactical asset allocation
• Improve model for international equity strategies
• Improvements of up to 82 in cumulative portfolio
  value as compared with passive benchmark model

29
Evolutionary Programming
Essential Common Themes

• Stochastic decision making GP uses pseudo-random
  numbers to mimic the randomness of natural
  evolution.
• Program structures GP assembles variable length
  program structures from basic units called
  functions and terminals.
• Genetic operators GP transforms the initial
  programs in the population using genetic
  operators (e.g. crossover, mutation,
  reproduction)
• Simulated evolution of a population by means of
  fitness-based selection.

30
Primitives of Genetic Programs

• Terminal set terminals are the inputs to the GP
  program (named by fact they are terminals of a
  tree structure)

31
GP Primitives (Cont)

• Function set these are the statements,
  operators, and functions available to the GP
  system.
• Boolean functions ( AND, NOT, OR, XOR etc.)
• Arithmetic functions (plus, minus, multiply etc.)
• Variable assignment functions ( x 3)
• Indexed memory functions
• Conditional statements (IF, THEN, ELSE, CASE)
• Control transfer statements (GOTO, CALL, JMP)
• Loop statements (WHILEDO, REPEAT)
• Subroutines (for example a robot turn left,
  stop)

32
Executable Program Structures

• Functions and terminals must be assembled into
  some structure that can be executed and
  evaluated.
• Two methods of initializing tree structure
• Full method of tree initialization
• Pick functions as nodes until maximum depth
  reached
• Pick terminals (leaves) of the tree from terminal
  set
• Grow method of tree initialization
• Pick randomly from functions and terminal sets
• If function selected continue with that branch
• If terminal selected the branch ends

33
Execution of Structure

• In the tree structure use the pre-order traversal
  to test fitness of all individuals of the
  population
• Apply selection algorithm
• Use crossover, mutation and reproduction to
  create new population
• Continue until termination criterion met or
  maximum number of generations reached

34
Genetic Programming -- Case Study

• GP paradigm tool STROGANOFF (1990s)
• Used in predicting USD/SF currency exchange rates
• Used in predicting movements of the Dow Jones
  industrial index
• Used in predicting movements of the Nikkei225
  index of the Tokyo Stock Exchange
• Other applications derivative markets, banking
  and finance, marketing