Contents

1
Contents

• Problems
• Basic Concepts Representation, objective,
  evaluation problem definition, neighbourhoods
  and optima
• Basic Problems Linear, SAT, TSP, NLP
  Constraint satisfaction problems
• Basic Techniques Exhaustive, Local Search
  Simplex method.
• Methods 1 Greedy, A, Branch Bound Dynamic
  programming, and Divide Conquer.
• Methods 2 Simulated Annealing, Tabu Search
• Methods 3 Evolutionary Approaches
• Constraint Handling Techniques
• Hybridise and Tune Practical tips
• Test are you sure that you have the best
  solution?

2
Garbage In Garbage Out

• Often blind acceptance of inputs
• Often blind generation of outputs
• Practical need to
• Verify
• Validate
• Test

3
Training, Testing Validation Sets

• Assuming noisy training data and a large hidden
  layer with backprop learning
• poor generalization performance can result from
  "overtraining" the network
• The reason for this phenomenon is that the
  network "overfits" the data, i.e., the network
  tries to fit the noise in the data as well as the
  underlying function to be approximated.
• The method of cross-validation is commonly used
  to overcome the problem of data overfitting.
• In this case, the data set is broken into 3 sets
  
• a training set,
• a validation set, and
• a testing set.
• The network's performance on the validation set
  is used to determine when to stop training.
• http//neuron.eng.wayne.edu/bpFunctionApprox/bpFun
  ctionApprox.html

4
V Fold Cross Validation

• Cross-validation is a method for estimating
  generalization error based on "resampling"
• The resulting estimates of generalization error
  are often used for choosing among various models,
  such as different network architectures.
• In k-fold cross-validation
• Slpit the data into k subsets of (approximately)
  equal size.
• Train k times, each time leaving out one of the
  subsets from training, but using only the omitted
  subset to compute whatever error criterion
  interests you.
• If k equals the sample size, this is called
  "leave-one-out" cross-validation.
• "Leave-v-out" is a more elaborate and expensive
  version of cross-validation that involves leaving
  out all possible subsets of v cases e.g., 10.
• Note that cross-validation is quite different
  from the "split-sample" or "hold-out" method that
  is commonly used for early stopping in NNs.

http//www.faqs.org/faqs/ai-faq/neural-nets/part3/
section-12.html
5
Standard Deviation
In probability and statistics, the standard
deviation is the most commonly used measure of
statistical dispersion. The standard deviation
is defined as the square root of the variance.
This means it is the root mean square (RMS)
deviation from the average. It is defined this
way in order to give us a measure of dispersion
that is (1) a non-negative number, and (2) has
the same units as the data. A distinction is
made between the standard deviation s (sigma) of
a whole population or of a random variable, and
the standard deviation s of a subset-population
sample. Wikipedia
6
Students t-test

• "Student" was the pen name of William Sealy
  Gosset, a statistician for Guinness brewery in
  Dublin, Ireland.
• Gosset was hired as a result of an innovative
  policy of Claude Guinness to recruit the best
  graduates from Oxford and Cambridge for the
  application of biochemistry and statistics to
  Guinness's industrial processes. Gosset published
  the t-test in Biometrika in 1908, but was forced
  to use a pen name by his employer who regarded
  the fact that they were using statistics as a
  trade secret. In fact, Gosset's identity was
  unknown not only to fellow statisticians but to
  his employer - the company insisted on the
  pseudonym so that it could turn a blind eye to
  the breach of its rules.
• Gosset invented the t-statistic to enable the
  quality of beer brews to be monitored in a
  cost-effective manner.
• Today, it is more generally applied to the
  confidence that can be placed in judgements made
  from small samples.

7
Students t-test

• This test for compares the means of two
  treatments, even if they have different numbers
  of replicates.
• In simple terms, the t-test compares the actual
  difference between two means in relation to the
  variation in the data (expressed as the standard
  deviation of the difference between the means).
• SE standard error of the difference.
• Take the variance for each group and divide it
  by the number of people in that group. Add these
  two values and then take their square root.
• http//helios.bto.ed.ac.uk/bto/statistics/tress4a.
  html
• http//www.socialresearchmethods.net/kb/stat_t.htm
  

8
The Analysis of Variance (ANOVA)

• ANOVA is a family of general technique used to
  test the hypothesis that the means among two or
  more groups are equal, R. Fisher 1920s
• - under the assumption that the sampled
  populations are normally distributed.
• Multiple t-tests are not useful as the number of
  groups for comparison grows.
• As the number of comparison pairs grows, the
  more likely we are to observe things that happen
  only 5 of the time
• Thus P.05 for one pair cannot be considered
  significant.
• ANOVA puts all the data into one number (F) and
  gives us one P for the null hypothesis.
• The Bonferroni method allows many comparison
  statements to be made (or confidence intervals to
  be constructed)
• Bonferroni can be overly conservative

9
Significance

• In statistical hypothesis testing, two hypotheses
  are stated,
• only one of which can be true, and one or the
  other must be true.
• The null hypothesis, is what is presumed to be
  true
• The alternative hypothesis, is will be
  considered true only if the facts are strong
  enough.
• The statistical hypothesis testing procedure
  (e.g. t-test) produces a value,
• If the t value that is calculated is greater than
  the threshold chosen for statistical significance
  (usually the 0.05 level), then the null
  hypothesis that the two groups do not differ is
  rejected in favour of the alternative hypothesis,
  
• which typically states that the groups do
  differ.
• Cats prefer Whiskers goat flavour to Felix
  artichoke flavour

10
Confusion Matrix

• A visualization tool typically used in supervised
  learning
• Each column of the matrix represents the
  instances in a predicted class, while each row
  represents the instances in an actual class.
• One benefit of a confusion matrix is that it is
  easy to see if the system is confusing two
  classes (i.e. commonly mislabelling one as an
  other).
• Often used in the diagnosis of diseases
• Example confusion matrix

11
Confusion Matrix

• True Positive an individual classified as
  positive by the test and verified by the gold
  standard
• True Negative an individual classified as
  negative by the test and verified by the gold
  standard
• False-Positive and False-Negative also used
• Sensitivity True Positive Decisions
• All Gold Standard Positives
• Specificity True Negative Decisions
• All Gold Standard Negative

12
Receiver Operating Characteristic

• The Receiver Operating Characteristic (ROC) curve
  the arises the relationship between the true
  positive and false-positive rates
• The Area Under the ROC Curve (AUC) is important
  to classify both true positives and true
  negatives.
• Often used in conjunction with other statistics
  in medical domains