Probably Approximately Correct Learning

1
Probably Approximately Correct Learning pac Model

fixed but unknown
distribution
according to an

• When we evaluate the quality of a hypothesis

(classification function)
we should take the
into account
unknown
distribution
error or expected error
)
made by the

• We call such measure risk functional and denote

it as
2

Generalization Error of pac Model
3
Probably Approximately Correct

• We assert

or
4
Find the Hypothesis with Minimum Expected Risk?

• The ideal hypothesis

should has the smallest
expected risk
Unrealistic !!!
5
Empirical Risk Minimization (ERM)
are not needed)
(
and

• Only focusing on empirical risk will cause
  overfitting

6
VC Confidence
(The Bound between )
7
Capacity (Complexity) of Hypothesis Space
VC-dimension
8
Shattering Points with Hyperplanes in
Can you always shatter three points with a line in
?
9
Definition of VC-dimension

• The Vapnik-Chervonenkis dimension,

, of
hypothesis space
defined over the input space
is the size of the (existent) largest finite
subset
shattered by
of