Advanced Digital Signal Processing

1
9Maximum Likelihood EstimatorI

• Asst. Prof. Dr. Peerapol Yuvapoositanon, PhD, DIC
• Department of Electronic Engineering
• and
• Graduate School of Electrical Engineering
• Mahanakorn University of Technology

2
PDF v.s. Likelihood function

• ??????????????????????????????????????
  ?????????????????? ???????? pdf
  ????????????????????????? x
• ??????????????????????????????????????????????????
  ?????? (???? A ?? xAw )
• ???????????????????????????? (Likelihood function)

3
PDF
4
Likelihood
5
Likelihood function
6

• ???????? ????????????????????????????????????????
  ??????????

7

• ?????????????????????????????????? ????????????
  ???????? Joint pdf ??????????????????

8
PDF ???
9
???????????
10

• Q ????????????????????????????????????????????????
  ?
• A ???????????????????????????(Observation data)
  ???? ????????????????????????
• ?????????? x(1) ???
• ?????????? x(8) ???

11

• ????????????????????

12

• ??????????? 6.1 ?????????????????????????????????
  ??????????????????????? ????????????????????????
  ????????????? ????????????????????????????????

13

• ?????? ??????? (6.4) ????????????????????????????
  ?????????????????????

14

• ???????
• ???????????????????????????????

15
Likelihood function

• ?????????????? pdf ?????????????????????????????
  ? ???????????????????

16
Log -Likelihood function

• ????????

17
(No Transcript)
18

• ???????????????????????????????????????? A

19
(No Transcript)
20
Maximum Likelihood Estimator

• ??????????????????????????? (Maximum Likelihood
  Estimator) ?????????

21

• ??????????? 6.2 ?????????????????????????
  ??????????????????????? ??????????????????????????
  ???
• ????????????? 6.2 ??? ???????????????????????????
  ?? ???????? WGN ???????????????????????????????
  DC (A) ???????????? ???? x(n)

22
???????????????????????
23
Likelihood function
24
Log-Likelihood function
25
(No Transcript)
26

• ?????????????????????? ???????????????????????????
  ??????????? MVUE ?????????????????????????????????
  ???????
• ???????????????????????????????????? MVUE
  ????????????????????????????

27

• ?????????????????????? 6.2 ??? ???????????????????
  ????????????????????????????????????????????????
  ????????????????????? ????????????????????????????
  ????????????????????????? ???????? (6.4)
  ????????????????????????????? ???

28
???????????????????????????????????????

• MLE ??? ??????????????????????????????????????????
  ???????????????????????????????????????

29
?????????????????????? (Biased Estimator)
30

• ?????????????????????????????????????????????
  (Law of Large Number)

31

• ????????????????? (Sample mean)
  ?????????????????????? (Mean value)

32
??????????????????????????????? (Asymptotically
Efficient)

• ??????????????????????????????? (Asymptotically
  Efficient) ???

33
Log-Likelihood Function
34
?????????????????
35
??????????????

• ??????

36
?????????????????
37

• ??? (???????????)

38
???????????????

• ??? ????????????

39

• ??????????

40
???
41
???????
42

• ????????????????????????????? ???????? MLE
  ?????????????????????????????? (Biased Estimator)

43
???????????? ??????????

• ????????????????? (Sample mean)
  ?????????????????????? (Mean value)

44
??????????????? (Taylor Series)

• ?????????????????????????????????????
  ??????????????? (Taylor Series) ???????????????
  ??????????? ??? ???? ????????????????????????????
  ????????? (First-order Taylor Series)
  ?????????????????
•  

45

• ??? u ????

46
????????????

• ??????????????????????????? u

47

• ???

48
(No Transcript)
49

• ??????????????????????????

50

• ???????????????????????????????????????????
  (Asymptotically unbiased estimator)

51
??????????????????????????? (Asymptotic variance)

• ???????????????????????????????????? (Asymptotic
  variance)

52
(No Transcript)
53
(No Transcript)
54
(No Transcript)
55
CRLB

• ??? ????? CRLB ??????

56

• ??? Log-Likelihood

57
(No Transcript)
58
(No Transcript)
59
(No Transcript)
60

• ????????????????????? ???????????????????????????
  ? (Asymptotically Efficient)
• ??????????? MVUE

61
????

• ??????????????????????????? ??????????????????????
  ?????????????
• ??????????????????? MVUE ?????????????????????????
  ? ??? ?????????????????????