What STATISTICS can do?

1
What STATISTICS can do?

• ----some examples
• ? ? ?
• zwp_at_ustc.edu.cn
• (O) 3600565
  

????? -----A Modern Introduction to Probability
and Statistics Understanding Why and How,
Springer, 2005
2
What is Statistics?

• American Heritage Dictionary ??,
  ??????????????, ????????????????.
• ???????????, ??????????????. ??????????????????.

3
????????

• ???????????????????,????????????
• ???????????,????????????,??Galton???????????,????
  ??????
• ??????????????????????????????????????????

4
????????

• ??????
• ?????---?????????????????
• ?????????
• ????????
• ??????
• ?????????
• ?????????????
• ????

5
?? Economics, Engineering, Marketing, Computer
Science
???? Astronomy, Chemistry, Physics
?? ??
???? Genetics, Clinical Trials, Epidemiology, Phar
macology
?? Agriculture, Ecology, Forestry, Animal
Populations
?? Census, Law, National Defense
6
Ex1 ?????

• 1932-1998
• ?????
• ?????
• ?????
• ???

7

• ???1998???????13???,2002?2????????????,????2002???
  ????
• ????2002?????????34?,????????
• 2006????????????,2002????????12,????2006???????,?
  ?95?????

8

• ????????????vs??????(???????),??Poisson??
  (?????????????????)?

9

• ??1998???13?,2002??????????16?(??Poisson??????)?
• ???????34?,???????????????0.000065,??2002????????
  ??????????,?????????????

10

• 2002??????12?,?1998?????,2006??????14.7?,95?????
  8,23
• ????,???2006?????????????11???,????????12?(???????
  ?????????????),????20.
• ???2006?????????26???,????1998?????????????!

11
Ex2 ?????

• ?????????

12

• ??
• ????
• ????????
• ???? 49.294,
• 95?????41.05171, 57.50797.

13
Ex3 Killer football

• Cardiovascular(????) mortality in Dutch men
  during 1996 European football championship
  longitudinal population study, 2000,British
  Medical Journal,3211552-1554
• ????6?22??????????????????????!

14

• ?????????????????????????????6?22???????????(?????
  )???????????????
• ????????????????!

15
95????
???????6?17?6?27????????????,????? ????????95????
?????????????????
16
Ex4 ?????

• E.S. Pearson ? 18931898 ????????n1375?65??????1
  8??????????, Pearson and Lee (1903) ??????,??
• ????????????????????????

17
????http//mil.qianlong.com/4919/2003/09/21/42_at_16
05800.htm
18

• ??????
• Dheight 30.48690.5326Mheight
• ????????63.78 inches (162cm), ??????????(163.7cm)
  
• 64.4561330.48690.532663.78
• ???, ???95?????152.3cm, 175.14cm

19
Ex5???????

• ????,?????????????????????????????????
• ???????????????????????
• ??????????????,?????????????????????????,????????
  ?????????????

20

• ??????????????,???????????
• ?????????
• ????
• ???
• ?????????,?????????????

21

• ???????????????????????????????
• ???????????????????????
• ??????????????????????????,???????????????????????
  ??

22

• ??????????????????????????????
• ???????????????????,??????????????????12???,??????
  ???3-6???

23
?????????????????
24

• ???????????????????????????
• ???,???????????1????????N,????????????
• ?????????????????????????N????

25

• ????1939?1943???,??5??????1400???,????????????
• ?????1943???????????????????,?????????????????????
  ????????,???????????,?????????????????90 0000
  ?120 0000!

26
An Empirical Approach to Economic Intelligence in
World War II Richard Ruggles, Henry Brodie,
JASA, Vol. 42, No. 237 (Mar., 1947), pp. 72-91
27
Ex6????

• 1983?????????----?????(?)?1/299792458?????????
  ??????????
• ???????????(?)???!

28

• ????????????????????
• Albert Michelson ?1879??6??7??????????????????,???
  ???????

29

• ??????????,????????????????,???????????????
• ????????,???????????,?????????????????????,??????
  ????

30

• ?Michelson???,?????????300000 km/s,?????????????!
• ?????Michelson?????

31
(No Transcript)
32

• ?????????
• Distance
• Radius
• Displacement
• Revolutions per seconds (rps)
• ????????????!?????????????????!

33

• Michelson???distance???2000??,?????????100????????
  ????????????100.006??(????)?????5?,?????1984.94?19
  85.17????,?????????true distance?(??????)!
• ??????????!???????(???,????)?299 944 /- 51 km/s

34

• ??Michelson??????,??????????!
• ??????????????????95?????299 944/-15.5km/s!

http//en.wikipedia.org/wiki/Albert_Abraham_Michel
son
35
Ex7??????

• ????????????

36

• ???? N ????????(??1),??????????? n,???????? S
  ????? n/N ?? (N???)
• n/N ? S
• (????)???????????? n ??????,??????????, ???? a
  ???????,???????? m ,???? n/ma ??S????

37
Ex8 Benford??

• ??????,?????1,2,..,9??????,???1/9
• ?????????,??????,???????????????? ,???? Benford??

??? 1 2 3 4 5 6 7 8 9
?? 0.301 0.176 0.125 0.097 0.079 0.067 0.058 0.051 0.046
38

• Benford???Newcomb(1881)?Benford(1938)?????????
• ????k????
• P(???k)log10(11/k),k1, 2,..., 9
• ??????????????????
• ?? ????????????????????????

39

• ?. Standard Poors (SP)?500?Index??????????(
  1986.1.2 1995.12.29 )?????????Benford???

??? 1 2 3 4 5 6 7 8 9
?? 735 432 273 266 200 175 169 148 126
??? 760 445 315 245 200 169 146 129 116
--- ???(Pearson)????????????
40
Ex 9 ???????

• ????(spam)??????????????. ????,???????????????????
  ????????.

41
Problem examples

• You have won!!!!, you are almost winner of ...
• Viagra, generic Viagra available order now
• Your order, your item have to be hipped
• Lose your weight, no subscription required
• Assistance required, an amount of million 25
  US
• Get login and password now, age above 18
• Check this, hi, your document has error
• Download it, free celebrity wallpapers download

42
How to fight? (techniques)

• ????
• if SENDER contains schacht ACTIONINBOX
• if SUBJECT contains Win ACTIONDELETE
• if BODY contains Viagra ACTIONDELETE
• ?? static rules, language dependent, how many
  rules, and who should define them?
• ???????????(??)?
• Decision Trees
• Support Vector Machines
• Naïve Bayes Classifier
• ?? when no features can be extracted??? Error
  loss?

43
Model overview

• Flow (training/test)

??
??
??????
??
Email?
Spam?
??
44
????

• ?????A???B?????
• P(B A) P(AB) P(B) / P(A)

SPAM
EMAIL
A
Feature set
B
P that A and B occurred together
Calculate P that these features belong to SPAM or
EMAIL class
45
Results
46

• More Amazing Applications of Probability and
  Statistics, visit
• http//www.intuitor.com/statistics/ or
• Search Google
• Read How to lie with statistics