??? ?????

1
?????? ????????? ??, ??????????? ? ???????

• ????? ????????

2
? ??? ?? ????? ????????

• ??? ????? ????? ???????, ??? ?????? ??????
• ?????? ?? ?????? ?????????? ?????????? ???????
• ??? ????? ?????????????????? ? ???????????????????
  
• ??? ????????????? ???????????????? ?
  ???????????????? ???
• ??????????? ?? ?????? ????????? ????
• ??????????? ?????? ???????

3
??? ????? ??????, ? ??? ????? ?????

• ??????? ???????? ???, ??? ????
• ????????????? ????????? ??????? ? ?????????
  ?????????
• ????? ???????
• ??????? ?????, ?????????? ?/??? ????????
  ???????????, ??????? ?????? ??? ???????? ??
  ???????????

4
??? ?????????? ????? ???????
???????
????????????
?????????
????
??????
?????

• ?????????? ???????? ?????? ???????
• ??????? ?????
• ??????? ?????????? ?/??? ???????? ???????????,
• ??????? ?????? ??? ???????? ?? ???????????

• ???-?? ????? ?? ???
• ????? ????????? ??? ??????? ????????,
• ????????????????? ??????? ????????????,
• ????????????? ????????
• ?????????? ?????? (???)

• ????????? ??????? ??????????? ?? ???????????
• ?????????????? ???? ??????? ?? ???????

5
??? ?????? ??????
6
?????? ??????????? ??????
Business-critical????? ? ????????????
Non-critical ???????? ????????????
Mission-critical????????????? ?
?????????????????
Life-critical???????? ? ?????

• ??? ????????? ???????, ??? ?????? ?????????????
  ????????? ?? ?????
• ?????????????? ?? ???? ?????? ?????????????
  ?????? ????? ?????? ??????.

7
????? ?? ???????? ???????????

• Airline 5 501
• 4 ???? 1996 ?. ??????-????????, ?????????? ESA,
  ?????????? ?? 37-?? ??????? ??????.
• ????? 7 ???? - ?????????? 500 ?????????
  ?????? ? ????.
• ??????? ?????????????? ?????????? arithmetic
  overflow ??? ?????????? 64-bit fp ? 16-bit signed
  int
• Therac-25
• 1985-87 ??? 6 ???????, ??????????? ???????
  ???????, ???????? ????????????? (? ??????? ???)
• ????? ??? ??????? 2 ?????? ??????????????? ??
  ?????????????, ? ????????? ??????? ?????? ?
  ??????????? ??????.
• ??????? race condition ??? ????????????? ??????,
  ?????????????? ????????????, ? ??????,
  ?????????????? ??????? ?????????

8
?????? ?? ?????? ????????? ??????????? ??????,
?????????? ??????????
9
??? ??????????????? ?????????
????????????
??????
??????????
????????????
???????????? ? ?????????
??? ? ??? ?????? ?????? ????????
????? ?????? ???? ???????, ????? ??? ??? ??????
?????? ?? ???????!
??????? ???????? ? ????????? ???? ???????
????????? ?????????? ?? ? ??? ??, ??? ???? ??????
?????????? ?????????? ? ????????, ??????
????????????
??????? ??? ??????????? ??????? ??????????
???????????? ?????????? ???????, ??????????????
????????, ?????? ????????????? (????)
????? ?????? ???????? ?? ??????-????????
??????? ???????? ???????????, ????????? ??????
..? ??? ? ??? ????????????? ??????
10
??? ??????? ?? ???????? ??????
????????????
??????
??????????
????????????
???????????? ? ?????????

• ?????????????? ??????
• ????????????? ???????????? ? ???????, ????????
  ?????????,
• ????? ???????,
• ????????????????

• ??????????? ? ??????????? ??????
• ????? ????,
• ????????????,
• ???????

???????? ?????????? ??
11
? ????? ????????? ???? ????. ? ?????.

• ?????????? ????????? ??? ???????. ??? ??????
  ????????? ?????? ?????????? (??? ??????
  ????????)
• ??????, ???????????, ?????? ???????? 2
  ????????? ?? 3?
• ??????? ????????? ?????? gt ??????? ??????? ???
  ??????
• ???????????? ??????
• There are two ways to write error-free programs
  only the third one works. (?)Alan J. Perlis

12
??? ????????? ??????????? ?????? ????????
13
Expect the unexpected!
????????? ??????? ????????? ???????????
??????? ????? ???????? (??? ???????)
?????? ????????? ????????????? ????????, ??????????
??????, ???? ?????????? ?????? ???????? ?
?????????? ?? ? ?????? ?????????
14
????????? ??????? ??????? ? ?????? ????????
??????????? ????????????? ????????? ???????????
? ????? (??????? ??? ??????) ????????.
15
?????????????????? ? ??????????????????
???????
?????????
????????????
????
??????
????????? ??????
?????????? ??????????, ??????? ??????
?????
?.????????????
?.????????????
16
????? ????????? ???????????? ???????????? ?
??????? ? ??????? ?? ????????
??????????????????
??????????????????
Business-critical????? ? ????????????
??????????? ?????? ????? ???????? ?
???????????????? ????????????
??????????? ???????????????? ????? ???????? ?
???????????????? ????????????
Mission-critical????????????? ?
?????????????????
Life-critical???????? ? ?????
Non-critical ???????? ????????????
?????????????? ????? ?????? ????? ?? ???????
?????? ? ?????? ????????????
??? ????????? ??????
17

• ??? ??????????? ?????????????????? ?
  ?????????????????? ??????

18

• ???????? ?????,
• ?????? ? ???????

19
?????? ? ????? ????? ?????????? ?? ?????
???????
????????????
?????????
?
????????? ? ????? ??????
?????????? ?????? ??????????
20
????????? ????? ????? ???? ???????? ????? ??????,
? ????????? ?????? ???????? ??????
???????
?????????
?????????? ????????? ???? ????????????
???????? ?????? ???? RAID1 ???????
?????????? ??????????????????? ???????
???????????? ?? ????????? ?? ??????
???????? ?????? ???? RAID1 ???????
??????? ?? ????????, ??? ???????? ??????
????????? ???
21
???? ???? ????? ???? ???????? ?????????? ??????
???????
?????????
?????????? ????????? ???? ???????
???????? ????
?????????? ????????? ??? ????????? ??????
????? ? ??????????? ?????????? ????? ? ???????
???????? ???, ??? ?????? ??????????? ??????
22
?????? ???????
????-gt??????-gt?????-gt????-gt??????...
January 15, 1990 ATT Network Outage
??????????
??????????
????????????
??????????
????????????
??????????
114 ???????????? ??????????????? ?????? 6 ??????,
??????? 60000 ??????? ??? ????????????? ????? ??
9 ?????
????????????
??????????
????????????
???, ?????????? ? ???????????? ??????? ???
????????? ????????? ? ????
????
23
???????? ?????, ?????? ? ??????? (????)

• ???????????? ??????, ????? ?? ??????? ??? ??????
  ????????
• ????? ??????????? ?? ?????
• ????????? ????? ?? ??????
• ?????? ???????

24
??? ????? ??????? ??? ????????????? ??????

• ?????????? ? ???????????????
• ???????????
• ?? ????????? ??????????????? ? ?????? ???????
• ??????????
• ???????????? ????????????????? ???????.
• ???? ??????????? ??? ????????? ??? ???????????
  ??????????? ?????? ? ????????? ? ????????? ??????
• ???????????????? ?? ??????

25
???????????? ???????? ???
26
?????????????? ????????????????

• ??????????? ?????????? ??????? ?????????????,
  ??????? ?????? ???
• ?????????????, ???????, ?????????? ?????????,
  ????? ???????? ? ???????????? ?????? ?????????
• ?????????? ?????????????
• ??? ????? ?????, ???? ?? ???????? ????????
• ??? ?????? ??????????, ???? ?? ???????? ????????
• ???? ??? ????????? ??????, ???? ?? ????????
  ????????
• A good programmer is someone who always looks
  both ways before crossing a one-way street. 
  (?)Doug Linder

27
???????????? ??????????? ? ???????? ??????

• ?????? ?? ????????????? ????????????? ??????????
  ? ?????????? ????
• ????????? ???????????? ??????? ??????
• ??? ?????, ???? ?????????? ??????? ?????????????
  ??? ?????? ?? ???????
• ?????? ?? ?????? ? ?????????? ????
• ?? ?????????? ????????????? ??????
• ?????? ????????? ????????? ?????????? ???????? ?
  ???????????? ???????????? ??????

28
???????????? ????????? ??????

• ???????????
• ?????????
• Retry
• ????????????? ???????? ?? ?????????
• ?????????????? ???? ?????????
• ?????????? ??????????
• ??????????? ??? ?????????? ???????, ???????? ?
  ????????????? ??????, ???????? ??????????
  ??????????? ????

29
???????????? ?????? ???????
30
???????

• ?????????? ? ???????????????
• ???????? ?????????????? ???????????
• ???????? ????????????????? ping/echo, heartbeat
• ????????? ?????????????? (deadlocks, ??????
  ??????)
• ???????????
• fault containers
• ??????????
• ?????????
• Roll-back ??????? ? ??????????? ????????
  ????????? (ex. ??????????, ?????????, ??????
  ??????)
• Roll-forward ????? ?????????? ???????? ?????????
  ? ??????? ? ???? (??. ??????????????, ????????
  ?????? ??? ???????????? ????????)
• ???????? ????????????, ????? ??????? ?????
  ?????????? ???????????????? ? ?????? ??????
  ????????? ?????????

31
????????????? ???????????? ??? ??????????????????

• Redundancy
• ???????? ??????????, ?????????? ??????????, ??
  ??????? ????? ????????????? ? ?????? ??????
  ???????? ?????????? (failover)
• Replication
• ????????? ?????????? ????????? ????????? ???? ?
  ?? ?? ?????? ???????????, ????????? ??????????
  ????????????
• Diversity
• ????????? ????????? ?????????? ??????????
  ????????? ???? ? ?? ?? ?????? ???????????,
  ????????? ?????????? ????????????

32
??? ? ??????? ????????????? ????????????????
?????????
????????????
??????
??????????
????????????
???????????? ? ?????????
???? ? ?????? ?????????? ????? ??????? ???????
????????? Graceful degradation ??? shutdown?
?????????? ????????? ????????? ??????, ??????????
???????? ??????????????, ??? ????????????? ??????
????????????
Defensive Programming ????? ???, ???????
????????? ???????? ? ????? ?????????
????????? ????????? ??????? ? ????????
Fault injection ???????????? ???????? ?????,
????? ??????????? ???, ?????????????? ??????
33
??????????

• ????? ??????????? ??????? ???????? ??????,
  ???????????? ??????? ? ?? ??????.
• ??????????? ??????? ??????????? ?? ?????????? ??
  ????????????????.
• Expect unexpected ??????? ?????? ???? ?????? ?
  ?????
• ?????????????????? ??????????? ???????????
  ???????
• ?????????????????? ?????????? ???????????
  ???????
• ?????? ??? ??????????? ?????????????????
  ?????????
• ?? ??????????? ??????????????
  ????????????????
• ?? ?????? ??????? ???????? ??????????????
  ??????????? ??? ??????????? ? ??????????????
• The most likely way for the world to be
  destroyed, most experts agree, is by accident. 
  Thats where we come in were computer
  professionals.  We cause accidents. Nathaniel
  Borenstein

34
????????

• olga.sharonova_at_gmail.com

35
???????!
36
??????????? ????????? ??????? ???????
Business-critical????? ? ????????????
Non-critical ???????? ????????????
Mission-critical ????????????? ?
?????????????????
Life-critical???????? ? ?????
?????? ? ??????? ??????????? ?? ????????
??????????? 2 ???? ? ????
???????? ? ????????? ????????? 90 ???????
?????????? ????? ???????? ? ???????????? ??
????????
?????????? ? ?????? ????? ???????? ???????? ?
?????????
?????? ???????? ??? ?????? ? ??????? 5 ???
??????? ?????????? ??????? ? ????????? ? ????????