Hotmail Israel Hosted Exchange System

1
Hotmail IsraelHosted Exchange System
MOM 2000

• Idan Lerer
• System Security Manager
• Eyron.net
• idanl_at_eyron.net

Powered ByEyron.net
msn
msn.co.il
2
System Overview

• More than 200,000 active mailboxes (Total of
  380GB)
• Average of 3,000 Outlook Web Access concurrent
  connections
• 50,000 unique logins per day
• More than 1,500 new mailboxes opened every
  day(Option to recovery mailbox name)
• More than 500,000 legitimate inbound / outbound
  messages per day
• More than 300,000 inbound messages per day
  blocked by the Anti-Spam system
• Average number of inbound and outbound messages
  per second 12

3
System Overview

• Server
• Windows 2000 Server IIS5
• Exchange 2000 Back-End Cluster A/P
• Exchange 2000 Front-End
• SQL 2000 Server
• MOM (Microsoft Operation Manager)
• Custom Develop System Tools
• Cluster services

• Hardware
• More than 20 Servers
• 2-Way / 4-WayServers

• SAN Storage System
• Fiber Channel
• Total of more than 1.7Tb

4
Mission Challenge

• Collecting events and performance counters to a
  central point of management
• Maintain system health
• Foresee a problems before they occur
• Catch problems ASAP
• Know about problems before users do
• Capacity planning

5
MOM Monitoring tools

• Windows 2000 Event Viewer
• Windows 2000 Performance monitor
• Scripts (most of them built in)
• WMI scripts (most of them built in)
• Read 3rd Party application log and sys log
• Run command / scripts while problems occurs
  (e.g. System dump file)

6
MOMMicrosoft Operations Manager

• Exchange management pack include more than 1,700
  build-in monitoring rules scripts developed by
  Exchange developer group.
• More than 150 build-in reports.
• Collecting important events to a central
  location.
• Using performance monitor data, which can
  indicate that the server is running low on
  resources.
• Monitoring all Exchange SMTP queues length.
• Monitoring SMTP denialofservice attack.
• Monitoring mail flow between servers in the
  organization.
• MAPI client logon check.
• Monitoring Exchange cluster health.
• Sending alerts on a critical events

7
MOM - Interface
8
Web interface - MOM Dashboard
OWA Current Sessions 3,321 Sessions , SMTP
Current Sessions - 363 Sessions
9
Customize web folder
10
Databases health check
11
Databases health check
12
AD response time
13
MOM SMTP attack notification
14
SMTP queue monitoring -Problems with outbound
messages
15
Tracking mail flow problems
16
How can we plan ahead using MOM ?
17
Plan ahead - Servers
18
Plan ahead - Servers
19
Plan ahead - Bandwidth
20
Plan ahead - Disks
21
Response on problems
22
Exchange 2003 And MOM
23
Aberrant mount of messages received for user
24
Aberrant mount of messages sent by user
25
Receiving domain
26
Top Mailboxes size (consider moving to another
database)
1000
27
Databases size
28
3rd Party Application Log (Anti Spam)
29
Monitor sys log and all networking equipment
30
Summary

• The concept - maintaining a server to collect
  only necessary and critical events - not to
  "drown" in huge number of events.
• Always think ahead, don't wait until problems
  will appear
• Monitor CPU usage - if its over 90 for more than
  5 min - you've got a problem !
• Monitor Disk Free Space - if its less than 20 -
  the problem is near.
• If youve got a problem, think how what can you
  monitor in order to prevent this problem in the
  future.
• Performance monitor is a strong tool - MOM, makes
  it stronger.
• Useful tools for capacity planning and budget
  planning.

31
Question

• ?

Idan Lerer System Security Manager Eyron.net ida
nl_at_eyron.net