Troubleshooting Message Flow in Microsoft Exchange 2000 Tom Jones PSS Technical Lead Enterprise Mess

1
Troubleshooting Message Flow in Microsoft
Exchange 2000 Tom Jones (PSS)Technical Lead
Enterprise Messaging SupportMicrosoft
Corporation
2
Presentation Outline

• Review
• Review of transport core
• Review of message flow
• Troubleshooting
• Overview of troubleshooting approach
• Gathering basic details
• Checking for common issues
• Gathering greater details
• Tools of the trade
• Questions and Answers

3
Transport Core High Level
Submission
Delivery
4
Transport Core Basic Components
Submission
Delivery
Transport Core
Categorizer
Advanced Queuing
Routing
Remote Delivery
5
Transport Core Basic Operations
Submission
Delivery
Transport Core
Advanced Queuing
SMTP
SMTP
Local Store
Local Store
Routing
Legacy MTA
Legacy MTA
6
Message Flow Components
Internet
Client
Information Store
\Exchsrvr\Mailroot\ Vs1\Queue
7
Message Flow Outbound to Internet

• Client submits message to information store
• Information store submits message to Advanced
  Queuing (AQ)
• AQ submits the message to the categorizer
• The categorizer (Cat) does the following
• Resolves and validates all recipients
• Determines limits and restrictions for all
  recipients
• Expands distribution lists (DLs) if necessary
• Determines if the message is for local or remote
  delivery
• The categorizer returns the message to AQ

8
Message Flow Outbound to Internet (2)

• AQ requests the routing engine determine a route
• After a route is found, AQ submits the message to
  remote delivery
• Remote delivery consists of the following
• The domain mapping table (DMT) sorts messages by
  priority, domain, and delivery time
• The connection manager establishes and manages
  connections with remote hosts
• SMTP transfers the message

9
Message Flow Outbound to Internet (3)
Client
Internet
3
AQ
Categorizer
1
5
4
2
Information Store
8
Remote Delivery Queues SMTP
Routing
6
7
10
Message Flow Inbound from Internet

• Remote SMTP host connects to our SMTP service
• SMTP receives the message and writes it to the
  \Queue folder
• \Program Files\Exchsrvr\Mailroot\1\Queue
• SMTP submits the message to AQ
• AQ submits the message
• Categorizer for categorization
• Routing for routing decision if message is not
  local
• Remote delivery if the message is for another
  Exchange 2000 server
• Information store if the message is for local
  delivery
• AQ submits the message to the information store
  (because the user is locally homed in this
  example)
• Information store delivers the message to the
  users mailbox

11
Message Flow Inbound from Internet (2)
Internet
Client
SMTP Client
AQ
Categorizer
Information Store
Remote Delivery Queues SMTP
Routing
\Queue
12
Message Flow Exchange 2000 to Exchange 2000

• Local Exchange 2000 server
• Client submits to the store
• Store submits to AQ
• AQ manages the following operations
• Submits to categorizer for categorization
• Categorizer recognizes the message is for a
  remote user
• Categorizer returns the message to AQ
• AQ requests a route for the routing engine
• Routing engine returns a route
• AQ submits the message to remote delivery queue
• SMTP on local server sends the message to the
  remote server

13
Message Flow Exchange 2000 to Exchange 2000 (2)

• Remote Exchange 2000 server
• SMTP receives the message and persists the
  message in the \Exchsrvr\Mailroot\Vsi1\Queue
  folder
• SMTP submits the message to AQ
• AQ does the following
• Sends the message to the categorizer
• Categorizer detects the message is for local
  delivery
• Categorizer returns the message to AQ
• AQ submits the message to store
• Store on remote Exchange 2000 server delivers the
  message

14
Message Flow Exchange 2000 to Exchange 2000 (3)
Server1
Server2
Client
Information Store
Information Store
SMTP
SMTP Client
15
Message Flow Exchange 2000 to Exchange 5.5

• Exchange 2000 server
• Client submits to store
• Store submits to AQ
• AQ does the following
• Submits the message to categorizer for
  categorization
• Categorizer determines the message is bound for
  a legacy e-mail system and returns the message
  to AQ
• AQ returns the message to the store
• Store passes the message to the MTA
• Exchange 2000 MTA sends to Exchange 5.5 MTA

16
Message Flow Exchange 2000 to Exchange 5.5 (2)

• Exchange 5.5 server
• MTA receives message from Exchange 2000 MTA
• MTA looks up the recipient in the Exchange 5.5
  directory
• MTA passes the message to the local information
  store
• The information store delivers the message

17
Message Flow Exchange 2000 to Exchange 5.5 (3)
Exchange 2000
Exchange 5.5
Client
Information Store
Information Store
X.400
MTA
MTA
Directory
18
Message Flow Exchange 2000 to Foreign E-Mail
System

• Follows the same path as Exchange 2000 to
  Exchange 5.5, up to the point the AQ returns the
  message to the information store
• THEN
• Information store moves the message to the
  MTS-OUT folder of the connectors mailbox
• The connector converts the message to the foreign
  format and moves the message to RDY-OUT
• The connector connects to the foreign system and
  transmits the message

19
Message Flow Exchange 2000 to Foreign E-Mail
System (2)
Exchange 2000
Client
Information Store
MTS- OUT
RDY- OUT
Foreign E-mail System
Foreign Connector
20
Troubleshooting Overview

• Troubleshooting is a methodical approach to
  identifying and solving a problem
• Define the problem as narrowly as possible
• Get basic configuration details
• Check to see if the problem is a known issue
• If not, gather more details through logging,
  tracing, dumps, et cetera
• Research in Knowledge Base, TechNet, and other
  online sources
• Consult with peers

21
Basic Details Whats the Problem?

• The problem is (fill in the blank)
• Most message flow problems start out as
• Mail is not being delivered
• Mail is being returned undeliverable
• Mail is queuing but not moving
• Mail is vanishing
• Mail is being misrouted

22
Basic Details Whats the Problem? (2)

• Ask these questions to clarify
• What do you mean when you say mail isnt being
  delivered?
• Where is the mail going?
• Is this a problem with inbound or outbound?
• Is the problem between Exchange 2000 and
  Exchange 5.5?
• What exactly does the NDR say? Can you send me
  the NDR?
• What exactly does the queue status say? Can you
  send me a screen shot?

23
Basic Details Configuration

• Gather details about Exchange
• What versions of Exchange?
• What service packs of Exchange and Microsoft
  Windows 2000 ?
• What versions of foreign e-mail system?
• How many Exchange servers in the RG?
• What types of servers (mailbox, bridgehead, et
  cetera)?
• Gather details about Windows
• How many domain controllers? Global catalogs? DNS
  servers? Windows 2000 sites?
• How is DNS configured on Exchange 2000?
• What type of server is Exchange 2000 (member or
  domain controller)?

24
Basic Details Scope of the Problem

• When did this problem begin?
• Has this ever worked?
• How many users is this affecting?
• Can this be reproduced with a new user?
• Can this be reproduced with different client
  software?
• Can this be reproduced from a different Exchange
  server?
• Is the problem affecting only inbound or only
  outbound?

25
Basic Details NDRs

• Exchange 2000 non-delivery reports (NDRs) follow
  Internet standard RFC 1893
• See also Q256321 and Q284204
• The code itself is very revealing
• Pay attention to the server that generated the
  NDR
• Example
• JoeUser_at_Domain.com on 2/11/2002 1235 PM
• There was a SMTP communication problem with
  the recipient's email server. Please contact
  your system administrator.
• ltExchange1.YourDomain.com 5.5.0 smtp550
  unknown user ltJoeUser_at_Domain.comgtgt

26
Basic Details A Little More Basic Info

• What events are reported in the Event Viewer?
• Get the event logs.
• Turn up Diagnostic Logging through the registry
  HKLM\SYSTEM\CCS\Services\MSExchangeTransport\Diag
  nostics set ALL categories to 7
• (Note the registry key above has been line
  wrapped for readability.)
• Where is the message going?
• Use the Queue Viewer and message tracking
• Get screen shots of Queue Viewer and message
  tracking
• Search the KB or TechNet using
• The NDR code
• The event IDs and errors in the description of
  the events
• Status of queues (that is, Remote domain not
  responding)
• Transport component where mail is stalled (AQ,
  CAT, SMTP, RESVC)

27
Common Issues Outbound to Internet

• Common scenario Cant send to the Internet, but
  inbound message flow is fine, and user to user
  message flow is fine.
• Common issues
• Recipient policies are misconfigured
• Use LDP or LDIFDE to get the recipient policies
• Make sure the SMTP address is valid for the
  external domain
• Internet message formats are misconfigured
• Make sure there is a Default entry where the
  domain is
• Make sure there are no restrictions that would
  prevent delivery
• SMTP connector has an invalid address space
• Address space should usually be
• Customers sometimes enter their local domain for
  the address space
• Router/firewall/proxy is blocking port 25 or PIX
  (MailGuard feature)
• DNS misconfigurations

28
Common Issues Inbound from Internet

• Common scenario cant receive from the Internet,
  but outbound is fine, and user to user message
  flow is fine.
• Common issues
• Router/firewall/proxy misconfigurations
• Anti-virus gateways
• Smart hosts
• MX record local domain is missing or
  misconfigured
• Use Nslookup to troubleshoot
• Run queries from a couple of different name
  servers
• Recipient policies are misconfigured
• Use LDP or LDIFDE to get the recipient policies
• Make sure the SMTP address is valid for the
  external domain
• Access or relay settings on the SMTP virtual
  server are misconfigured
• Enable all methods of access
• Allow relay but set restrictions (otherwise you
  are an open relay!)

29
Common Issues Exchange 2000 to Exchange 5.5

• Common scenario cant send to/from Exchange 5.5,
  but message flow between users on same system is
  fine.
• Common issues
• Recipient CA is misconfigured or malfunctioning
• Carefully verify settings in the recipient CA
• Verify successful replication of recipient data
  between systems
• Recipient policy is misconfigured
• Recipient Update Service isnt stamping User
  objects with correct attributes
• Get a dump of User objects
• Check for required attributes (Q281761)
• Check for TargetAddress on User objects (should
  only be on Contacts)
• Default public folder store value on mailbox
  store properties points to an invalid or
  unavailable public folder store
• MTA to MTA communication
• Make sure the MTA Stacks service is started
• Name resolution problems (Q303156)

30
Common Issues Exchange 2000 to Exchange 2000

• Common scenario cant send to other users on the
  same or adjacent Exchange 2000 servers. If you
  track the messages, they seem to stall in the
  Messages awaiting directory lookup queue.
• Common issues
• Router/firewall/proxy misconfigurations
• Use Telnet to simulate mailflow server to server
• Pay careful attention to the ESMTP verbs returned
  when you issue EHLO (Q290290)
• Public folder store value on mailbox store
  properties points to an invalid or unavailable
  public folder store
• Recipient policies are misconfigured or Recipient
  Update Service is not stamping the users with
  valid attributes
• Domain controller/global catalog replication
  problems

31
Common Issues Exchange 2000 to Foreign E-Mail
System

• Common scenario cant send to/from foreign
  e-mail system, but user to user message flow on
  the same system is just fine. Often, this was
  working with Exchange 5.5, but has now stopped
  working.
• Common issues
• Connectivity component to foreign system isnt
  functioning properly
• MTA transport stacks is not started
• One mail message is blocking the entire queue
• Recommendations
• Turn up diagnostics logging to get more details
• Lotus Notes Connector Install a different
  version of the Notes Client
• GroupWise Connector Verify permissions to the
  NetWare server
• Gather details and call Product Support Services
  (PSS)

32
Advanced Details Queues and What They Mean
33
Advanced Details Queues and What They Mean (2)
34
Advanced Details Queue Viewer
35
Advanced Details Message Tracking
36
Advanced Details Logging

• Diagnostic logging
• Configure Transport through the registry
• HKLM\SYSTEM\CCS\Services\MSExchangeTransport\Diagn
  ostics
• Set all categories to 7
• Configure MTA diagnostics through the ESM
• Set X.400 Service and Interface to maximum
• Transport event sinks
• Protocol logging Protolog.dll (obtain from PSS)
• Message archival ArchiveSink.dll (ships with
  Exchange 2000 SP2)

37
Advanced Details Configuration

• Ipconfig /all gt ipconfig.txt
• Netdiag /Debug /L
• DCDiag

38
Advanced Details Dumps

• Get LDP dumps of the following (depending on the
  scenario)
• User objects
• Contact objects
• SMTP virtual server
• Recipient policies
• Mailbox store
• Get admin dumps from Exchange 5.5
• Log in with raw mode, hold CTRL key down while
  selecting Raw Properties
• Site Addressing object, Connectors, Users, and
  MTA
• Output from various tests
• Telnet
• Nslookup
• Adsutil.vbs /smtpsvc /enumall

39
Advanced Details Traces

• Winroute (Q281382)
• Looks at the link state routing table
• Look for bad address spaces or red Xs
• Regtrace (Q238614)
• Captures debug level information
• Must be sent to PSS for analysis
• Network Monitor
• Captures network traffic
• Ships with Windows 2000

40
Transport Troubleshooting Logic Summary

• Did the message return undeliverable? Whats the
  NDR?
• If not, wheres the message? What queue? What is
  the status of that queue?
• Track the message. Where does it end up?
• Turn up diagnostic logging. What events are
  there?
• Troubleshooting guides from PSS
• Q281800 XCON Troubleshooting Message Failures
  in Exchange 2000
• Q257265 XCON General Troubleshooting for
  Exchange 2000 Transport Issues
• http//www.microsoft.com/exchange/techinfo/adminis
  tration/2000/MessageFlow.asp

41
Transport Tools Summary

• Nslookup
• Telnet
• Queue Viewer
• Event Viewer
• Message Tracking Center
• Winroute
• Regtrace
• Network Monitor
• LDP, LDIFDE, CSVDE, and ADSIEdit
• MetaEdit and Adsutil.vbs

42
Questions and Answers
43

• Thank you for joining us for todays Microsoft
  Support
• WebCast.
• For information about all upcoming Support
  WebCasts
• and access to the archived content (streaming
  media
• files, PowerPoint slides, and transcripts),
  please visit
• http//support.microsoft.com/webcasts/
• We sincerely appreciate your feedback. Please
  send any
• comments or suggestions regarding the Support
• WebCasts to feedback_at_microsoft.com and include
• Support WebCasts in the subject line.