Using Heterogeneous Paths for Interprocess Communication in a Distributed System

1
Using Heterogeneous Paths for Inter-process
Communication in a Distributed System

• Vimi Puthen Veetil

Instructor Pekka Heikkinen M.Sc.(Tech.)
Nokia Siemens Networks Supervisor Professor
Raimo Kantola
2

• Agenda
• Background
• What is Performance Based Path Determination?
• Objectives
• Performance Tests
• Delay measurements
• Throughput measurements
• Applying PBPD

3
Background (1/4)

• Performance of inter-process communication in
  distributed systems is of utmost importance
• Influences the performance of the entire system
• Heterogeneity in the communication needs of
  processes running in distributed processing
  systems.
• Some events require low latency communication,
  some other might need high bandwidth
• Several different communication networks such as
  ATM, HIPPI, FDDI etc. available today
• Each originally developed for a different
  application domain.
• Different performance for different types of
  communication
• Processing nodes having in-built support for
  multiple heterogeneous networks is becoming
  common
• Simultaneous use of multiple networks for
  performance improvement not in wide use.

4
Background (2/4)

• What is Performance Based Path Determination ?
• A method of utilizing multiple heterogeneous
  networks available in a distributed system to
  enhance the performance of each type of
  communication between processes.
• Utilizing the difference in the performance
  characteristics of different paths
• Two techniques
• Performance Based Path Selection (PBPS)
• Performance Based Path Aggregation (PBPA)

5
Background (3/4)

• Performance Based Path Selection

• Applicable when one of the networks exhibits
  better performances over the other(s) in one
  situation, while another is better in another
  situation.
• Dynamically select the appropriate communication
  path for a given communication event
• Using some message parameters such as message
  size, the type of communication etc.

6
Background (4/4)

• Performance Based Path Aggregation

• Applicable when two co-existíng networks show
  similar characteristics
• Two identical networks are aggregated into a
  single virtual network
• Each message divided into submessages and
  transferred over available networks
  simultaneously.
• Segmentation into submessages and reassembly at
  destination can add substantial overhead.

Submessage size can be calculated by
f1(x)x m1 t1 a1m1 l1 f2(x)x m2 t2
a2m2 l2 t1 t2 m m1 m2 Solving.
7
Objectives

• Measure and compare the performance of ATM and
  Fast Ethernet as node inter-connects in a
  distributed system.
• Investigate the possibility of achieving
  performance enhancement in inter-process
  communication in an experimental distributed
  system, using the techniques of Performance Based
  Path Determination, when ATM and Ethernet
  co-exist.

8
Performance Tests

• Tests done on a mini-network
• Intel PCs connected back to back (i586 i686)
• Independent connections with Fast Ethernet and
  ATM
• Operating System RH Linux (2.4.18)
• Ethernet NIC of type 3COM 3c905c-Tx
• ATM NIC of type ENI 155p-MF
• Measurement from the perspective of application
  programs
• UDP/IP and TCP/IP protocols used over Ethernet.
• AAL5 used for ATM
• No load in the network or on CPU

• Tested Parameters
• Delay as a function of message size
• Throughput as a function message size

9
Delay measurements

• Round trip delay measured
• Message size range 5bytes 60k bytes used
• Results
• For a minimal size message
• UDP/IP has one way delay of 63µs
• TCP/IP has one way delay of 84.5
• ATM has one way delay of 82.5
• Below 200 bytes Ethernet with UDP has lower delay
  (25 better)
• For bigger messages, ATM performed better
• E.g., for 2000 bytes ATM showed about 50
  improvement

10

• Delay (Contd.)
• Delay breakdown
• Propagation delay negligible in our test
  results
• Transmission delay Significant as message size
  increases
• E.g., for 1000 bytes, for Ethernet this value is
  80µs and for ATM 51µs
• Significant portion of total delay for 1000
  bytes 150µs and 123µs respectively
• Nodal processing - Major contributor
• Found to be ? 29µs for sending and ? 65µs for
  receiving for Ethernet using UDP for 1000 bytes
• Most expensive - Interrupt handling and copying
  data to user space

11
Throughput measurements

• Receiving throughput measured
• Faster processor used as receiver
• Results
• Ethernet reached a maximum throughput of 93.3Mbps
  with TCP
• Ethernet reached a maximum throughput 95.64Mbps
  with UDP
• ATM reached a maximum throughput of 135Mbps
• Overhead due to protocols reduces the maximum
  achievable throughput
• Below 1000 bytes, Ethernet offers higher
  bandwidth!

12
Delay measurements with background traffic

• Delay with load in the network and on the CPU
• Multiple applications communicating
• Similar kind of traffic in the background
• Results
• Performance of Ethernet degrades.
• ATM has no significant impact on delay
• Available throughput dropped

13
Performance Tests

• Conclusions from the tests
• ATM has higher throughput for bigger messages and
  smaller delay for bigger messages
• Delay comparable to Ethernet for small messages
  (Depends on transport protocol)
• Better performance on a loaded network
• Overall ATM gave a better performance
• We need more information, e.g., goodput ratio,
  connection set up time, reliability etc.
• Suitability to a system depends on the
  application domain, nature of inter-process
  messaging of the system etc.

But...
14
Applying PBPD to the test network

• Performance Based Path Selection
• Delay
• For messages smaller than 200 bytes, 20µs
  improvement if Ethernet used with UDP
• And above 200 bytes, ATM behaves better.
• By dynamically selecting appropriate path, delay
  performance improves
• Applicability in real systems depends on the
  system
• Throughput
• Below 1000 bytes Ethernet offered higher
  throughput
• For bigger messages ATM is better
• Yes! Excellent possibility for improvement!

15
Applying PBPD to the test network

• Performance Based Path Aggregation

• Implemented user process, that can segment and
  sent messages over the two paths using two
  threads
• Used pre-determined sizes for test purposes
• Similar process for receiving
• Delay
• Performance degrades
• Segmentation and reassembly adds substantial
  overhead
• Also other factors sending and receiving slower
  than for a single network
• Throughput
• Significant improvement
• Better than using Ethernet alone, for the entire
  tested message size range
• Better than ATM for above 2500 bytes

16
Applying PBPD

• Conclusions
• PBPD can offer performance improvement in some
  systems.
• Not a surefire solution for all performance
  problems.
• Depends on many factors including processing
  power of computing nodes, used protocols etc.

17
Future Work

• Numerous possibilities!
• Cost effectiveness of PBPD
• Whether additional hardware cost and RD costs
  are justified by the performance improvement
• Goodput Ratio and connection setup time for ATM
  and Ethernet

18
Thank you!