MRTG The Multi Router Traffic Grapher

1
MRTG The Multi RouterTraffic Grapher

• Tobias Oetiker, MRTG - The Multi Router Traffic
  Grapher, Proceedings of the Twelfth Systems
  Administration Conference (LISA 98) Boston,
  Massachusetts, December 6-11, 1998

2
Motivation of the MRTG

• In Summer 1994, the De Montfort University in
  Leicester, UK, had one 64 kBit Internet link for
  more than 1000 networked computers.
• Everybody was interested in knowing how the link
  was performing
• Multi Router Traffic Grapher (MRTG) was developed
  by Tobias Oetiker to provide the users on campus
  with current and detailed information about the
  status of the link.
• MRTG 1.0 was published on the Internet in spring
  1995

3
What does the MRTG 1.0 do?

• It used the SNMP to query the Octet Counters''
  of the university's Internet gateway router
• The CMU SNMP package is used as an external
  utilities to do SNMP queries
• The average transfer rate of the Internet link
  was derived for every five-minute interval
• A web page was generated with four graphs showing
  the transfer rates for the last day, week, month,
  and year
• MRTG logged its data to an ASCII file called
  logfile, rewriting it every five minutes
• The pnmtogif tool from the PBM (Portable BitMap)
  package was used to convert graphs to GIF format
• The visual presentation on the Web allowed
  everyone with a web browser to monitor the status
  of the link

4
MRTG snapshot
5
MRTG snapshot
6
MRTG snapshot
7
MRTG snapshot
8
Problems of MRTG 1.0

• Scalability
• Rewriting of the logfile was not fast enough to
  monitor larger number of links
• The pnmtogif tool from the PBM (Portable BitMap)
  package is not fast enough to monitor larger
  number of links
• Portability
• The CMU SNMP package proved to be rather
  difficult to compile on various platforms at that
  time

9
MRTG-2

• Developed by Dave Rand, released in January 1997
• Dave Rand wrote a small C program called rateup
• The rateup implements the two most CPU intensive
  tasks in C
• logfile rewriting
• graph generation
• rateup uses Thomas Boutell's GD library which
  enabled it to generate GIF files much faster than
  pnmtogif (Thomas Boutell, GD Lib, a graphics
  library for fast creation of GIF images,
  http//www.boutell.com/gd/)
• The SNMP portability problem was solved by
  switching from CMU's snmpget to Simon Leinen's
  Perl SNMP module,
• Simon Leinen's Perl SNMP module is written in
  pure Perl
• thus making it virtually platform independent.
• (Simon Leinen, Perl 5 SNMP Module, an SNMP
  client implemented entirely in Perl,
  http//www. switch.ch/misc/leinen/snmp/perl/)

10
More on MRTG-2

• MRTG-2 is faster than MRTG 1.0
• MRTG-2 is more user friendly
• A cfgmaker tool is included for building a
  skeleton configuration file for a router by
  reading its interface table via SNMP
• This allows a lot of people to successfully
  configure MRTG even when
• they do not know too much about SNMP
• they do not know about how to find out which
  physical router interface is mapped to which SNMP
  variable

11
The logfile in MRTG-2

• The logfile stores traffic data with a decreasing
  resolution into the the past
• Data older than two years is dropped from the
  logfile
• MRTG-2 logfiles are stored in plain ASCII
• Each line starts with a time stamp followed by
  the corresponding traffic data
• The file starts with the most current entry and
  ends about two years in the past

12
Logfile Processing

• A logfile is read as a whole, processed in memory
  and written back to disk

13
Data Consolidation in logfile
14
Key Elements to the Success of MRTG-2

• Simple Setup The configuration is done through
  simple ASCII text files
• Easy Maintenance The logfiles do not grow in
  size
• Friendliness The HTML pages created by MRTG are
  easy to understand
• Integrated Solution
• MRTG performs all the tasks required for traffic
  monitoring
• No external database or SNMP packages are
  required to make it work

15
Limits of MRTB-2

• Limited to 600 router ports queried every five
  minutes due to the way the logfiles are updated
• Does not support non traffic data sources

16
MRTG-3

• The goals for MRTG - 3
• Speed to support monitoring of more data
  sources per five minutes
• Flexibility to support non traffic data
  sources

17
Round Robin Database

• A new mechanism for data storage in MRTG-3
• The Round Robin Database is so much faster and
  more configurable than MRTG-2
• The new design allows to store in the order of a
  thousand data values per second in a Round Robin
  Database
• It is implemented in a C program called rrdtool

18
Round Robin Database

• The RRD format uses doubles for data storage
• This gets rid of the integer overflow problems
  seen when monitoring really fast routers with
  MRTG-2
• It allows to log small numbers like the load of a
  machine without scaling
• The RRD can also store unknown data values
• allows it to distinguish between situations where
  the data input is zero and those when no new
  valid data can be obtained.
• Parameters are configurable
• The number of log entries
• The resolution of the log
• The number of data sources to log in parallel
• The data values in the RRD are stored in native
  binary format
• This makes access to the data more efficient,
  because no conversions are necessary anymore
• A cookie in the header of the RRD is used to test
  if the RRD is compatible with the architecture it
  is being read from

19
Round Robin Database

• Can accept multiple data sources (DS1, DS2, DS3)
  in parallel
• One Round Robin Database (RRD) can contain one or
  more Round Robin Archives (RRA)
• Each Round Robin Archive has its special
  properties for time resolution, size and
  consolidation method

20
Round Robin Database

• The update interval of an RRA must be a multiple
  of the base update interval of the RRD
• Several values at the RRD's base resolution are
  consolidated into one value at the RRA's
  resolution using the consolidation method defined
  for this RRA
• An array of pointers identifies the most current
  entry in each RRA, such that only one write
  operation is necessary to update an RRA

21
Data Sources Re-sampling

• The re-sampling of the data takes care of the
  problem that it is not always possible to get new
  data at the desired point in time
• storage is much simpler when the data is equally
  spaced along the time axis

22
Graph Generation

• The rrdtool also does graph generation
• It allows
• to produce graphs of any size, spanning an
  arbitrary time period
• to draw data from a number of data sources stored
  in different RRDs
• The graphing engine determines sensible default
  values for its configurable parameters, allowing
  the user to concentrate on the fine tuning
• Almost every aspect of the graph's visual
  appearance is configurable by overriding the
  automatic default values
• Often configuration is not necessary, because the
  RRD tool has several functions which
  automatically tune features like axis labels and
  scaling to fit the displayed data.