The eMule application of the eDonkey protocol

1
The eMule application of the eDonkey protocol

• Yoram Kulbak

Advisor Danny Bickson Academic supervisor Prof.
Scott Kirkpatrick
2
Agenda

• Introduction
• The protocol
• Client - Server protocol
• Client to Client protocol
• The application
• Queue management
• Credit system
• Download part ordering
• Corrupted download recovery

3
Introduction

• eMule is a P2P file sharing network
• Based on an extended eDonkey protocol
• Uses both TCP and UDP
• Client Server architecture
• Open source client
• High rate of expansions and improvements
• Latest sources can be found at sourceforce
• There is also an official website

4
The eMule Network
5
Terminology

• Client ID 32 bit identifier provided by the
  server upon connection
• Valid only through the session lifetime.
• May be either a high ID or a low ID
• User ID A 128 bit value produced by
  concatenating random numbers
• Used to identify a client across sessions
• File ID A 128 bit hash calculated using MD4.
• A file is divided to 9.28 MB parts which are
  hashed separately and then combined

6
The Protocol

• Each message starts with a 6 byte header
  protocol, size ,type
• Empty messages are allowed
• Encoded are Integers, Strings and bit sets
• Strings are encoded using a 2 byte length, value
  encoding
• Message tags

7
Agenda

• Introduction
• The protocol
• Client - Server protocol
• Client to Client protocol
• The application
• Queue management
• Credit system
• Download part ordering
• Corrupted download recovery

8
Connection establishment

• Client Identification
• Determines whether the client gets a low ID

9
Connection startup

• This scenario assumes that the client has a
  requested file list
• The client sends his files
• The server replies with status information
• The client asks for sources for his files
• The server replies with his available sources

10
File search

• Search packet supports Boolean conditions
• A search usually follows by a source request

11
Callback mechanism

• Allows a high ID client to connect to a low ID
  client by asking a call back
• Both clients must be connected to the same server

12
Agenda

• Introduction
• The protocol
• Client - Server protocol
• Client to Client protocol
• The application
• Queue management
• Credit system
• Download part ordering
• Corrupted download recovery

13
Initial Handshake

• The handshake is symmetric
• The eMule info message is used mainly to
  determine peer capabilities
• Messages marked in gray are part of the eMule
  protocol extensions

14
Secure identification

• Used to prevent user impersonation motivated by
  the credit system
• Defined in the extended protocol, RSA based,
  simple challenge response scheme

15
File requests

• There is redundancy in the messages
• The sources request and answer complement the
  server source query and is useful when the other
  client is still downloading the file

16
Enlisting in the upload queue

• When As queue is not empty B is enlisted into
  As queue
• When B reaches the top of As queue, A connects
  to B in order to start the upload

17
Data Transfer

• The major part of eMules network activity
• The extended protocol allows sending compressed
  parts

18
Agenda

• Introduction
• The protocol
• Client - Server protocol
• Client to Client protocol
• The application
• Queue management
• Credit system
• Download part ordering
• Corrupted download recovery

19
Download queue

• The eMule client connects to sources in the order
  they were received from the server / UDP
  connection
• There is a special mechanism which deals with the
  corner case where the remote client is a
  potential source for several files

20
Upload queue

• A priority queue, based on waiting time
• At the head of the queue are clients with the
  highest Score
• Score Rating x (seconds in queue)/100
• Or 0x0FFFFFFF (infinite) in case of a friend
• The rating initial value is 100
• Banned users rating value is 0

21
Upload queue, cont

• Downloading clients initial rating is different
• In the first 15 minutes of a download the initial
  rating is 200
• A new downloading client with higher rating may
  push a downloading client out
• The rating is modified according to these factors
• Credits (modifier x1 - x10)
• Shared file priority (modifier x0.2 x1.8)

22
Agenda

• Introduction
• The protocol
• Client - Server protocol
• Client to Client protocol
• The application
• Queue management
• Credit system
• Download part ordering
• Corrupted download recovery

23
The credit system

• The credit system awards users for uploading
  files
• Client credits are not global, it is a value
  assigned to every ordered (client, client) pair
• The credit values range between 1-10
• Secure identification prevents false credit claims

24
Calculating the credit value

• The minimum of the following
• Uploaded Total x 2 / Downloaded Total
• SQRT(Uploaded Total 2)
• Boundary conditions
• If Uploaded Total is less than 1MB than Modifier
  1
• If Downloaded Total 0 then Modifier 10
• The modifier may not be lower than 1 or higher
  than 10

25
Agenda

• Introduction
• The protocol
• Client - Server protocol
• Client to Client protocol
• The application
• Queue management
• Credit system
• Download part ordering
• Corrupted download recovery

26
Selecting parts to download

• A file is divided to 9.28 MB parts
• Each part is divided to 180 KB blocks
• The downloading client determines which parts are
  downloaded
• All the blocks that are requested from the same
  source reside in the same part

27
Selecting parts, cont.

• A part is selected as follows
• Rare parts are downloaded first
• Preview parts (first and last)
• Spread requests among sources
• Complete partially received parts
• The principles impose part rating
• 0-9999 unrequested and requested very rare
  chunks
• 10000-19999 unrequested rare and preview chunks
  
• 20000-29999 unrequested most complete common
  chunk
• 30000-39999 requested rare and preview chunks
• 40000..49999 requested uncompleted common chunks
• Lowest rated part is downloaded first

28
Agenda

• Introduction
• The protocol
• Client - Server protocol
• Client to Client protocol
• The application
• Queue management
• Credit system
• Download part ordering
• Corrupted download recovery

29
Basic corruption handling

• The default corruption handling mechanism named
  ICH
• The algorithm
• When a part download completes, compute and
  compare its hash against the source
• In case of a mismatch, blocks are downloaded one
  by one until the corruption is corrected

30
Advanced corruption handling

• A new feature (since 2004) also called AICH
• Seems that it is soon to be replaced
• A tree based recovery mechanism

31
Backup slides

• Screen shots
• Client server UDP communication
• Client to Client UDP communication

32
The End

• Thank you

33
Servers screen
34
Search screen
35
Download screen
36
Client Server UDP communication

• UDP is used for
• periodic status checks
• File search enhancement
• Source search enhancement
• UDP is optional

37
Client to Client UDP communication

• UDP is used for verifying As position in Bs
  upload queue

38
AICH
The image was taken from the eMule website