Grid InterNetworking - PowerPoint PPT Presentation

1 / 65
About This Presentation
Title:

Grid InterNetworking

Description:

Problem scope (Grid introduction) Grid InterNetworking. Identifying a research gap ... C. Kesselman (eds): 'The Grid: Blueprint for a New Computing Infrastructure' ... – PowerPoint PPT presentation

Number of Views:99
Avg rating:3.0/5.0
Slides: 66
Provided by: telekoop
Category:

less

Transcript and Presenter's Notes

Title: Grid InterNetworking


1
Grid InterNetworking
Michael Welzl http//www.welzl.at DPS NSG Team
http//dps.uibk.ac.at/nsg Institute of Computer
Science University of Innsbruck
Politecnico di Torino Torino, Italy 26
November, 2007
2
Outline
  • Who am I?
  • Problem scope (Grid introduction)
  • Grid InterNetworking
  • Identifying a research gap
  • Some issues and proposed solutions
  • Conclusion
  • Practical problems
  • Future perspectives
  • Final words

3
Who am I?
  • Born in Innsbruck, Austria, 1973
  • Studied Computer Science in Linz end 1998, best
    markfor MSc. thesis on NetMusic started Ph.D.
    there
  • Passed Ph.D. defense with distinction November
    2002at TU Darmstadt (advisor Prof. Max
    Mühlhäuser)
  • Co-advisor Prof. Jon Crowcroft, Cambridge
    University
  • Thesis published as a Kluwer (now Springer) book,
    August 2003
  • Received Best Dissertation Award 2004 from
    German GI/ITG KuVS
  • Went back to Innsbruck in 2001 (new comp. science
    began)
  • Collaboration with Thomas Fahringer on Grid
    Computing
  • Started writing project proposals
  • Wrote Network Congestion Control Managing
    Internet Traffic
  • John Wiley Sons July 2005 first introductory
    book on this topic
  • Submitted this as habilitation thesis to TU
    Darmstadt in 2006,passed talk in June 2007
  • IRTF Internet Congestion Control Research Group
    (ICCRG) chair since May 2006

4
The NSG team
5
Problem scope
  • Shrinking the problem space

6
Introducing the Grid
  • History parallel processing at a growing scale
  • Parallel CPU architectures
  • Multiprocessor machines
  • Clusters
  • (Massively Distributed) computers on the
    Internet
  • GRID
  • logical consequence of HPC
  • metaphor power gridjust plug in, dont care
    where (processing) power comes from,dont care
    how it reaches you
  • Common definitionThe real and specific problem
    that underlies the Grid concept is coordinated
    resource sharing and problem solving in dynamic,
    multi institutional virtual organizationsIan
    Foster, Carl Kesselman and Steven Tuecke, The
    Anatomy of the Grid Enabling Scalable Virtual
    Organizations, International Journal on
    Supercomputer Applications, 2001

7
Scope
  • Definition quite broad (resource sharing)
  • Reasonable - e.g., computers also have harddisks
  • But also led to some confusion - e.g., new
    research areas / buzzwordsWireless Grid, Data
    Grid, Semantic / Knowledge Grid, Pervasive
    Grid,this space reserved for your favorite
    research area Grid
  • Example of confusion due to broad Grid
    interpretationOne of the first applications
    of Grid technologies will be in remote training
    and education. Imagine the productivity gains if
    we had routine access to virtual lecture rooms!
    (..) What if we were able to walk up to a local
    power wall and give a lecture fully
    electronically in a virtual environment with
    interactive Web materials to an audience gathered
    from around the country - and then simply walk
    back to the office instead of going back to a
    hotel or an airplane?I. Foster, C. Kesselman
    (eds) The Grid Blueprint for a New Computing
    Infrastructure, 2nd edition, Elsevier Inc. /
    MKP, 2004
  • ? Clear, narrower scope is advisable for
    thinking/talking about the Grid
  • Traditional goal processing power
  • Grid people parallel people thus, main goal
    has not changed much

8
The next Web?
  • Ways of looking at the Internet
  • Communication medium (email)
  • Truly large kiosk (web)
  • The Grid way of looking at the Internet
  • Infrastructure for Virtual Teams
  • Most of the time...
  • the real and specific goal is High Performance
    Computing
  • Virtual Organizations and Virtual Teams are well
    definedi.e. not an open system, e.g. security
    is a big issue
  • Virtual Teams
  • Geographically distributed
  • Organizationally distributed
  • Yet work on a common problem

But Web 2.0 is already here -)
It has been calledthe next web
9
Virtual Organizations and Virtual Teams
  • Distributed resources and people
  • Linked by networks, crossing admin domains
  • Sharing resources, common goals
  • Dynamic

10
The Grid and P2P systems
  • Look quite similar
  • Goal in both cases resource sharing
  • Major difference clearly defined VOs / VTs
  • No incentive considerations
  • Availability not such a big problem as in P2P
    case
  • It is an issue, but at larger time scales
  • (e.g. computers in student labs should be
    available after 2200,but are sometimes shut
    down by tutors)
  • Scalability not such a big issue as in P2P case
  • ...so far! ? convergence as Grids grow
  • coordinated resource sharing and problem solving
    in dynamic,multi institutional virtual
    organizations(Grid, P2P)

11
Austrian Grid E-science Grid applications
  • Medical Sciences
  • Distributed Heart Simulation
  • Virtual Lung Biopsy
  • Virtual Eye Surgery
  • Medical Multimedia Data Management and
    Distribution
  • Virtual Arterial Tree Tomography and Morphometry
  • High-Energy Physics
  • CERN experiment analyses
  • Applied Numerical Simulation
  • Distributed Scientific Computing Advanced
    Computational Methods in Life Science
  • Computational Engineering
  • High Dimensional Improper Integration Procedures
  • Astrophysical Simulations and Solar Observations
  • Astrophysical Simulations
  • Hydrodynamic Simulations
  • Federation of Distributed Archives of Solar
    Observation
  • Meteorologal Simulations
  • Environmental GRID Applications

12
Example CERN Large Hadron Collider
  • Largest machine built by humansparticle
    accelerator and collider with acircumference of
    27 kilometers
  • Will generate 10 Petabytes(107 Gigabytes) of
    information per year starting 2007 (?)
  • This information must be processed and stored
    somewhere
  • Beyond the scope of a singleinstitution to
    manage this problem
  • Projects LCG (LHC Computing Grid),EGEE
    (Enabling Grids for E-sciencE)

13
Complexity
  • Grid poses difficult problems
  • Heterogeneity and dynamicity of resources
  • Secure access to resources with different users
    in various roles,belonging to VTs which belong
    to VOs
  • Efficient assignment of data and tasks to
    machines (scheduling)

14
Grid requirements
  • Computer scientists can tackle these problems
  • Grid application users and programmers are often
    not computer scientists
  • Important goal ease of use
  • Programmer should not worry (too much) about the
    Grid
  • User should worry even less
  • Ultimate goal write and use an application as if
    using a single computer(power grid metaphor)
  • How do computer scientists simplify?
  • Abstraction.
  • We build layers.
  • In a Grid, we typically have Middleware.

15
Toolkits
  • Most famous Globus Toolkit
  • Evolution from GT2 via GT3 to GT4 influenced the
    whole Grid community
  • Reference implementation of Open Grid Forum (OGF)
    standards
  • Other well-known examples
  • Condor
  • Exists since mid-1980s
  • No Grid back then - system gradually evolved
    towards it
  • Traditional goal harvest CPU power of normal
    user workstations? many Grid issues always had
    to be addressed anyway
  • Special interfaces now enable Condor-Globus
    communication (Condor-G)
  • Unicore (used in D-Grid)
  • gLite (used in EGEE)
  • Issues that these middlewares (should) address
  • Load Balancing, error management
  • Authentification, Authorization and Accounting
    (AAA)
  • Resource discovery, naming
  • Resource access and monitoring

16
Evolution moving towards an architecture
  • OGSI / OGSA Open Grid Service Infrastructure /
    Architecture
  • Open Grid Forum (OGF) standards
  • OGSA service-oriented architecture key concept
    for virtualizationuse a resource call a
    service
  • OGSI Web Services state management
  • failed too complex, not compliant with Web
    Service standards

Source Globus presentation by Ian Foster
17
Current SoA
  • Standards are only specified when mechanisms are
    known to work
  • Globus only includes such working elements
  • Lots of important features missing
  • Practical issues with existing middlewares
  • Submitting a Globus job is very slow (Austrian
    Grid approx. 20 seconds)? significant
    granularity limit for parallelization!
  • Globus is a huge piece of software
  • Currently, some confusion about right location of
    features
  • On top of middleware? (research on top of Globus)
  • In middleware? (other Middleware projects)
  • In the OS? (XtreemOS)
  • ? Upcoming slides concern mechanisms which are
    mostly on topand partially within middleware

18
Automatic parallelization in Grids
  • Scheduling important issue for power outlet
    goal!
  • Automatic distribution of tasks and inter-task
    data transmissions scheduling
  • Grid scheduling encompasses
  • Resource Discovery
  • Authorization Filtering, Application Requirement
    Definition,Minimal Requirement Filtering
  • System Selection
  • Dynamic Information Gathering
  • System Selection
  • Job Execution
  • (optional) Advance Reservation
  • Job Submission
  • Preparation Tasks
  • Monitoring Progress
  • Job Completion
  • Clean-up Tasks
  • So far, most scheduling efforts consider
    embarassingly parallelapplications - typically
    parameter sweeps (no dependencies)

19
Grid workflow applications
  • Dependencies between applications (or large parts
    of applications) typically specified in Directed
    Acyclic Graph (DAG)
  • Condor DAG manager (DAGMan) uses .dag file for
    simple dependencies
  • Do not run job B until job A has completed
    successfully
  • DAGMan scheduling for all tasks do...
  • Find task with earliest starting time
  • Allocate it to processor with Earlierst Finish
    Time
  • Remove task from list
  • GriPhyN (Grid Physics Network) facilitates
    workflow designwith Pegasus (Planning for
    Execution in Grids) framework
  • Specification of abstract workflow identify
    application components, formulate workflow
    specifying the execution order, usinglogical
    names for components and files
  • Automatic generation of concrete workflow (map
    components to resources)
  • Concrete workflow submitted to Condor-G/DAGMan

20
Grid Workflow Applications /2
  • Components are built, Web (Grid) Services are
    defined,Activities are specified
  • Several projects (e.g. K-WF Grid) and systems
    (e.g. ASKALON) exist
  • Most applications have simple workflows
  • E.g. Montage dissects space image, distributes
    processing, merges results

21
Source http//www.dps.uibk.ac.at/projects/teuta/
22
Grid InterNetworking
23
Research gap Grid-specificnetwork enhancements
Bringing the Grid to its full potential !
Applications with specialnetwork properties
andrequirements
Driving a racing caron a public road
Traditional Internet applications(web browser,
ftp, ..)
24
Grid-network peculiarities
  • Special behavior
  • Predictable traffic pattern - this is totally new
    to the Internet!
  • Web users create traffic
  • FTP download starts ... ends
  • Streaming video either CBR or depends on
    content! (head movement, ..)
  • Could be exploited by congestion control
    mechanisms
  • Distinction Bulk data transfer (e.g. GridFTP)
    vs. control messages (e.g. SOAP)
  • File transfers are often pushed and not
    pulled
  • Distributed System which is active for a while
  • overlay based network enhancements possible
  • Multicast
  • P2P paradigm do work for others for the sake of
    enhancing the whole system (in your own
    interest) can be applied - e.g. act as a PEP,
    ...
  • sophisticated network measurements possible
  • can exploit longevity and distributed
    infrastructure
  • Special requirements
  • file transfer delay predictions
  • note useless without knowing about shared
    bottlenecks
  • QoS, but for file transfers only (advance
    reservation)

25
What is EC-GIN?
  • European project Europe-China Grid
    InterNetworking
  • STREP in IST FP6 Call 6
  • 2.2 MEuro, 11 partners (7 Europe 4 China)
  • Networkers developing mechanisms for Grids

26
Research Challenges
  • Research Challenges
  • How to model Grid traffic?
  • Much is known about web traffic (e.g.
    self-similarity) - but the Grid is different!
  • How to simulate a Grid-network?
  • Necessary for checking various environment
    conditions
  • May require traffic model (above)
  • Currently, Grid-Sim / Net-Sim are two separate
    worlds(different goals, assumptions, tools,
    people)
  • How to specify network requirements?
  • Explicit or implicit, guaranteed or elastic,
    various possible levels of granularity
  • How to align network and Grid economics?
  • Combined usage based pricing for various
    resources including the network
  • What P2P methods are suitable for the Grid?
  • What is the right means for storing short-lived
    performance data?

27
Open issue abstract-concrete WF mapping
Tasks T1, T2, T3, T4Resources R1, R2, R3,
R4Data transfers D1, D2, D3, D4
Unnoticed by scheduling algorithms!
28
Large stacks
Grid apps
Middleware
WS-RF
SOAP
HTTP
TCP
IP
29
Open issue layering inefficiency
Grid Service
Breaking the chain
Stateful
Web Service
Stateless
SOAP
Doesnt care, can do both
HTTP 1.0
Stateless
Connection state
TCP
Connection state
IP
Stateless
30
NWS The Network Weather Service
  • Most common tool for performance prediction
  • Important for making good scheduling decisions
  • Distributed system consisting of
  • Name Server (boring)
  • Sensor - actual measurement instance, regularly
    stores values in......
  • Persistent State
  • Forecaster (calculations based on data in
    Persistent State)
  • Interesting parts
  • SensorMeasured resources availableCpu,
    bandwidthTcp, connectTimeTcp, currentCpu,
    freeDisk, freeMemory, latencyTcp
  • ForecasterApply different models for prediction,
    compare with actual measurement data, choose best
    match

Duration of a long TCP transfer
RTT of a small message
31
NWS critique
  • Architecture (splitting into sensors, forecaster
    etc.) seems reasonableopen source ? consider
    integrating new work in NWS
  • Sensor
  • active measurements even though non-intrusiveness
    was an important design goal - does not passively
    monitor TCP (i.e. ignores available data)
  • strange methodology(Large message throughput)
    Empirically, we have observed that a message
    size of 64K bytes (..) yields meaningful results
  • ignores packet size ( measurement granularity )
    and path characteristics
  • trivial method - much more sophisticated methods
    available
  • point-to-point measurements distributed
    infrastructure not taken into account
  • Forecaster
  • relies on these weird measurements, where we
    dont know much about the distribution (but we do
    know some things about net traffic IFF properly
    measured)
  • uses quite trivial models (but they may in fact
    suffice...)

ssthresh
32
NWS measurements (Austrian Grid)Muhammad Murtaza
Yousaf, Michael Welzl, Malik Muhammad Junaid
"Fog in the Network Weather Service A case for
novel Approaches", MetroGrid workshop, co-located
with GridNets 2007, Lyon, France, 19 October 2007.
  • Salzburg-Linz (left) more than 20 MB needed to
    saturate link
  • Within Innsbruck (right), Gigabit link around
    100 MB needed
  • NWS supposedly designed to be non-intrusive...

33
The impact of shared bottlenecks
  • Example problem
  • C allocates tasks to A and B (CPU, memory
    available) both send results to C
  • B hinders A - task of B should have been kept at
    C!
  • Path changes are rare - thus, possible to detect
    potential problem in advance
  • generate test messages from A, B to C - identify
    signature from B in As traffic
  • Another issue in this scenario how valid is a
    prediction that A obtains if a measurement /
    prediction system does not know about the shared
    bottleneck?

34
EC-GIN Large File Transfer Scenario (LFTS)
Multipath file transfer (A?B A?C?B) beneficial
Multipath file transfer not beneficial due to
shared bottleneck
Questions when does this make sense, how to
expose this functionality, how to authenticate
and authorize?
35
Shared bottleneck detection with SVDMuhammad
Murtaza Yousaf, Michael Welzl, Bulent Yener
(2007) under submission
  • Input end-to-end forward delays of multiple
    flows
  • Analysis
  • Multivariate Analysis Method SVD (Singular Value
    Decomposition)
  • Matrix operation which yields clustered values
    for correlating flows
  • Calculate differences between values, consider
    changes between clusters as outliers, apply
    simple outlier detection method
  • Output clusters of flows which share a
    bottleneck
  • Very precise, easy to calculate, can cluster
    multiple flows at the same time (other work uses
    pairwise cross correlation)

36
Extending the Padhye equation to N flowsDragana
Damjanovic, Werner Heiss, Michael Welzl "An
Extension of the TCP Steady-State Throughput
Equation for Parallel TCP Flows", poster, ACM
SIGCOMM 2007, 27-31 August, Kyoto, Japan.
  • Fair amount of work done, but so far, no (easily
    usable) approximation exists which also takes
    loss into account
  • Useful in a Grid for multiple reasons
  • Prediction of GridFTP throughput (multiple TCP
    flows)
  • Protocol with tunable aggression (MulTFRC)
  • Because, if a Grid application uses two flows
    which share a bottleneck, flow 1 may be 3.7 times
    as important to it as flow 2 (e.g. if flow 2 is
    from replication)
  • For fairness if we take a break, we may earn
    aggression points

37
Other current EC-GIN work
  • INRIA UIBK working on scheduling of advance
    reservations for bulk data transfers (using
    high-speed congestion control mechanisms)
  • WP2 dedicated to modeling (and ns-2 simulation
    code)
  • Lead ULANC currently collecting measurements
    from everywhere...
  • Traffic model from INRIA
  • UIBK developed a Grid-Net scheduling simulator
    using ns-2
  • ISCAS developed a high-speed SOAP engine
  • UniZH working on P2P incentive mechanisms for the
    Grid security
  • ...stay tuned for more!

38
Conclusion
  • Practical problems
  • Future perspectives
  • Final words

39
Problem How Grid people see the Internet
Just like Web Service community
  • Abstraction - simply use what is available
  • still performance main goal
  • Existing transport system(TCP/IP Routing ..)
    works well
  • QoS makes things better, the Grid needs it!
  • we now have a chance for that, thanks to IPv6

Absolutely not like Web Service community !
Wrong.
  • Quote from a paper review
  • In fact, any solution that requires changing the
    TCP/IP protocol stack is practically unapplicable
    to real-world scenarios, (..).
  • How to change this view
  • Create awareness - e.g. GGF GHPN-RG published
    documents such asnet issues with grids,
    overview of transport protocols
  • Develop solutions and publish them! (EC-GIN,
    GridNets)

40
A time-to-market issue
Typical Grid project
Result thesis running codetests in
collaboration withdifferent research areas
Typical Network project
Result thesis simulationcode perhaps early
real-lifeprototype (if students did well)
41
Machine-only communication
  • Trend in networks from support of Human-Human
    Communication
  • email, chat
  • via Human-Machine Communication
  • web surfing, file downloads (P2P systems),
    streaming media
  • to Machine-machine Communication
  • Growing number of commercial web service based
    applications
  • New hype technologies Sensor nets, Autonomic
    Computing vision
  • Semantic Web (Services) first big step for
    supporting machine-only communication at a high
    level
  • So far, no steps at a lower level
  • This would be like RTP, RTCP, SIP, DCCP, ... for
    multimedia appsnot absolutely necessary, but
    advantageous

42
The long-term value of Grid-net research
  • Key for achieving this change viewpoint
    fromwhat can we do for the Grid to what can
    the Grid do for us(or from what does the Grid
    need to what does the Grid mean to us)
  • A subset of Grid-net developments willbe useful
    for other machine-onlycommunication systems!

43
Conclusion
  • Grid applications show special requirements and
    properties from a network perspective
  • and it is reasonable to develop tailored Internet
    technology for them.
  • There is another class of such applications...
  • Multimedia.
  • For multimedia applications, an immense number of
    network enhancements (even IETF standards) exist.
  • For the Grid, there is nothing.
  • This is a research gap lets fill it together!
  • submit a paper to GridNets 2008 in Beijing! -)

Reminder if done right, such research is also
applicable to other systems with machine-only
traffic
44
More informationhttp//www.ec-gin.eu
Thank you! Questions?
45
Backup slides
46
Exploiting longevity
  • Time scale of traffic fluctuations lt time scale
    of path changes? knowledge of link capacities
    may be more useful than traffic estimate
  • Underlying technique packet pair
  • send two packets p1 and p2 in a row high
    probability that p2 is enqueued exactly behind p1
    at bottleneck
  • at receiver calculate bottleneck bandwidth via
    time between p1 and p2
  • minimize error via multiple probes
  • TCP with Delayed ACK receiver automatically
    sends packet pairs? passive TCP receiver
    monitoring is quite good!

47
Grid-Network Simulation
48
Procedure
  • Grid simulator only simulates one execution on
    one machine
  • File transfers generate scenario invoke
    network simulator
  • Possibility data transfers influencing each
    other
  • No problem
  • Stop ns-2
  • Add or terminate flows
  • Continue

49
Example scenario
Tasks T1, T2, T3, T4Resources R1, R2, R3,
R4Data transfers D1, D2, D3, D4
50
Conclusion
  • Implementation finished!
  • Very soon available from EC-GIN website
  • Input concrete workflow (XML), output XML GUI
    gnuplot exporting...
  • Method tackles an important and current problem,
    but...
  • Time consuming
  • Repeated simulation of data transfers
  • Total parameters ( Grid parameters) (
    network parameters)
  • On the other hand...
  • User Research group which also runs a Grid
  • Easy to distribute (parameter study) ? Grid
    simulation on the Grid
  • Seems strange (recursion), but makes sense one
    Grid application for carrying out an analysis
    with numerous environment conditions

51
QoS for the Grid
52
QoS the state-of-the-art -(
  • Papers from SIGCOMM03 RIPQOS Workshop Why do
    we care, what have we learned?
  • QoSs Downfall At the bottom, or not at all! Jon
    Crowcroft, Steven Hand, Richard Mortier,Timothy
    Roscoe, Andrew Warfield
  • Failure to Thrive QoS and the Culture of
    Operational Networking Gregory Bell
  • Beyond Technology The Missing Pieces for QoS
    Success Carlos Macian, Lars Burgstahler, Wolfgang
    Payer, Sascha Junghans, Christian Hauser, Juergen
    Jaehnert
  • Deployment Experience with Differentiated
    Services Bruce Davie
  • Quality of Service and Denial of Service
    Stanislav Shalunov, Benjamin Teitelbaum
  • Networked games --- a QoS-sensitive application
    for QoS-insensitive users? Tristan Henderson,
    Saleem Bhatti
  • What QoS Research Hasnt Understood About Risk
    Ben Teitelbaum, Stanislav Shalunov
  • Internet Service Differentiation using Transport
    Optionsthe case for policy-aware congestion
    control Panos Gevros

53
Key reasons for QoS failure
  • Required participation of end users and all
    intermediate ISPs
  • normal Internet users want Internet-wide QoS,
    or no QoS at all
  • In a Grid, a virtual team wants QoS between its
    nodes
  • Members of the team share the same ISPs - flow of
    is possible
  • Technical inability to provision individual
    (per-flow) QoS
  • normal Internet users
  • unlimited number of flows come and go at any time
  • heterogeneous traffic mix
  • Grid users
  • number of members in a virtual team may be
    limited
  • clear distinction between bulk data transfer and
    SOAP messages
  • appearance of flows mostly controlled by
    machines, not humans
  • ? QoS can work for the Grid !

54
Proposed architecture
  • Goal efficient per-flow QoS without signaling to
    routers
  • Idea use traditional coarse-grain QoS (DiffServ)
    to differentiate between
  • long-lived bulk data transfer with advance
    reservation (EF) and
  • everything else ( SOAP etc. over TCP) (best
    effort)
  • Allows us to assume isolated traffic planned to
    drop this requirement later
  • Because data transfers are long lived, apply
    admission control
  • Flows signal to resource broker (RB) when joining
    or leaving the network
  • Mandate usage of one particular congestion
    control mechanism for all flows in the EF
    aggregate
  • Enables efficient resource usage because flows
    are elastic

55
Key ingredients of our QoS soup
  • Link capacities must be known, paths should be
    stable(capacity information should be updated
    upon routing change)
  • Shared bottlenecks must be known
  • Bottlenecks must be fairly shared by congestion
    control mechanism irrespective of RTT (max-main
    fairness required, i.e. all flows must increase
    their rates until they reach their limit)
  • No signaling to routers no way to enforce
    proper behavior? there must be no cheaters
  • User incentive fair behavior among cooperating
    nodes among which Grid application is distributed
  • Unfair behavior between Grid application 1 and 2
    in same Grid neglected(usually acceptable, as
    used by same Virtual Organization)

56
Link capacities must be known
  • Can be attained with measurements
  • Working on permanently active, (mostly) passive
    measurement system for the Grid that detects
    capacity with packet pair
  • send two packets p1 and p2 in a row high
    probability that p2 is enqueued exactly behind p1
    at bottleneck
  • at receiver calculate bottleneck bandwidth via
    time between p1 and p2
  • e.g. TCP Delayed ACKreceiver automatically
    sendspacket pairs? passive TCP
    receivermonitoring is quite good!
  • exploit longevity - minimizeerror by listening
    for along time

57
Shared bottlenecks must be known
  • Simple basis distributed traceroute tool
  • enhancement traceroute terminates early upon
    detection of known hop
  • Handle black holes in traceroute
  • generate test messages from A, B to C - identify
    signature from B in As traffic
  • method has worked in the past controlled
    flooding for DDoS detection

58
Congestion Control mechanism must be max-min fair
  • Was once said to be impossible without per-flow
    state in routers
  • not true XCP and some others
  • but these explicit require router support...
  • Main problem dependence on RTT
  • three good indications that this can be removed
    without router support
  • CADPC/PTP (my Ph.D. thesis)...
  • max-min fairness based on router feedback, but
    only capacity and available bandwidth (could also
    be obtain by measuring)
  • Personal comment by Sally Floyd
  • Reference to old paper on phase effects
  • TCP Libra
  • Problem efficiency - no max-min fair
    high-speed CC mechanism without router support
  • searched for a long time
  • now plan to change existing one based on
    knowledge from above examples

59
Per-flow QoS without signaling to routers
Traditional method signaling to edge routers
(e.g. with COPS) at this point!
Synchronization ofdistributed (P2P
based)database link capacitiesknown to all
brokers
Synchronization ofdistributed (P2P
based)database all flows knownto all brokers
Synchronization ofdistributed (P2P
based)database all flows knownto all brokers
continuous measurementsupdate to BB upon path
change
60
Efficiency via elasticity
  • QoS guarantees in Grid File will be transferred
    within X seconds? enables flexible resource
    usage

61
Efficiency via elasticity /2
  • Flow 1 stopped, flows 2-4 automatically increase
    their rates
  • leading to earlier termination times E2-E4
    known to (calculated by) BB

62
Efficiency via elasticity /3
  • Flow 5 asks BB for admission
  • BB knows about current rates and promised E2-E4,
    grants access

63
Efficiency via elasticity /4
Additional flow admitted and earlier termination
times than promised!
  • Flow 2 terminates in time
  • Flows 3-5 will also terminate in time

64
Elasticity without Congestion Control?
  • Significant amount of additional signaling
    necessary

As flow 5 is admitted, signal reduce your rates
toflows 2-4 required!
As Flow-1 stops, Flows 2-4 could increase their
rates
Without congestion control, signal increase your
rates to flows 2-4 required!
65
Additional considerations
  • How to assign different rates to different flows?
  • max-min fairness if a sender acts like two, it
    obtains twice the rate
  • consider rate consisting of slots (e.g. 1 kbit/s
    1 slot)
  • flows can consist of several slots
  • let congestion control mechanism operate on slots
  • Possibility admit new flows even in scenario
    below

Must introduce unfairness only flow 2 can reduce
rate
Disadvantage more signaling again!
66
Difficult distant future work
  • Drop requirement of traffic isolation via
    DiffServ
  • constantly obtain and update conservative
    estimate of available bandwidth using packet pair
    (works without saturating link)
  • ensure that limit is never exceeded condition
    red otherwise!
  • Some open questions...
  • does this require the CC mechanism to be
    TCP-friendly?
  • condition red reduce slots, or let flows be
    aggressive for a short time?
  • How to handle routing changes
  • will be noticed, but can reduce capacity ? break
    QoS guarantee
  • condition red can happen in worst case, but to
    be avoided at all cost
  • mitigation methods
  • very conservative estimate of available
    bandwidth leave headroom
  • tell senders to reroute via intermediate end
    systems
  • Bottom line lots of complicated issues, but
    possible to solve them
Write a Comment
User Comments (0)
About PowerShow.com