Websight Visualizing the Execution of Web Services

1
WebsightVisualizing the Execution of Web Services

• Wim De Pauw, Bob Hoch, Sophia Krasikov, Michelle
  Lei, John Morar, Raju Pavuluri, Amit Phalgune, Ed
  Pring, Senthil Velayudham
• Contact wim_at_us.ibm.com

2
Outline

• Motivation
• Debugging Web Services
• Profiling and understanding Web Services
• Dealing with complexity
• Technology
• Conclusions

Agenda slide
3
What are Web Services?

• "A WSDL described interface that defines a
  collection of network accessible operations"
• Modular
• Composed
• Described
• Published
• Found
• Bound
• Invoked

4
Evolution of Web Services

• Internal Enterprise Application Integration
• External Integration
• Dynamic adaptation in e-business
• gt Lingua Franca for (loosely coupled)
  distributed systems

5
Web Services the Challenge

• Do you know how your Web Services are interacting
  ?
• Ever tried to read their trace logs?
• Ever tried to collate ten trace logs?
• Where are the bottlenecks?
• Is your workflow correct?

6
Websight Problem Determination for Web Services

• Debugging and understanding
• Analysis at application level
• Understand message flow of Web Services in
  complex applications
• Total processing time vs. business logic time
• Experimental research tool

7
Sample Target eServicePac
BPEL Engine
Compatibility
BP Model for SPV
Exclusivity
Partner World
Check warranty period
SAP-F
Check Model Serial number
Java to Web Services
Web Services wrapper
Validation Functions
8
Lifecycle
Operation Maintenance Problem Determination
Implement Deploy
Design
Business requirements
9
Websight demo
10
Debugging Web Services

• Syntax errors
• Semantic errors
• Resource unavailable errors
• Business logic errors

11
Syntax Errors

• Often relate to representing unusual values
• Example ambiguous type conversions XML Java
• In subsequent versions of a SOAP engine
• java.util.GregorianCalendar ?? xsddate
• java.util.GregorianCalendar ?? xsddateTime
• Example encoding of null String vs. empty String
• Example problem with out of range values

12
Find Syntax Errors by Inspecting the Content
13
Semantic Errors Discordant Meanings

• Even if the syntax is correct, service providers
  and consumers may associate different meanings to
  certain expressions
• Examples
• Empty field means wildcard or missing
  information?
• Timestamp (omitting or ignoring time zones)
• 2004-07-11T104924.938000-0400

14
Semantic Errors Unexpected Modes of Execution

• Example
• Client node sends (unexpectedly!) two identical
  messages for every transaction.
• Functional Testing did not reveal this problem

15
Semantic Error Unexpected Modes of Execution

• Unexpected Flow Order
• Architect did not expect stackingChecker before
  compatibilityChecker because workflow engine
  was in parallel instead of serial mode.

16
Resource Unavailable Errors

• Find out if/where a node is down
• Effect / cause may be in different places
• Examples
• Database server down, workflow engine continues
• Server throws an HTTP error, fed into SOAP parser
• We use reactive approach
• Proactive approach synthetic heartbeat

17
Business logic error

• Example
• For this operation we didnt need to call
  StackingChecker service
• Unit testing did not expose this problem since
  the existing test data generated the correct
  result

18
Profiling is Understanding

• Where is time spent?
• Network vs. business logic?
• Chatty communication?

19
Profiling Sorting Statistics

• Table view helps find outliers by sorting Network
  Duration in table view for several transactions
• Selecting the outliers in table highlights
  corresponding messages in other views

20
Understanding Web Services Configurations

• Example Two concurrent Transactions using two
  workflow engines

21
Understanding Web Services Configurations

• Supply Chain Management from WS-I

• From Design Document SCM

• Traced Topology SCM

22
Dealing with complexity

• High traffic, high number of nodes, complex
  interactions
• Pattern extraction
• Selective tracing

23
Fifty Concurrent Transactions
Dealing with complexity Pattern Extraction
24
Selective Tracing for Heavy Loads

• Tracing everything during heavy load is
  undesirable too much perturbation, too much data
• Start with marked transaction (carrying
  correlator)
• Tracing will be turned on when this transaction
  is handled
• Reduces amount of data to be traced
• Easier to understand 1 typical transaction than
  10000

25
Technology based on Eclipse and Hyades

• Hyades
• Integrated test, trace and monitoring environment
• Open source basic infrastructure
• Tool interoperability across testing/profiling
  process
• www.eclipse.org/hyades
• Components
• Information model (EMF)
• Communication framework (Remote Agent Controller)
• Data collection

26
Typical architecture for existing
testing/profiling tools
27
Hyades provides a common infrastructure
28
Deployment

• Collect runtime
• information

2) Present to user
29
Websight Runtime (experimental setup)
Axis Parser
Websphere Parser
Non-SOAP protocols
Apache SOAP Parser
Websight Interface
Websight Runtime
Websphere Runtime
Correlation
Logging
Remote Agent Controller
Visualizer
Websight Trace File
30
Correlating Distributed Events

• Problem correlating messages
• Goal trace end-to-end message flow across
  system

?
31
Correlating Distributed Events

• Correlator technology
• ARM correlator technology
• Web Services correlator
• Websight SOAP header carries correlator
• Propagation through business logic handled by
  middleware

Receive Correlator
Receive Correlator
Attach Correlator
Attach Correlator
32
Summary

• Debugging Web Services requires new methodologies
  and tools
• Complexity at the application level
• Opportunities for work flow

33
Contactwim_at_us.ibm.com

Closing slide
34
Backup foils
35
New Order Transaction
36
Existing Order Transaction
37
Two Concurrent Transactions using Two Workflow
Engines
38
Navigation between Views
Selecting an item in one view
highlights corresponding items in the other views
39
Supply Chain Management Maximal
Transaction(understanding the complex flow)