An Evaluation of Contemporary Commercial SOAP Implementations

1
An Evaluation of Contemporary Commercial SOAP
Implementations
Alex Ng Department of Computing, Macquarie
University
Shiping Chen CSIRO ICT Centre
Paul Greenfield CSIRO ICT Centre
AWSA 2004, Melbourne, 13-14 April 2004

• Presented by Alex Ng

2
Outline

• Background
• Experimental design
• Result discussion
• Related work
• Future work

3
Web Services Definition (W3C)

• A Web service is a software system designed to
  support interoperable machine-to-machine
  interaction over a network. It has an interface
  described in a machine-processable format
  (specifically WSDL). Other systems interact with
  the Web service in a manner prescribed by its
  description using SOAP-messages, typically
  conveyed using HTTP with an XML serialisation in
  conjunction with other Web-related standards.

W3C Web Services Architecture Working Draft (8
August 2003)
4
W3C Web Services Conceptual Stack
Source Ferguson D. Secure, Reliable, Transacted
Web Services Architecture and Composition. MSDN
technical article, Sept. 2003.
5
SOAP (Simple Object Access Protocol)

• Simple
• Extensible
• Based on XML
• De facto standard for XML Messaging in supporting
  Web Services
• V1.1(May/00)
• V1.2 (June/03)

Request POST /StockQuote HTTP/1.1 Host
www.stockquoteserver.com Content-Type text/xml
Content-Length nnnn SOAPAction
Some-URI ltSOAPEnvelope xmlnsSOAP"urnschemas.
xmlsoap.orgsoap.v1"gt ltSOAPHeadergt lttTransactio
n xmlnstURI mustUnderstand1gt5lt/tTransacti
ongt lt/SOAPHeadergt ltSOAPBodygt ltmGetLastTradePri
ce xmlnsm"URI"gt ltsymbolgtDISlt/symbolgt lt/mGetL
astTradePricegt lt/SOAPBodygt lt/SOAPEnvelopegt
6
Other Competitors of SOAP Web Services

• Use different transport mechanism
• Microsoft DIME (Direct Internet Message
  Encapsulation)
• Work with or without SOAP HTTP
• send attachments of various types
• BEEP (Blocks Extensible Exchange Protocol)
• Connection oriented, min 257 channels,
  asynchronous message exchange
• Work with or without HTTP
• Use different Web Architecture
• REST (Representational State Transfer)
• An architectural style (Roy Fielding)
• Use XML/HTML, no need for SOAP

7
Performance Issues of SOAP Web Services

• Serialisation and de-serialisation
• Message size ( 6-8 times larger than binary)
• Introduce latency in transmission, storing or
  retrieving XML data
• XML Processing
• Parsing (encode/decode XML elements)
• Validation (conformation to a pre-defined schema)
• Transformation (XML lt-gt other format)

8
Network settings Issues

• HTTP 1.1 chunking (avoids content-length
  calculation)
• HTTP 1.1 Keep-Alive (reduces connection set up
  cost)
• HTTP 1.1 Pipelining (handles multiple
  requests/connection)
• TCP-Delayed-ACK (defers receiver acknowledgement)
• Nagle Algorithm (defers the sending of small
  packets)

9
Evaluation of contemporary commercial SOAP
implementations

• (1) What level of performance is shown by major
  commercial SOAP Web Services implementations?
• (2) How does the performance of SOAP compare to
  that offered by conventional non-SOAP
  technologies?

SOAP Body
Document
RPC
Document/Literal RPC/Literal
Document/Encoded RPC/Encoded
Literal
SOAP Parameter
Encoded
10
Experimental Design
Property.xml
4x1.6Ghz, 3.6Gb
Test Driver
Result Log
Platform Specific Runner
100Mbps Ethernet
Target Application Server
Target Web Service
Test Document
4x1.6Ghz, 3.6Gb
11
Result Message Size
No.of Char.
12
Result Latency
13
Result Simple Message Throughput
14
Result Medium Message Throughput
15
Result Complex Message Throughput
16
Result Serialisation/De-serialisation
13
23
10
30
17
Evaluation Summary

• SOAP Performance is affected by
• Encoding style (Document/Literal encoding is the
  choice)
• Product of implementation (a sizable gap between
  two products using the same document/literal
  encoding)
• Performance delivered by the majority SOAP
  implementations
• Simple Message able to deliver reasonably good
  performance (4 to 6ms vs 1ms non-SOAP)
• Medium Message (10 to 22ms vs 7ms non-SOAP)
• Complex Message (20 to 63ms vs 15ms non-SOAP)
• De-serialisation overhead is higher than that of
  serialisation.

18
Related Work (1)

• The Extreme Lab, Indiana University
• Govindaraju, et. al.
• sending different size of linked list of doubles
  using different implementations SOAPRMI, SUNRMI,
  and NEXUSRMI, and TCP.
• TCP is two orders of magnitude better than
  SOAPRMI
• Serialisation and de-serialisation are the
  bottlenecks.
• Chiu, et.al.
• sending different arrays of mathematical doubles.
  
• Suggested Optimization Pull Parser,
  schema-specific XML parser, trie and HTTP 1.1.
• Kohlhoff and Steele Compares SOAP against FIX
  and CDR
• SOAP latency is 2 to 3 times worse than CDR
• FIX is a text based encoding and outperforms CDR
  and CDR

19
Related Work (2)

• Dhawan Compares the performance of .NET Remoting
  against ASP.NET
• ASP.NET XML serialisation is more efficient than
  .NET Remoting SOAP serialisation
• Davis and Parashar
• Different SOAP Implementations SOAPRMI/Java1.1,
  Microsoft SOAP Toolkit SP2 in VB, Perl
  SOAPLite, Apache SOAP 2.2, and Apache Axis
  Alpha 3.
• inappropriate TCP options had caused 200ms delays
  in some implementations (MS SOAP, Apache SOAP and
  SOAPLite )
• Conclusion SOAP is in the orders of magnitude
  slower than JavaRMI and CORBA

20
Related Work (3)

• Benkner, et. al. Compares the performance of
  JAXM implementations against JAXRPC in different
  package sizes (1x1024 -gt 8x128)
• Different Implementations Apache Axis 1.1 RC2,
  Sun JWSDP 1.1, Mind Electric GLUE4.0, Systinet
  WASP 4.5, Novell jBroker Web 2.0, and X-Treme
  XSOAP 1.2.20.
• Conclusion
• JAXM JAXRPC produce similar results
• Best performance result is achieved when using
  large packages of data, i.e. splitting 1MB of
  source data into four packages of 256KB each for
  JAXM in Systinet WASP implementation and 2x 512
  for JAXRPC in Apache implementation.

21
Future Work

• Extend Evaluation to other SOAP Implementations,
  such as Apache Axis
• Identify other factors when SOAP over HTTP is
  used across wide-area networks

22
Your Feedback
23
Reference

• Chiu K., et. al. (2002) Investigating the Limits
  of SOAP Performance for Scientific Computing In
  Proceedings of 11th. IEEE International Symposium
  on High Performance Distributed Computing HPDC-11
  2002 (HPDC'02)
• Benkner, S., Brandic, I., Dimitrov, A., et al.
  Performance of Java Web Services Implementations.
  In Proceedings of ICWS'03. Las Vegas, 2003
• Davis, D. and Parashar, M. Latency Performance of
  SOAP Implementations. In Proceedings of IEEE
  Cluster Computing and the GRID 2002 (CCGRID'02)
• Dhawan, P. Performance Comparison Exposing
  Existing Code as a Web Service, October 2001
  (on-line) Accessed Date 10 October 2002
• http//msdn.microsoft.com/library/default.asp?url
  /library/en-us/dnbda/html/bdadotnetarch11.asp
• Kohlhoff, C. and Steele, R. (2003) Evaluating
  SOAP for High Performance Business Applications
  RealTime Trading Systems. In Proceedings of
  WWW2003. Budapest, Hungary.
• Govindaraju, M., A. Slominski, et al.
  Requirements for and Evaluation of RMI Protocols
  for Scientific Computing. Supercomputing. IEEE,
  2000