Distributed Systems and Technology Overview

1
Distributed Systems and Technology Overview

• Ian Gorton, Paul Greenfield
• Software Arhitectures and Component Technologies,
• CSIRO, Sydney

2
Who is CMIS?

• CSIRO is Australias largest scientific RD
  organisation and has around 7000 staff
• CSIRO Mathematical and Information Sciences is
• Australias premier group of mathematicians and
  Information Technology specialists working to
  improve performance in Australian industry.

3
SACT

• 13 person RD group, Sydney and Canberra
• Involved in software engineering RD
• Specifically looking at the problems of
  distributed systems, eg
• architecture, components, technologies
• testing and analysis
• performance, scalability, reliability

4
Architecture Issues
Physical constraints
Security
Transactions
Management
Solution Space
Technologies, Components
Fault Tolerance
Performance Requirements/Scalability
Testing
Trade-offs between competing forces can be
extremely complex
5
SACT capabilities

• Designing software architectures
• Expertise in enterprise distributed technologies
  (eg COM, ORBs, Java), transactions, databases
• Research based around software architectures
• In-depth knowledge of Internet and e-commerce
  issues

6
Course Aims

• Improve OO design and programming skills
• doing lots of it
• more advanced use of language constructs
• thinking architecturally
• design patterns
• exposure to expert code in Orbacus (open source
  product)

7
Course Aims

• Introduce distributed object technology
• teach CORBA programming
• teach distributed object system design
• understand and use CORBA services, including
  Naming and Transactions
• understand the fundamentally difficult issues in
  disttributed systems

8
Assessments

• 2 major assessments worth 20 each
• involve programming, experimentation
• Orbacus is freely available, so work can be done
  off-campus
• work in pairs (unless youve got a bullet-proof
  excuse!)
• End of session exam
• more on this nearer time

9
Week 2 Distributed Systems And Technology
Overview

• Acronym frenzy!!

10
What is Middleware?
Software plumbing and associated services for
building distributed systems
11
Basic distributed systems blocks

• Provides high(ish) level facilities (RPC, RMI)
  for building distributed system communication
  infrastructures
• Hides low level issues from the programmer
  through abstraction

The Next Big Thing??
Component Technologies
CORBA/RMI/COM
TCP/IP
Network/OS
12
Associated Services
Security
Transactions
Directory
Messaging

• Basic middleware add-ons
• Fulfill common application needs in distributed
  systems
• See CORBA Services spec for an example and
  comprehensive list

Middleware Plumbing
13
Middleware Technologies

• CORBA
• COM
• Java
• DCE
• Proprietary, ie
• Forte, Tuxedo, CICS, etc..
• TIBCO, MQ, etc..

14
So What is it Good For?

• Geek answer
• Middleware makes it feasible for mere mortals
  to build distributed software systems that are
• high-performance, scalable
• reliable, high availability
• Dont be fooled. Distributed systems are
  inherently difficult to build.

15
Er..try me again?

• The bottom line answer...
• Businesses need a WWW presence
• secure, transactional accesses to business
  systems
• Integration
• mergers, acquisitions, virtual enterprises
• Internet competition and time-to-market

16
Some Gartner Survey Figures

• To develop a currently competetive e-commerce
  site
• 5 months average duration (some 1 year)
• uses more than 1 consulting organization
• cost average US1 million (350K-2m)
• 79 costs were labor, 10 s/w
• No enterprise WWW site was on budget

17
Gartner Group Predictions

• Simple site - 300k-1 million
• Competetive site - 1-5 million
• Market leader - 5-20 million
• Site costs will increase 25 annually in next few
  years

18
Some Sample Applications

• Australian Stock Exchange
• NSW Roads and Traffic Authority
• many more...

19
Solving Business Problems with Middleware

• Legacy system access
• Data integrity
• Scalability
• Availability
• Client access

20
Legacy System Access
CICS\VSAM
AS400

• Ease legacy system integration by providing
  standard interfaces (wrappers)
• Hides details of legacy systems from clients

MQ
CICS Gateway
Server
Server
Middleware Wrappers
Clients
Clients
21
Data Integrity
SQL Server
Oracle

• Ensuring data integrity with distributed
  databases requires transactions
• Transactions ensure ACID properties

TP Service
SQL
ODBC
Server
Server
Clients
Clients
22
Scalability

• Middleware supports scaling system performance by
  service replication
• This isnt free - it requires good architecture!

Server
Server
Clients
Clients
23
Availability

• Replication enhances fault tolerance and improves
  availability
• Graceful degradation, lower performance but it
  keeps running

Server
Server
Clients
Clients
24
Client Access

• Client access enhanced due to
• physical distribution
• variety of access technologies (ie Internet
  protocols, IIOP, COM) and available bridges.

Clients
HTTP
WWW Server
Server
IIOP
COM Bridge
IIOP
Clients
Clients
25
Enterprise Middleware Products

• Some terminology first
• Middleware - basic products
• Orbix, Visibroker, DCE, Java RMI, COM
• Object Transaction Monitor (OTM)
• middleware plus enterprise systems capabilities,
  OrbixOTM, BEA WebLogic
• Application Server
• cynical answer - new name for OTMs with Java 2
  Enterprise Edition support -

26
N-Tier System Architecture
Client layer (browser, applets, apps)
http, IIOP, COM, RMI, XML
WWW server (WWW server, JSPs, ASPs)
Security
IIOP, COM, RMI, XML
Transactions
Business Logic (CORBA objects, EJBs, COM)
SQL, ODBC, JDBC, XA
Data Access (DBMSs)
27
Standards (wot standards?)

• CORBA - OMG
• Java 2 Enterprise Edition (EJBJMSJNDI) - Sun
• COM - Microsoft
• MQ Series - IBM
• TIBCO multicast protocol..
• ???

28
CORBA

• OMG is vendor consortium
• CORBA Interface Definition Language (IDL)
• Internet Inter-ORB Protocol (IIOP) (GIOP over
  TCP/IP)
• CORBA services, eg transactions, naming, events,
  security

29
Java

• Remote Method Invocation (RMI)
• Suns JRMP or RMI-over-IIOP
• Enterprise Java Beans (EJBs)
• Java Transaction Service (JTS)
• Java mapping to CORBA OTS
• Java Naming and Directory Interface (JNDI)
• Java Messaging Service (JMS)

30
EJBs

• EJBs are server-side components
• Separate business logic from infrastructure code

EJB
EJB
EJB
EJB Container
31
EJBs

• EJBs capture design patterns
• Stateless Session Bean
• Stateful Session Bean
• Entity Bean (bean or container managed)
• Containers/EJB Servers provide
• transaction policies
• security policies
• database connection pooling
• multi-threading

32
Transaction Service

• Essential for distributed transaction processing
• Manages consistent updates to multiple databases
• CORBA OTS, Java JTS, Msoft MTS
• X/Open DTP standards - XA

33
ACID properties

• A set of operations that have ACID properties
• Atomic
• Consistent
• Isolated
• Durable
• Classic example - a bank account transfer

34
Java Transaction Example

• try
• // Create a transaction.
• transaction.begin()
• // Invoke operations in transaction
• savingsAccount.makeWithdrawal(50)
• chequeAccount.makeDeposit(50)
• // Commit the transaction.
• transaction.commit(true)
• catch (APP_ERROR ex)
• transaction.rollback()
• catch (TRANSACTION_ROLLEDBACK ex)
• // handle rollback

35
Two-Phase Commit
36
Directory Service

• Clients need to get references to server objects
• Server objects advertise their reference in a
  directory service
• Clients query the directory service to retrieve
  desired reference
• CORBA Naming service, JNDI, LDAP

37
Example Name Space
38
Security

• Provide
• user authentication
• user/service authorization
• encryption
• Secure Socket Layer (SSL)
• public key infrastructure
• asymmetric private and public keys
• X.509 digital certificates

39
Messaging

• Many apps require asynchronous communications
• CORBA event/notifcation/messaging service
• Java Messaging Service
• TIBCO/Rendezvous
• MQ Series
• others
• More on these later...

40
WWW Server Tier

• Need to accept HTML requests via HTTP post or get
• CGI scripts first used to add processing
  capability
• spawn new process for each request
• slow, doesnt scale
• Proprietary improvents
• Netscapes NSAPI, Microsofts ISAPI

41
Java Servlets/Server Pages

• Java platforms includes specs for
• servlets
• Java Server Pages (JSPs)
• Java servlets launched by WWW Server in response
  to URL request
• JSPs allow code (eg Java beans) for formatting
  dynamic content to be embedded in HTML page

42
Java Servlets/Server Pages
invoke
query
URL
JSP
App. Server Component
results
html
WWW Server
invoke
query
Servlet
HTML
results
html
43
Extensible Markup Language (XML)

• Subset of SGML
• Standard textual format for structured documents
• Document Type Definition (DTD) defines document
  structure with tags for each entry in the
  document
• XML document instance contains both tags and
  document data
• Text format is easy to process

44
Sample DTD

• lt?xml encoding"US-ASCII"?gt
• lt!ELEMENT orders (order)gt
• lt!ELEMENT order (header,item,total)gt
• lt!ELEMENT header (billing_info,shipping_info)gt
• lt!ELEMENT billing_info (name,address,credit_card)gt
  
• lt!ELEMENT shipping_info (name,address)gt
• lt!ELEMENT name (given,family)gt
• lt!ELEMENT address (street,city,state,zipcode,count
  ry,phone)gt
• lt!ELEMENT item (product_id,product_name,quantity,p
  rice)gt
• lt!ELEMENT credit_card (PCDATA)gt
• lt!ELEMENT given (PCDATA)gt
• lt!ELEMENT family (PCDATA)gt
• ELEMENTS MISSING..
• lt!ELEMENT product_id (PCDATA)gt
• lt!ELEMENT product_name (PCDATA)gt
• lt!ELEMENT quantity (PCDATA)gt
• lt!ELEMENT price (PCDATA)gt
• lt!ELEMENT total (PCDATA)gt

45
Sample (Incomplete) XML Document

• lt?xml version"1.0"?gt
• lt!DOCTYPE orders SYSTEM "orders.dtd"gt
• ltordersgt
• ltordergt
• ltheadergt
• ltbilling_infogt
• ltnamegt
• ltgivengtJohnlt/givengt
• ltfamilygtDoelt/familygt
• lt/namegt
• ltaddressgt
• ltstreetgt555 Main
  Streetlt/streetgt
• ltcitygtMill Valleylt/citygt
• ltstategtCalifornialt/stategt
  
• ltzipcodegt94520lt/zipcodegt
• ltcountrygtUSAlt/countrygt
• ltphonegt707
  555-1000lt/phonegt
• lt/addressgt

46
Extensible Style Language (XSL)

• XML ignores presentation of data
• XSL stylesheets define presentation of XML doc in
  some format

XML Doc
HTML, PDF, etc...
Compiler
XSL
in browser or WWW server
47
XML/XSL Impact

• Major vendor support/momentum
• Presentation
• Decoupling content/presentation is good!
• Presentation is the browsers job
• Data interchange
• business-to-business e-commerce
• DOM API - XML parsers produce a DOM
  representation of an XML document
• Microsofts SOAP

48
XML example

• From
• http//www.microsoft.com/msj/0100/com/com0100.asp
• ltorder orderno"33512"gt
• 
  ltcustomer custno"4462" /gt
• ltitem
  itemno"3352" /gt
• ltitem
  itemno"1829" /gt
• lt/ordergt

49
XSL example

• lt?xml version"1.0" ?gt
• 
  ltxslstylesheet xmlnsxsl"http//www.w3.org/TR/WD
  -xsl"gt
• 
  ltxsltemplate match"/"gt
• 
  ltxslapply-templates /gt
• 
  lt/xsltemplategt
• 
  ltxsltemplate match"" gt
• 
  ltxslelementgt
• 
  ltxslfor-each select"_at_" gt
• 
  ltxslelementgt
• 
  ltxslvalue-of /gt
• 
  lt/xslelementgt
• 
  lt/xslfor-eachgt
• 
  ltxslapply-templates select""/gt
• 
  lt/xslelementgt
• 
  lt/xsltemplategt
• 
  lt/xslstylesheetgt

50
XML Output

• ltordergt
• 
  ltordernogt33512lt/ordernogt
• 
  ltcustomergtltcustnogt4462lt/custnogtlt/customergt
• 
  ltitemgtltitemnogt3352lt/itemnogtlt/itemgt
• 
  ltitemgtltitemnogt1829lt/itemnogtlt/itemgt
• lt/ordergt

51
Example Code

• var dom new ActiveXObject("Microsoft.XMLDOM")
• dom.load("input.xml")
• var xsl new ActiveXObject("Microsoft.XMLDOM")
• xsl.load("transform.xsl")
• WScript.echo(dom.transformNode(xsl))

52
Another Example

• lt?xml version"1.0" ?gt
• 
  ltxslstylesheet xmlnsxsl"http//www.w3.org/TR/WD
  -xsl"gt
• 
  ltxsltemplate match"/"gt
• 
  ltxslapply-templates select"order" /gt
• 
  lt/xsltemplategt
• 
  ltxsltemplate match"order" gt
• Order
  ltxslvalue-of select"_at_orderno" /gt
• Customer
  ltxslvalue-of select"customer/_at_custno" /gt
• Items
• 
  ltxslapply-templates select"item" /gt
• 
  lt/xsltemplategt
• 
  ltxsltemplate match"order/item" gt Item
  ltxslvalue-of select"_at_itemno" /gt
• 
  lt/xsltemplategt
• lt/xslstylesheetgt

Order 33512
Customer
4462 Items
Item 3352
Item 1829
53
Summary

• Distributed object system technology is a key
  part of building enterprise distributed systems
• Theres a lot of
• design issues
• technology issues
• plain hard issues (ie testing)
• This course is just the entrée...