COMS W4156: Advanced Software Engineering

1
COMS W4156 Advanced Software Engineering

• Prof. Gail Kaiser
• Kaiser4156_at_cs.columbia.edu
• http//bank.cs.columbia.edu/classes/cs4156/

2
Topics covered in this lecture

• Distributed computing overview
• More on CORBA client-server
• COM client-server
• MTS extends COM to 3-tier
• Note NO materials adapted from slides provided
  by the textbook author/publisher

3
Distributed Computing Overview
4
Distributed Computing 101Plumbing

• How does client identify server?
• How does server make its functionality known and
  available to prospective clients?
• How does client make its request?
• How does server return its response?
• What design-time support is available?
• What run-time support is available?

5
How does client identify server?

• One alternative hard-wired solution
• Tightly coupled code where any change to server
  may require modifications to client and vice
  versa
• Cannot plug-replace with new server from another
  vendor (better, faster, cheaper), possibly not
  even with a different version of the original
  server
• System administration nightmare
• Better alternative employ some standard
  discovery protocol that all prospective vendors
  agree to abide by

6
How does server make its functionality known and
available to prospective clients?

• Are separately specified interfaces used?
• If so, how expressive?
• Are interfaces enforced?

7
How does client make its request?How does server
return its response?

• Communication protocol
• How complicated? How (potentially) buggy?
• Percentage of code and development effort devoted
  to mechanism?

8
What design-time support is available?

• Quality Assurance
• Errors caught at runtime after deployment 1000x
  more expensive to fix than at design time
• Reduce errors in first place through intuitive
  idioms
• Type-safety checks incredibly useful (do
  parameters supplied by clients match those
  anticipated by server? do responses from server
  match those expected by client?)
• Interoperability
• Must client and server use same development
  language?
• Is third-party code reuse possible?

9
What run-time support is available?

• Standardized infrastructure crucial
• Reduced training, design, coding and testing
  costs
• More reliable and robust
• But need to consider impact on performance
• Interoperability (across vendors) essential
• Countless incompatible proprietary standards

10
Motivation for Component Model Frameworks

• Provide standard answers to the Distributed
  Computing 101 questions
• Designing, implementing, testing and deploying
  the plumbing is difficult and error-prone
• But nearly the same across many applications
• Put system programmers and distributing computing
  experts effort into doing it once per framework
  rather than once per application
• Leaves application logic (and business value) to
  application programmers and domain experts

11
CORBA
12
CORBA

• Common Object Request Broker Architecture
• Historically, one of the first organized
  frameworks for distributed computing (c. 1991)
• Specification developed and periodically revised
  by the Object Management Group
• Extremely influential
• Used especially as middleware in enterprise and
  business-critical infrastructures
• Not quite a component model as more recently
  envisioned, but on the way there (later CCM
  Corba Component Model)

13
CORBA Big Picture
Client
Server
IDLstub
IDLskeleton
ORB Object Request Broker
Request
Response
IDL Interface Description Language
14
CORBA Big Picture

• Server interface specified in language-independent
  IDL notation
• Client communicates request to ORB
• IDL compiler generates stub (in clients
  implementation language) to hide complexity
• Stub compiled and linked together with client
• ORB delivers request to Server
• IDL compiler generates skeleton (in servers
  implementation language) to hide complexity
• Skeleton compiled and linked together with server
• Analogous return path for response

15
Objects

• CORBA terminology can be confusing what
  Distributed Computing 101 calls a Server is
  instead called an Object
• What CORBA calls a Server is something else, in
  particular a runtime host process for one or more
  Objects
• CORBA Objects are object-oriented in the sense
  that they are individual units of running
  software that provide interfaces and combine
  functionality and data
• Typically, there are many instances of an Object
  of a single type e.g., an e-commerce website
  would have many shopping cart object instances
• For some types, there may be only one instance
  e.g., when a legacy application, such as an
  accounting system, is wrapped as a CORBA Object
  and opened up to clients on the network

16
CORBA Big Picture (Refined)

• Object type interface specified in
  language-independent IDL notation
• Client communicates request to ORB
• IDL compiler generates stub (in clients
  implementation language) to hide complexity
• Stub compiled and linked together with client
• ORB delivers request to Server host, which in
  turn delivers request to Object instance selected
  by Server
• IDL compiler generates skeleton (in servers
  implementation language) to hide complexity
• Skeleton compiled and linked together with server
• Analogous return path for response

17
Object Interfaces

• For each Object type, an interface is defined in
  OMGs IDL (Interface Description Language)
• The interface is the syntax part of the
  contract that the Object offers to the clients
• Any client that wants to invoke an operation on
  the Object must use this IDL interface to specify
  the operation it wants to perform, and to marshal
  the arguments that it sends
• When the invocation reaches the target Object,
  the same interface definition is used there to
  demarshal the arguments so that the Object can
  perform the requested operation
• And analogously wrt marshalling/demarshalling
  response

18
Object Marshalling/Demarshalling

• When objects in memory are to be passed across a
  network to another host or persisted to storage,
  their in-memory representation must be converted
  to a suitable out-of-memory format.
• This process is called marshalling (or
  serializing), and converting back to an in memory
  representation is called demarshalling (or
  deserializing).

19
Object Marshalling/Demarshalling

• During marshalling
• Objects must be represented with enough
  information that the destination host can
  understand the type of object being created.
• The objects state data must be converted to the
  appropriate format.
• Complex object trees that refer to each other via
  object references (or pointers) need to refer to
  each other via some form of ID that is
  independent of any memory model.
• During demarshalling
• The destination host must reverse all that.
• And must also validate that the objects it
  receives are consistent with the expected object
  type (i.e., it validate that it doesnt get a
  string where it expects a number).

20
CORBA Big Picture (Refined)
Client
Server
IDLstub
IDLskeleton
ORB
Object
Object IDL file
21
IDL Interface Description Language

• Neutral wrt implementation language
• IDL notation looks and feels remarkably like C,
  with some Pascal concepts added
• There are defined (or at least draft) mappings to
  Ada, C, C, Java, C, Python, Perl, Ruby, Lisp,
  XML, numerous others
• IDL Compilers generate native code in target
  language

22
IDL HelloWorld example

• module HelloApp
• interface Hello
• string sayHello()

• Module is a scoping unit
• Interface is set of Object method signatures
• Base types defined by CORBA include string, int,
  double, float, boolean, etc

23
More complicated example

• module StockObjects
• struct Quote
• string symbol
• long at_time
• double price
• long volume
• exception Unknown
• interface Stock
• Quote get_quote() raises (Unknown)
• void set_quote (in Quote stock_quote)
• readonly attribute string description

• Exception associated with modules may be raised
  by methods
• Attribute provides additional information

24
IDL expressiveness

• Method Signatures
• Declare arguments as inoutinout
• Can raise exceptions
• Can return a value
• Declarative Attributes
• readonly attribute type name
• Equivalent to having _get_att/_set_att(in p)
• Multiple Inheritance
• Interface ISpecI1,I2

25
CORBA Client-side

• Connect to ORB
• Contact NameService (standard service provided by
  any CORBA implementation)
• Locate (resolve) Object by name
• Typecast (narrow) to specific Interface
• Invoke desired method

26
Client-side perspective
Client

• Client shielded by Interface
• Client accesses ORB services
• Client communicates with stub proxy

interface
IDLstub
ORB
27
CORBA Server-side

• Connect to ORB
• Contact NameService
• Register (rebind) Object by name

28
Server-side perspective

• ORB receives call
• ORB passes request to Object implementation
  through skeleton
• Response sent back from Object to skeleton
• Sent back to client

Server
3. Response
IDLskeleton
1. Call
2. Invoke
4. To Client
ORB
29
Server-side perspective (Refined)

• ORB receives call
• ORB activates server
• Server calls BOAimpl_is_ready
• BOA instantiates Object in Server
• BOA passes request to Object implementation
  through skeleton
• Response sent back from object to skeleton
• Sent back to client

IDLskeleton
1. Call
BOA
ORB
BOA Basic Object Adapter
30
CORBA Evaluation

• Strengths
• Interfaces hide complexities
• Automatic language interoperability
• Weaknesses
• Client must know servers interface(s)
• Java RMI and other modern language facilities do
  everything CORBA does
• And todays component model frameworks do even
  more

31
CORBA Limitations

• Before version 3.0, c. 2002, which includes
  CORBA Component Model
• No common (mandatory) set of services implemented
  by all ORBs
• No standard way of deploying server Objects
  (adding new server Objects to an ORB)
• Each ORB infrastructure implementation had
  different approach to IMR (IMplementation
  Repository)

32
CORBA Limitations

• No support for common programming idioms
• Most server Objects implemented as factories,
  creating a new Object instance to deal with each
  new client, but new factory code needs to be
  written for each case
• Every programmer has same choices to make,
  between persistent and transient references,
  Objects identified or not by a primary key,
  Objects maintain persistent state or not, and
  then has to implement the decisions

33
CORBA Needs Components

• Binary (executable) units that can really be used
  interchangeably with any ORB
• Allows graceful evolution by replacing one
  component with another
• Eases porting to another ORB (better, faster,
  cheaper)
• Applications can then be built by assembling
  components
• Components must define what they need and what
  they offer
• Once assembled, deployment must be semi-automatic
  
• Need standard development, deployment and runtime
  environment

34
CORBA Component Model (CCM)

• Part of CORBA 3.0 specification, June 2002 (most
  recently revised in CORBA 3.1, January 2008)
• Extends CORBA object model
• New component meta-type
• Development by composition
• Similar to EJB (Enterprise Java Beans)
• Not widely used

35
COM/DCOM
36
Component Object Model (COM)

• COM specifies an object (or component) model, and
  programming and compiler requirements, that
  enable COM objects to interact with other COM
  objects
• A COM object is made up of a set of data and the
  functions that manipulate the data
• A COM objects data is accessed exclusively
  through one or more sets of related functions
  called interfaces
• COM requires that the only way to gain access to
  the methods of an interface is through a pointer
  to the interface
• Interfaces defined in Microsoft Interface
  Definition Language (analogous to CORBA IDL, but
  NOT the same)

37
Component Object Model (COM)

• COM is a binary standarda standard that applies
  after a program has been translated to binary
  machine code
• COM methods and interfaces must follow a
  prescribed in-memory layout
• Objects can be written in different languages,
  including languages that dont have objects
• COM allows objects to interact across process and
  machine boundaries as easily as within a single
  process
• Marshalling/demarshalling method parameters and
  return values across process and machine
  boundaries handled by operating system (in
  Windows COM implementation)

38
COM Clients and Servers

• A COM client is whatever code or object gets a
  pointer to a COM server and uses its services by
  calling the methods of its interface(s)
• A COM server is any object that provides services
  to clients
• These services are in the form of COM interface
  implementations that can be called by any client
  that is able to get a pointer to one of the
  interfaces on the server object

39
COM Overview
40
Types of COM Server

• An in-process server resides in a dynamic link
  library (DLL) and runs in the same address space
  as the COM client
• A local server resides in its own executable
  (e.g., .exe file), in a different process but on
  the same machine as the COM client
• A remote server runs on a different machine than
  the client

41
Same Machine

• For clients and servers on the same machine, the
  CLSID of the server is all the client ever needs
• On each machine, COM maintains a database (using
  the system registry on Windows) of all the CLSIDs
  for the servers installed on the system
• This is a mapping between each CLSID and the
  location of the DLL or EXE that houses the code
  for that CLSID
• COM consults this database whenever a client
  wants to create an instance of a COM class and
  use its services, so the client never needs to
  know the absolute location

42
Different Machines

• For distributed systems, COM provides registry
  entries that allow a remote server to register
  itself for use by a local client
• Applications need know only a server's CLSID,
  because they can rely on the registry to locate
  the server
• However, COM allows clients to override registry
  entries and specify server locations

43
COM vs. DCOM

• COM client applications do not need to be aware
  of how server objects are packaged, whether they
  are packaged as in-process objects (in DLLs) or
  as local or remote objects (in EXEs)
• Distributed COM (DCOM) is not separateit is just
  COM with a longer wire

44
COM Limitations

• Same as CORBA No support for common programming
  idioms (other than RPC)
• Unlike CORBA Has one main implementation -
  from Microsoft for Windows, so by definition
  compatible
• Needs component services distributed
  transactions, resource pooling, disconnected
  applications, event publication and subscription,
  better memory and processor (threads) management,
  
• COM 1993, DCOM 1996, (D)COM MTS 1998, COM
  2000, partially superseded by .NET 2002 (but
  compatible)

45
COM MTS
46
MTS Microsoft Transaction Server

• Enables the transactional requirements of each
  component to be set administratively, so
  components can be written separately and then
  grouped together as needed to form a single
  transaction

47
What Transactional Requirements?

• Many applications write to some database or other
  data repository
• An application that makes more than one change to
  a database may want to group those changes into a
  single unit called a transaction
• The goal is to make sure that either all of those
  changes take place or that none of them doa mix
  of success and failure isnt possible

48
What is a Transaction?

• ACID properties
• Atomicity (all or nothing)
• Consistency (no integrity constraints violated)
• Isolation (no one else sees intermediate states,
  more formally serializability)
• Durability (persistence failure recovery)
• Component model frameworks generally only
  concerned with atomicity and durability

49
Handling Transactions within One Data Source

• If all the data accessed are contained in a
  single database, the database itself probably
  provides transactions
• The programmer need only indicate when a
  transaction begins and when it ends
• The database will make sure that all data
  accesses within these boundaries are either
  committed or rolled back

50
Handling Transactions Across Multiple Data Sources

• The problem gets harder when an application
  accesses data contained in more than one database
  or involves other data resource managers such as
  a message queue
• Its probably not possible for the transaction
  support built into any one of those systems to
  coordinate the commitment or roll-back of all
  data accesses
• Need some kind of independent transaction
  coordinating service

51
What does the Transaction Coordinating Service do?

• Two-Phase Commit
• First phase transaction coordinator asks every
  resource manager (e.g., database) involved in the
  transaction if its prepared to commit
• Second phase
• If every resource manager says yes, then the
  transaction coordinator tells each one to commit
• If one or more of the resource managers involved
  in this transaction is unable or unwilling to
  commit (or does not respond within a timeout
  period), the transaction coordinator tells all of
  them to roll back the transaction

52
MTS Executive and Distributed Transaction
Coordinator (DTC)

• Invisible to a client, client sees just an
  ordinary COM object exposing some number of
  interfaces
• Executive intercepts every call the client makes
  on the objects it controls
• DTC manages the two-phase commit

53
MTS Application Structure
54
Two-Phase Commit
55
Context Object

• Every object involved in a transaction is
  associated with a context object
• If a transaction needs to span the functions of
  multiple different objects, then same context
  object coordinates the activities of all of those
  objects
• Allows the transaction semantics to be separated
  from the application code

56
Automatic Transactions

• In traditional transaction systems, the
  application code indicates when a transaction
  begins and ends
• MTS also supports automatic transactions,
  allowing business logic to remain unaware of when
  transactions start and end

57
Transaction Semantics Defined Declaratively

• Transaction semantics declared when components
  are assembled into an application package
• Each component is assigned a transaction
  attribute, one of four values
• Required
• Requires New
• Supported
• Not Supported

58
Transaction Options

• REQUIRED - The component must execute within the
  context of a transaction, although it does not
  care where the transaction context originates
• If the caller has a transaction (context object),
  then the called component will use the existing
  transaction
• If the caller does not have a transaction, then
  MTS automatically creates a new transaction
  context for the component

59
Transaction Options

• REQUIRES NEW - indicates that the component must
  always establish its own transaction context
• Regardless of whether or not the calling
  application has a transaction context, MTS
  automatically creates a new transaction context
  object for the component

60
Transaction Options

• SUPPORTED - indicates that the component does not
  care whether or not there is a transaction
  context in place
• NOT SUPPORTED - indicates that the component
  cannot support a transaction

61
Commit or Abort

• The component operating on the DB can tell the
  MTS executive when its task is complete
• Everything has gone just fine and the transaction
  is ready to be committed the component calls
  IObjectContextSetComplete (or just returns)
• Something has gone wrong perhaps one attempt to
  access data resulted in an error and the entire
  transaction should be rolled back the component
  calls IObjectContextSetAbort (or just crashes,
  hangs, etc.)

62
Coordination of Multi-Component Transactions

• Calling SetComplete does not necessarily mean
  that the transaction will be committed right then
• If this component implements all the work
  required for an entire transaction, then MTS will
  perform the commit

63
Coordination of Multi-Component Transactions

• But this component may be part of a group of
  components, all of which collectively participate
  in a single transaction
• Each component will call SetComplete (or
  terminate) when its work is done, but MTS wont
  begin the commit process until all components
  within the transaction have completed
  successfully
• The code for the component looks the same in
  either case

64
And a few other services
65
Just-In-Time (JIT) Activation

• Also known as deferred activation
• When a client makes a call to an object to create
  an instance, COM provides that client a
  reference to a context object instead of a
  reference to the object
• Client gets a real reference to the object, and
  the object is activated, when client calls a
  method of that object
• Object deactivated when method returns and
  reactivated when next method called
• Deactivated object releases all resources,
  including locks on data stores
• Allows server resources to be used more
  productively

66
Object Pooling

• Recycling of objects
• When a client releases an object that supports
  object pooling, or such an object is deactivated,
  instead of destroying that object completely,
  COM recycles it
• When another client requests or reactivates the
  same kind of object, COM gives an instance from
  the pool
• Since these component instances are already
  loaded in memory (up to maximum size of pool),
  they are immediately available for use
• If you intend to make only one call at a time on
  a pooled object, it is a good idea to enable JIT
  activation with object pooling if you intend to
  get a reference and make multiple calls on it,
  using object pooling without JIT activation may
  result in better performance.

67
Connection Pooling

• Opening and closing connections to a database can
  be time-consuming
• Reuse existing database connections rather than
  create new ones
• A resource dispenser caches resources such as
  ODBC (Open DataBase Connectivity) connections to
  a database, allowing components to efficiently
  reuse them

68
Role-Based Security

• Role a logically related group of users that
  share the same permissions to access a defined
  subset of an applications functionalities
• Assign different permissions for different roles
  on a class, interface or method
• Can set either administratively or via
  programming
• Dont need to write security-related logic into
  components (but can do so if desired)

69
So How Does MTS Fit With COM?
70
MTS Extends COM to 3-tier Architecture
71
Client-Server Architecture
Client
Server
72
2-tier Architecturewith Basic Component
Middleware
Client
Server
Component middleware
73
3-Tier Architecturewith Component Services
Middleware
Document Storage
Client
Database
Application logic components
LDAP
Component services middleware
74

• Final Notes

75
ReminderProject Concept due soon!

• Teams posted on the website team page
• Project concept due September 23rd
• Each team submits one document as a group
• Submit on courseworks

76
Upcoming Deadlines

• Team project concept due September 23th
• Project concept feedback by September 30th
• First iteration begins September 30th

77
COMS W4156 Advanced Software Engineering

• Prof. Gail Kaiser
• Kaiser4156_at_cs.columbia.edu
• http//bank.cs.columbia.edu/classes/cs4156/