Overview of Pegasus An Open-Source WBEM implementation

1
Overview ofPegasusAn Open-SourceWBEM
implementation

• 12 June 2001
• Michael Brasher (m.brasher_at_bmc.com)
• Karl Schopmeyer(k.schopmeyer_at_opengroup.org)

Version 1.0
2
Agenda

• Overview -What (and why)is Pegasus?
• The Pegasus Environment
• The Pegasus Software Architecture
• Pegasus Status Today
• Plans
• The Pegasus Project
• Pegasus and ???
• A Challenge for the Future

3
1. Overview
4
What is Pegasus?

• Pegasus is an open-source reference
  implementation of the DMTF WBEM specifications
• Pegasus is a work project of the TOG Enterprise
  Management Forum
• Pegasus is a platform for building application
  management
• Pegasus is a function-rich, production-quality
  open-source implementation designed to be used in
  high volume server implementations.

5
Why Produce Pegasus?

• Demonstrate manageability concepts.
• Provide additional standards for WBEM
• Provide a working implementation of WBEM
  technologies
• Provide an effective modular implementation
• Support other TOG manageability standards
• Base Platform for Open Group Application
  management Projects

6
Origins of Pegasus

• Pegasus was initiated in 2000 by the Open Group
  in collaboration with
• BMC Software
• IBM
• Tivoli Systems

7
Key Pegasus Objectives

• Open source
• MIT License, open availability
• Portable
• Code designed for portability
• Efficient and Lightweight
• Implemented in C
• Standards Based
• DMTF CIM/WBEM compliant
• Modular and extensible
• Use both internal and external modularity

8
Pegasus Working Group Philosophy

• Manageability not management
• The working groups objective is not to manage
  systems but to make them manageable by promoting
  a standard instrumentation environment
• The actual management of systems is left to
  systems management vendors
• No standards without implementation
• The process of implementation provides a rigorous
  process for testing the validity of standards
• Therefore all standards must be validated by
  implementation

9
Major Objectives of Pegasus

• Standards Based and Compliant
• DMTF CIM/WBEM
• Interoperable
• DMTF cim-xml Interface
• Efficient and Lightweight
• Implemented in C
• Portable
• NT, Linux, and others planned
• Modular
• Replaceable modules for function change and
  growth
• Extensible
• Manageability Service extensions

NOTE THIS SLIDE GOES AWAY
10
Open Source

• Code and documentation freely available
• Open Group and Source Forge
• MIT source License
• Open to contributions

11
Portable

• Designed for multi-platform, mulit-os,
  multi-compilers.
• Platforms today
• UNIX (AIX, HPUX, Solaris)
• Linux
• Windows Platforms (NT, 2000, 9x)
• Compaq Himalaya (Tandem)

12
Efficient and Lightweight

• Core written in C
• Designed for execution efficiency

13
Standards Based

• Based on DMTF CIM and CIM-XML specifications
• Open Group is active partner in DMTF
• Growth through participation in specification
  growth
• Commitment to continue DMTF compliance

14
Modular and Extensible

• Minimize core object broker.
• Maximize extensibility through plug-in components
• Component types
• Providers
• Provider interfaces
• Clients
• Repositories (additional repository handlers)
• Manageability service extensions
• Protocol Adapters
• Modules (extend and modify core functions)

Modularity is key to doing parallel development
and allowto extensibility
15
2. The Pegasus Environment
16
Pegasus Architecture
Standard Interfaces
Interoperable
CIM Server
Interoperable
CIM/HTTP
In-Process
17
Key Interoperability Interfaces
Management System
Enterprise Management Console

• Manageability to Manager
• Multiple management systems
• Common open manageability

CIM Object Manager

• Object Manager / Providers
• Multiple Providers
• Encourage common providers

CIM Providers

• Provider / Resource Interface
• Protect Applications
• Make application management easy

Application
Application
Application
Application
18
The CIM Operations
Standard Interfaces
CIM Operations
Interoperable
CIM Object Mgr
Repository
Repository
Interoperable
CIM/HTTP
In-Process
Indicators
19
Operations Routing

• Class Operations
• Routed to the Class Repository
• Instance Operations
• To Provider if Provider Qualifier exists
• To Instance repository if no Provider
• Instance routing at Class Level Today
• Issues Routing at instance level

20
Key Characteristics

• Open source
• Available Today
• Portable
• Designed to build and run on wide variety of
  platforms
• C core
• C CIM Objects
• C Operation/Provider/Service/Repository
  interfaces
• Modular and extensible
• Modular components to extend the core
• Manageability service extensions to extend
  functionality
• Light weight

21
Modularity and Extensibility

• Providers
• Grow with DMTF provider concepts
• Provider Interfaces
• Protocol Adapters (connectors)
• Client - Xml-cim today (Soap, etc. in future)
• Provider, service, repository, etc.
• Modules
• Modularize core so it can be extended and
  modified through attachable modules
• Manageability Service Extensions
• Think super providers

22
Building A Modular Manageability Environmnent
Core Object Broker
Connector
Connector
. . .
Provider
23
Pegasus Manageability Environment
Management System
Management System
CIMOM
Services core additional
Application Consumer
Application Consumer
Management System Connector
Application Consumer
Consumers Gateways Apps
XML/CIM Connector
Management System Connector

• CIM Object Broker
• Provider Registration
• Service Registration
• Request Routing
• Securiy

Broker
Class Repository
Instance Repository
AIC Provider
ARM Provider
SNMP Provider
. . .
Providers
Interface For Spec
Apps
OS
Etc.
Resource
24
Provider Interoperability

• In the classical architecture, interoperability
  is only supported between the client and server.
• In addition, the Pegasus architecture aims to
  support provider/server interoperability.
• Goal
• Write a provider once and run it under any CIM
  server implementation.
• Provider/Server Interoperability
• Participating in efforts to standardize the
  Provider/Server protocol.
• Proposing provider API standards.
• Writing adapters enabling Pegasus providers to
  run under other CIM servers.
• Adapters enabling other providers to run under
  Pegasus

25
Important Provider Interfaces

• SUN WBEM Provider Interface
• Java based
• Classes, etc. similar to Pegasus
• C Provider Interface
• Sun has a specification here.
• We will support multiple provider interfaces and
  language bindings.

26
In-Process and Out-of-process Providers

• Today Pegasus based on shared Library Providers
• Extend to
• Internal Providers
• IPC based Providers
• Providers in Remotes systems
• Objectives
• Write Provider once and compile/link for
  different environments
• Technique
• Use connectors as basis for provider/CIMOM
  communication
• Issues
• Security, discovery

27
Modules

• The core server functions are organized into
  loadable modules.
• Standard APIs are defined for each module.
• Alternative implementations can be provided later
  without recompiling the Pegasus server.

28
Manageability Service Extensions

• Super Providers
• Access to the Core Broker

29
Example Services

• Indication Management service.
• Query engine service.
• Class repository service.
• Instance repository service.

30
Connectors (Protocol Adapters)

• Functions
• Adapt to different protocols
• Characteristics
• Protocol
• Encoding
• Security
• Discovery
• Examples
• Xml-CIM
• Local Protocols
• Soap
• WMI

Xml-cim Client
Xml-cim Client
Xml-cim Connector
Soap Connector
Pegasus Core
Pegasus Provider
Connector
Connector
Java Provider
Remote Provider
31
Pegasus Interfaces

• Common Interface base for
• Clients, providers, services, connectors
• Based on CIM Operations over HTTP
• Additional functions for each interface
• Interfaces separated from implementation

Core Object Broker
32
3. The Pegasus Software Architecture
33
Introduction
34
Programming Language Support

• The Pegasus core is implemented in C and hence
  client and provider interface are provided for
  C.
• An integrated JVM is planned to allow providers
  to be developed in Java.
• Of course it is possible to use existing Java
  clients to interact with the Pegasus CIMOM.

35
Major Components
Client
Client
CIM Clients
Repository
CIM Server
Client
Client
CIM Providers
36
Topics

• Communication.
• Representation of CIM Elements.
• The Client Interface.
• The CIM Object Manager.
• The Provider Interface.
• The Repository Interface.

37
Communication
38
Pathways of Communication
Client
Client
CIM Clients
CIM Repository
CIM Server
Client
Client
CIM Providers
39
Component Location

• A component may be located in one of three places
  with respect to the CIM Server.
• In-process.
• Local out-of-process (on the same machine).
• Remote out-of-process (on another machine).
• For example, a provider may be in-process, local,
  or remote.

40
Component Location in Pegasus Today
41
Possible Communication Mechanisms

• Components could potentially communicate with the
  CIM Server using the following mechanisms
• CIM/HTTP (remote).
• Proprietary TCP-based protocol (remote).
• Direct call (in process).
• Shared memory (local).
• Named pipes (local).

42
Communication Mechanisms in Pegasus
43
Client Communication

• Uses CIM/HTTP as sole protocol.
• Asynchronous socket I/O.
• An efficient XML parser.
• Fast enough to eliminate the need for a
  proprietary protocol.

44
An Efficient XML Parser

• No memory heap usage during parsing.
• Modifies message in place to avoid copying.
• Non-validating parser (loose validation).

45
HTTP Implementation

• Uses asynchronous socket I/O in conjunction with
  message queues to achieve optimal throughput.
• Provides entry points to adding web server
  capabilities such as putting and getting of
  documents (to support remote upgrade and
  deployment later on).

46
Proposals

• Support out-of-process providers (local and
  remote).
• Support out-of-process repositories (local and
  remote).
• Location independent provider development.

47
Representation of CIM Elements

• Representing CIM Elements in Pegasus with C

48
CIM Data Types in C

• Uint8
• Sint8
• Uint16
• Sint16
• Uint32
• Sint32
• Uint64
• Sint64
• Real32
• Real64

• Boolean
• Char16
• String
• CIMDateTime
• CIMReference

49
CIM Values in C

• CIM values (property, parameter, and qualifier
  values) are represented using the CIMValue class.
  This class
• Encapsulates a union of all CIM data types.
• Has a type member indicating the type currently
  being represented.
• Provides access/modifier methods overloaded for
  each CIM data type.

50
CIM Elements in C

• CIMClass
• CIMInstance
• CIMProperty
• CIMMethod
• CIMParameter
• CIMQualifierDecl
• CIMQualifier

51
Class Declaration Example (Part 1)

• Consider the following MOF class declaration
• class Alarm
• key
• uint64 id
• string message none

52
Class Declaration Example (Part 2)

• This class is defined in C as follows
• CIMClass alarmClass(Alarm)
• CIMProperty id(id, Uint32(0))
• id.addQualifier(CIMQualifier(key, true))
• CIMProperty message(message, none)
• alarmClass.addProperty(id)
• alarmClass.addProperty(message)

53
Class Declaration Example (Part 3)

• Or more succinctly like this
• CIMClass alarmClass(Alarm)
• alarmClass
• .addProperty(CIMProperty(id, Uint32(0))
• .addQualifier(CIMQualifier(key, true)))
• .addProperty(CIMProperty(message, none))

54
Property Iteration Example

• The properties of a class may be iterated like
  this
• CIMClass c
• for (Uint32 i c.getPropertyCount() i lt n
  i)
• CIMProperty p c.getProperty(i)

55
The Client Interface
56
The Client Interface
Client
Client
CIM Clients
Repository
CIM Server
Client
Client
CIM Providers
57
The Client Interface

• A C interface for interacting with the Pegasus
  Server (or any CIM Server).
• Uses CIM/HTTP to communicate.
• Provides a method for each CIM operation defined
  in the CIM Operations over HTTP, V1.0
  specification.

58
The CIM Operations

• GetClass
• GetInstance
• DeleteClass
• DeleteInstance
• CreateClass
• CreateInstance
• ModifyClass
• ModifyInstance
• EnumerateClasses
• EnumerateClassNames
• EnumerateInstances
• EnumerateInstanceNames
• ExecQuery

• Associators
• AssociatorNames
• References
• ReferenceNames
• GetProperty
• SetProperty
• GetQualifier
• SetQualifier
• InvokeMethod

59
CIM Operation Example
ltinstancegt GetInstance ( IN
ltinstanceNamegt InstanceName,
IN,OPTIONAL boolean LocalOnly true,
IN,OPTIONAL boolean IncludeQualifiers false,
IN,OPTIONAL boolean IncludeClassOrigin
false, IN,OPTIONAL,NULL string
PropertyList NULL )
CIM Operations Specification
virtual CIMInstance getInstance( const
String nameSpace, const CIMReference
instanceName, Boolean localOnly true,
Boolean includeQualifiers false, Boolean
includeClassOrigin false, const
ArrayltStringgt propertyList NULL_PROPERTY_L
IST)
Pegasus Class Method
60
Client Connection Example

• A client connects to the CIM Server on the host
  called saturn at port 5988 like this
• Selector selector
• CIMClient client(selector)
• client.connect(saturn5988)

61
GetClass Example

• A client gets a class like this
• CIMClass alarmClass
• client.getClass(root/cimv2, Alarm)

62
Client Proposals

• Asynchronous Interface APIs
• Interface independent of local/remote client

63
The CIM Object Manager

64
The Pegasus CIMOM
Client
Client
CIM Clients
Repository
CIM Server
Client
Client
CIM Providers
65
Request Lifecycle (Part I)
Incoming Request
Channel
1. Receive TCP Message
Protocol
2. Process HTTP Request
Encodings
3. Decode from XML
Dispatcher
Repository
4. Dispatch Request
66
Request Lifecycle (Part II)
Outgoing Response
Channel
8. Transmit TCP Message
Protocol
7. Form HTTP Response
Encodings
6. Encode to XML
Aggregator
Repository
5. Aggregate Results
67
CIMOM Modules

• Channel - Implements transport mechanism (e.g.,
  TCP).
• Protocol - Implement application protocol (e.g.,
  HTTP).
• Encodings - Implements encoding and decoding of
  messages (e.g., CIM-XML).
• Dispatcher - Dispatches messages to
  provider(s)/repository.
• Repository - Implements CIM Repository.
• Aggregator - Aggregates results.

68
Module Concept (Proposed)

• The internal CIMOM architecture is being refined
  into a set of modules.
• Each module handles part of the request
  lifecycle.
• Alternative implementations may be provided for
  each module type.
• New module implementations can be used to modify
  or refine the behavior of the CIMOM and to
  facilitate porting.

69
Module Implementations (Proposed)

• Channel - provide a named-pipe implementation.
• Protocol - provide a proprietary binary protocol.
• Encodings - provide a binary encoding scheme.
• Repository - provide a relational-based
  implementation.
• Traffic Encryption - SSL implementation.
• Authentication Challange and response

70
Module Definition and Loading (Proposed)

• Modules implement one of the module interfaces
  (e.g., Repository).
• Modules are configured as dynamic libraries.
• Modules listed in modules file.
• Modules loaded by CIMOM on startup.

71
Channel Module Example

• A channel connects two endpoints and enables them
  to communicate.
• Uses acceptor/connector pattern from ACE
  Wrappers.
• Pegasus provides a TCP implementation which works
  on Windows and Unix.
• Other implementations could be provided for named
  pipes, SNA, or for TCP on other platforms.

72
The Channel Module
Server
Client
1. Connect
Acceptor
Connector
3. Accept
2. Create
4. Create
Channel
Channel
Read/Write
73
The Provider Interface
74
The Provider Interface
Client
Client
CIM Clients
Repository
CIM Server
Client
Client
CIM Providers
75
The Provider Interface

• The provider interface is defined by the
  CIMProvider class.
• The provider interface has a method for each CIM
  operation in the CIM Operations over HTTP, V1.0
  specification.
• CIMProvider is a base provider interface (more
  refined provider interfaces are being developed).

76
CIMProvider Methods

• getClass()
• getInstance()
• deleteClass()
• deleteInstance()
• createClass()
• createInstance()
• modifyClass()
• modifyInstance()
• enumerateClasses()
• enumerateClassNames()
• enumerateInstances()
• enumerateInstanceNames()
• execQuery()

• Associators()
• associatorNames()
• references()
• referenceNames()
• getProperty()
• setProperty()
• getQualifier()
• setQualifier()
• invokeMethod()

77
Example of modifyInstance() Method

• The provider developer derives from CIMProvider
  and overrides the modifyInstance() method.
• class MyProvider public CIMProvider
• virtual void MyProvidermodifyInstance(
• const String nameSpace,
• CIMInstance modifiedInstance)
• // Modify the instance here!

78
Provider Registration and Loading

• Providers are configured as dynamic libraries
  with an entry point function for creating the
  provider instance.
• The provider() qualifier is placed on the
  corresponding class in the repository (this
  qualifier specifies the library name containing
  the provider implementation) .
• The provider is loaded on demand (when instances
  of that class are requested).

79
Refined Provider Interfaces (Work in Progress)

• Refined provider interfaces are now being
  developed. The CIM operations are being assigned
  to different types of providers
• Instance provider
• Method provider
• Query provider
• Event provider
• Association provider

80
Provider Interface Managers (Planned)

• Provider Interface Managers map the base provider
  interface onto one or more other provider
  interfaces. This scheme may be used to
• Refine the base provider interface into several
  provider interfaces (as mentioned above).
• Map the provider interface onto other language
  provider interfaces (e.g., Java, Perl, Tcl).

81
Provider Proposals

• Interoperability with SUN Wbem providers proposal
• We are extending other interoperability ideas

Pegasus CIMOM
SUN WBEM CIMOM
Microsoft CIMOM
Pegasus Providers
SUN WBEM Providers
Microsoft WMI Providers
82
The Repository Interface
83
The Repository Interface
Client
Client
CIM Clients
Repository
CIM Server
Client
Client
CIM Providers
84
The Repository Interface

• Defines the interface to manipulating the CIM
  repository.
• Alternative implementations of this interface may
  be provided.
• Pegasus provides a simple default implementation.

85
The Default Repository Implementation

• Each CIM object is stored in its own file.
• CIM objects are encoded as XML in files.
• Namespaces are represented using file system
  directories.
• Single reader/writer at a time.
• Example the class X (subclass of Y) which
  resides in the root/cimv2 namespace would be
  stored in this file
• ltrepository_rootgt/rootcimv20/classes/X.Y

86
The Default Repository Limitations

• Adequate for classes and qualifiers (which tend
  to be few).
• Okay for a few hundred instances.
• Not intended to scale to thousands of instances
  (uses a lot of disk space due to internal
  fragmentation).
• Good for scenario in which a few instances come
  from the repository and many instances come from
  providers.

87
Alternative Repository Modules

• Repository modules may be developed to achieve a
  highly scalable repository. Possibilities
  include
• Basing repository on a relational database.
• Basing repository on Unix DB.
• Making the repository remote.
• Improving space utilization of existing
  implementation by storing objects in binary
  format (rather than as XML).

88
New Default Repository Implementation (Proposed)

• A new repository implementation is under
  consideration to improve scalability
• Encode objects in binary on disk (rather than
  XML). This will reduce size by three to one.
• Combine like objects into a single file rather
  than separate files (this will reduce internal
  fragmentation).
• Provide a fast indexing scheme for keys (disk
  hashing or B-Trees). This will improve lookup
  time.

89
4. The Pegasus Implementation Today And Tomorrow
90
Status Today

• Phase 1 almost complete (0.98 today)
• Phase 1
• CIMOM including
• HTTP Server and CIM-xml Protocol
• Provider interfaces and dispatch
• Repository with Instances, Classes, Associations
• C CIM Object Model representation
• Admin and support functions
• MOF Compiler
• Client Interface and test clients
• Test providers and beginning of some real
  providers
• Integrated unit and client server tests
  (regression)
• Beta level code quality
• Needs More testing

91
CIM Functionality Missing From Phase 1

• Events
• Security
• WQL and Query

92
Important Tasks Phase 1

• Testing
• Pegasus
• Interoperability
• More Providers for testing and Demos
• Better Clients
• Documentation
• Binary Release

93
Planned Extensions

• CIMOM Functions
• Security
• Indications
• Threading
• Async CIM Operations APIs
• More Modularity
• Enhance Service Configuration
• Expanding Provider Characteristics
• Out-of-Process Providers
• WBEM Functions
• Discovery
• CIMOM Object Manager Characteristics
• Provider Registration and Operations

• Clients
• Object Browser
• Protocol Adapters
• Add protocol Adapters
• Providers
• Test
• Sample Implementations
• Platforms
• Easier portability
• More platforms

94
Phase 2 Priority tasks

• Threaded Model
• Basic Indications Support
• CIM Object Manager Capabilities Reporting
• Discovery Demonstration
• Additional Modularization

95
5. The Pegasus Project
96
Overview of the Project

• Active project of the Enterprise Management Forum
  of the Open Group
• Produce
• Pegasus open-source Implementation
• Core, clients, providers, repositories
• SDKs (Provider and Client)
• Documentation for use and development
• Specifications for major Interfaces
• Continue support and growth of Pegasus
• Portability
• New functions
• New Standards requirements
• New Providers
• Tools

97
Pegasus Status Today

• Phase 1 of 4 Phases
• Version 0.98 Available
• Source Code available today
• Preliminary documentation available
• Multiple users evaluating today
• Tested on the platforms defined

98
Pegasus Project Phases

• Phase 1 (June 2001)
• Goals
• Model Validation
• Client and Provider development
• Basic Environment
• Core model
• Cim-xml Operations
• Class and instance repositories
• Providers

• Phase 2
• Goals
• Production Environment
• Additions
• Threading
• Configuration
• Security
• Service Extensions
• Indications
• Phase 3
• Remote Providers
• Discovery
• Other extensions including other Language
  Interfaces (ex. Java connector)

99
Participants

• BMC Software
• Compaq Computer
• Focal Point
• Hermes Softlab
• Hewlett Packard
• IBM
• SIAC

• The Open Group
• Research Institute
• Management Forum
• Tivoli

100
Working Together

• Open Source but coordinated
• Executive Committee controls strategy
• Single Architect
• Work from Proposals
• Agree to objectives before doing the work
• Regular, Stable Releases
• Developers style guides
• Project scheduling and planning

101
Additional Activities

• Providers
• Clients
• Growth of functionality with DMTF
• Discovery
• Provider standardization (registration,
  interfaces)
• Next generation interoperability

102
Pegasus Manageability Environment
Object Browser Editor
Client SDK
Service extensions
Consumers Gateways Apps
XML/CIM Connector
Security

• CIM Object Broker Broker)
• Provider Registration
• Service Registration
• Request Routing

Broker
Queuing
Class Repository
Security
Events
MOF Compiler
Instance Repository
Provider SDK
ARM Provider
. . .
Provider SDK
Providers
Remote Provider
Interface For Spec
Resource
103
Pegasus and Other Manageability Projects

• AIC Application and Control
• AIC as a Pegasus Provider
• ARM Applications Response Measurement
• ARM and DMTF DAP Information as Pegasus Provider
• Other possible Providers
• JMX (Java)
• SNMP (mapping already defined)
• DMI (mapping already defined)

104
6. A Challenge for all of us
105
CIMOMs - Basic Concepts

• Tool to create Management Interoperability
• Infrastructure for manageability
• Manageability interoperability
• Xml-cim today, ??? Tomorrow
• Instrumentation Interoperability
• Many providers, few CIMOMs
• Lots of applications limited numbers of
  providers

106
However

• We do not make money off of infrastructure
• If we dont have common interfaces we will not
  have interoperability.
• CIM is not Easy. Creating complete and Correct
  CIM environments is not easy
• There is a lot of work to do with a common
  environment and much more with many different
  environments

107
The Alternatives

• Creating a common interoperability environment
• Management Manageability xmp-cim
• Providers APIs and protocols
• Provider building Common object implementations
• The choice
• Build a common structure with contributions
• Everybody does their own thing. (Many
  incompatible and incomplete WBEM Implentations

108
WBEMsource Consortium

• Consortium to create common WBEM manageability
• In fomative stages today
• About 10 involved organizations today
• Sun, Compaq,IBM, Tivoli, Open Group, SNIA,
  Caldera, Novell, Nokia, Intel
• Open Group Proposing to host
• Announced at JavaOne 2001

109
WBEMsource Objectives
Create a library of providers and tools
Create an environment of open-source WBEM
implementations

• Create common interfaces and APIs

WBEMsource
Create commonality between implementations and
integrate different implementations.
Create an environment of conformance
110
The Challenge!!!

• Can we create a common WBEM infrastructure?
• OR
• do we all go our own way?

111
Where to Get More Information

• Pegasus is public and Open Source
• www.opengroup.org/management/pegasus
• Pegasus WEB Site
• Source Code
• Builds on Linux and Windows
• Snapshots and CVS
• Binary Release
• Documentation
• Pegasus Working Group

This presentation is on the Pegasus Site
Contributors and Users of the Code are Welcome
112
7. QUESTIONS?