Database Access Using D0OM

1
Database Access Using D0OM

• H. Greenlee
• Calibration Database Browser Workshop
• Apr. 8, 2002

2
D0OM Components

• D0OM Object Model.
• Preprocessor and dictionary system.
• I/O Interface.
• Non-database d0Stream.
• Open/close/read/write.
• Database d0StreamDB.
• Query.
• Calibration Data Model.
• Superset of D0OM object model.
• I/O back ends.
• DSPACK/EVPACK.
• Direct Oracle (d0omORACLE).
• C to database mapping dictionary.
• Client-Server Oracle (d0omCORBA).
• C to database mapping dictionary.

3
D0OM I/O Classes
4
Calibration Architectures
Direct Database Connection
Database Client-Server Architecture
Oracle Database
5
Client Server Architecture

• Client and server use corba protocol to
  communicate.
• Client written in C (e.g part of d0reco).
• Uses orbacus corba implementation.
• Dynamic corba (no compiled in knowledge of
  database schema).
• Only type any (no dynany).
• Server written entirely in python.
• Uses omniorb corba implementation.
• Consists of high level part (schema independent)
  and low level part (schema-specific generated idl
  and python).
• Schema specific part generated using fnal product
  db_server_gen (script dbgen.py).

6
D0OM Object Model

• Atomic types.
• Strings.
• Arrays.
• Bare pointers.
• STL collections.
• D0OM base class d0_Object.
• D0OM smart pointer d0_RefltTgt.
• User-defined classes.

7
Calibration Data Model

• Database base class d0_DBObjectltKgt.
• Query Classes.
• d0_QueryltT,Cgt and d0_Query_Base.
• Database Pointers.
• d0_DynRefltTgt.
• d0_RefToltT,Kgt.
• Database Collections
• d0_Query_CollectionltT,Cgt.
• d0_RefByltT,K,Cgt.
• Database key class.
• Abstract class d0_DBKey.
• Concrete classes d0_DBKey_1ltXgt, d0_DBKey_2ltX,Ygt,
  etc.

8
Database Base Class

• Class d0_DBObjectltKgt contains database key.
• Template parameter class K must fulfill d0_DBKey
  interface requirement.
• Attribute name fields of key are filled
  automatically from dictionary when d0_DBObjectltKgt
  is constructed.

9
Database Pointer Classes

• The database pointer classes inherit D0OM smart
  pointer interface (d0_RefltTgt).
• The database pointer classes support true
  deferred I/O
• They can be initialized to point to an object in
  a database without doing any I/O. (This was not
  true for dspack d0_RefltTgt.)
• d0_DynRefltTgt represents a relationship based on a
  general query object, which can be specified at
  run-time.
• Can not be stored in database.
• d0_RefToltT,Kgt represents a static one-to-one
  relationship based on key class K.
• Key is automatically converted to query.
• Points to d0_DBObjectltKgt or d0_RefToltgt/d0_RefByltgt
  (symmetric relationship).
• Can be stored in database.

10
Smart Pointer Class Diagram
11
Database Collections

• The database collection classes have interfaces
  similar to an STL collection.
• The database collection classes also support true
  deferred I/O.
• d0_Query_CollectionltT,Cgt represents a run-time
  collection based on a general query (plural
  version of d0_DynRefltTgt).
• d0_RefByltT,K,Cgt represents a static one-to-many
  relationship based on key K (plural version of
  d0_RefToltT,Kgt).

12
Query Classes

• Query classes d0_QueryltT,Cgt and d0_Query_Base act
  as front end to query method d0StreamDBquery.
• d0_QueryltT,Cgt returns collection of d0_RefltTgts
  when executed.
• d0_Query_Base returns a collection of
  d0_Ref_Anys when executed.

13
Database Key Classes

• Abstract class d0_DBKey and subclasses.
• Unlike ordinary persistent data, database keys
  know the names of their database columns.
• Keys can easily be turned into query objects.
• Keys can be constructed by hand, or extracted
  from database base objects.

14
Public Indirect Pointers

• New public indirect pointer class
  d0om_Query_Indptr is not specific to any D0OM I/O
  mechanism (but requires d0StreamDBquery to be
  implemented by back end).

15
C to Database Mapping

• Goals
• Full support for D0OM object model, including
  polymorphism (e.g. for writing event data).
• Flexible C to database mapping (e.g. for
  reading a human designed database).

16
Database Representation of C Objects.

• Ordinary (non-database) persistent objects
  (derived from d0_Object).
• Database persistent objects (derived from
  d0_DBObjectltKgt.
• Object id. reference count columns are optional.

17
Database Representation (cont.)

• Ordinary pointers and d0_RefltTgts.
• Dynamic type column and type table are optional.
• Not use in D0.

18
Database Representation (cont.)

• Database smart pointer d0_RefToltT,Kgt.
• Dynamic pointers are not polymorphic.
• Object id and reference count columns not used in
  D0.

19
Database Representation (cont.)

• STL Collection.
• Not used in D0.

20
Database Representation (cont.)

• Database collection d0_RefByltT,K,Cgt.
• Object id and reference count not used in D0.

21
External Database Dictionary

• D0omORACLE and d0omCORBA use an external python
  mapping dictionary.
• The following elements of C to database mapping
  can be controlled.
• Class name Table name.
• Data member name Column name.
• Server-based foreign key relationships.
• Metadata table and column names (not currently
  used).
• External dictionary is input to db_server_gen.
• I believe that it is possible to generate
  database dictionary automatically (i.e. from
  oracle designer), but I dont know how.

22
Generating server files using db_server_gen

• Need database dictionary files.
• GENLIST.py (contains list of table modules)
• MASTER.py (list of directories)
• One python file for each database table.
• Python modules must be on PYTHONPATH.
• Run script dbgen.py to generate low level server
  files
• setup db_server_gen
• dbgen.py d0om
• Files generated
• Base layer idl files (one per table).
• Python server files.
• C header and implementation files.

23
Programming Using d0omORACLE/d0omCORBA

• Use stream factory to open stream.
• d0StreamFactory factory d0StreamFactorylocate
  StreamFactory()
• d0StreamDB stream factory-gtmake_d0StreamDB(name
  , , iosin)
• Specific stream type is ORACLE, CORBA, or
  (empty string).
• For d0omORACLE, specify stream name as database
  connect string username/password_at_database.
  (password optional).
• For d0omCORBA, specify stream name as corba name
  of server (as registered with name server).

24
Programming Using d0omORACLE/d0omCORBA

• Linking.
• Force load LoadORACLE.o or LoadCORBA.o.
• Link with D0 libraries ld0omORACLE or
  -ld0omCORBA.
• For product libraries use srt_d0 makefile
  fragments arch_spec_oracle.mk or
  arch_spec_corba_common.mk and arch_spec_orbacus.mk
  (automatic with ctbuild).

25
Running d0omCORBA programs

• Start the server.
• Add dictionary files (GENLIST.py, etc.) and
  generated python files to PYTHONPATH.
• Setup products omniOrb, dcoracle, and sam_util
  (or sam).
• Specify database user name/password using env
  variables DB_SERVER_USERNAME and
  DB_SERER_PASSWORD.
• Start server
• DbListener.py nltnamegt -td0om
  cServantFactory mltmodulegt
• Running the client.
• Setup fnal product sam_util or sam (makes corba
  naming service available).
• Run client program.
• Client can run on different node than server.
• Password is not required for client.

26
Documentation

• D0omCORBA and d0omORACLE
• d0omCORBA/doc/database_access.ps
• d0omCORBA/doc/database_dictionary.ps.
• Calibration data model.
• d0stream/doc/calibration_classes.ps.
• D0om generally.
• d0om/d0c/d0om_user_guide.ps.