Strategies for All Your Data

1
Session id 40236
Strategies for All Your Data
Sandeepan Banerjee Vishu Krishnamurthy Oracle
Corporation
2
Where are you spending your money ?

• Data Management
• Labor
• Software Integration
• Hardware and System Integration

3
Too much information in too many places
Specialized

• Specialty Servers For Different Kinds Of Data
• Data Isolation
• High Systems Admin And Management Costs
• Scalability Problems
• High Training Costs
• Complex Support Problems

Relational
Messages
Documents
XML
Multimedia
Location
4
One Management System for All Your Data
RelationalCharacters, Numbers and Dates

• Complete
• Integrated
• Robust
• Scalable
• Secure
• Available on all platforms

XML DBIntegrated Native XML Database
Oracle Text Ultra SearchText management and
search
Oracle Locator SpatialLocation and Proximity
Searching
Oracle interMediaMultimedia management
Oracle Collaboration SuiteUnified Messaging and
Files
Extensibility FrameworkChemical, Genetic,
Engineering,
5
What is Oracle XML DB?

• Database support for the XML data model
• XMLType, XMLSchema, DOM Fidelity, Xpath,
• Hierarchical organization of the data
• WebDAV compliant with indexing for fast access
• Transparent storage optimizations
• Query Language SQLX and XQuery

6
Classes of XML DB Applications

• Exchanging Structured Documents
• Well-formed templated business-documents e.g.
  Purchase Orders, Phone Bills,
• Managing Unstructured Documents
• Documents, Messages, Instructions
• Integrating and normalizing data from diverse
  sources

7
Structured Document Exchange

• Relational storage remains the right way to
  store highly structured data
• As an XML programmer, you do not want to think
  about tables
• A hierarchical data model is what you want to
  manipulate
• XML DBs XMLType is about preserving the XML
  paradigm while getting the benefits of relational
  performance and scalability

8
Structured Document Exchangewith Oracle XML DB

• XML data model and APIs familiar to XML
  programmers
• XML Schema, Schema Validation, Dom Fidelity
• JNDI, DOM, XPATH, SQLX, XQuery
• Enterprise Class Performance Scalability
• Piecewise updates
• Schema caching
• Lazy materialization
• Server-based XSL transformations

9
Structured Data Temenos

• GLOBUS Banking platform 1 selling platform,
  major banks worldwide
• Contract-based system, deeply nested data model,
  user-customizable
• 80 major subsystems, 6000 Tables, 100s of GB

Using Oracle XML DB, we successfully
benchmarked 22 million banking transactions per
day, which translated to 2500 database-transaction
s-per-second, for Temenos' GLOBUS banking
platform. Oracle XML DBs performance assured us
that powerful XML innovations can be
operationalized and deployed without sacrificing
enterprise-class scalability. -
TEMENOS
10
Managing Unstructured Data

• More and more content is being produced as XML
  (Microsoft Word, Corel XMetal, Arbortext Epic, )
• Markup improves search, processing, organization,
  
• XML DBs Repository enables XML document content
  to be stored as files in folders without
  losing strong-management, queryability,
  unbreakable security etc.
• XML is doing for unstructured data what
  Relational did for structured create a standard
  way to store, query and manage unstructured data

11
Managing Unstructured Data with Oracle XML DB

• XML data model and APIs familiar to Content
  Developers
• Integrated Repository
• WebDAV compliant
• Xpath index for fast traversal of foldering
  hierarchies
• SQL Queryable
• Integrated Text Processing
• Optimizations such as tag aware search

12
Reed Elsevier

• Large technical publishing conglomerate
• More than 1700 scientific, technical medical
  peer-reviewed journals
• Over 59 million abstracts
• Over two million full-text scientific journal
  articles , another one million full-text articles
  via CrossRef (http//www.crossref.org/) to other
  publishers' platforms
• XML DB chosen as Repository Database

13
10g Whats new in XML DB

• Broad Performance Improvements
• SQLX query rewrites
• XSLT optimizations
• Repository Access and Query optimizations
• Direct loader support, loading large XML
  documents
• Storage optimizations
• I18N support for differing character sets on
  client and server
• Schema Evolution
• Transparently achieves data load/reload
• Unified XML API between XDK and XML DB
• Unified C interfaces

14
XML-based Integration XQuery

• Why XQuery ?
• Declarative way to query XML documents
• Why Java?
• Run in mid-tier or database
• Future server implementation in C
• Why XML Database ?
• Native XML storage
• XML data management
• Performance optimizations
• SQL/XML or XQuery depending on data
• Status
• OTN downloads (pending W3C standard finalization
  in 04)

iAS J2EETM Platform
XML DB
Server JVM
15
XQuery Example

• Assume a document emp.xml
• ltempsetgt
• ltemp empno21 enameSCOTT
  salary120000/gt
• ltemp empno22 enameJONES
  salary344000/gt
• lt/empsetgt
• To get the names of employees with salary gt
  200000
• for i in document(emp.xml)/empset
• let j 200000
• where i/_at_salary gt j
• return i/_at_ename
• Result (attribute node)
• JONES

16
Differences from SQL

• Navigation-oriented (using XPath expressions)
• Different type system (XMLSchema based simple
  types)
• Identity-based (XML Node identities and document
  order)
• Namespace aware name-resolution (functions,
  variables, element creation)
• Row based versus Item based
• Results are heterogeneous sequences
• Does not have all SQL extensions (e.g, OLAP,
  Full-Text..)

17
Oracle XQuery API

• JXQI Java API (ongoing standards discussions)
• import oracle.xquery
• XQueryContext ctx new XQuerycontext()
• Reader strm new FileReader(exmpl1.xml)
• XQueryPreparedStatement
• xq ctx.prepareStatement(strm)
• XQueryResultSet rset xq.executeQuery()
• while (rset.next())
• rset.getNode().print(System.out)
• XQLPlus tool! (like SQLPlus)

18
Datasources

• Enables arbitrary input sources
• files, cache, JCA datasources
• xmldatasrc Oracle language addition
• Datasource API
• initialize
• describe
• execute
• Fetch
• Bind (an existing DOM)

19
Rewrite to SQL

• XQuery over Oracle databases Rewrite!
• for i in view(scott.emp)/ROW
• where i/SALARY gt 200000
• return i/ENAME
• -- is translated to ---
• select i.ename
• from scott.emp i
• where i.salary gt 200000

20
More SQL rewrite
for i in view(purchaseOrder)/ROW/PurchaseOrder
where i/ShipAddr/City San Francisco return
ltPO ponumi/_at_Poidgt lti/ShipAddrgt lt/POgt
select xmlelement(PO, XMLAttributes(extractv
alue(i,/PurchaseOrder/_at_Poid) as ponum)),
extract(i, /PurchaseOrder/ShipAddr)) f
rom scott.purchaseorder i where
extractvalue(i, /PurchaseOrder/ShipAddr/City
)

San Francisco
21
D E M O N S T R A T I O N
XQuery
22
Oracle Text

• Rich Full-Text Capabilities built into the Oracle
  database
• Integrated Search support for Applications
• OCS, Portal, Ebusiness Suite
• Catalog Search
• Document Archives and Warehouses
• Infrastructure for Intranet and Extranet Search
  (via Ultra Search.)

23
Oracle Text Rich Full-Text
24
10g Whats new in Oracle Text?

• Supervised Classification Rule-based and SVM
• Unsupervised Classification (Clustering) KMeans
  and Hierarchical
• Query-Log Analysis
• Query-Templating for Progressive-Relaxation,
  Query-rewriting, Alternative scoring etc.
• Index creation improvements -- Real-time
  synchronization
• Better Partitioning Create local-partitioned
  indexes in parallel
• Filtering enhancements
• Filter and index RFC-822 email messages
• Language Enhancements
• Japanese stemming, Customization of Japanese
  Chinese Lexicons
• Information Visualization Stretch viewer

25
Oracle Ultra Search

• Out-of-the-box heterogeneous search-and-locate
  capabilities
• DB, Web Servers, Files, E-Mail, Apps
• High performance threaded Java crawlers
• Web-style interface
• Extensible, customizable (Java API)
• Customizable metadata search
• Custom crawling
• Custom rendering
• Integrated administration
• Fully multilingual and globalized
• Integrated with Oracle Portal (repository,
  portlet) and Oracle Collaboration Suite

26
(No Transcript)
27
10g Whats new in Ultra Search?

• Enhanced Security
• Secure Crawling (https support)
• Better Authentication
• http Digest and Forms
• ACL-secured search hitlist
• Role-based ACLs per datasource
• Or custom ACLs stamped by crawler
• Federated Search
• JCA-compliant Searchlet API
• Unified Search
• Secure Crawler API
• OID Integration

28
D E M O N S T R A T I O N
Information Visualization
29
The Media-enabled Oracle Platform

• Oracle Database 10g
• Storage, management, retrieval of image, audio,
  video data
• Native format understanding, metadata extraction,
  methods for image processing
• Support for leading streaming media servers
• Oracle Application Server 10g
• JSP, servlet and PL/SQL application development
  support
• Media Adaptation Services for Wireless
• JDeveloper (BC4J/UIX) and Portal integration
• Oracle Collaboration Suite
• Metadata extraction for OCS Files

30
New Oracle10g Multimedia Features

• Standards Support SQL/MM Still Image
• New version of Java Advanced Imaging (JAI
  1.1.1_01) and additional image processing
  operators
• Support for additional media formats
• Microsoft ASF, MPEG2 MPEG4
• Microsoft Windows Media Server Plugin
• Real Server Plugin for Helix Server
• XML DB integration

31
How Oracles Multimedia capabilites are better

• Only Oracle10g
• Supports media content natively
• No manual initiation of separate processes to
  enable database tablespace to accept media data.
  
• No need for DBAs to initiate these processes for
  each table where they wish to store media data
• Stores all media and its metadata in the same
  table as the associated relational data
• No triggers on each and every media object
  created to update the separate administration
  tables that contain media objects and metadata.
• No added processing and I/O overhead for access
  and retrieval
• Provides Java class libraries and JSP Tag
  libraries for application development and media
  access.

32
Oracle is the Leading Spatial Database

• In repeated surveys, IDC has found that Oracle
  is used in an 80-90 share of Spatial
  Information Management oriented database
  installations.
• IDC, December 2002
• Oracle 10g Locator feature Beginning with
  Oracle9i LOCATION capabilities have been part of
  EVERY database at NO ADDITIONAL COST
• Enables business, web and LBS applications
• Oracle Spatial 10g Enterprise Edition Option
• Supports advanced Land Management, GIS,
  Transportation,Energy / Utilities, Remote
  Sensing, Defense and Intelligence applications

33
Oracle10g Location Features

• Locator
• Points, lines, polygons
• 2D, 3D, 4D data
• Spatial Operators
• Distance
• Relationships
• Coordinate Systems
• Long Transactions
• Table Partitioning
• Object Replication
• Parallel Query NEW!
• Deferred Spatial Indexes NEW!

• Spatial (Enterprise Option)
• All Locator features
• Spatial functions
• area/length calculation
• buffer, centroid, intersection, union, etc.
• Linear Referencing
• Spatial Aggregates
• Coordinate Transforms
• GeoRaster NEW!
• Topology Data Model NEW!
• Network Data Model NEW!
• GeoCoder NEW!
• Spatial Data Analysis Mining NEW!

Requires Enterprise Edition with Partitioning
Option Some replication features on Enterprise
Ed. only
34
Location features in the Oracle Stack
Any device
CRM ERP Applications
TCA schema
Web Services
e-Business Suite
Application Server
B2B, B2E, B2C
iAS MapViewer / JDeveloper
iAS LBS Components
SOAP, WSDL
Oracle Application Server 10g
Data Server
Online Service
Spatial
Locator
Oracle Database 10g
Oracle Location Technology
Oracle core technologies
35
Oracles Extensibility Framework

• Open API to plug in new data types and access
  methods
• Specialty Data Types
• Chemical
• Genetic
• Engineering
• Biometric
• Multimedia
• Driven by specialized-domain ISVs --MDL, NetGene,
  Informax, Protegrity,

36
Extensibility In Silico Chemistry

• Chemistry searching requires special techniques
• Chemical name is not unique

Viagra
sildenafil citrate

• The solution
• A graphical search engine
• Specialized operators such as substructure
  search (sss) a chemical contains

37
Oracle Collaboration Suite

• Consolidate management of unstructured data
  (email, shared documents and other collaborative
  content)
• Before grid computing, resources such as storage
  and CPUs had to be managed separately for each
  component of the suite (e.g. email vs files vs
  web conferencing).
• OCS 10g takes advantage of grid infrastructure
  for greater efficiency, reduced cost and easier
  management

38
Extended Data Management
Oracle Collaboration Suite, Oracle Portal,
eBusiness Suite provide solutions

• Ultra Search crawls and (where desirable)
  federates non-Oracle or legacy sources, and bring
  these in the ambit of uniform access
• Search, Interchange, Visualization
• Analytics and Mining

• Oracle provides the most robust open and
  extensible platform and the important services
  for all your data
• Storage and Management
• Search, Interchange, Visualization
• Analytics and Mining

• Structured data will stay Relational
• Documents Messages will move to XML
• Multimedia will be in BLOBs, with metadata
  annotated in XML

39
A
40
(No Transcript)