Jonathan Robie

1
Jonathan Robie

• XQuery Technology Lead
• DataDirect Technologies

2
DataDirect XQuery 2.0
3
Data Integration without DataDirect XQuery
Java/JSP codeaccessing the variousJava APIs and
generating the HTML report
SOAP through AXIS
Web Browser
dBASE IV APIs
JDBC
4
The DataDirect XQuery Solution
Java/JSP
5
Dealing with multiple, heterogeneous Data
SourcesA dangerous approach
Data Access Layer
6
The XQuery Vision
XQuery
7
Querying XML and Relational the same way!

• for h in doc("holdings.xml")/holdings/entryfor
  c in collection("companies")/companieswhere
  h/userid "Minollo"   and c/ticker
  h/stocktickerreturn  ltcompany ticker"
  c/ticker "gt      c/companyname      
  c/annualrevenues   lt/companygt

8
XQuery Data Integration? DataDirect!

• In-memory XQuery implementations
• Only for physical XML
• Database XQuery implementations
• Only for their database
• Relational Database XQuery implementations
• XML Database XQuery implementations
• Full-text XQuery implementations
• Server-based XQuery implementations
• Only for their server
• Application Server
• Integration Server
• DataDirect XQuery is Open!
• All leading relational databases
• XML
• Many other data sources
• Any app server or none

9
DataDirect XQuery

• Accesses almost any data source
• No dependency on servers
• Standards-based
• High performance
• Scalable
• Embeddable
• Plugs into any Java architecture supports
  XQuery for Java API (XQJ)

10
What is XQJ?

• XQuery API for Java (XQJ) JSR 225
• The JDBC for XQuery

11
DataDirect XQuery Input, Output

• Relational Databases
• SQL Server 2000, 2005
• Oracle 9i, 10gR1, 10gR2
• DB2 Windows/UNIX/Linux v8.x, v9
• DB2 iSeries V5R2, V5R3
• DB2 z/OS  v8
• Sybase  12.5.x, 15
• XML sources
• http ftp and file schemes
• XML in DOM trees
• Drastically reduced memory requirements for large
  XML documents.
• Custom URI Resolvers
• Legacy sources
• Converters for CSV, EDI, and more
• SOA Integration
• Web Service calls

• XML Output
• XML as text
• XML in DOM trees
• XML in SAX streams
• XML in StAX streams

12
XML Processing without XQuery

• lt?xml version"1.0" encoding"UTF-8"?gtltsoapEnvel
  ope xmlnssoap"http//schemas.xmlsoap.org/soap/en
  velope/" xmlnsxsi"http//www.w3.org/2001/XMLSche
  ma-instance" xmlnsxsd"http//www.w3.org/2001/XML
  Schema"gt  ltsoapBodygt    ltGetQuotesResponse
  xmlns"http//swanandmokashi.com"gt      ltGetQuote
  sResultgt        ltQuotegt          ltCompanyNamegtAP
  PLE COMPUTERlt/CompanyNamegt          ltStockTickergt
  AAPLlt/StockTickergt          ltStockQuotegt74.17lt/St
  ockQuotegt          ltLastUpdatedgt9/14/2006
  401pmlt/LastUpdatedgt          ltChangegt1.17lt/Chang
  egt          ltPercentChangegt1.82lt/PercentChangegt
            ltOpenPricegtN/Alt/OpenPricegt          ltDa
  yHighPricegtN/Alt/DayHighPricegt          ltDayLowPri
  cegtN/Alt/DayLowPricegt          ltVolumegt0lt/Volumegt
            ltMarketCapgt63.266Blt/MarketCapgt         
   ltYearRangegt47.87 - 86.40lt/YearRangegt          ltE
  xDividendDategt21-Nov-95lt/ExDividendDategt         
   ltDividendYieldgtN/Alt/DividendYieldgt          ltDiv
  idendPerSharegt0.00lt/DividendPerSharegt        lt/Qu
  otegt      lt/GetQuotesResultgt    lt/GetQuotesRespo
  nsegt  lt/soapBodygtlt/soapEnvelopegt

13
XML Processing without XQuery

• lt?xml version"1.0" encoding"UTF-8"?gtltsoapEnvel
  ope xmlnssoap"http//schemas.xmlsoap.org/soap/en
  velope/" xmlnsxsi"http//www.w3.org/2001/XMLSche
  ma-instance" xmlnsxsd"http//www.w3.org/2001/XML
  Schema"gt  ltsoapBodygt    ltGetQuotesResponse
  xmlns"http//swanandmokashi.com"gt      ltGetQuote
  sResultgt        ltQuotegt          ltCompanyNamegtAP
  PLE COMPUTERlt/CompanyNamegt          ltStockTickergt
  AAPLlt/StockTickergt          ltStockQuotegt74.17lt/St
  ockQuotegt          ltLastUpdatedgt9/14/2006
  401pmlt/LastUpdatedgt          ltChangegt1.17lt/Chang
  egt          ltPercentChangegt1.82lt/PercentChangegt
            ltOpenPricegtN/Alt/OpenPricegt          ltDa
  yHighPricegtN/Alt/DayHighPricegt          ltDayLowPri
  cegtN/Alt/DayLowPricegt          ltVolumegt0lt/Volumegt
            ltMarketCapgt63.266Blt/MarketCapgt         
   ltYearRangegt47.87 - 86.40lt/YearRangegt          ltE
  xDividendDategt21-Nov-95lt/ExDividendDategt         
   ltDividendYieldgtN/Alt/DividendYieldgt          ltDiv
  idendPerSharegt0.00lt/DividendPerSharegt        lt/Qu
  otegt      lt/GetQuotesResultgt    lt/GetQuotesRespo
  nsegt  lt/soapBodygtlt/soapEnvelopegt

• Without XQuery
• Parse XML
• Navigate (DOM) or Capture Events (SAX)
• Cast XML values to Java values
• Create XML for output
• With XQuery
• declare default element namespace
  "http//swanandmokashi.com"for q in
  doc("aapl.xml")//Quotewhere q/PercentChange gt
  1return  ltQuotegt     q/StockTicker     
  q/StockQuote      q/PercentChange
    lt/Quotegt

14
DataDirect XQuery is fast for XML files!

• lt?xml version"1.0" encoding"UTF-8"?gtltsoapEnvel
  ope xmlnssoap"http//schemas.xmlsoap.org/soap/en
  velope/" xmlnsxsi"http//www.w3.org/2001/XMLSche
  ma-instance" xmlnsxsd"http//www.w3.org/2001/XML
  Schema"gt  ltsoapBodygt    ltGetQuotesResponse
  xmlns"http//swanandmokashi.com"gt      ltGetQuote
  sResultgt        ltQuotegt          ltCompanyNamegtAP
  PLE COMPUTERlt/CompanyNamegt          ltStockTickergt
  AAPLlt/StockTickergt          ltStockQuotegt74.17lt/St
  ockQuotegt          ltLastUpdatedgt9/14/2006
  401pmlt/LastUpdatedgt          ltChangegt1.17lt/Chang
  egt          ltPercentChangegt1.82lt/PercentChangegt
            ltOpenPricegtN/Alt/OpenPricegt          ltDa
  yHighPricegtN/Alt/DayHighPricegt          ltDayLowPri
  cegtN/Alt/DayLowPricegt          ltVolumegt0lt/Volumegt
            ltMarketCapgt63.266Blt/MarketCapgt         
   ltYearRangegt47.87 - 86.40lt/YearRangegt          ltE
  xDividendDategt21-Nov-95lt/ExDividendDategt         
   ltDividendYieldgtN/Alt/DividendYieldgt          ltDiv
  idendPerSharegt0.00lt/DividendPerSharegt        lt/Qu
  otegt      lt/GetQuotesResultgt    lt/GetQuotesRespo
  nsegt  lt/soapBodygtlt/soapEnvelopegt

• General XQuery rewrites
• Elimination of common sub-expressions, loop
  rewrites, ordering rewrites, etc
• Document projection
• XML construction accounts for much of the cost
• Dont build parts of the document that the query
  doesnt need!
• Document streaming
• Discard parts of the document when no longer
  needed
• Makes memory usage near constant with size of
  file
• Multiple Gigabytes can be queried

15
DataDirect XQuery for Relational Data
ltportfolio gt    ltcompany ticker"AMZN"gt        lt
companynamegtAmazon.com, Inc.lt/companynamegt       
 ltannualrevenuesgt7780lt/annualrevenuesgt    lt/compa
nygt    ltcompany ticker"EBAY"gt        ltcompanyna
megteBay Inc.lt/companynamegt        ltannualrevenues
gt22600lt/annualrevenuesgt    lt/companygt    ltcompan
y ticker"IBM"gt        ltcompanynamegtInt'l
Business Machines Clt/companynamegt        ltannualr
evenuesgt128200lt/annualrevenuesgt    lt/companygt   
 ltcompany ticker"PRGS"gt        ltcompanynamegtProg
ress Softwarelt/companynamegt        ltannualrevenue
sgt493.4lt/annualrevenuesgt    lt/companygtlt/portfoli
ogt
16
Database Publishing without DataDirect XQuery

• Without XQuery
• Use SQL for Relational Data
• Use JDBC for database connection
• Use Java XML API to build XML hierarchies from
  flat result sets
• Use XSLT for transformations
• With XQuery
• for h in collection("HOLDINGS")/holdingsfor
  c in collection("COMPANIES")/companieswhere
  h/ticker c/ticker  and h/userid
  "Jonathan"return  ltportfoliogt    ltcompany
  ticker"c/ticker"gt      ltcompanynamegtstring(
  c/name)lt/companynamegt      ltannualrevenuesgtstri
  ng(c/annualrevenues)lt/annualrevenuesgt    lt/comp
  anygt  lt/portfoliogt

ltportfolio gt    ltcompany ticker"AMZN"gt        lt
companynamegtAmazon.com, Inc.lt/companynamegt       
 ltannualrevenuesgt7780lt/annualrevenuesgt    lt/compa
nygt    ltcompany ticker"EBAY"gt        ltcompanyna
megteBay Inc.lt/companynamegt        ltannualrevenues
gt22600lt/annualrevenuesgt    lt/companygt    ltcompan
y ticker"IBM"gt        ltcompanynamegtInt'l
Business Machines Clt/companynamegt        ltannualr
evenuesgt128200lt/annualrevenuesgt    lt/companygt   
 ltcompany ticker"PRGS"gt        ltcompanynamegtProg
ress Softwarelt/companynamegt        ltannualrevenue
sgt493.4lt/annualrevenuesgt    lt/companygtlt/portfoli
ogt
17
DataDirect XQuery is fast for Relational Data!
ltportfolio gt    ltcompany ticker"AMZN"gt        lt
companynamegtAmazon.com, Inc.lt/companynamegt       
 ltannualrevenuesgt7780lt/annualrevenuesgt    lt/compa
nygt    ltcompany ticker"EBAY"gt        ltcompanyna
megteBay Inc.lt/companynamegt        ltannualrevenues
gt22600lt/annualrevenuesgt    lt/companygt    ltcompan
y ticker"IBM"gt        ltcompanynamegtInt'l
Business Machines Clt/companynamegt        ltannualr
evenuesgt128200lt/annualrevenuesgt    lt/companygt   
 ltcompany ticker"PRGS"gt        ltcompanynamegtProg
ress Softwarelt/companynamegt        ltannualrevenue
sgt493.4lt/annualrevenuesgt    lt/companygtlt/portfoli
ogt

• Highly optimized for relational sources
• Minimizes retrieval of data
• No more rows than needed
• No more columns than needed
• Uses database functionality
• Joins
• Sorting
• Etc..
• Optimizes for each SQL dialect
• Efficient JDBC retrieval
• Embeds DataDirect JDBC technology
• Optimizations added to support XQuery
• Supports incremental retrieval
• Optimizes for XML hierarchies
• Sort-merge algorithm
• Minimal cost of XML construction
• Leverages SQL library
• Recognizes equivalences
• Supports hints

18
XML Converters
EDI File ISA00DATADIRECT00STYLUS200601DATA
DIRECT 01STYLUS STUDIO 06050412120050
32006543210I' GSBFDATADIRECTSTYLUS2006200
60504121212256X005030' ST1053389' BGN28102
420060504121212GM' NM12L4Progress Software
Corporation' N314 Oak Park Drive' N4BedfordMA0
1730USAA' REF1ZPRGS' NM12L4Apple Computer,
Inc.' N31 Infinite Loop' N4CupertinoCA95014US
AA' REF1ZAAPL' SE113389' GE1256' IEA12006
54321'
19
XML Converters
EDI File ISA00DATADIRECT00STYLUS200601DATA
DIRECT 01STYLUS STUDIO 06050412120050
32006543210I' GSBFDATADIRECTSTYLUS2006200
60504121212256X005030' ST1053389' BGN28102
420060504121212GM' NM12L4Progress Software
Corporation' N314 Oak Park Drive' N4BedfordMA0
1730USAA' REF1ZPRGS' NM12L4Apple Computer,
Inc.' N31 Infinite Loop' N4CupertinoCA95014US
AA' REF1ZAAPL' SE113389' GE1256' IEA12006
54321'
doc("adapter//EDI?ticker-request.edi") ltX12gt   
 ltISAgt        ltISA01gtlt!--I01 Authorization
Information Qualifier--gt00lt!--No Authorization
Information Present (No Meaningful Information in
I02)--gtlt/ISA01gt        ltISA02gtlt!--I02
Authorization Information--gtDATADIRECTlt/ISA02gt   
     ltISA03gtlt!--I03 Security Information
Qualifier--gt00lt!--No Security Information Present
(No Meaningful Information in I04)--gtlt/ISA03gt    
    ltISA04gtlt!--I04 Security Information--gtSTYLUS2
006lt/ISA04gt        ltISA05gtlt!--I05 Interchange
ID Qualifier--gt01lt!--Duns (Dun amp
Bradstreet)--gtlt/ISA05gt        ltISA06gtlt!--I06
Interchange Sender ID--gtDATA DIRECT
lt/ISA06gt        ltISA07gtlt!--I05 Interchange ID
Qualifier--gt01lt!--Duns (Dun amp
Bradstreet)--gtlt/ISA07gt        ltISA08gtlt!--I07
Interchange Receiver ID--gtSTYLUS STUDIO
lt/ISA08gt        ltISA09gtlt!--I08 Interchange
Date--gt060504lt!--2006-05-04--gtlt/ISA09gt        ltIS
A10gtlt!--I09 Interchange Time--gt1212lt/ISA10gt     
   ltISA11gtlt!--I65 Repetition Separator--gtlt/ISA11
gt        ltISA12gtlt!--I11 Interchange Control
Version Number--gt00503lt!--Standards Approved for
Publication by ASC X12 Procedures Review Board
through October 2005--gtlt/ISA12gt        ltISA13gtlt!-
-I12 Interchange Control Number--gt200654321lt/ISA1
3gt        ltISA14gtlt!--I13 Acknowledgment
Requested--gt0lt!--No Interchange Acknowledgment
Requested--gtlt/ISA14gt        ltISA15gtlt!--I14
Interchange Usage Indicator--gtIlt!--Information--gtlt
/ISA15gt        ltISA16gtlt!--I15 Component Element
Separator--gtlt/ISA16gt    lt/ISAgt    ltGSgt        
ltGS01gtlt!--479 Functional Identifier
Code--gtBFlt!--Business Entity Filings
(105)--gtlt/GS01gt        ltGS02gtlt!--142
Application Sender's Code--gtDATADIRECTlt/GS02gt    
    ltGS03gtlt!--124 Application Receiver's
Code--gtSTYLUS2006lt/GS03gt        ltGS04gtlt!--373
Date--gt20060504lt!--2006-05-04--gtlt/GS04gt        ltG
S05gtlt!--337 Time--gt121212lt/GS05gt        ltGS06gtlt!
--28 Group Control Number--gt256lt/GS06gt        ltG
S07gtlt!--455 Responsible Agency
Code--gtXlt!--Accredited Standards Committee
X12--gtlt/GS07gt        ltGS08gtlt!--480 Version /
Release / Industry Identifier Code--gt005030lt!--Sta
ndards Approved for Publication by ASC X12
Procedures Review Board through October
2005--gtlt/GS08gt    lt/GSgt
20
SOA - Web Services
declare function localamazon-listing(isbn)   
 lttnsRequestgt      lttnsConditiongtAlllt/tnsCondi
tiongt      lttnsDeliveryMethodgtShiplt/tnsDelivery
Methodgt      lttnsFutureLaunchDate/gt      lttnsI
dTypegtASINlt/tnsIdTypegt      lttnsItemIdgt isbn
lt/tnsItemIdgt      lttnsResponseGroupgtMediumlt/tn
sResponseGroupgt    lt/tnsRequestgt let loc
ltlocation address"http//soap.amazon.com/onca/
soap?ServiceAWSECommerceService"
soapaction"http//soap.amazon.com" /gtlet
payload localamazon-listing("0395518482")ret
urn wscall(loc, payload)
21
DataDirect Stylus Studio IDE

• DataDirect Stylus Studio
• General-purpose XML IDE
• XQuery Editor
• Ad-hoc queries
• Database Connections Window
• XQuery Mapper
• XQuery Debugger
• XML Publisher
• XML Pipeline

22
ltoXygen/gt plugin for Eclipse (DataDirect Edition)

• ltoXygen/gt plugin for Eclipse (DataDirect
  Edition)
• Eclipse-based
• XQuery Editor
• Ad-hoc queries
• Database Connections Window
• XQuery Perspective

23
Why DataDirect XQuery?

• Performance
• Generates highly optimized SQL code for
  relational databases
• Excellent performance for very large XML files
• Programs written with DataDirect XQuery generally
  outperform programs written with Java XML APIs
  JDBC
• Programmer productivity
• Using XQuery means 7 to 20 times less code
• Using XQuery means code is easier to read and
  maintain
• XQuery IDEs increase productivity
• Support for most common data sources
• XML, Relational, Flat File, EDI, Web Service
  calls
• Extensible via Java
• Fits in almost any architecture
• Supports all leading relational databases
• Easily embeddable - does not require any server
• Supports any J2EE Application Server
• Runs on any Java platform

24
Getting Started

• www.xquery.com
• Free Trial Download
• Tutorials
• XQuery
• XQJ
• DataDirect XQuery
• Support Forums