The MegaStore Application

1
Databases and Web-Based Applications
The MegaStore Application Advanced Internet-based
Electronic Commerce Technologies for Music
Industry
Ammar Benabdelkader University of Amsterdam
December 1, 2003 UvA, Amsterdam
2
Motivation

• Large number of emerging e-commerce applications.
  These applications comprise large multimedia
  data, for which
• Data classification and cataloguing are required
• Related information needs to be properly
  inter-linked
• Efficient storage, and access of large multimedia
  data is supported
• Short response time for on-line requests must be
  guaranteed
• High data transfer rates must be provided
• Furthermore, public information must be separated
  from the private information that need to be
  securely kept at the place where it belongs.

3
Approach

• An efficient architecture must be designed
  addressing the need to provide the user of
  electronic commerce with an environment through
  which he can experience as sufficiently close to
  real life shopping environment. The approach
  considers the following aspects
• Security for data access and users authentication
• Suitable user friendly interfaces
• Performance issues must be addressed
• Private data need to be protected
• Public data is made available to the Internet
  users
• The MegaStore system aims at the design and
  set-up of the necessary database structure and
  platform architecture for advanced e-commerce
  applications

4
Problem Analysis

• From the analysis of music industry application,
  the following aspects are identified
• The data is geographically distributed over the
  network
• Information about music is classified into two
  main categories
• The general information stored at the directory
  services (database catalogue) and accessible by
  all the Internet users
• The raw music data that can only be accessible by
  the music storekeepers at music centers or
  burning towers
• Depending on the user profile and authorization,
  only a part of the information can be accessed
  and users need not to know about the data
  distribution
• The real music data must be securely transferred
  through a dedicated network
• High bandwidth connection is necessary to handle
  raw music data that need to be transferred
  between the music storage centers and the burning
  towers
• Low latency network connection is necessary to
  support the huge number of users expected to
  connect to the system

5
Database Design
6
The MegaStore System Architecture (1)

• The designed system architecture involves the
  following components
• The back-end system, including the database
  engine and the predefined networking connection
  between the MegaStore system components
• The front-end system, including
• The InternetShop interface, where a user from
  home (or work place) can search for music,
  listen/watch to the audio/video clips, and order
  CDs, and
• The Shop-in-a-Shop interface, where the music
  storekeeper can fetch on-line the real music data
  from its original source in order to burn at
  run-time the requested music CDs

7
The MegaStore System Architecture (2)
Shop-in-a-shop Server
Parallel Distributed Database Server
Internet
I n t r a n e t
Internet
Shop-in-a-shop Interface
Shop-in-a-shop Interface
8
Activity Diagram for the Internet-Shop Interface
9
Internet Web Interface (1)
10
Internet Web Interface (2)
(a) Song Search
(b) Album Search
11
Dynamic Browsing of Interface
(A) Artist Songs(B) Album Songs
(B)
(A)
12
MegaStore Ordering System
State Diagram for Orders
(a)Standard Order
(b)Custom Order
13
System Implementation

• The database schema is implemented on top of the
  Matisse object-oriented database system
• The Matisse back-end database runs on a cluster
  composed of 20 nodes.
• The MegaStore Internet interface is implemented
  using an NT front-end machine, that is in turn
  connected to the underlying back-end database.
• The Internet-shop server prototype is implemented
  using a combination of the following software
  technologies
• JavaScript and Vbscript for tips programming,
• Active Database Objects (ADO) for database
  access, and
• HTML for text formatting
• the server implementation is made possible using
  the Active Server Pages (ASP) environment that
  allows the combination of different software
  technologies in one single environment.

14
Information Management Strategy
F- C
F- A
F- B
Legacy systems (Flat-files and Databases)
DBA Program
Database access - ODL - OIF
Adapter
Adapter
Adapter
C
Access independent Framework
ODBMS
JAVA
Facilitating Applications - ODBC - XML - JDBC,
Java
SQL
User Interfaces
SQL3
Object-oriented

• Databases
• ODL is used to support the portability of
  database schemas across conforming ODBMSs.
• OIF is used to exchange objects between
  databases and provide database documentation.
• Applications
• Universal data access through standards and
  middle ware solutions (XML, ODBC, JDBC, JAVA,
  etc.).

15
Management of Large Multi-Media Data

• An efficient architecture must be designed for
  manipulation of large data sets. Therefore, Data
  management mechanisms must be addressed in such a
  way that
• Data needs to be properly searched, retrieved,
  published, inter-linked, and compared to other
  data,
• Information security is preserved,
• Performance issues are improved,
• Private data is protected, while published data
  is made available to the outside users.

16
Data Storage Approaches

• File System Approach and HFS (Hierarchical File
  System)
• External Data Link Approach
• database catalogue refers to NFS (network file
  system)
• One-Database Storage Approach
• Parallel/Distributed Database Server
• etc.

17
1- File System Approach

• Inefficient in
• Maintaining the link between the inter-related
  pieces of information,
• Comparing related data in different applications,
• Supporting the ability to query and modify the
  data using appropriate query languages,
• Searching the stored information and supporting
  efficient access to data items,
• Preserving the system coherency, data is
  scattered in various files of different formats.
• Applications are based on direct access to local
  file system they are hard to maintain and to
  extend.

Disk
18
2- External Data Link Approach

• A database catalogue is used together with the
  file system
• Database provides references to all objects
  stored locally or remotely
• This approach solves problems related to database
  overload and improves DB performance
• Data is distributed so that it is physically
  located closest to intensive usage sites
• Result files can be archived at (or close to) the
  point where they are generated
• Proper data distribution reduces access
  bottlenecks at individual sites

Link
Disk
19
2- External Data Link Approach (Cont.)

• Two problems are faced when using this approach
• Database catalogue consistency referenced binary
  objects can be updated/removed without notifying
  the database catalogue maintainer
• Solution a specific module that automatically
  and periodically checks the availability of the
  referenced objects against the database catalogue
• Security issue referenced objects, which are
  usually stored in a public location, are not
  secure.
• Solution development of a remote file server,
  through which, the file access and user
  authentication are controlled based on the
  database catalogue information

20
3- One-Database Storage Solution

• In ideal case, if a database is being created to
  manage the meta-data, then that database must
  also store the large scientific data
• This approach solves the problem of keeping the
  meta-data synchronized with the archive
• However, since the scientific binary data is of
  huge size
• It is very costly to store/access the data as
  large binary objects within the database itself
• Access mechanisms to binary objects require extra
  encoding/decoding facilities

Database
21
4- Parallel/Distributed Database Server

• This approach uses a database repository to store
  the general information, and a distributed
  database server to store large objects
• The database repository is better exploited for
  cataloguing, indexing, and searching facilities
• The database server enforces the issues related
  to security for access, concurrency control, and
  information visibility rights.

User Queries
Query Processor
Transaction Manager
Storage Manager
DBMS Software
Repository Metadata
Database Server
Database System
22
Parallel/Distributed Database Server Example
23
Distributed Parallel Server Extension

• The parallel/distributed database framework
  provides the MegaStore web server with efficient
  access to the raw music data. The nodes (music
  stores) of the distributed MegaStore server are
  inter-connected, making it possible for specific
  users to connect to any node in the distributed
  server and to request an object, without the need
  to know where that object actually resides
• The distributed database supports the following
  required functionalities
• Provides a way for managing huge amount of data
• Data is securely kept at geographically
  distributed music centers
• Data is stored only at the point(s) where it
  belongs
• Data is visible from any node (music center)
  within the cooperation community
• Data is efficiently transferred between the nodes
  in short response time

24
The MegaStore Prototypes - The FRS Application -
The LuisterPaal Interface - Sheet Music Server
25
Achievement

• Design and development of an e-MegaStore
  application that seamlessly fits several emerging
  applications and supports their model of
  operations
• Design Methodology
• A conceptual model
• A computer system architecture
• An Implementation
• Prototype

26
Conceptual Model
Information Integration and Value Adding
Value Adding Partners
27
System Architecture
Retail Value Adding
Suppliers
Major Content Suppliers
Upload Storage
External Links
Internet
FireWall
28
The FRS-Pilot Application
Music Content
Internet
29
Security for the Multimedia DB
Short Clips Audio
Medium Tracks Audio/Video
High Quality Audio Tracks
Data is Secure but Not Encrypted
Data is Secure and Encrypted
Data is Secure and Highly Encrypted
30
FRS Application Main Interface
31
FRS Application LuisterPaal Interface
32
Sheet Music Application Main Interface
33
Sheet Music Application Subscription Interface
34
Sheet Music Application Music Book Search
Interface
35
Sheet Music Application Composer Search Interface
36
Sheet Music Application Music Notes Example
37
Conclusion

• The main idea behind the developed framework is
  to design a comprehensive system to support
  advanced web-applications with two specific
  characteristics
• to hold large data sets and
• to manage multimedia information
• Thus, the MegaStore system can be considered as a
  general implementation approach that proves the
  validity of the proposed architecture and design.
• From this framework, other applications in
  biology, medicine, and system engineering that
  share the same characteristics can benefit.

38
References

• Personnel Home Page
• http//carol.wins.uva.nl/ammar
• Cooperative Information Management (CO-IM) Home
  Page
• http//carol.wins.uva.nl/netpeer/
• Matisse DBMS
• http//www.matisse.com
• Related Publications
• A. Benabdelkader, H. Afsarmanesh, L. O.
  Hertzberger. MegaStore Advanced Internet-based
  Electronic Commerce Service for Music Industry.
  In proceedings of 11th IEEE International
  Conference on Database and Expert Systems
  Applications - DEXA'2000, Pages 869-878, London -
  Greenwich, United Kingdom, 2000.
• A. Benabdelkader, H. Afsarmanesh, L. O.
  Hertzberger. The Virtual MegaStore System
  Implementation. Technical Report CS-99-05,
  Faculty of Science, Research Institute Computer
  Science, University of Amsterdam, 1999.
• A. Benabdelkader, H. Afsarmanesh, L. O.
  Hertzberger. The Virtual MegaStore System
  Architecture Analysis and Design. Technical
  Report CS-99-04, Faculty of Science, Research
  Institute Computer Science, University of
  Amsterdam, 1999.