Database Design Goals

1
Database Design Goals

• Match Structure with Algorithms
• Retain Original Data
• Store Intermediate Results
• Efficient Processing Storage
• Intuitive Reporting and Analysis

2
Database Bottlenecks

• Temporary Table Creation
• Data Replication
• Complex Primary and Foreign Keys
• Missing Indexes
• Ambiguous Indexing
• Cartesian Joins
• Small Repetitive Inserts

3
Coding Issues

• PL/pgSQL Stored Procedures
• Standard Format for SQL Code
• Explicit Table Join Format
• Index Specification (If Ambiguous)
• Kickoff Script (perl)
• Input Data Using copy Via psql
• Execution Master in PL/pgSQL
• Execution Logs Status

4
Execution Process

• Verify Run Parameters Files (perl)
• Execute Imports (perl)
• Kick Off Execution Master (perl)
• QA Inputs
• Execute Steps
• Status Tables ? Monitoring Application
• Restart Capability