Panel on New Research Directions in KDD

1
Panel onNew Research Directionsin KDD
Disclaimer Views are My Own, not necessarily
those of DARPA, Department of Defense, or the US
Government
Ted E. Senator 703-696-2231 tsenator_at_darpa.mil
2
Some Important (Research ?) Problems

• Structured/Linked Data (much more than structural
  models)
• Examples Human Organizations (formal or
  informal), (Bio)Chemistry, WWW
• Characteristics
• Non-Homogeneous Entities and Links
  Non-Independent Examples
• Significance of Connected Components (i.e.,
  network fragments) is ApparentOnly When
  Entities/Events are Linked
• Rare but Important Events Micro-Mining
• Structural Understanding Explore Space of
  Features
• Continuous Knowledge Discovery and Management
• Move from Project Orientation to Process
  Orientation
• Institutionalize Use and Management of Discovered
  Knowledge
• Dynamic, Ongoing Domains Problem Evolution
• Repeatable, Automatable ? Autonomous/On-Line
• Tight Coupling with Database/DataWarehouse
• Metrics and Measures of Effectiveness (vice
  Measures of Performance)
• Increased Statistical Rigor/Foundations
• Use of Domain/Background Knowledge
• Innovative Combination of Techniques for Data
  Exploration

3
Problems Receiving Too Much Attention

• Algorithm (Fine) Tuning and Extensions
• A New Decision Tree, Association Rule,
  Clustering Algorithm for
• Research Masquerading as Applications
• Bicycles Have Two Wheels Results
• Look Ma, No Hands Results
• Claims of Look Ma, No Wheels Results
• Knowledge-Free Data Mining

4
Potential KDD Applications and Contributions

• NOT Marketing or Customer Profiling,Perhaps
  Personalization if Privacy Can Be Maintained
• Support for Strategic Business Decisions
• New Products
• New Markets
• New Processes
• Policy Formulation

5
Commodity Software ?

• Is This Really a Good Thing ? Is it What We (KDD
  Community) Want ?
• Commodity low-cost producers of standard
  product wins
• High-End versus Low-End
• Are More Powerful Tools More Easily Misused ? And
  Results Misinterpreted ?
• Barriers
• Recognized Advantages overcome Costs
• Alternative Strategies
• Results of Ongoing Data Mining Competitive
  Advantage for Information Services Providers
• Domain-Specific Analytical Techniques
• Will Fate of KDD Be Similar to That of Expert
  Systems ?
• Hype ? Disillusionment
• Techniques Embodied in Widely-Used Database
  Technology,or in Enterprise Management Software
• PredictionLess Capable Add-Ons will Dominate
  More Capable Stand-Alone Applications at both Low
  End (e.g., Excel) and High End (e.g., Oracle)