Introduction to class

1
Introduction to class
2
Outline

• Introduction to class
• Introduction to machine learning / data mining
• Introduction to the Life Sciences
• Principles of bioinformatics
• Probabilistic framework

3
Introduction to Class

• This class focuses on learning how to apply data
  mining to biological and medical fields to solve
  some of their problems.
• Does not require prior knowledge in the
  application areas.
• Does not require prior knowledge in machine
  learning and/or data mining.

4
Introduction to Class

• Data mining specialized in
• Statistical data analysis and inference SPSS
• Clustering Clementine
• Machine learning Hidden-Markov Models, decision
  trees.
• Classification.
• Requirement use biological datasets and/or
  medical datasets.
• Seattle area has many renowned research
  institutes.

5
Human Genome Program, U.S. Department of Energy,
Genomics and Its Impact on Medicine and Society
A 2001 Primer, 2001
6
The Human Genome Project

• The Human Genome Project

7
Data Mining Motivation Necessity is the Mother
of Invention

• Data explosion problem
• Automated data collection tools and mature
  database technology lead to tremendous amounts of
  data stored in databases, data warehouses and
  other information repositories
• We are drowning in data, but starving for
  knowledge!
• Solution Data warehousing and data mining
• Data warehousing and on-line analytical
  processing
• Extraction of interesting knowledge (rules,
  regularities, patterns, constraints) from data
  in large databases

8
What Is Data Mining?

• Data mining (knowledge discovery in databases)
  
• Extraction of interesting (non-trivial, implicit,
  previously unknown and potentially useful)
  information or patterns from data in large
  databases
• Alternative names and their inside stories
• Data mining a misnomer?
• Knowledge discovery(mining) in databases (KDD),
  knowledge extraction, data/pattern analysis, data
  archeology, data dredging, information
  harvesting, business intelligence, etc.
• What is not data mining?
• (Deductive) query processing.
• Expert systems or small ML/statistical programs
  are often a part of data mining

9
What Is Data Mining?

• Data mining (knowledge discovery in databases) is
  the process of discovering interesting knowledge
  from large amounts of data stored either in
  databases, data warehouses, or other information
  repositories.
• Machine learning and knowledge discovery are
  interested in the process of discovering
  knowledge that may be structurally or
  semantically more complex models, graphs, new
  theorems or theories in particular to assist
  scientific discovery.

10
Why Data Mining? Potential Applications

• Database analysis and decision support
• Market analysis and management
• target marketing, customer relation management,
  market basket analysis, cross selling, market
  segmentation
• Risk analysis and management
• Forecasting, customer retention, improved
  underwriting, quality control, competitive
  analysis
• Fraud detection and management
• Other Applications
• Text mining (news group, email, documents) and
  Web analysis.
• Intelligent query answering.
• Medical decision support.

11
Data Mining A KDD Process
Knowledge
Pattern Evaluation

• Data mining the core of knowledge discovery
  process.

Data Mining
Task-relevant Data
Selection
Data Warehouse
Data Cleaning
Data Integration
Databases
12
Steps of a KDD Process

• Learning the application domain
• relevant prior knowledge and goals of application
• Creating a target data set data selection
• Data cleaning and preprocessing (may take 60 of
  effort!)
• Data reduction and transformation
• Find useful features, dimensionality/variable
  reduction, invariant representation.
• Choosing functions of data mining
• summarization, classification, regression,
  association, clustering.
• Choosing the mining algorithm(s)
• Data mining search for patterns of interest
  (machine learning)
• Pattern evaluation and knowledge presentation
• visualization, transformation, removing redundant
  patterns, etc.
• Use of discovered knowledge

13
Data Mining On What Kind of Data?

• Relational databases
• Data warehouses
• Transactional databases
• Advanced DB and information repositories
• Object-oriented and object-relational databases
• Spatial databases - images
• Time-series data and temporal data, sequence data
• Text databases and multimedia databases
• Heterogeneous and legacy databases
• WWW
• Data streams of sensors
• Structured data networks, graphs
• Spatiotemporal - video

14
Machine Learning Functionalities (1)

• Concept description Characterization and
  discrimination
• Generalize, summarize, and contrast data
  characteristics, e.g., dry vs. wet regions
• Association (correlation and causality)
• Multi-dimensional vs. single-dimensional
  association
• age(X, 20..29) income(X, 20..29K) à buys(X,
  PC) support 2, confidence 60
• contains(T, computer) à contains(x, software)
  1, 75
• Diaper ? Beer 0.5, 75

15
Machine Learning Functionalities (2)

• Classification and Prediction
• Finding models (functions) that describe and
  distinguish classes or concepts for future
  prediction
• E.g., classify countries based on climate, or
  classify cars based on gas mileage
• Presentation decision-tree, classification rule,
  neural network
• Prediction Predict some unknown or missing
  numerical values
• Cluster analysis
• Class label is unknown Group data to form new
  classes, e.g., cluster houses to find
  distribution patterns
• Clustering based on the principle maximizing the
  intra-class similarity and minimizing the
  interclass similarity

16
Machine Learning Functionalities (3)

• Outlier analysis
• Outlier a data object that does not comply with
  the general behavior of the data
• It can be considered as noise or exception but is
  quite useful in fraud detection, rare events
  analysis
• Trend and evolution analysis
• Trend and deviation regression analysis
• Sequential pattern mining, periodicity analysis
• Similarity-based analysis
• Other pattern-directed or statistical analyses

17
Are All the Discovered Patterns Interesting?

• A data mining or machine learning system/query
  may generate thousands of patterns, not all of
  them are interesting.
• Suggested approach Human-centered, query-based,
  focused mining
• Interestingness measures A pattern is
  interesting if it is easily understood by humans,
  valid on new or test data with some degree of
  certainty, potentially useful, novel, or
  validates some hypothesis that a user seeks to
  confirm
• Objective vs. subjective interestingness
  measures
• Objective based on statistics and structures of
  patterns, e.g., support, confidence, etc.
• Subjective based on users belief in the data,
  e.g., unexpectedness, novelty, actionability, etc.

18
Can We Find All and Only Interesting Patterns?

• Find all the interesting patterns Completeness
• Can a data mining or machine learning system find
  all the interesting patterns?
• Association vs. classification vs. clustering
• Search for only interesting patterns
  Optimization
• Can a data mining or machine learning system find
  only the interesting patterns?
• Approaches
• First general all the patterns and then filter
  out the uninteresting ones.
• Generate only the interesting patternsmining
  query optimization

19
Data Mining Confluence of Multiple Disciplines
Database Technology
Statistics
Data Mining
Machine Learning
Visualization
Information Science
Other Disciplines
20
Data Mining Classification Schemes

• General functionality
• Descriptive data mining
• Predictive data mining
• Different views, different classifications
• Kinds of databases to be mined
• Kinds of knowledge to be discovered
• Kinds of techniques utilized
• Kinds of applications adapted

21
A Multi-Dimensional View of Data Mining
Classification

• Databases to be mined
• Relational, transactional, object-oriented,
  object-relational, active, spatial, time-series,
  text, multi-media, heterogeneous, legacy, WWW,
  etc.
• Knowledge to be mined
• Characterization, discrimination, association,
  classification, clustering, trend, deviation and
  outlier analysis, etc.
• Multiple/integrated functions and mining at
  multiple levels
• Techniques utilized
• Database-oriented, data warehouse (OLAP), machine
  learning, statistics, visualization, neural
  network, etc.
• Applications adapted
• Retail, telecommunication, banking, fraud
  analysis, DNA mining, stock market analysis, Web
  mining, Weblog analysis, etc.

22
Architecture of a Typical Data Mining System
Graphical user interface
Pattern evaluation
Data mining engine
Knowledge-base
Database or data warehouse server
Filtering
Data cleaning data integration
Data Warehouse
Databases
23
Major Issues in Data Mining (1)

• Mining methodology and user interaction
• Mining different kinds of knowledge in databases
• Interactive mining of knowledge at multiple
  levels of abstraction
• Incorporation of background knowledge
• Data mining query languages and ad-hoc data
  mining
• Expression and visualization of data mining
  results
• Handling noise and incomplete data
• Pattern evaluation the interestingness problem
• Performance and scalability
• Efficiency and scalability of data mining
  algorithms
• Parallel, distributed and incremental mining
  methods

24
Major Issues in Data Mining (2)

• Issues relating to the diversity of data types
• Handling relational and complex types of data
• Mining information from heterogeneous databases
  and global information systems (WWW)
• Issues related to applications and social impacts
• Application of discovered knowledge
• Domain-specific data mining tools
• Intelligent query answering
• Process control and decision making
• Integration of the discovered knowledge with
  existing knowledge A knowledge fusion problem
• Protection of data security, integrity, and
  privacy

25
Summary

• Data mining / machine learning discovering
  interesting patterns from large amounts of data
• A natural evolution of database technology, in
  great demand, with wide applications
• A KDD process includes data cleaning, data
  integration, data selection, transformation, data
  mining, pattern evaluation, and knowledge
  presentation
• Mining can be performed in a variety of
  information repositories
• Data mining functionalities characterization,
  discrimination, association, classification,
  clustering, outlier and trend analysis, etc.
• Classification of data mining systems
• Major issues in data mining

26
Where to Find References?

• Data mining and KDD (SIGKDD member CDROM)
• Conference proceedings KDD, and others, such as
  PKDD, PAKDD, etc.
• Journal Data Mining and Knowledge Discovery
• Database field (SIGMOD member CD ROM)
• Conference proceedings ACM-SIGMOD, ACM-PODS,
  VLDB, ICDE, EDBT, DASFAA
• Journals ACM-TODS, J. ACM, IEEE-TKDE, JIIS, etc.
• AI and Machine Learning
• Conference proceedings Machine learning, AAAI,
  IJCAI, etc.
• Journals Machine Learning, Artificial
  Intelligence, etc.
• Statistics
• Conference proceedings Joint Stat. Meeting, etc.
• Journals Annals of statistics, etc.
• Visualization
• Conference proceedings CHI, etc.
• Journals IEEE Trans. visualization and computer
  graphics, etc.

27
Introduction to the Life Sciences

• What is human DNA ?
• DNA stands for DeoxyriboNucleic Acid
• DNA stores the genetic material chromosomes in
  each cell nucleus
• DNA is transcribed into RNA out of the nucleus
  (transcription)
• RNA stands for RiboNucleic Acid
• RNA is translated into proteins in a cytoplasm
  organism called a ribosome (translation)
• DNA ? RNA ? proteins

28
Introduction to the Life Sciences
DNA
transcription
mRNA
rRNA
tRNA
Ribosome
translation
Protein
29
Introduction to the Life Sciences

• DNA and RNA are composed of
• Nucleotides (nucleic acid molecules)
• Pyrimidines
• Cytosine (C) (DNA RNA)
• Thymine (T) (DNA)
• Uracil (U) (RNA)
• purines
• Adenine (A) (DNA RNA)
• Guanine (G) (DNA RNA)
• Oses (Ribose for RNA, Deoxyribose for DNA)

30
Introduction to the Life Sciences

• Succession of nucleotides composes a single
  strand in DNA
• Two strands of DNA pair themselves in the 3-D
  shape of a double helix, where bases are paired
  (bp base pair)
• Pairing of the bases (AT, G C) provides
  chemical bonds responsible for the double helix
  shape.

31
Introduction to the Life Sciences
32
Human Genome Program, U.S. Department of Energy,
Genomics and Its Impact on Medicine and Society
A 2001 Primer, 2001
33
Introduction to the Life Sciences

• Genes
• A gene is a part of the genome that can be
  translated
• A gene may encode a protein or RNA sequence
• Genes are separated by non coding regions
• Genes are concentrated in certain regions of the
  genome rich in G and C
• Regions rich in A and T do not contain genes
• Between the two, CpG islands (repetition of C and
  G) separate coding regions from non coding ones
• Non coding regions can be parts of genes

34
Introduction to the Life Sciences

• Genomes, diversity, size, structure
• Profound diversity of living organisms genome.
• DNA (cells), DNA or RNA (phage, virus)
• Direction from 5 to 3 of molecule (double
  stranded DNA), or both directions (single
  stranded)
• Genome organized or not in chromosomes
• Human genome 22 chromosomes, 3 billion bases,
  30,000 genes
• Other species genome vary in size and number of
  genes
• Human genome has only twice as many genes than a
  primitive worm
• GenBank database

35
Introduction to the Life Sciences

• Proteomes
• The proteome is the set of proteins that can be
  expressed from a genome
• Determination of
• Sequence of encoding genes
• Location of the genes
• Function of protein encoding genes
• Different biochemical states (phosphorylation,
  glycosylation, co-enzymes)

36
Introduction to the Life Sciences

• Gene ontologies
• Gene ontology consortium
• Dynamic controlled vocabulary to describe
• Molecular function (Ex DNA polymerase, )
• Biological process (Ex DNA synthesis,
  respiration, )
• Cellular component (Ex nucleus, ribosome, )

37
Principles of Bioinformatics

• Biological information
• Molecules at the basis of life can be represented
  as digital symbol strings (DNA, RNA, )
• Digital symbols (monomers) constitute an alphabet
• Unique representation
• Importance of probabilistic models

38
Principles of Bioinformatics

• Database annotation quality
• In addition to natural noise, data are distorted
  by peoples annotations (curation of the data)
• Resulting error is very significant
• Reasons
• Storage of positions in a sequence, not content
• Difficulty of storing content
• Need to check the data

39
Principles of Bioinformatics

• Database redundancy
• Different representations RNA, cDNA
  (corresponding complementary)
• Different methods single-pass sequence,
  multi-fold repetition of a sequence
• Different fragments pre-mRNA can lead to several
  levels of splicing in cDNA, alternative splicing
• Redundancy is source of error
• Bias of over represented fragments for closely
  related segments
• Bias of over represented fragments for
  correlations
• Overestimate prediction if input and output are
  related

40
Principles of Bioinformatics

• Database redundancy
• Better to clean the data first
• Data mining cleaning methods apply
• Difficulty to differentiate between true
  analogous sequences, and related ones
• Sequence profile describes amino acid variation
  in a family of sequences

41
Principles of Bioinformatics

• Main bioinformatics questions
• Determine the exact transition between coding and
  non coding regions of genes
• Find genes in prokaryotes and eukaryotes
• Determine transcription initiation and
  termination
• Sequence clustering and cluster topology
• Protein structure prediction
• Protein function prediction
• Protein family classification

42
Principles of Bioinformatics

• Question
• Find questions pertinent for bioinformatics
• Find questions pertinent for medical informatics