Title: Chapter 6 Date Warehousing
1??? ?????????
- 6.1 ?????????
- 6.2 ????????????
- 6.3 ??????????????
2 Chapter 6 Date Warehousing Data Mining
- 6.1 Definitions of data warehousing data mining
- 6.2 Approach and technologies of data mining
- 6.3 Application of data mining in CRM
36.1 ?????????
???????
- E????????
- E????,?????????13?
- ?7????????????
- ?????????????????????,????????????
- ????
- ??????/??/??/????
- ??????/????/????
- ?????????????????????????????????/????????
46.1 Definitions of Data Warehousing Data Mining
Background of Business Intelligence (BI)
- Information features in the e-century
- In E era, the amount of electrical data grows 1
to 3 times per year. - only 7 of existing data are properly analyzed
and applied. - Demand for information application data -gt
information -gt knowledge -gt wisdom. - Goals
- To assist organization in obtaining /
generalizing / explaining /analyzing data. - Profit analysis / Relationship marketing /
Customer management - Application domain customer contribution
analysis / market segmentation / risk management
/ cross-sells analysis / product profile analysis
/ investment portfolio analysis
56.1 ?????????
????(BI)??
- ???????????????????,?????????????????
- ??
- ????????????????
- ?????????????
- ?????????????????
- Provide the Right Information to the Right
Persons at the Right Time.
BI????????????????????????????????,?????????
66.1 Definitions of Data Warehousing Data Mining
Definition of Business Intelligence (BI)
- Definition The ability to rapidly analyze and
synthesize enterprise data in order to improve
its decision quality and business performance. - Features
- Analyze business development trend.
- Decision supports.
- Provide the Right Information to the Right
Persons at the Right Time.
BI techniques can provide statistical analysis,
data mining and analysis, and transform great
quantity of data into meaningful knowledge to
support the decision making.
76.1 ?????????
???????(1)
- ???????,??Bottom-Up??????,???????,??????(Row
Data)? - ??????,???????????????,????????????(Organized
Information)? - ??????????(domain know-how),????????(Knowledge)?
- ???????????,???????,????(Wisdom)?
BI????Bottom-Up??
86.1 Definitions of Data Warehousing Data Mining
Evolution of BI (1)
- In the organization, using the Bottom-Up process
to collect and save the record that forms the Raw
Data. - On the basis of making a decision, selecting,
reading, handling, analyzing the raw data the
that obtain the Organized Information. - Merging with organized information and domain
know-how of the company that transforms the
Knowledge. - Added the experience of the decision-makers,
fully utilized the knowledge that produce Wisdom.
The approach to constitute BI is Bottom-Up
process
96.1 ?????????
???????(2)
- Data?????(?????)
- Information ?????Data(?????)
- Knowledge Information????(?????)
- Wisdom ???????(???????)
106.1 Definitions of Data Warehousing Data Mining
Evolution of BI (2)
- Dataraw data (ex experiment data)
- Information collection and arrangement of data
- (ex experiment result)
- Knowledge Integration of information and
experience - (ex experiment conclusion)
- Wisdom Heuristic knowledge
- (ex application of experiment conclusion)
116.1 ?????????
???????
- ?????????????????????
- ???????????/???????????,???????????
- ????????????????,?????????????
- ???????????????(Bottom-Up)???????
- E?????????(1)????/???(2)????/??/??????
BI????Top-Down??
126.1 Definitions of Data Warehousing Data Mining
Application of BI (1)
- The high-level executives often need deeper
knowledge to make crucial decisions. - The application of e-technology can be used to
improve (1) information extraction/understanding
and (2)decision making /transaction.
The applications of BI are Top-Down processes
136.1 ?????????
??????????
- ????????????????(DW?DM)
- ??????IT?????????????????(KM)
- ???????????????/????(GIS)
- ??????????????,??????????(KM?VR)
- ??????????????????
- ?????????????????????????????(ERP?PM)
- ??????????????????????(KM?PM)
- ??/???????????????(CRM)
- ?????????????????(CRM)
146.1 Definitions of Data Warehousing Data Mining
Application of BI (2)
- Data analysis Discover the trend and model of
the great quantity data(DW?DM) - Intelligence disposition Apply IT solution for
commerce analysis and knowledge extraction.(KM) - Geographical (spatial) analysis Integrate with
business data and geographic or population
information. (GIS) - Data Visualization Use of the GUI to view data
and to support the optimal business decisions.
(KM?VR)
156.1 Definitions of Data Warehousing Data Mining
- BSC (Balance Score Card) Inspect the performance
of the managing of enterprise correctly. - Project management Fully grasp information in
order to make decision of resource allocation and
project choosing (ERP?PM). - Collaborative intelligence accumulation Use the
staffs experience (KM?PM). - Business / marketing analysisFully control the
sales data and trends (CRM). - Customer information analysisUnderstand
customers behavior and preferences (CRM).
Application of BI (3)
166.1 ?????????
CRM????????
- ??1????????,??????????,????????????
- ?????
- ????????
- ???????
- ??????
- ??2??????????????????????,??????,?????????,??????
???????? - ?????????????????????????????????????????????????
????
176.1 Definitions of Data Warehousing Data Mining
Definition of CRM
- Definition 1 Through effective communication and
understanding customer's behavior, enterprises
can reach the connection and expand the customer
source goal. - Increase new customer.
- Take precautions against the existing customer
lost. - Improve customer's loyalty and the profit of the
company. - Definition 2It is a kind of repeated course that
changes customers information into customers
positive relation. Using of information
technology, strengthen the practicability and
speed of administrative decision. - ConclusionCRM can offer more relevant
information to CSR, MIS, sales, and executive
about relation BI.
186.1 ?????????
CRM????
- CRM???????????????????????
- CRM? ??????????????(????????),????????????????????
?????? - ???????????,???????????????????????
- ??????????????????????????,??????????????
- Note?????????
- ????????????????
?????BI??????,????????????????,?????????????
196.1 Definitions of Data Warehousing Data Mining
Topics of CRM
- CRM should integrate all of the operations,
members , transactions of the company. - Customer relationship of CRM is not only trade
(products or the service), but expect customers
purchase continually or other valued behaviors. - Communication must be two-way, integration, have
records, management that make customer
relationship is really exist. - Contact Center is the first step of CRM.
Integrate the concepts and technologies of BI
collection, extraction, analyze services
information effectively, support decision making
of the organization.
206.1 ?????????
BI?CRM?? (1)
- ???????????????????,??/????,??????????????????
- ????????????????,??????
- ??E?????????(Transaction)??????,??????????????????
??? - ??BI???????,???????????,???????
- ?CRM????????????????????????(BI)???????
216.1 Definitions of Data Warehousing Data Mining
Relationship between BI and CRM (1)
- Digital information includes various kinds of
data that enterprises have accumulated for many
years. Internal and external information and
various types of accumulated experiences with
customers or other enterprises - Most of enterprise information systems only deal
with transactional data. - BI data mining helps better understand enterprise
operational models and customers characteristics.
- CRM integrates marketing and customer information
systems and BI decision analysis systems.
226.1 ?????????
?????CRM??(2)
Source http//bi.fast.com.tw/newpages/bi01.htm
Source http//www.bitech.com.tw/
236.1 Definitions of Data Warehousing Data Mining
Relationship between BI and CRM (2)
Source http//bi.fast.com.tw/newpages/bi01.htm
Source http//www.bitech.com.tw/
24?????CRM??(3)
Data
Information
Knowledge
??BI??????????????????Bottom-Up????,????Top-Down??
????
Wisdom
25Relationship between BI and CRM (3)
Data
Information
Knowledge
Button-Up collect related marketing, services
and customers data Top-Down strategies and
applications.
Wisdom
266.1 ?????????
????????????
Database Design
276.1 Definitions of Data Warehousing Data Mining
Steps of building and analyzing data of contact
center
Database Design
286.1 ?????????
?????????
- ??????????????????????,??????????,????????????
- ??????????????????,??????????????
- ?????????????????????????????,?????????????
296.1 Definitions of Data Warehousing Data Mining
Database system application
- Database is an IT application that collects and
marshals various data, and then saving them by
effective and organizational approach. - Database is widely used in various areas, i.e.,
library management.
306.1 ?????????
?????????
- File(??)???????????????????????????????????,?????
?????? - Record(??)
- Field(??)
- Character(??)
- Entity(??)
- Attribute(????????)
- ????????,?Database???Record???Record?Field???,??Fi
eld?????????,??Data Dictionary???????
316.1 Definitions of Data Warehousing Data Mining
Database system (DBMS) related vocabularies
- File
- Record
- Field
- Character
- Entity
- Attribute
- A database system includes various Records that
are form by Field. Field can be any data type,
and using Data Dictionary to class the context.
326.1 ?????????
???????????
- ????????
- ????????
- ????????
- ??????
- ???????????,????
336.1 Definitions of Data Warehousing Data Mining
Bottleneck of file systems for data storage
- Lack security of the data
- Lack commonality of the data
- Lack flexibility of the data
- Repeat records
- Hard to maintain the system because of tight
linkages between data and programs.
346.1 ?????????
????????????
- ????????????????????,????????????????????????????
- ?????????????????,??????????
- ???????????????????,????????????????????
- ????????,????????????????????????????????
356.1 Definitions of Data Warehousing Data Mining
Advantages of database system for data storage
- Storage, search, indexing capabilities
- Extraction of information for different commerce
applications. Generate the necessary reports. - Internet access.
- Flexible search functions. Efficiently manage
large amount of data.
366.1 ?????????
?????(1)
- Selection????????????????(Row)?
- Projection????????????(Column)?
- Product????Table??,?Table 1?N Rows?I
ColumnsTable 2?M Rows?J Columns,????????????NM
Rows?IJ Columns??Table? - ??Table???N Rows?M Rows,?Union????NM
Rows(????Table?Schema??,?????)?
376.1 ?????????
?????(2)
- Set Difference?????Relation???,?????Relation???,?R
elation??????A-(A-B)??? - Join???????????Field???Tuple??,??Tuple????????Tupl
e???????????,??Selection?Product???,??????????????
???????????????View??,???Row??????
386.1 Definitions of Data Warehousing Data Mining
Operations of database
- Selection is choosing the pass muster Row from
table. - Projection is moving out specific Column from
table. - Product is the multiplication of two tables. If
Table 1 has N rows, I columns, and Table 2 has M
rows, J columns, then the product is a new table
that has NM rows, IJ columns. - If two table is N rows, M rows respectively, then
Union result is N M Rows . - Set Difference is to find that data exists in
one Relation and not exists in another. The
interaction of two Relations is A-(A-B). - Join is to combine with two mutual Fields.
396.1 ?????????
SQL (Structured Query Language)
- SQL????????Relational Algebra?????????
- ?????????????????????
- ?IBM?1970???,????ANSI???
- ???RDBMS???,SQL?????Table Schema??????????????
- ?Create Table???????????Delete???Row?Insert
Into???Row??Table?)?
406.1 Definitions of Data Warehousing Data Mining
SQL (Structured Query Language)
- SQL standards for Structured Query Language and
is developed according to Relational Algebra. - SQL is developed for allowing high level language
to access the database. - SQL is developed by IBM in 1970 and is an ANSI
standard language. - In addition to query the data of RDBMS, SQL can
define the meaning of Table Schema, data built up
and translation. - Ex Creating Table can set up data scheme and
attribute, Delete can delete a row, Insert
Into is added a row to a table.
416.1 ?????????
RDBMS vs OODBMS
- ?RDBMS?,??????????????????(Object-Oriented
Database Management SystemOODBMS)? - ?OODBMS???,????????????,?????Object ID????
- ??????????Object ID?Object Link??(??Join??)?
- OODBMS??????Schema Querying(Run-Time Schema
Querying),???????????Meta-Data(?)??? - ????Hybrid OO-Relational???????,????RDBMS?????????
?,?OODBMS?????
426.1 Definitions of Data Warehousing Data Mining
RDBMS vs OODBMS
- Except RDBMS, DBM gradually develops
OODBMS(Object-Oriented Database Management
System) way. - In the OODBMS environment, all objects save in
one space and tracking by Object ID and Object
Link. - OODBMS provides really time Schema
Querying(Run-Time Schema Querying) to query
Meta-Data of application object. - Using Hybrid OO-Relational database systems now
is more, it offers the expression method with
objects of RDBMS, pure OODBMS is comparatively
used few.
436.2 ????????????
?????????
- ???????????????????????
- ??????????????-?????,?Entity-Relation
Model(??ER Model) - ????????????????????????????,???????????????????
- ???????????????????????DBMS?????,???????????????
?? - ??????(Normalization)?????????????????,?????????
- ??(Primary Key)????????(Key),??????(Primary
Key)????????????
446.2 Approach and technologies of data mining
Database design of the contact center
- First Step The demand specification of DB
should be clear out. - Second Step concept designbuild up
Entity-Relation Model (ER-Model) - Third Step logic desgntransform into decided
DBMS. (Ex the logic data model of RDBMS) - Fourth Step reality design transform the logic
data model into hardware type of DBMS to decide
data storage structure and search route. - Normalizationseek relation, and set up
structural format. - Decide Primary KeyPrimary Key represents the
characteristic of this table uniquly.
456.2 ????????????
ER Model??
466.2 Approach and technologies of data mining
ER Model example
476.2 ????????????
?????????(1)
- ???????????,??????????????????,???????????
- ????????????????,??????????(Extraction?Transformat
ion?Loading, ETL)?????,?????????? - ???????????
- ???(?????????)
- ???(?????????)
- ????
- ?????(??????)??????????
486.2 Approach and technologies of data mining
Data Warehousing of a contact center (1)
- Data Warehousing is to bring together
information from multiple operation systems as to
provide a consistent database source for decision
support. - Collection of data by CSR (Customer Service
representative) that is deal with extraction,
transformation, loading, ETL and saving in the
data base. - Data is the database should be
- correctness
- completeness
- Integration
- Classification according to translation (Ex
customers, products).
496.2 ????????????
?????????(2)
- ??????????????,???????????
- ??(Multi-Dimension)???????
- ????(Client/Server Architecture)
- ????(Middleware)
- ???????(GUI)
- ????(Replication)
- ????(Parallel Processing)
506.2 Approach and technologies of data mining
Data Warehousing of a contact center (2)
- Data Warehouse is the base of Decision Support
System (DSS) and the technologies are
multi-dimension and complicated - Multi-Dimension Database Management System (DBMS)
- Client / Server Architecture
- Middleware softwre
- GUI (Graphical user interface)
- Replication
- Parallel Processing
516.2 ????????????
?????????(3)
- ????
- ????????
- ??????
- ?????????????
- ??????????
- ?????????????????
526.2 Approach and technologies of data mining
Data Warehousing of a contact center (3)
- Fundamental attributes
- Classification according to translation
- Integration of multiple data
- Users can not change data without authorization
- Data changes with time constantly
- To help enterprise makes faster and better
decisions.
536.2 ????????????
?????????(4)
- ???????????????????,???????????
- ????????,?????????
- ???????????,?????????????
- ??????????????,???????
- ??????????,?????????????????
- ??????????????
- ??????????????
546.2 Approach and technologies of data mining
Data Warehousing of a contact center (4)
- Whether construction the data warehousing project
is succeeded and obtained its benefit, there are
some essential factors that should have high
dependence as following - Set up the support department and support the
projects to carry out. - Define the clear goal and demand range and let
the data warehousing platform accord with the
requirement. - It should have the high-level executive support,
a good trans-departmental communicative channel,
and setting up the corporate accountability. - The data warehousing platform must possesses
opening, expanding, stable, attributes, keeping
the same interface specification of the front
system .
556.2 ????????????
????????
566.2 Approach and technologies of data mining
Data Warehousing Process
576.2 ????????????
????????
- ????????????????????????
- ????????????????????????????????
- ??????????????
- ???????????????Data Mining?OLAP????????
- ???????????????
586.2 Approach and technologies of data mining
Benefits of data warehousing
- Promote the operation ability and analysis
efficiency of the system for users. - Train enterprises the ability of obtaining
information rapidly, shorten the reflect time of
executive . - Strengthen enterprise's information
centralization and integration ability. - Offer a new approach of analysis to enterprises,
support analysis tasks , such as Data Mining ,
OLAP ,etc.. - Improve enterprises to analyze the business trend.
596.2 ????????????
??????
- ?????????,???????????????????????????,????????????
??? - ????????(???????),???????????(?????????
- ???????????????????????????,
- ?????????????????????????????,????????????????????
?????,????????????????? - ????,??????????,??????????????????,???????????????
????,??????????????
606.2 Approach and technologies of data mining
Future trend of data warehousing
- The data warehousing technology overcomes the
bottleneck of the information technology
gradually, i.e., portability of cross-platform,
analysis efficiency, and the huge amount data
storage. - The subject applied to CRM at present will be
developed towards the depth value of information
analysis gradually. - Using the data warehousing technology, that can
transform a great quantity, unorderly data into
BI, and then help enterprises to analysis the
customers and assistance in the product
planning, increasing the competitiveness of
enterprises and profit-making chance.
616.2 ????????????
????
- ????(Data Mining)?????????????????????????,???????
??????? - ????????????????????
- ???????????????????????????????????
- ??????????????????????????????????????????????????
????? - ????????????????????????????????????(Baysian
Network)?Nearest Neighbor?Attributed-Oriented
Induction?Binary/Quantitative Association Rules??
626.2 Approach and technologies of data mining
Data Mining
- Data Mining discusses how to explore hiding
useful information and trend among a large amount
of data, in order to offer decision supports. - Data Mining process can be viewed as a KDD
process, including data selecting,
pre-processing, data translation, data mining,
explanation, and estimation. - Related research areas with the data mining
include DB technology, AI, expert system, data
visualization, statistics. - The popular models and technologies at present
include decision tree, neural network, Baysian
Network, Nearest Neighbor, Attributed-Oriented
Induction, Binary/Quantitative Association Rules.
636.3 ??????????????
?????????(1)
- ????
- ??????????????,???????,?????(Class)
- ?????????????????????,??????????
- ????,??????????????????????,??????????????????????
- ???????????????????????????
- ?????????????????????,???????????(Segmentation)
646.3 Application of data mining to CRM
Data Mining of the contact center (1)
- Data Mining Functionalities
- Classification Classification is subdivided by
assigning each element or record to a predefined
class on the basis of a model developed through
training on pre-classified examples. - Estimation Estimation deals with continuously
valued outcomes and given some related input data
to come up with a value for unknown continuous
variable.
656.3 Application of data mining to CRM
Data Mining of the contact center(2)
- Data Mining Functionalities
- Prediction Prediction is similar classification
and estimation except that the records are
classified according to some predicted future
behavior or estimated future value according to
observation value of the past. - Affiliation Grouping Affinity grouping is to
determine which things go together. - Clustering Clustering is to segment a
heterogeneous population into a number of more
homogeneous subgroups or clusters.
666.3 ??????????????
??????
676.3 Application of data mining to CRM
Data Mining Approach
686.3 ??????????????
??????
- ???????????????????????
- ????????????????????????
- ???????????????????,???????????
- ?????????OLAP????????????
- ????????????????????,????????????,????????
- ?????????????????????????????
- ???????????????????,??????????????????,??????????
??????
696.3 Application of data mining to CRM
Data Mining Plan
- Confirm problemConfirm the potential problems
that are wanted to deal with. - Decide data sourceDetermine where the basic data
comes from . - Decide requirementDecide interview and seek the
relevant problem data. - Establish ModelOLAP or neurual network.
- Data arrangementTransform the data because of
different data models and data demand. - Apply data warehousingData mining process must
be supported with data warehousing. - Software applicationSet up model through the
specialized statistical analysis software.
706.3 ??????????????
??????
716.3 Application of data mining to CRM
Data Mining Model
726.3 ??????????????
?????????(1)
- ???????????,??????????????????,???????????
- ????????????????,??????????(Extraction?Transformat
ion?Loading, ETL)?????,?????????? - ???????????
- ???(?????????)
- ???(?????????)
- ????
- ?????(??????)??????????
736.3 Application of data mining to CRM
Data Warehousing of the contact center(1)
- Data mining A huge database using structural
ways to store data that was been related from
different operation systems. - Extracting, transforming, loading, the data
received from customer service personnels
on-line process system. Then store it in the
database. - Must ensure the data in the warehouse is
- Correctness(There are not wrong materials mixed
among them ) - Completeness(essential data are all stored )
- Combine each other
- Regard trade subject (such as customers,
products) as its store categorized basis
746.3 ??????????????
?????????(2)
- ??????????????,???????????
- ?????????????????????????????,????????????????
- ?????????????(??????????????)????????????????????
?????,??????????? - ??????????????????????????,???????,???????,??????
??????? - ??????????????????,?????????????
756.3 Application of data mining to CRM
Data Mining of the contact center (2)
- All information collected is valuable assets, the
importance of data mining is very high. - Data mining is use automation or semi-automatic
way to carry on trend analysis to a large amount
of data, and to seek the meaningful relation or
rule to enterprises. - Customer segment Analyzing and clustering the
customers according to various kinds of attribute
of them(like gender, profession, income and
Education degree). The same group of customers
means that their entirety attribute is similar.
According to this, we can proceed difference
marketing. - Customer segment by goals Record and investigate
the unity consumption habits of all customers to
segment them. And according to customers
characteristics, using decision tree structure
to build segment model. - Characteristic (features) analysis Utilize the
cluster analyze model to segment customers, and
Step forward to analyze each groups
characteristics.
766.3 ??????????????
?????????(3)
- ???????????????????????????????????????,?????????
? - ?????????????,????????????
- ??????????????,?????????????
- ??????????????,????????????
- ??????????????????????????,????????
- ??????????????,?????????????????,???????????????
? - ?????????????????????????,???????????,???????????
???? - ???????????????,????????????,????????????????????
??????
776.3 Application of data mining to CRM
Data Mining of the contact center (3)
- Contribution analysis It means to analyze
customer's grade to enterprise's contribution
degree. It is to analyze certain consumption
attribute for all customers to segment and
cluster customers. - Period analysis According to the purchase
recently to analyze every the shopper's
characteristic during each period. - Frequency analysis According to the customer
buys frequency recently to analyze each shopping
frequency of the customers characteristic. - Amount analysis According to the customers
shopping amount to analyze each amount of
customers characteristic
786.3 Application of data mining to CRM
Data Mining of the contact center (4)
- Lifetime value analysis according to customers
buying day, frequency and amount to generalize
the value grades. - Cross-sell rule analysis Analyzing the detail
ledger of customer to know which goods they tends
to buying at the same time while doing shopping,
to recommend goods conform to customer's
interests . - Sequential-sell analysis Seek the priority
relation that the customer does shopping in
different shopping experience to make products
marketing orientation more correct and to slash
marketing and advertisement cost - Forecast (prediction) analysis According to
various kinds of attribute of the potential
customers, through built customer segment model
to accurately predict the customer of marketing
in future will belong to what kinds customer
types.
796.3 ??????????????
?????????(4)
- ????
- ???????????,???????
- ??????????????
- ????????????????????
- ?????????????????
- ??????????????
- ????????,?????????
806.3 Application of data mining to CRM
Data Mining of the contact center (5)
- Typical application
- According to the tendency of customer surf the
webpage to infer their preferences. - According to the sale data to Explore customer's
consumption habits. - Early warning credit card debt from customers
consume and the data of paying. - Product sales affiliation (co-relations) in a
large amount of trade data. - Discover the hot topics in a large amount of
customer service data - Look for the authorize rule of credit card
according to the history verify data.
816.3 ??????????????
????????(1)
- ????
- ????????????????????,?????????????????????????????
??????????????,???????????,???????????????? - ?????
- ???????????,??????????,???????????????????,???????
???,?????????,?????????????????
826.3 ??????????????
????????(2)
- ???
- ??????(The Data Mining Group)???????????????(Predi
ctive Model Markup Language, PMML)?PMML???XML??,??
????????,????????????????????????????? - ???
- ?????????????????????????,????????????????????????
??,?????????????????????,?????????????????
836.3 Application of data mining to CRM
Future trend of data mining
- Elasticity improving Because data mining tools
are high relation with sample, so the tool should
have high expandable property, store more
multiple higher-dimension data. - Economy improving The conclusion of analysis can
be implemented in industry's demand, offer the
rate of returns of investment. - StandardizationThe Data Mining Group start to
develop PMML (Predictive Model Markup Language)
that is a XML standard to describe the common
prediction model for other data mining, BI
application. - IntegrationIn order to promote the operation
efficiency of data mining model, some data mining
functions integrate RDBMS to promote query and
selecting efficacy.