?.?. ????? ?.?.?., ????????? ??????????? ???????? ???

1
?.?. ????? ?.?.?., ????????????????????
???????? ??? ??-??????
??????? ?????? ??????? ????, ?????? ???
?????????? ??????? ????????? ?????? ????
????????? ?????? ??? ?????????? ???????? ???????
2
(No Transcript)
3
????? ?????? ?????? ????? ??? ?????????
???????? ????????
???????? 260 1018 ???? 1024 ????????
3
4
??????? ?????????? ?? ???????
4
5
????????? ?????????
??????? ?????? ????????????? ? ????????????
???????????? ?????? ??????????? ? ???????????
??????
???????? - ?????????
????????? - ????????
?? ???????????? ????????? ??? ??????????? ???????
?????? ?????????? ??? ????????
?? ????????????? ?????? ??? ?????? ?? ??????
?????? ????????? ??? ????? ????????
????????? ? ?????? ????????? ????????? ???????????
????????
???????? ?????????? ?????? ???????????? ??????
?????
??????????? ??????
??????????? ??????????????????
6
?????????? IBM Big Data ??????? ????????? ??????
???? ????????? ??????
??????????? ?????? ???? ?????? ? ???????
???????????????? ??????????? ? ???????????
?????????? ??????????
????????? Big Data
?????????????-??? ??????
??????????????????? ??????
????????? ??????
7
?????????? ????????? ? ????????????? ???
????????? ?????? ? ?? ???????????? ????
Social Media
Website
Billing
Network Switches
ERP
CRM
RFID
8
?????????? ??????

• ?????????? ?????? ????????????
• ???????? ?? ???????????? ????????????? ??????
  ?????????? ?????????????-????????????? ?????? ???
  ???????????? ??????? ??????? ?????? ??? ?????????
  ???????? ??????? ?????? ??????????? ????????????
  ? ?????????? ??????? ?????????? ??? ?????????
  ??-???????????? ?? ????????????
• ??? ????? ??????????
• ??????????? - ????? ??????????????????
  ???????????? ??? ???????? ?????????? ??
  ????????????? ????????? ??????? ???????
  ?????????? ? ????????? ???????? ??????
• ??????????? ???????? ?????????????-?????????????
  ?????? ? ???????? ????? ? ? ???????? ???????
• ??????????? ???????? ???????????? ? ????????
  ?????????? ????? ? ??????, ???????? ???????
  ??????? ? ???????
• ??????????? ?????????? ?????? ? ?? ??????????
  ?????????????
• ????????????? ???????????????? ??????? ? ????????
  ???????? ?????? ????????

9
(No Transcript)
10
?????????? Big Data

• ?????? ??? ????? ?????
• ????? ? ?????? ??????? ????????? ??????????
  ?????????
• ????????? ????????? ?????? ??? ???????
  ???????????? ? ????????

11
??????????? ???????? ????????? Big Data
1 ????? ? ???????? ?????? ? ??????? ?????? IBM
Data Explorer
3 ????????? ????????? Netezza
2 ?????? ????? ?????? InfoSphere BigInsights
5 ?????? ????????? ?????? InfoSphere Streams
4 ?????????? ?????? ? ??????? Hadoop InfoSphere
BigInsights
11
12
?????????? IBM ??? ????????????? ? ???????? Big
Data

• ????????????? ??????
• IBM Cognos
• IBM SPSS
• ?????????? ??????
• IBM InfoSphere Information Server
• IBM Change Data Capture
• ??????-??????
• IBM InfoSphere Master Data Management Server
• ?????? ??? ??????
• InfoSphere Guardium

• IBM Big Data platform
• InfoSphere Streams
• InfoSphere BigInsights
• InfoSphere Data Explorer
• PureData for Analytics (Netezza)
• ????????????
• ?????? ???????
• ????????
• ???-??????
• ?????
• ???????????????? ??????
• ???????????????? ??????
• ??????????

13
????? ????? ??????????? ????????? Big Data
SPSS (????????????? ???? PMML)
Streams
NZ
DE
Big Ins
????????????? ?????
??????? ???????? ??????????
???????????
??????????
Cognos BI
????? ???????????????? 3-?? ????????? Java,
C/C, Python, Perl
14
??????????? Streams ????????? ????????? ??????????
Streams
????????????? ???? Stream Processing Language
(SPL)
??????? ???????? ?????????? (????????????
??????????) ?????? ??????? ????????????????????
?????? ???-?????? ????? ????????????????
?????? ???????????????? ?????? ?????????? ??????
???????? ???????? (???) ?????? ?????? ?? ?????
(???)
??????????? Standard Toolkit Internet
Toolkit Database Toolkit Financial Toolkit Data
Mining Toolkit Big Data toolkit Text Toolkit
?????????? Netezza Connector Hadoop Connector
????? ???????????????? 3-?? ????????? Java,
C/C, Python, Perl, JavaScript, Ruby ? ?.?.
15
????????????? ???? SPL - ??????????? ?????
?????????? Streams-??????????

• ???????? ?????????? ? ??????? ??????????????
  ??????????
• ??????? ??????? ??????????
• ??????? ? ???????? ??? ?? SPL ?????????????
  ????????????????

16
??????????? BigInsights ?????? ????? ?????? ?
?????????? ?????? ?? ????????
BigInsights
????????????? ????? Annotation Query Language
(AQL), JaQL (Query Language for JSON (JavaScript
Object Notation)), Pig Latin, HiveQL, R
???????? ? ??????????? ????????? Flume Hive Lucen
e Zookeeper Avro HBase HCatalog Sqoop Oozie
?????????? Netezza Connector Streams Connector
????? ???????????????? 3-?? ????????? Java, C/C
17
??????????? Data Explorer ???????? ????????????,
???????????? ?????? ? ????????? ???????
Data Explorer
????????????? ????? ?? ????????????
???????? ?????????? ? ????????? Application
Builder Search Engine
?????????? Framework Connector (30 ??????????,
??????? Streams ? BigInsights) Mature Connector
18
??????????? Pure Data (Netezza) ?????????
???????? ??????? ? ???????? ?????????????????
??????
Pure Data (Netezza)
????????????? ????? SQL, nz/PLSQL
?????????? Hadoop Connector Streams Connector
?????????? ESRI/OpenGIS nzMatrix
????? ???????????????? 3-?? ????????? Java, C /
C, FORTRAN, nzLua, Python, Ruby, JavaScript,
Perl, ? ??.
19
????? ??????????? ?? IBM Big Data

• ?????????? ????? ??????????? ??-??????, IBM ?
  ????? ??????
• ????? ???????? ?????? ??????????? ???????
  ????????????? ??????????? ???????? ?????????? IBM
  Big Data ????????????? ????? ?????? ?
  ????????????? ??????????? ???????, ?
  ?????????????? ???????????? ??????????
  ????????????, ???????????? ??????????? ?
  ???????????? ??-?????? ??? ????????? IBM ??
  ??????, ????????????? ? ?????????????
  ?????????????? ??????????? ? ?? ?????? ??????????
  ?????, ??????????? ???????????

???????? ?????? ??????

• ???????? ???????????????? ??????? IBM Big Data
• ???????? ????????????? ?????? ? ???????
  ???????????????? ???????? ????? ?? ???? ??????
• ???????? ??????? ?????? ??? ???????? ?????????
  IBM Big Data

?????????? ???????????????? ??????????? ?
??????????????
20
????????? ?????? ???????????
??? ??????????? ???????? ????????? IBM Big Data
20
21
??????? ?? ????????!
????????
22
?????????? ???????????? Streams
????????????? ????? ????????????? ?????
Stream Processing Language (SPL) ???????????? ??? ?????????? ?????????? ????????? ????????? ??????
????? ???????????????? 3-?? ????????? ????? ???????????????? 3-?? ?????????
Java, C / C, Python, Ruby, JavaScript, Perl, ? ??. ???????????? ? Streams ??? ???????????????? ??????????. ???? ?????? ????? ???????? ?? ??????????? ??????/???????
?????????? ??????????
Netezza Connector ????????? ?????? ? ?????? ?????? ?? Netezza
Hadoop Connector ????????? ?????? ? ?????? ?????? ?? Hadoop
???????????? ?????????? ???????????? ??????????
?????? ??????? ????? ???????????????? ???????, ??????? ????? ???? ???????????? ??? ????????? ?????????? ?????????? ????? ??????? ?????? ?????? ????? ? ??? ????, ? ??????? ??? ????????? ?? ???? ?????????? ?????????? ????????? ??????????
???????????????????? ?????? ????? ???????????????? ???????, ??????? ????? ???? ???????????? ??? ????????? ?????????? ?????????? ????? ??????? ?????? ?????? ????? ? ??? ????, ? ??????? ??? ????????? ?? ???? ?????????? ?????????? ????????? ??????????
???-?????? ????? ???????????????? ???????, ??????? ????? ???? ???????????? ??? ????????? ?????????? ?????????? ????? ??????? ?????? ?????? ????? ? ??? ????, ? ??????? ??? ????????? ?? ???? ?????????? ?????????? ????????? ??????????
????? ????? ???????????????? ???????, ??????? ????? ???? ???????????? ??? ????????? ?????????? ?????????? ????? ??????? ?????? ?????? ????? ? ??? ????, ? ??????? ??? ????????? ?? ???? ?????????? ?????????? ????????? ??????????
???????????????? ?????? ????? ???????????????? ???????, ??????? ????? ???? ???????????? ??? ????????? ?????????? ?????????? ????? ??????? ?????? ?????? ????? ? ??? ????, ? ??????? ??? ????????? ?? ???? ?????????? ?????????? ????????? ??????????
???????????????? ?????? ????? ???????????????? ???????, ??????? ????? ???? ???????????? ??? ????????? ?????????? ?????????? ????? ??????? ?????? ?????? ????? ? ??? ????, ? ??????? ??? ????????? ?? ???? ?????????? ?????????? ????????? ??????????
?????????? ????? ???????????????? ???????, ??????? ????? ???? ???????????? ??? ????????? ?????????? ?????????? ????? ??????? ?????? ?????? ????? ? ??? ????, ? ??????? ??? ????????? ?? ???? ?????????? ?????????? ????????? ??????????
?????? ???????? ???????? ????? ???????????????? ???????, ??????? ????? ???? ???????????? ??? ????????? ?????????? ?????????? ????? ??????? ?????? ?????? ????? ? ??? ????, ? ??????? ??? ????????? ?? ???? ?????????? ?????????? ????????? ??????????
?????? ?????? ?? ????? ????? ???????????????? ???????, ??????? ????? ???? ???????????? ??? ????????? ?????????? ?????????? ????? ??????? ?????? ?????? ????? ? ??? ????, ? ??????? ??? ????????? ?? ???? ?????????? ?????????? ????????? ??????????
23
?????????? ???????????? Streams
Toolkits Toolkits
Standard Toolkit ??????????? ????? ??????????, ??????? ???????????? ?????? ?? Streams
Internet Toolkit ???????? ??? ?????????? ?????? ?? URL ?????????. ?????????????? ???? ????????? HTTP, HTTPS, HTTP ????? RSS, RSS ????? HTTPS, FTP, FTPS, ? ????
Database Toolkit ????? ?????????? SPL, ??????? ????????? ????? ????????????? ? ???????? ????????? ??????
Financial Toolkit ????? ?????????? ????????? ??? ????????? ?????????? Streams c ?????? ????????????? ??????????? ? ???????????? ? ?????????? ?????
Data Mining Toolkit ???????? ????????? ????????????????? ??????? ??????. ????????? ?????????? ???????? PMML
Big Data toolkit ????? ?????????, ??????? ????????? ????????????????? (?????? ? ??????) ?????????? Streams ? HDFS
Text Toolkit ???????? ???????? ??? ??????? AQL ???????? ?? ?????????? ????????? ? ??????? ??? ???????? AQL ????????
24
?????????? ???????????? BigInsights
????????????? ????? ????????????? ?????
Annotation Query Language (AQL) ????, ??????? ???????????? ??? ????????? ???????????????????? ?????
JaQL (Query Language for JSON (JavaScript Object Notation) ) ?????????????? ???? ???????? ??? ?????????? ??????? ??????????, ??????????? ? ??????????? JSON-??????. ????? ????????? ???????????????? ??????? ??? ????????????? ? ??????????? ??????????
Pig Latin ???? ????????? Pig ??? ???????? MapReduce ??????????.
HiveQL ????, ??????? ???????????? ? Hadoop ? ???????? ????? ????????
R ????, ??????? ???????????? ??? ?????????????? ????????? ?????? (??????????????? ???????)
?????????? ??????????
Netezza Connector ????????? ?????? ? ?????? ?????? ?? Netezza
Streams Connector ????????? ?????? ? ?????? ?????? ?? Streams
????? ???????????????? 3-?? ????????? ????? ???????????????? 3-?? ?????????
Java, C / C ???????????? ? BigInsights ??? ???????? ??????????
25
?????????? ???????????? BigInsights
???????? ? ??????????? ????????? ???????? ? ??????????? ?????????
Flume ????????, ??????? ????????? ???????? ?????? ? ????????? ???????????
Hive ????????, ??????????? ???????? ??????????-??????????????-???????? (ETL) ? ?????????? ? ??????? ??????? ??????? ??????, ?????????? ? ?????????????? ???????? ??????? Hadoop (HDFS). ??? ????????? ??????????????? SQL-??????? ? ???? ???????? JaQL ? ????????? MapReduce-?????????. InfoSphere BigInsights ???????? JDBC-????????, ??????? ????? ???????????? ??? ???????????????? ? Hive ? ??? ??????????? ? ???????????? ??????????? Cognos Business Intelligence
Lucene ??? Jaql ??????, ??????? ????????? ?????????, ??????????? ? ??????????? Lucene ???????
Zookeeper ??? ???????????????? ?????? Apache ??? ?????????? ?????????? ? ????????????, ????????????, ??????????? ?????????????? ?????????????, ? ????? ?????????????? ?????? ?????
Avro ??? ????????? ??? ???????????? ? ?????????????? ??????.
HBase ?????????? HBase ????????? ?????????????? ?????? ?????? ?? ??????? HBase ????? ??????? InfoSphere BigInsights
HCatalog ?????? ?????????? ???????? ?????? ? Hadoop
Sqoop ???????????? ??? ??????????? ?????? ????? BigInsights InfoSphere ?????????????? ???????? ??????? ? ???????????? ????????? ?????????? ?????? ??????
Oozie ???????? ?????????? ????? Apache Hadoop
26
?????????? ???????????? Data Explorer
???????? ????????? ? ?????????? ???????? ????????? ? ??????????
Application Builder ????????, ??????? ????????? ??????? ????????????? ???-??????, ??????????????? ????????????? ????????? ? ??????????, ??????? Data Explorer ????? ??????????? ? ?????????????
Search Engine ????????, ??????? ?????????? ????????? ?????????? (??????????, ??????????, ???????????)
?????????? ??????????
Connector Framework ???????????? ????? 30 ????? ???????????? ?????????? ??????, ??????? CRM ???????, ?????? ??????????? ????, ? ????? Streams ? BigInsights.
Mature Connector ???????????? ??? ???????? ?????????????? ??????????? ? ??????????? ?????????? ??????.
27
?????????? ???????????? Pure Data (Netezza)
????????????? ????? ????????????? ?????
SQL, nz/PLSQL ???????????? ??? ????????? ???????? ??? ?????? ? ????????????????? Netezza
????? ???????????????? 3-?? ????????? ????? ???????????????? 3-?? ?????????
Java, C / C, FORTRAN, nzLua, Python, Ruby, JavaScript, Perl, ? ??. ??? ????? ???????? ?????? IBM NETEZZA ANALYTICS. ?????????? ????? ??????????? - ???? ??????????? ????????????? ????????? ????? ??????? ? ??????? ??? ????????? ?????? ? Netezza.
?????????? ??????????
ESRI/OpenGIS ?????????? ????????????? ????????? ??????, ????????? ? ??????????????????? ????????.
nzMatrix ?????????? ????????????? ????????? ??????, ????????? ? ??????? ? ?????????
?????????? ??????????
Streams Connector ????????? ?????? ? ?????? ?????? ?? Streams
Hadoop Connector ????????? ?????? ? ?????? ?????? ?? Hadoop