Unlock bigdata analytics efficiency with storage disaggregation.pdf

《Unlock bigdata analytics efficiency with storage disaggregation.pdf》由会员分享，可在线阅读，更多相关《Unlock bigdata analytics efficiency with storage disaggregation.pdf（29页珍藏版）》请在得力文库 - 分享文档赚钱的网站上搜索。

1、Unlock bigdata analytics efficiency with storage disaggregation 软件工程经理 张建AgendaData Growth TrendChallenges of scaling bigdata Intels role in storage Accelerate Bigdata analytics with disaggregated storage Summary 2AgendaData Growth TrendChallenges of scaling bigdata Intels role in storage Accelerate

2、 Bigdata analytics with disaggregated storage Summary 3Trend #1: Explosive growth in Data spherePicture Source: IDCs Data Age 2025 study, sponsored by Seagate, April 20174Trend #2: Rapid growth of data criticality 5Criticality ComputerData CenterMobile PhonePower SupplyMedical CareAutonomous Driving

3、Trend #3: Analytics and Artificial Intelligence change the landscape6DataValueTransportEnergyIndustryEnterpriseOtherIdentify and focus on the critical data that result in the highest impact.ConsumerHealthFinanceRetailGovernmentDataData InputFlume*Kafka*StorageHbase*HDFS*Resource MgmtIntel & Ali will

4、 continued collaboration in bigdata on object storageHadoop over object storage with Intel technologies delivers optimum performance while achieving scalability and flexibility 26No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document

5、.Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.This document contain

6、s information on products, services and/or processes in development. All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps.The products and services described may contain defects or er

7、rors known as errata which may cause deviations from published specifications. Current characterized errata are available on request.Intel, the Intel logo, 3D Xpoint, Optane are trademarks of Intel Corporation in the U.S. and/or other countries. *Other names and brands may be claimed as the property

8、 of others 2017 Intel Corporation.Legal noticesBackup 28Bigdata on object storage POC cluster configuration295x Compute Node Intel Xeon processor E5-2699 v4 2.2GHz, 128GB mem 2x10G 82599 10Gb NIC 1x 480GB S3510 and 1x 400GB S3700 as shuffle storage Software: Hadoop 2.7.3， Spark 2.1.1， Hive 2.2.1， Pr

9、esto 0.177， RHEL7.3 5x Storage Node, 2 RGW nodes, 1 HA nodes Intel(R) Xeon(R) CPU E5-2699v4 2.20GHz 128GB Memory 2x 82599 10Gb NIC 1x Intel P3700 1.0TB SSD as WAL and rocksdb 4x 1.6TB Intel SSD DC S3510 , 4x 400GB s3700 as data drive 1 OSD instances one each S3510 SSD RHEl7.3 RHCS 2.3 *Other names a

10、nd brands may be claimed as the property of others.OSD1MON OSD1OSD8Hadoop Hive Spark Presto1x10Gb NICOSD2OSD3OSD4OSD52x10Gb NICHadoop Hive Spark PrestoHadoop Hive Spark PrestoHadoop Hive Spark PrestoHadoop Hive Spark PrestoRGW1RGW2HA Head4x10Gb NIC2x10Gb NIC1x10Gb NICWorkloads With selected queries from TPC- DS*