Tuesday, September 29, 2015

Big Data/Analytics Performance – Driving IBM Power System Success

By Rich Ptak

It’s no secret that the action today is in Big Data and the associated Analytics! Whether for business, retail, education, government, medical, media, whatever, the focus is on data! Lots of it! Coming from every direction in every conceivable source and format. It is structured, unstructured, transactional, audio and visual flowing from IoT, mobile, social, production…to the tune of some 2.5 quintillion new bytes generated every single day.

IT is tasked with processing this raw data into the insights, wisdom, knowledge that result in new services or deliver solutions to previously impenetrable mysteries. The ultimate goal is to deliver benefits and provide value to users, clients and customers. Processing large amounts of data has been computing’s forte since their inception. BUT now the processing of data and generating results is immensely more complicated and must be delivered more quickly and economically than ever before.

IBM’s Power8 was specifically designed as a Big Data server with industry leading memory bandwidth, thread density, and cache architecture. It has the analytics tools[1], operating systems[2], databases[3] to be the System of Insight equipped to deal with the software, performance and management challenges of Big Data analysis, integration and governance.

And, in discussions with users, we’ve seen that it delivers. See our blogs about customer[4] success at dealing with Big Data challenges using Power8 systems.  Whether the goal is near real-time response (1.5 microsecond Algo-Logic’s Tick-to-Trade); significant cost savings with improved performance (IBM Platinum Partner Redis Labs processes more REDIS-NoSQL transactions with faster response times with fewer CAPI-Power8 servers); or  TalkTalk[5], a UK communications service provider, updating their network and improving the service to their customers by switching to Power-CAPI powered servers. 

No industry-standard benchmark existed for Apache Spark[6] until IBM developed the SparkBench benchmark suite. The first version includes 10 benchmarks covering four use cases: Machine Learning, Graph, SQL and Streaming Spark. The results are that a wide variety of Spark workloads consistently run 2x faster on POWER8 than competitor platforms. (FACT: POWER8 with 24 processor cores runs 37% faster than Haswell with 36 processor cores.) You can get SparkBench details and results here[7]. And, if you want to make sure that the SparkBench is the REAL thing, it is available to the public here[8]. IBM recently announced LinuxONE[9] for the mainframe world, we expect more interesting information in the October 5th webcast on new capabilities and products. We’ve registered and suggest that you do so also at:

[1] Hadoop, Big Insights, DB2BLU and Spark
[2] Red Hat, SuSE and Ubuntu
[3] Oracle, DB2LUW, MariaDB, MongoDB, PostgreSQL
[6] An in-memory distributed compute engine to complete analysis on large-scale data sets up to 100X faster than current technologies. More info on Apache Spark here: