DataDirect Networks Annual High Performance Computing Storage Trends Survey Reveals Why Users Look to DDN to Solve Their Toughest Big Data Problems

HPC End Users Confirm Mixed IO is the Hardest Problem to Solve Locally, and that Security and Data Sharing Complexity are the toughest problems for Multisite Collaboration

News provided by

DataDirect Networks (DDN)

Nov 17, 2014, 08:30 ET

NEW ORLEANS, Nov. 17, 2014 /PRNewswire/ -- SC14 (Booth #2539)

News Facts

Addressing the mixed I/O demands on the storage infrastructure of local High Performance Computing (HPC) environments continues to be the biggest challenge for HPC storage architects as they look to solve their Big Data and storage needs in 2015 according to the results of a DataDirect Networks (DDN) High Performance Computing (HPC) Trends survey, released today.
Conducted by TechValidate, a leading customer insights firm, the survey polled a cross-section of 112 HPC end users representing hundreds of petabytes of storage investment, which included individuals responsible for HPC allocations comprising high availability storage and file systems across a variety of industries. Of those surveyed, 36 percent manage or use at least one petabyte of storage, and over a quarter manage or use 6 PBs or more.
According to the survey results, more than two-thirds (67%) of those polled agree that performance is the number one storage and Big Data challenge with nearly half the respondents citing mixed I/O performance as the biggest bottleneck. In comparison to last year's survey results, this represents a 13 percent increase indicating that performance related issues have surged in the face of increasing IO demand.
In addressing this challenge, three quarters of respondents believe that burst buffers are the most likely technology to push storage to the next level, with response rates significantly higher (79%) for sites with over 6PB of storage. This suggests that planning for exascale, accelerating time to discovery and extracting results from massive data sets is exhausting the capabilities of today's architectures, leading more than 25 percent of respondents to seek faster and more efficient alternatives for provisioning I/O and accelerating applications.
In terms of remote sites, while multi-site HPC collaboration may be on the rise, security and data sharing complexity remain the biggest impediments according to 75 percent of those surveyed, with the issue becoming even more serious (78%) among respondents using or managing more than 6PBs of storage.

Tweet This: Mixed I/O is No. 1 #BigData challenge in @DDN_Limitless #HPC customer survey w/burst buffer tech a possible panacea http://bit.ly/1zukpxu

User Insights Reveal that Storage Performance Continues to be Critical for HPC and Analytics Workflows

High Performance Computing (HPC) is a unique environment that places special demands on the storage infrastructure, creating a slew of storage and Big Data challenges for HPC users. Looking to 2015 more than half of the survey respondents cite mixed I/O performance the primary issue (51%), with managing rapid data growth and managing bandwidth performance tying for second place at almost 20% each.
The storage I/O challenge extends to a variety of workflows including analytics with nearly three quarters (74%) of those polled indicating their analytics systems are hitting a wall as they approach Big Data scale, suggesting that performance bottlenecks of MapReduce is no longer the main challenge for these platforms.
With storage performance a critical requirement especially at today's petascale levels, of those respondents with an opinion, over 80 percent agree that site-wide file systems like those at Los Alamos National Labs, The University of Florida's Interdisciplinary Center for Biotechnology Research (ICBR) and Indiana University continue to be a significant infrastructure trend in HPC environments, allowing HPC architects to either consolidate multiple computers on the same system and/or the flexibility to upgrade storage and servers as needed.
Finally, despite recent economic recovery and some increase in grant funding, HPC organizations are still under increasing funding scrutiny, where, according to nearly three quarters of the survey respondents, it continues to be difficult for HPC sites to obtain funding for compute infrastructure without demonstrating a clear return on investment (ROI).

Supporting Quotes

Molly Rector, Chief Marketing Officer, DDN: "The results of DDN's annual HPC Trends Survey reiterate what HPC end users continue to tell us – storage I/O is a strategic issue for HPC sites and becoming more critical every year. Users rely on DDN to deliver continued technology innovation which enables research and business organizations to accelerate business insights and in turn bottom line results, year-over-year. DDN is focused on conquering the I/O performance, content distribution and collaboration demands of the HPC industry faster, smarter, more efficiently and most cost effectively."

Supporting Resources

Survey results available upon request at [email protected]
Connect with DDN at SC14
More on DDN HPC Storage Products
Follow DDN via Blog and Twitter

DataDirect Networks
DataDirect Networks (DDN) is a leader in data center infrastructure for Big Data and Cloud Storage applications. Our data storage and processing solutions and professional services enable content-rich and high growth IT environments to achieve the highest levels of systems scalability, efficiency and simplicity. DDN enables enterprises to extract value and deliver results from their information. Our customers include the world's leading online content and social networking providers, high performance cloud and grid computing, life sciences, financial services, media production, and security and intelligence organizations. Deployed in thousands of mission critical environments worldwide, DDN's solutions have been designed, engineered and proven in the world's most scalable data centers to ensure competitive business advantage for today's information powered enterprise. For more information, go to www.ddn.com or call +1-800-TERABYTE (837-2298).

DDN Media Contact:
Dani Kenison
Sr. Director, Global Corporate Communications
Email: [email protected]
Work: 408-990-2658

Logo - http://photos.prnewswire.com/prnh/20130223/MM65263LOGO

SOURCE DataDirect Networks (DDN)