Bringing the Power of HPC to Big Data
General Dynamics, making use of an Internal Research and Development (IRAD) investment, has combined the processing power of HPC with customized open source technologies to provide real-time analysis and understanding of Big Data.
Key results were:
- Achieved a 10x performance improvement in the advanced processing of very large and dynamic data sets
- Customized Apache Hama from the distributed programming model to the parallel programming model that is best utilized in HPC systems
- Displayed data using a 3D visualization tool that allows users to look at vast quantities of data quickly, helping to accelerate discovery and insight.
For a more detailed look at the methodology and results, download the Experiences Porting the Apache Hama Graph Analytics Framework to an HPC InfiniBand Connected Cluster white paper.