"Scalable parallel I/O on a Blue Gene/Q supercomputer using compression, topology-aware data aggregation, and subfiling"

Scalable parallel I/O on a Blue Gene/Q supercomputer using compression, topology-aware data aggregation, and subfiling

Bui, H., Leigh, J., Vishwanath, V., Finkel, H., Habib, S., Heitmann, K., Papka, M., Harms, K.

• Publication: Turin, Italy
• Publication: www.pdp2014.org/index.html
• PDF: pid3018107_huybui.pdf

IBM Blue Gene/Q at Argonne National Laboratory

In this paper, we propose an approach to improving the I/O performance of an IBM Blue Gene/Q supercomputing system using a novel framework that can be integrated into high performance applications. We take advantage of the systems’ tremendous computing resources and high interconnection bandwidth among compute nodes to efficiently exploit I/O bandwidth. This approach focuses on loss-less data compression, topology-aware data movement, and subfiling. The efficacy of this solution is demonstrated using micro-benchmarks and an application-level benchmark.