The research group of Xian-He Sun, Distinguished Professor of Computer Science, has received a grant from the National Science Foundation (NSF) for the project, “Framework: Software: NSCI: Collaborative Research: Hermes: Extending the HDF Library to Support Intelligent I/O Buffering for Deep Memory and Storage Hierarchy Systems.” The award is expected to total $3 million over four years with $2.85 million going to Illinois Tech and $150,000 going to University of Illinois Urbana-Champaign.

Big data applications, as their name states, have big data. These big data must be maintained, managed, and operated on computer storage systems via input/output (I/O) systems. Performance improvement of disk-based storage systems has been much slower than that of memory, creating a significant I/O performance gap. To reduce this gap, storage subsystems are undergoing extensive changes, adopting new technologies, and adding more layers into the memory/storage hierarchy. With a deeper memory hierarchy, the data movement complexity of memory systems is increased significantly, making it harder to utilize the potential of the deep memory and storage hierarchy (DMSH) design.

The main objective of this project is to develop an I/O buffering system that will utilize DMSH and significantly accelerate I/O performance. Sun’s group will collaborate with the HDF Group, the developer of the widely used HDF5 high performance I/O library, to extend the library with their newest findings and to transform their methodologies into HDF5 software libraries to benefit the general big data and high-performance computing community.