Efficient Journaling for the Spider Storage System

Abstract

Journaling is a widely used technique to increase file system robustness against meta data and/or data corruptions. While the overhead of journaling can be negligible for small-scale file systems, we found that two aspects of local back-end file system journaling significantly lower the overall performance of a large-scale parallel file system such as Lustre: extra head seeks and serialization of incoming client requests. Journal transactions reside on a separate area of the disk that the file data, and each commit of the journal requires a head seek. Incoming client requests become serialized and take a latency hit by waiting for a commit to occur before the reply is sent. In this paper we present two different approaches to increase the local back-end file system journaling efficiency, thus increasing the overall aggregate parallel file system efficiency. First, we present a hardware-based solution where a solid-state device is used as an external journaling device to minimize the disk head seek. Second, we introduce a software-based optimization to allow asynchronously commit multiple journal transactions on the local back-end file system to minimize the penalty of serialization. Both our solutions are experimentally tested on Oak Ridge National Laboratory's large-scale Spider storage system and our tests show that both methods nearly double the overall parallel write performance.