Distributed storage system mush keep data available against several hardware/software failures. To achieve this, conventional distributed storage systems employ replication which is a simple but effective method to make distributed storage systems resilient. However, replicating a significant amount of data incurs substantial storage overheads. Therefore, erasure coding becomes an alternative to replication as it can offer the same reliability as or higher reliability than triple replication (denoted by “3-replication”), with much lower storage overheads. The Reed-Solomon (RS) code is one of the most popular erasure code due to its optimal storage space utilization. When an RS code is employed, all input data are stored as chunks with a fixed size. In general, RS(k,m) computes m numbers of coding chunk for k numbers of the data chunk, and the system distributes the data and coding chunks across different storage devices or nodes. For example, RS(6,3) splits original data into six data chunks and generates additional three coding chunks.

Block traces are widely used for system studies, model verifications, and design analyses in both industry and academia. While such traces include detailed block access patterns, existing trace-driven research unfortunately often fails to find true-north due to a lack of runtime contexts such as user idle periods and system delays, which are fundamentally linked to the characteristics of target storage hardware. Thus, we propose TraceTracker, a novel hardware/software co-evaluation method that allows users to reuse a broad range of the existing block traces by keeping most their execution contexts and user scenarios while adjusting them with new system information. Specifically, our TraceTraker’s software evaluation model can infer CPU burst times and user idle periods from old storage traces, whereas its hardware evaluation method remasters the storage traces by interoperating the inferred time information, and updates all inter-arrival times by making them aware of the target storage system. We apply the proposed co-evaluation model to 577 traces, which were collected by servers from different institutions and locations a decade ago, and revive the traces on a high-performance flash-based storage array. You can download the traces here.

We are collecting traces of KANDEMIR machine which is all-flash array based High Performance Compuing (HPC) testbed of CAMELab. Not only for normal user behaviors, also different kinds of emerging applications are executed for various workload scenarios. It will be prepared soon.