SNIA: LTFS Format Spec V2.5 for Magnetic Tape Approved

Storing large number of files on single large capacity tape cartridge

The SNIA LTFS (Linear Tape File System) Technical Work Group (TWG) announces that the new version of LTFS Format Specification has just been approved.

LTFS provides an industry standard format for recording data on magnetic tape. It is a file system that allows those stored files to be accessed in a similar fashion to those on disk or removable flash drives.

The SNIA standard, also known as an ISO standard ISO/IEC 20919:2016, defines the LTFS format requirements for interchanged media that claims LTFS compliance. Those requirements are specified as the size and sequence of data blocks and file marks on the media, the content and form of special data constructs (the LTFS Label and LTFS Index), and the content of the partition labels and use of MAM parameters.

The data content (not the physical media) of the LTFS format shall be interchangeable among all storage systems claiming conformance to this format. Physical media interchange is dependent on compatibility of physical media and the media access devices in use.

Q&A with Takeshi Ishimoto, co-chair, LTFS Technical Work Group, to learn what it means.

Q. What is this standard all about?
A. LTFS utilizes modern tape storage hardware to store files and their metadata on the same tape and makes the data accessible through the file system interface. The SNIA LTFS Format Specification standard defines the data recording structure on tape and the metadata format in XML, so that the LTFS-based tape is interchangeable between LTFS implementations. Because LTFS is an open and self-describing format, the data on tape are transportable and accessible across different locations and on different OS platforms, and thus the tape storage becomes suitable for long term archival and bulk transfer at lower cost.

What are the new revisions?
The LTFS Format Specification version 2.5 extends the standard to use a new incremental index method, in addition to the legacy full index method. This new incremental index method gives a journal-like capability where only changes to the index need to be recorded frequently. A complete index is required to be written at unmount time (but may also be written at any other time).

What are the benefits of these revisions to the end users?
The incremental index will improve the TCO by reducing the tape space occupied by the indexes, thereby allowing more user data to be stored on tape. It also improves the overall write performance of the file system by reducing the time required to write the index to tape. With the incremental index method, LTFS can be applied for the user with many small files, such as the user managing the data from IOT sensors, without compromising the space efficiency and performance. The incremental index method has been designed to be backwards compatible with previous generations for all normal usage.

What problems will this new 2.5 revision solve?
With the evolution of tape-recording density, current tape hardware on the market can store >10TB in a single palm-size tape cartridge, and future recording technology is projected to go beyond 300TB in the same format factor. The new LTFS Format Specification version 2.5 addresses the challenges in storing a large number of files on a single large capacity tape cartridge. LTFS is widely adopted by various tape hardware vendors as well as many software vendors.