Replacing old DIVA installations with XenData - Migrating from DIVA

Comparing XenData X100 and DIVA Video Archives

DIVA LTO archives use overly complicated architectures and consequently have extremely high support costs. We learned recently that Telestream, the latest owner of DIVA, has greatly increased the cost of DIVA support. Now is the time for existing DIVA customers to switch to a XenData LTO archive. Typically, the cost of the XenData replacement is paid for within a year or two by savings in the astronomical DIVA support costs.

Introduction

This white paper is aimed at readers who are familiar with DIVA and want to know how XenData X100 archives compare.

DIVA was first developed in the 1990s at StorageTek, before being spun-out into a new company, Front Porch Digital, in 2000. It was acquired by Oracle in 2014, then sold to a start-up, EcoDigital LLC which, in turn, was purchased by Telestream in October 2020. The DIVA product suite is well respected but employs an antiquated and complex architecture that requires many servers to achieve acceptable performance. This in turn means it has a high on-going support cost.

In contrast, XenData X100 Archives employ a simple architecture with a standard Windows Server 2022 cluster and a high-performance RAID cache. The XenData approach takes advantage of the massive advances that have occurred in computer technology over the last two decades since the DIVA architecture was conceived. Furthermore, XenData takes advantage of modern industry standards, including offering a high-performance file system interface, compliance with Active Directory and use of the industry standard LTFS tape format.

Architectures

A typical DIVA system has a pair of metadata servers that keep track of file locations in its database and control the robot within an LTO library. In addition, there are multiple servers for data moving. These are termed ‘Actors’ and they connect directly to the LTO drives within the LTO library. The multiple DIVA servers communicate with each other through a TCP/IP control network.

The XenData X100 is available as a clustered system with no single point of failure. An example configuration is illustrated to the right which shows an X100 system managing two LTO libraries and mirroring files across them. Other configurations with a single enterprise- class library are supported.

When archiving files with a XenData system, they are first written to RAID and then automatically to a designated group of LTO cartridges.

Archive policies are highly configurable, allowing different folders to be written to specified groups of tapes. LTO cartridges may be automatically replicated, either within a library or across different libraries. And disk retention policies allow specified file types and folders to be given different retention periods on the external RAID.

When restoring files, they are read directly from LTO, unless an instance of the file is available from the RAID, in which case it will be read directly from the disk array. The system is optimized for large numbers of simultaneous restores from multiple LTO drives.

A XenData Archive is fully compliant with the Microsoft security model based on Active Directory and is easily installed into an existing Windows Domain.

Interfaces

The XenData X100 offers three interfaces which can be used simultaneously:

• ➢ File-Folder Interface which scales to billions of files and hundreds of petabytes. Writing to and reading from this interface is just like writing to a disk-based volume. It supports SMB, NFS, FTP network protocols as well as local file transfers.
• ➢ Object Storage Interface which is ideal for remote access. It has an S3 interface, supporting HTTPS and HTTP protocols. It presents all or a portion of the archive as a private cloud.
• ➢ XML API which provides frame accurate timecode based partial file restores for video archives. The XML instructions include the ability to pull assets from a source location and push them back to that location or another destination.

DIVA has APIs for archiving and restoring assets, where an asset is a group of files or just a single file. The DIVA API can restore assets to multiple end-point locations. The same is true for a XenData archive, using the XML API.

MAM Intergration

Many Media Asset Management systems have been modified to work with DIVArchive. In addition, the optional DIVA View is a simple MAM that works with the core DIVA archive.

XenData does not offer a MAM itself but X100 systems are supported by a wide range of media asset management systems.

XenData also has many proven integrations with leading MAMs, including with Cinegy, Dalet Galaxy and Flex, Interplay Production1, Vizrt and VSN MAM. In many cases, the MAM uses XenData’s Workflow API but, in others, the integration is simply via XenData’s file system interface.

LTO Formats

Early DIVA installations wrote to LTO cartridges using proprietary formats, unique to Front Porch Digital (FPD). From late 2008, FPD lead a standardization effort to create a standard format that became AXF. Front Porch eventually released their implementation in April 2011. Unfortunately, AXF obtained minimal adoption outside of FPD.

In April 2010, one year prior to FPD’s release of AXF, IBM launched the LTFS interchange format for LTO. LTFS is an acronym for Linear Tape File System and it was quickly adopted as the industry standard. It is used by HPE, IBM, Masstech, Oracle StorageTek, Quantum, XenData and many other providers of LTO systems.

LTO cartridges written using the LTFS format can be exchanged between any system that supports this industry standard. By adopting LTFS, XenData has avoided locking its customers into a non-mainstream LTO format and makes it easy to switch to a different system supplier in the future.

Cloud Support

DIVA allows archived assets to be stored in public clouds. It supports Partial File Restore which is an important characteristic when Internet bandwidth is limited and high egress (download) charges apply.

A XenData Archive system may be extended by adding a XenData Cloud File Gateway. Such a system supports both LTO and cloud object storage from multiple public cloud offerings including Amazon Web Services S3, Microsoft’s Azure blob storage and Wasabi S3. The XenData Cloud File Gateway supports Partial File Restore and streaming of files. It is ideal for storing a DR copy of the archive in the cloud and for sharing all or select assets from the archive.

Syncronizing Across Locations

DIVAnet connects independent DIVArchive systems and provides a replicate feature that allows content mirroring between two DIVArchive systems.

XenData goes one step further with Multi-Site Sync for LTO, a synchronization service that links multiple archives in different locations, creating a single global file system accessible anywhere worldwide. As soon as a file is archived to LTO at one location, it becomes available as a stub file within the global file system. When a user makes a change by writing, overwriting, or deleting a file, that change is propagated to all locations. This provides a consistent up-to-date set of files across the entire distributed organization. When files are restored from another location, they are transferred directly using peer to peer multi- threaded HTTPS which delivers secure fast file transfers. The Multi-Site Sync service uses a cloud database, but the files themselves are never stored in public cloud object storage, avoiding cloud storage and egress fees. Furthermore, because the service only synchronizes file system metadata, it requires minimal Internet bandwidth.

About XenData

XenData specializes in developing and providing massively scalable active archive systems. XenData LTO archive systems using robotic libraries are installed in over 90 countries.

Customers using XenData products for media and entertainment storage applications include TV stations, video production companies, post-production organizations and the media departments of large corporations and government organizations.

Contact your BackupWorks Account Rep today and ask about Xendata at 866 801 2944