At the NAB Show in April, the folks from Saturday Night Live delivered a great talk on their next-generation active archive. They are migrating to Cloudian object storage, and away from tape, a change that is already delivering operational benefits. This project is worth a close look, as it speaks to the unique attributes of object storage and why it’s ideally suited to media archive applications. (Storage Switzerland just posted a blog post on this talk here)
The talk was delivered by Matt Yonks, who has spent the last 19 years at SNL as their post-production supervisor. Matt discussed three fundamental challenges SNL was looking to solve, all of which are familiar to anyone managing a media archive:
- Scaling the environment: A perennial challenge for any archivist is ensuring sufficient capacity on hand. SNL has an impressive archive, with more than 40 years of history, all of which is now digitized. SNL has multiple PB’s of data, but the problem can be just as vexing in any studio. Many production companies now shoot exclusively in 4K, consuming a TB of storage with just 3 hours of shooting.
- Grappling with the chain of dependencies: Ensuring access to assets requires that all parts of the delivery stack work together. Application software, drivers, components and operating systems must all work together. If any part of the chain becomes obsolete or out-of-support, it can immediately affect access. For a media archive, where longevity and assured access are fundamental assumptions, this is an ongoing risk. You can keep certain systems “under glass” to maintain access, but this approach ultimately has its limits.
- Facilitating search: Matt put this succinctly in his talk. “An asset is only as good as your ability to find it.” The job of finding media is typically left to the media asset manager software, but this too has its limits. Assets move across regions. MAMs can themselves become obsolete. Making sure assets are findable, even after 20 or 40 years, is the ultimate goal of an archive.
Object storage addresses each of these with elements unique to this storage type.
Scalability: Object storage is not limited in scale. With a flat file layout and shared-nothing cluster architecture, capacity and performance both expand with added nodes.
Open architecture: The chain of dependencies that vexes other storage types becomes a non-issue with object storage for three reasons. First, object storage employs internet protocols and an API. There is no specific driver software. Second, objects themselves are portable. Users can, for example, migrate objects from Cloudian to Amazon S3 and immediately access them with cloud-based applications. Third, object storage is built on industry-standard hardware, thus eliminating dependence on proprietary hardware.
Metadata for search: Object storage is built for search. Each object includes metadata that describes the contents. Generated by applications or users, that metadata facilitates search irrespective of where the object is located. Whether on-prem or in the cloud, any specific asset can always be found with a search tool. In the case of SNL, their Evolphin MAM maintains a copy of the metadata set within the MAM itself, and a second copy with each object, thus ensuring long-term access and peace-of-mind for the archive manager.
The SNL example is a great use case that demonstrates the value of object storage and its key attributes — scalability, metadata, open architecture – in solving large-scale storage challenges.