Replication Factor Storage Policy for Multi-Cloud
When creating a multi-cloud environment, securing your data across more than one cloud is the name of the game, as this gives resilience to an entire-cloud failure. In this blog, I will explain how Cloudian Hyperstore protects your data automatically using data replication (RF).
Cloudian HyperStore Supports Replication Factor 3
Cloudian HyperStore supports multiple storage protection policies to help protect your data. The simplest mechanism for protecting your data is to make multiple copies. The number of copies you create is called a replication factor or RF. The smallest RF supported by HyperStore is RF3 (three copies of each object).
These three copies can be stored anywhere nodes are located, either all in a single DC (which is common for on-prem deployments), or across multiple sites (as in a multi-cloud deployment).
Storage policies can be implemented on a per-bucket basis, which ensures that your protection profile matches the specific business requirement for that data.
Customizing HyperStore Replication Factor Policies
Sometimes, higher levels of data availability or durability are required. Customisable policies with higher levels of redundancy, such as RF4, 5, and 6, can be created to obtain stronger protection at the expense of storage efficiency. In the example in the video below, we use RF3 with one copy residing in each public cloud provider. This RF3 allows for one public cloud to fail whilst still allowing read and write operations.
Configuring Consistency Settings for HyperStore Multi-Cloud
Consistency settings allow fine-grained control of read/write behaviour. With RF3, a minimum quorum of two copies is required to accept a write, but you can select a read with only one copy available. With this setting, two public cloud providers could be down, but we could still read data back. You would, however, still require a second available public cloud to write data.
There are many more configurations possible for this, depending on the number of data centres and the protection scheme used. But for RF multi-cloud clusters, these are the options:
- All – All objects must be written
- Each Quorum – Each of the DC’s systems must succeed in reading/writing a majority of object replicas
- Quorum – The system must succeed in reading/writing a majority of object data replicas
- Local Quorum – The system must succeed in reading/writing a majority of the local DC data replicas
- One (reads only) – The system must succeed in reading one object data replica
This is a simplified version of consistency settings, which is a complex topic that could justify a blog post of its own.
Other Storage Configuration Options
Other storage options such as compression and encryption can also be specified, along with application-specific policies depending on the data being stored.
Data replication storage policies are the simplest (but not necessarily the most efficient) way to store data across multiple clouds whilst offering protection against cloud failure. They are highly configurable to suit individual deployment characteristics or the data profile being protected. They are commonly used within each cloud provider to protect their own data.