Gary Ogasawara, CTO, Cloudian
We recently announced general availability of Cloudian’s HyperStore object storage on VMware Cloud Foundation with Tanzu. HyperStore® is S3-compatible object storage, which now integrates with VMware’s vSAN Data Persistence platform to provide a containerized version of HyperStore managed by Kubernetes. In this blog, I describe the steps to set up HyperStore in this environment so that apps can consume S3 object storage. The setup can either use kubectl command-line operations or the vSphere Client User Interface (VC UI). Here we focus on using the VC UI. As a prerequisite, we assume VMware Cloud Foundation with Tanzu, including the vSAN Data Persistence platform, is installed. The vSAN Data Persistence platform enables HyperStore to use vSAN storage with a shared-nothing architecture (SNA) where the data durability and availability is managed by HyperStore instead of vSAN.
Enabling HyperStore
As a virtual infrastructure (VI) admin you can enable HyperStore in the VC UI Supervisor Services section. Enabling HyperStore triggers the creation of a new Kubernetes Namespace for HyperStore and the download and creation of a HyperStore Operator Pod and a HyperStore UI Plugin Pod. It also creates two vSAN storage policies (vSAN SNA, vSAN Direct) for HyperStore if those vSAN resources are available. Because these vSAN storage policies do not do data replication and rebuild, they are a good fit with software-defined storage software like HyperStore that itself manages data replication and rebuild.
In the VC UI, from the Workload-Cluster, select Configure → Supervisor Services → Services, select Cloudian HyperStore from the list of available services, and click ENABLE.
This brings up the Enable Supervisor Service screen where you can set the HyperStore Operator version and other parameters.
For the “Version” field pull-down menu, select “v1.0.0” or a later version. If you want to use a custom Docker image repository, then set the parameters for the Repository endpoint, Username, and Password. The images must have previously been stored in this repository. This method is how an air-gapped installation can be done. If the repository endpoint is not set, then the default is to use https://quay.io as the registry where the images are available.
Optionally, custom parameters can be added by setting the Key-Value pairs under “Advanced settings”. HyperStore supports timeout parameters before starting certain rebuild actions. The above figure shows a custom parameter “rebuildTimerEMM.” Details about the custom parameters can be found in the documentation, but for a standard installation, they can be left unspecified.
After the parameters screen, the Cloudian End-User License Agreement (EULA) URL is displayed. Click through and read the agreement carefully before selecting the checkbox to accept the terms of the license agreement and clicking “FINISH”.
HyperStore is now enabled. This creates a new Kubernetes namespace prefixed with “hyperstore-domain-” and starts a HyperStore Operator Pod and UI Plugin Pod in that namespace. In the below picture in the left pane, the newly created namespace “hyperstore-domain-c8” and the Operator and UI Plugin Pods can be seen.
The HyperStore Operator Pod uses the Operator SDK to manage HyperStore using Kubernetes principles, notably a control loop to reconcile desired and current states.
The UI Plugin Pod implements a VC UI plugin to configure and monitor HyperStore. For example, the UI Plugin is used to configure and create a new HyperStore Instance.
Creation of a HyperStore Instance
A HyperStore Instance is a StatefulSet of Pods that store data using replication and/or erasure coding, providing object storage capabilities with an S3 API. Using VC UI and the underlying Kubernetes infrastructure, a new HyperStore Instance can be created simply.
Role-based access control (RBAC) is enforced at the Kubernetes Namespace level. In VC UI, a Namespace can be created and then configured for permissions and storage policies. The “edit” permission in the Namespace is required for a user to create a HyperStore Instance in that Namespace. For storage policies, two HyperStore-specific storage policies are available for vSAN Direct and vSAN SNA policies.
A new HyperStore Instance is created by using the VC UI under the VC cluster Configure → Cloudian HyperStore → Overview, and then clicking “NEW INSTANCE”.
As an alternative to the UI Plugin, a new HyperStore Instance can be created by kubectl apply of a Custom Resource (CR) file that has the configuration parameters to use. Below is an example CR file:
After entering the parameters and clicking “OK”, a new HyperStore Instance is created in the specified Namespace with a limited and temporary HyperStore license. The HyperStore image is downloaded from the image registry and used in a new HyperStore Pod. Once the image is downloaded and started, the Pod transitions from Pending to Running status, and the HyperStore installation process starts to create and configure the additional Pods in the StatefulSet.
One function of the HyperStore Operator is to report on the cluster health status that the VC UI monitoring uses. During the initial installation, the health status is RED. When the instance’s health status changes to GREEN, the HyperStore Instance is ready for S3 traffic.
The vSAN Data Persistence platform on VMware Cloud Foundation with Tanzu provides a powerful framework to deploy and manage HyperStore S3 object storage. As a foundation, Kubernetes provides functions like auto-scaling, resource scheduling, and role-based access control. Layering on VMware’s vSAN Data Persistence platform enables efficient use of vSAN storage with management functions like maintenance mode and health monitoring. The result is an environment for apps managed within VMware Cloud Foundation with Tanzu where HyperStore S3 object storage can be created and monitored from the VC UI, a convenient “single pane of glass.”
To see a demo of this new combined solution, go to cloudian.com/resource/demos-and-videos/demo-vmware-vsan/.
To learn more about Cloudian solutions for VMware environments, go to cloudian.com/vmware/.
