Modern vector databases like Milvus are increasingly being deployed at enterprise scale, where traditional local storage solutions may not meet the demands for durability, scalability, and cost-effectiveness required by AI and machine learning workloads.

Cloudian HyperStore, with its S3-compatible object storage architecture, provides an ideal backend storage solution for Milvus deployments that need to handle massive vector datasets while maintaining high availability and seamless horizontal scaling.

By configuring Milvus to leverage HyperStore’s distributed storage capabilities, organizations can achieve the perfect balance of performance and economics—storing frequently accessed vectors in high-speed tiers while automatically tiering older or less critical data to more cost-effective storage layers, all while maintaining the familiar S3 API compatibility that simplifies integration and reduces operational complexity.

Installing Milvus via Helm with HyperStore

These steps assume that you already have a kubernetes cluster setup and helm installed.

Collecting Hyperstore Configuration

We’ll need to create a bucket for Milvus to store its objects in, and we’ll need the S3 access credentials with access to that bucket. Store these, along with the other details of your Hyperstore in a Helm values.yaml which we’ll use later to install Milvus. For example:

$ cat values.yaml
externalS3:
  enabled: true
  host: my-hyperstore.cloudian.com
  port: 80
  useSSL: false
  bucketName: milvus-bucket
  accessKey: 5a0e021c09aaabe3344f
  secretKey: BBTRVwA0cKTMGP0BnevKLz6474SGFqSahDslV6JX
  region: region-1
minio:
  enabled: false

Alternatively, sensitive fields can be left out of the values.yaml and specified later using the –set option when we install the helm chart.

Installing Milvus

Add the Milvus Helm Repository:

helm repo add milvus https://zilliztech.github.io/milvus-helm/
helm repo update

Next we just need to install Milvus, either distributed or standalone. If you missed out sensitive fields earlier, these can be specified as shown in the below example commands.

# Distributed
helm install -f values.yaml mymilvus milvus/milvus

# Distributed with sensitive fields
helm install -f values.yaml mymilvus milvus/milvus --set externalS3.secretKey=$MY_SECRET_KEY

# Standalone
helm install -f values.yaml mymilvus milvus/milvus --set cluster.enabled=false --set etcd.replicaCount=1 --set pulsar.enabled=false

Further Reading

For more configuration options when installing Milvus via Helm: https://milvus.io/docs/install_cluster-helm.md

As an alternative to helm, install with object storage via the Milvus Operator: https://milvus.io/docs/object_storage_operator.md

Helm Chart Readme on external object storage: https://artifacthub.io/packages/helm/milvus-helm/milvus#milvus-with-external-object-storage

Learn more at cloudian.com

Or, sign up for a free trial