Modern vector databases like Milvus are increasingly being deployed at enterprise scale, where traditional local storage solutions may not meet the demands for durability, scalability, and cost-effectiveness required by AI and machine learning workloads.
Cloudian HyperStore, with its S3-compatible object storage architecture, provides an ideal backend storage solution for Milvus deployments that need to handle massive vector datasets while maintaining high availability and seamless horizontal scaling.
By configuring Milvus to leverage HyperStore’s distributed storage capabilities, organizations can achieve the perfect balance of performance and economics—storing frequently accessed vectors in high-speed tiers while automatically tiering older or less critical data to more cost-effective storage layers, all while maintaining the familiar S3 API compatibility that simplifies integration and reduces operational complexity.
Installing Milvus via Helm with HyperStore
These steps assume that you already have a kubernetes cluster setup and helm installed.
Collecting Hyperstore Configuration
We’ll need to create a bucket for Milvus to store its objects in, and we’ll need the S3 access credentials with access to that bucket. Store these, along with the other details of your Hyperstore in a Helm values.yaml which we’ll use later to install Milvus. For example:
$ cat values.yaml externalS3: enabled: true host: my-hyperstore.cloudian.com port: 80 useSSL: false bucketName: milvus-bucket accessKey: 5a0e021c09aaabe3344f secretKey: BBTRVwA0cKTMGP0BnevKLz6474SGFqSahDslV6JX region: region-1 minio: enabled: false
Alternatively, sensitive fields can be left out of the values.yaml and specified later using the –set option when we install the helm chart.
Installing Milvus
Add the Milvus Helm Repository:
helm repo add milvus https://zilliztech.github.io/milvus-helm/ helm repo update
Next we just need to install Milvus, either distributed or standalone. If you missed out sensitive fields earlier, these can be specified as shown in the below example commands.
# Distributed helm install -f values.yaml mymilvus milvus/milvus # Distributed with sensitive fields helm install -f values.yaml mymilvus milvus/milvus --set externalS3.secretKey=$MY_SECRET_KEY # Standalone helm install -f values.yaml mymilvus milvus/milvus --set cluster.enabled=false --set etcd.replicaCount=1 --set pulsar.enabled=false
Further Reading
For more configuration options when installing Milvus via Helm: https://milvus.io/docs/install_cluster-helm.md
As an alternative to helm, install with object storage via the Milvus Operator: https://milvus.io/docs/object_storage_operator.md
Helm Chart Readme on external object storage: https://artifacthub.io/packages/helm/milvus-helm/milvus#milvus-with-external-object-storage
Learn more at cloudian.com
Or, sign up for a free trial