Reduce Splunk storage costs by 70% while increasing storage scalability. Splunk’s new SmartStore feature plus Cloudian HyperStore provides an exabyte-scalable, on-prem storage pool that is separate from your Splunk indexers.

The Challenge

Data growth and the need to extract timely insights from this increasingly strategic asset are two of the biggest challenges for enterprises today. The growth of machine and unstructured data is breaking the distributed scale-out mode that combines compute and storage in the same devices and limits your flexibility to choose the ideal resources for each. This drives up costs and makes it more difficult to accommodate growing data storage and analytics demands.

Solution

Splunk SmartStore and Cloudian together let you decouple the compute and Splunk storage layers, so you can independently scale those resources to best serve workload demands. With SmartStore, Splunk Indexers retain data only in hot buckets that contain newly indexed data. Older data resides in the warm buckets in the on-prem Cloudian storage.

Splunk SmartStore manages where data is stored. It moves data among indexers and the external Cloudian storage based on data age, priority and users’ access patterns.

Splunk storage with SmartStore for scalable storage in a single data lake

Unlike other types of Splunk storage (such as NAS or SAN), Cloudian offers modular growth, letting you expand from terabytes to an exabyte without disruption. Embedded data redundancy features provide up to 14 nines data durability, removing the necessity of a separate data backup process. Compared with traditional enterprise storage — or with storage on compute-intensive servers — Cloudian saves up to 70% on TCO. Cloudian saves on space, too, with the industry’s highest density: up to 840TB capacity in each 4U high chassis.

Cloud Enabled

Cloudian is on-prem Splunk storage. It lets you start small and scale, growing non-disruptively to an exabyte in your data center.

Cloudian can also integrate with public cloud storage services. Employ policy-based tools to replicate or tier data to AWS, GCP or Azure for offsite DR, capacity expansion or data analysis in the cloud. This built-in capability requires no additional software or licenses.

Secure

To ensure data security, Cloudian HyperStore provides AES-256 server-side encryption for data at rest, SSL for data in transit (HTTPS), role-based access controls and storage policies that can be applied at an object and bucket-level.

Multi-Purpose Scalable Storage

Cloudian offers the industry’s most compatible S3 API, so it integrates seamlessly with S3-compatible applications. In addition to Splunk SmartStore, it also provides scalable storage for data management applications from Rubrik, Veeam, Commvault, Veritas, Pure Storage, Quantum, and others.

Deploy as Software or Appliances

Cloudian storage for Splunk is available as preconfigured appliances, with capacities from 96TB to 1.5PB. It can also be deployed as software for either bare metal servers or VMs.

Cloudian HyperStore

Cloudian HyperStore delivers a scalable, secure, highly resilient and flexible Splunk storage infrastructure at the industry’s lowest TCO. With a single namespace across all locations, HyperStore provides unified visibility and management of all data, with granular, bucket-level storage policies.

Cloudian deploys easily, beginning with configurations of just three nodes and scaling to hundreds without disruption. The modular scaling model eliminates costly over-provisioning and time-consuming data migrations.

HyperStore’s automatic data verification and self-healing functions provide reliability and resilience against hardware failures, while its data encryption, both in-flight and at rest, safeguard valuable assets against threats of deletion or theft via malware. HyperStore provides unmatched data durability with flexible, policy-based data protection options including erasure coding and replication across clusters and sites.

Splunk Storage with SmartStore

As Splunk storage volume increases, storage demand often outpaces demand for compute resources. Splunk SmartStore lets you easily add storage without adding compute cost.  You can manage Splunk storage and compute resources in a cost-effective manner by scaling those resources separately. Data may reside either locally on indexers or on Cloudian’s remote storage tier, with data movement between the two managed by the SmartStore cache manager.

Indexing and searching of data occur on the indexer, just as in a traditional deployment. The key difference with SmartStore is that the remote Cloudian HyperStore object store becomes the location for master copies of warm buckets, while the indexer’s local storage is used to cache copies of warm buckets currently participating in a search or that have a high likelihood of participating in a future search.

SOLUTION BENEFITS

  • Independently scale compute and Splunk storage resources
  • On-prem data lake with exabyte scalability
  • Data location is managed by Splunk
  • Modular design for non-disruptive storage growth
  • Built-in data protection; up to 14 nines data durability
  • 70% TCO savings vs indexer-based storage
  • Public cloud compatible; integrates seamlessly with AWS, GCP, and Azure
  • Hybrid cloud-ready for DR
  • Faster recovery from indexer node failure
  • Simplified indexer updates; new cluster can inherit data from the original

YOU MAY ALSO BE INTERESTED IN

Making Big Data Analytics Smarter with Cloudian and Cloudera
Optimize your Big Data Analytics environment for performance, scale, and economics