A prominent telecommunications company in the United States, serving a massive customer base in the wireless segment, faced a significant challenge in managing its data. With the need to store and analyze data from hundreds of millions of wireless calls daily, the company required a robust and scalable storage solution to manage log data for analytics purposes.
Solution Exploration
The company considered various storage options, including on-premises solutions and cloud storage. However, due to the prohibitive costs associated with cloud storage for such vast amounts of data, the focus shifted to an object storage solution that promised cost-effective scalability. The company evaluated several technologies, including Pure Storage, NetApp Storage Grid, IBM Cloud Object Storage, and MinIO, before making a decision.
Implementation
After a thorough assessment, the firm opted for a Cloudian data lake, which offered limitless scalability and was compatible with the company’s existing data management and analysis tools, Apache Druid, and MemSQL. Cloudian stood out for its high level of S3 API compatibility, seamless scalability, and its proven track record in large-scale data management use cases.
Seamless expansion over four years
Beginning with a small number of storage nodes, the data lake deployment ultimately grew over a period of four years to include a total of 65 Cloudian appliances across three sites with nine storage clusters and a total capacity of 28PB. The majority of the appliances employed HDD-based storage for the lowest TCO. For the MemSQL use case the firm leveraged a combination of flash-based appliances to enhance performance and HDD-based appliances. For the Druid use case, 200 servers running Druid were connected to 14 nodes of Cloudian storage providing 4PB of usable capacity.
To provide storage infrastructure observability and analytics, Cloudian HyperIQ was employed, enabling the firm to monitor and analyze the infrastructure effectively.
Conclusion
The company successfully established a scalable and cost-effective data analytics infrastructure with Cloudian, demonstrating superior performance and durability in a large-scale production environment. This strategic implementation has enabled the firm to retain and analyze vast amounts of call data efficiently, leveraging the full potential of their chosen analytics tools and significantly enhancing their data management capabilities.
