5 Reasons to Adopt Hybrid Cloud Storage for your Data Center

Are you weighing the benefits of cloud storage versus on-premises storage? If so, the right answer might be to use both–and not just in parallel, but in an integrated way. Hybrid cloud is a storage environment that uses a mix of on-premises and public cloud services with data mobility between the two platforms.

IT professionals are now seeing the benefit of hybrid solutions. According to a recent survey of 400 organizations in the U.S. and UK conducted by Actual Tech, 28 percent of firms have already deployed hybrid cloud storage, with a further 40 percent planning to implement within the next year. The analyst firm IDC agrees: In its 2016 Futurescape research report, the company predicted that by 2018, 85 percent of enterprises will operate in a multi-cloud environment.

Hybrid has piqued interest as more organizations look to the public cloud to augment their on-premises data management. There are many drivers for this, but here are five:

  1. We now have a widely-accepted standard interface.

The emergence of a common interface for on-prem and cloud storage changes everything. The world of storage revolves around interface standards. They are the glue that drives down cost and ensures interoperability. For hybrid storage, the defacto standard is the Amazon S3 API, an interface that began in cloud storage and is now available for on-premises object storage as well. This standardization is significant because it gives storage managers new flexibility to deploy common tools and applications on-prem and in the cloud, and easily move data between the two environments to optimize cost, performance, and data durability.

  1. Unprecendented hybrid scalability delivers operational efficiency.

Managing one large, scalable pool of storage is far more efficient than managing two smaller ones. And hybrid storage is hands-down the most scalable storage model ever devised. It combines on-prem object storage – which is itself scalable to hundreds of PBs – with cloud storage that is limitlessly scalable, for all practical purposes. This single-pool storage model reduces data silos, and simplifies management with a single namespace and a single view — no matter where the data originated or where it resides. Further, hybrid allows you to keep a copy of all metadata on-premises, ensuring rapid search across both cloud and on-premise data.

  1. Best-of-breed data protection is now available to everyone.

Data protection is fundamental to storage. A hybrid storage model offers businesses of all sizes incredible data protection options, delivering data durability that previously would have been affordable to only the most well-heeled storage users. In a hybrid configuration, you can backup data to object storage on premises, then automatically tier data to the cloud for long-term archive (Amazon Glacier, Google Coldline, Azure Blob). This gives you two optimal results: You have a copy of data on-site for rapid recovery when needed, and a low-cost, long-term archive offsite copy for disaster recovery. Many popular backup solutions including Veritas, Commvault and Rubrik provide Amazon S3 connectors that enable this solution as a simple drop-in.

  1. Hybrid offers more deployment options to match your business needs.

Your storage needs have their own nuances, and you need the operational flexibility to address them. Hybrid can help with more deployment options than other storage models. For the on-premise component, you can select from options that range from zero-up-front cost software running on the servers you already own, to multi-petabyte turnkey systems. For the cloud component, a range of offerings meet both long-term and short-term storage needs. Across both worlds, a common object storage interface lets you mix-and-match the optimal solution. Whether the objective is rapid data access on-premises or long-term archival storage, these needs can be met with a common set of storage tools and techniques.

  1. Hybrid helps meet data governance rules.

External and internal data governance rules play a big part in data storage planning.  In a recent survey, 59% of respondents reported the need to maintain some of their data on premises. On average, that group stated that only about half of their data can go to the cloud. Financial data and customer records in particular are often subject to security, governance and compliance rules, driven by both internal policy and external regulation. With a hybrid cloud model, you can more easily accommodate the changing needs. With hybrid, you can set policies to ensure compliance, tailoring migration and data protection rules to specific data types.

While many are seeing the natural advantages of hybrid, some are still unsure. What other factors play in that I haven’t mentioned? With more and more being digitized and retained into perpetuity, what opportunities is your organization exploring to deal with the data deluge?

IBM Spectrum Protect with Amazon S3 Cloud Storage

IBM Spectrum Protect (formerly IBM Tivoli Storage Manager) solution provides the following benefits:

  • Supports software-defined storage environments
  • Supports cloud data protection
  • Easily integrates with VMware and Hyper-V
  • Enables data protection by minimizing data loss with frequent snapshots, replication, and DR management
  • Reduce the cost of data protection with built-in efficiencies such as source-side and target-side deduplication

IBM Spectrum Protect has also enhanced its offerings by providing support for Amazon S3 cloud storage (version 7.1.6 and later) and IBM Spectrum Protect version 7.1.6 was just released on June 17th, 2016. I was actually a little nervous and excited at the same time. Why? Because Cloudian HyperStore has a S3 guarantee. What better way to validate that guarantee than by trying a plug-and-play with a solution that has just implemented support for Amazon S3?

Overview of IBM Spectrum Protect with Amazon S3 cloud storage

And the verdict? Cloudian HyperStore configured as “Cloud type: Amazon S3” works right off the bat with IBM Spectrum Protect. You can choose to add a cloud storage pool from the V7.1.6 Operations Center UI or use the Command Builder. The choice is yours.

We’ll look at both the V7.1.6 Operations Center UI and the Command Builder to add our off-premise cloud storage.

NOTE: Cloudian HyperStore can be deployed as your on-premise S3 cloud storage but it has to be identified as an Amazon S3 off-premise cloud storage and you have to use a signed SSL certificate.

Here’s how you can add an Amazon S3 cloud storage or a Cloudian HyperStore S3 cloud storage into your IBM Spectrum Protect storage pool:

From the V7.1.6 Operations Center UI

 

From the V7.1.6 Operations Center console, select “+Storage Pool”.

Adding 'Storage Pool' to the IBM Spectrum Protect V7.1.6 Operations Center console

In the “Add Storage Pool:Identity” pop-up window, provide the name of your cloud storage and the description. In the next step of the “Add Storage Pool:Type”, select “Container-based storage:Off-premises cloud”.

IBM Spectrum Protect cloud storage description

Click on “Next” to continue. The next step in the “Add Storage Pool:Credentials” page is where it gets exciting. This is where we provide the information for:

  • Cloud type: Amazon S3 (Amazon S3 cloud type is also used to identify a Cloudian HyperStore S3)
  • User Name: YourS3AccessKey
  • Password: YourS3SecretKey
  • Region: Specify your Amazon S3 region (for Cloudian HyperStore S3, select “Other”)
  • URL: If you had selected an Amazon S3 region, this will dynamically update to the Amazon region’s URL. If you are using a Cloudian HyperStore S3 cloud storage, input the S3 Endpoint Access (HTTPS).

Complete the process by clicking on “Add Storage Pool”.

IBM Spectrum Protect

NOTE: Be aware that there is currently no validation performed to verify your entries when you click on “Add Storage Pool”. Your S3 cloud storage pool will be created. I believe the IBM Spectrum Protect group is addressing this with a validation process for the creation of a S3 cloud storage pool. I hope the step-by-step process that I have provided will help minimize errors with your Amazon S3 cloud storage pool setup.

From the V7.1.6 Operations Center Command Builder

 

From the V7.1.6 Operations Center Command Builder, you can use the following define stgpool command and you are done adding your off-premise S3 cloud storage pool:

  • define stgpool YourCloudName stgtype=cloud pooltype=primary cloudtype=s3 cloudurl=https://s3.cloudianstorage.com:443 access=readwrite encrypt=yes identity=YourS3AccessKey password=YourS3SecretKey description=”Cloudian”

NOTE: You can review the server instance dsmffdc log if there’s errors. It is located in the server instance directory. There’s also a probability that the signed SSL certificate might not be correct.

For example:

06-20-2016 11:58:26.150][ FFDC_GENERAL_SERVER_ERROR ]: (sdcloud.c:3145) com.tivoli.dsm.cloud.api.ProviderS3 handleException com.amazonaws.AmazonClientException Unable to execute HTTP request: com.ibm.jsse2.util.h: PKIX path building failed: java.security.cert.CertPathBuilderException: unable to find valid certification path to requested target
[06-20-2016 11:58:26.150][ FFDC_GENERAL_SERVER_ERROR ]: (sdcntr.c:8166) Error 2903 creating container ibmsp.a79378e1333211e6984b000c2967bf98/1-a79378e1333211e6984b000c2967bf98
[06-20-2016 11:58:26.150][ FFDC_GENERAL_SERVER_ERROR ]: (sdio.c:1956) Did not get cloud container. rc = 2903

 

Importing A Signed SSL Certificate

 

You can use the IBM Spectrum Protect keytool –import command to import the signed SSL certificate. However, before you perform the keytool import process, make a copy of the original Java cacerts.

The Java cacerts is located in IBM_Spectrum_Protect_Install_Path > TSM > jre > security directory.

You can run the command from IBM_Spectrum_Protect_Install_Path > TSM > jre > bin directory.
For example, on Windows:

    • ./keytool –import ../lib/security/cacerts –alias Cloudian –file c:/locationofmysignedsslcert/admin.crt

 

Enter the keystore password when prompted. If you haven’t updated your keystore password, the default is changeit and you should change it for production environments. When you are prompted to “Trust this certificate?”, input “yes”.

NOTE: Keep track of the “Valid from: xxxxxx” of your signed SSL certificate, you will have to import a new certificate when the current one expires.

By the way, if you encounter error “ANR3704E sdcloud.c(1636): Unable to load the jvm for the cloud storage pool on Windows 2012R2”, update the PATH environment variable on the Spectrum Protect Server:
IBM_Spectrum_Install_Path\Tivoli\TSM\jre\bin\j9vm and also set the JVM_LIB to jvm.dll.

Here’s what your Amazon S3 cloud storage type looks like from IBM Spectrum Protect V7.1.6 Operations Center console:

Operations Center console final result after adding Amazon S3 cloud storage to IBM Spectrum Protect V7.1.6

And you’re off! If you encounter any issues during this process, feel free to reach out to our support team.

You can also learn more by downloading our solution brief.

How-To: S3 Your Data Center

As the Storage Administrator or a Data Protection Specialist in your data center, you are likely looking for some alternative storage solution to help store all your big data growth needs. And with all that’s been reported by Amazon (stellar growth, strong quarterly earnings report), I am pretty sure their Simple Storage Service (S3) is on your radar. S3 is a secure, highly durable and highly scalable cloud storage solution that is also very robust. Here’s an API view of what you can do with S3:

S3 API view

As a user or developer, you can securely manage and access your bucket and your data, anytime and anywhere in the world where you have web access. As a storage administrator, you can easily manage and provision storage to any group and any user on always-on, highly scalable cloud storage. So if you are convinced that you want to explore S3 as a cloud storage solution, Cloudian HyperStore should be on your radar as well. I believe a solution that is easy to deploy and use helps accelerates the adoption of the technology. Here’s what you will need to deploy your own cloud storage solution:

  • Cloudian’s HyperStore Software – Free Community Edition
  • Recommended minimum hardware configuration
    • Intel-compatible hardware
    • Processor: 1 CPU, 8 cores, 2.4GHz
    • Memory: 32GB
    • Disk: 12 x 2TB HDD, 2 x 250GB HDD (12 drives for data, 2 drives for OS/Metadata)
    • RAID: RAID-1 recommended for the OS/Metadata, JBOD for the Data Drives
    • Network: 1x1GbE Port


You can install a single Cloudian HyperStore node for non-production purposes, but it is best practice to deploy a minimum 3-node HyperStore cluster so that you can use logical storage policies (replication and erasure coding) to ensure your S3 cloud storage is highly available in your production cluster. It is also recommended to use physical servers for production environments.

Here are the steps to set up a 3-node Cloudian HyperStore S3 Cluster:

  1. Use the Cloudian HyperStore Community Edition ISO for OS installation on all 3 nodes. This will install CentOS 6.7 on your new servers.
  2. Log on to your servers
    1. The default root password is password (Update your root access for production environments)
  3. Under /root, there are 2 Cloudian directories:
    1. CloudianTools
      1. configure_appliance.sh allows you to perform the following tasks:
        1. Change the default root password
        2. Change time zone
        3. Configure network
        4. Format and mount available disks for Cloudian S3 data storage
          1. Available disks that were automatically formatted and mounted during the ISO install for S3 storage will look similar to the following /cloudian1 mount:
            Format and mount available disks for Cloudian S3 data storage
    2. CloudianPackages
      1. Run ./CloudianHyperStore-6.0.1.2.bin cloudian_xxxxxxxxxxxx.lic to extract the package content from one of your nodes. This will be the Puppet master node.
        S3 Puppet master mode
      2. Copy sample-survey.csv survey.csv
        sample-survey.csv
      3. Edit the survey.csv file
        Edit survey.csv
        In the survey.csv file, specify the region, the node name(s), IP address(s), DC, and RAC of your Cloudian HyperStore S3 Cluster.

        NOTE: You can specify an additional NIC on your x86 servers for internal cluster communication.

      4. Run ./cloudianInstall.sh and select “Install Cloudian HyperStore”. When prompted, input the survey.csv file name. Continue with the setup.
        NOTE: If deploying in a non-production environment, it is possible that your servers (virtual/physical) may not have the minimum resources or a DNS server. You can run your install with ./cloudianInstall.sh dnsmasq force. Cloudian HyperStore includes an open source domain resolution utility to resolve all HyperStore service endpoints.
      5. v. In the following screenshot, the information that we had provided in the survey.csv file is used in the Cloudian HyperStore cluster configuration. In this non-production setup, I am also using a DNS server for domain name resolution with my virtual environment.Cloudian HyperStore cluster configuration
      6. Your Cloudian HyperStore S3 Cloud Storage is now up and running.
        Cloudian HyperStore S3 cloud storage
      7. Access your Cloudian Management Console. The default System Admin group user ID is admin and the default password is public.
        Cloudian Management Console
      8. Complete the Storage Policies, Group, and SMTP settings.
        Cloudian HyperStore - near final

Congratulations! You have successfully deployed a 3-node Cloudian HyperStore S3 Cluster.

Shifting Technology Habits and the Growth of Object Storage

Technology is, for many of us, a vital and inextricable part of our lives. We rely on technology to look up information, keep in touch with friends and family, monitor our health, entertain ourselves, and much more.

space

However, technology wasn’t always so ubiquitous – it wasn’t too long ago that our wireless phones had limited features and even fewer users actually using these features. Here’s the breakdown from 2004, according to a study from the Yankee Group:

This means that just over 10 years ago, less than 50% of cell phones had internet access and less than 10% had cameras. Even with 50% of phones having internet access, only 15% of users took advantage of this feature.

pew research center

By contrast, look at this survey conducted by Pew Research in 2014:

Among the 18-29 age group, text messaging and internet are more frequently used features than phone calls, which is indicative of the tremendous shift in technology use over the past few years. This study doesn’t even cover a major feature that many users use their phones for: pictures. As younger users turn almost exclusively to smartphone cameras for their photos (and, of course, #selfies), they turn to photo-sharing sites to host and display their images.

Photos are just one type of the ever-growing deluge of unstructured data, though. For enterprises, unstructured data also includes emails, documents, videos, audio files, and more. In order for companies to cost-effectively store this data (while keeping it protected and backed up for end-users), many of them are starting to turn to object storage over traditional network-attached storage (NAS).

Some of the benefits of object storage include a lower total cost of ownership (TCO) and the ability to easily scale up as data needs grow. That by itself is not enough, though. With a solution like our very own HyperStore, in addition to the affordable price (as low as 1c per GB per month) and infinite scalability (from tens of terabytes to hundreds of petabytes), we offer easy management and access control, plus strong data protection with both erasure coding and replication settings. You can read about all of HyperStore’s features and benefits here.

Unstructured data use is only going to continue to grow. Smartphones and other data-intensive technologies will only become more prevalent, and you’ll want to be prepared to meet that growth. Learn more about Cloudian’s hardware and software solutions today.

Lenovo Solves Data Storage Needs with a New Appliance

As our lives become increasingly digital, we’ll generate more and more data. By current estimates, storage needs are doubling in size every two years. That means that by 2020, we will reach 44 zettabytes – or 44 trillion gigabytes – of data, with most of that growth as unstructured data for backups, archives, cloud storage, multimedia content, and file data. This growth in data is quickly outpacing IT budgets. It’s clear we need a new storage approach if we hope to keep up with this deluge of data.

Introducing a New Appliance by Lenovo and Cloudian

 

Lenovo, together with Cloudian, is attacking the $40B storage market with a new, innovative capacity storage appliance for low-cost, scalable storage which addresses 80% of customer’s data needs. We are proud to introduce the Lenovo DX8200C powered by Cloudian as the storage building block which can scale to this challenge and further drive datacenter efficiency and investment protection.

Lenovo DX8200C powered by CloudianThe Lenovo DX8200C powered by Cloudian is an affordable and scalable object storage solution.

Offered as part of Lenovo’s StorSelect software-defined storage program, this factory integrated appliance is built upon Lenovo’s industry-leading servers and features:

  • S3: S3 is the de facto cloud storage standard as stated by Gartner. Cloudian is the only native S3-compatible mass capacity storage solution on the market, enabling customers and partners to take advantage of the $38B AWS ecosystem
  • Affordability: Lower the total cost of ownership (TCO) to $0.1 per GB per month
  • Scalability: The flexible design allows you to start small and scale up to 112 TB of storage capacity per node
  • Security: Utilize always-on erasure coding and replication to ensure your data is protected
  • Simplicity: Single SKU for full appliance and support

The Lenovo DX8200C powered by Cloudian delivers a fully-integrated and ready-to-deploy capacity storage system, reducing risks and variables in the datacenter. Global support is provided by Lenovo’s top-rated support team.

Additionally, what sets this appliance apart from others is the use of Cloudian’s HyperStore storage platform, bringing with it a full host of key features, including:

 

In a news announcement today, David Lincoln, GM of the Data Center Group at Lenovo, stated that “the Cloudian HyperStore solution enables us to deliver leading innovative, software-defined storage capabilities to enterprises and service providers worldwide.”

Michael Tso, CEO and co-founder of Cloudian, reiterated this point by stating that “enterprises and value-added resellers (VARs) can maximize their business investment and revenue opportunities with this fully turnkey, channel-ready, 100 percent S3 object storage solution.”

With more and more industries requiring massive amounts of data to be stored, this partnership with Lenovo represents a vital next step – one where pre-loaded appliances make it easy for companies to both integrate with existing infrastructure and scale out for large deployments.

The Lenovo DX8200C powered by Cloudian will be available worldwide in the third quarter of 2016 but Lenovo and Cloudian are working closely together to address all customer needs in the meantime.

Betting on Software-Defined Storage

Picking a company to advise is not always easy, but sometimes it just clicks

Take, for example, my recent decision to join Cloudian’s advisory board. As outlined in my recent blog post, I look at several factors before deciding to advise a company:

  • The potential for growth
  • How well they know their target audience
  • The quality of the team
  • How passionate everyone is (not just the employees of the company, but also my own passion and excitement)

Of course, these are broad factors I always consider. When it came to Cloudian, I had plenty of other questions as well:

  • Does the product/service actually work?
  • Does it scale?
  • Does it save money?
  • Does it enable a more agile operations environment?

Everything I’ve heard from customers – and from the team, of course – indicates that the answer is a definite YES. But before I go into more detail on what Cloudian does, a bit of background on object storage.

Storage: Then and Now

Storage was long dominated by firms that did a good job protecting your data – and serving it up for the then dominant vertically scalable workloads – and that, in turn, locked you into their proprietary hardware and software, resulting in the largest margins in IT infrastructure. Back in 2008, many firms such as Intel and LSI and the whitebox server providers and entrepreneurs including myself thought: hmm, storage is taking more and more of the IT budget but is not keeping up with new application architectures, new patterns of data generation, and a new generation of storage managers. There has got to be a better way.

And that better way is now called software-defined storage.

Today, storage is much better than it was in 2008, with far better economics, business models that pass the benefits of flash and network improvements on to customers, and a shift towards scale-out, developer-friendly architectures. Much of this change has been forced by Amazon and S3, who set the bar quite high for easy-to-use, massively scalable, and comparatively less expensive storage.

How Cloudian Fits In

Cloudian provides on-premise software-defined storage at 1c/GB. This by itself does not set the company apart, but they made a smart move early on – they bet on Amazon’s S3 API. Instead of inventing another proprietary API in a sea of proprietary APIs, they decided to focus on S3 from day one. This gives them a unique offering – 100% S3-compliant storage that uses metadata in interesting and intelligent ways.

So if you and your developers and the software you are running can interact with S3 – it can interact with Cloudian. Cloudian’s management interfaces blow away the AWS storage GUI, incidentally – giving you a firm-wide view into your data, grouped into blobs, and the policies you’ve applied to these blobs. So your developers are happy because it scales and it is basically S3 out the front and your storage teams are happy because they retain the control and visibility they need to do their jobs. And that’s not to mention “native” multi-tenancy, which is one of many reasons service providers like NTT rely on Cloudian.

The results have been outstanding, with customers providing excellent reviews – and this growing word of mouth has led to a sales acceleration.

I went into the Cloudian offices recently to talk a bit more about my reasons for joining the advisory board, and you can watch the video here:

I’m excited to get a front-row seat as Cloudian continues to ramp up their momentum and grow, and I’m looking forward to getting to know everyone in this wonderful team of people.

Data Availability & Data Protection for the IoT World

New York cityscape

New York, “The City That Never Sleeps”. A very fitting moniker for a city that is full of energy and excitement. Servers located in data centers all around the world are constantly crunching numbers and generating analytics in every financial institution in New York. Why are some of these servers located worldwide? Well, for a variety of reasons, but in my humble opinion, it is to ensure that data is always on and always available. After all, we are talking about billions of dollars in capital electronically managed by the New York Stock Exchange alone.

By 2020, it is predicted that there will be at least 20+ billion internet connected devices. As your business grows, so will the amount of data and storage that you will need. We’ll obviously need solutions to protect our data on-premise or in the cloud. A company that can make sure customers data is always on, secure, highly available, and also protected, rules the IoT WORLD.

modern storage requirementsBut in order to serve and protect your data for the always on, always available IoT world, what requirements should we take into account before deploying any data protection or storage solution? If you are a data protection geek, you’ll most likely see some of your requirements being listed on the right. If you are a data protection solutions provider, you guys definitely rock! Data protection solutions such as Commvault, NetBackup, Rubrik, Veeam, etc. are likely the solutions you have in-house to protect your corporate data centers and your mobile devices. These are software-defined and they are designed to be highly available for on-premise or in-the-cloud data protection.

What about storage? What would you consider? I am sure there are many well-known storage providers you can easily name. But with the new kids on the block disrupting the storage market, would lowering your operating costs ($0.005/GB per month) and meeting the above-listed requirements pique your interest?

Amazon S3 and Cloudian
Cloudian is a software-defined storage company. The solution is fully S3 compliant, which means that if you are familiar with Amazon S3, you’ll love the features that comes with this solution. If you are not, as a data protection geek with more than 15 years of experience, I invite you to give Cloudian HyperStore free trial a shot. The features and capabilities of Cloudian HyperStore as a scale-out storage solution with true multi-tenancy is pretty cool in my books. Imagine being able to deploy and grow storage as you need it for your corporate user home directories, backups, archiving, and even object storage for virtualization solutions (i.e. Red Hat OpenStack). The use cases for scale-out storage solutions are vast. There is no more hardware vendor lock-in as you can easily select between the options of a Cloudian HyperStore appliance or commodity servers to roll-your-own scale-out storage with Cloudian HyperStore software.

Imagine that you, as a storage administrator, can easily provide Storage as a Service (STaaS) to all your users. Take a look at the image below. The granular object level management that is available on a per user basis is pretty sweet. I can provide access to my files/objects with read and/or write permissions, with object level ACL and share the object via a public URL access.
Cloudian object level management

To top it all off, I can also limit the maximum number of downloads of that specific object that I want to share. As a service provider, you can also use the analytics inherent in the solution to implement chargeback to your customers on every account that you manage using Cloudian HyperStore smart storage solution.

Best of all, if you decide that you want to move your data to Amazon, use Cloudian Hyperstore’s built-in auto-tiering feature. Dynamically move your data to Amazon S3 if you choose to do so. You don’t have to take my word for it. Cloudian will provide you with a 45-day free trial. Try it out today.

Harness the Power of Software-Defined Infrastructure to Help Solve the Biggest Big Data Storage Challenges

With the popularity of rich media, the proliferation of mobile devices and the digitization of content, there has been an exponential growth in the amount of unstructured data that IT is managing (think medical images or large research data sets).  And this growth is not slowing but increasing. This unprecedented growth is just not sustainable for IT organizations as they try to control costs and limit or reduce the operational complexity in the datacenter.  Enter the need for an evolution in IT – for Software-Defined Infrastructure, and software-defined storage solutions to help store and manage the Big Data tidal wave.

 

Intel and Cloudian understand the powerful benefits behind software-defined infrastructure, and the real value it delivers to businesses and to the IT organizations behind them. We have been working together to deliver the true benefits of SDI to our joint customers with the world’s largest Big Data storage needs. From enterprises in Life Sciences, Media & Entertainment, and Financial Services, to Service Providers like ScaleMatrix, who serve their customers with new, innovative hosted cloud storage solutions, we help them store and manage petabyte-scale data, easily, securely, and cost-effectively.

Cloudian develops 100-percent software-defined object storage software.  We’ve joined Intel’s Storage Builders alliance, an elite group of independent software vendors focused on delivering on the benefits of software-defined infrastructure with innovative software-defined storage solutions. We’re pleased to join the larger Intel’s Builders family of Independent Software Vendors (ISVs), operating system vendors (OSVs), and original equipment manufacturers (OEMs) to support the group’ mission to accelerate global adoption of software-defined infrastructure, including cloud, storage, and networking technologies. We’ve contributed Cloudian’s validated software-defined object storage designs, along with Cloudian HyperStore – our scale-out, ready-to go, object storage software – to deliver customers all the benefits of software-defined storage: scalability, agility, and the choice to build on-premise or hybrid clouds at the lowest cost. We’ve qualified the Intel Xeon processor D-1500 product family for use with Cloudian HyperStore. We’ve also developed a comprehensive reference architecture for Cloudian HyperStore powered by Lenovo hardware.

Customers love the flexibility and scalability of software-defined storage. They can quickly deploy HyperStore for on-premise cloud storage that’s easy to manage but that also allows choice to automatically tier some or all of their data to any S3-compatible cloud in a hybrid model. Global enterprises use HyperStore to easily and securely manage their data stored across multiple datacenters, and Service Provider businesses take advantage of HyperStore’s software-defined architecture to deliver Storage as a Service (StaaS) product offerings to their customers. They are able to quickly deploy and manage these services while easily making updates to continuously deliver new features to their customers.

Together, Cloudian, Intel, and other leading vendors, including Lenovo, are helping organizations embrace the business value of software-defined infrastructure.  We put the power of software-defined object storage in the hands of our joint customers to make their IT organizations, and their infrastructures strong, agile, and ready to meet the demands of their Big Data.

Learn more about Cloudian and Intel Storage Builders.

Cloudian HyperStore Integration with Symantec NetBackup

Starting with Symantec NetBackup 7.7, administrators will find an exciting new feature for cloud storage backup: Cloudian HyperStore®. The NetBackup Cloud Storage Connector enables the NetBackup software to back up data to and from Cloudian HyperStore straight out of the box without additional software installations or plugins. HyperStore is an option in the “Cloud Storage Server Configuration Wizard”. Users can simply add their S3 account information such as endpoint, access key, and secret key to begin the process of backing up their data to Cloudian HyperStore storage.

cloudian hyperstore 4000

Cloudian HyperStore and Symantec NetBackup together deliver the following benefits:

  • Enterprise-level backup
  • Complete integrated data center solution: computing, networking, and storage
  • Reduced total cost of ownership (TCO) that continues to improve as the solution scales out
  • Operational efficiency
  • Agility and scalability with the scale-out architectures of Cloudian HyperStore
  • Complete Amazon Simple Storage Service (S3) API–compatible geographically federated object storage platform
  • Enterprise-class features: multi-tenancy, quality of service (QoS), and dynamic data placement in a completely software-defined package
  • Policy-based tiering between on-premises hybrid cloud storage platform and any S3 API–compliant private or public cloud
  • Investment protection: mix and match different generations and densities of computing platforms to build your storage environment; more than 400 application vendors support S3

The seamless integration allows IT Departments to manage cloud storage for backup and recovery as easily as on-premise storage, but with lower costs. Finally, this integrated solution helps deliver an automated and policy-based backup and recovery solution. Organizations can also leverage the cloud as a new storage tier or as a secondary off-site location for disaster recovery.

For more information, please see the Symantec NetBackup and Cloudian HyperStore Solution Brief.

 

Next Generation Storage: integration, scale & performance

Guest Blog Post by Colm Keegan from Storage Switzerland

Various industry sources estimate that data is doubling approximately every two years and the largest subset of that growth is coming from unstructured data. User files, images, rich multimedia, machine sensor data and anything that lives outside of a database application can be referred to collectively as unstructured data.

Storage Scaling Dilemma

3d-man-growing-data-centerThe challenge is that traditional storage systems, which rely on “scale-up” architectures (populating disk drives behind a dual controller system) to increase storage capacity, typically don’t scale well to meet the multi PB data growth which is now occurring within most enterprise data centers. On the other hand, while some “scale-out” NAS systems can scale to support multiple PB’s of storage within a single filesystem, they are often not a viable option since adding storage capacity to these systems often requires adding CPU and memory resources at the same time – resulting in a high total cost of ownership.

Commoditized Storage Scaling

Businesses need a way to cost effectively store and protect their unstructured data repositories utilizing commodity, off the shelf storage resources and/or low cost cloud storage capacity. In addition, these repositories need to be capable of scaling massively to support multiple PB’s of data and enable businesses to seamlessly share this information across wide geographical locations. But in addition to storage scale and economy, these resources should also be easy to integrate with existing business applications. And ideally, they should be performance optimized for unstructured data files.

Software Driven Capacity

Software defined storage (SDS) technologies are storage hardware agnostic solutions which allow businesses to use any form of storage to build-out a low cost storage infrastructure. Internal server disk, conventional hard disk drives inside a commodity disk array or even a generic disk enclosure populated with high density disk can be used. Likewise, with some SDS offerings, disk resources in the data center can be pooled with storage in secondary data center facilities located anywhere in the world and be combined with cloud storage to give businesses a virtually unlimited pool of low-cost storage capacity.

Plug-and-Play Integration

From an integration perspective, some of these solutions provide seamless integration between existing business applications and cloud storage by providing native support for NFS and CIFS protocols. So instead of going through the inconvenience and expense of re-coding applications with cloud storage compatible API’s like REST, SWIFT or Amazon’s S3 protocol, these technologies essentially make a private or hybrid cloud data center object storage deployment a plug-and-play implementation but still provide the option to go “native” in the future.

Tapping Into Cloud Apps

But storage integration isn’t just limited to on premise applications, it also applies to cloud based applications as well. Today there is a large ecosystem of Amazon S3 compatible applications that businesses may want to leverage. Examples include backup and recovery, archiving, file sync and share, etc. Gaining access to these software offerings by utilizing an S3 compatible object storage framework, gives businesses even more use cases and value for leveraging low-cost hybrid cloud storage.

Data Anywhere Access

Now businesses can provision object storage resources on-premises and/or out across public cloud infrastructure to give their end-users ubiquitous access to data regardless of their physical location. This enables greater data mobility and can serve to enhance collaborative activities amongst end-users working across all corners of the globe. Furthermore, by replicating data across geographically dispersed object storage systems, businesses can automatically backup data residing in remote offices/branch offices to further enhance data resiliency.

With data intensive applications like big data analytic systems and data mining applications clamoring for high speed access to information, object storage repositories need to be capable of providing good performance as well. Ideally, the storage solution should be tuned to read, write and store large objects very efficiently while still providing high performance.

 Stem The Data Tide

Businesses today need a seamless way to grow out low-cost abundant, hybrid cloud storage resources across the enterprise to meet the unstructured data tsunami that is flooding their data center environments. In addition to providing virtually unlimited scaling from a storage capacity perspective, these resources need to easily integrate into existing application environments and provide optimal performance access to large unstructured data objects. Cloudian’s HyperStore solution provides all of these capabilities through a software defined storage approach which gives businesses the flexibility to choose amongst existing commodity disk assets in the data center and/or low cost object storage in the cloud, to help stem the unstructured data tide.

 

About Author

Colm Keegan is a 23 year IT veteran, Colm’s focus is in the enterprise storage, backup and disaster recovery solutions space at Storage Switzerland.

Enterprise Storage – Stop the Madness

Guest Blog Post by John Bennett

Recently I was visiting my favorite co-location data center in Tokyo when I saw two young technologists attempting to push a heavily laden cart of brand new gear, still in neonatal ESD bags, over a door jam. Their ID badges revealed them as employees of a well-known global investment bank. In a thinly veiled maneuver to satisfy my curiosity, I offered to help. After a few tedious moments we had surmounted the obstacle. Panting a bit, the two young men thanked me profusely for lending an extra back to their burden. It was then that I realized what I had been lifting.  A brand new disk array with Fibre Channel storage processors.

Fibre Channel… in 2014.

Well, I thought, perhaps they were adding storage to a mainframe, or it was an upgrade to an existing solution. My curiosity piqued, I asked.

No, they said. It was storage for a new component of a customer facing web application.

The exchange bothered me for the rest of the afternoon. When I arrived at the office the next day, I penned some rough specifications, put in a request for a budgetary quotation and scribbled out a high-level WBS and rough order of magnitude estimate for a project to deliver 100TB of replicated, geographically diverse disk using a similar technology to what I had seen the day before.

A couple of days later the numbers came back from the storage vendor. When I put it all together, what I discovered was shocking. The effective all-in 5 year cost of ownership for the disk array I had pushed over a 1 cm piece of aluminum the day before was somewhere around $2.3 million USD. This includes the cost of the array, networking, installation and project labor, cabling, rack space, power and maintenance.

Most of us have had to help a business executive through technology sticker shock before. I’m sure this project had been no exception. These conversations typically contain catchphrases like “investing in scalability” and “enterprise-grade availability and fault tolerance” and typically last as long as it takes for the person holding the purse strings to glaze over and open their wallet. But at this point we’ve been preaching about the cost savings of virtualization and private clouds for well over a decade. How many of us are still spending megabucks on old legacy or, even worse, new legacy disk arrays and SAN fabric switches? When will our adherence to now ancient technologies become an existential risk to the enterprise technologist as a species? A reasonable argument could be made that we’ve all been made obsolete and we just don’t know it yet.

We have to stop the storage madness before it’s too late.

The narrative of the counter argument goes something like this: Infrastructure, properly run, is a utility service. As such it is largely defined by the requirements of the layers of the technology stack that depend on it. Infrastructure technologists can only independently change their part of the investment insofar as that change doesn’t impact the layers above it. Put another way, no matter how much shiny new whiz-bang object store capacity I can offer in my data center, it does absolutely no good if the business I support runs on applications that were written when monolithic RDBMSs dominated the earth. In this context, it’s understandable why some might think enterprise storage a lost cause. I’d like to argue that enterprise storage presents a ripe opportunity to add value.

The fact of the matter is that now is the time to be vocal about next-generation infrastructure technologies of all stripes. “Big Data” is no longer just a buzz word. It’s a reality, and often an unwelcome one for established firms. Pressure from without as more agile cloud-native ventures close in on the market share of more mature firms is converging with pressure from within to add features and capacity to legacy BI and CRM systems.  Legacy platforms the world over are straining under impossible loads and technology departments are straining under demands to meet bottom line targets that simply can’t be met with technology architectures from 1988.

As it was when mainframes gave way to midrange UNIX and when the Internet changed everything forever, the big winners were the ones who could optimize technological transformations for their stakeholders. For enterprise storage a similar change is happening right now. The past has shown us that leading a revolution is much preferable to being swept away by it.

About Author

John is a technologist with 20 years of experience on the front lines of technology infrastructure and operations. John’s focus is the application of scientific,  data-driven quality management techniques to high-risk technology operations. He is currently located in Tokyo.