New HyperStore 4000: Highest density storage

Rack space and budget. Most data centers are short on both. Yet somehow, you’re expected to accommodate a 50% increase in unstructured data volume annually. That’s a problem.

The new solution is the HyperStore 4000. With 700TB in just 4U of rack height, it’s nearly 2X the density of our earlier models. And it delivers storage in your data center at prices on par with the public cloud: about ½ cent per GB per month. Space savings and cost savings in one.

The HyperStore 4000, by the numbers

The HyperStore 4000 appliance was built to handle massive amounts of data. It’s housed in a 7” high 4U enclosure, with 2 nodes and a max capacity of 700TB. The drive sizes range from 4TB to 10TB and has 256GB of memory (128GB per node).

Better yet, the appliance reduces storage costs by 40% (versus other Cloudian solutions) – data management now costs half a cent per GB per month. Even with the reduced cost, there is no drop in data availability – with a three-appliance cluster, you’ll still see 99.999999% data durability.

For the full list of specs, check out our datasheet.

Save time, your scarcest commodity

With most storage systems, as your needs grow, your management headaches grow with them. But the HyperStore 4000 grows painlessly. You just add nodes. No disruption and no data migration.

We specifically designed the HyperStore 4000 appliance to help customers in industries with huge data needs such as life sciences, healthcare, and entertainment. These are the industries where data growth is exploding every year and where modern data centers feel the most burden, as they need high-density storage with peak performance for data protection, video surveillance, research, archival, and more. Now you can meet these growing needs without growing pains.

Finally, the HyperStore 4000 has a 100% native S3 API, and has the industry’s highest level of S3 API compatibility. In fact, we guarantee it to work with your S3-enabled applications.

Be sure to also take a look at Cloudian’s other solutions to see which one is right for you.

AWS CLI with S3-Compatible Storage

There’ve been a lot of discussions about Amazon’s Simple Storage Service (S3) and Amazon Web Services (AWS). It seems to me that everyone is saying that they are Amazon S3-compatible or that they work with S3 storage. That makes me wonder, what is the best way to validate a solution or test it out to see if the storage solution will meet my object storage needs? Well, why not just use Amazon’s own S3APIs and AWS Command Line Interface (CLI)?

AWS CLI is a unified tool developed to help manage AWS services. I believe this is the best way to test out any solution that says they are an S3 compatible storage such as Cloudian HyperStore. So let’s hop on to it and get started. The following shows the steps on how to install and use AWS CLI with Cloudian HyperStore on your Linux server.

Prerequisite:

You will need to install PIP to simplify your AWS CLI installation, you can copy the following python script to your Linux server and it will help you install pip and awscli. The script is provided as-is but feel free to copy, modify and improve it to your liking.

import urllib

import os

PIP=’get-pip.py’

urllib.urlretrieve (“https://bootstrap.pypa.io/get-pip.py”, PIP)

os.system(“python get-pip.py”)

os.system(“pwd”)

os.system(“pip install awscli”)

Process:

  1. Download the following dc_getpip.py to your Linux server. The script has been tested on RHEL and CentOS. The Cloudian S3 region used in this example is s3-region.addomain.local
  2. Run python dc_getpip.py. This script will download pip and install AWS CLI for you.
  3. When the AWS CLI is successfully installed, continue with configuring AWS CLI with Cloudian HyperStore.
  4. Execute aws configure and provide the Cloudian credential along with the Cloudian S3 region information. For example:
  5. cd ~/./.aws because the config and the credential files for aws is located in your user directory. In this example, this is the root user directory.AWS CLI root user directory
  6. There are 2 files in .aws directory:
    1. config
    2. credentials
  7. Update the config file with the Cloudian region information. Include [cloudian] in your update.AWS CLI Cloudian regional information
  8. Update the credentials files with the Cloudian information, include [cloudian] in your update.AWS CLI credentials file
  9. Run the following aws command to validate connectivity to your Cloudian HyperStore cluster. Using s3 ls will list the buckets of the tenant that was configured.
    1. aws –profile=cloudian –endpoint-url=http://s3-region1.addomain.local s3 lsAWS CLI validate connectivity to Cloudian HyperStore cluster
    2. Replace s3-region1.addomain.local with your Cloudian region.
    3. You can use aws –profile=cloudian –endpoint-url=http://s3-region1.addomain.local s3 cp file s3://bucket to test upload to your s3 bucket.
  10. Your AWS CLI is successfully configured with Cloudian HyperStore S3.

 

If you are curious to learn more about S3, download Cloudian HyperStore’s community edition and validate the solution for yourself.

Learn more about hybrid cloud management here.

IBM Spectrum Protect with Amazon S3 Cloud Storage

IBM Spectrum Protect (formerly IBM Tivoli Storage Manager) solution provides the following benefits:

  • Supports software-defined storage environments
  • Supports cloud data protection
  • Easily integrates with VMware and Hyper-V
  • Enables data protection by minimizing data loss with frequent snapshots, replication, and DR management
  • Reduce the cost of data protection with built-in efficiencies such as source-side and target-side deduplication

IBM Spectrum Protect has also enhanced its offerings by providing support for Amazon S3 cloud storage (version 7.1.6 and later) and IBM Spectrum Protect version 7.1.6 was just released on June 17th, 2016. I was actually a little nervous and excited at the same time. Why? Because Cloudian HyperStore has a S3 guarantee. What better way to validate that guarantee than by trying a plug-and-play with a solution that has just implemented support for Amazon S3?

Overview of IBM Spectrum Protect with Amazon S3 cloud storage

And the verdict? Cloudian HyperStore configured as “Cloud type: Amazon S3” works right off the bat with IBM Spectrum Protect. You can choose to add a cloud storage pool from the V7.1.6 Operations Center UI or use the Command Builder. The choice is yours.

We’ll look at both the V7.1.6 Operations Center UI and the Command Builder to add our off-premise cloud storage.

NOTE: Cloudian HyperStore can be deployed as your on-premise S3 cloud storage but it has to be identified as an Amazon S3 off-premise cloud storage and you have to use a signed SSL certificate.

Here’s how you can add an Amazon S3 cloud storage or a Cloudian HyperStore S3 cloud storage into your IBM Spectrum Protect storage pool:

From the V7.1.6 Operations Center UI

 

From the V7.1.6 Operations Center console, select “+Storage Pool”.

Adding 'Storage Pool' to the IBM Spectrum Protect V7.1.6 Operations Center console

In the “Add Storage Pool:Identity” pop-up window, provide the name of your cloud storage and the description. In the next step of the “Add Storage Pool:Type”, select “Container-based storage:Off-premises cloud”.

IBM Spectrum Protect cloud storage description

Click on “Next” to continue. The next step in the “Add Storage Pool:Credentials” page is where it gets exciting. This is where we provide the information for:

  • Cloud type: Amazon S3 (Amazon S3 cloud type is also used to identify a Cloudian HyperStore S3)
  • User Name: YourS3AccessKey
  • Password: YourS3SecretKey
  • Region: Specify your Amazon S3 region (for Cloudian HyperStore S3, select “Other”)
  • URL: If you had selected an Amazon S3 region, this will dynamically update to the Amazon region’s URL. If you are using a Cloudian HyperStore S3 cloud storage, input the S3 Endpoint Access (HTTPS).

Complete the process by clicking on “Add Storage Pool”.

IBM Spectrum Protect

NOTE: Be aware that there is currently no validation performed to verify your entries when you click on “Add Storage Pool”. Your S3 cloud storage pool will be created. I believe the IBM Spectrum Protect group is addressing this with a validation process for the creation of a S3 cloud storage pool. I hope the step-by-step process that I have provided will help minimize errors with your Amazon S3 cloud storage pool setup.

From the V7.1.6 Operations Center Command Builder

 

From the V7.1.6 Operations Center Command Builder, you can use the following define stgpool command and you are done adding your off-premise S3 cloud storage pool:

  • define stgpool YourCloudName stgtype=cloud pooltype=primary cloudtype=s3 cloudurl=https://s3.cloudianstorage.com:443 access=readwrite encrypt=yes identity=YourS3AccessKey password=YourS3SecretKey description=”Cloudian”

NOTE: You can review the server instance dsmffdc log if there’s errors. It is located in the server instance directory. There’s also a probability that the signed SSL certificate might not be correct.

For example:

06-20-2016 11:58:26.150][ FFDC_GENERAL_SERVER_ERROR ]: (sdcloud.c:3145) com.tivoli.dsm.cloud.api.ProviderS3 handleException com.amazonaws.AmazonClientException Unable to execute HTTP request: com.ibm.jsse2.util.h: PKIX path building failed: java.security.cert.CertPathBuilderException: unable to find valid certification path to requested target
[06-20-2016 11:58:26.150][ FFDC_GENERAL_SERVER_ERROR ]: (sdcntr.c:8166) Error 2903 creating container ibmsp.a79378e1333211e6984b000c2967bf98/1-a79378e1333211e6984b000c2967bf98
[06-20-2016 11:58:26.150][ FFDC_GENERAL_SERVER_ERROR ]: (sdio.c:1956) Did not get cloud container. rc = 2903

 

Importing A Signed SSL Certificate

 

You can use the IBM Spectrum Protect keytool –import command to import the signed SSL certificate. However, before you perform the keytool import process, make a copy of the original Java cacerts.

The Java cacerts is located in IBM_Spectrum_Protect_Install_Path > TSM > jre > security directory.

You can run the command from IBM_Spectrum_Protect_Install_Path > TSM > jre > bin directory.
For example, on Windows:

    • ./keytool –import ../lib/security/cacerts –alias Cloudian –file c:/locationofmysignedsslcert/admin.crt

 

Enter the keystore password when prompted. If you haven’t updated your keystore password, the default is changeit and you should change it for production environments. When you are prompted to “Trust this certificate?”, input “yes”.

NOTE: Keep track of the “Valid from: xxxxxx” of your signed SSL certificate, you will have to import a new certificate when the current one expires.

By the way, if you encounter error “ANR3704E sdcloud.c(1636): Unable to load the jvm for the cloud storage pool on Windows 2012R2”, update the PATH environment variable on the Spectrum Protect Server:
IBM_Spectrum_Install_Path\Tivoli\TSM\jre\bin\j9vm and also set the JVM_LIB to jvm.dll.

Here’s what your Amazon S3 cloud storage type looks like from IBM Spectrum Protect V7.1.6 Operations Center console:

Operations Center console final result after adding Amazon S3 cloud storage to IBM Spectrum Protect V7.1.6

And you’re off! If you encounter any issues during this process, feel free to reach out to our support team.

You can also learn more by downloading our solution brief.

How-To: S3 Your Data Center

As the Storage Administrator or a Data Protection Specialist in your data center, you are likely looking for some alternative storage solution to help store all your big data growth needs. And with all that’s been reported by Amazon (stellar growth, strong quarterly earnings report), I am pretty sure their Simple Storage Service (S3) is on your radar. S3 is a secure, highly durable and highly scalable cloud storage solution that is also very robust. Here’s an API view of what you can do with S3:

S3 API view

As a user or developer, you can securely manage and access your bucket and your data, anytime and anywhere in the world where you have web access. As a storage administrator, you can easily manage and provision storage to any group and any user on always-on, highly scalable cloud storage. So if you are convinced that you want to explore S3 as a cloud storage solution, Cloudian HyperStore should be on your radar as well. I believe a solution that is easy to deploy and use helps accelerates the adoption of the technology. Here’s what you will need to deploy your own cloud storage solution:

  • Cloudian’s HyperStore Software – Free Community Edition
  • Recommended minimum hardware configuration
    • Intel-compatible hardware
    • Processor: 1 CPU, 8 cores, 2.4GHz
    • Memory: 32GB
    • Disk: 12 x 2TB HDD, 2 x 250GB HDD (12 drives for data, 2 drives for OS/Metadata)
    • RAID: RAID-1 recommended for the OS/Metadata, JBOD for the Data Drives
    • Network: 1x1GbE Port


You can install a single Cloudian HyperStore node for non-production purposes, but it is best practice to deploy a minimum 3-node HyperStore cluster so that you can use logical storage policies (replication and erasure coding) to ensure your S3 cloud storage is highly available in your production cluster. It is also recommended to use physical servers for production environments.

Here are the steps to set up a 3-node Cloudian HyperStore S3 Cluster:

  1. Use the Cloudian HyperStore Community Edition ISO for OS installation on all 3 nodes. This will install CentOS 6.7 on your new servers.
  2. Log on to your servers
    1. The default root password is password (Update your root access for production environments)
  3. Under /root, there are 2 Cloudian directories:
    1. CloudianTools
      1. configure_appliance.sh allows you to perform the following tasks:
        1. Change the default root password
        2. Change time zone
        3. Configure network
        4. Format and mount available disks for Cloudian S3 data storage
          1. Available disks that were automatically formatted and mounted during the ISO install for S3 storage will look similar to the following /cloudian1 mount:
            Format and mount available disks for Cloudian S3 data storage
    2. CloudianPackages
      1. Run ./CloudianHyperStore-6.0.1.2.bin cloudian_xxxxxxxxxxxx.lic to extract the package content from one of your nodes. This will be the Puppet master node.
        S3 Puppet master mode
      2. Copy sample-survey.csv survey.csv
        sample-survey.csv
      3. Edit the survey.csv file
        Edit survey.csv
        In the survey.csv file, specify the region, the node name(s), IP address(s), DC, and RAC of your Cloudian HyperStore S3 Cluster.

        NOTE: You can specify an additional NIC on your x86 servers for internal cluster communication.

      4. Run ./cloudianInstall.sh and select “Install Cloudian HyperStore”. When prompted, input the survey.csv file name. Continue with the setup.
        NOTE: If deploying in a non-production environment, it is possible that your servers (virtual/physical) may not have the minimum resources or a DNS server. You can run your install with ./cloudianInstall.sh dnsmasq force. Cloudian HyperStore includes an open source domain resolution utility to resolve all HyperStore service endpoints.
      5. v. In the following screenshot, the information that we had provided in the survey.csv file is used in the Cloudian HyperStore cluster configuration. In this non-production setup, I am also using a DNS server for domain name resolution with my virtual environment.Cloudian HyperStore cluster configuration
      6. Your Cloudian HyperStore S3 Cloud Storage is now up and running.
        Cloudian HyperStore S3 cloud storage
      7. Access your Cloudian Management Console. The default System Admin group user ID is admin and the default password is public.
        Cloudian Management Console
      8. Complete the Storage Policies, Group, and SMTP settings.
        Cloudian HyperStore - near final

Congratulations! You have successfully deployed a 3-node Cloudian HyperStore S3 Cluster.

Lenovo Solves Data Storage Needs with a New Appliance

As our lives become increasingly digital, we’ll generate more and more data. By current estimates, storage needs are doubling in size every two years. That means that by 2020, we will reach 44 zettabytes – or 44 trillion gigabytes – of data, with most of that growth as unstructured data for backups, archives, cloud storage, multimedia content, and file data. This growth in data is quickly outpacing IT budgets. It’s clear we need a new storage approach if we hope to keep up with this deluge of data.

Introducing a New Appliance by Lenovo and Cloudian

 

Lenovo, together with Cloudian, is attacking the $40B storage market with a new, innovative capacity storage appliance for low-cost, scalable storage which addresses 80% of customer’s data needs. We are proud to introduce the Lenovo DX8200C powered by Cloudian as the storage building block which can scale to this challenge and further drive datacenter efficiency and investment protection.

Lenovo DX8200C powered by CloudianThe Lenovo DX8200C powered by Cloudian is an affordable and scalable object storage solution.

Offered as part of Lenovo’s StorSelect software-defined storage program, this factory integrated appliance is built upon Lenovo’s industry-leading servers and features:

  • S3: S3 is the de facto cloud storage standard as stated by Gartner. Cloudian is the only native S3-compatible mass capacity storage solution on the market, enabling customers and partners to take advantage of the $38B AWS ecosystem
  • Affordability: Lower the total cost of ownership (TCO) to $0.1 per GB per month
  • Scalability: The flexible design allows you to start small and scale up to 112 TB of storage capacity per node
  • Security: Utilize always-on erasure coding and replication to ensure your data is protected
  • Simplicity: Single SKU for full appliance and support

The Lenovo DX8200C powered by Cloudian delivers a fully-integrated and ready-to-deploy capacity storage system, reducing risks and variables in the datacenter. Global support is provided by Lenovo’s top-rated support team.

Additionally, what sets this appliance apart from others is the use of Cloudian’s HyperStore storage platform, bringing with it a full host of key features, including:

 

In a news announcement today, David Lincoln, GM of the Data Center Group at Lenovo, stated that “the Cloudian HyperStore solution enables us to deliver leading innovative, software-defined storage capabilities to enterprises and service providers worldwide.”

Michael Tso, CEO and co-founder of Cloudian, reiterated this point by stating that “enterprises and value-added resellers (VARs) can maximize their business investment and revenue opportunities with this fully turnkey, channel-ready, 100 percent S3 object storage solution.”

With more and more industries requiring massive amounts of data to be stored, this partnership with Lenovo represents a vital next step – one where pre-loaded appliances make it easy for companies to both integrate with existing infrastructure and scale out for large deployments.

The Lenovo DX8200C powered by Cloudian will be available worldwide in the third quarter of 2016 but Lenovo and Cloudian are working closely together to address all customer needs in the meantime.

Start Small and Grow with Unlimited Scale

It seems that much of the current conversation around data revolves around how much of it there is and how much there will be in the coming years. While this macro level perspective is important and should help inform how data is stored, it’s also important to focus in on the micro level use cases.

space
Many companies tout that they can start big and go bigger. The issue with this approach is that it ignores a large swath of customer needs. What if you don’t need hundreds of TBs of storage immediately? What if you want to start small, but anticipate growth down the line?

Cloudian HyperStore 6.0

Scale as you grow with Cloudian HyperStore

 

Cloudian offers the flexibility to start small without sacrificing any of the robust features in our HyperStore operating environment. We offer both software and hardware solutions so you can start with as little as tens of TB of storage and scale up to hundreds of PBs.

Cloudian HyperStore can be deployed on off-the-shelf commodity hardware for 1c per GB per month, making it both easy and affordable to scale out as your data grows. As you add more data, HyperStore will automatically divert from highly used disks to less used disks to avoid imbalance. Of course, as you scale, security and data resiliency become more and more vital, which is why this smart disk balancing is only one part of the wider array of protection features in HyperStore.

Big protection for all your data

 

No matter how much data you’re storing, we’ve built in some of the most robust security features possible to protect your data. On a read request on your data, all replicas are checked and missing or out-of-date replicas are automatically updated or replaced. As a result, you don’t have to worry about restoring to outdated data.

The Cloudian Management Console lets you monitor your system’s health and get alerts when things are off. Be proactive by utilizing replication or erasure coding (or both!) to properly protect your data. Plus, spread your data out among geographically independent data centers as an added contingency against data loss. If you need to conduct a more granular check-up on your system, we’ve implemented an “object GPS” so you can quickly and easily locate any specific object within a given bucket.

As your organization grows, your access needs will change as well. HyperStore gives you multi-tenancy controls so that you can give role-based access to administrators and users.

From the very beginning, we believed strongly in providing customers with all the tools they needed to create the storage platform that works for them. In addition to the HyperStore software, we also have turnkey appliances that enable small deployments with the potential to scale up to many PBs.

Cloudian HyperStore Appliance 1500 The Cloudian HyperStore 1500 Appliance offers hot-swappable hardware, automated data tiering, and unlimited scale.

If you’d like to try Cloudian HyperStore for yourself, sign up for a free trial today.

Simplifying Enterprise Data Protection with Rubrik

Data center sprawl can be a real pain to manage, and it’s only made worse when dealing with legacy architecture. As data needs continue to grow exponentially, having an efficient and cost-effective data protection solution in place is more necessary than ever. If you’re still dealing with outdated hardware and software, then it will only become increasingly complex (and frustrating) to deal with data migration, backup, and recovery.

space

Working with Rubrik for Better Data Protection

 

To help address these pain points, Cloudian has partnered with Rubrik to bring simple, seamless, and secure backup and long-term data retention solutions to enterprises.

Rubrik acts as a sort of ‘time machine’ for VMs. Backup software, catalog management, replication, and deduplicated storage are all brought into a single appliance. As a result, you get incredible ease of use and infinite scalability. Rubrik is able to deliver near-zero recovery times without the need for rehydration.

How Rubrik and Cloudian work together

How Rubrik and Cloudian HyperStore work hand-in-hand to provide efficient and affordable data backup and protection.

How Cloudian Fits In

 

Rubrik serves as a smart on-ramp to Cloudian HyperStore – no additional software installations or plugins necessary to connect the two. This, in turn, lets IT automate backup, replication, and archival via a policy-based engine. By sending the deduplicated data to HyperStore, IT can save money on data transfer and storage costs.

Furthermore, Cloudian HyperStore is fully software-defined with no affinity to hardware. It is a scale-out, 100% native S3 object storage platform designed for large but flexible storage solutions – ideal for storing unstructured data and content. HyperStore is robust and durable, but also flexible, and we kept ease of management and usability in mind from the start. Additionally, HyperStore provides seamless tiering and replication to public cloud providers such as AWS, Glacier, and other S3-compatible endpoints, including tape.

You can read more about Cloudian HyperStore’s features and benefits here.

—

Having ironclad data protection in place is vital. Read more about how Rubrik and Cloudian can help in our joint solution brief.

Betting on Software-Defined Storage

Picking a company to advise is not always easy, but sometimes it just clicks

Take, for example, my recent decision to join Cloudian’s advisory board. As outlined in my recent blog post, I look at several factors before deciding to advise a company:

  • The potential for growth
  • How well they know their target audience
  • The quality of the team
  • How passionate everyone is (not just the employees of the company, but also my own passion and excitement)

Of course, these are broad factors I always consider. When it came to Cloudian, I had plenty of other questions as well:

  • Does the product/service actually work?
  • Does it scale?
  • Does it save money?
  • Does it enable a more agile operations environment?

Everything I’ve heard from customers – and from the team, of course – indicates that the answer is a definite YES. But before I go into more detail on what Cloudian does, a bit of background on object storage.

Storage: Then and Now

Storage was long dominated by firms that did a good job protecting your data – and serving it up for the then dominant vertically scalable workloads – and that, in turn, locked you into their proprietary hardware and software, resulting in the largest margins in IT infrastructure. Back in 2008, many firms such as Intel and LSI and the whitebox server providers and entrepreneurs including myself thought: hmm, storage is taking more and more of the IT budget but is not keeping up with new application architectures, new patterns of data generation, and a new generation of storage managers. There has got to be a better way.

And that better way is now called software-defined storage.

Today, storage is much better than it was in 2008, with far better economics, business models that pass the benefits of flash and network improvements on to customers, and a shift towards scale-out, developer-friendly architectures. Much of this change has been forced by Amazon and S3, who set the bar quite high for easy-to-use, massively scalable, and comparatively less expensive storage.

How Cloudian Fits In

Cloudian provides on-premise software-defined storage at 1c/GB. This by itself does not set the company apart, but they made a smart move early on – they bet on Amazon’s S3 API. Instead of inventing another proprietary API in a sea of proprietary APIs, they decided to focus on S3 from day one. This gives them a unique offering – 100% S3-compliant storage that uses metadata in interesting and intelligent ways.

So if you and your developers and the software you are running can interact with S3 – it can interact with Cloudian. Cloudian’s management interfaces blow away the AWS storage GUI, incidentally – giving you a firm-wide view into your data, grouped into blobs, and the policies you’ve applied to these blobs. So your developers are happy because it scales and it is basically S3 out the front and your storage teams are happy because they retain the control and visibility they need to do their jobs. And that’s not to mention “native” multi-tenancy, which is one of many reasons service providers like NTT rely on Cloudian.

The results have been outstanding, with customers providing excellent reviews – and this growing word of mouth has led to a sales acceleration.

I went into the Cloudian offices recently to talk a bit more about my reasons for joining the advisory board, and you can watch the video here:

I’m excited to get a front-row seat as Cloudian continues to ramp up their momentum and grow, and I’m looking forward to getting to know everyone in this wonderful team of people.

Data Availability & Data Protection for the IoT World

New York cityscape

New York, “The City That Never Sleeps”. A very fitting moniker for a city that is full of energy and excitement. Servers located in data centers all around the world are constantly crunching numbers and generating analytics in every financial institution in New York. Why are some of these servers located worldwide? Well, for a variety of reasons, but in my humble opinion, it is to ensure that data is always on and always available. After all, we are talking about billions of dollars in capital electronically managed by the New York Stock Exchange alone.

By 2020, it is predicted that there will be at least 20+ billion internet connected devices. As your business grows, so will the amount of data and storage that you will need. We’ll obviously need solutions to protect our data on-premise or in the cloud. A company that can make sure customers data is always on, secure, highly available, and also protected, rules the IoT WORLD.

modern storage requirementsBut in order to serve and protect your data for the always on, always available IoT world, what requirements should we take into account before deploying any data protection or storage solution? If you are a data protection geek, you’ll most likely see some of your requirements being listed on the right. If you are a data protection solutions provider, you guys definitely rock! Data protection solutions such as Commvault, NetBackup, Rubrik, Veeam, etc. are likely the solutions you have in-house to protect your corporate data centers and your mobile devices. These are software-defined and they are designed to be highly available for on-premise or in-the-cloud data protection.

What about storage? What would you consider? I am sure there are many well-known storage providers you can easily name. But with the new kids on the block disrupting the storage market, would lowering your operating costs ($0.005/GB per month) and meeting the above-listed requirements pique your interest?

Amazon S3 and Cloudian
Cloudian is a software-defined storage company. The solution is fully S3 compliant, which means that if you are familiar with Amazon S3, you’ll love the features that comes with this solution. If you are not, as a data protection geek with more than 15 years of experience, I invite you to give Cloudian HyperStore free trial a shot. The features and capabilities of Cloudian HyperStore as a scale-out storage solution with true multi-tenancy is pretty cool in my books. Imagine being able to deploy and grow storage as you need it for your corporate user home directories, backups, archiving, and even object storage for virtualization solutions (i.e. Red Hat OpenStack). The use cases for scale-out storage solutions are vast. There is no more hardware vendor lock-in as you can easily select between the options of a Cloudian HyperStore appliance or commodity servers to roll-your-own scale-out storage with Cloudian HyperStore software.

Imagine that you, as a storage administrator, can easily provide Storage as a Service (STaaS) to all your users. Take a look at the image below. The granular object level management that is available on a per user basis is pretty sweet. I can provide access to my files/objects with read and/or write permissions, with object level ACL and share the object via a public URL access.
Cloudian object level management

To top it all off, I can also limit the maximum number of downloads of that specific object that I want to share. As a service provider, you can also use the analytics inherent in the solution to implement chargeback to your customers on every account that you manage using Cloudian HyperStore smart storage solution.

Best of all, if you decide that you want to move your data to Amazon, use Cloudian Hyperstore’s built-in auto-tiering feature. Dynamically move your data to Amazon S3 if you choose to do so. You don’t have to take my word for it. Cloudian will provide you with a 45-day free trial. Try it out today.

What is Hybrid Cloud Storage and What Can it Do For You?

IT departments today are forced to come up with innovative ways to deal with the amount of unstructured data that they must manage. Now many of these places are looking to merge the flexibility and scale of the cloud with the security and control of their on-premises IT environment by using a hybrid cloud storage solution. However, until recently, the hybrid cloud storage solution was a mere dream.

CloudianHybridCloud

With the release of the Hyperstore 4.0, Cloudian has transcended this split data system and created a cost-effective data storage solution that begins in the on-premises IT environment, but also integrates with the Amazon cloud infrastructure.

Basic Benefits of Using Hybrid Cloud Storage

Moving towards a hybrid cloud storage system such as the newly developed Cloudian Hyperstore, companies are now able to reap a number of previously unattainable benefits including:

–          Reducing storage acquisition costs

–          Reducing cost of managing storage environment

–          Enjoying Amazon S3 compatible and complementary applications

–          Easily expanding from small (Terabytes) to Petabytes as unstructured data grows

–          Balancing SLAs (service level agreements) using S3 bucket lifecycle policies

 

Cloudian Hyperstore Infrastructure

 

Cloudian Hybrid Cloud

The Cloudian Hyperstore is a hybrid cloud storage software solution for service providers and enterprises. It is fully Amazon S3 compliant and can be seamlessly integrated. It begins with an on-premises data solution and integrates with a cloud data solution for the best of both worlds, particularly for those with large growing amounts of unstructured data.

Hybrid Cloud Storage with Cloudian Hyperstore and Amazon S3

 

– Simone Morellato Director of Technical & Solutions Marketing

The Right S3-Compatible Storage System: Choose Wisely

Applications are driving the escalating need for cloud storage and the public cloud providers saw it coming years ago. Amazon’s foresight dates back to early 2006 when they launched the Amazon Simple Storage Service (S3), a massively scalable and cost effective cloud storage solution developed specifically to house the massive influx of data being created by organizations worldwide. We have since witnessed several milestones for the public cloud giant including the number of objects stored in S3 grow to over 2 trillion, 40 price drops on the service, and the development of a burgeoning ecosystem of over 350+ compatible applications. It’s clear that Amazon has established itself as the dominant leader in public cloud storage. Now hybrid cloud storage is getting its turn in the limelight. For the enterprise, flexibility, control, and resilience are at the forefront of their concerns and hybrid cloud is rapidly shaping up to be the solution of choice for their storage needs. In fact, a recent Rackspace survey indicated that over 60% of the enterprise IT departments surveyed planned to deploy hybrid cloud over the next 3 years.

Amazon S3 holds twice the market share of all its closest competitors combined, so it’s likely that the storage platform of choice for on-premise hybrid or private cloud deployments largely depends on its compatibility with S3. With no standards enforced for claiming S3 compatibility, choosing the right storage platform can be tenuous.

So what does it mean to be S3 compatible? And why does it matter?

Why it matters…

If you have applications that speak to S3 and you are looking to deploy a hybrid cloud storage solution that allows for applications to simply switch between storage targets, then compatibility matters. If you utilize any of the 350+ applications that speak S3 and you want them to operate seamlessly in an on-premise or hybrid cloud environment, compatibility matters. If you have written or plan to write applications that utilize S3 and want them to continue to run without having to rewrite their APIs, then compatibility matters. If ease of migration, cost efficiency, and TCO are important to you, then compatibility matters.

Compatibility with the S3 API can mean the difference between a good decision and costly and time intensive mistake. Understanding the S3 API and the varying levels of compatibility storage platforms offer can seriously impact the outcome for those of you that plan to stand up hybrid or on-premise clouds.

The S3 API

To make S3 simple for applications to access, Amazon built and continues to refine the operations available to applications through the S3 API. The API enables operations to be performed on the S3 Service, S3 Buckets, and S3 Objects. There are 51 total operations available through it. Compatibility is based on a storage platform’s ability to perform some, many, or all of the 51 operations available through S3 API.

S3 Compatibility (1)

Simple Compatibility

There are 9 simple operations available through the S3 API. The “Simple” subset of operations should act as the barebones set of operations that a storage platform must perform in order to claim compatibility. These 9 operations allow for very basic manipulation of data through the API, though it is important to remember that 42 additional operations still remain that are not being executed by a storage platform falling under this category, even though it may boast “S3 compatibility”. The chart below shows the 9 “Simple” operations:

Screen Shot 2014-04-24 at 2.57.08 PM

An example of a storage platform that can claim simple compatibility is SwiftStack. Using middleware called Swift3, SwiftStack can perform the above operations through the S3 API.

Moderate Compatibility

If an application requires just one more operation to be performed through the S3 API other than the 9 “Simple” operations listed above AND you don’t want to have to rewrite your application APIs, then you need to look to storage platforms that carry a bit more robust compatibility. The chart below shows the 9 simple operations (in red) with the addition of 18 moderately complex operations (in yellow). In order to be considered moderately compatible, a storage platform should be able to perform the majority of the 27 operations listed below:

Screen Shot 2014-04-24 at 2.57.21 PM

 

One such storage platform is the open-source unified storage platform, Ceph. Based on the above chart, it can boast “Moderate” S3 compatibility, though robust as it is, even it cannot perform 100% of the operations listed above.

Advanced Compatibility

For those of you that desire peace of mind, knowing that the applications you have written that speak S3 and/or all 350+ S3 compatible applications will continue to work seamlessly with your hybrid or on-premise cloud, then choosing a storage platform that boasts advanced compatibility with the S3 API is vital. Of the 51 operations available through the S3 API, 24 of them are considered advanced. Below is the total set of operations available through the API including simple (red), moderate (yellow), and advanced (green):

Cloudian is the only storage platform that can boast full “Advanced Compatibility”. Additionally, Cloudian is the only storage platform in the “Advanced Compatibility” tier that allows developers continued use of Amazon’s S3 SDK, which significantly eases their workload. Finally, Cloudian is the only storage platform that automatically tiers data between on-premise deployments and Amazon’s S3 public cloud while representing it under a single name space. With this set of advanced functionality, Cloudian stands by its claim as a bug-for-bug match with S3 for on-premise and hybrid cloud deployments.

Too Long; Didn’t Read…

When looking at deploying an open-hybrid cloud and/or moving data between S3 and your private cloud, it is of utmost importance that you understand the level of compatibility your storage platform claims versus its compatibility in reality. S3 is not going anywhere and if you’re reading this, there is a good chance you currently use or plan to use it, either exclusively or in a hybrid cloud environment. Choosing the right storage platform for your hybrid or private cloud can save you tons of money and shave months off of your time to deploy. Compatibility matters.

So, choose wisely.

– Steven Walchek, Business Development

Categories