Enterprise Storage Trends Driving New Features in OpenStack

By Ed Balduf, Cloud Solutions Architect, SolidFire

OpenStack has become successful as a cloud software option, and is increasingly being looked to by enterprises as a cost-effective alternative to commercial infrastructure software.  As enterprises consider OpenStack, differences in the requirements and expectations of enterprise customers are pushing OpenStack to change or add features to accommodate them. Let’s explore how these changes are pushing the Cinder block storage subsystem to adapt.

I have had several discussions over the past few months with a number of enterprise OpenStack customers or potential customers who were inquiring about features around:

  • Live migration
  • Quality of Service (QoS) quotas and/or hierarchical quotas
  • Replication
  • Fibre Channel
  • High availability connections
  • Special catalogs for specific users or groups

These feature requests indicate two trends in my opinion: 1) a reluctance to embrace the cloud programming model of application-based failure recovery, and 2) a lack of a universal catalog and pricing chargeback (or at least look-back). Let’s look at each of these in a bit more detail and examine how their requests represent the above changes and their effects on Cinder.

Trend #1: Protect Legacy Applications

Live migration is the epitome of the regression from an application failure recovery scenario to that of an enterprise siloed application model (some people call these the cattle versus pets models). Live migration has generally been associated with shared storage, but properly-designed block storage can also accommodate live migration. OpenStack has accommodated both for a while, but we have discovered and fixed some improper checking in the check code run before a migration.

In addition, we have run into a couple of customers who have built their infrastructure and networks in a way that prevents block storage-based migrations. Specifically, configuration drives that have been constructed on local ephemeral storage cannot be migrated.  You are probably asking, “How this is related to networking?” The alternative to a config drive is a metadata service, but that requires networking facilities (i.e. network address translation) to redirect metadata requests to an appropriate service mechanism.

To date I have not heard of a request for storage live migration, but I can imagine it won’t be long.

Trend #2: Custom Quotas and Provisioning

A request that surprised me the first time, but has subsequently come up several times since, is for the ability to limit the the amount of IOPS a user or project can allocate. Upon the second request, I started asking much deeper questions to find the root of the requests.

Service providers typically produce a catalog and price services such that they can make money. They want customers to buy as much as they need. Enterprises with a fixed budget for hardware (and lack of chargeback) want to carve up chunks of capacity and performance for certain groups, but then allow the groups to carve within the chunk as the group desires. One customer specifically told me, “I want to give the Oracle group 10,000 IOPS and let them do with it what they want: one big volume or 10 at 1,000 IOPS each.”

In a similar line of thinking to quotas, Fibre Channel continues to be an enterprise request. It allows storage admins to continue to control a storage network of their own, and not get mixed up in the corporate networking group.

Cinder’s “hidden volume types” feature continues the enterprise theme of grouping and specialization.  Hidden volume types has already been added to Cinder in the Kilo release, and enterprises clearly are going to take advantage of it to give some special groups abilities that others won’t even know about.

Replication and high availability connections to storage (multi-attach) are two other requests coming principally from the enterprise, because multi-attached volumes provide an additional level of security for a siloed application that doesn’t have application-level redundancy. Storage system replication fits in the same theme, and OpenStack Cinder is working on accommodating both of these techniques in near future code releases.

I have detailed some of the changes in progress or needed in the OpenStack block storage project to better accommodate enterprise needs.  It is clear to me from this list that enterprises like the self-service aspect of OpenStack, but are having trouble embracing the cloud-programming model. As one customer explained to me, “No matter how much I try to change the expectation of users to align with ‘cloud,’ they still complain when I reboot things. So I gotta have live migration.”

As containerization comes to the enterprise, how will it affect the storage requirements?  Currently there is a lot of discussion in the community on how to attach instances to containers dynamically and automatically. Stay tuned for details on how Cinder volumes will work with OpenStack Magnum.

To learn more about how enterprise needs are affecting storage changes in OpenStack, attend “Storage Trends Driving New Features in OpenStack” at OpenStack Silicon Valley on Wednesday, August 26, at 2:05 pm in Breakout 3.

Image by Georgie Pauwels

Leave a Reply

Your email address will not be published. Required fields are marked *