The rise of container orchestration storage standards

orchestra

Containers are everywhere and thanks to Docker, CoreOS, and the Cloud Native Computing Foundation (CNCF), we have container standards pretty much nailed down. When it comes to container orchestration, we have three major contenders, Docker Swarm, Mesosphere, and Kubernetes, with Kubernetes pulling into the lead. But, when it comes to container orchestration storage standards we have — uh, well — a mess. Today, every container management system demands its own storage driver. That’s pretty darn silly. So, there’s an industry afoot to come up with a standard: Container Storage Interface (CSI).

CSI is supported by Google, Red Hat, Hitachi Data Systems, IBM, Mesosphere, and NetApp. It also has support from Docker, Kubernetes, and Cloud Foundry Foundation. Other companies are watching it closely. SUSE CTO Thomas Di Giacomo told me at SUSECon Prague that container orchestration storage “needs to be done,” and that SUSE is watching CSI closely.

Why? Mesosphere put it well, “Docker has DVDI; Kubernetes has Flex volume; and DC/OS supports local volume, DVDI, and REX-Ray. These implementations have different design criteria, incompatible APIs, and implementations with various degrees of quality and reliability. From the storage vendor side, the story looks even more grim. Vendors are forced to prioritize development for one platform over another and embrace a non-trivial testing matrix. The result is a fragmented container storage ecosystem and a rough experience for end users. We as a community can do better.”

CSI’s goal is to have a single, cluster-level volumes plugin API that is shared by all orchestration programs. So, for example, conformant storage plugins written for Docker would run unmodified in Kubernetes (and vice-versa). It was first  proposed by Mesosphere’s founder Benjamin Hindman with support from all the cloud orchestration developers.

The first software plugin to use CSI is incorporated into REX-Ray. This works with the Network File System (NFS) protocol, virtual file system (VFS), and block devices. What it doesn’t do yet is actually work with orchestration programs. Whoops. Mesosphere is already working on this. Soon thereafter, CSI should be working with Apache Mesos.

CSI may soon get a kick in the pants though. Sources say that Red Hat will spearhead efforts to incorporate CSI into Kubernetes. If this happens it might replace Kubernetes FlexVolume. That may be easier than it sounds. FlexVolume enables users to write their own storage volumes drivers and is an alpha feature. That’s a lot of work and CSI can take care of the storage issues in a standard way, Kubernetes users may well welcome it.

The long-term plan is to make CSI a CNCF standard. The project is currently directed by a CNCF working group, WG-Storage. If all goes well by 2018, container orchestration storage will have a real standard and it will be much easier to interoperate between the major container orchestration programs. That will make life much easier for both developers and corporate container users. 

RELATED LINKS

Building the case for Cloud Orchestration – Part 1

CoreOS moves in on cloud DevOps with Kubernetes

Docker now includes built-in container orchestration

Trackbacks

  1. […] The rise of the container orchestration storage standards (Article) […]

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: