The rise of container orchestration storage standards


Containers are everywhere and thanks to Docker, CoreOS, and the Cloud Native Computing Foundation (CNCF), we have container standards pretty much nailed down. When it comes to container orchestration, we have three major contenders, Docker Swarm, Mesosphere, and Kubernetes, with Kubernetes pulling into the lead. But, when it comes to container orchestration storage standards we have — uh, well — a mess. Today, every container management system demands its own storage driver. That’s pretty darn silly. So, there’s an industry afoot to come up with a standard: Container Storage Interface (CSI).

CSI is supported by Google, Red Hat, Hitachi Data Systems, IBM, Mesosphere, and NetApp. It also has support from Docker, Kubernetes, and Cloud Foundry Foundation. Other companies are watching it closely. SUSE CTO Thomas Di Giacomo told me at SUSECon Prague that container orchestration storage “needs to be done,” and that SUSE is watching CSI closely.

Why? Mesosphere put it well, “Docker has DVDI; Kubernetes has Flex volume; and DC/OS supports local volume, DVDI, and REX-Ray. These implementations have different design criteria, incompatible APIs, and implementations with various degrees of quality and reliability. From the storage vendor side, the story looks even more grim. Vendors are forced to prioritize development for one platform over another and embrace a non-trivial testing matrix. The result is a fragmented container storage ecosystem and a rough experience for end users. We as a community can do better.”

CSI’s goal is to have a single, cluster-level volumes plugin API that is shared by all orchestration programs. So, for example, conformant storage plugins written for Docker would run unmodified in Kubernetes (and vice-versa). It was first  proposed by Mesosphere’s founder Benjamin Hindman with support from all the cloud orchestration developers.

The first software plugin to use CSI is incorporated into REX-Ray. This works with the Network File System (NFS) protocol, virtual file system (VFS), and block devices. What it doesn’t do yet is actually work with orchestration programs. Whoops. Mesosphere is already working on this. Soon thereafter, CSI should be working with Apache Mesos.

CSI may soon get a kick in the pants though. Sources say that Red Hat will spearhead efforts to incorporate CSI into Kubernetes. If this happens it might replace Kubernetes FlexVolume. That may be easier than it sounds. FlexVolume enables users to write their own storage volumes drivers and is an alpha feature. That’s a lot of work and CSI can take care of the storage issues in a standard way, Kubernetes users may well welcome it.

The long-term plan is to make CSI a CNCF standard. The project is currently directed by a CNCF working group, WG-Storage. If all goes well by 2018, container orchestration storage will have a real standard and it will be much easier to interoperate between the major container orchestration programs. That will make life much easier for both developers and corporate container users. 


Building the case for Cloud Orchestration – Part 1

CoreOS moves in on cloud DevOps with Kubernetes

Docker now includes built-in container orchestration


  1. […] The rise of the container orchestration storage standards (Article) […]

Speak Your Mind


This site uses Akismet to reduce spam. Learn how your comment data is processed.