Chapter 1: Why a CSI Driver

The problem CSI solves

Before 2019, if you wanted Kubernetes to talk to a new kind of storage, you wrote your driver inside Kubernetes itself — in the k8s.io/kubernetes tree, in Go, released on Kubernetes's schedule. Your storage bug fix waited for the next Kubernetes minor version. Your driver shipped with every cluster whether anyone used it or not.

The Container Storage Interface (CSI) is Kubernetes's answer: a small, language-agnostic gRPC contract that lives outside core Kubernetes. You implement the contract, ship your driver as its own container image, and Kubernetes talks to it over gRPC. Your release cycle is yours. Kubernetes doesn't need to know your storage system exists — it only needs to know your driver speaks CSI.

That's the whole idea. Everything else in this book is working out the mechanics of "speaks CSI."

The three services

A CSI driver implements up to three gRPC services. You don't have to implement all of them in one binary — many drivers ship the Controller service in one Deployment and the Node service in a DaemonSet — but conceptually there are three jobs:

  • Identity — "who are you, what can you do?" Every driver implements this. It's how the sidecars (more on those in a moment) discover your driver's capabilities before calling anything else.
  • Controller — the cluster-wide brain. Creates and deletes volumes, attaches and detaches them from nodes, takes snapshots. Runs once (or as a leader-elected few) per cluster, because "create a volume" is a cluster-level decision, not a per-node one.
  • Node — runs on every node as a DaemonSet. Once a volume exists and is attached to a node, the Node service is what actually mounts it into a pod's filesystem, and unmounts it when the pod goes away.

Here's the mental model that makes the split click: Controller decides, Node executes. In the common case, the Controller service handles provisioning and attachment while the Node service handles mounting — this driver's own Controller happens to touch the filesystem too, because "provisioning" for a local-directory backend just is creating a directory, but a real network-storage driver's Controller would only ever call its backend's API, never touch a filesystem directly. Either way, the Node service's job stays the same: run mount on whatever the Controller already made sure exists and is attached to the local node, never talk to the backend's API itself.

Why gRPC, and why that matters to you

CSI is defined as .proto files — Protocol Buffer service definitions. This is worth sitting with for a second, because it explains a lot about how you'll spend your time in this book: you are not going to design an API. The API is already designed, by the CSI spec, and it doesn't change based on what storage backend you're writing. Your job is to implement Go functions with signatures that already exist.

That's a very different exercise from typical Go service development. You won't be deciding "what should CreateVolume return?" — the spec already decided. You'll be deciding "given what CreateVolume must return, how do I make that true for my storage backend?"

This also means the official csi Go package — generated from those same .proto files — gives you every request/response struct, every error code, before you write a line of driver logic. We'll pull it in next chapter.

Sidecars: the part nobody explains well

New CSI developers often get confused here, so let's be explicit: your driver never talks to the Kubernetes API server directly. You don't watch for PersistentVolumeClaims. You don't create PersistentVolume objects. A set of Kubernetes-maintained sidecar containers does that translation for you:

  • external-provisioner watches PVCs, and when one needs a volume, calls your Controller service's CreateVolume over a shared Unix socket.
  • external-attacher watches VolumeAttachment objects and calls your Controller's ControllerPublishVolume.
  • node-driver-registrar tells kubelet "here's a CSI driver at this socket path, here's its name" by calling your Identity service.
  • external-resizer and external-snapshotter do the equivalent for expansion and snapshots.

Each sidecar is a small, boring, well-tested Go program maintained by kubernetes-csi on GitHub. You never write one. You run them as extra containers in the same pod as your driver, sharing a Unix socket over an emptyDir volume. Your driver's entire job is to answer the gRPC calls those sidecars make.

This is why, when you go looking at a CSI driver pod later in this book, you'll see four or five containers in it and only one of them is "yours."

What "idempotent" means here, concretely

The spec requires most RPCs to be idempotent, and this isn't a nice-to-have — sidecars will retry. CreateVolume might get called twice for the same name because the provisioner didn't see your first response before a restart. Your driver has to recognize "I already created a volume for this name" and return the existing volume instead of creating a second one or erroring.

We'll hit this for real in Chapter 7, but it's worth previewing now because it shapes how you should read the spec: every RPC's doc comment is really answering two questions — "what does this do?" and "what does this do if it's called again with the same arguments?" Both matter equally.

What you'll have by the end of this chapter

Nothing runnable yet — that starts next chapter. But you should now be able to answer, without looking anything up:

  • What problem CSI solves that in-tree drivers didn't
  • The job of each of the three services, and the "Controller decides, Node executes" split
  • Why you'll never see your driver call the Kubernetes API directly
  • Why idempotency isn't optional

Chapter 2 sets up the project: Go module, the csi package, and a kind cluster to deploy into. By the end of Chapter 3 you'll have a driver answering its very first real gRPC call over a Unix socket; Chapter 4 is where it actually gets deployed into that cluster and registered with kubelet.