Storage

Attaching volume is like attaching a disk onto VM or mounting a dev onto a folder. Frequent used functions of volume API are: local PV: can mount disk, partition or directory, e.g /mnt/disks/ssd1. hostpath: can only mount local host directory. emptyDir: non-existed space, can be used as shared disk between 2 container in 1 pod. PVC: a way to claim durable storage without knowing the details of the cloud environment. configMap: can mount as disk or file. secret: can mount as disk or file.

Pods can choose above options to mount with mountPath. If you need to limit mounted folder in a specific size and reusable, use PVC (it provides space control capability and persistent, while others such as hostpath/emptyDir shares space with host nodes, are not designed for persistent case). It isolates diskspace and cuts space from PV, which is dedicated used by a pod. PV is associated with storageclass which defines what storage underlay to be used.

When a pod consumes volume PVC, it goes through this process: | POD -> PVC -> PV -> Storageclass | |namespace user| admin user | PVC claims what it wants to use, then it will dig into PV to find whatever matches. User only needs to define PVC if Storageclass is used, k8s can dynamically generate and lock PV if it has a matched storage class. This way, a developer who deployes pod doesn’t need to know how admin defined what storage he’ll use(PVC is per namespace), all he needs to do is choose a class with size. How much effort an admin spend on setting up a storageclass Gold which requires cinder integration(global namespace with admin right), he doesn’t care.

Example of Pod using PVC with cinder storageclass

A pod consumes PVC task-pv-claim

kind: Pod
apiVersion: v1
metadata:
  name: task-pv-pod
spec:
  volumes:
    - name: task-pv-storage
      persistentVolumeClaim:
       claimName: task-pv-claim
  containers:
    - name: task-pv-container
      image: nginx
      ports:
        - containerPort: 80
          name: "http-server"
      volumeMounts:
        - mountPath: "/usr/share/nginx/html"
          name: task-pv-storage

A 3GB standard PVC. This is actually good enough, no need to define PV explicitly as we have local storageclass standard defined.

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: task-pv-claim
spec:
  storageClassName: standard
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 3Gi

A local pv with storageclass standard:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: local-pv-1
  labels:
    type: local
spec:
  capacity:
    storage: 5Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: standard
  hostPath: 
    path: /tmp/data/pv-1

A local storageclass standard, it will consume pv of which has storageclass name field as standard.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: standard
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

Storageclass is like a pool of volumes, like cinder’s volume type in openstack.

create a cinder backend storageclass manually:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: standard
provisioner: kubernetes.io/cinder
parameters:
  availability: nova

Example of EmptyDir

A temp cache folder can be shared accessible by mutliple containers inside a pod.

apiVersion: v1
kind: Pod
metadata:
  name: test-pd
spec:
  containers:
  - image: k8s.gcr.io/test-webserver
    name: test-container
    volumeMounts:
    - mountPath: /cache
      name: cache-volume
  volumes:
  - name: cache-volume
    emptyDir: {}

Example of hostPath

If nothing exists at the given path, an empty file will be created there as needed with permission set to 0644, having the same group and ownership with Kubelet. the files or directories created on the underlying hosts are only writable by root. You either need to run your process as root in a privileged container or modify the file permissions on the host to be able to write to a hostPath volume.

apiVersion: v1
kind: Pod
metadata:
  name: test-pd
spec:
  containers:
  - image: k8s.gcr.io/test-webserver
    name: test-container
    volumeMounts:
    - mountPath: /test-pd
      name: test-volume
  volumes:
  - name: test-volume
    hostPath:
      # directory location on host
      path: /data
      # this field is optional
      type: Directory

Example of Local

apiVersion: v1
kind: PersistentVolume
metadata:
  name: example-pv
spec:
  capacity:
    storage: 100Gi
  # volumeMode field requires BlockVolume Alpha feature gate to be enabled.
  volumeMode: Filesystem  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Delete
  storageClassName: local-storage
  local:
    path: /mnt/disks/ssd1
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - example-node

Example of Binding secret

Generate base64 encoded content

$ echo -n "admin" | base64
YWRtaW4=
$ echo -n "1f2d1e2e67df" | base64
MWYyZDFlMmU2N2Rm
apiVersion: v1
kind: Secret
metadata:
  name: mysecret
type: Opaque
data:
  password: ####
  username: YWRtaW4=

Mount secret as volume

apiVersion: v1
kind: Pod
metadata:
  labels:
    name: db
  name: db
spec:
  volumes:
  - name: secrets
    secret:
      secretName: mysecret
  containers:
  - image: gcr.io/my_project_id/pg:v1
    name: db
    volumeMounts:
    - name: secrets
      mountPath: "/etc/secrets"
      readOnly: true
    ports:
    - name: cp
      containerPort: 5432
      hostPort: 5432

Mount secret as env

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: wordpress-deployment
spec:
  replicas: 2
  strategy:
      type: RollingUpdate
  template:
    metadata:
      labels:
        app: wordpress
        visualize: "true"
    spec:
      containers:
      - name: "wordpress"
        image: "wordpress"
        ports:
        - containerPort: 80
        env:
        - name: WORDPRESS_DB_USER
          valueFrom:
            secretKeyRef:
              name: mysecret
              key: username
        - name: WORDPRESS_DB_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysecret
              key: password