• Recent blogs

    AWS: How to Resize/Extend Persistent Volumes in Kubernetes

    AWS: How to Resize/Extend Persistent Volumes in Kubernetes 

    Resizing Persistent Volumes in Kubernetes

    Disk Expansion in Kubernetes
    Think of a situation where your Workload's Persistent Volume is almost full. How can you expand it? With Kubernetes 1.11 and above, this can now easily be done by just updating the Persistent Volume Claim storage specification.

    My example in this post is tested in AWS hosted opensource Kubernetes Engine.

    Let's get started!

    You can only expand a PVC if its storage class’s allowVolumeExpansion field is set to true.

    Enabling Volume Expansion in Storage Class
    To enable the disk resizing feature, ensure that the Storage Class that your Workload's Persistent Volume Claim uses is configured with "allowVolumeExpansion: true"

    [root@k8s-master manasmonitoring]# kubectl describe storageclass gp2-ebs
    Name:            gp2-ebs
    IsDefaultClass:  Yes
    Annotations:     kubectl.kubernetes.io/last-applied-configuration={"allowVolumeExpansion":true,"apiVersion":"storage.k8s.io/v1","kind":"StorageClass","metadata":{"annotations":{"storageclass.beta.kubernetes.io/is-default-class":"true"},"creationTimestamp":"2020-03-24T14:23:22Z","labels":{"k8s-addon":"storage-aws.addons.k8s.io"},"name":"gp2-ebs","resourceVersion":"282","selfLink":"/apis/storage.k8s.io/v1/storageclasses/gp2","uid":"7de3480c-9711-4087-a8f4-4b7c515e9d91"},"parameters":{"type":"gp2"},"provisioner":"kubernetes.io/aws-ebs","reclaimPolicy":"Retain","volumeBindingMode":"Immediate"}
    Provisioner:           kubernetes.io/aws-ebs
    Parameters:            type=gp2
    AllowVolumeExpansion:  True
    MountOptions:          <none>
    ReclaimPolicy:         Retain
    VolumeBindingMode:     Immediate
    Events:                <none>
    [root@k8s-master manasmonitoring]#

    Resizing a Persistent Volume Claim

    Find the Persistent Volume Claim to resize.

    [root@k8s-master ~]# kubectl get pvc  -n kube-system
    NAME                                      STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
    alertmanager                              Bound    pvc-17d71d1f-d2ea-482a-b37d-648a9f820a2a   5Gi        RWO            gp2-ebs        41h
    grafanastorage-pvc                        Bound    pvc-2ee50955-db6a-4752-9c8f-429708ffc9b7   5Gi        RWO            gp2-ebs        42h
    prometheus-data-prometheus-prometheus-0   Bound    pvc-a4eb87f4-dac0-4e80-952e-6b26b5bd3da0   12Gi       RWO            gp2-ebs        41h
    [root@k8s-master ~]#

    [root@k8s-master grafana-new-manas]# kubectl edit pvc grafanastorage-pvc -n kube-system

    change 5Gi to 6Gi

    apiVersion: v1
    kind: PersistentVolumeClaim
        pv.kubernetes.io/bind-completed: "yes"
        pv.kubernetes.io/bound-by-controller: "yes"
        volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/aws-ebs
        volume.kubernetes.io/storage-resizer: kubernetes.io/aws-ebs
      creationTimestamp: "2020-07-14T10:20:42Z"
      - kubernetes.io/pvc-protection
        addonmanager.kubernetes.io/mode: EnsureExists
        kubernetes.io/cluster-service: "true"
      name: grafanastorage-pvc
      namespace: kube-system
      resourceVersion: "19621218"
      selfLink: /api/v1/namespaces/kube-system/persistentvolumeclaims/grafanastorage-pvc
      uid: 2ee50955-db6a-4752-9c8f-429708ffc9b7
      - ReadWriteOnce
          storage: 6Gi --> Here
      storageClassName: gp2-ebs
      volumeMode: Filesystem
      volumeName: pvc-2ee50955-db6a-4752-9c8f-429708ffc9b7
      - ReadWriteOnce
        storage: 6Gi  --> Here
      phase: Bound

    After successfully modifying the Persistent Volume Claim, Check its status

    So the operation is based on 2 things:

    1. StorageClass should be dynamic.
    2. Kubernetes Version.

    This procedure surely work for version 1.17 and higher as I've tested it. If you have a version Lower than this and If the solution do not work, you can do the following:

    After successfully modifying the Persistent Volume Claim, Check its status

    kubectl get pvc <pvc name> -o yaml

    It should show you type as "type: FileSystemResizePending".

      status: "True"
        type: FileSystemResizePending
      phase: Bound

    Restart the Pod
    The next step is to restart the Pod as recommended by Kubernetes.
    kubectl delete pod <pod name>
    Then wait until the Pod starts running again.

    Check the Status again:
    kubectl get pvc <pvc name> -o yaml

    It should show you now the changed Size.

    No comments