Overview
Official GitHub and reference documentation: GitHub – kubernetes-sigs/nfs-subdir-external-provisioner: Dynamic sub-dir volume provisioner on a remote NFS server.
Deploy nfs-subdir-external-provisioner to provide StorageClass service
Steps
nfs server preparation
/etc/exports
# cat /etc/exports /nfsshare *(rw,no_root_squash,sync)
install nfs-utils
yum -y install nfs-utils systemctl enable --now nfs
namespace
apiVersion: v1
kind: Namespace
metadata:
labels:
kubernetes.io/metadata.name: nfs-server
name: nfs-server
spec:
finalizers:
-kubernetes
class
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: nfs-client
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner # or choose another name, must match deployment's env PROVISIONER_NAME'
parameters:
archiveOnDelete: "true"
# Define path format
pathPattern: "${.PVC.namespace}/${.PVC.annotations.nfs.io/storage-path}"
different presentation methods
rbac
apiVersion: v1
kind: ServiceAccount
metadata:
name: nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: nfs-server
---
kind:ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: nfs-client-provisioner-runner
rules:
- apiGroups: [""]
resources: ["nodes"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "create", "delete"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: run-nfs-client-provisioner
subjects:
- kind: ServiceAccount
name: nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: nfs-server
roleRef:
kind:ClusterRole
name: nfs-client-provisioner-runner
apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: leader-locking-nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: nfs-server
rules:
- apiGroups: [""]
resources: ["endpoints"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: leader-locking-nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: nfs-server
subjects:
- kind: ServiceAccount
name: nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: nfs-server
roleRef:
kind: Role
name: leader-locking-nfs-client-provisioner
apiGroup: rbac.authorization.k8s.io
deployment
apiVersion: apps/v1
Kind: Deployment
metadata:
name: nfs-client-provisioner
labels:
app: nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: nfs-server
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app: nfs-client-provisioner
template:
metadata:
labels:
app: nfs-client-provisioner
spec:
serviceAccountName: nfs-client-provisioner
containers:
- name: nfs-client-provisioner
# image: registry.k8s.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2
image: registry.cn-beijing.aliyuncs.com/xngczl/nfs-subdir-external-provisione:v4.0.0
volumeMounts:
- name: nfs-client-root
mountPath: /persistentvolumes
env:
- name: PROVISIONER_NAME
value: k8s-sigs.io/nfs-subdir-external-provisioner
#Set high availability to allow elections
- name: ENABLE_LEADER_ELECTION
value: "True"
- name: NFS_SERVER
value: 192.168.164.16
- name: NFS_PATH
value: /nfsshare
volumes:
- name: nfs-client-root
nfs:
server: 192.168.164.16
path: /nfsshare
Verify command
kubectl logs -f --tail=20 nfs-client-provisioner-5856c5fc68-2r4k2 -n nfs-server
kubectl get sc
Test
pvc
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: test-claim
annotations:
volume.beta.kubernetes.io/storage-class: "nfs-client"
spec:
storageClassName: nfs-client
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Mi
pod
kind: Pod
apiVersion: v1
metadata:
name: test-pod
spec:
containers:
- name: test-pod
image: busybox:stable
command:
- "/bin/sh"
args:
- "-c"
- "touch /mnt/SUCCESS & amp; & amp; exit 0 || exit 1"
volumeMounts:
- name: nfs-pvc
mountPath: "/mnt"
restartPolicy: "Never"
volumes:
- name: nfs-pvc
persistentVolumeClaim:
claimName: test-claim
Error
No such file or directory
Warning FailedMount 15s (x8 over 79s) kubelet MountVolume.SetUp failed for volume “nfs-client-r oot” : mount failed: exit status 32
Mounting command: mount
Mounting arguments: -t nfs 192.168.164.16:/nfsshare /etc/cni/net.d/pods/67154eef-96d4-46cc-af01-653496a4f1a7/vo lumes/kubernetes.io~nfs/nfs-client-root
Output: mount.nfs: mounting 192.168.164.16:/nfsshare failed, reason given by server: No such file or directory
Reason for error: Configuration file writing error:
After modifying the correct configuration file /etc/exports, you need to restart the nfs service to take effect.
mount mount file, there is a bad super block
Mounting arguments: –description=Kubernetes transient mount for /data/kubernetes/kubelet/pods/2ca70aa9-433c-4d10-8f87-154ec9569504/volumes/kubernetes.io~nfs/nfs-client-root –scope — mount – t nfs 172.16.41.7:/data/nfs_storage /data/kubernetes/kubelet/pods/2ca70aa9-433c-4d10-8f87-154ec9569504/volumes/kubernetes.io~nfs/nfs-client-root
Output: Running scope as unit: run-rdcc7cfa6560845969628fc551606e69d.scope
mount: /data/kubernetes/kubelet/pods/2ca70aa9-433c-4d10-8f87-154ec9569504/volumes/kubernetes.io~nfs/nfs-client-root: bad option; for several filesystems (e.g. nfs, cifs) you might need a /sbin/mount.
helper program. Warning FailedMount 10s kubelet, node1.ayunw.cn MountVolume.SetUp failed for volume “nfs-client-root” : mount failed: exit status 32
Mounting command: systemd-run
Reason for the error: Each Kubernetes node needs to install nfs-utils
yum -y install nfs-utils can be installed
waiting for a volume to be created
Normal ExternalProvisioning 8s (x17 over 3m42s) persistentvolume-controller waiting for a volume to be created, either by external provisioner “k8s-sigs.io/nfs-subdir-external-provisioner” or manually created by system administrator
pvc has been in the pending state, and the event has been waiting.
Check the class.yaml file and deployment file
Reference documentation
mount file, there is a bad super block (solution) – CodeLeading.com (codeleading.com)
k8s uses the new version of NFS Provisioner to configure subdir – Zhihu (zhihu.com)
StorageClass + NFS using persistent volumes in k8s-1.22.3 version (zhihu.com)
k8s-1.22.3 version deploys persistent storage StorageClass + NFS_nfs deploys storageclass_Guihai Tingxue’s blog-CSDN blog
Handling and answering frequently asked questions about K8s production environment (serial, irregular updates) – Alibaba Cloud Developer Community (aliyun.com)
Releases · kubernetes-sigs/nfs-subdir-external-provisioner (github.com)
k8s learning: Deploy dynamic pvc (nfs-subdir-external-provisioner)_dynamic pvc_Hahahu 123’s blog-CSDN blog