k8s对接Samba&CIFS文件共享存储

发表于 分类于

简介

kubernetes提供CSI驱动将外部存储提供给pod中使用，例如我们可以通过CSI驱动对接Samba/CIFS共享文件存储

前提

  • Kubernetes 1.16+

安装

安装SMB CSI驱动（二选一）

通过kubectl安装

1
curl -skSL https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/v0.6.0/deploy/install-driver.sh | bash -s v0.6.0 --

通过helm安装

1
2
helm repo add csi-driver-smb https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/charts
helm install csi-driver-smb csi-driver-smb/csi-driver-smb --namespace kube-system

查看pod状态是否正常

1
kubectl -n kube-system get pod |grep csi-smb

在kubernetes集群中安装Samba服务（可选）

首先创建一个secret保存用户和密码

1
kubectl create secret generic smbcreds --from-literal username=rancher --from-literal password="rancher"

接着创建Samba服务

1
kubectl create -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/deploy/example/smb-provisioner/smb-server.yaml

查看Samba服务状态是否正常

1
kubectl get pod |grep smb

创建storage class

新建storage-class.yaml文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: smb
provisioner: smb.csi.k8s.io
parameters:
  source: "//smb-server.default.svc.cluster.local/share"
  csi.storage.k8s.io/node-stage-secret-name: "smbcreds"
  csi.storage.k8s.io/node-stage-secret-namespace: "default"
  createSubDir: "false"  # optional: create a sub dir for new volume
reclaimPolicy: Retain  # only retain is supported
volumeBindingMode: Immediate
mountOptions:
  - dir_mode=0777
  - file_mode=0777
  - uid=1001
  - gid=1001

其中csi.storage.k8s.io/node-stage-secret-name对应的是上一步创建的secret

创建storage class

1
kubectl create -f storage-class.yaml

部署应用

创建一个statefulset类型的应用，并挂载Samba卷

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: statefulset-smb
  labels:
    app: nginx
spec:
  serviceName: statefulset-smb
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
    spec:
      nodeSelector:
        "kubernetes.io/os": linux
      containers:
        - name: statefulset-smb
          image: mcr.microsoft.com/oss/nginx/nginx:1.19.5
          command:
            - "/bin/bash"
            - "-c"
            - set -euo pipefail; while true; do echo $(date) >> /mnt/smb/outfile; sleep 1; done
          volumeMounts:
            - name: persistent-storage
              mountPath: /mnt/smb
  updateStrategy:
    type: RollingUpdate
  selector:
    matchLabels:
      app: nginx
  volumeClaimTemplates:
    - metadata:
        name: persistent-storage
        annotations:
          volume.beta.kubernetes.io/storage-class: smb
      spec:
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 10Gi

创建该应用

1
kubectl apply -f statefulset-smb.yaml

查看应用状态

1
kubectl get pod

在pod中执行df -h查看Samba挂载情况

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# kubectl exec -it statefulset-smb-0 sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
# df -h
Filesystem                                    Size  Used Avail Use% Mounted on
overlay                                        40G   19G   20G  50% /
tmpfs                                          64M     0   64M   0% /dev
tmpfs                                         3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/mapper/ubuntu--vg-ubuntu--lv              40G   19G   20G  50% /etc/hosts
//smb-server.default.svc.cluster.local/share   40G   21G   20G  52% /mnt/smb
shm                                            64M     0   64M   0% /dev/shm
tmpfs                                         3.9G   12K  3.9G   1% /run/secrets/kubernetes.io/serviceaccount
tmpfs                                         3.9G     0  3.9G   0% /proc/acpi
tmpfs                                         3.9G     0  3.9G   0% /proc/scsi
tmpfs                                         3.9G     0  3.9G   0% /sys/firmware

可以看到，/mnt/smb目录挂载了//smb-server.default.svc.cluster.local/share Samba文件存储

对接 Windows 共享文件夹

创建新的storage class yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: smb-windows
provisioner: smb.csi.k8s.io
parameters:
  source: "//172.16.30.8/share"
  csi.storage.k8s.io/node-stage-secret-name: "smbcreds"
  csi.storage.k8s.io/node-stage-secret-namespace: "default"
  createSubDir: "false"  # optional: create a sub dir for new volume
reclaimPolicy: Retain  # only retain is supported
volumeBindingMode: Immediate
mountOptions:
  - dir_mode=0777
  - file_mode=0777
  - uid=1001
  - gid=1001

其中source填写对应的Windows主机的IP地址和共享文件路径

创建storage class

1
kubectl create -f storage-class-win.yaml

其他的操作都一样，正常创建工作负载对接PVC即可

参考：https://github.com/kubernetes-csi/csi-driver-smb