Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
144 changes: 144 additions & 0 deletions deploy/kubernetes/setup-csi-snapshotter.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
# This YAML file contains all API objects that are necessary to run external
# CSI snapshotter.
#
# In production, this needs to be in separate files, e.g. service account and
# role and role binding needs to be created once, while stateful set may
# require some tuning.
#
# In addition, hostpath CSI driver is hardcoded as the CSI driver.
apiVersion: v1
kind: ServiceAccount
metadata:
name: csi-snapshotter

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: external-snapshotter-runner
rules:
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "create", "delete"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we may do not need create and delete for persistentvolumes

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually this is needed:

E0914 14:17:54.987113 1 reflector.go:205] github.com/kubernetes-csi/external-provisioner/vendor/github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:665: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:serviceaccount:default:csi-snapshotter" cannot list resource "persistentvolumes" in API group "" at the cluster scope

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but the error is for list PVCs, you don't need create/delete ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like I didn't read the comments carefully:). I'll verify.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified that I need to keep create/delete too:

I0915 03:26:41.931543 1 controller.go:1113] provision "default/hpvc" class "csi-hostpath-sc": failed to save persistentvolume "pvc-1e1481d2-b897-11e8-944c-000c2967769a": persistentvolumes is forbidden: User "system:serviceaccount:default:csi-snapshotter" cannot create resource "persistentvolumes" in API group "" at the cluster scope

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xing-yang Yes, you are right, we deploy external-provisioner together with external-snapshotter, so we need this.

- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["list", "watch", "create", "update", "patch"]
- apiGroups: [""]
resources: ["endpoints"]
verbs: ["list", "watch", "create", "update", "delete", "get"]
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "list"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotcontents"]
verbs: ["create", "get", "list", "watch", "update", "delete"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshots"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["apiextensions.k8s.io"]
resources: ["customresourcedefinitions"]
verbs: ["create", "list", "watch", "delete"]

---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: csi-snapshotter-role
subjects:
- kind: ServiceAccount
name: csi-snapshotter
namespace: default
roleRef:
kind: ClusterRole
name: external-snapshotter-runner
apiGroup: rbac.authorization.k8s.io

---
kind: Service
apiVersion: v1
metadata:
name: csi-snapshotter
labels:
app: csi-snapshotter
spec:
selector:
app: csi-snapshotter
ports:
- name: dummy
port: 12345

---
kind: StatefulSet
apiVersion: apps/v1
metadata:
name: csi-snapshotter
spec:
serviceName: "csi-snapshotter"
replicas: 1
selector:
matchLabels:
app: csi-snapshotter
template:
metadata:
labels:
app: csi-snapshotter
spec:
serviceAccount: csi-snapshotter
containers:
- name: csi-provisioner
image: xyang105/csi:csi-provisioner
args:
- "--provisioner=csi-hostpath"
- "--csi-address=$(ADDRESS)"
- "--connection-timeout=15s"
env:
- name: ADDRESS
value: /csi/csi.sock
imagePullPolicy: "IfNotPresent"
volumeMounts:
- name: socket-dir
mountPath: /csi
- name: csi-snapshotter
image: xyang105/csi:csi-snapshotter
args:
- "--csi-address=$(ADDRESS)"
- "--connection-timeout=15s"
env:
- name: ADDRESS
value: /csi/csi.sock
imagePullPolicy: "IfNotPresent"
volumeMounts:
- name: socket-dir
mountPath: /csi
- name: hostpath
image: xyang105/csi:csi-hostpath
args:
- "--v=5"
- "--endpoint=$(CSI_ENDPOINT)"
- "--nodeid=$(KUBE_NODE_NAME)"
env:
- name: CSI_ENDPOINT
value: unix:///csi/csi.sock
- name: KUBE_NODE_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
imagePullPolicy: IfNotPresent
securityContext:
privileged: true
volumeMounts:
- name: socket-dir
mountPath: /csi
volumes:
- name: socket-dir
emptyDir:
11 changes: 11 additions & 0 deletions examples/kubernetes/pvc.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: hpvc
spec:
storageClassName: csi-hostpath-sc
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
15 changes: 15 additions & 0 deletions examples/kubernetes/restore.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: hpvc-restore
spec:
storageClassName: csi-hostpath-sc
dataSource:
name: new-snapshot-demo
kind: VolumeSnapshot
apiGroup: snapshot.storage.k8s.io
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
9 changes: 9 additions & 0 deletions examples/kubernetes/snapshot.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
apiVersion: snapshot.storage.k8s.io/v1alpha1
kind: VolumeSnapshot
metadata:
name: new-snapshot-demo
spec:
snapshotClassName: csi-hostpath-snapclass
source:
name: hpvc
kind: PersistentVolumeClaim
5 changes: 5 additions & 0 deletions examples/kubernetes/snapshotclass.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
apiVersion: snapshot.storage.k8s.io/v1alpha1
kind: VolumeSnapshotClass
metadata:
name: csi-hostpath-snapclass
snapshotter: csi-hostpath
7 changes: 7 additions & 0 deletions examples/kubernetes/storageclass.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-hostpath-sc
provisioner: csi-hostpath
reclaimPolicy: Delete
volumeBindingMode: Immediate