Z.S.K.'s Records

Kubernetes学习(使用PodSecurityPolicy)

PodSecurityPolicy(psp)是kubernetes中提供的一种安全策略, 默认情况下,Kubernetes 允许创建一个有特权容器的 Pod,这些容器很可能会危机系统安全,而 Pod 安全策略(PSP)则通过确保请求者有权限按配置来创建 Pod,从而来保护集群免受特权 Pod 的影响

PodSecurityPolicy 是 Kubernetes API 对象,你可以在不对 Kubernetes 进行任何修改的情况下创建它们,但是,默认情况下不会强制执行我们创建的一些策略,我们需要一个准入控制器、kube-controller-manager 配置以及 RBAC 权限配置

注: kubeadm搭建的集群默认情况下没有开启PodSecurityPolicy, 需要在api-erver的启动参数中开启, 直接修改api-erver的静态yaml文件, –enable-admission-plugins=NodeRestriction,PodSecurityPolicy

下面通过一个例子来说明PodSecurityPolicy的使用

在指定namespace下, 先增加一个禁止pod使用hostNetwork,然后添加一个serviceaccount,可以使用这个特定的serviceaccount能够使用hostNetwork, 其它pod则不可以

在默认情况下,所有pod可以直接指定 hostNetwork=true,要想禁止使用,则需要增加一个psp

因此, 我们将创建2个策略,第一个是提供限制访问的“默认”策略,保证使用特权设置(例如使用 hostNetwork)无法创建 Pod。第二种是一个“提升”的许可策略,允许将特权设置用于某些 Pod

Admission Controller

Admission Controller(准入控制器)拦截对 kube-apiserver 的请求,拦截发生在请求的对象被持久化之前,但是在请求被验证和授权之后.

PodSecurityPolicy添加到 kube-apiserver 上的--enabled-admission-plugins参数中,然后重启 kube-apiserver, 目前参数如下:

1
--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota,NodeRestriction,PersistentVolumeLabel

用于禁止使用hostNetwork的PodSecurityPolicy

新建PodSecurityPolicy

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: restrictive
spec:
privileged: false
hostNetwork: false
allowPrivilegeEscalation: false
defaultAllowPrivilegeEscalation: false
hostPID: false
hostIPC: false
runAsUser:
rule: RunAsAny
fsGroup:
rule: RunAsAny
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
volumes:
- 'configMap'
- 'downwardAPI'
- 'emptyDir'
- 'persistentVolumeClaim'
- 'secret'
- 'projected'
allowedCapabilities:
- '*'

创建clusterrole

1
2
3
4
5
6
7
8
9
10
11
12
13
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: psp-restrictive
rules:
- apiGroups:
- extensions
resources:
- podsecuritypolicies
resourceNames:
- restrictive
verbs:
- use

创建rolebinding

1
2
3
4
5
6
7
8
9
10
11
12
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: psp-default
subjects:
- kind: Group
name: system:serviceaccounts
namespace: kube-system
roleRef:
kind: ClusterRole
name: psp-restrictive
apiGroup: rbac.authorization.k8s.io

上述的对象实现restrictive策略和系统中所有的控制器(对象加在system:serviceaccounts下) ServiceAccount 进行绑定, 这样,所有pod在使用sa时都将会禁止使用hostNetwork.

kubectl apply 之后,对于新生成的pod对象(如果是deployment对象, deployment对象可以创建成功,但是pod的event log会提示以下错误信息),是无法指定hostNetwork, 对于已经使用的pod不影响

1
2
3
#提示的错误信息如下: 
Error creating: pods "nginx-hostnetwork-deploy-6647d65899-" is forbidden: unable to
validate against any pod security policy: [spec.securityContext.hostNetwork: Invalid value: true: Host network is not allowed to be used]

用于允许使用hostNetwork的PodSecurityPolicy

新建PodSecurityPolicy

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: permissive
spec:
privileged: true
hostNetwork: true
hostIPC: true
hostPID: true
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
fsGroup:
rule: RunAsAny
hostPorts:
- min: 0
max: 65535
volumes:
- '*'

创建sa

1
kubectl create sa psp-sa

创建clusterrole

1
2
3
4
5
6
7
8
9
10
11
12
13
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: psp-permissive
rules:
- apiGroups:
- extensions
resources:
- podsecuritypolicies
resourceNames:
- permissive
verbs:
- use

创建clusterrolebinding

1
2
3
4
5
6
7
8
9
10
11
12
13
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: specialsa-psp-permissive
namespace: default
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: psp-permissive
subjects:
- kind: ServiceAccount
name: psp-sa
namespace: default

测试

pod中使用该psp-sa的就可指定hostNetwork,没有使用该sa的无法使用hostNetwork

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-hostnetwork-deploy
namespace: default
labels:
app: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.15.4
hostNetwork: true
serviceAccount: psp-sa #指定sa

如果没有指定serviceAccount=psp-sa, 则会提示以下错误

1
2
3
4
5
6
7
$ kubectl describe rs nginx-hostnetwork-deploy-74c8fbd687
Name: nginx-hostnetwork-deploy-74c8fbd687
......
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedCreate 80s (x15 over 2m42s) replicaset-controller Error creating: pods "nginx-hostnetwork-deploy-74c8fbd687-" is forbidden: unable to validate against any pod security policy: [spec.securityContext.hostNetwork: Invalid value: true: Host network is not allowed to be used]

从这个例子可以了现, psp虽然能够在pod之间做一些限制, 但是力度还是非常粗糙的,如果需要更加细粒度地对流量进行控制,psp做不到, 可以考虑服务网格.

参考文章:

转载请注明出处https://izsk.me

Z.S.K. wechat
Scan Me To Read on Phone
I know you won't do this,but what if you did?