Working with Workload Resources

Workload Validation

Workload resources are Kubernetes resources like Deployments or DaemonSets that create Pods by-way-of a controller. Because many Gatekeeper validation policies are written to enforce against Pods, like those found in the Gatekeeper policy library, it is important to recognize that Gatekeeper Pod violation messages will not be directly reported to the user when using the library as those Pods are created from workload resources.

To reject workload resources that can create a resource that violates a constraint, checkout the Validation of Workload Resources feature available in Gatekeeper v3.10+.

Example

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sPSPPrivilegedContainer
metadata:
  name: psp-privileged-container
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]

The above constraint matches on Pods that pass through the Gatekeeper admission controller. If you create a Deployment with the following PodTemplateSpec, then the Deployment itself will not be blocked, even though the containers in the Deployment violate the constraint.

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: i-wont-be-blocked
  name: i-wont-be-blocked
spec:
  replicas: 1
  selector:
    matchLabels:
      app: i-wont-be-blocked
  template:
    metadata:
      labels:
        app: i-wont-be-blocked
    spec:
      containers:
      - image: nginx
        name: nginx
        securityContext:
          privileged: true

Instead, the Pods that are created from the Deployment will be blocked, and the Gatekeeper denial messages will be found in the workload object responsible for creating the Pods (in this case, the ReplicaSet created by the Deployment). The denial message will eventually make its way into the Deployment's status as well.

status:
  conditions:
  - message: 'admission webhook "validation.gatekeeper.sh" denied the request: [psp-privileged-container]
      Privileged container is not allowed: nginx, securityContext: {"privileged":true}'

Gatekeeper violation messages within statuses can be retrieved using a kubectl command like the following:

$ kubectl get replicaset i-wont-be-blocked-755547df65 -o jsonpath='{ .status.conditions[].message }'
admission webhook "validation.gatekeeper.sh" denied the request: [psp-privileged-container] Privileged container is not allowed: nginx, securityContext: {"privileged": true}

$ kubectl get deploy i-wont-be-blocked -o jsonpath='{ .status.conditions[*].message }'
Deployment does not have minimum availability. admission webhook "validation.gatekeeper.sh" denied the request: [psp-privileged-container] Privileged container is not allowed: nginx, securityContext: {"privileged": true} ReplicaSet "i-wont-be-blocked-755547df65" has timed out progressing.

Note that adding workload objects to the "kinds" list in the Gatekeeper policy library constraints will not block and alert on workload resources. This is because most of the source Rego code for the library constraints match on the spec.containers[_] field instead of the spec.template.spec.containers[_] field that is often the format used by the PodTemplateSpec in workload resources.

input_containers[c] { 
  c := input.review.object.spec.containers[_] 
} 
input_containers[c] { 
  c := input.review.object.spec.initContainers[_] 
} 

Workload Mutation

Similar to Gatekeeper validation policies, Gatekeeper mutation policies can act on individual Pods or the workload resources that generate those Pods. Mutations on workload resource PodTemplateSpecs will implicitly mutate the Pods that are generated by that workload resource. In contrast, mutations on individual Pods will not bubble up to the parent workload resource's PodTemplateSpec.

Use the extent of changes section in mutation policies to granularly specify the scope of a mutation.