Understanding OpenShift Security

OpenShift is, as per its definition, a container platform, and its role is to run and orchestrate lots and lots of containers. But containers are nothing more than isolated Linux processes, running under very tight conditions and the control of several agents.

The biggest threat on such a platform is that of privilege escalation. Rogue containers might include malicious code that tries to break out from the container where it is running, using mechanisms such as the "mknod" or "setuid" system calls. Privilege escalation is a real threat, and there have been many documented cases of Kubernetes clusters suffering from such attacks.

Red Hat has designed OpenShift to be secure by default by placing more restrictive conditions around containers than other Kubernetes distributions. Such security defaults have a common side effect, well-known by developers working on OpenShift: many container images that work with other Kubernetes distributions do not work with OpenShift. Such behavior can be confusing for developers, and in this section, we will review some of those mechanisms so that your container images can run on OpenShift without issues.

Users and Roles

To be able to operate on an OpenShift project, a DevOps engineer should have at least the "edit" role.

There are also service accounts, used in conjunction with external services such as CI/CD systems, which must be able to access the cluster in a limited but secure way.

Security Context Constraints

OpenShift is configured by default using Security Context Constraints, or SCCs. The values of the SCCs provided by OpenShift are secure by default. Still, they are simply Custom Resource Definitions (CRDs), and your cluster administrator can technically modify them, even if this practice is far from recommendable.

Always check with your cluster administrator to ensure the default security settings of your cluster before building or deploying your cluster images.

Base Images

All container images "inherit" lots of their behavior from their base images. As such, DevOps developers must always consider their images' origin and safety. If your container image is not working on OpenShift, some setting on your base image prevents you from reaching your goal. Review your base images carefully, and ensure they are not running privileged processes or requesting resources unavailable to regular users.

Red Hat provides a range of base images called the Red Hat Universal Base Images, or UBI. These images, distributed through the Red Hat Quay registry, are validated by Red Hat and can be considered a safe option for your container images.

There are Dockerfile examples of non-OpenShift compatible containers in the 02_01 branch of the GitHub repository for this course.

Building Images

Whenever you build images for OpenShift, remember to use the USER command with a value greater than 1000; this will instruct the container runtime to run your application not as root but as a regular user. Your containers must run using non-root accounts to be usable within OpenShift.

Suppose you do not specify a default USER for your image. In that case, OpenShift will run your container using arbitrarily assigned user IDs, which can be problematic if your image requires storing data on a file system.

Network Ports

Another common problem with container images on OpenShift is their use of system network ports, ranging from 0 to 1023. Those ports are privileged ports, and only processes running under the root account can use them. Because your images cannot run on OpenShift as root, you cannot use those ports.

Remember to end your Dockerfiles with the instruction EXPOSE and a number greater or equal to 1024.

A Dockerfile to create containers compatible with Red Hat OpenShift
FROM docker.io/library/nginx:stable

EXPOSE 8080

RUN \
    # Support running as arbitrary user
    chmod g+rwx /var/cache/nginx /var/run /var/log/nginx && \
    # Users are not allowed to listen to privileged ports
    sed -i.bak 's/listen\(.*\)80;/listen 8080;/' /etc/nginx/conf.d/default.conf && \
    # Comment user directive as master process is run as different user
    sed -i.bak 's/^user/#user/' /etc/nginx/nginx.conf

USER 1001:0

Storage

If your container must access data stored on a file system, you might need to "chgrp" and "chmod" the paths in your persistent volumes so that your container can access them. Pay attention to the user ID of the container; run your containers with a defined ID if possible.

Apply "12 Factor" Best Practices

Remember to apply the principles of the Twelve-Factor App when creating your containers. In particular:

  1. Remember to store configuration values as environment variables.

  2. Separate build and run stages for your images using multiple-step builds.

  3. Deploy a single binary per container.

  4. Log everything to "stdout."

  5. Divide your application into various stateless processes.

And always keep your container images as small as possible. Remember that OpenShift can scale your deployments vertically if needed, just like any Kubernetes cluster, so using small and lean images will make your life easier.

Learn more about the Twelve-Factor App at 12factor.net.