OpenShift 4 with Kerberos authentication (Request Header)

Why This Article

Well, because some organization haven’t switched to OpenID Connect yet , Kerberos is the closest Protocol to allow then connect with a Single Sign On to application within the organization

In An Enterprise Company

As we all know in any Large Enterprise company there is a central identity management which holds all of the organization users information.
all of the internal application (should) use that central identity for the triple-A standard which stand for Authentication , Authorization and Audit.

when we are going through an authentication process we are basically telling the application “here is prof the we are who we say we are”. A real life example is showing your ID card at a bank to prove you are who you say you are.
In our tutorial our Authentication mechanism is Kerberos which used a ticket (after first login) in order to check if the user is the user which claims to be

After the user has been authenticated the application needs to go and check if the user is allowed to access the application (or a specific information/page with in the application) and returns the data accordingly.In real life is like a backstage pass at a concert.
In our example there actually 2 authorization mechanism, the first is the Kerberos which needs to make sure the application has permission provide service over Kerberos and the second is a usually a group based authorization which means you have to be a part of a specific group in order to gain access.

The Audit part is unfortunately sometimes neglected but is a very critical part in our process.
Every login no matter if it is successful or unsuccessful should be audit and saved. the best way to find security breaches ore attempts is to go over the logs and see if anyone had login in unreasonable hours or from a far away location which is just not possible that it is the same user.

Kerberos is a computer-network authentication protocol that works on the basis of tickets to allow nodes communicating over a non-secure network to prove their identity to one another in a secure manner.

Apache httpd has a lot of authentication modules. One of the authentication modules is for Kerberos authentication (in combined with and LDAP module can provide authorization as well) which named mod_auth_gssapi .

request Header

A request header identity provider identifies users from request header values, such as X-Remote-User. It is typically used in combination with an authenticating proxy, which sets the request header value. The request header identity provider cannot be combined with other identity providers that use direct password logins, such as HTPasswd, Keystone, LDAP or Basic authentication.

The Architecture

Due to the fact that we want to use this solution for an enterprise environment we need to make sure the solution is highly available and we want the cluster to be autonomous so it’s needs to (or at least should) run on the infra servers ( I case you are using an Infra Server it can run on those as well).

WorkFlow Design

Getting Things Done

The following action can only be run with the permission of a cluster admin

First we will create a directory on out bastion server named openshift-rh

# mkdir openshift-rh
# cd openshift-rh/

Now let’s create a project and add the node selector annotation so that pods will only run on our infra servers :

# oc create ns openshift-request-header

And let’s add a monitoring label so it will be monitored :

# oc label namespace openshift-request-header ""

Now we need to add annotation to the namespace to make sure it will only run on our dedicated servers:

# oc patch ns/openshift-request-header --type merge --patch '{"metadata":{"annotations":{"":""}}}'

And switch our workspace to our newly created namespace :

# oc project openshift-request-header

In this tutorial we will use the name of the route in our scripts so let’s create the service and the route in advanced :

First we will ceate a YAML directory to store the configurations :

# mkdir YAML 
# cd YAML/

Next we will create the service :

# cat > rh-service.yaml << EOF
apiVersion: v1
kind: Service
name: httpd-rh-svc
app: httpd-rh-proxy
namespace: openshift-request-header
- name: login
port: 8443
app: httpd-rh-proxy

And the Route :

# cat > rh-route.yaml << EOF
kind: Route
app: httpd-rh-proxy
name: rh-proxy
namespace: openshift-request-header
targetPort: login
kind: Service
name: httpd-rh-svc
termination: passthrough

Now we need to apply our 2 objects :

# oc apply -f rh-service.yaml -f rh-route.yaml

In any Organization the Security/admin team should maintain an enterprise CA. That CA should sign our Server certificate which will be used by our request header we server.
As an alternative we can use our wildcard certificate which should already be present in our cluster (and signed by our Organization CA).

In our example we will create a Custom CA for both the ingress router and for the client authentication and apply it in out cluster proxy , our bastion trust ca list and in our browser (so we can login from our desktop).

Orginized our Directories :

# cd ..
# mkdir TLS
# cd TLS

Now Let’s create an answer file for our CA :

# cat > openssl_config.cnf << EOF
[ server_auth ]
nsCertType = server
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
[ client_auth ]
nsCertType = client, email
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth
[ v3_ca ]
basicConstraints = critical,CA:true
keyUsage = cRLSign, keyCertSign
[ req ]
default_bits = 2048
prompt = no
default_md = sha512
distinguished_name = dn
attributes = req_attributes
x509_extensions = v3_ca
[ req_attributes ]
challengePassword = A challenge password
challengePassword_min = 4
challengePassword_max = 20
unstructuredName = An optional company name
[ dn ]
ST=New York
L=New York
CN =

Now that our CA answer file is ready we can go ahead and create a Certificate Authority key :

# openssl genrsa -out rootCA.key 4096

And let’s create the CA certificate :

# openssl req -x509 -new -nodes -key rootCA.key -sha512 -days 3655 -out rootCA.crt -extensions v3_ca -config openssl_config.cnf

Once the CA is in place we will create our Server Certificate

you can skip the next part if you already have an ingress router certificate witch is signed by the organization CA.

For our Server authentication request we need to make sure the CN and the DNS alter names are configured by the route FQDN.

# ROUTER_FQDN=$(oc get route rh-proxy -o jsonpath='{}')

Now we will create an answer file for our server CSR :
(change the vaule in the dn section to fit your organization)

# cat > server_csr.txt << EOF[req]
default_bits = 2048
prompt = no
default_md = sha512
req_extensions = req_ext
distinguished_name = dn
[ dn ]C=US
ST=New York
L=New York
[ req_ext ]
subjectAltName = @alt_names
[ alt_names ]

Let’s create a key for the server :

# openssl genrsa -out server.key 4096

And the Certificate request :

# openssl req -new -key server.key -out server.csr -config <(cat server_csr.txt)

Before we sign the CSR we need to modify our openssl config file with our router FQDN :

# export SAN="DNS:${ROUTER_FQDN}"
# sed -i 's/SUBJECT_ALTER_NAME=/subjectAltName=${ENV::SAN}/g' openssl_config.cnf

Now Let’s go ahead and sign the certificate :

# openssl x509 -req -in server.csr -CA rootCA.crt -CAkey rootCA.key -CAcreateserial -out server.crt -days 1024 -sha256 -extfile openssl_config.cnf -extensions server_auth

We can test the alter DNS names with the following command :

# openssl x509 -in server.crt -noout -text | grep DNS

For the client authentication we need to make sure the proxy can authentication over X509 certificate in order to pass the header back to OpenShift so we will create a new key and certificate request and sign it with out new CA (you can sign it with your organization CA just make sure it is “client authenitcaion” enabled.

Let’s create the key :

# openssl genrsa -out client.key 4096

And create the CSR (we don’t need a DNS alter name for this one)

# openssl req -new -sha256 -key client.key \
-subj "/O=system:authenticated/CN=kube-proxy" \
-out client.csr

Now , sign the certificate :

# openssl x509 -req -in client.csr -CA rootCA.crt \
-CAkey rootCA.key -CAcreateserial -out client.crt \
-days 1024 -sha256 \
-extfile openssl_config.cnf -extensions client_auth

In order for everything to work we need to merge between all CA certificates and between the certificate and key of the authentication proxy

First of our CA obtain your Organization CA and copy all of it’s content to a file named router-ca.crt

# cat > router-ca.crt << EOF
< Organization CA >

More so I would suggest adding the cluster CA bundle list :

# oc get cm -n openshift-config-managed default-ingress-cert -o template='{{ index .data "ca-bundle.crt"}}' >> router-ca.crt

Merge the file with our custom CA :

# cat rootCA.crt router-ca.crt > ultimateca.crt

And for our proxy authentication file :

# cat client.{crt,key} > client.pem

Now that we have all the signatures in place we can start and deploy our request header service :


I will cover this part in an other section , we just need to take into our consideration that we have a keytab with the principle of http/<request-header-route>@KERBEROS.DOMAIN.

Custom Image

In our scenario we will want to create and run an httpd image that can run within OpenShift without any special priviliges and provide the request header it exepting for.
In order to atchieve that we will now create a startup file that will generate a VirtualHost file from our environment variables and then start the httpd over the pods.

Go Back one directory and create a new one named image :

# cd .. && mkdir image && cd image/

Now the file looks as such :

# cat > << EOF
if [ -z ${PROXY_ROUTE} ]; then
echo "Environment variable SERVER_FQDN undefined"
exit 1
elif [[ -z ${OAUTH_ROUTE} ]]; then
echo "Environment variable OUATH_FQDN undefined"
exit 1
elif [[ -z ${SSL_CERT_FILE} ]]; then
echo "Environment variable SSL_CERT_FILE undefined"
exit 1
elif [[ -z ${SSL_KEY_FILE} ]]; then
echo "Environment variable SSL_KEY_FILE undefined"
exit 1
elif [[ -z ${SSL_CA_FILE} ]]; then
echo "Environment variable SSL_CA_FILE undefined"
exit 1
elif [[ -z ${PROXY_CA_FILE} ]]; then
echo "Environment variable PROXY_CA_FILE undefined"
exit 1
elif [[ -z ${PROXY_CERT_FILE} ]]; then
echo "Environment variable PROXY_CERT_FILE undefined"
exit 1
echo "
LoadModule mpm_event_module modules/
LoadModule request_module modules/
LoadModule authn_core_module modules/
LoadModule authn_file_module modules/
LoadModule authz_core_module modules/
LoadModule authz_user_module modules/
LoadModule auth_basic_module modules/
LoadModule proxy_module modules/
LoadModule proxy_http_module modules/
LoadModule headers_module modules/
LoadModule ssl_module modules/
LoadModule socache_shmcb_module modules/
LoadModule log_config_module modules/
LoadModule rewrite_module modules/
LoadModule unixd_module modules/
User apache
Group apache
LogLevel trace5
Listen 8443
ServerName ${PROXY_ROUTE}
<VirtualHost *:8443>
" > /tmp/reverse.conf
echo '
CustomLog "logs/ssl_request_log" "%t U=%{SSL_CLIENT_SAN_OTHER_msUPN_0}x %h \"%r\" %b"
' >> /tmp/reverse.conf
echo "
SSLEngine on
SSLCertificateFile ${SSL_CERT_FILE}
SSLCertificateKeyFile ${SSL_KEY_FILE}
SSLCACertificateFile ${SSL_CA_FILE}

SSLProxyEngine on
SSLProxyCACertificateFile ${PROXY_CA_FILE}
# It is critical to enforce client certificates. Otherwise, requests can
# spoof the X-Remote-User header by accessing the /oauth/authorize endpoint
# directly.
SSLProxyMachineCertificateFile ${PROXY_CERT_FILE}
SSLProxyVerify none
RewriteEngine on
RewriteRule ^/challenges/oauth/authorize(.*) https://${OAUTH_ROUTE}/oauth/authorize$1 [P,L]
RewriteRule ^/web-login/oauth/authorize(.*) https://${OAUTH_ROUTE}/oauth/authorize$1 [P,L]
<Location /challenges/oauth/authorize>
AuthType Kerberos
AuthName "Kerberos Login"
KrbMethodNegotiate On
Krb5KeyTab /opt/app-root/rh-proxy.keytab
KrbDelegateBasic on
KrbSaveCredentials On
Require valid-user
RequestHeader set \"X-Remote-User\" %{REMOTE_USER}s
<Location /web-login/oauth/authorize>
AuthType Kerberos
AuthName "Kerberos Login"
KrbMethodNegotiate On
Krb5KeyTab /opt/app-root/rh-proxy.keytab
KrbDelegateBasic on
KrbSaveCredentials On
Require valid-user
RequestHeader set \"X-Remote-User\" %{REMOTE_USER}s
ErrorDocument 401 '<html><body>Client certificate authentication failed</body></html>'
RequestHeader unset X-Remote-User
" >> /tmp/reverse.conf
mv /tmp/reverse.conf /opt/app-root/reverse.conf/usr/sbin/httpd -DFOREGROUND

Once the file has been created we need to make sure it is an executable :

# chmod a+x

Next we will create a Dockerfile that will craete our needed image.
The Dockerfile should look as such :

# cat > Dockerfile << EOF
FROM centos
MAINTAINER Oren Oichman <Back to Root>
RUN yum module enable mod_auth_openidc -y && \
yum install -y httpd mod_auth_gssapi mod_session \
apr-util-openssl mod_auth_gssapi mod_ssl \
mod_auth_openidc openssl && \
yum clean all
COPY /usr/sbin/run-httpd.shRUN echo "PidFile /tmp/" >> /etc/httpd/conf/httpd.conf
RUN echo 'IncludeOptional /opt/app-root/*.conf' >> /etc/httpd/conf/httpd.conf
RUN sed -i "s/Listen\ 80/Listen\ 8080/g" /etc/httpd/conf/httpd.conf
RUN sed -i "s/\"logs\/error_log\"/\/dev\/stderr/g" /etc/httpd/conf/httpd.conf
RUN sed -i "s/CustomLog \"logs\/access_log\"/CustomLog \/dev\/stdout/g" /etc/httpd/conf/httpd.conf
RUN rm -f /etc/httpd/conf.d/ssl.conf
RUN mkdir /opt/app-root/ && chown apache:apache /opt/app-root/ && chmod 777 /opt/app-root/
USER apacheEXPOSE 8080 8443
ENTRYPOINT ["/usr/sbin/"]

Once we have the 2 file ready we can start the image building :

# buildah bud -f Dockerfile -t <registry>/httpd-request-header

And Push it to the registry :

#buildah push <registry>/httpd-request-header

Now that we have the image ready we can move on to the deployment.


we know that we already created a service and a route so now let’s create the secrets , the configMaps and the daemonset.

First let’s start by creating a TLS secret for our Server key and certificate :

# oc create secret tls server-tls --key=server.key --cert=server.crt 

Once the secret is created we can move on for the configMaps.

First we will create a configMap which will contain our request header route and our oauth route.
In order to create the cm we need first to set the 2 variales

# OAUTH_ROUTE=$(oc get route -n openshift-authentication oauth-openshift -o template='{{ }}')
# PROXY_ROUTE=$(oc get route -n openshift-request-header rh-proxy -o jsonpath='{ }')

And now create it :

# oc create cm routes --from-literal=oauth_route="$OAUTH_ROUTE" --from-literal=proxy_route="$PROXY_ROUTE"

Now we need to create the config map for the CA and for the proxy authentication.

# oc create cm custom-ca --from-file=ca.crt=ultimateca.crt
(for the CA file)
# oc create cm proxy-auth --from-file=proxy-auth.crt=client.pem

For our Kerberos configuration we are going to create 2 more configMaps. One for the keytab and one for the krb5.conf file:
(Assuming that the 2 files are under the krb directory)

# oc create cm krb5-conf --from-file=krb5.conf=../krb/krb5.conf
# oc create cm krb5-keytab --from-file=krb5.keytab=../krb/krb5.keytab

Now all we need is to create a daemon set in order to deploy our image with everything put together :

# cd .. && mkdir YAML && cd YAML
# cat > rh-daemonset.yaml << EOF
apiVersion: apps/v1
kind: DaemonSet
name: rh-proxy
app: httpd-rh-proxy
app: httpd-rh-proxy
replicas: 3
app: httpd-rh-proxy
- name: httpd-rh
image: <your registry>/httpd-request-header:latest
- containerPort: 8443
protocol: TCP
- name: server-tls
mountPath: /opt/app-root/server/
- name: custom-ca
mountPath: /opt/app-root/ca/ca.crt
subPath: ca.crt
- name: proxy-auth
mountPath: /opt/app-root/proxyauth/proxy-auth.crt
subPath: proxy-auth.crt
- name: krb5-keytab
mountPath: /etc/krb5.keytab
subPath: krb5.keytab
- name: krb5-conf
mountPath: /etc/krb5.conf
subPath: krb5.conf
name: routes
key: oauth_route
name: routes
key: proxy_route
value: /opt/app-root/server/tls.crt
- name: SSL_KEY_FILE
value: /opt/app-root/server/tls.key
- name: SSL_CA_FILE
value: /opt/app-root/ca/ca.crt
value: /opt/app-root/ca/ca.crt
value: /opt/app-root/proxyauth/proxy-auth.crt
- name: server-tls
secretName: server-tls
- name: custom-ca
name: custom-ca
- name: proxy-auth
name: proxy-auth
- name: krb5-conf
name: krb5-conf
- name: krb5-keytab
name: krb5-keytab

Chage the registry to your image registry and apply the daemon set :

# oc apply -f rh-daemonset.yaml

Once every this is in place all we need to do is to update our cluster’s oauth type.

First we will create a YAML file with the necessary configuration :

# cat > oauth-cluster.yaml << EOF
kind: OAuth
name: cluster
- name: requestHeadersIdP
type: RequestHeader
name: request-header-ca
headers: ["X-Remote-User"]
loginURL: "https://${PROXY_ROUTE}/web-login/oauth/authorize?${query}"
challengeURL: "https://${PROXY_ROUTE}/challenges/oauth/authorize?${query}"

Now all we need to do is apply the YAML file :

# oc apply -f oauth-cluster.yaml


The test Is very simple , from a server/workstation which is part of our Kerberos Domain just make sure you have a kerberos ticket :

# klist -e

And now try to login with the oc command (or through the web console):

# oc login

That is it

If you have any question feel free to responed/ leave a comment.
You can find on linkedin at :
Or twitter at :




Open Source contributer for the past 15 years

Love podcasts or audiobooks? Learn on the go with our new app.

Bridging the Cross-Sector Gap: Breakthrough for the Next Gen of Cyber Security Professionals

📜 What is Rust?

CI/CD is an Agile and DevOps best practice?

Folder Protocol ($FOL) will be listed on @BittrexGlobal FOL-BTC/FOL-USDT market today!

Building something from Scratch — Final

Microservices Communication: Spring Boot + OpenFeign + Client Credentials

21 keys to building successful business software

How to Learn Tekla Structure Design Software?

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Oren Oichman

Oren Oichman

Open Source contributer for the past 15 years

More from Medium

Kubernetes Security Series - Part 1

Node Affinity In Kubernetes

Visualize Prometheus metrics in Grafana

Db2 on Kubernetes