HAproxy with X-forward-for pass to OpenShift 4

Into the thick of it

When we deploy OpenShift 4 in a full Open Source environment it is recommended to use HAproxy as the cluster external Load Balancer.
HAproxy has to ways of load balancing the traffic , The first and the most used is by TCP so the packet does not go through any inspection from the HAproxy side and just change to source IP which is sent onto it’s destination (the router or the API).

In some cases the end user wants to control which source IP addresses are allowed and which denied from using the application. For that they will need the original IP address of the client.

Since HAproxy in TCP mode will not pass the original source IP address ( while not using proxy protocol) we need to use HAproxy with HTTP mode and set it to forward the source IP address with X-forward-for.

Overview

In this tutorial we will set the HAproxy to send X-forward-for to OpenShift 4 and we will show how set up the OpenShift’s route to Allow/Deny the traffic from the source IP.

HAproxy Configuration

First we need to set up HAproxy with the right set of options on the back end and front end services which are relevant to the ingress operator.

In full stack configuration the HAproxy will handle traffic on ports : 80 , 443 , 6443 and 22623. In our case we will change only ports 80 and 443.

Port 80 is simple then 443 (due to the SSL part which we will touch base on later). All we need to do is change the mode from TCP to HTTP and make sure the X-forward-for options are set.

The Following is an example of out relevant section in the haproxy.conf configuration file

We will switch From this :

frontend  openshift-app-http
bind *:80
default_backend openshift-app-http
mode tcp
option tcplog
backend openshift-app-http
balance roundrobin
mode tcp
server worker-01 192.168.1.9:80 check
server worker-02 192.168.1.10:80 check
server worker-03 192.168.1.11:80 check

To this :

frontend  openshift-app-http
bind *:80
mode http
option httplog
default_backend openshift-app-http
option http-server-close
option forwardfor
http-request add-header X-Forwarded-Proto http
http-request add-header X-Forwarded-Port 80
backend openshift-app-http
balance roundrobin
mode http
server worker-01 192.168.1.9:80 check
server worker-02 192.168.1.10:80 check
server worker-03 192.168.1.11:80 check

Now that we’ve finished with the HTTP part we can move on to the HTTPS part which will need use some SSL/TLS skills.

For out HTTPS configuration we need to obtain the wildcard SSL/TLS certificate and add it to our HAproxy configuration to make sure we are addressing OpenShift with SSL enabled.

On our configuration file, We will change on the From this :

frontend  openshift-app-https
bind *:443
default_backend openshift-app-https
mode tcp
option tcplog
backend openshift-app-https
balance roundrobin
mode tcp
server worker-01 192.168.1.9:443 check
server worker-02 192.168.1.10:443 check
server worker-03 192.168.1.11:443 check

To the following :

frontend  openshift-app-https
bind *:443 ssl crt /etc/haproxy/certs.d/wildcard.example.com.crt no-sslv3
mode http
option httplog
option http-server-close
option forwardfor
http-request add-header X-Forwarded-Proto https
http-request add-header X-Forwarded-Port 443
# set HTTP Strict Transport Security (HTST) header (Optional)
http-response add-header Strict-Transport-Security max-age=15768000
# the Backend
default_backend openshift-app-https
backend openshift-app-https
balance roundrobin
server worker-01 192.168.1.9:443 ssl verify check
server worker-02 192.168.1.10:443 ssl verify check
server worker-03 192.168.1.11:443 ssl verify check

In the example above we have a bundle of the CA + certificate including the private key of the wildcard certificate.

In order to concatenate the needed files we can run the following :

#cat wildcard.crt ca.crt wildcard.key > /etc/haproxy/certs.d/wildcard.example.com.crt

In the above example the mentioned files are :

  • wildcard.crt — the wildcard certificate file
  • ca.crt — the certificate authority file
  • wildcard.key — the wildcard certificate key file.

Now all we need to do is restart the haproxy service and make sure all the configuration are in order

In order to test our configuration we can add an annotation to the route and specified the source IP address

metadata:
annotations:
haproxy.router.openshift.io/ip_whitelist: 192.168.1.0/24 192.168.2.10

That’s it

If you have any question feel free to respond / leave a comment.
You can find me on linkedin at : https://www.linkedin.com/in/orenoichman
Or twitter at : https://twitter.com/ooichman

--

--

--

Open Source contributer for the past 15 years

Love podcasts or audiobooks? Learn on the go with our new app.

Create fixed(Elastic) IP lambda in just 3 minutes.

How to set up and run Cypress Test Cases in Google Cloud?

Cypress google cloud integration

Utilising different frontend technologies with Decoupled Drupal architecture

100 Days of AWS — Day 1 — IAM User

Algorithm Practice: Reverse Word walkthrough

Creating a Ledge Grab in Unity (2.5D Platformer) (2) — Climbing Up

Require PHP packages without destroying whole dependencies

This is How to Use ITIL, DevOps, and SRE Best Practices

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Oren Oichman

Oren Oichman

Open Source contributer for the past 15 years

More from Medium

OpenShift 4.8 (and above) with NFS Subdir External Provisioner

Scaling Openshift UPI Clusters Using Central Infrastructure Management

Setup Cortex (Grafana Labs) on GKE

RHACM: Managing Multi-Cloud Multi-Cluster Openshift/Kubernetes