OpenShift 4 with MacVLAN and whereabouts

About this tutorial

One of the must important things when transitioning into Micro Services is to make sure we provide a solution for edge scenarios such as Multicast communication or random clients request from a specific scope.
For those type of scenarios (and more) in Openshift 4 we can setup a CNI named “Multus” which enables us to add another interface to our running pod and those connecting it to another network other then OpenShift SDN.

How do it works ?

Basically because Multus is already part of OpenShift 4 we need to take a few simple steps :

1. setting up the Network Operator
2. adding the network to the namespace
3. configuring the pod with another network (we can also set it up to be the only network).

Option 1 — Network Operator

In order to edit and add another network we need to run “oc edit”:

# oc edit networks.operator.openshift.io cluster

And then add the following values to the spec section :

# cat > cluster-macvlan-network.yaml << EOF
apiVersion: operator.openshift.io/v1 
kind: Network 
metadata:   
  name: cluster
spec:
  additionalNetworks:     
    - name : net-test        
      namespace: openshift-multus    
      type: Raw         
      rawCNIConfig: |-         
        { "cniVersion": "0.3.1", 
          "name": "test", 
          "type": "macvlan", 
          "master": "bond0.100", 
          "ipam": { 
               "type": "whereabouts", 
               "range": "10.10.10.0/24",
               "exclude": [ 
               "10.10.10.0/32",
               "10.10.10.1/32",
               "10.10.10.254/32"
            ],
               "routes": [{"dst": "10.10.10.224/32"}] 
          } 
         }
EOF

My adding the part Above we are creating a new network dedicated for the namespace “test” and we are naming the network “net-test”.

More so we are defining the fact that it is a MacVLAN network and we are using whereabouts as it’s type.

After that comes the interesting part , here we letting the network it’s IP pool and Range and excluding the IP address we do NOT want the whereabouts to attach to one of the pods.

NOTE!!
the IP of the default gateway must be in the range of the pool or it will not work (we will configure the default gateway later on in the pod annotation).

Adding the Network

Now let’s save the file we created and apply it :

# oc apply -f cluster-macvlan-network.yaml

Option 2 — Network Attached definition

As an Alternative we can add A custom resource for the configuration.
The configuration for an additional network is specified from a YAML configuration file, such as in the following example:

# cat > nad-macvlan-network.yaml << EOF
apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  name: net-test
  namespace: openshift-multus
spec:
  config: |-
    { "cniVersion": "0.3.1", 
      "name": "test", 
      "type": "macvlan", 
      "master": "bond0.100", 
      "ipam": { 
        "type": "whereabouts", 
        "range": "10.10.10.0/24",
           "exclude": [ 
             "10.10.10.0/32",
             "10.10.10.1/32",
             "10.10.10.254/32"
           ],
             "routes": [{"dst": "10.10.10.224/32"}] 
      } 
    }
EOF

Adding the Network

Now let’s save the file we created and apply it :

# oc apply -f nad-macvlan-network.yaml

Optional

In case we want the default gateway to remain on the OpenShiftSDN we can add a “gateway” section and destination routes to use that gateway and for all the other network it will use the OpenShiftSDN :

"ipam": { 
  "type": "whereabouts", 
  "range": "10.10.10.0/24",
  ....
  "gateway": "10.10.10.254"
  "routes": [{"dst": "192.168.4.0/24"}]
}

In our scenario we will not use this option.

Confirm that the CNO created the NetworkAttachmentDefinition CR by running the following command. Replace test with the namespace that you specified when configuring the network attachment. There might be a delay before the CNO creates the CR.

# oc get network-attachment-definitions -n openshift-multus
NAME                 AGE 
net-test             14m

Pod Configuration

For the Last part we need to tell the pod to use additional networks and to use our Multus network as our Default Gateway.

In order to set it up we are going to add the following annotation :

annotations:
  k8s.v1.cni.cncf.io/networks: |-
      [
        {
          "name": "net-test", 
          "namespace": "openshift-multus", 
          "default-route": ["10.10.10.254"] 
        }
      ]

Once does configuration have been added the Pod will be available in both networks the OpenShiftSDN and the Multus network (net-test).

As you can notice from the configuration the pod will get the default gateway from the Multus Network but will still have access to the services and the other Pods network (in it’s namespace).

If you have any question feel free to responed/ leave a comment.
You can find on linkedin at : https://www.linkedin.com/in/orenoichman
Or twitter at : https://twitter.com/ooichman

That is it !!!