Testing Ports availability from OpenShift to external Service

Internal work processes

In Today’s IT world firewall is a very big part of the internal (east to west) communication just as the external (north to south) communication and more the once we are asking the security team to open ports for use and as you know where there is a human intervention there is bound to be obstacles.

Micro service world

In a micro service world sense we are dealing with a “cluster” of servers more then once maybe one or more cluster nodes can be left out of the firewall rule which as a result will prevent our micro service to access an external service if it spawn one of those nodes. So a question may rise. How do we test a micro service on a specific node we suspect ?

Portcheck Applicaion

In our line of work we will (or we have) came across this scenario so we want to be prepared if such scenario occur.An open source project named “portcheck” (the code source can be found here).

Obtaining portcheck.

There are 2 ways of obtaining portcheck

1. download the source and create the image
2. download the image directly

Compiling from source

Compiling from source is a simple process. we need to obtain the source from git and build the image from the source code.
If you are in a disconnected environment then you will need to modify the Containerfile files with the available images in your internal directory.

Download the git from the source code :

# git clone https://gitlab.com/two.oes/portcheck.git && cd portcheck

As a good practice set the registry FQDN as an environment variable in your running session :

# export REGISTRY=<your registry>

First we will create the “portcheck” image. for that we can go ahead and run the following command :

# buildah bud -f Containerfile.portcheck -t ${REGISTRY}/portcheck/portcheck

Download the Image

to download the image all you need to do is to access the gitlab registry and obtain the image :

# podman pull registry.gitlab.com/two.oes/portcheck/portcheck:latest

Deployment

Now that the image is ready you can go ahead and deploy the image on our cluster. Because the image is very lite and we want to make sure we cover the nodes randomly we can use the DaemonSet and make sure we have the tool on each node.
Modify the image accordingly (if you are in a disconnected environment ) and apply the Daemonset :

# oc apply -f  Deployment/portcheck-daemonset.yaml

Once all the Pods are running we can create the service and the route for our application :
For the Service :

# oc apply -f Deployment/portcheck-svc.yaml

For the route:

# oc apply -f Deployment/portcheck-route.yaml

Testing

portcheck expect a JSON content data in a POST method over HTTP for several reasons. The main reason is to make it a simple integration tool with other workflows you many have or for example integrate it with Ansible playbook using the URI module.
To make usage much more simple I wrote A BASH script that will help us run curl with the expected data.

under the “test” directory we can run the “test_portcheck.sh” script with arguments.
For example , if we want to test if port 8080 TCP is open on server with the IP address of 1.1.1.1 and our route URL is portcheck.example.com then the command should look as such :

# ./test/test_portcheck.sh -r 1.1.1.1 -p TCP -n 8080 \
   -u http://portcheck.example.com

This will trigger portcheck to run the test and return a JSON if the port is Open or close.

NOTE
Feel free to look at the script and fit it to your needs as you wish.

Spearedge application

portcheck is great to check whether a random node in the cluster has access to the remote port or not but in some cases we would like to test a specific node which for this purpose we will need spearedge.
The way that spearedge work is it allow us to list all the nodes in the cluster with the given URI of <FQDN>/listnodes and then run the same JSON data as portcheck with the node option that will spawn a portcheck Pod on that node, run the JSON test , get the return code and delete the POD once the process test is completed.

Compiling from source

Just as we did with portcheck , As a good practice set the registry FQDN as an environment variable in your running session :

# export REGISTRY=<your registry>

First we will create the “spearedge” image. for that we can go ahead and run the following command :

# buildah bud -f Containerfile.spearedge -t ${REGISTRY}/portcheck/spearedge

Download the Image

to download the image all you need to do is to access the gitlab registry and obtain the image :

# podman pull registry.gitlab.com/two.oes/portcheck/spearedge:latest

deployment

Before we deploy the spearedge application with need to create a service account and provide it role and role binding so it will be able to list all the nodes in the cluster and create the portcheck pod where it needs to:

# oc apply -f Deployment/serviceaccount.yaml

For the role and role binding :

# oc apply -f Deployment/clusterRole-listNodes.yaml \
   -f Deployment/clusterRoleBinding.yaml

And to allow it to create the portcheck pod :

# oc apply -f Deployment/role-pods.yaml \
  -f Deployment/rolePodBinding.yaml

Now we need to deploy the deployment :

# oc apply -f Deployment/spearedge-deployment.yaml

Let’s make sure the application is accessible through the route :

# oc apply -f Deployment/spearedge-svc.yaml \
  -f Deployment/spearedge-route.yaml

Testing spearedge

Like portcheck we have a bash script to simple test the spearedge as well. we can run the command with the relevant argument.

Run the following command :

# ./test/test_spearedge.sh [ -l ] || -u <URL> -p <Protocol> -n <Port Number> -r <remote host> -h <OpenShift Hostname>

For example if we want to list the nodes on the cluster we can run :

# ./test/test_spearedge.sh -l -u \
https://$(oc get route spearedge -o jsonpath='{.spec.host}')

Another example is if we want to run a port test from node name “node01” on port UDP 4431 where the remote destination is 1.1.1.1 we will run the following command :

# ./test/test_spearedge.sh -l -u \
https://$(oc get route spearedge -o jsonpath='{.spec.host}') \
-p UDP -n 4431 -r 1.1.1.1 -h node01

That is it !!!

If you have any question feel free to respond/ leave a comment.
You can find me on linkedin at : https://www.linkedin.com/in/orenoichman
Or twitter at : https://twitter.com/ooichman