Monitor HTTP response time with curl statistics on OpenShift

Why this article ?

In some cases during our work over Kubernetes Infrastructure we need to run a response test over HTTP to see how long does the response takes to complete.
for that purpose we need a simple command line tools such as CURL.
In sort , cURL is a command-line tool for getting or sending data including files using URL syntax. Since cURL uses libcurl, it supports every protocol libcurl supports.

Where to Begin ?

we want the cURL binary available on our Kubernetes/OpenShift environment and for that we need an image that will contain it. Locally Red Hat Provides use with a ubi-minimal image that comes with curl. but if you don’t have a Red Hat account to can use the minimal image I prepared and can be pulled from : quay.io/ooichman/mini-curl

Now that our tool is in place we need to tell it the format of the output we want to have. The output will go to STDOUT so we do not need persistent storage for this deployment.

The curl binary can get an argument which points to a statics format file with the “-w” argument.
As stated in the man page :
-w, — write-out <format>
Make curl display information on stdout after a completed transfer. The format is a string that may contain plain text mixed with any number of variables. The format can be specified as a literal “string”, or you can have curl read the format from a file with “@filename” and to tell curl to read the format from stdin you write “@-”.

Running on Kubernetes

In some cases running the command once it not enough, you want to create a graph out of those statistics , in order to do that we need to run the script in a loop or through Conman Gate Interface Binary (E.G cgi-bin)

The script is a basically a shell script so we can build a loop with our favorite shell and run the curl command in it.

Let’s go ahead and create the format file. First create the file named loop_curl_statistics.txt

# touch loop_curl_statistics.txt

Next give it the following content :

# cat > loop_curl_statistics.txt << EOF
time_namelookup: %{time_namelookup}s\n
time_connect: %{time_connect}s\n
time_appconnect: %{time_appconnect}s\n
time_pretransfer: %{time_pretransfer}s\n
time_redirect: %{time_redirect}s\n
time_starttransfer: %{time_starttransfer}s\n
----------\n
time_total: %{time_total}s\n
EOF

Now let’s use it as a test run :

# curl -w "@loop_curl_statistics.txt" -o /dev/null -s "http://google.com/"

A good output from out test should be something like :

time_namelookup:  0.091651s
time_connect: 0.152425s
time_appconnect: 0.000000s
time_pretransfer: 0.152458s
time_redirect: 0.000000s
time_starttransfer: 0.232729s
----------
time_total: 0.232847s

For our example I am going to user BASH :

# echo '#!/bin/bash

if [[ -z $DESTINATION_URL ]]; then
echo "No DESTINATION_URL variable was defined"
exit(0);
fi

if [[ -z $TIME_INTERVAL ]]; then
echo "No TIME_INTERVAL variable is set"
exit(1)
fi

while true; do
curl -w "@/opt/app-root/loop_curl_statistics.txt" -o /dev/null -s "$DESTINATION_URL"
sleep $TIME_INTERVAL
done
' > run.sh

Now that the script is in place let’s make it executable :

# chmod a+x run.sh

In our test case we would want to run this loop with in an OpenShift (Or Kubernetes) Cluster to check external service. In order to achieve that we need to run the script in a pod and make sure we give in all the needed environment variables.

In our example we may want to change the loop_curl_statistics.txt file as we go without rebuild the image every time. for that we will create it as a configMap file and make sure it is mounted for our pod deployment.

# oc create configmap loop-curl-statistics --from-file=loop_curl_statistics.txt=loop_curl_statistics.txt

Now let’s create an image for our Pod to use. We will start with the Containerfile :

# cat > Containerfile.loop << EOF
FROM ubi8/ubi-minimal
COPY run.sh /opt/app-root/
RUN chmod a+x /opt/app-root/run.sh
ENTRYPOINT ["/opt/app-root/run.sh"]
CMD ["/opt/app-root/run.sh"]
EOF

And let’s go ahead and build the image

# buildah bud -f Containerfile.loop -t loop-curl-statisticsSTEP 1/5: FROM ubi8/ubi-minimal
Resolved "ubi8/ubi-minimal" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull registry.access.redhat.com/ubi8-minimal:latest...
Getting image source signatures
Copying blob dde93efae2ff done
Copying blob 94249d6f79d2 done
Copying config 30557e4f1c done
Writing manifest to image destination
Storing signatures
STEP 2/5: COPY run.sh /opt/app-root/
STEP 3/5: RUN chmod a+x /opt/app-root/run.sh
STEP 4/5: ENTRYPOINT ["/opt/app-root/run.sh"]
STEP 5/5: CMD ["/opt/app-root/run.sh"]
COMMIT curl-statistics
Getting image source signatures
Copying blob 54e42005468d skipped: already exists
Copying blob 0b911edbb97f skipped: already exists
Copying blob 9db4ab7518b5 done
Copying config 7ddc792ffa done
Writing manifest to image destination
Storing signatures
--> 7ddc792ffa2
Successfully tagged localhost/loop-curl-statistics:latest
7ddc792ffa2144dc2e010527bcaf831fdea89390ace7bac46430000f9952631a

Once the process is complete you can make sure you see the image :

# podman image list | grep loop-curl-statistics
localhost/loop-curl-statistics latest 7ddc792ffa21 13 seconds ago 104 MB

Re Tag it and push in to your registry (for example you can use docker.io or quay.io )

# podman tag localhost/loop-curl-statistics:latest <your registry>/<some library>/loop-curl-statistics:latest# podman push <your registry>/<some library>/loop-curl-statistics:latest

Now that the image and the configMap are in our registry we can go ahead and build our deployment.
First create the following file :

# cat > loop-curl-deployment.yaml << EOF 
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: loop-curl-statistics
name: loop-curl-statistics
spec:
replicas: 1
selector:
matchLabels:
app: loop-curl-statistics
template:
metadata:
labels:
app: loop-curl-statistics
spec:
containers:
- env:
- name: DESTINATION_URL
value: 'https://www.example.com/'
- name: TIME_INTERVAL
value: '5'
image: registry.example.com/library/loop-curl-statistics:latest
imagePullPolicy: Always
name: loop-curl-statistics
volumeMounts:
- mountPath: /opt/app-root/loop_curl_statistics.txt
name: loop-curl-statistics
subPath: loop_curl_statistics.txt
volumes:
- name: loop-curl-statistics
configMap:
defaultMode: 420
items:
- key: loop_curl_statistics.txt
path: loop_curl_statistics.txt
name: loop-curl-statistics
EOF

NOTE!!!
(Make sure you set the image to your registry and mage path)

As you can see in our example we are running a test to www.example.com in a 5 second interval

And let’s go ahead and apply it :

# oc apply -f loop-curl-deployment.yaml

Now that everything is set you can look at the logs of the pod and see the output we are looking for ….

# oc logs $(oc get pods -o name | grep loop-curl-statistics)

That is it

Have fun

--

--

--

Open Source contributer for the past 15 years

Love podcasts or audiobooks? Learn on the go with our new app.

How to load assets asynchronously with Phaser 3

A Little Story About REALTOR Software

Solidity Guide For Beginners

✍Aenco Academy #14 What is KYC?✍

Async Python : it’s easy to do it wrong

Creating Kubernetes cluster using kubeadm on AWS

Scratching the Firebase services with your iOS app

Learning Ruby By Reprogramming a Python Driven Snake Game

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Oren Oichman

Oren Oichman

Open Source contributer for the past 15 years

More from Medium

Rotating AKS Cluster Certificates

Extending VNET for AKS Cluster

3 Steps Creating Self-managed Kubernetes High Availability in Azure for Open5gs [part 3]