I’ve recently been working to diagnose intermittent latency issues from some HTTP calls that go through an outbound proxy. In an effort to determine whether the latency is at the destination server itself or is coming from the proxy I needed to so some digging.

After spending some time crafting a script in Python I found that I wanted more granular output than what I was getting from the options available using Python requests. This is when I stumbled upon this great github project called HTTPStat that leverages some fancy curl options and displays latency of the request in a waterfall style right in your terminal. https://github.com/reorx/httpstat

The process to install is a simple pip install httpstat. To use simply follow the readme on the github repo. For my case I needed to repeat a request several times in rapid succession and needed to include a few hundred lines of SOAP xml in my request. To do that I did the following:
– Create a soap.xml file containing just the SOAP xml
– Create a python script such as the example below

from subprocess import call

call("httpstat https://example.com -X POST --data-binary @/absolute/path/to/soap.xml --header 'Content-type: text/xml'", shell=True)

Assuming that you’re updated the above python snippet with a proper URI, absolute path, and the headers required you can now call your python script from the cli and you will see output similar to this (note the IPs have been obscured on purpose)

Connected to 111.222.333.444:443 from 444.555.666.777:37436

HTTP/1.1 100 Continue

HTTP/1.1 200 OK
Cache-Control: private, max-age=0
Content-Type: application/soap+xml; charset=utf-8
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Tue, 21 May 2019 21:15:28 GMT
Content-Length: 1254

Body stored in: /tmp/tmpHm_Ts0

  DNS Lookup   TCP Connection   TLS Handshake   Server Processing   Content Transfer
[     4ms    |       9ms      |     83ms      |       215ms       |       377ms      ]
             |                |               |                   |                  |
    namelookup:4ms            |               |                   |                  |
                        connect:13ms          |                   |                  |
                                    pretransfer:96ms              |                  |
                                                      starttransfer:311ms            |
                                                                                 total:688ms

For bonus points if you want to repeatedly call the request to generate multiple samples you can simply run a bash one liner for loop such as this:

 for i in `seq 1 20` ; do python myhttptest.py ; done 

Hopefully this has been helpful.