How to create custom HTTP monitors with Postman, curl, and Python

The following is a guide on how to monitor your service/microservices using Postman, curl, and Python. BIG-IP enables you to monitor your applications health by creating monitors that emulate your end-user clients. When creating a monitor for API services it can be useful to use the Postman Chrome App to create these monitors.

Monitoring 101

If you’ve gone through the process of creating a custom HTTP monitor you’re familiar with the process of creating the correct “Send String” to use in the monitor. SOL2167 has some examples including the following for a HTTP/1.1 post request:

POST /form.cgi HTTP/1.1\r\nHost: host.domain.com\r\nConnection: Close\r\n\r\nFirst=Joe&Last=Cool

A previous DevCentral article “HTTP Monitoring with POST Request” dives into the details of building the request from scratch, but this can be a bit intimidating. Instead we’re going to use the Postman Chrome App to build the requests.

An alternate way to build a monitor

The Postman Chrome App is a useful add-on for the Google Chrome browser that enables a developer to troubleshoot/test APIs. Features include the ability to modify arbitrary HTTP headers and inspect responses from endpoints.

The first step to building a monitor is to use Postman to construct the request. In the following example I’m sending a request to a simple REST API (httpbin, Python application very useful for testing). In this case I’m sending a POST request with a custom User-Agent to distinguish the monitor from other clients. I’m also specifying a “Accept-Encoding” header to avoid getting back a compressed response and setting the “Connection” header to “Close” to avoid lingering TCP connections on the server.

The body of the request contains a JSON payload that includes some newline characters:

Once I click “Send” in Postman I can see the response from the server. In this case httpbin also sends the headers back to the client in the response:

Creating the new monitor with Python

Now that I have a working request/response I need to convert this into a BIG-IP monitor. The first step is to change the Postman settings to remove the no-cache/Postman Token header (additional headers that are not required). The updated settings look like the following:

Once this is complete click on the “Generate Code” link and select “curl” as the output.

Click on the “Copy to Clipboard” and connect to your BIG-IP via the shell and run the command to verify that your new monitor will also work from the BIG-IP.

Once this is done you can either:

a. Change the output to HTTP and do some finangling to replace newline characters and quote strings

b. Run my Python script from CodeShare that converts the curl input to a BIG-IP monitor Send String

For option b.

After you install the Python script from CodeShare (in this case it is running on the BIG-IP, but you could also run this on your local computer or another server that has Python 2.x available). In this example I replace “curl” with “python curl_to_send_string.py”.

 

Take the output from the script and enter it as the Send String on the BIG-IP

 

Attach the new monitor to your pool and you should be all set. This same method can also be applied to an HTTPS monitor as well.

Applying the monitor

Putting this all together with a Local Traffic Policy you can serve up RESTful APIs that can monitor the health of the supporting services behind it.

The following example is creating a separate policy for requests that start with v1 vs. v2 and sending these to separate pools with monitors that track the health of the service.

From the Network Map you can quickly check on the health of services that are behind an API endpoint.

 

Distributed Systems are Hard

Microservices provide many values for decoupling monolithic systems into more manageable slices, but can lead to an increase in complexity and obscuring of issues. Using BIG-IP and monitors can help reduce the headache of tracking down issues and make it easier to keep your systems up and customers happy.