cancel
Showing results for 
Search instead for 
Did you mean: 
Eric_Chen
F5 Employee
F5 Employee

Problem this snippet solves:

Convert curl commands into a HTTP or HTTPS monitor Send String.

How to use this snippet:

Save this code into a Python script and run as a replacement for curl.

Related Article: How to create custom HTTP monitors with Postman, curl, and Python

An example would be:

% python curl_to_send_string.py -X POST -H "Host: api.example.com" -H "User-Agent: Custom BIG-IP Monitor" -H "Accept-Encoding: identity" -H "Connection: Close" -H "Content-Type: application/json" -d '{
  "hello":
      "world"
}
' "http://10.1.10.135:8080/post?show_env=1"
SEND STRING:
POST /post?show_env=1 HTTP/1.1\r\nHost: api.example.com\r\nUser-Agent: Custom BIG-IP Monitor\r\nAccept-Encoding: identity\r\nConnection: Close\r\nContent-Type: application/json\r\n\r\n{\n  \"hello\":\n      \"world\"\n}\n

Code :

Python 2.x

import getopt
import sys
import urllib

optlist, args = getopt.getopt(sys.argv[1:], 'X:H:d:')
flat_optlist = dict(optlist)

method = flat_optlist.get('-X','GET')

(host,uri) = urllib.splithost(urllib.splittype(args[0])[1])
protocol = 'HTTP/1.1'

headers = ["%s %s %s" %(method, uri, protocol)]
headers.extend([h[1] for h in optlist if h[0] == '-H'])
if not filter(lambda x: 'host:' in x.lower(),headers):
    headers.insert(1,'Host: %s' %(host))


send_string =  "\\r\\n".join(headers)
send_string += "\\r\\n\\r\\n"

if '-d' in flat_optlist:
    send_string += flat_optlist['-d'].replace('\n','\\n')
send_string = send_string.replace("\"", "\\\"")
print "SEND STRING:"
print send_string

Python 3.x

import getopt
import sys

from urllib.parse import splittype, urlparse

optlist, args = getopt.getopt(sys.argv[1:], 'X:H:d:')
flat_optlist = dict(optlist)

method = flat_optlist.get('-X','GET')

parts = urlparse(splittype(args[0])[1])
(host,uri) = (parts.hostname,parts.path)
protocol = 'HTTP/1.1'

headers = ["%s %s %s" %(method, uri, protocol)]
headers.extend([h[1] for h in optlist if h[0] == '-H'])
if not filter(lambda x: 'host:' in x.lower(),headers):
    headers.insert(1,'Host: %s' %(host))


send_string =  "\\r\\n".join(headers)
send_string += "\\r\\n\\r\\n"

if '-d' in flat_optlist:
    send_string += flat_optlist['-d'].replace('\n','\\n')
send_string = send_string.replace("\"", "\\\"")
print("SEND STRING:")
print(send_string)

Tested this on version:

11.5

Comments
Danny_Arroyo
Cirrus
Cirrus

Eric,

 

I just tried your script on v11.4.1 and it worked perfectly. Thanks!!!

 

Buddy_Edwards_1
Nimbostratus
Nimbostratus

I'd like to say first that this is a great piece of code that has saved me a lot of headaches so thank you for creating it. The original script runs great in Python2 but had need for it to run natively in Python3 as the interpreter. This is a modified version that works in Python3.

 

Code import getopt import sys from urllib.parse import splithost, splittype optlist, args = getopt.getopt(sys.argv[1:], 'X:H:d:') flat_optlist = dict(optlist) method = flat_optlist.get('-X', 'GET') (host, uri) = splithost(splittype(args[0])[1]) protocol = 'HTTP/1.1' headers = ["%s %s %s" % (method, uri, protocol)] headers.extend([h[1] for h in optlist if h[0] == '-H']) if not fil`text`ter(lambda x: 'host:' in x.lower(), headers): headers.insert(1, 'Host: %s' % (host)) send_string = "\\r\\n".join(headers) send_string += "\\r\\n\\r\\n" if '-d' in flat_optlist: send_string += flat_optlist['-d'].replace('\n', '\\n') send_string = send_string.replace("\"", "\\\"") print("SEND STRING: " + send_string)
Eric_Chen
F5 Employee
F5 Employee

Awesome! Thank you for sharing.

 

scyonix_360392
Nimbostratus
Nimbostratus

Hi I tried this but health monitoring is showing down, Logs says its a malformed request

 

this is my curl command which works as expected:

 

curl -X POST -H 'content-type: application/json' -H 'host: host.domain.com' -d '{"queryParam":"cleaning","start":"0","rows":"20","searchType":["ALL"],"facetSelection":["SC:catalogs"],"userRoles":[""]}'

 

here is the equivalent send string created using the script which is not working:

 

POST /globalSearch/basicSearch HTTP/1.1\r\ncontent-type: application/json\r\nhost: host.domain.com\r\n\r\n{\“queryParam\“:\“cleaning\“,\“start\“:\“0\“,\“rows\“:\“20\“,\“searchType\“:[\“ALL\“],\“facetSelection\“:[\“SC:catalogs\“],\“userRoles\“:[\“\”]}

 

Can anyone find out what the issue is?

 

Eric_Chen
F5 Employee
F5 Employee

I wonder whether your web server is expecting a content-length. Try

 

POST /globalSearch/basicSearch HTTP/1.1\r\ncontent-type: application/json\r\nhost: host.domain.com\r\nContent-Length: 120\r\n\r\n{\“queryParam\“:\“cleaning\“,\“start\“:\“0\“,\“rows\“:\“20\“,\“searchType\“:[\“ALL\“],\“facetSelection\“:[\“SC:catalogs\“],\“userRoles\“:[\“\”]}

 

scyonix_360392
Nimbostratus
Nimbostratus

no luck, does the braces need to be escaped?

 

Eric_Chen
F5 Employee
F5 Employee

In your curl example above was it http or https? Similar question about whether your monitor was configured to match the same protocol.

 

scyonix_360392
Nimbostratus
Nimbostratus

Curl is http and the protocol monitor is also http

 

Eric_Chen
F5 Employee
F5 Employee

Weird. It looks fine. tcpdump could verify whether any differences. Sending to the same port?

 

Eric_Chen
F5 Employee
F5 Employee

Also. What is the response size?

 

scyonix_360392
Nimbostratus
Nimbostratus

response size is 39Kb.

 

This is the actual request from the logs and i am getting HTTP 400 but curl works just fine:

 

send=POST /globalSearch/basicSearch HTTP/1.1\x0d\x0acontent-type: application/json\x0d\x0ahost: host.domain.com\x0d\x0aContent-Length: 120\x0d\x0a\x0d\x0a{\x0a"queryParam":"cleaning",\x0a"start":"0",\x0a "rows":"20",\x0a"searchType":[\x0a"ALL"\x0a],\x0a"facetSelection":[\x0a"SC:catalogs"\x0a ],\x0a"userRoles":[""\x0a ]\x0a}\x0a

 

Eric_Chen
F5 Employee
F5 Employee

I think you’re hitting this limit (5kb response): https://support.f5.com/csp/article/K3451

 

You could try adding a range header or modify your query to get less data back.

 

scyonix_360392
Nimbostratus
Nimbostratus

My response string appears in the 1st line of the response which is within 5,120 bytes so i doubt this is the problem

 

Eric_Chen
F5 Employee
F5 Employee

What version are you running, here's what I get from my monitor log. Running TMOS 13.1.0.5

 

[0][6132] 2018-05-04 01:07:41.595594: ID 46 :(_send_active_service_ping): writing [ tmm?=false td=true tr=false addr=::ffff:192.168.1.187:80 srcaddr=::ffff:192.168.1.250%0:56436 ] send=POST /test.json HTTP/1.1\x0d\x0acontent-type: application/json\x0d\x0ahost: host.domain.com\x0d\x0a\x0d\x0a{\\xe2\x80\x9cqueryParam\\xe2\x80\x9c:\\xe2\x80\x9ccleaning\\xe2\x80\x9c,\\xe2\x80\x9cstart\\xe2\x80\x9c:\\xe2\x80\x9c0\\xe2\x80\x9c,\\xe2\x80\x9crows\\xe2\x80\x9c:\\xe2\x80\x9c20\\xe2\x80\x9c,\\xe2\x80\x9csearchType\\xe2\x80\x9c:[\\xe2\x80\x9cALL\\xe2\x80\x9c],\\xe2\x80\x9cfacetSelection\\xe2\x80\x9c:[\\xe2\x80\x9cSC:catalogs\\xe2\x80\x9c],\\xe2\x80\x9cuserRoles\\xe2\x80\x9c:[\\xe2\x80\x9c\\xe2\x80\x9d]}

 

Are you using the GUI to create the monitor or using iControl REST?

 

Accenture_Lorea
Nimbostratus
Nimbostratus

Hello,

 

In this example, you are trying to send the monitoring traffic in http but in port tcp/8080 right ? But where are you mentioning that. I want to perform the same type of GET request but in a different port. Where should I mention that for a monitor.

Eric_Chen
F5 Employee
F5 Employee

By default a monitor will use the IP:PORT that is configured for the pool member. If you need to target a specific different port than what the service is using than you can specify a "Alias Service Port" as part of the monitor.

Accenture_Lorea
Nimbostratus
Nimbostratus

Yes Eric, I just found out that couple of minutes back. But thanks for the information..

Version history
Last update:
‎28-Jul-2022 09:02
Updated by:
Contributors