Issue with external monitor using curl on ntlm site
I need to create a monitor for our share point environment. I first tried with the built-in HTTPS monitor, but it gave a 401 error. After some investigation it seems there is an issue if the service is using NTLM and I was recommended to use an external monitor. With the information I found here I created the following script: !/bin/sh This script expects the following Name/Value pairs: URI = the URI to check USER = username PASSWORD = password RECV = the expected response (case sensitive) remove IPv&/IPv4 compatibility prefix (LTM passes addresses in IPv6 format) IP=`echo ${1} | sed 's/::ffff://'` PORT=${2} PIDFILE="/var/run/`basename ${0}`.${IP}_${PORT}.pid" kill of the last instance of this monitor if hung and log current pid if [ -f $PIDFILE ] then echo "EAV exceeded runtime needed to kill ${IP}:${PORT} $PIDFILE" | logger -p local0.error kill -9 `cat $PIDFILE` > /dev/null 2>&1 fi echo "$$" > $PIDFILE send request and check for expected response if [ $PORT -eq 443 ] then curl -kfNS --ntlm --user ${USER}:${PASSWORD} https://${IP}${URI} | grep "${RECV}" 2>&1 > /dev/null else curl -kfNS --ntlm --user ${USER}:${PASSWORD} http://${IP}:${PORT}${URI} | grep "${RECV}" 2>&1 > /dev/null fi mark node UP if expected response was received if [ $? -eq 0 ] then Remove the PID file rm -f $PIDFILE echo "UP" else Remove the PID file rm -f $PIDFILE fi exit Currently only using 443 so it on it will only use the first curl command, but wanted it able to handle both and I had some issue when I tried to use the second command. The curl command works fine from the F5 cli and if I use "run /util test-monitor intranet_sharepoint_monitor_ext address 10.xxx.xxx.xxx port 443" from tmsh it correctly marks the nodes as up or down. If do the test from the LTM Monitor I get "No successful responses received before deadline" if I try it on the share point that uses NTLM, if I try it on another share point that doesn't use NTLM it works fine. Why would it work with the test-monitor command but not with the actual monitor in the GUI, shouldn't they be the same?! Anyone have any suggestions on what I could do to solve this? Have checked so it doesn't contain any windows characters and I have checked so that file located in /config/filestore/files_d/Common_d/external_monitor_d/ has the correct permissions.API Calls to F5 limited to 1024 KB download
Hi, I am interacting with the F5 API in order to download ASM policies for the purpose of automating the backups. The process works fine however policies larger than 1024 KB are cut off at this size of 1024 KB. Initially I suspected that there was a default limit on the curl request however I have not been able to find information on how to increase this with the curl request. Is this a limitation on the F5 API or the Curl request? wget is not an option as this is not natively supported on the F5 virtual appliance. My script lives on the appliance, downloads the relevant policies and then pushes them to a SMB share. The only issue is that the ASM policies that are larger than 1024 KB are being cut off at 1024 KB. The API calls are as per the documentation here: http://cdn.f5.com/websites/devcentral.f5.com/downloads/icontrol-rest-api-user-guide-13-0-0.pdf specifically: GET https://x.x.x.x/mgmt/tm/asm/policies POST https://x.x.x.x/mgmt/tm/asm/tasks/export-policy GET https://x.x.x.x/mgmt/tm/asm/file-transfer/downloads/$asmPolicy Excluding the processing in my script the API calls I make are shown below: I expect the issue resides in the download api call. Is there a switch I can add to increase this limit? curl -ku 'username:password' -X GET https://x.x.x.x/mgmt/tm/asm/policies | jq '.items[] | "pol_name:" + .name + ";api_id:" + .id' >> $wdir/asmDetails.txt curl -ku 'username:password' -X POST https://x.x.x.x/mgmt/tm/asm/tasks/export-policy -H 'Content-Type: application/json' -d '{"filename":"'$asmPolicy'","policyReference":{"link":"https://localhost/mgmt/tm/asm/policies/'$asmIDs'"}}' curl -ku 'username:password' -X GET https://x.x.x.x/mgmt/tm/asm/file-transfer/downloads/$asmPolicy > $wdir/asmBackup/"$folderName"/$number-$asmPolicy-$hostname-"$dateStamp".xml Thanks
iControlREST and Curl to save and download ASM policies
Hi, I want to be able to save/export asm policies on the F5 and then download. I want to do this using iControlREST and curl. I am able to save UCS files with the post shown below: curl -v -sk -u admin:admin https://myF5/mgmt/tm/sys/ucs -H 'Content-Type: application/json' -X POST -d '{"command":"save","name":"blah.ucs"}' | jq However if I try to do something similar for asm I get errors. Below is what I was trying with asm. curl -v -sk -u admin:admin https://myF5/mgmt/tm/asm/policies/fn9GoMrandomGvoN2dD -H 'Content-Type: application/json' -X POST -d '{"command":"save","name":"as_test.xml"}' | jq The error I get is: { "code": 400, "message": "Could not parse/validate the Policy 'Security Policy /Common/as_test'. Unknown field 'command'", "originalRequestBody": "{\"command\":\"save\",\"name\":\"as_test.xml\"", "referer": "x.x.x.x", "restOperationId": 59083, "kind": ":resterrorresponse" } Thank youLet's Encrypt with Cloudflare DNS and F5 REST API
Hi all This is a followup on the now very old Let's Encrypt on a Big-IP article. It has served me, and others, well but is kind of locked to a specific environment and doesn't scale well. I have been going around it for some time but couldn't find the courage (aka time) to get started. However, due to some changes to my DNS provider (they were aquired and shut down) I finally took the plunges and moved my domains to a provider with an API and that gave me the opportunity to make a more nimble solution. To make things simple I chose Cloudflare as the community proliferation is enormous and it is easy to find examples and tools. I though think that choosing another provide with an open API isn't such a big deal. After playing around with different tools I realized that I didn't need them as it ended up being much easier to just use curl. So, if the other providers have just a somewhat close resemblance it shouldn't be such a big task converting the scripts to fit. There might be finer and more advanced solutions out there, but my goal was that I needed a solution that had as few dependencies as possible and if I could make that only Bash and Curl it would be perfect. And that is what I ended up with 😎 Just put 5 files in the same directory, adjust the config to your environment, and BAM you're good to go!!😻 And if you need to run it somewhere else just copy the directory over and continue like nothing was changed. That is what I call portability 😁 Find all the details here: Let's Encrypt with Cloudflare DNS and F5 REST API Please just drop me a line if you have any questions or feedback or find any bugs.Virtual server details get a 404 with CURL
Hi, I am trying to get the virtual server details from a CURL call: curl -svku "admin:admin" https://0.0.0.0/mgmt/tm/ltm/virtual/virtualtest And I keep getting the following error: {"code":404,"message":"01020036:3: The requested Virtual Server (/Common/virtualtest) was not found.","errorStack":[],"apiError":3} What am I missing? The virtual server does exist. I am using F5 version 12.1.2 Thank you.
Able to see live traffic stats on GUI , but not through REST API
Hi Team, I ran with a strange issue with our customer. We are able to see live traffic on the F5 GUI (V15) as well as when i run # tmsh show sys connection cs-server-addr 10.xxx.xxx.xx cs-server-port 443 , but at the same instance when we run the curl commnd #curl -sk https://10.xxx.xxx.xx/mgmt/tm/sys/connection. I am unable to see the connection details. Is there any know article i can refer to .
Download a BIG-IP UCS archive with "curl".
Problem this snippet solves: Download a BIG-IP UCS archive using the program "curl" and verifies the output file's signature. Tested on 13.1.1. How to use this snippet: Edit the code to input the hostname of your F5 UI, admin credentials, source UCS file name (defaults to config.ucs), and the output file name. Code : #!/bin/bash # # Download a UCS archive (across a stable network) with curl. # #------------------------------------------------------------------------- F5_HOST='myhost.example.com' CREDENTIALS='admin:admin' FINAL_FILE='/tmp/config.ucs' ARCHIVE_NAME_ON_SERVER='config.ucs' DEBUG='' #------------------------------------------------------------------------- # # Get the md5 checksum for the archive. # #------------------------------------------------------------------------- ARCHIVE_CHECKSUM=$(curl -sku $CREDENTIALS -X POST -H "Content-type: application/json" \ -d "{\"command\":\"run\", \"utilCmdArgs\": \"-c '/usr/bin/md5sum /var/local/ucs/$ARCHIVE_NAME_ON_SERVER'\"}" \ https://$F5_HOST/mgmt/tm/util/bash | awk -F':' '{print $NF}' | awk -F'"' '{ print $2 }' | awk '{print $1}') [ -z "$ARCHIVE_CHECKSUM" ] && echo "Failed to get archive signature. Aborting." && exit 1 [ ! -z "$DEBUG" ] && echo "Archive checksum: $ARCHIVE_CHECKSUM" #------------------------------------------------------------------------- # # Find out the size of the archive and the size of the data packet. # #------------------------------------------------------------------------- Content_Range=$(curl -I -kv -u $CREDENTIALS -H 'Content-Type: application/json' -X GET "https://$F5_HOST/mgmt/shared/file-transfer/ucs-downloads/$ARCHIVE_NAME_ON_SERVER" 2>/dev/null | grep "Content-Range: " | cut -d ' ' -f 2) FIRST_CONTENT_RANGE=$(echo -n $Content_Range | cut -d '/' -f 1 | tr -d '\r') [ ! -z "$DEBUG" ] && echo -n "FIRST_CONTENT_RANGE: " [ ! -z "$DEBUG" ] && echo $FIRST_CONTENT_RANGE NUMBER_OF_LAST_BYTE=$(echo -n $FIRST_CONTENT_RANGE | cut -d '-' -f 2) [ ! -z "$DEBUG" ] && echo -n "NUMBER_OF_LAST_BYTE: " [ ! -z "$DEBUG" ] && echo $NUMBER_OF_LAST_BYTE INITIAL_CONTENT_LENGTH=$NUMBER_OF_LAST_BYTE CONTENT_LENGTH=$(($NUMBER_OF_LAST_BYTE+1)) [ ! -z "$DEBUG" ] && echo -n "CONTENT_LENGTH: " [ ! -z "$DEBUG" ] && echo $CONTENT_LENGTH DFILE_SIZE=$(echo -n $Content_Range | cut -d '/' -f 2 | tr -d '\r' ) [ ! -z "$DEBUG" ] && echo -n "DFILE_SIZE: " [ ! -z "$DEBUG" ] && echo $DFILE_SIZE LAST_END_BYTE=$((DFILE_SIZE-1)) CUMULATIVE_NO=0 [ ! -z "$DEBUG" ] && echo "CUMULATIVE_NO: $CUMULATIVE_NO" SEQ=0 LAST=0 #------------------------------------------------------------------------- # # Clean up: Remove the previous output file. # #------------------------------------------------------------------------- /bin/rm $FINAL_FILE 2>/dev/null #------------------------------------------------------------------------- # # Get the archive file. # #------------------------------------------------------------------------- while true do if [ $LAST -gt 0 ]; then [ ! -z "$DEBUG" ] && echo 'End of run reached.' break fi if [ $SEQ -eq 0 ]; then NEXT_RANGE=$FIRST_CONTENT_RANGE CUMULATIVE_NO=$NUMBER_OF_LAST_BYTE CONTENT_LENGTH=$INITIAL_CONTENT_LENGTH else START_BYTE=$(($CUMULATIVE_NO+1)) END_BYTE=$(($START_BYTE + $CONTENT_LENGTH)) if [ $END_BYTE -gt $LAST_END_BYTE ]; then [ ! -z "$DEBUG" ] && echo "END_BYTE greater than LAST_END_BYTE: $END_BYTE:$LAST_END_BYTE" LAST=1 let END_BYTE=$LAST_END_BYTE [ ! -z "$DEBUG" ] && echo "Getting the last data packet." fi NEXT_RANGE="${START_BYTE}-${END_BYTE}" CUMULATIVE_NO=$END_BYTE fi [ ! -z "$DEBUG" ] && echo "NEXT_RANGE: $NEXT_RANGE" let SEQ+=1 [ ! -z "$DEBUG" ] && echo "SEQ: $SEQ" OUTPUT_FILE_NAME="/tmp/$$_downloaded_ucs_archive_file_part_$SEQ"; curl -H "Content-Range: ${NEXT_RANGE}/${DFILE_SIZE}" -s -k -u $CREDENTIALS -H 'Content-Type: application/json' -X GET "https://$F5_HOST/mgmt/shared/file-transfer/ucs-downloads/$ARCHIVE_NAME_ON_SERVER" -o $OUTPUT_FILE_NAME cat $OUTPUT_FILE_NAME >> $FINAL_FILE /bin/rm $OUTPUT_FILE_NAME [ ! -z "$DEBUG" ] && echo "End of loop $SEQ" done #------------------------------------------------------------------------- # # Verify downloaded file. # #------------------------------------------------------------------------- FINAL_FILE_CHECKSUM=$(/usr/bin/md5sum $FINAL_FILE | awk '{print $1}') if [ "$FINAL_FILE_CHECKSUM" == "$ARCHIVE_CHECKSUM" ]; then echo "Download completed and verified." else echo "Downloaded file has incorrect checksum." exit 1 fi # END -------------------------------------------------------------------- Tested this on version: 13.0
Exchange 2016 iApp LTM on newer versions (v14.1 and above)
Hi all, I recently upgraded a existing F5 LTM cluster running 12.1.5.x towards version 15.1.x On the cluster there where several Exchange 2016 iApps (f5.microsoft_exchange_2016.v1.0.2) with all pool monitoring features enabled (IMAPS inbox check). After the upgrade the IMAPS inbox health check with the external monitor "imap_adv_eav" failed. There were several logs regarding authentication errors. After investigation we saw a different behaviour of cURL which is executed withn the external monitor script. This is due to a new version of cURL (7.47.1 instead of 7.25 on 12.1.5). This newer version is in place in all images from 14.1.x till 16.0.x (this is what i tested). SOLUTION: If you want the external monitor to work again, you need to skip the newly added GSSAPI (Kerberos v5) which is build into the newer cURL version. This can be archived by adding the parameter --login-options "AUTH=NTLM" to the curl command in the external script. This will force the system to use NTLM auth like in previous versions and therefor will work as expected again. Greets Markus
External Health monitor scripts
Hello DevCentral Friends: Im having an issue with external monitor scripts, and i wonder if any of you can help. Im trying to create a script to monitor my service at application layer. In BIG IP LTM i add the following info to my external monitor: >ltm monitor external eav_test_monitor { defaults-from external destination *:* interval 5 run /Common/Trails time-until-up 0 timeout 16 user-defined HOST sitefoint.net user-defined URI /v/1/siteservice.svc user-defined RECV siteService Service } >I have around 40 different services (Pools name) all using the the same back-end Server IPs (10.X.X.60, 10.X.X.61 and 10.X.X.62). when applied my ext-monitor to siteinfo.net service, it is also shown on other services (all 40 instances).. >The attached scripts is applied to the ext monitor in BIG-IP. But when the ext health monitors is applied the pool it doesn't work. The Pool goes Down. Logs shows eav failed. Services down due to ext monitor. Any idea what is wrong on the scripts below, or what might be the problem? I have tried with no recv string set as well... #!/bin/sh # # (c) Copyright 1996-2007 F5 Networks, Inc. # # This software is confidential and may contain trade secrets that are the # property of F5 Networks, Inc.No part of the software may be disclosed # to other parties without the express written consent of F5 Networks, Inc. # It is against the law to copy the software.No part of the software may # be reproduced, transmitted, or distributed in any form or by any means, # electronic or mechanical, including photocopying, recording, or information # storage and retrieval systems, for any purpose without the express written # permission of F5 Networks, Inc.Our services are only available for legal # users of the program, for instance in the event that we extend our services # by offering the updating of files via the Internet. # # @(#) $Id: http_monitor_cURL+GET,v 1.0 2007/06/28 16:10:15 deb Exp $ # (based on sample_monitor,v 1.3 2005/02/04 18:47:17 saxon) # # these arguments supplied automatically for all external monitors: # $1 = IP (IPv6 notation. IPv4 addresses are passed in the form #::ffff:w.x.y.z #where "w.x.y.z" is the IPv4 address) # $2 = port (decimal, host byte order) # # Additional command line arguments ($3 and higher) may be specified in the monitor template # This example does not expect any additional command line arguments # # Name/Value pairs may also be specified in the monitor template # This example expects the following Name/Vaule pairs: #URI= the URI to request from the server #RECV = the expected response (not case sensitive) #HOST =the host name of the SNI-enabled site # # remove IPv6/IPv4 compatibility prefix (LTM passes addresses in IPv6 format) #IP=`echo ${1} | sed 's/::ffff://'` NODE=`echo ${1} | sed 's/::ffff://'` if [[ $NODE =~ ^[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}$ ]]; then NODE=${NODE} else NODE=[${NODE}] fi PORT=${2} PIDFILE="/var/run/`basename ${0}`.${HOST}_${PORT}_${NODE}.pid" # kill of the last instance of this monitor if hung and log current pid if [ -f $PIDFILE ] then echo "EAV exceeded runtime needed to kill ${HOST}_${PORT}_${NODE}" | logger -p local0.error kill -9 `cat $PIDFILE` > /dev/null 2>&1 fi echo "$$" > $PIDFILE # send request & check for expected response #curl -fNsk https://${IP}:${PORT}${URI} | grep -i "${RECV}" 2>&1 > /dev/null curl -fNsk --resolve $HOST:$PORT:$NODE https://$HOST$URI | grep -i "${RECV}" > /dev/null 2>&1 # mark node UP if expected response was received if [ $? -eq 0 ] then rm -f $PIDFILE echo "UP" else rm -f $PIDFILE fi exit
