Add or delete a parameter from multiple ASM policy or modify multiple ASM policy via API (iControlREST)
Problem this snippet solves: Sometimes it is necessary to add a parameter into multiple policy or all policies or to delete a parameter from multiple policies. If you have hundreds of asm polices and you try to do it via GUI, It takes long time and It is boring. For example, you have a new vulnerability scanner and you want to add all policies, or your contract with a security analysis company and you want to delete their IP address from all asm policies. If you have lots of policy, this gets big issue. How to use this snippet: I wrote a sample bash script, It adds an IP into the trusted IP list of multiple asm policy or deletes an IP from the trusted IP list of all asm policies. Firstly, you must choose which asm polices you want to change. Use this command to get list of the asm policies and write it into a file(asmPolicies.txt😞 curl -k -u <admin>:<password> -H "Content-Type: application/json" -X GET https://<F5 IP Address>/mgmt/tm/asm/policies?$select=id,name,fullPath | jq -r '.items[] | "\(.id) \(.name) \(.fullPath)"' > asmPolicies.txt This is the sample content of an asmPolicies.txt [root@f5 asmPolicies]# cat asmPolicies.txt x3yyOJTe3CvcWJDMqpnrgQ First /Common/First RqXf73h6qZY94EFGVDSlbg SecPolManual_First /Common/SecPolManual_First d928o8by0WBrWdW7oadMQg SecPol-Lab14 /Common/SecPol-Lab14 i4LnoF4GwMKRhTZ81RCeSQ SecPol-Lab14.2 /Common/SecPol-Lab14.2 kLoqhuDoa6bEeBjcrFo4VA SecPol-Lab15.1 /Common/SecPol-Lab15.1 DvE_fPp2tLUZvJi8cb8Rpg SecPol-Lab15.2 /Common/SecPol-Lab15.2 52dxLNxjExt6QRNvbg7fHA SecPol-Lab15.3 /Common/SecPol-Lab15.3 DcSvljkbLZQD19adkVdV3A SecPol-Lab16.2 /Common/SecPol-Lab16.2 rJ6Mt9sPxzgLu6WHyyifLg SecPol-Lab16.4 /Common/SecPol-Lab16.4 Sy_0vNh-5VXal-xDlMXMqw Single_URI /Common/Single_URI Hzyj8pZF6flV3VhTkCFkig SecPol-Lab22.2 /Common/SecPol-Lab22.2 sPR5LNQrrf29I1xZ8MtcRA SecPol-Lab16.4_2 /Common/SecPol-Lab16.4_2 Secondly, check the asmPolicies.txt, and erase the lines which policies you dont want to change Last, copy updateAsmPolicies.sh(attached) in a directory, then run updateAsmPolicies.sh with an appropriate command and parameter Usage: updateAsmPolicies.sh command parameter Commands: -a, -add add an IP address into the trusted IP list -d, -delete delete an IP address from the trusted IP list -c, -change <orgIP-newIP> delete the orgIP from the trusted IP list, then add the newIP into the trusted IP list updateAsmPolicies.sh -a 1.1.1.1 -> adds 1.1.1.1 into the trusted IP list updateAsmPolicies.sh -d 1.1.1.1 -> delete 1.1.1.1 from the trusted IP list that is it. This is just a sample. Code : #!/bin/bash #### #### AUTHOR: FARUK AYDIN --- farukaydin at yahoo.com #### DATE: 2018.01.25 #### This script adds or deletes or changes the trusted IP addresses in the asm policies #### #### Prerequest commands: ####echo ####curl ####jq ####shift ####cut ####cat function usage { echo "Usage: $0 command parameter" echo "Commands:" echo "-a, -add add an IP address into the trusted IP lists" echo "-d, -delete delete an IP address from the trusted IP lists" echo "-c, -change delete the orgIP from trusted IP lists, then add the newIP into the trusted IP lists" exit 0 } if [ ${#@} == 0 ]; then usage fi addingIP() { echo adding $2 into $1 policy; curl -sk -u ${f5user}:${f5pass} -H "Content-Type: application/json" -X POST -d '{"ipAddress":"'"$2"'","ipMask":"255.255.255.255","trustedByPolicyBuilder":true}' https://${f5host}/mgmt/tm/asm/policies/$1/whitelist-ips } deleteIP() { md5IP=$(curl -sk -u ${f5user}:${f5pass} -H "Content-Type: application/json" -X GET https://${f5host}/mgmt/tm/asm/policies/$1/whitelist-ips | jq -r '.items[] | select(.ipAddress=="'"$2"'") |"\(.id)"') if [ -z "$md5IP" ]; then echo $2 is not found in $1 policy; else echo deleting $1 from $1 policy; curl -sk -u ${f5user}:${f5pass} -H "Content-Type: application/json" -X DELETE https://${f5host}/mgmt/tm/asm/policies/$1/whitelist-ips/${md5IP} fi } UNKNOWN=() param=0 whatTodo="nothing" whatToDoN=0 f5user="admin" f5pass="password" f5host="192.168.1.245" while [[ $# -gt 0 ]] do key="$1" case $key in -a|--add) ((param++)) addIP="$2" whatToDo="adding a new trusted IP(${addIP}) to all asm policies" whatToDoN=1 shift # past argument shift # past value ;; -d|--delete) ((param++)) delIP="$2" whatToDo="deleting the trusted IP(${delIP}) from all asm policies" whatToDoN=2 shift # past argument shift # past value ;; -c|--change) ((param++)) changeIP="$2" orgIP=$(echo $changeIP | cut -f1 -d-) newIP=$(echo $changeIP | cut -f2 -d-) if [ "${orgIP}" == "${newIP}" ] ; then orgIP=$(echo $changeIP | cut -f1 -d:) newIP=$(echo $changeIP | cut -f2 -d:) fi whatToDo="changing the trusted IP(${orgIP}) with the new IP(${newIP}) in all asm policies" whatToDoN=3 shift # past argument shift # past value ;; --default) DEFAULT=YES ((param++)) shift # past argument ;; *) # unknown option UNKNOWN+=("$1") # save it in an array for later shift # past argument ;; esac done if [ "${param}" -gt 1 ] ; then echo "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!" echo "!!!!!!!! you used ${param} commands !!!!!!!!" echo "!!! you must use only one command !!!" echo "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!" usage fi echo "${whatToDo}", Option: "${whatToDoN}" for i in $(cat asmPolicies.txt | cut -d " " -f 1); do case $whatToDoN in 1) addingIP $i $addIP ;; 2) deleteIP $i $delIP ;; 3) deleteIP $i $orgIP addingIP $i $newIP ;; esac done Tested this on version: 12.1
BIG-IP : iControl LocalLBDataGroupFile.set_local_path()
F5 BIG-IP LTM VE v11.4.0 on ESXi iControl re-cache data-group operation : LocalLBDataGroupFile.set_local_path() For a live prod BIG-IP cluster with a VIP iRule that is actively reading data from the data-group , is the above considered a risky operation ? e.g. attempt live-swap of data-group's data-file, fails due to locks or other systems-level issues, data-group now unavailable to iRule I've attempted live-update of a data-file's contents via BIG-IP browser admin ( copy/paste new contents and click “Update” ) and seen the corresponding data-group become unavailable to the iRule. The paranoid approach is to create a new data-group/file and a copy of the iRule modified to point to this new data-group , and then in the VIP’s iRule list swap old/new iRules – so that never actually update a “live” data-group. So now I am concerned about performing similar operation via iControl LocalLBDataGroupFile.set_local_path() –- although apparently this API is not actually live-updating a cached data-file but rather re-pointing data-group to a new cached data-file ?BIG-IP : device-side timeout applied to iControl operations ?
F5 BIG-IP LTM VE v11.4.0 on ESXi The iControl API classes accept a timeout parameter - which to be safe I set to 3600000 ms = 1 hr ( how to set to infinite ? ) However calls to LocalLB.DataGroupFile.set_file_path() API ( that when successful complete in under 10 mins ) sometimes return this error : The underlying connection was closed: An unexpected error occurred on a send. Is it possible that BIG-IP is applying its own internal timeout to iControl requests ? If so, how to configure my BIG-IP device's internal timeout ?
Issue setting route domain parent with PowerShell / iControl
Good morning, I am attempting to script the configuration of an existing route domain. I am able to set the description, set strict isolation, and other settings, but when I attempt to set the parent ID I am running into some strange behavior. I am attempting to configure a route domain with an ID of 6 to have a parent that has an ID of 501. If I run the script below I receive the following error: error_string : 01070734:3: Configuration error: Invalid route domain modification. A domain id change from 4 to 6 is not supported" $objiControlBigIP1 = New-Object iControl.Interfaces $objiControlBigIP1.initialize(IPADDRESS, PORT, USERNAME, PASSWORD) $objiControlBigIP1.NetworkingRouteDomain.Set_Parent_ID(6,501) But when I look at the system audit list it appears to have taken the change, but the change is never made in the console: client Unknown, user admin - transaction 1665299-3 - object 0 - modify { route_domain { route_domain_id 6 route_domain_parent_name "/Common/PublicInternet" } } [Status=Command OK]: If I run the script with any number other than 501 it runs without an error, logs in the audit log with a success, but still does not make the change. Thank you for any information or assistance!
Cannot overwrite existing certificate/key using iControl certificate/key import_from_pem calls
I have been testing importing SSL certificate or key from PEM using iControl calls https://devcentral.f5.com/wiki/iControl.Management__KeyCertificate__certificate_import_from_pem.ashx (and key_import_from_pem). The two calls accept a boolean overwrite parameter which supposedly should allow overwriting of existing certificate/key. But this doesn't seem to work, my test Perl script (the actual API call part) looks like this: $soapResponse = $KeyCert->certificate_import_from_pem ( SOAP::Data->name(mode => MANAGEMENT_MODE_DEFAULT), SOAP::Data->name(cert_ids => [$sCert]), SOAP::Data->name(pem_data => [$CertPEM]), SOAP::Data->name(overwrite => 1) ); I've also tried "true" instead of 1 for the overwrite parameter, neither works, I'm getting errors like this: 01020066:3: The requested Certificate File (/Common/test-cert.crt) already exists in partition Common. My target is an F5 VM running BigIP v12.1.0, the script works fine when the cert/key didn't already exist on the F5.
Why do calls to REST API fail?
We are seeing intermittent failures for calls to the rest api on 11.5.1 HF6. The TCP handshake completes, client sends SSL Client Hello, BigIP sends ACK, and no further packets are seen. Is this related to iControl/REST connection limits? In the icrd log I see: Nov 1 12:45:34 mylb notice icrd_child[31618]: 31618,31633, RestRequestSender, INFO,Connection idle too long fd:9 cached. Nov 1 12:45:42 mylb notice icrd: 8195,13830, RestRequestSender, INFO,Connection idle too long fd:13 cached. Nov 1 12:50:34 mylb notice icrd_child[31655]: 31655,31670, RestRequestSender, INFO,Connection idle too long fd:10 cached. Nov 1 12:50:52 mylb notice icrd: 8195,13830, RestRequestSender, INFO,Connection idle too long fd:13 cached. Nov 1 12:55:33 mylb notice icrd_child[31717]: 31717,31732, RestRequestSender, INFO,Connection idle too long fd:10 cached. Nov 1 12:56:02 mylb notice icrd: 8195,13830, RestRequestSender, INFO,Connection idle too long fd:13 cached. Nov 1 12:56:08 mylb notice icrd: 8195,13817, RestServer, INFO,Connection idle too long fd:11 Nov 3 13:05:33 mylb notice icrd_child[31936]: 31936,31951, RestRequestSender, INFO,Connection idle too long fd:10 cached. Nov 3 13:05:51 mylb notice icrd: 8195,13830, RestRequestSender, INFO,Connection idle too long fd:13 cached. Nov 3 13:06:08 mylb notice icrd: 8195,13817, RestServer, INFO,Connection idle too long fd:11 The restjavad log doesn't register anything during the failure time, and lsof shows the below. [myuser@mylb:Active:Changes Pending] ~ lsof -nPu apache | grep -E "(TCP|COMMAND)" COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME httpd 12579 apache 3u IPv6 1272168705 TCP *:80 (LISTEN) httpd 12579 apache 5u IPv6 1272168710 TCP *:443 (LISTEN) httpd 12579 apache 18u IPv4 1554156729 TCP 127.0.0.1:52743->127.0.0.1:8100 (CLOSE_WAIT) ...several iterations of the above... httpd 16246 apache 3u IPv6 1272168705 TCP *:80 (LISTEN) httpd 16246 apache 5u IPv6 1272168710 TCP *:443 (LISTEN) httpd 16246 apache 16u IPv6 1555163788 TCP 10.1.1.1:443->10.10.10.10:50000 (ESTABLISHED) httpd 16246 apache 18u IPv4 1554419261 TCP 127.0.0.1:43108->127.0.0.1:8100 (CLOSE_WAIT)
Read Access Denied on LocalLBRule::get_metadata
I'm using the iControl Java lib and am getting "Read Access Denied" errors. public class F5Test implements Credentials { private Interfaces f5Interface = new Interfaces("my-lb.example.com", 443, UNAME, PSWD); public static void main(String[] args) { F5Test test = new F5Test(); test.test(); } void test() { try { f5Interface.getManagementPartition().set_active_partition("my-partition"); String[] rules = { "irule_my_rule" }; String[][] md = f5Interface.getLocalLBRule().get_metadata(rules); for(String[] mds : md) { for(String metadata : mds) { System.out.println(metadata); } } } catch (RemoteException e) { e.printStackTrace(); } catch (Exception e) { e.printStackTrace(); } } } When I run this I get: AxisFault faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Server faultSubcode: faultString: Exception caught in LocalLB::urn:iControl:LocalLB/Rule::get_metadata() Exception: Common::OperationFailed primary_error_code : 17238051 (0x01070823) secondary_error_code : 0 error_string : 01070823:3: Read Access Denied: user (zharvey) type (rule metadata) faultActor: faultNode: faultDetail: {http://xml.apache.org/axis/}stackTrace:Exception caught in LocalLB::urn:iControl:LocalLB/Rule::get_metadata() Exception: Common::OperationFailed primary_error_code : 17238051 (0x01070823) secondary_error_code : 0 error_string : 01070823:3: Read Access Denied: user (zharvey) type (rule metadata) at org.apache.axis.message.SOAPFaultBuilder.createFault(SOAPFaultBuilder.java:222) at org.apache.axis.message.SOAPFaultBuilder.endElement(SOAPFaultBuilder.java:129) ...rest of stack trace omitted for brevity I have confirmed that the iRule "irule_my_rule" is defined inside "my-partition" and our IT staff confirms I have full r/w permission for it. Am I using the API incorrectly or is this an F5 config issue?IllegalArgumentException using iControl 11.1 Java
We are using the iControl-11.1.0 Java library to interface with LTM version 11.5.1 Build 4.0.128 Hotfix HF4. Whenever we're trying to call LocalLB.PoolMember.get_all_statistics we are getting the following exception; all other calls seems to be working fine. Any ideas what might be causing this issue? Since iControl-11.1 have the same major branch (11.x) as BigIp 11.5.1 shouldn't these be compatible? ERROR 2015-01-05 06:15:02,797 Call schedulerFactoryBean_Worker-9 Exception: org.xml.sax.SAXException java.lang.IllegalArgumentException at org.apache.axis.encoding.ser.SimpleDeserializer.onEndElement(SimpleDeserializer.java:176) at org.apache.axis.encoding.DeserializerImpl.endElement(DeserializerImpl.java:502) at org.apache.axis.encoding.DeserializationContext.endElement(DeserializationContext.java:1087) at org.apache.axis.message.SAX2EventRecorder.replay(SAX2EventRecorder.java:171) at org.apache.axis.message.MessageElement.publishToHandler(MessageElement.java:1141) at org.apache.axis.message.RPCElement.deserialize(RPCElement.java:236) at org.apache.axis.message.RPCElement.getParams(RPCElement.java:384) at org.apache.axis.client.Call.invoke(Call.java:2467) at org.apache.axis.client.Call.invoke(Call.java:2366) at org.apache.axis.client.Call.invoke(Call.java:1812) at iControl.LocalLBPoolMemberBindingStub.get_all_statistics(LocalLBPoolMemberBindingStub.java:905) ... Caused by: java.lang.IllegalArgumentException at iControl.CommonStatisticType.fromValue(CommonStatisticType.java:1579) at iControl.CommonStatisticType.fromString(CommonStatisticType.java:1584) at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.apache.axis.encoding.ser.EnumDeserializer.makeValue(EnumDeserializer.java:53) at org.apache.axis.encoding.ser.SimpleDeserializer.onEndElement(SimpleDeserializer.java:172) ... 24 moreBIG-IP : iControl : System/ConfigSync::download_configuration() : error opening file for read operations
BIG-IP 11.4.1 Build 608.0 Final VE I'm in .NET 4.5 C : using proxy classes generated from WSDL, I call into iControl API to retrieve BIG-IP system configuration : var systemConfigSync = new SystemConfigSync("bigip", "admin", "admin", 600000); long offset = 0; SystemConfigSyncFileTransferContext context = systemConfigSync.download_configuration("bigip", 131072, ref offset); throws exception : Exception caught in System::urn:iControl:System/ConfigSync::download_configuration() Exception: Common::OperationFailed primary_error_code : 16908289 (0x01020001) secondary_error_code : 0 error_string : Error opening file for read operations On v11.4.0 , this call successfully retrieved the system configuration ( bigip.conf ). However, on upgrading to 11.4.1 it began failing.