Create an iRule that can search and replace
Looking for some irule help please this is a search and replace rule... Step one in the message payload Find <soapenv:Header/> and replace with <soapenv:Header> <job:roleIdentifier>SERVER1</job:roleIdentifier> </soapenv:Header> or if the message is from Server2 Find <soapenv:Header/> and replace with <soapenv:Header> <job:roleIdentifier>SERVER2</job:roleIdentifier> </soapenv:Header>Managing ZoneRunner Resource Records with Bigsuds
Over the last several years, there have been questions internal and external on how to manage ZoneRunner (the GUI tool in F5 DNS that allows you to manage DNS zones and records) resources via the REST interface. But that's a no can do with the iControl REST--it doesn't have that functionality. It was brought to my attention by one of our solutions engineers that a customer is using some methods in the SOAP interface that allows you to do just that...which was news to me! The things you learn... In this article, I'll highlight a few of the methods available to you and work on a sample domain in the python module bigsuds that utilizes the suds SOAP library for communication duties with the BIG-IP iControl SOAP interface. Test Domain & Procedure For demonstration purposes, I'll create a domain in the external view, dctest1.local, with the following attributes that mirrors nearly identically one I created in the GUI: Type: master Zone Name: dctest1.local. Zone File Name: db.external.dctest1.local. Options: allow-update from localhost TTL: 500 SOA: ns1.dctest1.local. Email: hostmaster.ns1.dctest1.local. Serial: 2021092201 Refresh: 10800 Retry: 3600 Expire: 604800 Negative TTL: 60 I'll also add a couple type A records to that domain: name: mail.dctest1.local., address: 10.0.2.25, TTL: 86400 name: www.dctest1.local., address: 10.0.2.80, TTL: 3600 After adding the records, I'll update one of them, changing the IP and the TTL: name: mail.dctest1.local., address: 10.0.2.110, ttl: 900 Then I'll delete the other one: name: www.dctest1.local., address: 10.0.2.80, TTL: 3600 And finally, I'll delete the zone: name: dctest1.local. ZoneRunner Methods All the methods can be found on Clouddocs in the ZoneRunner, Zone, and ResourceRecord method pages. The specific methods we'll use in our highlight real are: Management.ResourceRecord.add_a Management.ResourceRecord.delete_a Management.ResourceRecord.get_rrs Management.ResourceRecord.update_a Management.Zone.add_zone_text Management.Zone.get_zone_v2 Management.Zone.zone_exist With each method, there is a data structure that the interface expects. Each link above provides the details, but let's look at an example with the add_a method. The method requires three parameters, view_zones, a_records, and sync_ptrs, which the image of the table shows below. The boolean is just a True/False value in a list. The reason the list ( [] ) is there for all the attributes is because you can send a single request to update more than one zone, and add more than one record within each zone if desired. The data structure for view_zones and a_records is in the following two images. Now that we have an idea of what the methods require, let's take a look at some code! Methods In Action First, I import bigsuds and initialize the BIG-IP. The arguments are ordered in bigsuds for host, username, and password. If the default “admin/admin” is used, they are assumed, as is shown here. import bigsuds b = bigsuds.BIGIP(hostname='ltm3.test.local') Next, I need to format the ViewZone data in a native python dictionary, and then I check for the existence of that zone. zone_view = {'view_name': 'external', 'zone_name': 'dctest1.local.' } b.Management.Zone.zone_exist([zone_view]) # [0] Note that the return value, which should be a list of booleans, is a list with a 0. I’m guessing that’s either suds or the bigsuds implementation doing that, but it’s important to note if you’re checking for a boolean False. It’s also necessary to set the booleans as 0 or 1 as well when sending requests to BIG-IP with bigsuds. Now I will create the zone since it does not yet exist. From the add_zone_text method description on Clouddocs, note that I need to supply, in separate parameters, the zone info, the appropriate zone records, and the boolean to sync reverse records or not. zone_add_info = {'view_name': 'external', 'zone_name': 'dctest1.local.', 'zone_type': 'MASTER', 'zone_file': 'db.external.dctest1.local.', 'option_seq': ['allow-update { localhost;};']} zone_add_records = 'dctest1.local. 500 IN SOA ns1.dctest1.local. hostmaster.ns1.dctest1.local. 2021092201 10800 3600 604800 60;\n' \ 'dctest1.local. 3600 IN NS ns1.dctest1.local.;\n' \ 'ns1.dctest1.local. 3600 IN A 10.0.2.1;' b.Management.Zone.add_zone_text([zone_add_info], [[zone_add_records]], [0]) b.Management.Zone.zone_exist([zone_view]) # [1] Note that the strings here require a detailed understanding of DNS record formatting, the individual fields are not parameters that can be set like in the ZoneRunner GUI. But, I am confident there is an abundance of modules that manage DNS formatting in the python ecosystem that could simplify the data structuring. After creating the zone, another check to see if the zone exists results in a true condition. Huzzah! Now I’ll check the zone info and the existing records for that zone. zone = b.Management.Zone.get_zone_v2([zone_view]) for k, v in zone[0].items(): print(f'{k}: {v}') # view_name: external # zone_name: dctest1.local. # zone_type: MASTER # zone_file: "db.external.dctest1.local." # option_seq: ['allow-update { localhost;};'] rrs = b.Management.ResourceRecord.get_rrs([zone_view]) for rr in rrs[0]: print(rr) # dctest1.local. 500 IN SOA ns1.dctest1.local. hostmaster.ns1.dctest1.local. 2021092201 10800 3600 604800 60 # dctest1.local. 3600 IN NS ns1.dctest1.local. # ns1.dctest1.local. 3600 IN A 10.0.2.1 Everything checks outs! Next I’ll create the A records for the mail and www services. I’m going to add a filter to only check for the mail/www services for printing to cut down on the lines, but know that they’re still there going forward. a1 = {'domain_name': 'mail.dctest1.local.', 'ip_address': '10.0.2.25', 'ttl': 86400} a2 = {'domain_name': 'www.dctest1.local.', 'ip_address': '10.0.2.80', 'ttl': 3600} b.Management.ResourceRecord.add_a(view_zones=[zone_view], a_records=[[a1, a2]], sync_ptrs=[0]) rrs = b.Management.ResourceRecord.get_rrs([zone_view]) for rr in rrs[0]: if any(item in rr for item in ['mail', 'www']): print(rr) # mail.dctest1.local. 86400 IN A 10.0.2.25 # www.dctest1.local. 3600 IN A 10.0.2.80 Here you can see that I’m adding two records to the zone specified and not creating the reverse records (not included for brevity, but in prod would be likely). Now I’ll update the mail address and TTL. b.Management.ResourceRecord.update_a([zone_view], [[a1]], [[a1_update]], [0]) rrs = b.Management.ResourceRecord.get_rrs([zone_view]) for rr in rrs[0]: if any(item in rr for item in ['mail', 'www']): print(rr) # mail.dctest1.local. 900 IN A 10.0.2.110 # www.dctest1.local. 3600 IN A 10.0.2.80 You can see that the address and TTL updated as expected. Note that with the update_/N/ methods, you need to provide the old and new, not just the new. Let’s get destruction and delete the www record! b.Management.ResourceRecord.delete_a([zone_view], [[a2]], [0]) rrs = b.Management.ResourceRecord.get_rrs([zone_view]) for rr in rrs[0]: if any(item in rr for item in ['mail', 'www']): print(rr) # mail.dctest1.local. 900 IN A 10.0.2.110 And your web service is now unreachable via DNS. Congratulations! But there’s more damage we can do: it’s time to delete the whole zone. b.Management.Zone.delete_zone([zone_view]) b.Management.Zone.zone_exist([zone_view]) # [0] And that’s a wrap! As I said, it’s been years since I have spent time with the iControl SOAP interface. It’s nice to know that even though most of what we do is done through REST, imperatively or declaratively, that some missing functionality in that interface is still alive and kicking via SOAP. H/T to Scott Huddy for the nudge to investigate this. Questions? Drop me a comment below. Happy coding! A gist of these samples is available on GitHub.
Form Based Authentication with external SOAP web services
Problem this snippet solves: 1- You need to authenticate users against an external authentication system relying on SOAP calls. 2- The session identifier must be provided by an external third party system. How to use this snippet: Installation Files You need to upload an html login page using ifiles. You need to upload the SOAP body of the external web service using ifiles. irule You need to install the irule on your Virtual Server you need to protect. Variables set static::holdtime 3600 # session timeout set static::login_url "/login" # login url set static::sideband_vs "VS_EXTERNAL_AUTH_PROVIDER" # Virtual Server that publish the web service Features Version 1.0 Form based login (provided by a custom ifile) Authentication against an external SOAP web service Manage Session timeout Backlog Improve logging Allow 2-factor authentication (Challenge) Encrypt Session cookie Provide internal mecanism to generate a session cookie accept Basic Authentication External links Github : https://github.com/e-XpertSolutions/f5 Code : when RULE_INIT { set static::holdtime 3600 set static::login_url "/login" set static::sideband_vs "VS_EXTERNAL_AUTH_PROVIDER" } when HTTP_REQUEST { if { [HTTP::cookie exists SessionCook] and [table lookup -subtable "active_sessions" [HTTP::cookie SessionCook]] != "" } { return } else { if { [HTTP::path] eq $static::login_url } { if { [HTTP::method] eq "POST" } { if {[HTTP::header "Content-Length"] ne "" && [HTTP::header "Content-Length"] <= 1048576}{ set content_length [HTTP::header "Content-Length"] } else { set content_length 1048576 } if { $content_length > 0} { HTTP::collect $content_length } } else { HTTP::respond 200 content [ifile get login.html] "Cache-Control" "no-cache, must-revalidate" "Content-Type" "text/html" } } else { HTTP::respond 302 noserver "Location" $static::login_url "Cache-Control" "no-cache, must-revalidate" Set-Cookie "SessionCook=$result;domain=[HTTP::host];path=/" } } } when HTTP_REQUEST_DATA { set payload [HTTP::payload] set username "" set password "" regexp {Login1\%3AtxtUserName\=(.*)\&Login1\%3AtxtPassword\=(.*)\&Login1\%3AbtnSubmit\=(.*)} $payload -> username password garbage if {[catch {connect -timeout 1000 -idle 30 -status conn_status $static::sideband_vs} conn_id] == 0 && $conn_id ne ""}{ log local0. "Connect returns: $conn_id and conn status: $conn_status" } else { log local0. "Connection could not be established to sideband_virtual_server" } set content [subst -nocommands -nobackslashes [ifile get soap_body]] set length [string length $content] set data "POST /apppath/webservicename.asmx HTTP/1.1\r\nHost: www.hostname.com\r\nContent-Type: text/xml; charset=utf-8\r\nContent-Length: $length\r\nSOAPAction: http://schemas.microsoft.com/sqlserver/2004/SOAP\r\n\r\n$content" set send_bytes [send -timeout 1000 -status send_status $conn_id $data] set recv_data [recv -timeout 1000 $conn_id] # parse response to retrieve the authentication result, it gives 0 if authentication failed or a session_id if it succeed regexp { (.*) (.*)} $recv_data -> result garbage unset content unset length unset data unset recv_data close $conn_id # add a custom alert notification to the login page if { $result == 0 } { set alert " Invalid credentials. " HTTP::respond 200 content [subst -nocommands -nobackslashes [ifile get login.html]] "Cache-Control" "no-cache, must-revalidate" "Content-Type" "text/html" Set-Cookie "SessionCook=deleted;expires=Thu, 01-Jan-1970 00:00:10 GMT;domain=[HTTP::host];path=/" } else { HTTP::respond 302 noserver "Location" "/" "Cache-Control" "no-cache, must-revalidate" Set-Cookie "SessionCook=$result;domain=[HTTP::host];path=/" # save the cookie value in a cache for fast checking table add -subtable "active_sessions" $result $username indef $static::holdtime } } Tested this on version: 11.5
XML : Modifiy SOAP header
Hello, I need a to modify the soap header in order to add a specific parameter. After the tag " " i should be adding the parameter " soap:mustUnderstand="1" " but the data contained in the tag should not be modified. The soap envelope is like this : [DATA] Then soap envelope should like this : [DATA] I know that i need to use the function stream in order to make it work but i need guidance on this one. What should i put in the iRule in order to just add this parameter ? Thank you for your help
Modify SOAP To header based on target node
Hi, I have 2 kinds of virtual servers set up. The first has no pools assigned. It takes requests over SSL, terminates the SSL and routes the requests (in plaintext) to wholly different pools based on content. However, the SOAP To header from these requests needs to be modified. The protocol needs to be changed from https to http, the service URL is suffixed with a designator used for services without transport security and the port needs to be changed from 8443 to some other port. It's the "some other port" that is causing me some issues. Following is my iRule on the SSL-terminating, content-based routing virtual server: when HTTP_REQUEST { if { [HTTP::method] eq "POST" } { if { [HTTP::header exists "Content-Length"] } { set content_length [HTTP::header "Content-Length"] } else { set content_length 1048576 } HTTP::collect $content_length } else { HTTP::respond 405 content "Unsupported" Allow "POST" } } when HTTP_REQUEST_DATA { if { [HTTP::method] eq "POST" } { set payload [HTTP::payload] binary scan [sha1 [SSL::cert 0]] H* certHash set newSoapHeader "" if { [info exists certHash] } { append newSoapHeader $certHash } else { append newSoapHeader "cert_hash_retrieval_failure" } append newSoapHeader "" Try to stick the new header in the header collection set numMatches [regsub {.*\<[A-Za-z0-9:]*Header[^\>]*\>} $payload [concat {&} $newSoapHeader] modifiedPayload] if no matches from that, we know there was no header and, therefore, no subsitution. We need to introduce the whole Header block if { $numMatches == 0 } { set contentToInsert "" append contentToInsert $newSoapHeader append contentToInsert "" set numMatches [regsub {.*\<[A-Za-z0-9:]*Envelope[^\>]*\>} $payload [concat {&} $contentToInsert] modifiedPayload] } Empty it of content first HTTP::payload replace 0 [string length [HTTP::payload]] "" Then replace the empty payload HTTP::payload replace 0 0 $modifiedPayload set currentVirtualServer [virtual name] if { [string match *LEGACY* $currentVirtualServer] > 0 } { Replace the HTTP Location Header HTTP::header replace Location [string map { "https://" "http://" ".svc" "_U.svc" } [ HTTP::header Location]] Replace the SOAP To Header set badToHeaderPattern ".svc" set lenBadHeaderPattern [string length $badToHeaderPattern] set betterToHeaderPattern "_U.svc" set offset [string first $badToHeaderPattern [HTTP::payload]] if { $offset >= 0 } { HTTP::payload replace $offset $lenBadHeaderPattern $betterToHeaderPattern set badToHeaderPattern "https://" set lenBadHeaderPattern [string length $badToHeaderPattern] set betterToHeaderPattern "http://" set offset [string first $badToHeaderPattern [HTTP::payload]] if { $offset >= 0 } { HTTP::payload replace $offset $lenBadHeaderPattern $betterToHeaderPattern } NOW REPLACE THE 8443 PORT WITH... um... } } the trailing arguments align with the parts of the regex in parentheses. First is always the whole match, subsequent per parentheses pair regexp {(\<[A-Za-z0-9:]*Organisation[^\>]*\>)([A-Za-z]*)} $modifiedPayload wholeMatch xmlElementMatch organisationNameMatch if { [string length $organisationNameMatch] > 0 } { string map takes a list of replacement pairs (e.g [list needle1 replace1 needle2 replace2... needleN replaceN]) set targetPool [string map [list -D--- --- MULTI $organisationNameMatch RELEASE RELEASE-U DEBUG DEBUG-U LEGACY ""] $currentVirtualServer] pool $targetPool } else { HTTP::respond 400 content "Unknown" } } Ensure that the request is released back to F5 so it may take control of the underlying connection and complete the routing HTTP::release } I'll detail the HTTP_REQUEST if requested, but it's pretty much stock-standard stuff. All the magic is in the HTTP_REQUEST_DATA handling. I know a lot of my SOAP content replacement is really risky and needs better pattern matching/controls so I don't replace anything I shouldn't, so don't focus on that. But after I switch out the https with http in the SOAP To header (really every instance of https in the whole payload), I want to change the port 8443 to another port. The other port, however, is not known until I route it to the pool later (the "pool $targetPool" in the last couple of lines). That routing works just fine and the correct pool gets the requests and processes them. But I can't seem to manipulate the payload any further after I set the new pool. That means I can't, say, get "[LB::server port]" and replace the mentions of 8443 with it. It looks like I don't have it available to manipulate on the LB_SELECTED event either. So, how do I modify the port in the SOAP To Header (also need to modify the HTTP Location header) with a port I will only know after I've re-routed the request with the "pool" keyword?
Use APM HTTP Auth to send a SOAP Message for OTP
Hello, I recently read this article on implementing a OTP solution in our APM via SMS: https://devcentral.f5.com/articles/one-time-passwords-via-an-sms-gateway-with-big-ip-access-policy-manager In this article the OTP is sent to the client via an HTTP API, of the SMS provider, which used an HTTP Auth Server to communicate with the SMS provider. I would like to implement this same solution using a SOAP API, but I'm not sure what to populate the fields of the AAA HTTP Server with. I have successfully implemented this using an e-mail irule to the SMS provider and it works well, but we would like to encrypt the message via HTTPS and the provider only supports SOAP. I was hoping that I could just paste the following output (from soapclient.com) into the "Hidden Form Parameters/Values", but it doesn't seem to be that simple. Any suggestions or other F5 doco I could reference? 1234567890987654321Test MessageMessage The WSDL of the SMS Toolkit methods is located at: http://xml.redcoal.com/soapserver.dll/wsdl/ISoapServer Thanks for your help, -MikeSOAP HTTP Monitor - HTTP Error 400. The request has an invalid header name.
Hello All I am new to SOAP testing, I am using an http monitor to do a SOAP test but I keep receiving error 400 from the server. I am using a 3rd party SOAP client which gets a successful response, and it's using the same statement. I believe the problem is how I am constructing the request. I have read http://support.f5.com/kb/en-us/solutions/public/2000/100/sol2167.html but still don't understand why this is failing. This is what I am using on the 'Send String' portion of the test: POST /CurrencyConvertor.asmx HTTP/1.1\r\nAccept-Encoding: gzip,deflate\r\nContent-Type: text/xml;charset=UTF-8\r\nSOAPAction: \"http://www.webserviceX.NET/ConversionRate\"\r\nContent-Length: 345\r\nHost: www.webservicex.com\r\nConnection: Keep-Alive\r\nUser-Agent: Apache-HttpClient/4.1.1 (java 1.5)\r\n\r\r\n\r\n\r\n\r\nEUR\r\nAFA\r\n\r\n\r The Wireshark capture from my tests indicates: POST /CurrencyConvertor.asmx HTTP/1.1 Accept-Encoding: gzip,deflate Content-Type: text/xml;charset=UTF-8 SOAPAction: "http://www.webserviceX.NET/ConversionRate" Content-Length: 345 Host: www.webservicex.com Connection: Keep-Alive EUR AFA HTTP/1.1 400 Bad Request Content-Type: text/html; charset=us-ascii Server: Microsoft-HTTPAPI/2.0 Date: Fri, 04 Oct 2013 23:04:50 GMT Connection: close Content-Length: 339 Bad Request Bad Request - Invalid Header HTTP Error 400. The request has an invalid header name. The Wireshark from the successful connection using 3rd party shows: POST /CurrencyConvertor.asmx HTTP/1.1 Accept-Encoding: gzip,deflate Content-Type: text/xml;charset=UTF-8 SOAPAction: "http://www.webserviceX.NET/ConversionRate" Content-Length: 345 Host: www.webservicex.com Connection: Keep-Alive User-Agent: Apache-HttpClient/4.1.1 (java 1.5) EUR AFA HTTP/1.1 200 OK Cache-Control: private, max-age=0 Content-Type: text/xml; charset=utf-8 Content-Encoding: gzip Vary: Accept-Encoding Server: Microsoft-IIS/7.0 X-AspNet-Version: 4.0.30319 X-Powered-By: ASP.NET Date: Fri, 04 Oct 2013 22:24:10 GMT Content-Length: 311 0 I'd really appreciate any advice, many thanks…. G FYI I am using BIP-IP VE version 10.1.0
Event log soap[22458]
Hello, I try to understand a log message on our F5 Big IP 13.1.1.4. Under System -> Logs -> Local Traffic, I have several entries like LogLevel:info Service:soap[22458] Event:src=127.0.0.1, user= I precise there is nothing after user :) Anyone can explain me what it means and if it is possible to filter these entries? Best regards.
How to find the iControl jar version & i want to upgrade it to latest version?
I've been using iControl jar in my appliance to connect F5 devices using SOAP service, Due to security purposes i need to upgrade to latest jar. I didn't find any version in the contents of the jar. Please help me to find out the current jar version & point me to find the latest jar location like url or gradle path.
