BIGREST - A Python SDK for F5 iControl REST API
This article is written by, and published on behalf of, DevCentral MVP Leonardo Souza. --- Hello all, this is going to be my shortest article so far. As you probably know already both BIG-IP and BIG-IQ have an iControl REST API. However, if you play with that very often, you will find yourself creating some scripts to perform some common tasks. If you put those scripts together, you kind of have an SDK that other people can use to simplify the use of the API. Almost all vendors these days have an SDK for their products, and the language of choice is mainly Python because of the language simplicity. As the article title says, I wrote BIGREST that is a Python SDK to work with iControl REST API. The SDK fully supports both BIG-IP and BIG-IQ. I wanted to advance my Python andiControl REST knowledge, so this was a useful way of doing that. You may be wondering "Isn't there already a Python SDK foriControl REST?", so let me explain that part. I have used the existing SDK many times in the past, and it was very helpful. The existing Python SDK, the F5-SDK (https://github.com/F5Networks/f5-common-python) is limited, as it mainly supports BIG-IP, and the only supported BIG-IQ functionality is license pools. I wanted to help with the F5-SDK and extend it for BIG-IQ so I looked into the code but I decided the changes I wanted to make made more sense to start from scratch. Some details about these differences are here HTTP paths HTTP paths can be seen as just a tmsh command. In the following examples, HTTP path is “/mgmt/tm/ltm/pool”. F5-SDK mgmt.tm.ltm.pools.pool.create(name='mypool', partition='Common') Python code for every HTTP path; requires more code. BIGREST device.create("/mgmt/tm/ltm/pool", {"name": “mypool”, “partition”: “Common”}) The user tells the HTTP path they want to use. less code to write and support all current HTTP paths and new HTTP paths are automatically supported. BIG-IQ and Python Support F5-SDK created to support BIG-IP REST API supports Python 2 and Python 3 Python 2 was discontinued in 2020 BIGREST created to support BIG-IP and BIG-IQ. supports only Python 3 The code can use new Python 3 functionalities to make it simpler to write and read. Method Names F5-SDK uses some names of the REST API like collection. Example: mgmt.tm.ltm.pools.get_collection() BIGREST tries to use only tmsh names. Example: device.load("/mgmt/tm/ltm/pool") In this case, you load the objects to memory, and if you want you save them after. Similar to load the configuration from the disk using tmsh, and saving it to the disk after. I wrote a very extensive documentation explaining how the SDK works, so you will find all the details there. For more information, including the link for the code and documentation, go to the code share: https://devcentral.f5.com/s/articles/BIGREST
Displaying VS config using SDK
Problem this snippet solves: I have made a script in python using the SDK of F5 to display a brief information of the configuration of each existing VS. You can use this code as proof of concept of how to use F5 SDK. How to use this snippet: You need to replace this next line with the corresponding information before executing the script. session = ManagementRoot("F5_mgmt_IP","username","password",token=True) Code : import re from f5.bigip import ManagementRoot # ---------------------------------------------------------- session = ManagementRoot("F5_mgmt_IP","username","password",token=True) # CAPTURE LIST OF CLIENT SSL PROFILES client_ssls = session.tm.ltm.profile.client_ssls.get_collection() # https:// /mgmt/tm/ltm/profile/client-ssl listClientSsl = [] for client_ssl in client_ssls: listClientSsl.append(client_ssl.name) # CAPTURE LIST OF SERVER SSL PROFILES server_ssls = session.tm.ltm.profile.server_ssls.get_collection() # https:// /mgmt/tm/ltm/profile/server-ssl listServerSsl = [] for server_ssl in server_ssls: listServerSsl.append(server_ssl.name) # CAPTURE VIRTUAL SERVER INFORMATION virtuals = session.tm.ltm.virtuals.get_collection() # https:// /mgmt/tm/ltm/virtual for virtual in virtuals: print("------------") print("Partition: {}".format(virtual.partition)) if hasattr(virtual, 'subPath'): print("SubPath: {}".format(virtual.subPath)) else: print("SubPath: None") print("Virtual: {}".format(virtual.name)) if hasattr(virtual, 'description'): print("Description: {}".format(virtual.description)) else: print("Description: None") print("Destination: {}".format(re.search('[^\/]+$', virtual.destination).group(0))) listClientSsl_inUse = [] listServerSsl_inUse = [] for profile in virtual.profiles_s.get_collection(): # https:// /mgmt/tm/ltm/virtual/ /profiles if profile.name in listClientSsl: listClientSsl_inUse.append(profile.name) if profile.name in listServerSsl: listServerSsl_inUse.append(profile.name) if listClientSsl_inUse: for prof in listClientSsl_inUse: print("Client SSL: {}".format(prof)) else: print("Client SSL: None") if listServerSsl_inUse: for prof in listServerSsl_inUse: print("Server SSL: {}".format(prof)) else: print("Server SSL: None") if hasattr(virtual, 'rules'): for rule in virtual.rules: print("Rule: {}".format(re.search('[^\/]+$', rule).group(0))) else: print("Rule: None") if hasattr(virtual, 'persist'): for persist in virtual.persist: print("Persistence: {}".format(persist['name'])) else: print("Persistence: None") if hasattr(virtual, 'pool'): print("Pool: {}".format(re.search('[^\/]+$', virtual.pool).group(0))) if hasattr(virtual, 'subPath'): poolName = virtual.pool.split("/")[3] poolSubpath = virtual.pool.split("/")[2] poolPartition = virtual.pool.split("/")[1] pool = session.tm.ltm.pools.pool.load(name=poolName, subPath=poolSubpath, partition=poolPartition) # https:// /mgmt/tm/ltm/pool/ else: poolName = virtual.pool.split("/")[2] poolPartition = virtual.pool.split("/")[1] pool = session.tm.ltm.pools.pool.load(name=poolName, partition=poolPartition) # https:// /mgmt/tm/ltm/pool/ poolMembers = pool.members_s.get_collection() # https:// /mgmt/tm/ltm/pool/ /members if poolMembers: for member in poolMembers: print("Member: {}".format(member.name)) else: print("Member: None") else: print("Pool: None") print("Member: None") print("------------") # ---------------------------------------------------------- Tested this on version: 12.1
F5 and Promon Have Partnered to Protect Native Mobile Applications from Automated Bots - Easily
This DevCentral article provides details on how F5 Bot Defense is used today to protect against mobile app automated bot traffic from wreaking havoc with origin server infrastructure and producing excessive nuisance load volumes, and in turn how the Promon Mobile SDK Integrator is leveraged to speed the solution's deployment. The integrator tool contributes to the solution by allowing the F5 anti-bot solution to be inserted into customer native mobile apps, for both Apple and Android devices, with a simple and quick “No Code” approach that can be completed in a couple of minutes.No source code of the native mobile app need be touched, only the final compiled binaries are required to add the anti bot security provisions as a quick, final step when publishing apps. Retrieving Rich, Actionable Telemetry in a Browser World In the realm of Internet browser traffic, whether the source be Chrome, Edge, Safari or any other standards-based web browser, the key to populating the F5 anti-bot analytics platform with actionable data is providing browsers clear instructions of when, specifically, to add telemetry to transactions as well as what telemetry is required.This leads to the determination, in real time, of whether this is truly human activity or instead automated, negative hostile traffic. The “when” normally revolves around the high value server-side endpoint URLs, things like “login” pages, “create account” links, “reset password” or “forgot username” pages, all acting like honeypot pages where attackers will gravitate to and direct brute force bot-based attacks to penetrate deep into the service. All the while, each bot will be cloaked to the best of their abilities as a seemingly legitimate human user.Other high-value transaction endpoints would likely be URLs corresponding to adding items to a virtual shopping cart and checkout pages for commercial sites. Telemetry, the “what” in the above discussion, is to be provided for F5 analytics and can range from dozens to over one hundred elements.One point of concern is inconsistencies in the User Agent field produced in the HTTP traffic, where a bot may indicate a Windows 10 machine using Chrome via the User Agent header, but various elements of telemetry retrieved indicate a Linux client using Firefox.This is eyebrow raising, if the purported user is misleading the web site with regards to the User Agent, what is the traffic creator’s endgame? Beyond analysis of configuration inconsistencies, behavioral items are noted such as unrealistic mouse motions at improbable speeds and uncanny precision suggesting automation, or perhaps the frequent use of paste functions to fill in form fields such as username, where values are more likely to be typed or be auto filled by browser cache features. Consider something as simple as the speed of key depressed and the key being released upward, if signals indicate an input typing pace that defies the physics of a keyboard, just milliseconds for a keystroke, this is another strong warning sign of automation. Telemetry in the Browser World Telemetry can be thought of as dozens and dozens of intelligence signals building a picture of the legitimacy of the traffic source. The F5 Bot Defense request for telemetry and subsequently provided values are fully obfuscated from bad actors and thus impossible to manipulate in any way. The data points lead to a real time determination of the key aspect of this user:is this really a human or is this automation? This set of instructions, directing browsers with regards to what transactions to decorate with requested signals, is achieved by forcing browsers to execute specific JavaScript inserted into the pages that browsers load when interacting with application servers.The JavaScript is easily introduced by adding JavaScript tags into HTML pages, specifically key returned transactions with “Content Type=text/HTML” that subsequently lead to high value user-side actions like submission of user credentials (login forms). The addition of the JavaScript tags is frequently inserted in-line to returned traffic through a F5 BIG-IP application delivery controller, the F5 Shape proxy itself, or a third-party tag manager, such as Google Tag Manager.The net result is a browser will act upon JavaScript tags to immediately download the F5 JavaScript itself.The script will result in browsers providing the prescribed rich and detailed set of telemetry included in those important HTTP transactions that involve high value website pages conducting sensitive actions, such as password resetting through form submissions. Pivoting to Identify Automation in the Surging Mobile Application Space With native mobile apps, a key aspect to note is the client is not utilizing a browser but rather what one can think of a “thick” app, to borrow a term from the computer world.Without a browser in play, it is no longer possible to simply rely upon actionable JavaScript tags that could otherwise be inserted in flight, from servers towards clients, through application delivery controllers or tag managers.Rather, with mobile apps, one must adjust the app itself, prior to posting to an app store or download site, to offer instructions on where to retrieve a dynamic configuration file, analogous to the instructions provided by JavaScript tag insertion in the world of browsers. The retrieved configuration instruction file will guide the mobile app in terms of which transactions will require “decorating” with telemetry, and of course what telemetry is needed.The telemetry will often vary from one platform such as Android to a differing platform such as Apple iOS.Should more endpoints within the server-side application need decorating, meaning target URLs, one simply adjusts the network hosted configuration instruction file. This adjustment in operations of a vendor’s native mobile application behavior is achieved through F5 Bot Defense Mobile SDK (Software Development Kit) and representative provided telemetry signals might include items like device battery life remaining, device screen brightness and resolution and indicators of device setup, such as a rooted device or an emulator posing as a device.Incorrectly emulated devices, such as one displaying mutually exclusive iOS and Android telemetry, concurrently, allows F5 to isolate troublesome, unwanted automated traffic from the human traffic required for ecommerce to succeed efficiently and unfettered. The following four-step diagram depicts the process of a mobile app being protected by F5 bot defense, from step one with the retrieval of instructions that include what transactions to decorate with telemetry data, through to step four where the host application (the mobile app) is made aware if the transaction was mitigated (blocked or flagged) by means of headers introduced by the F5 Antibot solution. The net result of equipping mobile apps with the F5 Bot Defense Mobile SDK, whether iOS or Android, is the ability to automatically act upon automated bot traffic observed, without a heavy administrative burden. A step which is noteworthy and unique to the F5 mobile solution is the final, fourth step, whereby a feedback mechanism is implemented in the form of the Parse Response header value.This notifies the mobile app that the transaction was mitigated (blocked) en route.One possible reason this can happen is the app was using a dated configuration file, and a sensitive endpoint URL had been recently adjusted to require telemetry.The result of the response in step 4 is the latest version of the config file, with up-to-date telemetry requirements, will automatically be re-read by the mobile app and the transaction can now take place successfully with proper decorated telemetry included. Promon SDK Integrator and F5 Bot Defense: Effortlessly Secure Your Mobile Apps Today One approach to infusing a native mobile app with the F5 Bot Defense SDK would be a programmatic strategy, whereby the source code of the mobile app would require modifications to incorporate the F5 additional code.Although possible, this may not be aligned with the skillsets of all technical resources requiring frequent application builds, for instance for quality assurance (QA) testing. Another issue might be a preference to only have obfuscated code at the point in the publishing workflow where the security offering of the SDK is introduced.In simple terms, the core mandate of native mobile app developers is the intellectual property contained within that app, adding valuable security features to combat automation is more congruent with a final checkmark obtained by protecting the completed application binary with a comprehensive protective layer of security. To simplify and speed up the time for ingestion of the SDK, F5 has partnered with Promon, of Oslo, Norway to make use of an integrator application from Promon which can achieve a “No Code” integration in just a few commands.The integrator, technically a .jar executable known as the Shielder tool at the file level, is utilized as per the following logic. The workflow to create the enhanced mobile app, using the Promon integration tool, with the resultant modified mobile app containing the F5 (Shape) SDK functions within it, consists of only two steps. 1.Create a Promon SDK Integrator configuration file 2.Perform the SDK injection An iOS example would take the follow form: Step 1: python3 create_config.py --target-os iOS --apiguard-config ./base_ios_config.json --url-filter *.domain.com --enable-logs --outfile sdk_integrator_ios_plugin_config.dat Step 2: java -jar Shielder.jar --plugin F5ShapeSDK-iOS-Shielder-plugin-1.0.4.dat --plugin sdk_integrator_ios_plugin_config.dat ./input_app.ipa --out . /output_app.ipa --no-sign Similarly, an Android example would remain also a simple two-step process looking much like this: Step 1: python3 create_config.py --target-os Android --apiguard-config ./base_android_config.json --url-filter *.domain.com --enable-logs --outfile sdk_integrator_android_plugin_config.dat Step 2: java -jar Shielder.jar --plugin F5ShapeSDK-Android-Shielder-plugin-1.0.3.dat --plugin sdk_integrator_android_plugin_config.dat ./input_app.apk --output ./output_app.apk In each respective “Step 1”, the following comments can be made about the create_config.py arguments: target-os specifies the platform (Apple or Android). apiguard-config will provide a base configuration .json file, which will server to provide an initial list of protected endpoints (corresponding to key mobile app exposed points such as “create account” or “reset password”), along with default telemetry required per endpoint.Once running, the mobile app equipped with the mobile SDK will immediately refresh itself with a current hosted config file. url-filter is a simple means of directing the SDK functions to only operate with specific domain name space (eg *.sampledomain.com).Url filtering is also available within the perpetually refreshed .json config file itself. enable-logs allows for debugging logs to be optionally turned on. outfile specifies for file naming of the resultant configuration.dat file, which is ingested in step 2, where the updated iOS or Android binary, with F5 anti-bot protections, will be created. For Step 2, where the updated binaries are created, for either Apple iOS or Android platforms, these notes regard each argument called upon by the java command: shielder.jar is the portion of the solution from Promon which will adjust the original mobile application binary to include F5 anti-bot mobile security, all without opening the application code. F5ShapeSDK-[Android or iOS]-Shielder-plugin-1.0.3.dat is the F5 antibot mobile SDK, provide in a format consumable by the Promon shielder. The remaining three arguments are simply the configuration output file of step 1, the original native mobile app binary and finally the new native mobile app binary which will now have anti-bot functions. The only additional step that is optionally run would be re-signing the new version of the mobile application, as the hash value will change with the addition of the security additions to the new output file. Contrasting the Promon SDK Integrator with Manual Integration and Mobile Application Requirements The advantage of the Promon Integrator approach is the speed of integration and the lack of any coding requirements to adjust the native mobile application.The manual approach to integrating F5 Bot Defense Mobile SDK is documented by F5 with separate detailed guides available for Android and for iOS.A representative summary of the steps involved include the following check points along the path to successful manual SDK integration: oImporting the provided APIGuard library into Android Studio (Android) and Xcode (iOS) environments oThe steps for iOS will differ depending on whether the original mobile app is using a dynamic or static framework, commands for both Swift and Objective-c are provided in the documentation oThe F5 SDK code is already obfuscated and compressed; care should be followed not to include this portion of the revised application in existing obfuscation procedures oWithin Android Studio, expand and adjust as per the documentation the Gradle scripts oInitialize the Mobile SDK; specific to Android and the Application Class the detailed functions utilized are available with the documentation, including finished initialization examples in Java and Kotlin oSpecific to iOS, initialize the mobile SDK in AppDelegate, this includes adding APIGuardDelegate to the AppDelegate class as an additional protocol, full examples are provided for Swift and Obective-c oBoth Android and iOS will require a GetHeaders functions be invoked through code additions for all traffic potentially to be decorated with telemetry, in accordance with the instructions of the base and downloaded configuration files As demonstrated the by the list length of these high-level and manual steps above, which involve touching application code, the alternative ease and simplicity of the Promon Integrator two command offering may be significant in many cases. The platform requirements for the F5 Bot Defense and Promon paired solution are not arduous and frequently reflect libraries used in many native mobile applications developed today.These supported libraries include: oAndroid: HttpURLConnection, OkHttp, Retrofit oiOS: NSURLSession, URLSession, Alamofire Finally, mobile applications that utilize WebViews, which is to say applications using browser-type technologies to implement the app, such as support for Cascading Style Sheets or JavaScript, are not applicable to the F5 Bot Defense SDK approach.In some cases, entirely WebView implemented applications may be candidates for support through the browser style JavaScript-oriented F5 Bot Defense telemetry gathering. Summary With the simplicity of the F5 and Promon workflow, this streamlined approach to integrating the anti-bot technology into a mobile app ecosystem allows for rapid, iterative usage. In development environments following modern CI/CD (continuous integration/continuous deployment) paradigms, with build servers creating frequently updated variants of native mobile apps, one could invoke the two steps of the Promon SDK integrator daily, there are no volume-based consumption constraints.
How to install a cert and a key through Python SDK
I am trying to install a cert and a key through Python SDK. BIG-IP Version: 12.1.2 First.. I had no problem with uploading certs and keys to F5's /var/config/rest/downloads/ folder. Now when I tried the following python code to install cert and key, I didn't see any error. key = bitip.tm.sys.file.ssl_keys.ssl_key.create(name='test', partition='Common', sourcePath='file:/var/config/rest/downloads/test.key') cert = bitip.tm.sys.file.ssl_certs.ssl_cert.create(name='test', partition='Common', sourcePath='file:/var/config/rest/downloads/test.crt') However when I logged on BIG-IP device and clicked a cert or key to see their actual content, I couldn't see any content. "No certificate" under certificate tab and an error message of "An error has occurred while trying to process your request." under Key tab. Does any one know what is causing this problem? Or is there any way I can troubleshoot this issue? Thank you.
