F5 iApp Automated Backup
Problem this snippet solves: This is now available on GitHub! Please look on GitHub for the latest version, and submit any bugs or questions as an "Issue" on GitHub: (Note: DevCentral admin update - Daniel's project appears abandoned so it's been forked and updated to the link below. @damnski on github added some SFTP code that has been merged in as well.) https://github.com/f5devcentral/f5-automated-backup-iapp Intro Building on the significant work of Thomas Schockaert (and several other DevCentralites) I enhanced many aspects I needed for my own purposes, updated many things I noticed requested on the forums, and added additional documentation and clarification. As you may see in several of my comments on the original posts, I iterated through several 2.2.x versions and am now releasing v3.0.0. Below is the breakdown! Also, I have done quite a bit of testing (mostly on v13.1.0.1 lately) and I doubt I've caught everything, especially with all of the changes. Please post any questions or issues in the comments. Cheers! Daniel Tavernier (tabernarious) Related posts: Git Repository for f5-automated-backup-iapp (https://github.com/tabernarious/f5-automated-backup-iapp) https://community.f5.com/t5/technical-articles/f5-automated-backups-the-right-way/ta-p/288454 https://community.f5.com/t5/crowdsrc/complete-f5-automated-backup-solution/ta-p/288701 https://community.f5.com/t5/crowdsrc/complete-f5-automated-backup-solution-2/ta-p/274252 https://community.f5.com/t5/technical-forum/automated-backup-solution/m-p/24551 https://community.f5.com/t5/crowdsrc/tkb-p/CrowdSRC v3.2.1 (20201210) Merged v3.1.11 and v3.2.0 for explicit SFTP support (separate from SCP). Tweaked the SCP and SFTP upload directory handling; detailed instructions are in the iApp. Tested on 13.1.3.4 and 14.1.3 v3.1.11 (20201210) Better handling of UCS passphrases, and notes about characters to avoid. I successfully tested this exact passphrase in the 13.1.3.4 CLI (surrounded with single quote) and GUI (as-is): `~!@#$%^*()aB1-_=+[{]}:./? I successfully tested this exact passphrase in 14.1.3 (square-braces and curly-braces would not work): `~!@#$%^*()aB1-_=+:./? Though there may be situations these could work, avoid these characters (separated by spaces): " ' & | ; < > \ [ ] { } , Moved changelog and notes from the template to CHANGELOG.md and README.md. Replaced all tabs (\t) with four spaces. v3.1.10 (20201209) Added SMB Version and SMB Security options to support v14+ and newer versions of Microsoft Windows and Windows Server. Tested SMB/CIFS on 13.1.3.4 and 14.1.3 against Windows Server 2019 using "2.0" and "ntlmsspi" v3.1.0: Removed "app-service none" from iCall objects. The iCall objects are now created as part of the Application Service (iApp) and are properly cleaned up if the iApp is redeployed or deleted. Reasonably tested on 11.5.4 HF2 (SMB worked fine using "mount -t cifs") and altered requires-bigip-version-min to match. Fixing error regarding "script did not successfully complete: (can't read "::destination_parameters__protocol_enable": no such variable" by encompassing most of the "implementation" in a block that first checks $::backup_schedule__frequency_select for "Disable". Added default value to "filename format". Changed UCS default value for $backup_file_name_extension to ".ucs" and added $fname_noext. Removed old SFTP sections and references (now handled through SCP/SFTP). Adjusted logging: added "sleep 1" to ensure proper logging; added $backup_directory to log message. Adjusted some help messages. New v3.0.0 features: Supports multiple instances! (Deploy multiple copies of the iApp to save backups to different places or perhaps to keep daily backups locally and send weekly backups to a network drive.) Fully ConfigSync compatible! (Encrypted values now in $script instead of local file.) Long passwords supported! (Using "-A" with openssl which reads/writes base64 encoded strings as a single line.) Added $script error checking for all remote backup types! (Using 'catch' to prevent tcl errors when $script aborts.) Backup files are cleaned up after any $script errors due to new error checking. Added logging! (Run logs sent to '/var/log/ltm' via logger command which is compatible with BIG-IP Remote Logging configuration (syslog). Run logs AND errors sent to '/var/tmp/scriptd.out'. Errors may include plain-text passwords which should not be in /var/log/ltm or syslog.) Added custom cipher option for SCP! (In case BIG-IP and the destination server are not cipher-compatible out of the box.) Added StrictHostKeyChecking=no option. (This is insecure and should only be used for testing--lots of warnings.) Combined SCP and SFTP because they are both using SCP to perform the remote copy. (Easier to maintain!) Original v1.x.x and v2.x.x features kept (copied from an original post): It allows you to choose between both UCS or SCF as backup-types. (whilst providing ample warnings about SCF not being a very good restore-option due to the incompleteness in some cases) It allows you to provide a passphrase for the UCS archives (the standard GUI also does this, so the iApp should too) It allows you to not include the private keys (same thing: standard GUI does it, so the iApp does it too) It allows you to set a Backup Schedule for every X minutes/hours/days/weeks/months or a custom selection of days in the week It allows you to set the exact time, minute of the hour, day of the week or day of the month when the backup should be performed (depending on the usefulness with regards to the schedule type) It allows you to transfer the backup files to external devices using 4 different protocols, next to providing local storage on the device itself SCP (username/private key without password) SFTP (username/private key without password) FTP (username/password) SMB (now using TMOS v12.x.x compatible 'mount -t cifs', with username/password) Local Storage (/var/local/ucs or /var/local/scf) It stores all passwords and private keys in a secure fashion: encrypted by the master key of the unit (f5mku), rendering it safe to store the backups, including the credentials off-box It has a configurable automatic pruning function for the Local Storage option, so the disk doesn't fill up (i.e. keep last X backup files) It allows you to configure the filename using the date/time wildcards from the tcl [clock] command, as well as providing a variable to include the hostname It requires only the WebGUI to establish the configuration you desire It allows you to disable the processes for automated backup, without you having to remove the Application Service or losing any previously entered settings For the external shellscripts it automatically generates, the credentials are stored in encrypted form (using the master key) It allows you to no longer be required to make modifications on the linux command line to get your automated backups running after an RMA or restore operation It cleans up after itself, which means there are no extraneous shellscripts or status files lingering around after the scripts execute How to use this snippet: Find and download the latest iApp template on GitHub (e.g "f5.automated_backup.v3.2.1.tmpl.tcl"). Import the text file as an iApp Template in the BIG-IP GUI. Create an Application Service using the imported Template. Answer the questions (paying close attention to the help sections). Check /var/tmp/scriptd.out for general logs and errors. Tested this on version: 16.0
Understanding iApps
Understanding iApps iApps are powerful tools. Used by roughly one third of all F5 customers, they perform input validation and apply complex business logic for configuring a wide variety of applications. They hide complexity, sometimes driving hundreds of configuration parameters with just a handful of input values. They also provide deployment guidance, reducing the need for documentation and training. But iApps are often misunderstood and misused, sometimes with frustrating consequences. iApp, not Wizard It is helpful to understand the difference between an iApp and a wizard. A wizard is usually a script with a GUI. It is a tool that accepts a set of user inputs and performs a one-time procedure. When run twice, a wizard performs exactly the same steps during its second run as it did during its first. Although an iApp is also a script with a GUI, it behaves differently on re-entry. Unlike a wizard, an iApp maintains a relationship to the configuration that it generates. iApps have 5 critical properties: iApps always act atomically. The result of deployment is always either the entire intended configuration or none at all. iApp-driven configuration objects are marked so the iApp can track them throughout its lifecycle. A visualization of these configuration objects is presented in the popular TMUI “Component View.” iApps protect the configuration from accidental changes. iApp-driven elements may not be changed via the UI or CLI except through the iApp. iApps support re-entry. Since the iApp framework tracks the configuration objects that it manages, it can be intelligent about which elements are touched during reconfiguration. iApps automatically perform cleanup on deletion. The housekeeping is automatic, and the iApp author does not need to (and is not allowed to) write any delete-time code. Mark and Sweep On initial deployment, an iApp builds a configuration as directed by the TMSH commands in the iApp’s implementation section. On reconfiguration, the iApp framework again runs the implementation, but does not immediately submit the result to BIG-IP. Instead, it first compares the desired (new) configuration with the existing (old) configuration. All similarities are dropped from the workload. If existing elements are missing from the desired configuration, the framework automatically prepares delete requests for those elements. In the end, the changes submitted to BIG-IP are the minimum necessary to accomplish the desired configuration. TMSH “create” commands used to build config on the initial deployment are automatically changed to “modify” on re-entry. This mechanism, known as “mark-and-sweep,” reduces a scripted change set to its net effect, often allowing iApps to reconfigure BIG-IP with no disruption to data plane traffic. Strict Updates As noted above, an iApp tags all of the configuration objects under its control and BIG-IP prevents users from modifying these objects outside the iApp. This restriction, called “strict-updates”, can be disabled. The negative consequences of this action are often underestimated. To illustrate the typical frustration, consider a simple iApp that accepts 2 inputs and constructs a virtual server with a custom HTTP profile: presentation { section virtual { string ip required choice xff { “enabled”, “disabled” } } } implementation { tmsh::create ltm profile http ${tmsh::app_name}_http \ insert-xforwarded-for $::virtual__xff tmsh::create ltm virtual ${tmsh::app_name}_vs \ destination $::virtual__ip:80 \ profiles add \{ ${tmsh::app_name}_http \} } Suppose you deploy this iApp with X-Forwarded-For enabled, then disable strict-updates and edit the HTTP profile directly as shown: Now, re-enter and re-deploy the iApp without changing any inputs. What is the result? You might expect your manual customization of response chunking to remain in place, because there is no mention of response chunking in the iApp code. In this case, however, iApp re-deployment causes chunking to be set back to its default value. Consider another change, this time to the virtual server: Disable address translation, then re-enter and re-deploy the iApp again. What is the result this time? You might expect your customization to be overwritten as it was in the previous example. In this case, however, address translation remains disabled. The TMSH “modify” command does not handle defaults consistently across all key-value pairs. This makes tampering with an iApp-enforced configuration especially hazardous. Right Use Hopefully, this article provides some insights that will help you use iApps to your best advantage. An iApp is a scripted management tool that is dedicated to an application for life. If you find yourself wishing to disengage the iApp and tamper with its configuration, instead consider modifying the iApp code to accomplish your goals. If you absolutely must disable strict-updates, do so with crystal-clear expectations for managing that application going forward.
F5 Automated Backups - The Right Way
Hi all, Often I've been scouring the devcentral fora and codeshares to find that one piece of handywork that will drastically simplify my automated backup needs on F5 devices. Based on the works of Jason Rahm in his post "Third Time's the Charm: BIG-IP Backups Simplified with iCall" on the 26th of June 2013, I went ahead and created my own iApp that pretty much provides the answers for all my backup-needs. Here's a feature list of this iApp: It allows you to choose between both UCS or SCF as backup-types. (whilst providing ample warnings about SCF not being a very good restore-option due to the incompleteness in some cases) It allows you to provide a passphrase for the UCS archives (the standard GUI also does this, so the iApp should too) It allows you to not include the private keys (same thing: standard GUI does it, so the iApp does it too) It allows you to set a Backup Schedule for every X minutes/hours/days/weeks/months or a custom selection of days in the week It allows you to set the exact time, minute of the hour, day of the week or day of the month when the backup should be performed (depending on the usefulness with regards to the schedule type) It allows you to transfer the backup files to external devices using 4 different protocols, next to providing local storage on the device itself SCP (username/private key without password) SFTP (username/private key without password) FTP (username/password) SMB (using smbclient, with username/password) Local Storage (/var/local/ucs or /var/local/scf) It stores all passwords and private keys in a secure fashion: encrypted by the master key of the unit (f5mku), rendering it safe to store the backups, including the credentials off-box It has a configurable automatic pruning function for the Local Storage option, so the disk doesn't fill up (i.e. keep last X backup files) It allows you to configure the filename using the date/time wildcards from the tcl [clock] command, as well as providing a variable to include the hostname It requires only the WebGUI to establish the configuration you desire It allows you to disable the processes for automated backup, without you having to remove the Application Service or losing any previously entered settings For the external shellscripts it automatically generates, the credentials are stored in encrypted form (using the master key) It allows you to no longer be required to make modifications on the linux command line to get your automated backups running after an RMA or restore operation It cleans up after itself, which means there are no extraneous shellscripts or status files lingering around after the scripts execute I wasn't able to upload the iApp template to this article, so I threw it on pastebin: http://pastebin.com/YbDj3eMN Enjoy! Thomas Schockaert
F5 Analytics iApp
Problem this snippet solves: Analytics iApp v3.7.0 You can use this fully supported version of the analytics iApp template to marshal statistical and logging data from the BIG-IP system. The iApp takes this data and formats it as a JSON object which is then exported for consumption by data consumers, such as F5 BIG-IQ or applications such as Splunk. The Analytics iApp allows you to configure several categories of data to be exported. For data consumers like Splunk, the iApp lets you configure the network endpoint to which the data is sent. Version 3.7.0 of the iApp template is fully supported by F5 and available on downloads.f5.com. We recommend all users upgrade to this version. For more information, see https://support.f5.com/csp/article/K07859431. While this version of the iApp is nearly identical to the v3.6.13 which was available on this page, the major difference (other than being fully supported) is that ability to gather APM statistics using the iApp has been removed from BIG-IP versions prior to 12.0. Supported/Tested BIG-IP versions: 11.4.0 - 12.1.2. Data Sources: LTM, GTM, AFM, ASM, APM, SWG, and iHealth (APM statistics require 12.0 or later) Data Output Formats: Splunk, F5 Analytics, F5 Risk Engine Splunk App: https://apps.splunk.com/apps/id/f5 The new deployment guide can be found on F5.com: http://f5.com/pdf/deployment-guides/f5-analytics-dg.pdf Video Demo - https://player.vimeo.com/video/156773835 Solution Architecture - 20s Installation - 1m53s UI Demo Device Dashboard - 6m44s Application Issue Troubleshooting - 9m26s Application Team Self Service - 12m17s Code : https://downloads.f5.com/esd/ecc.sv?sw=BIG-IP&pro=iApp_Templates&ver=iApps&container=iApp-Templates
Creating, Importing and Assigning a CA Certificate Bundle
Within this article, I will be using a personal and relative use case to my own customers. While many organizations may only have one or two Root CA's to identify, the US Department of Defense has numerous CA's sometimes making it difficult for new F5 admins to grasp the concept of a certificate bundle and where to use it. In this article I wanted to take just a few minutes to walk you through the creation of a CA bundle, importing it into the BIG-IP and where you would apply the bundle to perform functions such as smart card authentication. If you would like to attempt to use the cert bundle iApp created by F5, the iApp deployment guide can be found using the link below though deploying that iApp is outside the scope of this document. With all of that, let's begin. https://f5.com/solutions/deployment-guides/ca-bundle-iapp-big-ip-v115-v12 Obtaining all CA's to Include in the Bundle For DoD customers, navigate to https://iase.disa.mil/pki-pke/Pages/index.aspx Select For Administrators, Integrators and Developers Select Tools and continue to browse until you locate PKI CA Certificate Bundles: PKCS#7 Select and download the certificate bundle that is appropriate for your organization though as an example I have selected For DoD PKI Only. Locate the Zip file in the directory you downloaded it to, right click and select extract. Copy Required Files to the BIG-IP After you have extracted all of the files within the .zip file, using your preferred SCP application, copy the file name with the extension of .pem.p7b as shown in this example Certificates_PKCS7_v5.3_DoD.pem.p7b. Note: For Windows users PowerShell can be used to SCP files though if you prefer WinSCP it is certainly acceptable. Run the command scp -p @: Convert the PKCS Certificate to PEM format SSH to the BIG-IP Navigate to the directory you placed the .p7b file during the SCP process above. Run the command openssl pkcs7 -in .p7b -text -out .pem -print_certs to convert the file from the p7b file format to pem. Once you have converted the file from p7b to pem format run the command tmsh install sys crypto cert from-local-file to import the file into the BIG-IP certificate store. To save the config, run the command save sys config Validate Cert Bundle was Imported Log into the BIG-IP TMUI >> System >> Certificate Management >> Traffic Certificate Management >> SSL Certificate List Locate the certificate name that you used in the previous step to import the pem file into the certificate store. To view the certificates included in that bundle, click the certificate bundle and within the certificate tab you can view all certificates. Assign the CA Bundle to a SSL Client Profile This task is most commonly used in SSL client profiles assigned to applications performing smart card or user certificate based authentication. Navigate to Local Traffic >> Profiles >> SSL >> Client Select the profile that will be used for client authentication Scroll until you reach Client Authentication From the drop down menu for Trusted Certificate Authorities, select the bundle created in the previous steps. From the drop down menu for Advertised Certificate Authorities, select the bundle created in the previous steps. Click Update The Trusted Certificate Authorities setting is required only if the BIG-IP system performs Client Certificate Authentication. This setting is specifies the BIG-IP system's Trusted Certificate Authorities store (the CAs that the BIG-IP system trusts when the system verifies a client certificate that is presented during Client Certificate Authentication). The Advertised Certificate Authorities setting is optional. You can use it to specify the CAs that the BIG-IP system advertises as trusted when soliciting a client certificate for client certificate authentication. If the Client Certificate setting is configured to Require or Request, you can configure the Advertised Certificate Authorities setting to send clients a list of CAs that the server is likely to trust. At this point you have successfully created, imported and assigned your new certificate bundle. If you would like to view a complete guide on configuring smart card authentication, please view my articles on DevCentral. Until next time!UDP TCP Packet Duplication
Problem this snippet solves: This iApp provides full configuration of UDP/TCP packet duplication. It is commonly used to duplicate Syslog, SNMP Traps, Netflow, and Sflow data streams to multiple vendor solutions or customers. It also provides fault tolerance capabilities within each duplicated destination. By pointing Network devices, Appliances, and Servers to a VIP distributing network management traffic modifying distribution of streams can be done in one centralized location. UDP packets retain the original source address when sending to the destination locations. Notes: Prior to 11.5 you must add an IPv6 address to any interface to allow for HSL traffic to be sent to the distribution virtual fdf5::1/64 fdf5::2/64 for an HA pair would do it.* TCP traffic does not maintain original source Internal F5 Resources can demo this solution within the UDF environment using the blueprint named "Traffic Duplication Demo" Contributed by: Ken Bocchino 20200807 - Updated to v2.2 How to use this snippet:Microsoft Skype for Business Server 2015
Problem this snippet solves: New release candidate iApp template and deployment guide for Microsoft Skype for Business Server 2015 (formerly Lync Server 2010/2013). For more information and complete guidance on configuring the iApp template, see the associated deployment guide: http://www.f5.com/pdf/deployment-guides/microsoft-skype-for-business-dg.pdf f5.microsoft_skype_server_2015.v1.0.0rc9: posted to downloads.f5.com in 11/2017 RC-9 was posted to downloads.f5.com (as will most new versions of this template). It contained the following changes: new BIG-IP AFM IP Intelligence threat categories to support BIG-IP v13.1 and support for route domain 0 from non-Common partitions. f5.microsoft_skype_server_2015.v1.0.0rc7: posted 09/21/2016 RC-7 provides additional SIP domain support within reverse proxy, a monitor schema change for reverse proxy to make use of the 200 OK response when querying lyncdiscover/lyncdiscoverinternal, support for the director service standalone use case(separate LTM from Front End service), added support to ask for the IP phone update url to allow connections through reverse proxy and added a port 80 Virtual Server in addition to the existing 443 Virtual Server for reverse proxy. RC 5 and 6 were never released to the public, this includes changes as a part of those RC's f5.microsoft_skype_server_2015.v1.0.0rc4: posted 02/16/2016 RC-4 Fixes a security log profile error when deploying on versions of BIG-IP earlier than 11.4, where AFM is not available. f5.microsoft_skype_server_2015.v1.0.0rc3: posted 01/22/2016 RC-3 attaches a supplemental ICMP monitor to the Edge internal UDP virtual server. See https://support.f5.com/kb/en-us/solutions/public/6000/100/sol6143.html for more information. f5.microsoft_skype_server_2015.v1.0.0rc2: posted 01/11/2016 RC-2 contains only a small correction to the iRule produced by the iApp template. The iApp will now always force the FQDN written to lowercase in the iRule, even if the user enters CAPITAL letters. f5.microsoft_skype_server_2015.v1.0.0rc1: posted 07/06/2015 New iApp template for Skype for Business. Code : 70782Complete F5 Automated Backup Solution
Problem this snippet solves: Hi all, Often I've been scouring the devcentral fora and codeshares to find that one piece of handywork that will drastically simplify my automated backup needs on F5 devices. Based on the works of Jason Rahm in his post "Third Time's the Charm: BIG-IP Backups Simplified with iCall" on the 26th of June 2013, I went ahead and created my own iApp that pretty much provides the answers for all my backup-needs. Here's a feature list of this iApp: It allows you to choose between both UCS or SCF as backup-types. (whilst providing ample warnings about SCF not being a very good restore-option due to the incompleteness in some cases) It allows you to provide a passphrase for the UCS archives (the standard GUI also does this, so the iApp should too) It allows you to not include the private keys (same thing: standard GUI does it, so the iApp does it too) It allows you to set a Backup Schedule for every X minutes/hours/days/weeks/months or a custom selection of days in the week It allows you to set the exact time, minute of the hour, day of the week or day of the month when the backup should be performed (depending on the usefulness with regards to the schedule type) It allows you to transfer the backup files to external devices using 4 different protocols, next to providing local storage on the device itself * SCP (username/private key without password) SFTP (username/private key without password) FTP (username/password) SMB (using smbclient, with username/password) Local Storage (/var/local/ucs or /var/local/scf) It stores all passwords and private keys in a secure fashion: encrypted by the master key of the unit (f5mku), rendering it safe to store the backups, including the credentials off-box It has a configurable automatic pruning function for the Local Storage option, so the disk doesn't fill up (i.e. keep last X backup files) It allows you to configure the filename using the date/time wildcards from the tcl clock command, as well as providing a variable to include the hostname It requires only the WebGUI to establish the configuration you desire It allows you to disable the processes for automated backup, without you having to remove the Application Service or losing any previously entered settings For the external shellscripts it automatically generates, the credentials are stored in encrypted form (using the master key) It allows you to no longer be required to make modifications on the linux command line to get your automated backups running after an RMA or restore operation It cleans up after itself, which means there are no extraneous shellscripts or status files lingering around after the scripts execute Enjoy! Thomas Schockaert Contributed by: Thomas Schockaert How to use this snippet: minimum version 11.4 Code : 67735 Tested this on version: 11.4
How to check the support id.
Hi Guys, I am new for F5. Dome times users can able to access our clients network from outside. User provides us a support id. Now my query is how to check this support id in LTM and allow that URL so that user's can able to access the URL.Kindly help me to fix issue. Please tell me the steps. Regards Tan_Sal
