F5 iApp Automated Backup
Problem this snippet solves: This is now available on GitHub! Please look on GitHub for the latest version, and submit any bugs or questions as an "Issue" on GitHub: (Note: DevCentral admin update - Daniel's project appears abandoned so it's been forked and updated to the link below. @damnski on github added some SFTP code that has been merged in as well.) https://github.com/f5devcentral/f5-automated-backup-iapp Intro Building on the significant work of Thomas Schockaert (and several other DevCentralites) I enhanced many aspects I needed for my own purposes, updated many things I noticed requested on the forums, and added additional documentation and clarification. As you may see in several of my comments on the original posts, I iterated through several 2.2.x versions and am now releasing v3.0.0. Below is the breakdown! Also, I have done quite a bit of testing (mostly on v13.1.0.1 lately) and I doubt I've caught everything, especially with all of the changes. Please post any questions or issues in the comments. Cheers! Daniel Tavernier (tabernarious) Related posts: Git Repository for f5-automated-backup-iapp (https://github.com/tabernarious/f5-automated-backup-iapp) https://community.f5.com/t5/technical-articles/f5-automated-backups-the-right-way/ta-p/288454 https://community.f5.com/t5/crowdsrc/complete-f5-automated-backup-solution/ta-p/288701 https://community.f5.com/t5/crowdsrc/complete-f5-automated-backup-solution-2/ta-p/274252 https://community.f5.com/t5/technical-forum/automated-backup-solution/m-p/24551 https://community.f5.com/t5/crowdsrc/tkb-p/CrowdSRC v3.2.1 (20201210) Merged v3.1.11 and v3.2.0 for explicit SFTP support (separate from SCP). Tweaked the SCP and SFTP upload directory handling; detailed instructions are in the iApp. Tested on 13.1.3.4 and 14.1.3 v3.1.11 (20201210) Better handling of UCS passphrases, and notes about characters to avoid. I successfully tested this exact passphrase in the 13.1.3.4 CLI (surrounded with single quote) and GUI (as-is): `~!@#$%^*()aB1-_=+[{]}:./? I successfully tested this exact passphrase in 14.1.3 (square-braces and curly-braces would not work): `~!@#$%^*()aB1-_=+:./? Though there may be situations these could work, avoid these characters (separated by spaces): " ' & | ; < > \ [ ] { } , Moved changelog and notes from the template to CHANGELOG.md and README.md. Replaced all tabs (\t) with four spaces. v3.1.10 (20201209) Added SMB Version and SMB Security options to support v14+ and newer versions of Microsoft Windows and Windows Server. Tested SMB/CIFS on 13.1.3.4 and 14.1.3 against Windows Server 2019 using "2.0" and "ntlmsspi" v3.1.0: Removed "app-service none" from iCall objects. The iCall objects are now created as part of the Application Service (iApp) and are properly cleaned up if the iApp is redeployed or deleted. Reasonably tested on 11.5.4 HF2 (SMB worked fine using "mount -t cifs") and altered requires-bigip-version-min to match. Fixing error regarding "script did not successfully complete: (can't read "::destination_parameters__protocol_enable": no such variable" by encompassing most of the "implementation" in a block that first checks $::backup_schedule__frequency_select for "Disable". Added default value to "filename format". Changed UCS default value for $backup_file_name_extension to ".ucs" and added $fname_noext. Removed old SFTP sections and references (now handled through SCP/SFTP). Adjusted logging: added "sleep 1" to ensure proper logging; added $backup_directory to log message. Adjusted some help messages. New v3.0.0 features: Supports multiple instances! (Deploy multiple copies of the iApp to save backups to different places or perhaps to keep daily backups locally and send weekly backups to a network drive.) Fully ConfigSync compatible! (Encrypted values now in $script instead of local file.) Long passwords supported! (Using "-A" with openssl which reads/writes base64 encoded strings as a single line.) Added $script error checking for all remote backup types! (Using 'catch' to prevent tcl errors when $script aborts.) Backup files are cleaned up after any $script errors due to new error checking. Added logging! (Run logs sent to '/var/log/ltm' via logger command which is compatible with BIG-IP Remote Logging configuration (syslog). Run logs AND errors sent to '/var/tmp/scriptd.out'. Errors may include plain-text passwords which should not be in /var/log/ltm or syslog.) Added custom cipher option for SCP! (In case BIG-IP and the destination server are not cipher-compatible out of the box.) Added StrictHostKeyChecking=no option. (This is insecure and should only be used for testing--lots of warnings.) Combined SCP and SFTP because they are both using SCP to perform the remote copy. (Easier to maintain!) Original v1.x.x and v2.x.x features kept (copied from an original post): It allows you to choose between both UCS or SCF as backup-types. (whilst providing ample warnings about SCF not being a very good restore-option due to the incompleteness in some cases) It allows you to provide a passphrase for the UCS archives (the standard GUI also does this, so the iApp should too) It allows you to not include the private keys (same thing: standard GUI does it, so the iApp does it too) It allows you to set a Backup Schedule for every X minutes/hours/days/weeks/months or a custom selection of days in the week It allows you to set the exact time, minute of the hour, day of the week or day of the month when the backup should be performed (depending on the usefulness with regards to the schedule type) It allows you to transfer the backup files to external devices using 4 different protocols, next to providing local storage on the device itself SCP (username/private key without password) SFTP (username/private key without password) FTP (username/password) SMB (now using TMOS v12.x.x compatible 'mount -t cifs', with username/password) Local Storage (/var/local/ucs or /var/local/scf) It stores all passwords and private keys in a secure fashion: encrypted by the master key of the unit (f5mku), rendering it safe to store the backups, including the credentials off-box It has a configurable automatic pruning function for the Local Storage option, so the disk doesn't fill up (i.e. keep last X backup files) It allows you to configure the filename using the date/time wildcards from the tcl [clock] command, as well as providing a variable to include the hostname It requires only the WebGUI to establish the configuration you desire It allows you to disable the processes for automated backup, without you having to remove the Application Service or losing any previously entered settings For the external shellscripts it automatically generates, the credentials are stored in encrypted form (using the master key) It allows you to no longer be required to make modifications on the linux command line to get your automated backups running after an RMA or restore operation It cleans up after itself, which means there are no extraneous shellscripts or status files lingering around after the scripts execute How to use this snippet: Find and download the latest iApp template on GitHub (e.g "f5.automated_backup.v3.2.1.tmpl.tcl"). Import the text file as an iApp Template in the BIG-IP GUI. Create an Application Service using the imported Template. Answer the questions (paying close attention to the help sections). Check /var/tmp/scriptd.out for general logs and errors. Tested this on version: 16.0
F5 Analytics iApp
Problem this snippet solves: Analytics iApp v3.7.0 You can use this fully supported version of the analytics iApp template to marshal statistical and logging data from the BIG-IP system. The iApp takes this data and formats it as a JSON object which is then exported for consumption by data consumers, such as F5 BIG-IQ or applications such as Splunk. The Analytics iApp allows you to configure several categories of data to be exported. For data consumers like Splunk, the iApp lets you configure the network endpoint to which the data is sent. Version 3.7.0 of the iApp template is fully supported by F5 and available on downloads.f5.com. We recommend all users upgrade to this version. For more information, see https://support.f5.com/csp/article/K07859431. While this version of the iApp is nearly identical to the v3.6.13 which was available on this page, the major difference (other than being fully supported) is that ability to gather APM statistics using the iApp has been removed from BIG-IP versions prior to 12.0. Supported/Tested BIG-IP versions: 11.4.0 - 12.1.2. Data Sources: LTM, GTM, AFM, ASM, APM, SWG, and iHealth (APM statistics require 12.0 or later) Data Output Formats: Splunk, F5 Analytics, F5 Risk Engine Splunk App: https://apps.splunk.com/apps/id/f5 The new deployment guide can be found on F5.com: http://f5.com/pdf/deployment-guides/f5-analytics-dg.pdf Video Demo - https://player.vimeo.com/video/156773835 Solution Architecture - 20s Installation - 1m53s UI Demo Device Dashboard - 6m44s Application Issue Troubleshooting - 9m26s Application Team Self Service - 12m17s Code : https://downloads.f5.com/esd/ecc.sv?sw=BIG-IP&pro=iApp_Templates&ver=iApps&container=iApp-TemplatesUDP TCP Packet Duplication
Problem this snippet solves: This iApp provides full configuration of UDP/TCP packet duplication. It is commonly used to duplicate Syslog, SNMP Traps, Netflow, and Sflow data streams to multiple vendor solutions or customers. It also provides fault tolerance capabilities within each duplicated destination. By pointing Network devices, Appliances, and Servers to a VIP distributing network management traffic modifying distribution of streams can be done in one centralized location. UDP packets retain the original source address when sending to the destination locations. Notes: Prior to 11.5 you must add an IPv6 address to any interface to allow for HSL traffic to be sent to the distribution virtual fdf5::1/64 fdf5::2/64 for an HA pair would do it.* TCP traffic does not maintain original source Internal F5 Resources can demo this solution within the UDF environment using the blueprint named "Traffic Duplication Demo" Contributed by: Ken Bocchino 20200807 - Updated to v2.2 How to use this snippet:Microsoft Skype for Business Server 2015
Problem this snippet solves: New release candidate iApp template and deployment guide for Microsoft Skype for Business Server 2015 (formerly Lync Server 2010/2013). For more information and complete guidance on configuring the iApp template, see the associated deployment guide: http://www.f5.com/pdf/deployment-guides/microsoft-skype-for-business-dg.pdf f5.microsoft_skype_server_2015.v1.0.0rc9: posted to downloads.f5.com in 11/2017 RC-9 was posted to downloads.f5.com (as will most new versions of this template). It contained the following changes: new BIG-IP AFM IP Intelligence threat categories to support BIG-IP v13.1 and support for route domain 0 from non-Common partitions. f5.microsoft_skype_server_2015.v1.0.0rc7: posted 09/21/2016 RC-7 provides additional SIP domain support within reverse proxy, a monitor schema change for reverse proxy to make use of the 200 OK response when querying lyncdiscover/lyncdiscoverinternal, support for the director service standalone use case(separate LTM from Front End service), added support to ask for the IP phone update url to allow connections through reverse proxy and added a port 80 Virtual Server in addition to the existing 443 Virtual Server for reverse proxy. RC 5 and 6 were never released to the public, this includes changes as a part of those RC's f5.microsoft_skype_server_2015.v1.0.0rc4: posted 02/16/2016 RC-4 Fixes a security log profile error when deploying on versions of BIG-IP earlier than 11.4, where AFM is not available. f5.microsoft_skype_server_2015.v1.0.0rc3: posted 01/22/2016 RC-3 attaches a supplemental ICMP monitor to the Edge internal UDP virtual server. See https://support.f5.com/kb/en-us/solutions/public/6000/100/sol6143.html for more information. f5.microsoft_skype_server_2015.v1.0.0rc2: posted 01/11/2016 RC-2 contains only a small correction to the iRule produced by the iApp template. The iApp will now always force the FQDN written to lowercase in the iRule, even if the user enters CAPITAL letters. f5.microsoft_skype_server_2015.v1.0.0rc1: posted 07/06/2015 New iApp template for Skype for Business. Code : 70782
Complete F5 Automated Backup Solution
Problem this snippet solves: Hi all, Often I've been scouring the devcentral fora and codeshares to find that one piece of handywork that will drastically simplify my automated backup needs on F5 devices. Based on the works of Jason Rahm in his post "Third Time's the Charm: BIG-IP Backups Simplified with iCall" on the 26th of June 2013, I went ahead and created my own iApp that pretty much provides the answers for all my backup-needs. Here's a feature list of this iApp: It allows you to choose between both UCS or SCF as backup-types. (whilst providing ample warnings about SCF not being a very good restore-option due to the incompleteness in some cases) It allows you to provide a passphrase for the UCS archives (the standard GUI also does this, so the iApp should too) It allows you to not include the private keys (same thing: standard GUI does it, so the iApp does it too) It allows you to set a Backup Schedule for every X minutes/hours/days/weeks/months or a custom selection of days in the week It allows you to set the exact time, minute of the hour, day of the week or day of the month when the backup should be performed (depending on the usefulness with regards to the schedule type) It allows you to transfer the backup files to external devices using 4 different protocols, next to providing local storage on the device itself * SCP (username/private key without password) SFTP (username/private key without password) FTP (username/password) SMB (using smbclient, with username/password) Local Storage (/var/local/ucs or /var/local/scf) It stores all passwords and private keys in a secure fashion: encrypted by the master key of the unit (f5mku), rendering it safe to store the backups, including the credentials off-box It has a configurable automatic pruning function for the Local Storage option, so the disk doesn't fill up (i.e. keep last X backup files) It allows you to configure the filename using the date/time wildcards from the tcl clock command, as well as providing a variable to include the hostname It requires only the WebGUI to establish the configuration you desire It allows you to disable the processes for automated backup, without you having to remove the Application Service or losing any previously entered settings For the external shellscripts it automatically generates, the credentials are stored in encrypted form (using the master key) It allows you to no longer be required to make modifications on the linux command line to get your automated backups running after an RMA or restore operation It cleans up after itself, which means there are no extraneous shellscripts or status files lingering around after the scripts execute Enjoy! Thomas Schockaert Contributed by: Thomas Schockaert How to use this snippet: minimum version 11.4 Code : 67735 Tested this on version: 11.4TCP iApp template
Problem this snippet solves: Use this TCP iApp template to configure high availability and optimization for TCP implementations. As of 3/29/17, this release is now available on downloads.f5.com in the Release-Candidates folder of the TCP directory. RC4 RC4 fixes an issue where the iApp would fail if using a non-default Route Domain. RC3 RC3 contains updates for new BIG-IP AFM IP Intelligence threat categories to support BIG-IP v13.1, and adds support for route domain 0 from non-Common partitions. RC2 RC2 of the iApp template solves an issue where the iApp deployment fails when a system management route is not present. RC1 This initial release of the iApp is available on downloads.f5.com in the RELEASE-CANDIDATE directory. Until the associated deployment guide is available, for detailed information and configuration assistance see the inline help. How to use this snippet: Extract (unzip) the latest version of the tcp iApp template file. From the BIG-IP system web-based Configuration utility. On the Main tab, expand iApp, and then click Templates. Click the Import button on the right side of the screen. Click a check in the Overwrite Existing Templates box. Click the Browse button, and then browse to the location you saved the iApp file. Click the Upload button. The iApp is now available for use. Code : https://downloads.f5.com/esd/index.jsp
Microsoft Active Directory Federation Services (AD FS) iApp Template
Problem this snippet solves: Use this iApp template for configuring standard load balancing, monitoring and TCP optimization for Microsoft Active Directory Federation Servers (AD FS and AD FS Proxy). If APM is provisioned, the template should support configuring pre-authentication for ADFS servers running in Windows Authentication mode. Minimum required BIG-IP version: 11.2. If you are deploying APM for authentication proxy to AD FS services, you must enable Windows Authentication in the Intranet section of the AD FS Global Authentication Policy. v1.2.0 iApps v1.2.0rc1 Added support for ADFS 4.0 Made 49443 device registration/certificate authentication objects optional via a question. Made ADFSPIP iRule automatic but only when APM set to yes. Added support for an existing APM profile to be selected from within the iApp. Added forms SSO for /adfs/ls endpoint into the iApp via a question v1.2.0rc2 Fixed an "app_health__frequency variable not found" issue when using a custom monitor Added support if a custom pool is chosen AND certificate authentication/device registration is set to yes to display an option for what pool to use for cert auth/device registration.(As the ports would be different) v1.1.0 iApps v1.1.0rc2 Added certificate auth objects(49443) and MS-ADFSPIP headers irule. Added iRule to disable APM for MS Federation Gateway endpoint(s) v1.0.0 iApps v1.0.0rc1 Initial release. v1.0.0rc2 Fixed an "iapp::template_start" error when importing the template. v1.0.0rc3 Fixed a "runtime exceeded" error caused by incorrect syntax in external SNI monitor. v1.0.0rc4 Corrected external monitor cURL command to fix issue with pool members being marked down incorrectly. v1.0.0rc5 Added support for FastL4 deployment. v1.0.0rc6 Fixed issue with broken APM Quick Start page previews. v1.0.0rc7 Changes to external monitor script: removed verbose flag; corrected output redirection. Fixed an issue with the associated cli script that could prevent users from importing iApp templates. Official release of 1.0.0 The official F5 supported version of this iApp is now on downloads.f5.com. See https://support.f5.com/kb/en-us/solutions/public/17000/000/sol17041.html for information. For the associated Deployment Guide, see http://www.f5.com/pdf/deployment-guides/microsoft-adfs-dg.pdf Code : 73996Exchange 2016
Problem this snippet solves: f5.microsoft_exchange_2016.v1.0.0 The new, fully supported iApp template for Exchange 2016 has been released to downloads.f5.com in the root directory. Use the deployment guide or the AskF5 solution for instructions on downloading the iApp template. An updated deployment guide is now available at http://f5.com/pdf/deployment-guides/microsoft-exchange-2016-dg.pdf Microsoft Exchange 2016 rc1 Use this F5 contributed, release candidate iApp template for Microsoft Exchange Server 2016 deployments. Until the accompanying deployment guide for 2016 is released, use the inline help for assistance in configuring the iApp for Exchange 2016. Code : https://support.f5.com/content/kb/en-us/solutions/public/k/11/sol11100442
HTTP Explicit Proxy - V11.5+
Problem this snippet solves: This iApp configures an Explicit Proxy using the new "Explicit" Proxy Mode that was introduced into the HTTP Profile in BIG-IP 11.5. You only need LTM or APM provisioned. It creates all configuration components required including: DNS Resolvers TCP Tunnel HTTP Profile (Explicit) Default Connect Handling set to Allow SNAT Pools (Optional) SNAT Default is Automap If you require the Explicit Proxy to listen on more than 1 port e.g 3128 and 8080, simply just create another Application Service. Contributed by: Brett Smith How to use this snippet:SMTP iApp Template - Early Release
Problem this snippet solves: INITIAL RELEASE Minimum required BIG-IP version: 11.4.0. Supported BIG-IP versions: 11.4.0-12.0 v1.0.0rc1 iApp template for configuring standard load balancing, monitoring, SSL offloading, and TCP optimization for Simple Mail Transfer Protocol (SMTP). The template also supports deploying F5's Advanced Firewall Manager (AFM), when AFM is licensed and provisioned. v1.0.0rc2 There were no changes to the functionality in this release. Minor changes to clarify some of the questions and answers. Added inline help entries. v1.0.0rc3 Fixed an issue with the associated cli script that could prevent users from importing iApp templates. v1.0.0rc4 Fixed an issue with selecting password-protected encryption keys. To use a password-protected encryption key, you must create an SSL profile that uses the key and specify that profile where indicated in the iApp template. v1.0.0rc5 Fixed an issue with incorrectly formatted external monitor scripts. v1.0.0rc7 Fixed an issue with monitors utilized in the server-side ssl scenarios, as a result the openssl eav monitor is used in the 'no msg submitted' monitor scenarios. A fifth monitor option was presented as well to break the 'auth/no msg' option into basic and ntlm so the iApp can use openssl if Basic(auth login) is selected. - This release also allows a custom receive string to be specified(advanced must be selected). v1.0.0rc8 Minor updates and enhancements to the monitor choices. For the associated deployment guide, see [http://www.f5.com/pdf/deployment-guides/f5-smtp-dg.pdf] Contributed by: F5 Code : 83126 Tested this on version: 12.0
