AS3 Best Practice
Introduction AS3 is a declarative API that uses JSON key-value pairs to describe a BIG-IP configuration. From virtual IP to virtual server, to the members, pools, and nodes required, AS3 provides a simple, readable format in which to describe a configuration. Once you've got the configuration, all that's needed is to POST it to the BIG-IP, where the AS3 extension will happily accept it and execute the commands necessary to turn it into a fully functional, deployed BIG-IP configuration. If you are new to AS3, start reading the following references: Products - Automation and orchestration toolchain(f5.com; Product information) Application Services 3 Extension Documentation(clouddocs; API documentation and guides) F5 Application Services 3 Extension(AS3) (GitHub; Source repository) This article describes some considerations in order to efficiently deploy the AS3 configurations. Architecture In the TMOS space, the services that AS3 provides are processed by a daemon named 'restnoded'. It relies on the existing BIG-IP framework for deploying declarations. The framework consists of httpd, restjavad and icrd_child as depicted below (the numbers in parenthesis are listening TCP port numbers). These processes are also used by other services. For example, restjavad is a gateway for all the iControl REST requests, and is used by a number of services on BIG-IP and BIG-IQ. When an interaction between any of the processes fails, AS3 operation fails. The failures stem from lack of resources, timeouts, data exceeding predefined thresholds, resource contention among the services, and more. In order to complete AS3 operations successfully, it is advised to follow the Best Practice outlined below. Best Practice Your single source of truth is your declaration Refrain from overwriting the AS3-deployed BIG-IP configurations by the other means such as TMSH, GUI or iControl REST calls. Since you started to use the AS3 declarative model, the source of truth for your device's configurations is in your declaration, not the BIG-IP configuration files. Although AS3 tries to weigh BIG-IP locally stored configurations as much as it can do, discrepancy between the declaration and the current configuration on BIG-IP may cause the AS3 to perform less efficiently or error unexpectedly. When you wish to change a section of a tenant (e.g., pool name change), modify the declaration and submit it. Keep the number of applications in one tenant to a minimum AS3 processes each tenant separately.Having too many applications (virtual servers) in a single tenant (partition) results in a lengthy poll when determining the current configuration. In extreme cases (thousands of virtuals), the action may time out. When you want to deploy a thousand or more applications on a single device, consider chunking the work for AS3 by spreading the applications across multiple tenants (say, 100 applications per tenant). AS3 tenant access behavior behaves as BIG-IP partition behavior.A non-Common partition virtual cannot gain access to another partition's pool, and in the same way, an AS3 application does not have access to a pool or profile in another tenant.In order to share configuration across tenants, AS3 allows configuration of the "Shared" application within the "Common" tenant.AS3 avoids race conditions while configuring /Common/Shared by processing additions first and deletions last, as shown below.This dual process may cause some additional delay in declaration handling. Overwrite rather than patching (POSTing is a more efficient practice than PATCHing) AS3 is a stateless machine and is idempotent. It polls BIG-IP for its full configuration, performs a current-vs-desired state comparison, and generates an optimal set of REST calls to fill the differences.When the initial state of BIG-IP is blank, the poll time is negligible.This is why initial configuration with AS3 is often quicker than subsequent changes, especially when the tenant contains a large number of applications. AS3 provides the means to partially modify using PATCH (seeAS3 API Methods Details), but do not expect PATCH changes to be performant.AS3 processes each PATCH by (1) performing a GET to obtain the last declaration, (2) patching that declaration, and (3) POSTing the entire declaration to itself.A PATCH of one pool member is therefore slower than a POST of your entire tenant configuration.If you decide to use PATCH,make sure that the tenant configuration is a manageable size. Note: Using PATCH to make a surgical change is convenient, but using PATCH over POST breaks the declarative model. Your declaration should be your single source of truth.If you include PATCH, the source of truth becomes "POST this file, then apply one or more PATCH declarations." Get the latest version AS3 is evolving rapidly with new features that customers have been wishing for along with fixes for known issues. Visitthe AS3 section of the F5 Networks Github.Issuessection shows what features and fixes have been incorporated. For BIG-IQ, check K54909607: BIG-IQ Centralized Management compatibility with F5 Application Services 3 Extension and F5 Declarative Onboarding for compatibilities with BIG-IQ versions before installation. Use administrator Use a user with the administrator role when you submit your declaration to a target BIG-IP device. Your may find your role insufficient to manipulate BIG-IP objects that are included in your declaration. Even one authorized item will cause the entire operation to fail and role back. See the following articles for more on BIG-IP user and role. Manual Chapter : User Roles (12.x) Manual Chapter : User Roles (13.x) Manual Chapter : User Roles (14.x) Prerequisites and Requirements(clouddocs AS3 document) Use Basic Authentication for a large declaration You can choose either Basic Authentication (HTTP Authorization header) or Token-Based Authentication (F5 proprietary X-F5-Auth-Token) for accessing BIG-IP. While the Basic Authentication can be used any time, a token obtained for the Token-Based Authentication expires after 1,200 seconds (20 minutes). While AS3 does re-request a new token upon expiry, it requires time to perform the operation, which may cause AS3 to slow down. Also, the number of tokens for a user is limited to 100 (since 13.1), hence if you happen to have other iControl REST players (such as BIG-IQ or your custom iControl REST scripts) using the Token-Based Authentication for the same user, AS3 may not be able to obtain the next token, and your request will fail. See the following articles for more on the Token-Based Authentication. Demystifying iControl REST Part 6: Token-Based Authentication(DevCentral article). iControl REST Authentication Token Management(DevCentral article) Authentication and Authorization(clouddocs AS3 document) Choose the best window for deployment AS3 (restnoded daemon) is a Control Plane process. It competes against other Control Plane processes such as monpd and iRules LX (node.js) for CPU/memory resources. AS3 uses the iControl REST framework for manipulating the BIG-IP resources. This implies that its operation is impacted by any processes that use httpd (e.g., GUI), restjavad, icrd_child and mcpd. If you have resource-hungry processes that run periodically (e.g., avrd), you may want to run your AS3 declaration during some other time window. See the following K articles for alist of processes K89999342 BIG-IP Daemons (12.x) K05645522BIG-IP Daemons (v13.x) K67197865BIG-IP Daemons (v14.x) K14020: BIG-IP ASM daemons (11.x - 15.x) K14462: Overview of BIG-IP AAM daemons (11.x - 15.x) Workarounds If you experience issues such as timeout on restjavad, it is possible that your AS3 operation had resource issues. After reviewing the Best Practice above but still unable to alleviate the problem, you may be able to temporarily fix it by applying the following tactics. Increase the restjavad memory allocation The memory size of restjavad can be increased by the following tmsh sys db commands tmsh modify sys db provision.extramb value <value> tmsh modify sys db restjavad.useextramb value true The provision.extramb db key changes the maximum Java heap memory to (192 + <value> * 8 / 10) MB. The default value is 0. After changing the memory size, you need to restart restjavad. tmsh restart sys service restjavad See the following article for more on the memory allocation: K26427018: Overview of Management provisioning Increase a number of icrd_child processes restjavad spawns a number of icrd_child processes depending on the load. The maximum number of icrd_child processes can be configured from /etc/icrd.conf. Please consult F5 Support for details. See the following article for more on the icrd_child process verbosity: K96840770: Configuring the log verbosity for iControl REST API related to icrd_child Decrease the verbosity levels of restjavad and icrd_child Writing log messages to the file system is not exactly free of charge. Writing unnecessarily large amount of messages to files would increase the I/O wait, hence results in slowness of processes. If you have changed the verbosity levels of restjavad and/or icrd_child, consider rolling back the default levels. See the following article for methods to change verbosity level: K15436: Configuring the verbosity for restjavad logs on the BIG-IP systemHow to get a F5 BIG-IP VE Developer Lab License
(applies to BIG-IP TMOS Edition) To assist DevOps teams improve their development for the BIG-IP platform, F5 offers a low cost developer lab license.This license can be purchased from your authorized F5 vendor. If you do not have an F5 vendor, you can purchase a lab license online: CDW BIG-IP Virtual Edition Lab License CDW Canada BIG-IP Virtual Edition Lab License Once completed, the order is sent to F5 for fulfillment and your license will be delivered shortly after via e-mail. F5 is investigating ways to improve this process. To download the BIG-IP Virtual Edition, please log into downloads.f5.com (separate login from DevCentral), and navigate to your appropriate virtual edition, example: For VMware Fusion or Workstation or ESX/i:BIGIP-16.1.2-0.0.18.ALL-vmware.ova For Microsoft HyperV:BIGIP-16.1.2-0.0.18.ALL.vhd.zip KVM RHEL/CentoOS: BIGIP-16.1.2-0.0.18.ALL.qcow2.zip Note: There are also 1 Slot versions of the above images where a 2nd boot partition is not needed for in-place upgrades. These images include_1SLOT- to the image name instead of ALL. The below guides will help get you started with F5 BIG-IP Virtual Edition to develop for VMWare Fusion, AWS, Azure, VMware, or Microsoft Hyper-V. These guides follow standard practices for installing in production environments and performance recommendations change based on lower use/non-critical needs fo Dev/Lab environments. Similar to driving a tank, use your best judgement. DeployingF5 BIG-IP Virtual Edition on VMware Fusion Deploying F5 BIG-IP in Microsoft Azure for Developers Deploying F5 BIG-IP in AWS for Developers Deploying F5 BIG-IP in Windows Server Hyper-V for Developers Deploying F5 BIG-IP in VMware vCloud Director and ESX for Developers Note: F5 Support maintains authoritativeAzure, AWS, Hyper-V, and ESX/vCloud installation documentation. VMware Fusion is not an official F5-supported hypervisor so DevCentral publishes the Fusion guide with the help of our Field Systems Engineering teams.
Decrypting TLS traffic on BIG-IP
1 Introduction As soon as I joined F5 Support, over 5 years ago, one of the first things I had to learn quickly was to decrypt TLS traffic becausemost of our customers useL7 applications protectedby TLS layer. In this article, I will show 4 ways to decrypt traffic on BIG-IP, including thenew one just release in v15.xthat is ideal for TLS1.3 where TLS handshake is also encrypted. If that's what you want to know just skip totcpdump --f5 ssl optionsection as this new approach is just a parameter added to tcpdump. As this article is very hands-on, I will show my lab topology for the tests I performed and then every possible way I used to decrypt customer's traffic working for Engineering Services at F5. 2 Lab Topology This is the lab topology I used for the lab test where all tests were performed: Also, for every capture I issued the followingcurlcommand: Update: the virtual server's IP address is actually 10.199.3.145/32 2 The 4 ways to decrypt BIG-IP's traffic RSA private key decryption There are 3 constraints here: Full TLS handshake has to be captured CheckAppendix 2to learn how to to disable BIG-IP's cache RSA key exchange has to be used, i.e. no (EC)DHE CheckAppendix1to understand how to check what's key exchange method used in your TLS connection CheckAppendix 2to understandhow to prioritise RSA as key exchange method Private key has to be copied to Wireshark machine (ssldump command solves this problem) Roughly, to accomplish that we can setCache Sizeto 0 on SSL profile and remove (EC)DHE fromCipher Suites(seeAppendix 1for details) I first took a packet capture using :pmodifier to capture only the client and server flows specific to my Client's IP address (10.199.3.1): Note: The0.0interface will capture any forwarding plane traffic (tmm) andnnnis the highest noise to capture as much flow information as possible to be displayed on the F5 dissector header.For more details about tcpdump syntax, please have a look atK13637: Capturing internal TMM information with tcpdumpandK411: Overview of packet tracing with the tcpdump utility. Also,we need to make sure we capture the full TLS handshake. It's perfectly fine to captureresumed TLS sessionsas long as full TLS handshake has been previously captured. Initially, our capture is unencrypted as seen below: On Mac, I clicked onWireshark→Preferences: ThenProtocols→TLS→RSA keys listwhere we see a window where we can reference BIG-IP's (or server if we want to decrypt server SSL side) private key: Once we get there, we need to add any IP address of the flow we want Wireshark to decrypt, the corresponding port and the private key file (default.crtfor Client SSL profile in this particular lab test): Note:For Client SSL profile, this would be the private key onCertificate Chainfield corresponding to the end entity Certificate being served to client machines through the Virtual Server. For Server SSL profile, the private key is located on the back-end server and the key would be the one corresponding to the end entity Certificate sent in the TLSCertificatemessage sent from back-end server to BIG-IP during TLS handshake. Once we clickOK, we can see the HTTP decrypted traffic (in green): In production environment, we would normally avoid copying private keys to different machines soanother option is usessldumpcommand directly on the server we're trying to capture. Again, if we're capturing Client SSL traffic,ssldumpis already installed on BIG-IP. We would follow the same steps as before but instead of copying private key to Wireshark machine, we would simply issue this command on the BIG-IP (or back-end server if it's Server SSL traffic): Syntax:ssldump-r<capture.pcap>-k<private key.key>-M<type a name for your ssldump file here.pms>. For more details, please have a look atK10209: Overview of packet tracing with the ssldump utility. Inssldump-generated.pms, we should find enough information for Wireshark to decrypt the capture: Syntax:ssldump-r<capture.pcap>-k<private key.key>-M<type a name for your ssldump file here.pms>. For more details, please have a look atK10209: Overview of packet tracing with the ssldump utility. Inssldump-generated.pms, we should find enough information for Wireshark to decrypt the capture: After I clickedOK, we indeed see the decrypted http traffic back again: We didn't have to copy BIG-IP's private key to Wireshark machine here. iRules The only constraint here is that we should apply the iRule to the virtual server in question. Sometimes that's not desirable, especially when we're troubleshooting an issue where we want the configuration to be unchanged. Note: there is abugthat affects versions 11.6.x and 12.x that was fixed on 13.x. It records the wrong TLS Session ID to LTM logs. The workaround would be to manually copy the Session ID from tcpdump capture or to use RSA decryption as in previous example. You can also combine bothSSL::clientrandomandSSL::sessionidwhich isthe ideal: Reference: K12783074: Decrypting SSL traffic using the SSL::sessionsecret iRules command (12.x and later) Again, I took a capture usingtcpdumpcommand: After applying above iRule to our HTTPS virtual server and taking tcpdump capture, I see this on /var/log/ltm: To copy this to a *.pms file wecanuseon Wireshark we can use sed command (reference:K12783074): Note:If you don't want to overwrite completely the PMS file make sure youuse >> instead of >. The endresult would be something like this: As both resumed and full TLS sessions have client random value, I only had to copy CLIENT_RANDOM + Master secret to our PMS file because all Wireshark needs is a session reference to apply master secret. To decrypt file on Wireshark just go toWireshark→Preferences→Protocols→TLS→Pre-Master Key logfile namelike we did inssldumpsection and add file we just created: As seen on LTM logs,CLIENTSSL_HANDSHAKEevent captured master secret from our client-side connection andSERVERSSL_HANDSHAKEfrom server side. In this case, we should have both client and server sides decrypted, even though we never had access to back-end server: Notice added anhttpfilter to show you both client and and server traffic were decrypted this time. tcpdump --f5 ssl option This was introduced in 15.x and we don't need to change virtual server configuration by adding iRules. The only thing we need to do is to enabletcpdump.sslproviderdb variable which is disabled by default: After that, when we take tcpdump capture, we just need to add --f5 ssl to the command like this: Notice that we've got a warning message because Master Secret will be copied to tcpdump capture itself, so we need to be careful with who we share such capture with. I had to update my Wireshark to v3.2.+ and clicked onAnalyze→Enabled Protocols: And enable F5 TLS dissector: Once we open the capture, we can find all the information you need to create our PMS file embedded in the capture: Very cool, isn't it? We can then copy the Master Secretand Client Random values by right clicking like this: And then paste it to a blank PMS file. I first pasted the Client Random value followed by Master Secret value like this: Note: I manually typedCLIENT_RANDOMand then pasted both values for both client and server sides directly from tcpdump capture. The last step was to go toWireshark→Preferences→Protocols→TLSand add it toPre-master-Secret log filenameand clickOK: Fair enough! Capture decrypted on both client and server sides: I usedhttpfilter to display only decrypted HTTP packets just like in iRule section. Appendix 1 How do we know which Key Exchange method is being used? RSA private key can only decrypt traffic on Wireshark if RSA is the key exchange method negotiated during TLS handshake. Client side will tell the Server side which ciphers it support and server side will reply with the chosen cipher onServer Hellomessage. With that in mind, on Wireshark, we'd click onServer Helloheader underCipher Suite: Key Exchange and Authentication both come before theWITHkeyword. In above example, because there's only RSA we can say that RSA is used for both Key Exchange and Authentication. In the following example, ECDHE is used for key exchange and RSAfor authentication: Appendix 2 Disabling Session Resumption and Prioritising RSA key exchange We can set Cache Size to 0 to disableTLS session resumptionand change the Cipher Suites to anything that makes BIG-IP pick RSA for Key Exchange:
BIG-IP Upgrade Procedure Using CLI (vCMP Guest & Host)
Problem this snippet solves: Next article describes an upgrade procedure to perform only using CLI commands. The idea is not to replace an official procedure, but to give a different approach for those guys who love using CLI and they want to execute an upgrade only using commands (without GUI access). The procedure is separated in 4 sections: Data Collection & Planning - for executing some days before the upgrade. Pre-Upgrade Tasks - for executing just before the upgrade (applies to all devices in the cluster). Upgrade Tasks - Only applies for one device in the cluster for each time (normally standby device). Post-Upgrade Tasks - for executing just after the upgrade (applies to all devices in the cluster). This procedure is valid for most of the BIP-IP set-ups: Standalone & clusters vCMP Host & vCMP Guests GTM/DNS Synchronization Groups Everything that helps to fix mistakes is great, so your comments are welcome. OFFICIAL REFERENCES: Release Notes - https://support.f5.com/csp/knowledge-center/software/BIG-IP General Upgrade Procedure - https://support.f5.com/csp/article/K84554955 GTM/DNS Upgrades - https://support.f5.com/csp/article/K11661449 VCMP Host Upgrades - https://support.f5.com/csp/article/K15930#p17 HW Life-Cycle - https://support.f5.com/csp/article/K4309 SW Life-Cycle - https://support.f5.com/csp/article/K5903 HW-SW Compatibility - https://support.f5.com/csp/article/K9476 Upgrade Path - https://support.f5.com/csp/article/K13845 How to use this snippet: >> DATA COLLECTION & PLANNING (ALL CLUSTER DEVICES) >> PRE-UPGRADE TASKS (ALL CLUSTER DEVICES) >> UPGRADE TASKS (ONE DEVICE AT TIME) >> POST-UPGRADE TASKS (ALL CLUSTER DEVICES) Code : ###################################################### ## DATA COLLECTION & PLANNING (ALL CLUSTER DEVICES) ## ###################################################### ## Capture Product Code & Serial Number tmsh show sys hardware ## Capture Management IP & Blade State tmsh show sys cluster ## Capture Provision State tmsh list sys provision ##Capture Release and Volume Info tmsh show sys software ## Capture Master-key tmsh show sys crypto ## Check Relicensing Needed tmsh show sys license | grep -i 'service check date' REF - https://support.f5.com/csp/article/K7727 ##Check Certificate Expiration openssl x509 -noout -text -in /config/httpd/conf/ssl.crt/server.crt | grep Validity -A2 REF - https://support.f5.com/csp/article/K6353 ##Check RAID Integrity tmsh show sys raid tmsh run util platform_check cat /var/log/user.log cat /var/log/kern.log ##Check Mirroring Enabled tmsh show sys connection type mirror tmsh show sys ha-mirror ## Check Upgrade Disk Space (At least 20Gb) vgs ## Check ZebOS Module Running vtysh zebos/rdX/ZebOS.conf >> 'X' REPRESENTS ROUTE DOMAIN ID ## ONLY GTM/DNS - Check Devices Managed by GTM tmsh show gtm iquery all ## ONLY GTM/DNS - Check if DNSSEC keys in FIPS are Synchronized tmsh show sys crypto fips ## Capture QKView (Upload to iHealth) qkview REF - https://ihealth.f5.com/qkview-analyzer/ ##Check Release Notes For Specific Details REF - https://support.f5.com/csp/knowledge-center/software/BIG-IP ## Upload Release Image scp -p / @ :/shared/images/ ## Upload MD5 Hash Image scp -p / @ :/shared/images/ ## Upload Script to Check Pool Status scp -p /Check_Pool_Status.sh @ :/shared/tmp/ REF - https://github.com/DariuSGB/F5_Bash/blob/master/Check_Pool_Status.sh ############################################# ## PRE-UPGRADE TASKS (ALL CLUSTER DEVICES) ## ############################################# ##Disable Virtual Server Mirroring REF - https://support.f5.com/csp/article/K13478 ## Disable Config Auto-Sync (if enabled) tmsh modify cm device-group auto-sync disabled ## ONLY GTM/DNS - Disable GSLB/ZoneRunner Synchronization tmsh modify gtm global-settings general { synchronization no synchronize-zone-files no auto-discovery no } ## Save Running Config tmsh save sys config ##Check HA Cluster Synchronization tmsh show cm sync-status tmsh run cm config-sync to-group ## Check Release Image Integrity cd /shared/images/ md5sum -c ##Create Initial UCS (Backup) tmsh save sys ucs /shared/tmp/$(date '+%Y%m%d')_initial.ucs ## Capture Initial Config tmsh save sys config file /shared/tmp/$(date '+%Y%m%d')_initial.scf no-passphrase ## Capture Initial Pool Status /shared/tmp/Check_Pool_Status.sh > /shared/tmp/$(date '+%Y%m%d')_initial_pools_output.txt ## Check No Upgrade Process Running tmsh show sys software status ## OPTIONAL - Get More Free Disk Space (At least 20Gb) tmsh delete sys software volume vgs ######################################## ## UPGRADE TASKS (ONE DEVICE AT TIME) ## ######################################## ## Restart AOM to Prevent Licensing Problems (iSeries) ipmiutil reset -k REF - https://support.f5.com/csp/article/K00415052 ## ONLY VCMP HOST - Check That All Guests Are In Standby tmsh show vcmp guest >> ACCESS INDIVIDUALLY TO EACH GUEST tmsh show cm sync-status ## ONLY VCMP HOST - Deprovision All Guests (Configured) tmsh show vcmp guest >> EXECUTE FOR EACH GUEST tmsh modify vcmp guest state configured tmsh save sys config ## Re-licensing Device >> BIG-IP WITH INTERNET ACCESS tmsh install sys license registration-key add-on-keys { } REF - https://support.f5.com/csp/article/K15055 >> BIG-IP WITHOUT INTERNET ACCESS cp /config/bigip.license /config/bigip.license.backup get_dossier -b -a ** ACCESS LICENSE ACTIVATION https://activate.f5.com/license/dossier.jsp ** PASTE LICENSE FILE (ENTER 'CTRL+D' AFTER PASTING) cat > /config/bigip.license reloadlic REF - https://support.f5.com/csp/article/K2595 ## Force Offline Mode tmsh run sys failover offline ## Verify Configuration Integrity tmsh load sys config verify ## Install Image tmsh install sys software image create-volume volume ## Check Installation State tmsh show sys software status cat /var/log/liveinstall.log ## OPTIONAL - Copy Configuration To New Volume ## (Only if you have made changes since installation) clsh --slot=X,Y cpcfg >> FROM VIPRION cpcfg >> FROM NOT VIPRION ## Boot On New Volume tmsh reboot volume ## ONLY VCMP GUEST - Check Boot Up Status >> FROM VCMP HOST vconsole ## Check Logs (LTM, APM, ASM,...) REF - https://support.f5.com/csp/article/K16197 ## Capture Final Config tmsh save sys config file /shared/tmp/$(date '+%Y%m%d')_final.scf no-passphrase ## Compare Initial-Final Config tmsh show sys config-diff /shared/tmp/$(date '+%Y%m%d')_initial.scf /shared/tmp/$(date '+%Y%m%d')_final.scf | egrep -e "\s{3}\|\s{3}" -e "[<]$" -e "^\s*[>]" ## Disable Force Offline tmsh run sys failover online ## ONLY GTM/DNS - Enable Metrics Collection tmsh start sys service big3d ## Capture Final Pool Status /shared/tmp/Check_Pool_Status.sh > /shared/tmp/$(date '+%Y%m%d')_final_pools_output.txt ## Compare Initial-Final Pool Status diff /shared/tmp/$(date '+%Y%m%d')_initial_pools_output.txt /shared/tmp/$(date '+%Y%m%d')_final_pools_output.txt ## ONLY VCMP HOST - Deploy All Guests (Deployed) tmsh show vcmp guest tmsh modify vcmp guest state deployed ## FROM ACTIVE NODE - Check Current Connections tmsh show sys traffic raw ## FROM ACTIVE NODE - Force Failover Event tmsh run sys failover standby ## Check CPU/Memory status tmsh show sys cpu tmsh show sys memory ## Check Current Connections tmsh show sys traffic raw ##Perfom Other Custom Tests Here ... ############################################## ## POST-UPGRADE TASKS (ALL CLUSTER DEVICES) ## ############################################## ## OPTIONAL - Install Big3d daemon in all managed members ## (Only necessary if you upgrade GTM/DNS before its members) big3d_install REF - https://support.f5.com/csp/article/K11661449#update-big3d ## ONLY GTM/DNS - Enable GSLB/ZoneRunner Synchronization tmsh modify gtm global-settings general { synchronization yes synchronize-zone-files yes auto-discovery yes } ## Re-enable Virtual Server Mirroring REF - https://support.f5.com/csp/article/K13478 ## Synchronize HA Cluster tmsh show cm sync-status tmsh run cm config-sync force-full-load-push to-group ## Re-enable Config Auto-Sync (if enabled) tmsh modify cm device-group auto-sync enabled ## Save running config tmsh save sys config ## Create Final UCS (Backup) tmsh save sys ucs /shared/tmp/$(date '+%Y%m%d')_final.ucs ##Delete Unused Images delete sys software image ## Delete Unused Volumes (Mandatory reboot) delete sys software volume Tested this on version: 12.1APM Configuration to Support Duo MFA using iRule
Overview BIG-IP APM has supported Duo as an MFA provider for a long time with RADIUS-based integration. Recently, Duo has added support for Universal Prompt that uses Open ID Connect (OIDC) protocol to provide two-factor authentication. To integrate APM as an OIDC client and resource server, and Duo as an Identity Provider (IdP), Duo requires the user’s logon name and custom parameters to be sent for Authentication and Token request. This guide describes the configuration required on APM to enable Duo MFA integration using an iRule. iRules addresses the custom parameter challenges by generating the needed custom values and saving them in session variables, which the OAuth Client agent then uses to perform MFA with Duo. This integration procedure is supported on BIG-IP versions 13.1, 14.1x, 15.1x, and 16.x. To integrate Duo MFA with APM, complete the following tasks: 1. Choose deployment type: Per-request or Per-session 2. Configure credentials and policies for MFA on the DUO web portal 3. Create OAuth objects on the BIG-IP system 4. Configure the iRule 5. Create the appropriate access policy/policies on the BIG-IP system 6. Apply policy/policies and iRule to the APM virtual server Choose deployment type APM supports two different types of policies for performing authentication functions. Per-session policies: Per-session policies provide authentication and authorization functions that occur only at the beginning of a user’s session. These policies are compatible with most APM use cases such as VPN, Webtop portal, Remote Desktop, federation IdP, etc. Per-request policies: Per-request policies provide dynamic authentication and authorization functionality that may occur at any time during a user’s session, such as step-up authentication or auditing functions only for certain resources. These policies are only compatible with Identity Aware Proxy and Web Access Management use cases and cannot be used with VPN or webtop portals. This guide contains information about setting up both policy types. Prerequisites Ensure the BIG-IP system has DNS and internet connectivity to contact Duo directly for validating the user's OAuth tokens. Configure credentials and policies for MFA on Duo web portal Before you can protect your F5 BIG-IP APM Web application with Duo, you will first need to sign up for a Duo account. 1. Log in to the Duo Admin Panel and navigate to Applications. 2. Click Protect an application. Figure 1: Duo Admin Panel – Protect an Application 3. Locate the entry for F5 BIG-IP APM Web in the applications list and click Protect to get the Client ID, Client secret, and API hostname. You will need this information to configure objects on APM. Figure 2: Duo Admin Panel – F5 BIG-IP APM Web 4. As DUO is used as a secondary authentication factor, the user’s logon name is sent along with the authentication request. Depending on your security policy, you may want to pre-provision users in Duo, or you may allow them to self-provision to set their preferred authentication type when they first log on. To add users to the Duo system, navigate to the Dashboard page and click the Add New...-> Add User button. A Duo username should match the user's primary authentication username. Refer to the https://duo.com/docs/enrolling-users link for the different methods of user enrollment. Refer to Duo Universal Prompt for additional information on Duo’s two-factor authentication. Create OAuth objects on the BIG-IP system Create a JSON web key When APM is configured to act as an OAuth client or resource server, it uses JSON web keys (JWKs) to validate the JSON web tokens it receives from Duo. To create a JSON web key: 1. On the Main tab, select Access > Federation > JSON Web Token > Key Configuration. The Key Configuration screen opens. 2. To add a new key configuration, click Create. 3. In the ID and Shared Secret fields, enter the Client ID and Client Secret values respectively obtained from Duo when protecting the application. 4. In the Type list, select the cryptographic algorithm used to sign the JSON web key. Figure 3: Key Configuration screen 5. Click Save. Create a JSON web token As an OAuth client or resource server, APM validates the JSON web tokens (JWT) it receives from Duo. To create a JSON web token: 1. On the Main tab, select Access > Federation > JSON Web Token > Token Configuration. The Token Configuration screen opens. 2. To add a new token configuration, click Create. 3. In the Issuer field, enter the API hostname value obtained from Duo when protecting the application. 4. In the Signing Algorithms area, select from the Available list and populate the Allowed and Blocked lists. 5. In the Keys (JWK) area, select the previously configured JSON web key in the allowed list of keys. Figure 4: Token Configuration screen 6. Click Save. Configure Duo as an OAuth provider APM uses the OAuth provider settings to get URIs on the external OAuth authorization server for JWT web tokens. To configure an OAuth provider: 1. On the Main tab, select Access > Federation > OAuth Client / Resource Server > Provider. The Provider screen opens. 2. To add a provider, click Create. 3. In the Name field, type a name for the provider. 4. From the Type list, select Custom. 5. For Token Configuration (JWT), select a configuration from the list. 6. In the Authentication URI field, type the URI on the provider where APM should redirect the user for authentication. The hostname is the same as the API hostname in the Duo application. 7. In the Token URI field, type the URI on the provider where APM can get a token. The hostname is the same as the API hostname in the Duo application. Figure 5: OAuth Provider screen 8. Click Finished. Configure Duo server for APM The OAuth Server settings specify the OAuth provider and role that Access Policy Manager (APM) plays with that provider. It also sets the Client ID, Client Secret, and Client’s SSL certificates that APM uses to communicate with the provider. To configure a Duo server: 1. On the Main tab, select Access > Federation > OAuth Client / Resource Server > OAuth Server. The OAuth Server screen opens. 2. To add a server, click Create. 3. In the Name field, type a name for the Duo server. 4. From the Mode list, select how you want the APM to be configured. 5. From the Type list, select Custom. 6. From the OAuth Provider list, select the Duo provider. 7. From the DNS Resolver list, select a DNS resolver (or click the plus (+) icon, create a DNS resolver, and then select it). 8. In the Token Validation Interval field, type a number. In a per-request policy subroutine configured to validate the token, the subroutine repeats at this interval or the expiry time of the access token, whichever is shorter. 9. In the Client Settings area, paste the Client ID and Client secret you obtained from Duo when protecting the application. 10. From the Client's ServerSSL Profile Name, select a server SSL profile. Figure 6: OAuth Server screen 11. Click Finished. Configure an auth-redirect-request and a token-request Requests specify the HTTP method, parameters, and headers to use for the specific type of request. An auth-redirect-request tells Duo where to redirect the end-user, and a token-request accesses the authorization server for obtaining an access token. To configure an auth-redirect-request: 1. On the Main tab, select Access > Federation > OAuth Client / Resource Server > Request. The Request screen opens. 2. To add a request, click Create. 3. In the Name field, type a name for the request. 4. For the HTTP Method, select GET. 5. For the Type, select auth-redirect-request. 6. As shown in Figure 7, specify the list of GET parameters to be sent: request parameter with value depending on the type of policy For per-request policy: %{subsession.custom.jwt_duo} For per-session policy: %{session.custom.jwt_duo} client_id parameter with type client-id response_type parameter with type response-type Figure 7: Request screen with auth-redirect-request (Use “subsession.custom…” for Per-request or “session.custom…” for Per-session) 7. Click Finished. To configure a token-request: 1. On the Main tab, select Access > Federation > OAuth Client / Resource Server > Request. The Request screen opens. 2. To add a request, click Create. 3. In the Name field, type a name for the request. 4. For the HTTP Method, select POST. 5. For the Type, select token-request. 6. As shown in Figure 8, specify the list of POST parameters to be sent: client_assertion parameter with value depending on the type of policy For per-request policy: %{subsession.custom.jwt_duo_token} For per-session policy: %{session.custom.jwt_duo_token} client_assertion_type parameter with value urn:ietf:params:oauth:client-assertion-type:jwt-bearer grant_type parameter with type grant-type redirect_uri parameter with type redirect-uri Figure 8: Request screen with token-request (Use “subsession.custom…” for Per-request or “session.custom…” for Per-session) 7. Click Finished. Configure the iRule iRules gives you the ability to customize and manage your network traffic. Configure an iRule that creates the required sub-session variables and usernames for Duo integration. Note: This iRule has sections for both per-request and per-session policies and can be used for either type of deployment. To configure an iRule: 1. On the Main tab, click Local Traffic > iRules. 2. To create an iRules, click Create. 3. In the Name field, type a name for the iRule. 4. Copy the sample code given below and paste it in the Definition field. Replace the following variables with values specific to the Duo application: <Duo Client ID> in the getClientId function with Duo Application ID. <Duo API Hostname> in the createJwtToken function with API Hostname. For example, https://api-duohostname.com/oauth/v1/token. <JSON Web Key> in the getJwkName function with the configured JSON web key. Note: The iRule ID here is set as JWT_CREATE. You can rename the ID as desired. You specify this ID in the iRule Event agent in Visual Policy Editor. Note: The variables used in the below example are global, which may affect your performance. Refer to the K95240202: Understanding iRule variable scope article for further information on global variables, and determine if you use a local variable for your implementation. proc randAZazStr {len} { return [subst [string repeat {[format %c [expr {int(rand() * 26) + (rand() > .5 ? 97 : 65)}]]} $len]] } proc getClientId { return <Duo Client ID> } proc getExpiryTime { set exp [clock seconds] set exp [expr $exp + 900] return $exp } proc getJwtHeader { return "{\"alg\":\"HS512\",\"typ\":\"JWT\"}" } proc getJwkName { return <JSON Web Key> #e.g. return "/Common/duo_jwk" } proc createJwt {duo_uname} { set header [call getJwtHeader] set exp [call getExpiryTime] set client_id [call getClientId] set redirect_uri "https://" set redirect [ACCESS::session data get "session.server.network.name"] append redirect_uri $redirect append redirect_uri "/oauth/client/redirect" set payload "{\"response_type\": \"code\",\"scope\":\"openid\",\"exp\":${exp},\"client_id\":\"${client_id}\",\"redirect_uri\":\"${redirect_uri}\",\"duo_uname\":\"${duo_uname}\"}" set jwt_duo [ ACCESS::oauth sign -header $header -payload $payload -alg HS512 -key [call getJwkName] ] return $jwt_duo } proc createJwtToken { set header [call getJwtHeader] set exp [call getExpiryTime] set client_id [call getClientId] set aud "<Duo API Hostname>/oauth/v1/token" #Example: set aud https://api-duohostname.com/oauth/v1/token set jti [call randAZazStr 32] set payload "{\"sub\": \"${client_id}\",\"iss\":\"${client_id}\",\"aud\":\"${aud}\",\"exp\":${exp},\"jti\":\"${jti}\"}" set jwt_duo [ ACCESS::oauth sign -header $header -payload $payload -alg HS512 -key [call getJwkName] ] return $jwt_duo } when ACCESS_POLICY_AGENT_EVENT { set irname [ACCESS::policy agent_id] if { $irname eq "JWT_CREATE" } { set ::duo_uname [ACCESS::session data get "session.logon.last.username"] ACCESS::session data set session.custom.jwt_duo [call createJwt $::duo_uname] ACCESS::session data set session.custom.jwt_duo_token [call createJwtToken] } } when ACCESS_PER_REQUEST_AGENT_EVENT { set irname [ACCESS::perflow get perflow.irule_agent_id] if { $irname eq "JWT_CREATE" } { set ::duo_uname [ACCESS::session data get "session.logon.last.username"] ACCESS::perflow set perflow.custom [call createJwt $::duo_uname] ACCESS::perflow set perflow.scratchpad [call createJwtToken] } } Figure 9: iRule screen 5. Click Finished. Create the appropriate access policy/policies on the BIG-IP system Per-request policy Skip this section for a per-session type deployment The per-request policy is used to perform secondary authentication with Duo. Configure the access policies through the access menu, using the Visual Policy Editor. The per-request access policy must have a subroutine with an iRule Event, Variable Assign, and an OAuth Client agent that requests authorization and tokens from an OAuth server. You may use other per-request policy items such as URL branching or Client Type to call Duo only for certain target URIs. Figure 10 shows a subroutine named duosubroutine in the per-request policy that handles Duo MFA authentication. Figure 10: Per-request policy in Visual Policy Editor Configuring the iRule Event agent The iRule Event agent specifies the iRule ID to be executed for Duo integration. In the ID field, type the iRule ID as configured in the iRule. Figure 11: iRule Event agent in Visual Policy Editor Configuring the Variable Assign agent The Variable Assign agent specifies the variables for token and redirect requests and assigns a value for Duo MFA in a subroutine. This is required only for per-request type deployment. Add sub-session variables as custom variables and assign their custom Tcl expressions as shown in Figure 12. subsession.custom.jwt_duo_token = return [mcget {perflow.scratchpad}] subsession.custom.jwt_duo = return [mcget {perflow.custom}] Figure 12: Variable Assign agent in Visual Policy Editor Configuring the OAuth Client agent An OAuth Client agent requests authorization and tokens from the Duo server. Specify OAuth parameters as shown in Figure 13. In the Server list, select the Duo server to which the OAuth client directs requests. In the Authentication Redirect Request list, select the auth-redirect-request configured earlier. In the Token Request list, select the token-request configured earlier. Some deployments may not need the additional information provided by OpenID Connect. You could, in that case, disable it. Figure 13: OAuth Client agent in Visual Policy Editor Per-session policy Configure the Per Session policy as appropriate for your chosen deployment type. Per-request: The per-session policy must contain at least one logon page to set the username variable in the user’s session. Preferably it should also perform some type of primary authentication. This validated username is used later in the per-request policy. Per-session: The per-session policy is used for all authentication. A per-request policy is not used. Figures 14a and 14b show a per-session policy that runs when a client initiates a session. Depending on the actions you include in the access policy, it can authenticate the user and perform actions that populate session variables with data for use throughout the session. Figure 14a: Per-session policy in Visual Policy Editor performs both primary authentication and Duo authentication (for per-session use case) Figure 14b: Per-session policy in Visual Policy Editor performs primary authentication only (for per-request use case) Apply policy/policies and iRule to the APM virtual server Finally, apply the per-request policy, per-session policy, and iRule to the APM virtual server. You assign iRules as a resource to the virtual server that users connect. Configure the virtual server’s default pool to the protected local web resource. Apply policy/policies to the virtual server Per-request policy To attach policies to the virtual server: 1. On the Main tab, click Local Traffic > Virtual Servers. 2. Select the Virtual Server. 3. In the Access Policy section, select the policy you created. 4. Click Finished. Figure 15: Access Policy section in Virtual Server (per-request policy) Per-session policy Figure 16 shows the Access Policy section in Virtual Server when the per-session policy is deployed. Figure 16: Access Policy section in Virtual Server (per-session policy) Apply iRule to the virtual server To attach the iRule to the virtual server: 1. On the Main tab, click Local Traffic > Virtual Servers. 2. Select the Virtual Server. 3. Select the Resources tab. 4. Click Manage in the iRules section. 5. Select an iRule from the Available list and add it to the Enabled list. 6. Click Finished.Client SSL Authentication on BIG-IP as in-depth as it can go
Quick Intro In this article, I'm going to explain how SSL client certificate authentication works on BIG-IP and explain what actually happens during client authentication as in-depth as I can, showing the TLS headers on Wireshark. This article is about the client side of BIG-IP (Client SL profile) authenticating a client connecting to BIG-IP. The Topology For reference so we can follow Wireshark output: How to Configure Client Certificate Authentication on Client SSL profile Essentially, what we're doing here is making BIG-IP verify client's credentials before allowing the TLS handshake to proceed. However, such credentials are in the form of a client certificate. The way to do this is to configure BIG-IP by: Adding a CA file to Trusted Certificate Authorities (ca-file in tmsh) to validate client certificate Optionally adding same CA that signed client certificate to Advertised Certificate Authorities Enforcing Client Certificate validation by setting Client Certificate option on BIG-IP to require Optionally setting the Frequency of such checks if we don't want to stick to the defaults. I'll go through each option now. Adding CA file to Trusted Certificate Authorities We should add to Trusted Certificate Authorities a single certificate file (*.crt) with one CA or concatenated file with 2 or more CAs with the purpose of validating client certificate, i.e. confirming client's identity. Upon receiving client certificate, BIG-IP will go through this list of CAs and confirm client's identity. It also has another purpose which is to authenticate BIG-IP to client but it's out of the scope of this article. Optionally add CA file to Advertised Certificate Authorities Trusted Certificate Authoritiesexplicitly tells BIG-IP the CA or chain of CAs it will use to validate client certificate whereasAdvertised Certificate Authorities tells client in advance what kind of CA BIG-IP trusts so that client can make the decision about which certificate to send to BIG-IP: Why would we use Advertised Certificate Authorities? Let's imagine a situation where Client's application has more than one Client Certificate configured. How is it going to figure out which certificate to send to BIG-IP? That's where Advertised Certificate Authorities comes to rescue us! When we add our CA bundle to Advertised Certificate Authorities, we're telling BIG-IP to add it to a header field named Distinguished Names within Certificate Request message. I dedicated the Appendix section to show you more in-depth how changing Advertised Certificate Authority affects Certificate Request header. Configuring BIG-IP to enforce Client Certificate validation To enable client certificate authentication on BIG-IP we change Local Traffic››Profiles : SSL : Client›› Client Certificate to request: The default is set to ignore where client certificate authentication is disabled. If we truly want to enable client certificate validation we need to select require. The reason why is because request makes BIG-IP request a client certificate from the client but BIG-IP will not perform the validation to confirm if the certificate sent is valid in this mode. The following subsections explain each option. Ignore Client Certificate Authentication is disabled (the default). BIG-IP never sends Certificate Request to client and therefore client does not need to send its certificate to BIG-IP. In this case, TLS handshake proceeds successfully without any client authentication: pcap:ssl-sample-peer-cert-mode-ignore.pcap Wireshark filter used:frame.number == 5 or frame.number == 6 Request BIG-IP requests Client Certificate by sending Certificate Request message but does not check whether client certificate is valid which is not really client authentication, is it? This means that ca-file(Trusted Certificate Authorities in the GUI) will not be used to validate client certificate and we will consider any certificate sent to us to be valid. For example, inssl-sample-peer-cert-mode-request-with-no-client-cert-sent.pcapwe now see that BIG-IP sends aCertificate Requestmessage and client responds with aCertificatemessage this time: Because I didn't add any client certificate to my browser, it sent a blank certificate to BIG-IP. Again, BIG-IP did not perform any validation whatsoever, so TLS handshake proceeded successfully. We can then conclude that this setting only makes BIG-IP request client certificate and that's it. Require It behaves just like Request but BIG-IP also performs client certificate validation, i.e. BIG-IP will use CA we hopefully added to ca-file (Trusted Certificate Authorities in the GUI) to confirm if client certificate is valid. This means that if we don't add a CA to Trusted Certificate Authorities (ca-file in tmsh) then validation will fail. Setting the Frequency of Client Certificate Requests This setting specifies the frequency BIG-IP authenticates client by enabling/disabling TLS session resumption. It has only 2 options: once BIG-IP requests client certificate during first handshake and no longer re-authenticates client as long as TLS session is reused and valid. The way BIG-IP does it is by using Session Resumption/Reuse. During first TLS handshake from client, BIG-IP sends a Session ID to Client within Server Hello header and in subsequent TLS connections, assuming session ID is still in BIG-IP's cache and client re-sends it back to BIG-IP, then session will be resumed every time client tries to establish a TLS session (respecting cache timeout). First time client sends Client Hello with blank session ID as it's cache is empty and then it is assigned a Session ID by BIG-IP (409f...): pcap:ssl-sample-clientcert-auth-once-enabled.pcap Wireshark filter used:filter:!ip.addr == 172.16.199.254 and frame.number > 1 and frame.number < 7 Certificate Requestconfirms BIG-IP is trying to authenticate client. Notice Session ID BIG-IP sent to client is 409f7... Then when Client tries to go through another TLS handshake and sends above session ID in Client Hello (packet #70 below). BIG-IP then confirms session is being resumed by sending samesession IDin Server Hello back to client: Wireshark filter used:!ip.addr == 172.16.199.254 and frame.number > 66 and frame.number < 73 This resumed TLS handshake just means we will not go through full handshake and will no longer need to exchange keys, select ciphers or re-authenticate client as we're reusing those already negotiated in the full TLS handshake where we first received Session ID 409f... always BIG-IP requests client certificate, i.e. re-authenticates client at every handshake. On BIG-IP, this is accomplished by disabling session reuse which makes BIG-IP not to sendSession IDback to Client in the beginning and forcing a full TLS handshake every time. pcap:ssl-sample-clientcert-auth-always-enabled.pcap Wireshark filter used: !ip.addr == 172.16.199.254 and ((frame.number > 1 and frame.number < 7) or (frame.number > 74 and frame.number < 80)) Appendix: Understanding how Advertised Certificate Authority field affects Certificate Request header For this test, I've got the following: myCAbundle.crt: concatenation of root_ca.crt and ltm2.crt (signed by root_ca.crt) client_cert.crt: added to Firefox and signed by ltm2.crt I've also added myCAbundle.crt to Trusted Certificate Authorities so BIG-IP is able to verify that client_cert.crt is valid. For each test, I will use change Advertised Certificate Authorities so we can see what happens. We'll go through 3 tests here: Setting Advertised Certificate Authority to None Setting Advertised Certificate Authority to a certificate that didn't sign client cert Setting Advertised Certificate Authority to a bundle that signed client cert Setting Advertised Certificate Authority to None Note that even though no CA was advertised in Certificate Request message, BIG-IP still advertises Certificate typesandSignature Hash Algorithmsso that client knows in advance what kind of certificate (RSA, DSS or ECDSA) and hash algorithms BIG-IP supports in advance. If client certificate had not been signed using any of the certificate types and hashing algorithms listed then handshake would have failed. However, in this case validation is successful as we can see on frame #8 that client certificate is RSA type and hashed with SHA1: pcap: ssl-sample-advcert-none.pcap filter used: !ip.addr == 172.16.199.254 and frame.number > 1 and frame.number < 16 It's worth noting that Distinguished Names is NOT populated and has length of zero because we didn't attach a bundle to Advertised Certificate Authorities. In this case, it worked fine because my client browser had only one certificate attached, so it shouldn't cause a problem anyway. Setting Advertised Certificate Authority to a certificate that didn't sign client cert pcap: ssl-sample-advcert-default-firefox.pcap Wireshark filter used: None I've set Advertised Certificate Authority to default.crt as this is NOT the CA that signed client's certificate: The difference here when compared to None is that now we can see that Distinguished Names is now populated with the certificate I added (default.crt): However, even though I added the correct certificate to my Firefox browser, it sent a blank certificate instead. Why? That's because BIG-IP signalled in Distinguished Names that default.crt is the CA that signed the certificate BIG-IP is looking for and as Firefox doesn't have any certificate signed by default.crt, it just sent a blank certificate back to BIG-IP. Also, because BIG-IP is now performing proper validation, i.e. comparing whatever client certificate is sent to it with the CA list added to Trusted Certificate Authorities, it knows a blank certificate is not valid and terminates the TLS handshake with a Fatal Alert. Setting Advertised Certificate Authority to a bundle that signed client cert pcap: ssl-sample-advcert-ltm2chainedwithrootca.pcap filter used: !ip.addr == 172.16.199.254 and frame.number > 1 Now I've set Advertised Certificate Authority to the correct bundle that signed my client certificate: And indeed handshake succeeds because: BIG-IP advertises myCAbundle.crt in Certificate Request >> Distinguished Names header, as per Advertised Certificate Authority configuration By reading Distinguished Names field, Client correctly sends the correct client certificate back to BIG-IP BIG-IP correctly validates client certificate using myCAbundle configured on Trusted Certificate Authorities Hope this article provides some clarification about these mysterious TLS headers.Happy 20th Birthday, BIG-IP TMOS!
I wasn’t in the waiting room with the F5 family, ears and eyes perked for the release announcement of BIG-IP version 9.0. I was a customer back in 2004, working on a government contract at Scott AFB, Illinois. I shared ownership of the F5 infrastructure, pairs of BIG-IPs running version 4.5 on Dell PowerEdge 2250 servers with one other guy. But maybe a month or two before the official first release of TMOS, my F5 account manager dropped off some shiny new hardware. And it was legit purpose-built and snazzy, not some garage-style hacked Frankenstein of COTS parts like the earlier stuff. And you wonder why we chose Dell servers! Anyway, I was a hard-core network engineer at this time, with very little exposure to anything above layer four, and even there, my understanding was limited to ports and ACLs and maybe a little high-level clarity around transport protocols. But application protocols? Nah. No idea. So with this new hardware and an entirely new full-proxy architecture (what’s a proxy, again?) I was overwhelmed. And honestly, I was frustrated with it for the first few days because I didn’t know what I didn’t know and so I struggled to figure out what to do with it, even to replicate my half-proxy configuration in the “new way”. But I’m a curious person. Given enough time and caffeine, I can usually get to the bottom of a problem, at least well enough to arrive at a workable solution. And so I did. My typical approach to anything is to make it work, make it work better, make it work reliably better, then finally make it work reliably and more performantly better. And the beauty here with this new TMOS system is that I was armed with a treasure trove of new toys. The short list I dug into during my beta trial, which lasted for a couple of weeks: The concept of a profile. When you support a few applications, this is no big deal. When you support hundreds, being able to macro configuration snippets within your application and across applications was revolutionary. Not just for the final solution, but also for setting up and executing your test plans. iRules. Yes, technically they existed in 4.x, but they were very limited in scope. With TMOS, F5 introduced the Tcl-based and F5 extended live-traffic scripting environment that unleashed tremendous power and flexibility for network and application teams. I dabbled with this, and thought I understood exactly how useful this was. More on this a little later. A host operating system. I was a router, switch, and firewall guy. Nothing I worked on had this capability. I mean, a linux system built in to my networking device? YES!!! Two things I never knew I always needed during my trial: 1) tcpdump ON BOX. Seriously--mind blown; and 2) perl scripting against config and snmp. Yeah, I know, I laugh about perl now. But 20 years ago, it was the cats pajamas. A fortunate job change Shortly after my trial was over, I interviewed for an accepted a job offer from a major rental car company that was looking to hire an engineer to redesign their application load balancing infrastructure and select the next gear purchase for the effort. We evaluated Cisco, Nortel/Alteon, Radware, and F5 on my recommendation. With our team’s resident architect we drafted the rubric with which we’d evaluate all the products, and whereas there were some layer two performance issues in some packet sizes that were arguably less than real-world, the BIG-IP blew away the competitors across the board. Particularly, though, in configurability and instrumentation. Tcpdump on box was such a game-changer for us. Did we have issues with TMOS version 9? For sure. My first year with TMOS was also TMOS's first year. Bugs are going to happen with any release, but a brand new thing is guaranteed. But F5 support was awesome, and we worked through all the issues in due time. Anyway, I want to share three wins in my first year with TMOS. Win #1 Our first production rollout was in the internet space, on BIG-IP version 9.0.5. That’s right, a .0 release. TMOS was a brand new baby, and we had great confidence throughout our testing. During our maintenance, once we flipped over the BIG-IPs, our rental transaction monitors all turned red and the scripted rental process had increased by 50%! Not good. “What is this F5 stuff? Send it back!!” But it was new, and we knew we had a gem here. We took packet captures on box, of course, then rolled back and took more packet captures, this time through taps because our old stuff didn’t have tcpdump on box. This is where Jason started to really learn about the implications of both a full proxy architecture and the TCP protocol. It turns our our application servers had a highly-tuned TCP stack on them specific to the characteristics of the rental application. We didn’t know this, of course. But since we implemented a proxy that terminates clients at the BIG-IP and starts a new session to the servers, all those customizations for WAN traffic were lost. Once we built a TCP profile specifically for the rental application servers and tested it under WAN emulation, we not only reached parity with the prior performance but beat it by 10%. Huzzah! Go BIG-IP custom protocol stack configuration! Win #2 For the next internal project, I had to rearchitect the terminal server farm. We had over 700 servers in two datacenters supporting over 60,000 thin clients around the world for rental terminals. Any failures meant paper tickets and unhappy staff and customers. One thing that was problematic with the existing solution is that sometimes clients would detach and upon reconnect would connect directly to the server, which skewed the load balancers view of the world and frequently overloaded some servers to the point all sessions on that server would hang until metrics (but usually angry staff) would notify. Remember my iRules comment earlier on differentiators? Well, iRules architect David Hansen happened to be a community hero and was very helpful to me in the DevCentral forums and really opened my eyes to the art of possible with iRules. He was able to take the RDP session token that was being returned by the client, read it, translate it from its Microsoft encoding format, and then forward the session on to the correct server in the backend so that all sessions continued to be accounted for in our load balancing tier. This was formative for me as a technologist and as a member of the DevCentral community. Win #3 2004-2005 was the era before security patching was as visible a responsibility as it is today, but even then we had a process and concerns when there were obstacles. We had an internal application that had a plugin for the web tier that managed all the sessions to the app tier, and this plugin was no longer supported. We were almost a year behind on system and application patches because we had no replacement for this. Enter, again, iRules.I was able to rebuild the logic of the plugin in an iRule that IIRCwasn’tmore than 30 lines. So the benefits ended up not only being a solution to that problem, but the ability to remove that web tier altogether, saving on equipment, power, and complexity costs. And that was just the beginning... TMOS was mature upon arrival, but it got better every year. iControl added REST-based API access; clustered multi-processing introduced tremendous performance gains; TMOS got virtualized, and all the home-lab technologists shouted with joy; a plugin architecture allowed for product modules like ASM and APM; solutions that began as iRules like AFM and SSLO became products. It’s crazy how much innovation has taken place on this platform! The introduction of TMOS didn’t just introduce me to applications and programmability. It did that and I’m grateful, but it did so much more. It unlocked in me that fanboy level that fans of sports teams, video game platforms, Taylor Swift, etc, experience. It helped me build an online community at DevCentral, long before I was an employee. Happy 20th Birthday, TMOS! We celebrate and salute you!Cipher Suite Practices and Pitfalls
Cipher Suite Practices and Pitfalls It seems like every time you turn around there is a new vulnerability to deal with, and some of them, such as Sweet32, have required altering cipher configurations for mitigation. Still other users may tweak their cipher suite settings to meet requirements for PCI compliance, regulatory issues, local compatibility needs, etc. However, once you start modifying your cipher suite settings you must take great care, as it is very easy to shoot yourself in the foot. Many misconfigurations will silently fail – seeming to achieve the intended result while opening up new, even worse, vulnerabilities. Let's take a look at cipher configuration on the F5 BIG-IP products to try stay on the safe path. What is a Cipher Suite? Before we talk about how they're configured, let's define exactly what we mean by 'cipher suite', how it differs from just a 'cipher', and the components of the suite. Wikipedia had a good summary, so rather than reinvent the wheel: A cipher suite is a named combination of authentication, encryption, message authentication code (MAC) and key exchange algorithms used to negotiate the security settings for a network connection using the Transport Layer Security (TLS) / Secure Sockets Layer (SSL) network protocol. When we talk about configuring ciphers on BIG-IP we're really talking about configuring cipher suites. More specifically the configured list of cipher suites is a menu of options available to be negotiated. Each cipher suite specifies the key exchange algorithm, authentication algorithm, cipher, cipher mode, and MAC that will be used. I recommend reading K15194: Overview of the BIG-IP SSL/TLS cipher suite for more information. But as a quick overview, let's look at a couple of example cipher suites. The cipher suite is in the format: Key Exchange-Authentication-Cipher-Cipher Mode-MAC Note that not all of these components may be explicitly present in the cipher suite, but they are still implicitly part of the suite. Let's consider this cipher suite: ECDHE-RSA-AES256-GCM-SHA384 This breaks down as follows: Key Exchange Algorithm: ECDHE (Elliptic Curve Diffie-Hellman Ephemeral) Authentication Algorithm: RSA Cipher: AES256 (aka AES with a 256-bit key) Cipher Mode: GCM (Galois/Counter Mode) MAC: SHA384 (aka SHA-2 (Secure Hash Algorithm 2) with 384-bit hash) This is arguably the strongest cipher suite we have on BIG-IP at this time. Let's compare that to a simpler cipher suite: AES128-SHA Key Exchange Algorithm: RSA (Implied) – When it isn't specified, presume RSA. Authentication Algorithm: RSA (Implied) – When it isn't specified, presume RSA. Cipher: AES128 (aka AES with a 128-bit key) Cipher Mode: CBC (Cipher Block Chaining) (Implied) – When it isn't specified, presume CBC. MAC: SHA1 (Secure Hash Algorithm 1; SHA-1 always produces a 160-bit hash.) This example illustrates that the cipher suite may not always explicitly specify every parameter, but they're still there. There are 'default' values that are fairly safe to presume when not otherwise specified. If an algorithm isn't specified, it is RSA. That's a safe bet. And if a cipher mode isn't specified it is CBC. Always CBC. Note that all ciphers currently supported on BIG-IP are CBC mode except for AES-GCM and RC4. ALL. I stress this as it has been a recurring source of confusion amongst customers. It isn't only the cipher suites which explicitly state 'CBC' in their name. Let's examine each of these components. This article is primarily about cipher suite configuration and ciphers, and not the SSL/TLS protocol, so I won't dive too deeply here, but I think it helps to have a basic understanding. Forgive me if I simplify a bit. Key Exchange Algorithms As a quick review of the difference between asymmetric key (aka public key) cryptography and symmetric key cryptography: With the asymmetric key you have two keys – K public and K private –which have a mathematical relationship. Since you can openly share the public key there is no need to pre-share keys with anyone. The downside is that these algorithms are computationally expensive. Key lengths for a common algorithm such as RSA are at least 1024-bit, and 2048-bit is really the minimally acceptable these days. Symmetric key has only K private . Both ends use the same key, which poses the problem of key distribution. The advantage is higher computational performance and common key sizes are 128-bit or 256-bit. SSL/TLS, of course, uses both public and private key systems – the Key Exchange Algorithm is the public key system used to exchange the symmetric key. Examples you'll see in cipher suites include ECDHE, DHE, RSA, ECDH, and ADH. Authentication Algorithms The Authentication Algorithm is sometimes grouped in with the Key Exchange Algorithm for configuration purposes; 'ECDHE_RSA' for example. But we'll consider it as a separate component. This is the algorithm used in the SSL/TLS handshake for the server to sign (using the server's private key) elements sent to the client in the negotiation. The client can authenticate them using the server's public key. Examples include: RSA, ECDSA, DSS (aka DSA), and Anonymous. Anonymous means no authentication; this is generally bad. The most common way users run into this is by accidentally enabling an 'ADH' cipher suite. More on this later when we talk about pitfalls. Note that when RSA is used for the key exchange, authentication is inherent to the scheme so there really isn't a separate authentication step. However, most tools will list it out for completeness. Cipher To borrow once again from Wikipedia: In cryptography, a cipher (or cypher) is an algorithm for performing encryption or decryption—a series of well-defined steps that can be followed as a procedure. An alternative, less common term is encipherment. To encipher or encode is to convert information into cipher or code. In common parlance, 'cipher' is synonymous with 'code', as they are both a set of steps that encrypt a message; however, the concepts are distinct in cryptography, especially classical cryptography. This is what most of us mean when we refer to 'configuring ciphers'. We're primarily interested in controlling the cipher used to protect our information through encryption. There are many, many examples of ciphers which you may be familiar with: DES (Data Encryption Standard), 3DES (Triple DES), AES (Advanced Encryption Standard), RC4 (Rivest Cipher 4), Camellia, RC6, RC2, Blowfish, Twofish, IDEA, SEED, GOST, Rijndael, Serpent, MARS, etc. For a little cipher humor, I recommend RFC2410: The NULL Encryption Algorithm and Its Use With IPsec. Roughly speaking, ciphers come in two types – block ciphers and stream ciphers. Block Ciphers Block ciphers operate on fixed-length chunks of data, or blocks. For example, DES operates on 64-bit blocks while AES operates on 128-bit blocks. Most of the ciphers you'll encounter are block ciphers. Examples: DES, 3DES, AES, Blowfish, Twofish, etc. Stream Ciphers Stream ciphers mathematically operate on each bit in the data flow individually. The most commonly encountered stream cipher is RC4, and that's deprecated. So we're generally focused on block ciphers, not that it really changes anything for the purposes of this article. All of the secrecy in encryption comes from the key that is used, not the cipher itself. Obtain the key and you can unlock the ciphertext. The cipher itself – the algorithm, source code, etc. – not only can be, but should be, openly available. History is full of examples of private cryptosystems failing due to weaknesses missed by their creators, while the most trusted ciphers were created via open processes (AES for example). Keys are of varying lengths and, generally speaking, the longer the key the more secure the encryption. DES only had 56-bits of key data, and thus is considered insecure. We label 3DES as 168-bit, but it is really only equivalent to 112-bit strength. (More on this later.) Newer ciphers, such as AES, often offer options – 128-bits, 192-bits, or 256-bits of key. Remember, a 256-bit key is far more than twice as strong as a 128-bit key. It is 2 128 vs. 2 256 - 3.4028237e+38 vs. 1.1579209e+77 Cipher Mode Cipher mode is the mode of operation used by the cipher when encrypting plaintext into ciphertext, or decrypting ciphertext into plaintext. The most common mode is CBC – Cipher Block Chaining. In cipher block chaining the ciphertext from block n feeds into the process for block n+1 – the blocks are chained together. To steal borrow an image from Wikipedia: As I mentioned previously, all ciphers on BIG-IP are CBC mode except for RC4 (the lone stream cipher, disabled by default starting in 11.6.0) and AES-GCM. AES-GCM was first introduced in 11.5.0, and it is only available for TLSv1.2 connections. GCM stands for Galois/Counter Mode, a more advanced mode of operation than CBC. In GCM the blocks are not chained together. GCM runs in an Authenticated Encryption with Associated Data (AEAD) mode which eliminates the separate per-message hashing step, therefore it can achieve higher performance than CBC mode on a given HW platform. It is also immune to classes of attack that have harried CBC, such as the numerous padding attacks (BEAST, Lucky 13, etc.) Via Wikipedia: The main drawback to AES-GCM is that it was only added in TLSv1.2, so any older clients which don't support TLSv1.2 cannot use it. There are other cipher suites officially supported in TLS which have other modes, but F5 does not currently support those ciphers so we won't get too deep into that. Other ciphers include AES-CCM (CTR mode with a CBC MAC; CTR is Counter Mode), CAMELLIA-GCM (CAMELLIA as introduced in 12.0.0 is CBC), and GOST CNT (aka CTR). We may see these in the future. MAC aka Hash Function What did we ever do before Wikipedia? A hash function is any function that can be used to map data of arbitrary size to data of fixed size. The values returned by a hash function are called hash values, hash codes, digests, or simply hashes. One use is a data structure called a hash table, widely used in computer software for rapid data lookup. Hash functions accelerate table or database lookup by detecting duplicated records in a large file. An example is finding similar stretches in DNA sequences. They are also useful in cryptography. A cryptographic hash function allows one to easily verify that some input data maps to a given hash value, but if the input data is unknown, it is deliberately difficult to reconstruct it (or equivalent alternatives) by knowing the stored hash value. This is used for assuring integrity of transmitted data, and is the building block for HMACs, which provide message authentication. In short, the MAC provides message integrity. Hash functions include MD5, SHA-1 (aka SHA), SHA-2 (aka SHA128, SHA256, & SHA384), and AEAD (Authenticated Encryption with Associated Data). MD5 has long since been rendered completely insecure and is deprecated. SHA-1 is now being 'shamed', if not blocked, by browsers as it is falling victim to advances in cryptographic attacks. While some may need to continue to support SHA-1 cipher suites for legacy clients, it is encouraged to migrate to SHA-2 as soon as possible – especially for digital certificates. Configuring Cipher Suites on BIG-IP Now that we've covered what cipher suites are, let's look at where we use them. There are two distinct and separate areas where cipher suites are used – the host, or control plane, and TMM, or the data plane. On the host side SSL/TLS is handled by OpenSSL and the configuration follows the standard OpenSSL configuration options. Control Plane The primary use of SSL/TLS on the control plane is for httpd. To see the currently configured cipher suite, use ' tmsh list sys http ssl-ciphersuite '. The defaults may vary depending on the version of TMOS. For example, these were the defaults in 12.0.0: tmsh list sys http ssl-ciphersuite sys httpd { ssl-ciphersuite DEFAULT:!aNULL:!eNULL:!LOW:!RC4:!MD5:!EXP } As of 12.1.2 these have been updated to a more explicit list: tmsh list sys http ssl-ciphersuite sys httpd { ssl-ciphersuite ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA384:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:AES128-SHA256:AES256-SHA256:ECDHE-RSA-DES-CBC3-SHA:ECDHE-ECDSA-DES-CBC3-SHA:DES-CBC3-SHA } You can change this configuration via ' tmsh modify sys http ssl-ciphersuite <value> '. One important thing to note is that the default is not just 'DEFAULT' as it is on the data plane. This is one thing that users have been caught by; thinking that setting the keyword to 'DEFAULT' will reset the configuration. As OpenSSL provides SSL/TLS support for the control plane, if you want to see which ciphers will actually be supported you can use ' openssl ciphers -v <cipherstring> '. For example: openssl ciphers -v 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA384:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:AES128-SHA256:AES256-SHA256:ECDHE-RSA-DES-CBC3-SHA:ECDHE-ECDSA-DES-CBC3-SHA:DES-CBC3-SHA' ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(256) Mac=AEAD ECDHE-RSA-AES128-SHA SSLv3 Kx=ECDH Au=RSA Enc=AES(128) Mac=SHA1 ECDHE-RSA-AES256-SHA SSLv3 Kx=ECDH Au=RSA Enc=AES(256) Mac=SHA1 ECDHE-RSA-AES128-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AES(128) Mac=SHA256 ECDHE-RSA-AES256-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AES(256) Mac=SHA384 ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AESGCM(128) Mac=AEAD ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AESGCM(256) Mac=AEAD ECDHE-ECDSA-AES128-SHA SSLv3 Kx=ECDH Au=ECDSA Enc=AES(128) Mac=SHA1 ECDHE-ECDSA-AES256-SHA SSLv3 Kx=ECDH Au=ECDSA Enc=AES(256) Mac=SHA1 ECDHE-ECDSA-AES128-SHA256 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AES(128) Mac=SHA256 ECDHE-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AES(256) Mac=SHA384 AES128-GCM-SHA256 TLSv1.2 Kx=RSA Au=RSA Enc=AESGCM(128) Mac=AEAD AES256-GCM-SHA384 TLSv1.2 Kx=RSA Au=RSA Enc=AESGCM(256) Mac=AEAD AES128-SHA SSLv3 Kx=RSA Au=RSA Enc=AES(128) Mac=SHA1 AES256-SHA SSLv3 Kx=RSA Au=RSA Enc=AES(256) Mac=SHA1 AES128-SHA256 TLSv1.2 Kx=RSA Au=RSA Enc=AES(128) Mac=SHA256 AES256-SHA256 TLSv1.2 Kx=RSA Au=RSA Enc=AES(256) Mac=SHA256 ECDHE-RSA-DES-CBC3-SHA SSLv3 Kx=ECDH Au=RSA Enc=3DES(168) Mac=SHA1 ECDHE-ECDSA-DES-CBC3-SHA SSLv3 Kx=ECDH Au=ECDSA Enc=3DES(168) Mac=SHA1 DES-CBC3-SHA SSLv3 Kx=RSA Au=RSA Enc=3DES(168) Mac=SHA1 Now let's see what happens if you use 'DEFAULT': openssl ciphers -v 'DEFAULT' ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(256) Mac=AEAD ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AESGCM(256) Mac=AEAD ECDHE-RSA-AES256-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AES(256) Mac=SHA384 ECDHE-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AES(256) Mac=SHA384 ECDHE-RSA-AES256-SHA SSLv3 Kx=ECDH Au=RSA Enc=AES(256) Mac=SHA1 ECDHE-ECDSA-AES256-SHA SSLv3 Kx=ECDH Au=ECDSA Enc=AES(256) Mac=SHA1 DHE-DSS-AES256-GCM-SHA384 TLSv1.2 Kx=DH Au=DSS Enc=AESGCM(256) Mac=AEAD DHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=DH Au=RSA Enc=AESGCM(256) Mac=AEAD DHE-RSA-AES256-SHA256 TLSv1.2 Kx=DH Au=RSA Enc=AES(256) Mac=SHA256 DHE-DSS-AES256-SHA256 TLSv1.2 Kx=DH Au=DSS Enc=AES(256) Mac=SHA256 DHE-RSA-AES256-SHA SSLv3 Kx=DH Au=RSA Enc=AES(256) Mac=SHA1 DHE-DSS-AES256-SHA SSLv3 Kx=DH Au=DSS Enc=AES(256) Mac=SHA1 DHE-RSA-CAMELLIA256-SHA SSLv3 Kx=DH Au=RSA Enc=Camellia(256) Mac=SHA1 DHE-DSS-CAMELLIA256-SHA SSLv3 Kx=DH Au=DSS Enc=Camellia(256) Mac=SHA1 ECDH-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH/RSA Au=ECDH Enc=AESGCM(256) Mac=AEAD ECDH-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH/ECDSA Au=ECDH Enc=AESGCM(256) Mac=AEAD ECDH-RSA-AES256-SHA384 TLSv1.2 Kx=ECDH/RSA Au=ECDH Enc=AES(256) Mac=SHA384 ECDH-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH/ECDSA Au=ECDH Enc=AES(256) Mac=SHA384 ECDH-RSA-AES256-SHA SSLv3 Kx=ECDH/RSA Au=ECDH Enc=AES(256) Mac=SHA1 ECDH-ECDSA-AES256-SHA SSLv3 Kx=ECDH/ECDSA Au=ECDH Enc=AES(256) Mac=SHA1 AES256-GCM-SHA384 TLSv1.2 Kx=RSA Au=RSA Enc=AESGCM(256) Mac=AEAD AES256-SHA256 TLSv1.2 Kx=RSA Au=RSA Enc=AES(256) Mac=SHA256 AES256-SHA SSLv3 Kx=RSA Au=RSA Enc=AES(256) Mac=SHA1 CAMELLIA256-SHA SSLv3 Kx=RSA Au=RSA Enc=Camellia(256) Mac=SHA1 PSK-AES256-CBC-SHA SSLv3 Kx=PSK Au=PSK Enc=AES(256) Mac=SHA1 ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AESGCM(128) Mac=AEAD ECDHE-RSA-AES128-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AES(128) Mac=SHA256 ECDHE-ECDSA-AES128-SHA256 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AES(128) Mac=SHA256 ECDHE-RSA-AES128-SHA SSLv3 Kx=ECDH Au=RSA Enc=AES(128) Mac=SHA1 ECDHE-ECDSA-AES128-SHA SSLv3 Kx=ECDH Au=ECDSA Enc=AES(128) Mac=SHA1 DHE-DSS-AES128-GCM-SHA256 TLSv1.2 Kx=DH Au=DSS Enc=AESGCM(128) Mac=AEAD DHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=DH Au=RSA Enc=AESGCM(128) Mac=AEAD DHE-RSA-AES128-SHA256 TLSv1.2 Kx=DH Au=RSA Enc=AES(128) Mac=SHA256 DHE-DSS-AES128-SHA256 TLSv1.2 Kx=DH Au=DSS Enc=AES(128) Mac=SHA256 DHE-RSA-AES128-SHA SSLv3 Kx=DH Au=RSA Enc=AES(128) Mac=SHA1 DHE-DSS-AES128-SHA SSLv3 Kx=DH Au=DSS Enc=AES(128) Mac=SHA1 DHE-RSA-SEED-SHA SSLv3 Kx=DH Au=RSA Enc=SEED(128) Mac=SHA1 DHE-DSS-SEED-SHA SSLv3 Kx=DH Au=DSS Enc=SEED(128) Mac=SHA1 DHE-RSA-CAMELLIA128-SHA SSLv3 Kx=DH Au=RSA Enc=Camellia(128) Mac=SHA1 DHE-DSS-CAMELLIA128-SHA SSLv3 Kx=DH Au=DSS Enc=Camellia(128) Mac=SHA1 ECDH-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH/RSA Au=ECDH Enc=AESGCM(128) Mac=AEAD ECDH-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH/ECDSA Au=ECDH Enc=AESGCM(128) Mac=AEAD ECDH-RSA-AES128-SHA256 TLSv1.2 Kx=ECDH/RSA Au=ECDH Enc=AES(128) Mac=SHA256 ECDH-ECDSA-AES128-SHA256 TLSv1.2 Kx=ECDH/ECDSA Au=ECDH Enc=AES(128) Mac=SHA256 ECDH-RSA-AES128-SHA SSLv3 Kx=ECDH/RSA Au=ECDH Enc=AES(128) Mac=SHA1 ECDH-ECDSA-AES128-SHA SSLv3 Kx=ECDH/ECDSA Au=ECDH Enc=AES(128) Mac=SHA1 AES128-GCM-SHA256 TLSv1.2 Kx=RSA Au=RSA Enc=AESGCM(128) Mac=AEAD AES128-SHA256 TLSv1.2 Kx=RSA Au=RSA Enc=AES(128) Mac=SHA256 AES128-SHA SSLv3 Kx=RSA Au=RSA Enc=AES(128) Mac=SHA1 SEED-SHA SSLv3 Kx=RSA Au=RSA Enc=SEED(128) Mac=SHA1 CAMELLIA128-SHA SSLv3 Kx=RSA Au=RSA Enc=Camellia(128) Mac=SHA1 PSK-AES128-CBC-SHA SSLv3 Kx=PSK Au=PSK Enc=AES(128) Mac=SHA1 ECDHE-RSA-RC4-SHA SSLv3 Kx=ECDH Au=RSA Enc=RC4(128) Mac=SHA1 ECDHE-ECDSA-RC4-SHA SSLv3 Kx=ECDH Au=ECDSA Enc=RC4(128) Mac=SHA1 ECDH-RSA-RC4-SHA SSLv3 Kx=ECDH/RSA Au=ECDH Enc=RC4(128) Mac=SHA1 ECDH-ECDSA-RC4-SHA SSLv3 Kx=ECDH/ECDSA Au=ECDH Enc=RC4(128) Mac=SHA1 RC4-SHA SSLv3 Kx=RSA Au=RSA Enc=RC4(128) Mac=SHA1 RC4-MD5 SSLv3 Kx=RSA Au=RSA Enc=RC4(128) Mac=MD5 PSK-RC4-SHA SSLv3 Kx=PSK Au=PSK Enc=RC4(128) Mac=SHA1 ECDHE-RSA-DES-CBC3-SHA SSLv3 Kx=ECDH Au=RSA Enc=3DES(168) Mac=SHA1 ECDHE-ECDSA-DES-CBC3-SHA SSLv3 Kx=ECDH Au=ECDSA Enc=3DES(168) Mac=SHA1 EDH-RSA-DES-CBC3-SHA SSLv3 Kx=DH Au=RSA Enc=3DES(168) Mac=SHA1 EDH-DSS-DES-CBC3-SHA SSLv3 Kx=DH Au=DSS Enc=3DES(168) Mac=SHA1 ECDH-RSA-DES-CBC3-SHA SSLv3 Kx=ECDH/RSA Au=ECDH Enc=3DES(168) Mac=SHA1 ECDH-ECDSA-DES-CBC3-SHA SSLv3 Kx=ECDH/ECDSA Au=ECDH Enc=3DES(168) Mac=SHA1 DES-CBC3-SHA SSLv3 Kx=RSA Au=RSA Enc=3DES(168) Mac=SHA1 PSK-3DES-EDE-CBC-SHA SSLv3 Kx=PSK Au=PSK Enc=3DES(168) Mac=SHA1 EDH-RSA-DES-CBC-SHA SSLv3 Kx=DH Au=RSA Enc=DES(56) Mac=SHA1 EDH-DSS-DES-CBC-SHA SSLv3 Kx=DH Au=DSS Enc=DES(56) Mac=SHA1 DES-CBC-SHA SSLv3 Kx=RSA Au=RSA Enc=DES(56) Mac=SHA1 EXP-EDH-RSA-DES-CBC-SHA SSLv3 Kx=DH(512) Au=RSA Enc=DES(40) Mac=SHA1 export EXP-EDH-DSS-DES-CBC-SHA SSLv3 Kx=DH(512) Au=DSS Enc=DES(40) Mac=SHA1 export EXP-DES-CBC-SHA SSLv3 Kx=RSA(512) Au=RSA Enc=DES(40) Mac=SHA1 export EXP-RC2-CBC-MD5 SSLv3 Kx=RSA(512) Au=RSA Enc=RC2(40) Mac=MD5 export EXP-RC4-MD5 SSLv3 Kx=RSA(512) Au=RSA Enc=RC4(40) Mac=MD5 export As you can see that enables far, far more ciphers, including a number of unsafe ciphers – export, MD5, DES, etc. This is a good example of why you always want to confirm your cipher settings and check exactly what is being enabled before placing new settings into production. Many security disasters could be avoided if everyone doublechecked their settings first. Let’s take a closer look at how OpenSSL represents one of the cipher suites: ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(256) Mac=AEAD The columns are: Cipher Suite: ECDHE-RSA-AES256-GCM-SHA384 Protocol: TLSv1.2 Key Exchange Algorithm (Kx): ECDH Authentication Algorithm (Au): RSA Cipher/Encryption Algorithm (Enc): AESGCM(256) MAC (Mac): AEAD Since the control plane uses OpenSSL you can use the standard OpenSSL documentation, so I won't spend a lot of time on that. Data Plane In TMM the cipher suites are configured in the Ciphers field of the Client SSL or Server SSL profiles. See K14783: Overview of the Client SSL profile (11.x - 12.x) & K14806: Overview of the Server SSL profile (11.x - 12.x), respectively for more details. It is important to keep in mind that these are two different worlds with their own requirements and quirks. As most of the configuration activity, and security concerns, occur on the public facing side of the system, we'll focus on the Client SSL Profile. Most of the things we'll cover here will also apply to the Server SSL profile. In the GUI it appears as an editable field: Presuming the profile was created with the name 'Test': tmsh list ltm profile client-ssl Test ltm profile client-ssl Test { app-service none cert default.crt cert-key-chain { default { cert default.crt key default.key } } chain none ciphers DEFAULT defaults-from clientssl inherit-certkeychain true key default.key passphrase none } Modifying the cipher configuration from the command line is simple. tmsh list ltm profile client-ssl Test ciphers ltm profile client-ssl Test { ciphers DEFAULT } tmsh modify ltm profile client-ssl Test ciphers 'DEFAULT:!3DES' tmsh list ltm profile client-ssl Test ciphers ltm profile client-ssl Test { ciphers DEFAULT:!3DES } Just remember the ' tmsh save sys config ' when you're happy with the configuration. Note here the default is just 'DEFAULT'. What that expands to will vary depending on the version of TMOS. K13156: SSL ciphers used in the default SSL profiles (11.x - 12.x) defines the default values for each version of TMOS. Or you can check it locally from the command line: tmm --clientciphers 'DEFAULT' On 12.1.2 that would be: tmm --clientciphers 'DEFAULT' ID SUITE BITS PROT METHOD CIPHER MAC KEYX 0: 159 DHE-RSA-AES256-GCM-SHA384 256 TLS1.2 Native AES-GCM SHA384 EDH/RSA 1: 158 DHE-RSA-AES128-GCM-SHA256 128 TLS1.2 Native AES-GCM SHA256 EDH/RSA 2: 107 DHE-RSA-AES256-SHA256 256 TLS1.2 Native AES SHA256 EDH/RSA 3: 57 DHE-RSA-AES256-SHA 256 TLS1 Native AES SHA EDH/RSA 4: 57 DHE-RSA-AES256-SHA 256 TLS1.1 Native AES SHA EDH/RSA 5: 57 DHE-RSA-AES256-SHA 256 TLS1.2 Native AES SHA EDH/RSA 6: 57 DHE-RSA-AES256-SHA 256 DTLS1 Native AES SHA EDH/RSA 7: 103 DHE-RSA-AES128-SHA256 128 TLS1.2 Native AES SHA256 EDH/RSA 8: 51 DHE-RSA-AES128-SHA 128 TLS1 Native AES SHA EDH/RSA 9: 51 DHE-RSA-AES128-SHA 128 TLS1.1 Native AES SHA EDH/RSA 10: 51 DHE-RSA-AES128-SHA 128 TLS1.2 Native AES SHA EDH/RSA 11: 51 DHE-RSA-AES128-SHA 128 DTLS1 Native AES SHA EDH/RSA 12: 22 DHE-RSA-DES-CBC3-SHA 168 TLS1 Native DES SHA EDH/RSA 13: 22 DHE-RSA-DES-CBC3-SHA 168 TLS1.1 Native DES SHA EDH/RSA 14: 22 DHE-RSA-DES-CBC3-SHA 168 TLS1.2 Native DES SHA EDH/RSA 15: 22 DHE-RSA-DES-CBC3-SHA 168 DTLS1 Native DES SHA EDH/RSA 16: 157 AES256-GCM-SHA384 256 TLS1.2 Native AES-GCM SHA384 RSA 17: 156 AES128-GCM-SHA256 128 TLS1.2 Native AES-GCM SHA256 RSA 18: 61 AES256-SHA256 256 TLS1.2 Native AES SHA256 RSA 19: 53 AES256-SHA 256 TLS1 Native AES SHA RSA 20: 53 AES256-SHA 256 TLS1.1 Native AES SHA RSA 21: 53 AES256-SHA 256 TLS1.2 Native AES SHA RSA 22: 53 AES256-SHA 256 DTLS1 Native AES SHA RSA 23: 60 AES128-SHA256 128 TLS1.2 Native AES SHA256 RSA 24: 47 AES128-SHA 128 TLS1 Native AES SHA RSA 25: 47 AES128-SHA 128 TLS1.1 Native AES SHA RSA 26: 47 AES128-SHA 128 TLS1.2 Native AES SHA RSA 27: 47 AES128-SHA 128 DTLS1 Native AES SHA RSA 28: 10 DES-CBC3-SHA 168 TLS1 Native DES SHA RSA 29: 10 DES-CBC3-SHA 168 TLS1.1 Native DES SHA RSA 30: 10 DES-CBC3-SHA 168 TLS1.2 Native DES SHA RSA 31: 10 DES-CBC3-SHA 168 DTLS1 Native DES SHA RSA 32: 49200 ECDHE-RSA-AES256-GCM-SHA384 256 TLS1.2 Native AES-GCM SHA384 ECDHE_RSA 33: 49199 ECDHE-RSA-AES128-GCM-SHA256 128 TLS1.2 Native AES-GCM SHA256 ECDHE_RSA 34: 49192 ECDHE-RSA-AES256-SHA384 256 TLS1.2 Native AES SHA384 ECDHE_RSA 35: 49172 ECDHE-RSA-AES256-CBC-SHA 256 TLS1 Native AES SHA ECDHE_RSA 36: 49172 ECDHE-RSA-AES256-CBC-SHA 256 TLS1.1 Native AES SHA ECDHE_RSA 37: 49172 ECDHE-RSA-AES256-CBC-SHA 256 TLS1.2 Native AES SHA ECDHE_RSA 38: 49191 ECDHE-RSA-AES128-SHA256 128 TLS1.2 Native AES SHA256 ECDHE_RSA 39: 49171 ECDHE-RSA-AES128-CBC-SHA 128 TLS1 Native AES SHA ECDHE_RSA 40: 49171 ECDHE-RSA-AES128-CBC-SHA 128 TLS1.1 Native AES SHA ECDHE_RSA 41: 49171 ECDHE-RSA-AES128-CBC-SHA 128 TLS1.2 Native AES SHA ECDHE_RSA 42: 49170 ECDHE-RSA-DES-CBC3-SHA 168 TLS1 Native DES SHA ECDHE_RSA 43: 49170 ECDHE-RSA-DES-CBC3-SHA 168 TLS1.1 Native DES SHA ECDHE_RSA 44: 49170 ECDHE-RSA-DES-CBC3-SHA 168 TLS1.2 Native DES SHA ECDHE_RSA Some differences when compared to OpenSSL are readily apparent. For starters, TMM kindly includes a column label header, and actually aligns the columns. The first column is simply a 0-ordinal numeric index, the rest are as follows: ID: The official SSL/TLS ID assigned to that cipher suite. SUITE: The cipher suite. BITS: The size of the key in bits. PROT: The protocol supported. METHOD: NATIVE (in TMM) vs. COMPAT (using OpenSSL code). CIPHER: The cipher. MAC: The hash function. KEYX: The Key Exchange and Authentication Algorithms Note that the MAC is a little misleading for AES-GCM cipher suites. There is no separate MAC as they're AEAD. But the hashing algorithm is used in the Pseudo-Random Function (PRF) and a few other handshake related places. Selecting the Cipher Suites Now we know how to look at the current configuration, modify it, and list the actual ciphers that will be enabled by the listed suites. But what do we put into the configuration? Most users won't have to touch this. The default values are carefully selected by F5 to meet the needs of the majority of our customers. That's the good news. The bad news is that some customers will need to get in there and change the configuration – be it for regulatory compliance, internal policies, legacy client support, etc. Once you begin modifying them, the configuration is truly custom for each customer. Every customer who modifies the configuration, and uses a custom cipher configuration, needs to determine what the proper list is for their needs. Let's say we have determined that we need to support only AES & AES-GCM, 128-bit or 256-bit, and only ECDHE key exchange. Any MAC or Authentication is fine. OK, let's proceed from there. On 12.1.2 there are six cipher suites that fit those criteria. We could list them all explicitly: tmm --clientciphers 'ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-CBC-SHA:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES128-CBC-SHA' That will work, but it gets unwieldy fast. Not only that, but in versions up to 11.5.0 the ciphers configuration string was truncated at 256bytes. Starting in 11.5.0 that was increased to 768bytes, but that can still truncate long configurations. We'll revisit this when we get to the pitfalls section. Fortunately, there is an alternative – keywords! This will result in the same list of cipher suites: tmm --clientciphers 'ECDHE+AES-GCM:ECDHE+AES' That specifies the ECDHE key exchange with AES-GCM ciphers, and ECDHE with AES ciphers. Let's take a closer look to help understand what is happening here. Keywords Keywords are extremely important when working with cipher suite configuration, so we'll spend a little time on those. Most of these apply to both the control plane (OpenSSL) and the data plane (TMM), unless otherwise noted, but we're focused on the data plane as that's F5 specific. Keywords organize into different categories. F5 specific: NATIVE: cipher suites implemented natively in TMM COMPAT: cipher suites using OpenSSL code; removed as of 12.0.0 @SPEED: Re-orders the list to put 'faster' (based on TMOS implementation performance) ciphers first. Sorting: @SPEED: Re-orders the list to put 'faster' (based on TMOS implementation performance) ciphers first. (F5 Specific) @STRENGTH: Re-orders the list to put 'stronger' (larger keys) ciphers first. Protocol: TLSv1_2: cipher suites available under TLSv1.2 TLSv1_1: cipher suites available under TLSv1.1 TLSv1: cipher suites available under TLSv1.0 SSLv3: cipher suites available under SSLv3 Note the 'Protocol' keywords in the cipher configuration control the ciphers associated with that protocol, and not the protocol itself! More on this in pitfalls. Key Exchange Algorithms (sometimes with Authentication specified): ECDHE or ECDHA_RSA: Elliptic Curve Diffie-Hellman Ephemeral (with RSA) ECDHE_ECDSA: ECDHE with Elliptic Curve Digital Signature Algorithm DHE or EDH: Diffie-Hellman Ephemeral (aka Ephemeral Diffie-Hellman) (with RSA) DHE_DSS: DHE with Digital Signature Standard (aka DSA – Digital Signature Algorithm) ECDH_RSA: Elliptic Curve Diffie-Hellman with RSA ECDH_ECDSA: ECDH with ECDSA RSA: RSA, obviously ADH: Anonymous Diffie-Hellman. Note the Authentication Algorithms don't work as standalone keywords in TMM. You can't use 'ECDSA' or 'DSS' for example. And you might think ECDHE or DHE includes all such cipher suites – note that they don't if you read carefully. General cipher groupings: DEFAULT: The default cipher suite for that version; see K13156 ALL: All NATIVE cipher suites; does not include COMPAT in current versions HIGH: 'High' security cipher suites; >128-bit MEDIUM: 'Medium' security cipher suites; effectively 128-bit suites LOW: 'Low' security cipher suites; <128-bit excluding export grade ciphers EXP or EXPORT: Export grade ciphers; 40-bit or 56-bit EXPORT56: 56-bit export ciphers EXPORT40: 40-bit export ciphers Note that DEFAULT does change periodically as F5 updates the configuration to follow the latest best practices. K13156: SSL ciphers used in the default SSL profiles (11.x - 12.x) documents these changes. Cipher families: AES-GCM: AES in GCM mode; 128-bit or 256-bit AES: AES in CBC mode; 128-bit or 256-bit CAMELLIA: Camellia in CBC mode; 128-bit or 256-bit 3DES: Triple DES in CBC mode; 168-bit (well, 112-bit really) DES: Single DES in CBC mode, includes EXPORTciphers;40-bit & 56-bit. RC4: RC4 stream cipher NULL: NULL cipher; just what it sounds like, it does nothing – no encryption MAC aka Hash Function: SHA384: SHA-2 384-bit hash SHA256: SHA-2 256-bit hash SHA1 or SHA: SHA-1 160-bit hash MD5: MD5 128-bit hash Other: On older TMOS versions when using the COMPAT keyword it also enables two additional keywords: SSLv2: Ciphers supported on the SSLv2 protocol RC2: RC2 ciphers. So, let's go back to our example: tmm --clientciphers 'ECDHE+AES-GCM:ECDHE+AES' Note that you can combine keywords using '+' (plus sign). And multiple entries in the ciphers configuration line are separated with ':' (colon). You may also need to wrap the string in single quotes on the command line – I find it is a good habit to just always do so. We can also exclude suites or keywords. There are two ways to do that: '!' (exclamation point) is a hard exclusion. Anything excluded this way cannot be implicitly or explicitly re-enabled. It is disabled, period. '-' (minus sign or dash) is a soft exclusion. Anything excluded this way can be explicitly re-enabled later in the configuration string. (Note: The dash is also usedinthe names of many cipher suites, such as ECDHE-RSA-AES256-GCM-SHA384 or AES128-SHA. Do not confuse the dashes that are part of the cipher suite names with a soft exclusion, which alwaysprecedes, or prefixes,the value being excluded. 'AES128-SHA': AES128-SHA cipher suite. '-SHA': SHA is soft excluded. '-AES128-SHA': the AES128-SHA cipher suite is soft excluded. Position matters.) Let's look at the difference in hard and soft exclusions. We'll start with our base example: tmm --clientciphers 'ECDHE+AES-GCM:DHE+AES-GCM' ID SUITE BITS PROT METHOD CIPHER MAC KEYX 0: 49200 ECDHE-RSA-AES256-GCM-SHA384 256 TLS1.2 Native AES-GCM SHA384 ECDHE_RSA 1: 49199 ECDHE-RSA-AES128-GCM-SHA256 128 TLS1.2 Native AES-GCM SHA256 ECDHE_RSA 2: 159 DHE-RSA-AES256-GCM-SHA384 256 TLS1.2 Native AES-GCM SHA384 EDH/RSA 3: 158 DHE-RSA-AES128-GCM-SHA256 128 TLS1.2 Native AES-GCM SHA256 EDH/RSA Now let's look at a hard exclusion: tmm --clientciphers 'ECDHE+AES-GCM:!DHE:DHE+AES-GCM' ID SUITE BITS PROT METHOD CIPHER MAC KEYX 0: 49200 ECDHE-RSA-AES256-GCM-SHA384 256 TLS1.2 Native AES-GCM SHA384 ECDHE_RSA 1: 49199 ECDHE-RSA-AES128-GCM-SHA256 128 TLS1.2 Native AES-GCM SHA256 ECDHE_RSA And lastly a soft exclusion: tmm --clientciphers 'ECDHE+AES-GCM:-DHE:DHE+AES-GCM' ID SUITE BITS PROT METHOD CIPHER MAC KEYX 0: 49200 ECDHE-RSA-AES256-GCM-SHA384 256 TLS1.2 Native AES-GCM SHA384 ECDHE_RSA 1: 49199 ECDHE-RSA-AES128-GCM-SHA256 128 TLS1.2 Native AES-GCM SHA256 ECDHE_RSA 2: 159 DHE-RSA-AES256-GCM-SHA384 256 TLS1.2 Native AES-GCM SHA384 EDH/RSA 3: 158 DHE-RSA-AES128-GCM-SHA256 128 TLS1.2 Native AES-GCM SHA256 EDH/RSA Note that in the second example, the hard exclusion, we used '!DHE' and even though we then explicitly added 'DHE+AES-GCM' those ciphers were not enabled. This is because, once excluded with a hard exclusion, ciphers cannot be re-enabled. In the third example, the soft exclusion, we used '-DHE' and then 'DHE+AES-GCM'. This time it did enable those ciphers, which is possible with a soft exclusion. You might be wondering what soft disabling is useful for; why would you ever want to remove ciphers only to add them again? Reordering the ciphers is a common use case. As an example, DEFAULT orders ciphers differently in different versions, but mainly based on strength – bit size. Let's say we know 3DES is really 112-bit equivalent strength and not 168-bit as it is usually labeled. For some reason, maybe legacy clients, we can't disable them, but we want them to be last on the list. One way to do this is to first configure the DEFAULT list, then remove all of the 3DES ciphers. But then add the 3DES ciphers back explicitly – at the end of the list. Let's try it – compare the following: tmm --clientciphers 'DEFAULT' ID SUITE BITS PROT METHOD CIPHER MAC KEYX 0: 159 DHE-RSA-AES256-GCM-SHA384 256 TLS1.2 Native AES-GCM SHA384 EDH/RSA 1: 158 DHE-RSA-AES128-GCM-SHA256 128 TLS1.2 Native AES-GCM SHA256 EDH/RSA 2: 107 DHE-RSA-AES256-SHA256 256 TLS1.2 Native AES SHA256 EDH/RSA 3: 57 DHE-RSA-AES256-SHA 256 TLS1 Native AES SHA EDH/RSA 4: 57 DHE-RSA-AES256-SHA 256 TLS1.1 Native AES SHA EDH/RSA 5: 57 DHE-RSA-AES256-SHA 256 TLS1.2 Native AES SHA EDH/RSA 6: 57 DHE-RSA-AES256-SHA 256 DTLS1 Native AES SHA EDH/RSA 7: 103 DHE-RSA-AES128-SHA256 128 TLS1.2 Native AES SHA256 EDH/RSA 8: 51 DHE-RSA-AES128-SHA 128 TLS1 Native AES SHA EDH/RSA 9: 51 DHE-RSA-AES128-SHA 128 TLS1.1 Native AES SHA EDH/RSA 10: 51 DHE-RSA-AES128-SHA 128 TLS1.2 Native AES SHA EDH/RSA 11: 51 DHE-RSA-AES128-SHA 128 DTLS1 Native AES SHA EDH/RSA 12: 22 DHE-RSA-DES-CBC3-SHA 168 TLS1 Native DES SHA EDH/RSA 13: 22 DHE-RSA-DES-CBC3-SHA 168 TLS1.1 Native DES SHA EDH/RSA 14: 22 DHE-RSA-DES-CBC3-SHA 168 TLS1.2 Native DES SHA EDH/RSA 15: 22 DHE-RSA-DES-CBC3-SHA 168 DTLS1 Native DES SHA EDH/RSA 16: 157 AES256-GCM-SHA384 256 TLS1.2 Native AES-GCM SHA384 RSA 17: 156 AES128-GCM-SHA256 128 TLS1.2 Native AES-GCM SHA256 RSA 18: 61 AES256-SHA256 256 TLS1.2 Native AES SHA256 RSA 19: 53 AES256-SHA 256 TLS1 Native AES SHA RSA 20: 53 AES256-SHA 256 TLS1.1 Native AES SHA RSA 21: 53 AES256-SHA 256 TLS1.2 Native AES SHA RSA 22: 53 AES256-SHA 256 DTLS1 Native AES SHA RSA 23: 60 AES128-SHA256 128 TLS1.2 Native AES SHA256 RSA 24: 47 AES128-SHA 128 TLS1 Native AES SHA RSA 25: 47 AES128-SHA 128 TLS1.1 Native AES SHA RSA 26: 47 AES128-SHA 128 TLS1.2 Native AES SHA RSA 27: 47 AES128-SHA 128 DTLS1 Native AES SHA RSA 28: 10 DES-CBC3-SHA 168 TLS1 Native DES SHA RSA 29: 10 DES-CBC3-SHA 168 TLS1.1 Native DES SHA RSA 30: 10 DES-CBC3-SHA 168 TLS1.2 Native DES SHA RSA 31: 10 DES-CBC3-SHA 168 DTLS1 Native DES SHA RSA 32: 49200 ECDHE-RSA-AES256-GCM-SHA384 256 TLS1.2 Native AES-GCM SHA384 ECDHE_RSA 33: 49199 ECDHE-RSA-AES128-GCM-SHA256 128 TLS1.2 Native AES-GCM SHA256 ECDHE_RSA 34: 49192 ECDHE-RSA-AES256-SHA384 256 TLS1.2 Native AES SHA384 ECDHE_RSA 35: 49172 ECDHE-RSA-AES256-CBC-SHA 256 TLS1 Native AES SHA ECDHE_RSA 36: 49172 ECDHE-RSA-AES256-CBC-SHA 256 TLS1.1 Native AES SHA ECDHE_RSA 37: 49172 ECDHE-RSA-AES256-CBC-SHA 256 TLS1.2 Native AES SHA ECDHE_RSA 38: 49191 ECDHE-RSA-AES128-SHA256 128 TLS1.2 Native AES SHA256 ECDHE_RSA 39: 49171 ECDHE-RSA-AES128-CBC-SHA 128 TLS1 Native AES SHA ECDHE_RSA 40: 49171 ECDHE-RSA-AES128-CBC-SHA 128 TLS1.1 Native AES SHA ECDHE_RSA 41: 49171 ECDHE-RSA-AES128-CBC-SHA 128 TLS1.2 Native AES SHA ECDHE_RSA 42: 49170 ECDHE-RSA-DES-CBC3-SHA 168 TLS1 Native DES SHA ECDHE_RSA 43: 49170 ECDHE-RSA-DES-CBC3-SHA 168 TLS1.1 Native DES SHA ECDHE_RSA 44: 49170 ECDHE-RSA-DES-CBC3-SHA 168 TLS1.2 Native DES SHA ECDHE_RSA tmm --clientciphers 'DEFAULT:-3DES:!SSLv3:3DES+ECDHE:3DES+DHE:3DES+RSA' ID SUITE BITS PROT METHOD CIPHER MAC KEYX 0: 159 DHE-RSA-AES256-GCM-SHA384 256 TLS1.2 Native AES-GCM SHA384 EDH/RSA 1: 158 DHE-RSA-AES128-GCM-SHA256 128 TLS1.2 Native AES-GCM SHA256 EDH/RSA 2: 107 DHE-RSA-AES256-SHA256 256 TLS1.2 Native AES SHA256 EDH/RSA 3: 57 DHE-RSA-AES256-SHA 256 TLS1 Native AES SHA EDH/RSA 4: 57 DHE-RSA-AES256-SHA 256 TLS1.1 Native AES SHA EDH/RSA 5: 57 DHE-RSA-AES256-SHA 256 TLS1.2 Native AES SHA EDH/RSA 6: 57 DHE-RSA-AES256-SHA 256 DTLS1 Native AES SHA EDH/RSA 7: 103 DHE-RSA-AES128-SHA256 128 TLS1.2 Native AES SHA256 EDH/RSA 8: 51 DHE-RSA-AES128-SHA 128 TLS1 Native AES SHA EDH/RSA 9: 51 DHE-RSA-AES128-SHA 128 TLS1.1 Native AES SHA EDH/RSA 10: 51 DHE-RSA-AES128-SHA 128 TLS1.2 Native AES SHA EDH/RSA 11: 51 DHE-RSA-AES128-SHA 128 DTLS1 Native AES SHA EDH/RSA 12: 157 AES256-GCM-SHA384 256 TLS1.2 Native AES-GCM SHA384 RSA 13: 156 AES128-GCM-SHA256 128 TLS1.2 Native AES-GCM SHA256 RSA 14: 61 AES256-SHA256 256 TLS1.2 Native AES SHA256 RSA 15: 53 AES256-SHA 256 TLS1 Native AES SHA RSA 16: 53 AES256-SHA 256 TLS1.1 Native AES SHA RSA 17: 53 AES256-SHA 256 TLS1.2 Native AES SHA RSA 18: 53 AES256-SHA 256 DTLS1 Native AES SHA RSA 19: 60 AES128-SHA256 128 TLS1.2 Native AES SHA256 RSA 20: 47 AES128-SHA 128 TLS1 Native AES SHA RSA 21: 47 AES128-SHA 128 TLS1.1 Native AES SHA RSA 22: 47 AES128-SHA 128 TLS1.2 Native AES SHA RSA 23: 47 AES128-SHA 128 DTLS1 Native AES SHA RSA 24: 49200 ECDHE-RSA-AES256-GCM-SHA384 256 TLS1.2 Native AES-GCM SHA384 ECDHE_RSA 25: 49199 ECDHE-RSA-AES128-GCM-SHA256 128 TLS1.2 Native AES-GCM SHA256 ECDHE_RSA 26: 49192 ECDHE-RSA-AES256-SHA384 256 TLS1.2 Native AES SHA384 ECDHE_RSA 27: 49172 ECDHE-RSA-AES256-CBC-SHA 256 TLS1 Native AES SHA ECDHE_RSA 28: 49172 ECDHE-RSA-AES256-CBC-SHA 256 TLS1.1 Native AES SHA ECDHE_RSA 29: 49172 ECDHE-RSA-AES256-CBC-SHA 256 TLS1.2 Native AES SHA ECDHE_RSA 30: 49191 ECDHE-RSA-AES128-SHA256 128 TLS1.2 Native AES SHA256 ECDHE_RSA 31: 49171 ECDHE-RSA-AES128-CBC-SHA 128 TLS1 Native AES SHA ECDHE_RSA 32: 49171 ECDHE-RSA-AES128-CBC-SHA 128 TLS1.1 Native AES SHA ECDHE_RSA 33: 49171 ECDHE-RSA-AES128-CBC-SHA 128 TLS1.2 Native AES SHA ECDHE_RSA 34: 49170 ECDHE-RSA-DES-CBC3-SHA 168 TLS1 Native DES SHA ECDHE_RSA 35: 49170 ECDHE-RSA-DES-CBC3-SHA 168 TLS1.1 Native DES SHA ECDHE_RSA 36: 49170 ECDHE-RSA-DES-CBC3-SHA 168 TLS1.2 Native DES SHA ECDHE_RSA 37: 22 DHE-RSA-DES-CBC3-SHA 168 TLS1 Native DES SHA EDH/RSA 38: 22 DHE-RSA-DES-CBC3-SHA 168 TLS1.1 Native DES SHA EDH/RSA 39: 22 DHE-RSA-DES-CBC3-SHA 168 TLS1.2 Native DES SHA EDH/RSA 40: 22 DHE-RSA-DES-CBC3-SHA 168 DTLS1 Native DES SHA EDH/RSA 41: 10 DES-CBC3-SHA 168 TLS1 Native DES SHA RSA 42: 10 DES-CBC3-SHA 168 TLS1.1 Native DES SHA RSA 43: 10 DES-CBC3-SHA 168 TLS1.2 Native DES SHA RSA 44: 10 DES-CBC3-SHA 168 DTLS1 Native DES SHA RSA I added something else in there which I'll come back to later. Pitfalls As should be clear by now cipher configuration is a powerful tool, but as the song says, every tool is a weapon if you hold it right. And weapons are dangerous. With a little careless handling it is easy to lose a toe – or a leg. Whenever you are working with cipher suite configuration the old rule of 'measure twice, cut once' applies – and then double-check the work to be certain. There are several common pitfalls which await you. Misuse Perhaps the most common pitfall is simply misuse – using cipher suite configuration for that which it is not intended. And the single most common example of this comes from using cipher configuration to manipulate protocols. Given the keywords, as described above, it seems common for users to presume that if they want to disable a protocol, such as TLSv1.0, then the way to do that is to use a cipher suite keyword, such as !TLSv1. And, indeed, this may seem to work – but it isn't doing what is desired. The protocol is not disabled, only the ciphers that are supported for that protocol are. The protocol is configured on the VIP independently of the ciphers. !TLSv1 would disable all ciphers supported under the TLSv1.0 protocol, but not the protocol itself. Note that the protocol negotiation and the cipher negotiation in the SSL/TLS handshake are independent. What happens if the VIP only supports TLSv1.0/v1.1/v1.2 and the client only supports SSLv3 & TLSv1.0? Well, they'd agree on TLSv1.0 as the common protocol. The cipher list the client sends in the Client Hello is independent of the protocol that is eventually negotiated. Say the client sends AES128-SHA and the server has that in its list, so it is selected. OK, we've agreed on a protocol and a cipher suite – only the server won't do any ciphers on TLSv1.0 because of '!TLSv1' in the ciphers configuration, and the connection will fail. That may seem like splitting hairs, but it makes a difference. If a scanner is looking for protocols that are enabled, and not the full handshake, it may still flag a system which has been configured this way. The protocol is negotiated during the SSL/TLS handshake before the cipher is selected. This also means the system is doing more work, as the handshake continues further before failing, and the log messages may be misleading. Instead of logging a protocol incompatibility the logs will reflect the failure to find a viable cipher, which can be a red herring when it comes time to debug the configuration. The right way to do this is to actually disable the protocol, which doesn't involve the cipher suite configuration at all. For the control plane this is done through the ssl-protocol directive: tmsh list sys http ssl-protocol sys httpd { ssl-protocol "all -SSLv2 -SSLv3" } For example, if we wanted to disable TLSv1.0: tmsh modify sys http ssl-protocol 'all -SSLv2 -SSLv3 -TLSv1' tmsh list sys http ssl-protocol sys httpd { ssl-protocol "all -SSLv2 -SSLv3 -TLSv1" } For the data plane this can be done via the Options List in the SSL Profile GUI, via the No SSL, No TLSv1.1, etc. directives: Or via the command line: tmsh list ltm profile client-ssl Test options ltm profile client-ssl Test { options { dont-insert-empty-fragments } } tmsh modify ltm profile client-ssl Test options {dont-insert-empty-fragments no-tlsv1} tmsh list /ltm profile client-ssl Test options ltm profile client-ssl Test { options { dont-insert-empty-fragments no-tlsv1 } } The values are slightly different on the command line, use this command to see them all: tmsh modify ltm profile client-ssl <profile-name> options ? Use the right tool for the job and you'll be more likely to succeed. Truncation As I previously mentioned, in versions up to 11.5.0 the ciphers configuration string was truncated at 256 bytes. Starting in 11.5.0 that was increased to 768 bytes (see K11481: The SSL profile cipher lists have a 256 character limitation for more information), but that can still silently truncate long configurations. This is not a theoretical issue, we've seen users run into this in the real world. For example, little over a year ago I worked with a customer who was then using 11.4.1 HF8. They were trying to very precisely control which ciphers were enabled, and their order. In order to do this they'd decided to enumerate every individual cipher in their configuration – resulting in this cipher suite configuration string: TLSv1_2+ECDHE-RSA-AES256-CBC-SHA:TLSv1_1+ECDHE-RSA-AES256-CBC-SHA:TLSv1_2+ECDHE-RSA-AES128-CBC-SHA:TLSv1_1+ECDHE-RSA-AES128-CBC-SHA:TLSv1_2+DHE-RSA-AES256-SHA:TLSv1_1+DHE-RSA-AES256-SHA:TLSv1_2+DHE-RSA-AES128-SHA:TLSv1_1+DHE-RSA-AES128-SHA:TLSv1_2+AES256-SHA256:TLSv1_1+AES256-SHA:TLSv1_2+AES128-SHA256:TLSv1_1+AES128-SHA:TLSv1+ECDHE-RSA-AES256-CBC-SHA:TLSv1+ECDHE-RSA-AES128-CBC-SHA:TLSv1+DHE-RSA-AES256-SHA:TLSv1+DHE-RSA-AES128-SHA:TLSv1+AES256-SHA:TLSv1+AES128-SHA:TLSv1+DES-CBC3-SHA That string would save in the configuration and it was there if you looked at the bigip.conf file, but it was silently truncated when the configuration was loaded. Since this was 11.4.1, only the first 256 bytes were loaded successfully, which made the running configuration: TLSv1_2+ECDHE-RSA-AES256-CBC-SHA:TLSv1_1+ECDHE-RSA-AES256-CBC-SHA:TLSv1_2+ECDHE-RSA-AES128-CBC-SHA:TLSv1_1+ECDHE-RSA-AES128-CBC-SHA:TLSv1_2+DHE-RSA-AES256-SHA:TLSv1_1+DHE-RSA-AES256-SHA:TLSv1_2+DHE-RSA-AES128-SHA:TLSv1_1+DHE-RSA-AES128-SHA:TLSv1_2+AES256-S Note the last suite is truncated itself, which means it was invalid and therefore ignored. If their configuration had worked they would've had nineteen protocol+suite combinations – instead they had eight. Needless to say, this caused some problems. This customer was missing ciphers that they expected to have working. That is bad enough – but it could be worse. Let's imagine a customer who wants to specify several specific ciphers first, then generally enable a number of other TLSv1.2 & TLSv1.1 ciphers. And, of course, they are careful to disable dangerous ciphers! TLSv1_2+ECDHE-RSA-AES256-CBC-SHA:TLSv1_1+ECDHE-RSA-AES256-CBC-SHA:TLSv1_2+ECDHE-RSA-AES128-CBC-SHA:TLSv1_1+ECDHE-RSA-AES128-CBC-SHA:TLSv1_2+DHE-RSA-AES256-SHA:TLSv1_1+DHE-RSA-AES256-SHA:TLSv1_2+DHE-RSA-AES128-SHA:TLSv1_1+DHE-RSA-AES128-SHA:TLSv1_2:TLSv1_1:!RC4:!MD5:!ADH:!DES:!EXPORT OK, that looks fairly solid, right? What do you suppose the problem with this is? This is the problem; in 11.4.1 and earlier it would truncate to this: TLSv1_2+ECDHE-RSA-AES256-CBC-SHA:TLSv1_1+ECDHE-RSA-AES256-CBC-SHA:TLSv1_2+ECDHE-RSA-AES128-CBC-SHA:TLSv1_1+ECDHE-RSA-AES128-CBC-SHA:TLSv1_2+DHE-RSA-AES256-SHA:TLSv1_1+DHE-RSA-AES256-SHA:TLSv1_2+DHE-RSA-AES128-SHA:TLSv1_1+DHE-RSA-AES128-SHA:TLSv1_2:TLSv1_1: All of the exclusions were truncated off! Now we have the opposite problem – there are a number of ciphers enabled which the customer expects to be disabled! And they're BAD ciphers – ADH, DES, MD5, RC4. So this customer would be at high risk without realizing it. Be aware of this; it is very sneaky. The configuration will look fine; the truncation happens in the code when it loads the configuration. This is also one reason why I always recommend listing your exclusions first in the configuration string. Then you can never accidentally enable something. Unintended Consequences Let's say a new CVE is announced which exposes a very serious vulnerability in SSLv3 & TLSv1.0. There is no way to mitigate it, and the only solution is to limit connections to only TLSv1.1 & TLSv1.2. You want a cipher configuration to accomplish this. It seems straight-forward – just configure it to use only ciphers on TLSv1.1 & TLSv1.2: tmsh modify ltm profile client-ssl <profile> ciphers 'TLSv1_2:TLSv1_1' Congratulations, you've solved the problem. You are no longer vulnerable to this CVE. You know there is a but coming, right? What's wrong? Well, you just enabled all TLSv1.2 & TLSv1.1 ciphers. That includes such gems as RC4-MD5, RC4-SHA, DES, and a few ADH (Anonymous Diffie-Hellman) suites which have no authentication. As recently as 11.3.0 you'd even be enabling some 40-bit EXPORT ciphers. (We pulled them out of NATIVE in 11.4.0.) So you just leapt out of the frying pan and into the fire. Always, always, always check the configuration before using it. Running that through tmm --clientciphers 'TLSv1_2:TLSv1_1' would've raised red flags. Instead, this configuration would work without causing those problems: tmsh modify ltm profile client-ssl <profile> ciphers 'DEFAULT:!TLSv1:!SSLv3' Another option, and probably the better one, is to disable the SSLv3 and TLSv1.0 protocols on the VIP. As I discussed above. Of course, you can do both – belt and suspenders. And just to show you how easy it is to make such a mistake, F5 did this! In K13400: SSL 3.0/TLS 1.0 BEAST vulnerability CVE-2011-3389 and TLS protocol vulnerability CVE-2012-1870 we originally had the following in the mitigation section: Note: Alternatively, to configure an SSL profile to use only TLS 1.1-compatible, TLS 1.2-compatible, AES-GCM, or RC4-SHA ciphers using the tmsh utility, use the following syntax: tmsh create /ltm profile client-ssl <name> ciphers TLSv1_1:TLSv1_2:AES-GCM:RC4-SHA Yes, I had this fixed long ago. Remember back in the section on keywords I had this comparison example: tmm --clientciphers 'DEFAULT' tmm --clientciphers 'DEFAULT:-3DES:!SSLv3:3DES+ECDHE:3DES+DHE:3DES+RSA' Who caught the '!SSLv3' in the second line? Why do you think I added that? Did I need to? Hint: What do you think the side effect of blanket enabling all of those 3DES ciphers would be if I didn't explicitly disable SSLv3? Cipher Ordering In SSL/TLS there are two main models to the cipher suite negotiation – Server Cipher Preference or Client Cipher Preference. What does this mean? In SSL/TLS the client sends the list of cipher suites it is willing and able to support in the Client Hello. The server also has its list of cipher suites that it is willing and able to support. In Client Cipher Preference the server will select the first cipher on the client's list that is also in the server's list. Effectively this gives the client influence over which cipher is selected based on the order of the list it sends. In Server Cipher Preference the server will select the first server on its own list that is also on the client's list. So the server gives the order of its list precedence. BIG-IP always operates in Server Cipher Preference, so be very careful in how you order your cipher suites. Preferred suites should go at the top of the list. How you order your cipher suites will directly affect which ciphers are used. It doesn't matter if a stronger cipher is available if a weak cipher is matched first. HTTP/2 How is HTTP/2 a pitfall? The HTTP/2 RFC7540 includes a blacklist of ciphers that are valid in TLS, but should not be used in HTTP/2. This can cause a problem on a server where the TLS negotiation is decoupled from the ALPN exchange for the higher level protocol. The server might select a cipher which is on the blacklist, and then when the connection attempts to step up to HTTP/2 via ALPN the client may terminate the connection with extreme prejudice. It is well known enough to be called out in the RFC – Section 9.2.2. F5 added support for HTTP/2 in 12.0.0 – and we fell into this trap. Our DEFAULT ciphers list was ordered such that it was almost certain a blacklisted cipher would be selected.; This was fixed in 12.0.0 HF3 and 12.1.0, but serves as an example. On 12.0.0 FINAL through 12.0.0 HF2 a simple fix was to configure the ciphers to be 'ECDHE+AES-GCM:DEFAULT'. ECDHE+AES-GCM is guaranteed to be supported by any client compliant with RFC7540 (HTTP/2). Putting it first ensures it is selected before any blacklisted cipher. 3DES Back in the section on ciphers I mentioned that we label 3DES as being 168-bit, but that it only provides the equivalent of 112-bit strength. So, what did I mean by that? DES operates on 64-bit data blocks, using 56-bits of key. So it has a strength of 2 56 . 3DES, aka Triple DES, was a stop-gap designed to stretch the life of DES once 56-bits was too weak to be safe, until AES became available. 3DES use the exact same DES cipher, it just uses it three times – hence the name. So you might think 3x56-bits = 168-bits. 2 168 strong. Right? No, not really. The standard implementation of 3DES is known as EDE – for Encrypt, Decrypt, Encrypt. (For reasons we don't need to get into here.) You take the 64-bit data block, run it through DES once to encrypt it with K 1 , then run it through again to decrypt it using K 2 , then encrypt it once again using K 3 . Three keys, that's still 168-bits, right? Well, you'd think so. But the devil is in the (implementation) details. First of all there are three keying options for 3DES: - Keying option 1: K1, K2, K3 – 168 unique bits (but only 112-bit strength!) - Keying option 2: K1, K2, K1 – 112 unique bits (but only 80-bit strength!) - Keying option 3: K1, K1, K1 – 56 unique bits, 56-bit strength (Equivalent to DES due to EDE!) F5 uses keying option one, so we have 168-bits of unique key. However, 3DES with keying option one is subject to a meet-in-the-middle cryptographic attack which only has a cost of 2 112 . It has even been reduced as low as 2 108 , as described in this paper. So it does not provide the expected 168-bits of security, and is in fact weaker than AES128. To add some confusion, due to an old issue we used to describe 3DES as being 192-bit. See: K17296: The BIG-IP system incorrectly reports a 192-bit key length for cipher suites using 3DES (DES-CBC3) for more details. Of course, with the appearance of the Sweet32 attack last fall I would encourage everyone to disable 3DES completely whenever possible. We're also seeing a growing number of scanners and audit tools recategorizing 3DES as a 'Medium' strength cipher, down from 'High', and correspondingly lowering the grade for any site still supporting it. If you don't need it, turn it off. See K13167034: OpenSSL vulnerability CVE-2016-2183 for more information. Conclusion Believe it or not, that's the quick overview of cipher suite configuration on BIG-IP. There are many areas where we could dig in further and spend some time in the weeds, but I hope that this article helps at least one person understand cipher suite configuration better, and to avoid the pitfalls that commonly claim those who work with them. Additional Resources This article is by no means comprehensive, and for those interested I'd encourage additional reading: BIG-IP SSL Cipher History by David Holmes, here on DevCentral Cipher Rules And Groups in BIG-IP v13 by Chase Abbott, also on DevCentral OpenSSL Cipher Documentation K8802: Using SSL ciphers with BIG-IP Client SSL and Server SSL profiles K15194: Overview of the BIG-IP SSL/TLS cipher suite K13163: SSL ciphers supported on BIG-IP platforms (11.x - 12.x) K13156: SSL ciphers used in the default SSL profiles (11.x - 12.x) K17370: Configuring the cipher strength for SSL profiles (12.x) K13171: Configuring the cipher strength for SSL profiles (11.x) K14783: Overview of the Client SSL profile (11.x - 12.x) K14806: Overview of the Server SSL profile (11.x - 12.x)L2 Deployment of BIG-IP with Gigamon
Introduction This article is part of a series on deploying BIG-IPs with bypass switches and network packet brokers. These devices allow for the transparent integration of network security tools with little to no network redesign and configuration change. For more information about bypass switch devices refer to https://en.wikipedia.org/wiki/Bypass_switch; for network packet brokers, refer to https://www.ixiacom.com/company/blog/network-packet-brokers-abcs-network-visibility and https://www.gigamon.com/campaigns/next-generation-network-packet-broker.html. The article series introduces network designs to forward traffic to the inline tools at layer 2 (L2). F5’s BIG-IP hardware appliances can be inserted in L2 networks. This can be achieved using either virtual Wire (vWire) or by bridging 2 Virtual LANs using a VLAN Groups. This document covers the design and implementation of the Gigamon Bypass Switch/Network Packet Broker in conjunction with the BIG-IP i5800 appliance and Virtual Wire (vWire). This document focuses on Gigamon Bypass Switch / Network Packet Broker. For more information about architecture overview of bypass switch and network packet broker refer to https://devcentral.f5.com/s/articles/L2-Deployment-of-vCMP-guest-with-Ixia-network-packet-broker?tab=series&page=1. Gigamon provides internal bypass switch within network packet broker device whereas Ixia has external bypass switch. Network Topology Below diagram is a representation of the actual lab network. This shows deployment of BIG-IP with Gigamon. Figure 1 - Topology before deployment of Gigamon and BIG-IP Figure 2 - Topology after deployment of Gigamon and BIG-IP Figure 3 - Connection between Gigamon and BIG-IP Hardware Specification Hardware used in this article are BIG-IP i5800 GigaVUE-HC1 Arista DCS-7010T-48 (all the four switches) Note: All the Interfaces/Ports are 1G speed Software Specification Software used in this article are BIG-IP 16.1.0 GigaVUE-OS 5.7.01 Arista 4.21.3F (North Switches) Arista 4.19.2F (South Switches) Gigamon Configuration In this lab, the Gigamon is configured with two type of ports, Inline Network and Inline Tool. Steps Summary Step 1 : Configure Port Type Step 2 : Configure Inline Network Bypass Pair Step 3 : Configure Inline Network Group (if applicable) Step 4 : Configure Inline Tool Pair Step 5 : Configure Inline Tool Group (if applicable) Step 6 : Configure Inline Traffic Flow Maps Step 1 : Configure Port Type First and Foremost step is to configure Ports. Figure 2 shows all the ports that are connected between Switches and Gigamon. Ports that are connected to switch should be configured as Inline Network Ports. As per Figure 2, find below Inline Network ports Inline Network ports: 1/1/x1, 1/1/x2, 1/1/x3, 1/1/x4, 1/1/x5. 1/1/x6, 1/1/x7, 1/1/x8 Figure 3 shows all the ports that are connected between BIG-IP and Gigamon. Ports that are connected to BIG-IP should be configured as Inline Tool Ports. As per Figure 3, find below Inline Tool ports Inline Tool ports: 1/1/x9, 1/1/x10, 1/1/x11, 1/1/x12, 1/1/g1, 1/1/g2, 1/1/g3, 1/1/g4 To configure Port Type, do the following Log into GigaVUE-HC1 GUI Select Ports -> Go to specific port and modify Port Type as Inline Network or Inline Tool Figure 4 - GUI configuration of Port Types Equivalent command for configuring Inline Network port and other port configuration port 1/1/x1 type inline-net port 1/1/x1 alias N-SW1-36 port 1/1/x1 params admin enable autoneg enable Equivalent command for configuring Inline Tool Port and other port configuration port 1/1/x9 type inline-tool port 1/1/x9 alias BIGIP-1.1 port 1/1/x9 params admin enable autoneg enable Step 2 : Configure Inline Network Bypass Pair Figure 1 shows direct connections between switches. An inline network bypass pair will ensure the same connections through Gigamon. An inline network is an arrangement of two ports of the inline-network type. The arrangement facilitates access to a bidirectional link between two networks (two far-end network devices) that need to be linked through an inline tool. As per Figure 2, find below Inline Network bypass pairs Inline Network bypass pair 1 : 1/1/x1 -> 1/1/x2 Inline Network bypass pair 2 : 1/1/x3 -> 1/1/x4 Inline Network bypass pair 3 : 1/1/x5 -> 1/1/x6 Inline Network bypass pair 4 : 1/1/x7 -> 1/1/x8 To configure the inline network bypass pair, do the following Log into GigaVUE-HC1 GUI Select Inline Bypass -> Inline Networks Figure 5 - Example GUI configuration of Inline Network Bypass Pair Equivalent command for configuring Inline Network Bypass Pair inline-network alias Bypass1 pair net-a 1/1/x1 and net-b 1/1/x2 physical-bypass disable traffic-path to-inline-tool Step 3 : Configure Inline Network Group An inline network group is an arrangement of multiple inline networks that share the same inline tool. To configure the inline network bypass group, do the following Log into GigaVUE-HC1 GUI Select Inline Bypass -> Inline Networks Groups Figure 6 - Example GUI configuration of Inline Network Bypass Group Equivalent command for configuring Inline Network Bypass Group inline-network-group alias Bypassgroup network-list Bypass1,Bypass2,Bypass3,Bypass4 Step 4 : Configure Inline Tool Pair Figure 3 shows connection between BIG-IP and Gigamon which will be in pairs. An inline tool consists of inline tool ports, always in pairs, running at the same speed, on the same medium. As per Figure 3, find below Inline Tool pairs. Inline Network bypass pair 1 : 1/1/x9 -> 1/1/x10 Inline Network bypass pair 2 : 1/1/x11 -> 1/1/x12 Inline Network bypass pair 3 : 1/1/g1 -> 1/1/g2 Inline Network bypass pair 4 : 1/1/g3 -> 1/1/g4 To configure the inline tool pair, do the following Log into GigaVUE-HC1 GUI Select Inline Bypass -> Inline Tools Figure 7 - Example GUI configuration of Inline Tool Pair Equivalent command for configuring Inline Tool pair inline-tool alias BIGIP1 pair tool-a 1/1/x9 and tool-b 1/1/x10 enable shared true Step 5 : Configure Inline Tool Group (if applicable) An inline tool group is an arrangement of multiple inline tools to which traffic is distributed to the inline tools based on hardware-calculated hash values. For example, if one tool goes down, traffic is redistributed to other tools in the group using hashing. To configure the inline tool group, do the following Log into GigaVUE-HC1 GUI Select Inline Bypass -> Inline Tool Groups Figure 8 - Example GUI configuration of Inline Tool Group Equivalent command for configuring Inline Tool Group inline-tool-group alias BIGIPgroup tool-list BIGIP1,BIGIP2,BIGIP3,BIGIP4 enable Step 6 : Configure Inline Traffic Flow Maps Flow mapping takes traffic from a network TAP or a SPAN/mirror port and sends it through a set of user-defined map rules to the tools and applications that secure, monitor and analyze IT infrastructure. As per Figure 2, it is the high-level process for configuring traffic to flow from the inline network links to the inline tool group, allowing you to test the deployment functionality of the BIG-IP appliances within the group. To configure the inline tool group, do the following Log into GigaVUE-HC1 GUI Select Maps -> New Figure 9 - Example GUI configuration of Flow Maps Note: Above configuration allows all traffic from Inline Network Group to flow through Inline Tool Group Equivalent command for configuring PASS ALL Flow Map map-passall alias Map1 to BIGIPgroup from Bypassgroup Flow Maps can be configured specific to certain traffic. For example, If LACP traffic should bypass BIG-IP and all other traffic should pass through BIG-IP. Find below command to achieve mentioned condition map alias inMap type inline byRule roles replace admin to owner_roles comment " " rule add pass ethertype 8809 to bypass from Bypassgroup exit map-scollector alias SCollector roles replace admin to owner_roles from Bypassgroup collector BIGIPgroup exit Note: For more details on Gigamon, refer https://docs.gigamon.com/pdfs/Content/Shared/5700-doclist.html BIG-IP Configuration In series of BIG-IP and Gigamon deployment, BIG-IP configured in L2 mode with Virtual Wire (vWire) Step Summary Step 1 : Configure interfaces to support vWire Step 2 : Configure trunk in LACP mode or passthrough mode Step 3 : Configure Virtual Wire Note: Steps mentioned above are specific to topology in Figure 2. For more details on Virtual Wire (vWire), refer https://devcentral.f5.com/s/articles/BIG-IP-vWire-Configuration?tab=series&page=1 and https://devcentral.f5.com/s/articles/vWire-Deployment-Configuration-and-Troubleshooting?tab=series&page=1 Step 1 : Configure interfaces to support vWire To configure interfaces to support vWire, do the following Log into BIG-IP GUI Select Network -> Interfaces -> Interface List Select Specific Interface and in vWire configuration, select Virtual Wire as Forwarding Mode Figure 10 - Example GUI configuration of interface to support vWire Step 2 : Configure trunk in LACP mode or passthrough mode To configure trunk, do the following Log into BIG-IP GUI Select Network -> Trunks Click Create to configure new Trunk. Enable LACP for LACP mode and disable LACP for LACP passthrough mode Figure 11 - Example GUI configuration of Trunk in LACP Mode Figure 12 - Example GUI configuration of Trunk in LACP Passthrough Mode As per Figure 2, when configured in LACP Mode, LACP will be established between BIG-IP and switches. When configured in LACP passthrough mode, LACP will be established between North and South Switches. As per Figure 2 and 3 , there will be four trunk configured as below, Left_Trunk 1 : Interfaces 1.1 and 2.3 Left_Trunk 2 : Interfaces 1.3 and 2.1 Right_Trunk 1 : Interfaces 1.2 and 2.4 Right_Trunk 2 : Interfaces 1.4 and 2.2 Left_Trunk ensure connectivity between BIG-IP and North Switches. Right_Trunk ensure connectivity between BIG-IP and South Switches. Note: Trunks can be configured for individual interfaces, if LACP passthrough configured as LACP frames not getting terminated at BIG-IP Step 3 : Configure Virtual Wire To configure trunk, do the following Log into BIG-IP GUI Select Network -> Virtual Wire Click Create to configure Virtual Wire Figure 13 - Example GUI configuration of Virtual Wire Above Virtual Wire configuration will work for both Tagged and Untagged traffic. Figure 2 and 3, requires both the Virtual Wire configured. This configuration works for both LACP mode and LACP passthrough mode. If each interface configured with specific trunk in passthrough deployment, then there will be 4 specific Virtual Wires configured. Note: In this series, all the mentioned scenarios and configuration will be covered in upcoming articles. Conclusion This deployment ensures transparent integration of network security tools with little to no network redesign and configuration change. The Merits of above network deployment are Increases reliability of Production link Inline devices can be upgraded or replaced without loss of the link Traffic can be shared between multiple tools Specific Traffic can be forwarded to customized tools Trusted Traffic can be Bypassed un-inspectedBIG-IP L2 Virtual Wire LACP Passthrough Deployment with IXIA Bypass Switch and Network Packet Broker (Single Service Chain - Active / Active)
Introduction This article is part of a series on deploying BIG-IPs with bypass switches and network packet brokers. These devices allow for the transparent integration of network security tools with little to no network redesign and configuration change. For more information about bypass switch devices refer tohttps://en.wikipedia.org/wiki/Bypass_switch; for network packet brokers, refer tohttps://www.ixiacom.com/company/blog/network-packet-brokers-abcs-network-visibilityandhttps://www.gigamon.com/campaigns/next-generation-network-packet-broker.html. The article series introduces network designs to forward traffic to the inline tools at layer 2 (L2). F5’s BIG-IP hardware appliances can be inserted in L2 networks. This can be achieved using either virtual Wire (vWire) or by bridging 2 Virtual LANs using a VLAN Groups. This document covers the design and implementation of the IXIA Bypass Switch/Network Packet Broker in conjunction with the BIG-IP i5800 appliance and Virtual Wire (vWire). This document focus on IXIA Bypass Switch / Network Packet Broker. For more information about architecture overview of bypass switch and network packet broker refer tohttps://devcentral.f5.com/s/articles/L2-Deployment-of-vCMP-guest-with-Ixia-network-packet-broker?tab=series&page=1. This article is continuation of https://devcentral.f5.com/s/articles/BIG-IP-L2-Deployment-with-Bypasss-Network-Packet-Broker-and-LACP?tab=series&page=1 with latest versions of BIG-IP and IXIA Devices. Also focused on various combination of configurations in BIG-IP and IXIA devices. Network Topology Below diagram is a representation of the actual lab network. This shows deployment of BIG-IP with IXIA Bypass Switch and Network Packet Broker. Figure 1 - Deployment of BIG-IP with IXIA Bypass Switch and Network Packet Broker Please refer Lab Overview section in https://devcentral.f5.com/s/articles/BIG-IP-L2-Deployment-with-Bypasss-Network-Packet-Broker-and-LACP?tab=series&page=1 for more insights on lab topology and connections. Hardware Specification Hardware used in this article are IXIA iBypass DUO ( Bypass Switch) IXIA Vision E40 (Network Packet Broker) BIG-IP Arista DCS-7010T-48 (all the four switches) Software Specification Software used in this article are BIG-IP 16.1.0 IXIA iBypass DUO 1.4.1 IXIA Vision E40 5.9.1.8 Arista 4.21.3F (North Switches) Arista 4.19.2F (South Switches) Switch Configuration LAG or link aggregation is a way of bonding multiple physical links into a combined logical link. MLAG or multi-chassis link aggregation extends this capability allowing a downstream switch or host to connect to two switches configured as an MLAG domain. This provides redundancy by giving the downstream switch or host two uplink paths as well as full bandwidth utilization since the MLAG domain appears to be a single switch to Spanning Tree (STP). Lab Overview section in https://devcentral.f5.com/s/articles/BIG-IP-L2-Deployment-with-Bypasss-Network-Packet-Broker-and-LACP?tab=series&page=1shows MLAG configuring in both the switches. This article focus on LACP deployment for tagged packets. For more details on MLAG configuration, refer tohttps://eos.arista.com/mlag-basic-configuration/#Verify_MLAG_operation Step Summary Step 1 : Configuration of MLAG peering between both the North Switches Step 2 : Verify MLAG Peering in North Switches Step 3 : Configuration of MLAG Port-Channels in North Switches Step 4 : Configuration of MLAG peering between both the South Switches Step 5 : Verify MLAG Peering in South Switches Step 6 : Configuration of MLAG Port-Channels in South Switches Step 7 : Verify Port-Channel Status Step 1 : Configuration of MLAG peering between both the North Switches MLAG Configuration in North Switch1 and North Switch2 are as follows North Switch 1: Configure Port-Channel interface Port-Channel10 switchport mode trunk switchport trunk group m1peer Configure VLAN interface Vlan4094 ip address 172.16.0.1/30 Configure MLAG mlag configuration domain-id mlag1 heartbeat-interval 2500 local-interface Vlan4094 peer-address 172.16.0.2 peer-link Port-Channel10 reload-delay 150 North Switch 2: Configure Port-Channel interface Port-Channel10 switchport mode trunk switchport trunk group m1peer Configure VLAN interface Vlan4094 ip address 172.16.0.2/30 Configure MLAG mlag configuration domain-id mlag1 heartbeat-interval 2500 local-interface Vlan4094 peer-address 172.16.0.1 peer-link Port-Channel10 reload-delay 150 Step 2 : Verify MLAG Peering in North Switches North Switch 1: North-1#show mlag MLAG Configuration: domain-id:mlag1 local-interface:Vlan4094 peer-address:172.16.0.2 peer-link:Port-Channel10 peer-config : consistent MLAG Status: state:Active negotiation status:Connected peer-link status:Up local-int status:Up system-id:2a:99:3a:23:94:c7 dual-primary detection :Disabled MLAG Ports: Disabled:0 Configured:0 Inactive:6 Active-partial:0 Active-full:2 North Switch 2: North-2#show mlag MLAG Configuration: domain-id:mlag1 local-interface:Vlan4094 peer-address:172.16.0.1 peer-link:Port-Channel10 peer-config : consistent MLAG Status: state:Active negotiation status:Connected peer-link status:Up local-int status:Up system-id:2a:99:3a:23:94:c7 dual-primary detection :Disabled MLAG Ports: Disabled:0 Configured:0 Inactive:6 Active-partial:0 Active-full:2 Step 3 : Configuration of MLAG Port-Channels in North Switches North Switch 1: interface Port-Channel513 switchport trunk allowed vlan 513 switchport mode trunk mlag 513 interface Ethernet50 channel-group 513 mode active North Switch 2: interface Port-Channel513 switchport trunk allowed vlan 513 switchport mode trunk mlag 513 interface Ethernet50 channel-group 513 mode active Step 4 : Configuration of MLAG peering between both the South Switches MLAG Configuration in South Switch1 and South Switch2 are as follows South Switch 1: Configure Port-Channel interface Port-Channel10 switchport mode trunk switchport trunk group m1peer Configure VLAN interface Vlan4094 ip address 172.16.1.1/30 Configure MLAG mlag configuration domain-id mlag1 heartbeat-interval 2500 local-interface Vlan4094 peer-address 172.16.1.2 peer-link Port-Channel10 reload-delay 150 South Switch 2: Configure Port-Channel interface Port-Channel10 switchport mode trunk switchport trunk group m1peer Configure VLAN interface Vlan4094 ip address 172.16.1.2/30 Configure MLAG mlag configuration domain-id mlag1 heartbeat-interval 2500 local-interface Vlan4094 peer-address 172.16.1.1 peer-link Port-Channel10 reload-delay 150 Step 5 : Verify MLAG Peering in South Switches South Switch 1: South-1#show mlag MLAG Configuration: domain-id : mlag1 local-interface : Vlan4094 peer-address : 172.16.1.2 peer-link : Port-Channel10 peer-config : consistent MLAG Status: state : Active negotiation status : Connected peer-link status : Up local-int status : Up system-id : 2a:99:3a:48:78:d7 MLAG Ports: Disabled : 0 Configured : 0 Inactive : 6 Active-partial : 0 Active-full : 2 South Switch 2: South-2#show mlag MLAG Configuration: domain-id : mlag1 local-interface : Vlan4094 peer-address : 172.16.1.1 peer-link : Port-Channel10 peer-config : consistent MLAG Status: state : Active negotiation status : Connected peer-link status : Up local-int status : Up system-id : 2a:99:3a:48:78:d7 MLAG Ports: Disabled : 0 Configured : 0 Inactive : 6 Active-partial : 0 Active-full : 2 Step 6 : Configuration of MLAG Port-Channels in South Switches South Switch 1: interface Port-Channel513 switchport trunk allowed vlan 513 switchport mode trunk mlag 513 interface Ethernet50 channel-group 513 mode active South Switch 2: interface Port-Channel513 switchport trunk allowed vlan 513 switchport mode trunk mlag 513 interface Ethernet50 channel-group 513 mode active LACP modes are as follows On Active Passive LACP Connection establishment will occur only for below configurations Active in both North and South Switch Active in North or South Switch and Passive in other switch On in both North and South Switch Note: In this case, all the interfaces of both North and South Switches are configured with LACP mode as Active. Step 7 : Verify Port-Channel Status North Switch 1: North-1#show mlag interfaces detail local/remote mlag state local remote oper config last change changes ---------- ----------------- ----------- ------------ --------------- ------------- --------------------------- ------- 513 active-full Po513 Po513 up/up ena/ena 4 days, 0:34:28 ago 198 North Switch 2: North-2#show mlag interfaces detail local/remote mlag state local remote oper config last change changes ---------- ----------------- ----------- ------------ --------------- ------------- --------------------------- ------- 513 active-full Po513 Po513 up/up ena/ena 4 days, 0:35:58 ago 198 South Switch 1: South-1#show mlag interfaces detail local/remote mlag state local remote oper config last change changes ---------- ----------------- ----------- ------------ --------------- ------------- --------------------------- ------- 513 active-full Po513 Po513 up/up ena/ena 4 days, 0:36:04 ago 190 South Switch 2: South-2#show mlag interfaces detail local/remote mlag state local remote oper config last change changes ---------- ----------------- ----------- ------------ --------------- ------------- --------------------------- ------- 513 active-full Po513 Po513 up/up ena/ena 4 days, 0:36:02 ago 192 Ixia iBypass Duo Configuration For detailed insight, refer to IXIA iBypass Duo Configuration section in https://devcentral.f5.com/s/articles/L2-Deployment-of-vCMP-guest-with-Ixia-network-packet-broker?page=1 Figure 2 - Configuration of iBypass Duo (Bypass Switch) Heartbeat Configuration Heartbeats are configured on both bypass switches to monitor tools in their primary path and secondary paths. If a tool failure is detected, the bypass switch forwards traffic to the secondary path. Heartbeat can be configured using multiple protocols, here Bypass switch 1 uses DNS and Bypass Switch 2 uses IPX for Heartbeat. Figure 3 - Heartbeat Configuration of Bypass Switch 1 ( DNS Heartbeat ) In this infrastructure, the VLAN ID is 513 and represented as hex 0201. Figure 4 - VLAN Representation in Heartbeat Figure 5 - Heartbeat Configuration of Bypass Switch 1 ( B Side ) Figure 6 - Heartbeat Configuration of Bypass Switch 2 ( IPX Heartbeat ) Figure 7 - Heartbeat Configuration of Bypass Switch 2 ( B Side ) IXIA Vision E40 Configuration Create the following resources with the information provided. Bypass Port Pairs Inline Tool Pair Service Chains Figure 8 - Configuration of Vision E40 ( NPB ) This articles focus on deployment of Network Packet Broker with single service chain whereas previous article is based on 2 service chain. Figure 9 - Configuration of Tool Resources In Single Tool Resource, 2 Inline Tool Pairs configured which allows to configure both the Bypass Port pair with single Service Chain. Figure 10 - Configuration of VLAN Translation From Switch Configuration, Source VLAN is 513 and it will be translated to 2001 and 2002 for Bypass 1 and Bypass 2 respectively. For more insights with respect to VLAN translation, refer https://devcentral.f5.com/s/articles/L2-Deployment-of-vCMP-guest-with-Ixia-network-packet-broker?page=1 For Tagged Packets, VLAN translation should be enabled. LACP frames will be untagged which should be bypassed and routed to other Port-Channel. In this case LACP traffic will not reach BIG-IP, instead it will get routed directly from NPB to other pair of switches. LACP bypass Configuration The network packet broker is configured to forward (or bypass) the LACP frames directly from the north to the south switch and vice versa.LACP frames bear the ethertype 8809 (in hex). This filter is configured during the Bypass Port Pair configuration. Note: There are methods to configure this filter, with the use of service chains and filters but this is the simplest for this deployment. Figure 11 - Configuration to redirect LACP BIG-IP Configuration Step Summary Step 1 : Configure interfaces to support vWire Step 2 : Configure trunk in passthrough mode Step 3 : Configure Virtual Wire Note: Steps mentioned above are specific to topology inFigure 2.For more details on Virtual Wire (vWire), referhttps://devcentral.f5.com/s/articles/BIG-IP-vWire-Configuration?tab=series&page=1andhttps://devcentral.f5.com/s/articles/vWire-Deployment-Configuration-and-Troubleshooting?tab=series&page=1 Step 1 : Configure interfaces to support vWire To configure interfaces to support vWire, do the following Log into BIG-IP GUI SelectNetwork -> Interfaces -> Interface List Select Specific Interface and in vWire configuration, select Virtual Wire as Forwarding Mode Figure 12 - Example GUI configuration of interface to support vWire Step 2 : Configure trunk in passthrough mode To configure trunk, do the following Log into BIG-IP GUI SelectNetwork -> Trunks ClickCreateto configure new Trunk. Disable LACP for LACP passthrough mode Figure 13 - Configuration of North Trunk in Passthrough Mode Figure 14 - Configuration of South Trunk in Passthrough Mode Step 3 : Configure Virtual Wire To configure trunk, do the following Log into BIG-IP GUI SelectNetwork -> Virtual Wire ClickCreateto configure Virtual Wire Figure 15 - Configuration of Virtual Wire As VLAN 513 is translated into 2001 and 2002, vWire configured with explicit tagged VLANs. It is also recommended to have untagged VLAN in vWire to allow any untagged traffic. Enable multicast bridging sys db variable as below for LACP passthrough mode modify sys db l2.virtualwire.multicast.bridging value enable Note: Make sure sys db variable enabled after reboot and upgrade. For LACP mode, multicast bridging sys db variable should be disabled. Scenarios As LACP passthrough mode configured in BIG-IP, LACP frames will passthrough BIG-IP. LACP will be established between North and South Switches.ICMP traffic is used to represent network traffic from the north switches to the south switches. Scenario 1: Traffic flow through BIG-IP with North and South Switches configured in LACP active mode Above configurations shows that all the four switches are configured with LACP active mode. Figure 16 - MLAG after deployment of BIG-IP and IXIA with Switches configured in LACP ACTIVE mode Figure 16shows that port-channels 513 is active at both North Switches and South Switches. Figure 17 - ICMP traffic flow from client to server through BIG-IP Figure 17shows ICMP is reachable from client to server through BIG-IP. This verifies test case 1, LACP getting established between Switches and traffic passthrough BIG-IP successfully. Scenario 2: Active BIG-IP link goes down with link state propagation enabled in BIG-IP Figure 15shows Propagate Virtual Wire Link Status enabled in BIG-IP. Figure 17shows that interface 1.1 of BIG-IP is active incoming interface and interface 1.4 of BIG-IP is active outgoing interface. Disabling BIG-IP interface 1.1 will make active link down as below Figure 18 - BIG-IP interface 1.1 disabled Figure 19 - Trunk state after BIG-IP interface 1.1 disabled Figure 19shows that the trunks are up even though interface 1.1 is down. As per configuration, North_Trunk has 2 interfaces connected to it 1.1 and 1.3 and one of the interface is still up, so North_Trunk status is active. Figure 20 - MLAG status with interface 1.1 down and Link State Propagation enabled Figure 20shows that port-channel 513 is active at both North Switches and South Switches. This shows that switches are not aware of link failure and it is been handled by IXIA configuration. Figure 21 - IXIA Bypass Switch after 1.1 interface of BIG-IP goes down As shown in Figure 8, Single Service Chain is configured and which will be down only if both Inline Tool Port pairs are down in NPB. So Bypass will be enabled only if Service Chain goes down in NPB. Figure 21 shows that still Bypass is not enabled in IXIA Bypass Switch. Figure 22 - Service Chain and Inline Tool Port Pair status in IXIA Vision E40 ( NPB ) Figure 22 shows that Service Chain is still up as BIG IP2 ( Inline Tool Port Pair ) is up whereas BIG IP1 is down. Figure 1 shows that P09 of NPB is connected 1.1 of BIG-IP which is down. Figure 23 - ICMP traffic flow from client to server through BIG-IP Figure 23 shows that still traffic flows through BIG-IP even though 1.1 interface of BIG-IP is down. Now active incoming interface is 1.3 and active outgoing interface is 1.4. Low bandwidth traffic is still allowed through BIG-IP as bypass not enabled and IXIA handles rate limit process. Scenario 3: When North_Trunk goes down with link state propagation enabled in BIG-IP Figure 24 - BIG-IP interface 1.1 and 1.3 disabled Figure 25 - Trunk state after BIG-IP interface 1.1 and 1.3 disabled Figure 15 shows that Propagate Virtual Wire Link State enabled and thus both the trunks are down. Figure 26 - IXIA Bypass Switch after 1.1 and 1.3 interfaces of BIG-IP goes down Figure 27 - ICMP traffic flow from client to server bypassing BIG-IP Conclusion This article covers BIG-IP L2 Virtual Wire Passthrough deployment with IXIA. IXIA configured using Single Service Chain. Observations of this deployment are as below VLAN Translation in IXIA NPB will convert real VLAN ID (513) to Translated VLAN ID (2001 and 2002) BIG-IP will receive packets with translated VLAN ID (2001 and 2002) VLAN Translation needs all packets to be tagged, untagged packets will be dropped. LACP frames are untagged and thus bypass configured in NPB for LACP. Tool Sharing needs to be enabled for allowing untagged packet which will add extra tag. This type of configuration and testing will be covered in upcoming articles. With Single Service Chain, If any one of the Inline Tool Port Pairs goes down, low bandwidth traffic will be still allowed to pass through BIG-IP (tool) If any of the Inline Tool link goes down, IXIA handles whether to bypass or rate limit. Switches will be still unaware of the changes. With Single Service Chain, if Tool resource configured with both Inline Tool Port pair in Active - Active state then load balancing will happen and both path will be active at a point of time. Multiple Service Chains in IXIA NPB can be used instead of Single Service Chain to remove rate limit process. This type of configuration and testing will be covered in upcoming articles. If BIG-IP goes down, IXIA enables bypass and ensures there is no packet drop.
