F5 Supporting Our Technical Community During the COVID-19 Outbreak
Our community health is always a top priority. That priority extends to all of you who support each other every day here at DevCentral. We're a global community and we know many of you are directly affected by the COVID-19 pandemic and we want to help. Many of us are now required to work from home, and for some of us that's hard to do. The last thing we want you to worry about is technical issues. Speaking with several of you and talking to support and our teams out in the field answering your questions, we're busy gathering content that will help us all during this trying time. Our Support During the Outbreak AskF5 K70811681: F5 response to the global impact of coronavirus - F5 Support published their policy March 4th and our ability to support you remains unaffected. We strive to meet our stringent business continuity management plans to provide you with the service you've come to expect from F5 even during events like this. Troubleshooting and Support for F5 Remote Access Solutions Finding out the limits of your configuration or license during unplanned global issues is stressful to say the least. To help you troubleshoot and get started resolving those issues we compiled the below list based on your questions. AskF5 K21883200: Emerging issues you may experience during the COVID-19 outbreak - Compiled from the incoming support calls received, this will be your best ongoing source of top issues our users are running into with the increased load for remote access functionality. If you're having BIG-IP APM performance issues, start here! AskF5 K20775035: BIG-IP APM Operations Guide - This is the a great place to start if you haven't implemented APM. Consider this your field guide. It provides topics ranging from end user clients, configuration examples, to the ever-important Chapter 10: Troubleshooting. AskF5 K05847240: Troubleshooting BIG-IP APM Networks Access issues related to lease pools - This has been a popular question where lease pools are running out because so many people are connecting in and saturating available IP space. AskF5 K7752: Licensing the BIG-IP System - If you received a new license to bump up your APM client count, here's how install your new license. F5.com: Configuring the BIG-IP APM as a SAML 2.0 Identity Provider for Common SaaS Applications - Long title I know but if you're scrambling to get some federated access up for your systems start here. From the Field With everyone working remotely the need for additional BIG-IP APM answers and solutions is evident. It may be increasing DHCP lease pools for your users, installing a new license to increase your APM client count, or using Per-App VPN App Tunnels, we're working on getting you the information you need. Determining if you're licensed for Remote Access Capabilities - Many of you may have a license where APM and client access licenes are provisioned but you're just not using them. This article will determine if you're ready to start configuring APM for your remote workers. Responding to the Coronavirus - Six Ways to Improve App Availability - It's not just remote access issues people are running into. Ensure your apps are tuned and working for any heightened traffic from the increased remote working requirements. AskF5: Customizing BIG-IP APM access policy error messages - We've heard a lot about this. If you don't have good error messaging pages your user won't know if they entered a bad password or failed the client policy check. Creating an SSL VPN Using F5 Full Webtop - For those of you who have Access Policy Manager (APM) licensed but haven't configured it yet, here's a great starting point. There's a big spike in traffic for this article so we'll make sure it's in your list of to-read content Azure Active Directory and BIG-IP APM Integration - Ease identity and access management by integrating your cloud directory structure. Another high traffic article for people needing help with integrating quickly. Configuring a Per-App VPN Using F5 App Tunnels - Taking the burden off your corporate infrastructure by allowing VPN to a single app. Not many of us need full VPN tunnels so why waste your bandwidth. AskF5 K16680: VoIP through Network Access connections - Our engineering support team asked us to include this AskF5 KB based off the increase in customer questions related to VoIP via VPN. AskF5 K12524516: APM Network Access (VPN) compression causes higher CPU usage - We're also seeing in uptick in customers calling in on this issue. If you're experiencing performance issues with BIG-IP APM, check here for a quick resolution. What to do if you're experiencing an attack? The darker side is malicious users are taking advantage of the business upheaval and trying out new attack vectors and old favorites too. If you're trying to balance out the needs of your user's remote issues, let us help you with managing the influx of bad actors. Contact our F5 Security Incident Response Team (SIRT) . They're here and ready to help. Contact Us We will continue to update this document as we gain more insights from the field. As always if you have questions please login (or sign up if you're new) and hop on over to the DevCentral Q&A where our community of F5 technical professionals are happy to assist. And if you found something useful that helped you manage your remote workers tell us! We'll be happy to spread the news. It's amazing to see the support everyone provides during these difficult times and we're always proud to work and support you.Hitting the Easy Button: Securing the Remote Desktop on F5 BIG-IP APM
Being able to provide the most effective remote access solution is critical, especially in these turbulent times. In this article printed with permission from authors Lucas Thompson & Michael Waechter, we're going to talk about Remote Desktop Web Access. Solution Brief In short, it enables end-users to access their Remote Desktop applications through the F5 APM Webtop. The benefits of utilizing Remote Desktop Web Access over a desktop can be many. With the only requirement being a compatible web browser, Microsoft RDP application (comes installed with all modern versions of Windows), and a backend server hosting the applications… the solution speaks for itself. When the Full Webtop is displayed, APM will fetch a list of RemoteApps available on the target Terminal Server via HTTPS (using the Server SSL profile on the APM virtual server) and the associated icons. They will then be presented to the end user. The connection is done over HTTPS to the APM, and APM uses RDP (port 3389) to the Terminal Server. In the classic Terminal Server Desktop use case, the user is assigned a ‘native’ type RDP resource. This icon is presented to the user on a Full Webtop. Access is made by selecting the icon. A .RDP file is downloaded to the end user client PC, and the browser will activate the OS’s native RDP client to proceed with the connection. The connection is done over HTTPS to the APM, and APM uses RDP (port 3389) to the Terminal Server. Not only is the client setup simple, but the administration part of the equation is equally as easy to enable. I’m running version 14.1 (LTS) and here are a few screenshots of the setup. To enable the solution, let’s click on Access -> Connectivity/VPN Go ahead and choose VDI/RDP -> Remote Desktops Add the relative information. (Note: It’s always best to have the host name be a FQDN, and add this as a LTM node for health monitoring.) Technical Workflow The user clicks a resource icon on the Full Webtop, an RDP file is downloaded and then executed by the TS client on the user’s PC. The RemoteApp use case has a few differences versus targeting a desktop, or terminal server directly. In this case APM Will: Obtain a list of the RD feeds. The list of RDP App Resources will be derived from the RemoteApp feed. The list of icons will be delivered to the end user’s browser. The end user’s browser will request the icon pictures via a proxy mechanism in the VDI module. Because the RemoteApp feed comes through HTTPS and IIS on the Terminal Server, we have to make sure that: BIG-IP data plane can route to the Terminal Server. BIG-IP can create a HTTPS connection to Terminal Server. Terminal Server is rendering the page correctly. When you browse to it (https://terminalserver/rdweb/) you should see something like this: 2008: 2012: Authenticate with the same credentials that the test user uses in APM, and you should see an App Feed or desktop feed: Solution FAQ What kind of licenses are used for RDP access? APM has two license types: CCU and Access Session. Access Sessions are used for each established session ID. CCU are used for Network VPNs and other things that require more advanced features. Native Mode RDP does NOT use a CCU (connectivity) license. Only a single Access Session license will be consumed by a connecting user. What RDP options are supported? All of them. They’re basically echoed back to the client in the .RDP file. Put your desired parameter into the Custom Parameters area. It’s OK to use session variables in %{session.variable} format as well. RDP Custom Parameters configuration Lists of the RDP options have been compiled by 3rd parties, including the one at https://www.donkz.nl/overview-rdp-file-settings/ which is quite comprehensive. Please note that the following options are reserved for APM RDG use. If you attempt to apply these custom parameters, they will be ignored and/or overwritten by APM: Gatewayusagemethod Gatewayprofileusagemethod Gatewayhostname Gatewaycredentialssource Gatewayaccesstoken authentication level full address server port enablecredsspsupport signscope signature prompt for credentials on client domain username alternate full address gatewaybrokeringtype RDP Window Title The maximized window title for MSTSC inherits the target device name (not the RD gateway host). The medium-sized window title for MSTSC inherits the RDP filename (which is always “launch” -- see RFE 610244). One interesting thing that is possible is to internally-redirect the RDP session so that the client THINKS its connecting to one site, but then re-assign the remote host variable to a different site during the RAP access policy execution. RemoteApps It’s possible to create a lot of apps by using a PowerShell script on a RemoteApp-enabled terminal server. Client Requirements Microsoft Remote Desktop Client is supported for both Windows and Mac. Because the protocol used utilizes the Remote Desktop Gateway functionality, only newer RDP clients work. Legacy clients will likely not be able to create connections. iOS/Android The latest iOS / Android App Store RDP clients from Microsoft are supported. There might be some version conflicts, but for the most part the latest and greatest will work Reconnections / Disruptions Reconnections work the same as normal RDP If the user disconnects and reconnects, the session will be resumed. The client instructs the RD Gateway (APM) to again establish the session. The Remote Desktop session will be resumed also, the same way as with normal RDP. If the session is deleted or timed out or otherwise destroyed, the connection will stop, RDP will try to reconnect, but it will fail, and you will see this message from the MSTSC client.
COVID-19; Lessons from Security Incident Response
For the past few decades, threats of an 'epidemic' or 'pandemic' nature have loomed over digital assets and infrastructures. Do you remember the DDoS attack in 2002 that targeted a dozen of DNS root servers in the US and almost brought the Internet to its knees? What about the ILOVEYOU virus, which affected more than 10% of the world’s computers and caused an estimated $10 billion worth of damages? Essentially, any zero-day attack targetingthe core internet infrastructure and popular applications is potentially disastrous. The risk is even higher given the impressive volume and frequency of threats (an attack occurs every 39 seconds, on average 2,244 times a day, according to University of Maryland). As a result, security professionals have enhanced their security incident response (SIR) mechanisms. With slight variations, SIRs follow the guidanceof NIST SP 800-61 and generallyconsist of four phases: preparation; detection and analysis; containment, eradication and recovery; and post-incident activity. As the world responds to COVID-19, what can we learn from SIR? Early detection In SIR, as with COVID-19, precursors on a subject (clues that an incident may occur in the future) are difficult to identify. It is difficultto detect a potential COVID-19 patient untilhe starts exhibitingthe symptoms. The good news is that COVID-19 is easily detectable. Indicators such as symptoms and abnormal behaviorson human subjects are well known. However, spotting an incident early is essential to mitigate it effects. In AppSec, traffic is continuously monitored and inspected 24/7 in real time, using rules-based and anomaly-based detection to detect traffic posing a threat. Artificial intelligence (AI) and machine learning (ML) augment detection by improving accuracyrates while reducing false positives. Similarly, deploying significant efforts in early detection of COVID-19 patients. A higher capacity to monitor the population for COVID-19 symptoms (analogy of rules-based detection) can lead to early detection. Early Containment Once a threat is identified, it needs to be contained. Containmentis a mitigation strategy enacted while a permanent fix is being develop. The main goal of containment is to reduce the speed of contamination by isolating affected subjects. My coworker, Raymond Pompon, has illustrated containment strategies similarities between SIR and the COVID-19 response inContainment is Never Perfect. Despite the residual risk, as with early detection, early containment is essential at reducing the attack surface. Moreover, containment provides an environment for information gathering in point- and contextual-threat analysis. In that regards, SIR strategies includes sandboxes and honeypots systems to aid further threat analysis. Tightening Security Posture As a threat is identified and containment strategies are implemented, when facing a looming threat, it is common practice in SIR to perform risk assessment and review and enhance the security posture of non-infected systems. Even when a permanent fix is not yet available, a looming threat imposes the need for a review of the security architecture and processes to identify and mitigate possible inflections points, threat actors, and attack vectors. With COVID-19, similar process is being observed and should be encouraged because organizations and households are reviewing theirprotocols, hygiene, and safety policies. Communication Plan In SIR as with the COVID-19, managing communication is a big challenge. To quote World Health Organization Director-General Tedros Adhanom Ghebreyesus, "Our greatest enemy right now is not the virus itself; it's fear, rumors,and stigma." Large organizations concerned for theirreputation have developedspecific security incident communication plan that reflects the nature, scope, risk, and impact of an attack. Communications are typically delivered by security leadership in the organization to stakeholdersfollowing the guidance of transparency. Special considerationare taken when a communication could be use for reverse engineering and be detrimental to the organization. However, an interesting model is the way Vulnerability Disclosure operates in computer security. An independentresearcher or ethical hacker not affiliated with an organization could discover a threat or vulnerability and report it directly to the affected organization or through a bounty program. Using such communication channel, an organization can take mitigation action. In SIR, as with COVID-19, a collaborative communication approach could hep in early detection, early containment, and tightening of the security posture.How AI Will Automate Cybersecurity in the Post-COVID World
Widespread remote working is accelerating the trend of digitization in society and a derivative trend of this acceleration is our increased reliance on online applications - which also means cybercrime is becoming more lucrative. Over on F5 Labs, Shuman Ghosemajumder briefly introduces the problem space and links to an article on VentureBeat about how AI will Automate cybersecurity in a Post-Covid world. https://www.f5.com/labs/articles/bylines/how-ai-will-automate-cybersecurity-in-the-post-covid-world
Rate Limiting SSL VPN User Traffic
With lots of people working at home, contention on VPNs is a real problem at the moment - license capacity, device CPU and throughput rate. One way to deal with this is to apply rate limits to user traffic. This can be done in a number of ways - applying a BWC policy in the Access Policy, using Traffic Classifiers, etc but I like simple solutions so i'm going to show you how to do it with virtual servers and iRules, and to take the easy way out you can use my iApp to do it for you! For a start, let's look at an SSL VPN in a bit more detail. Tunnels! The tunnel part of the SSL VPN is based around the Connectivity Profile - this specifies settings like compression and VPN settings. When you create a Connectivity Profile, this also creates a tunnel interface This tunnel interface is used as an internal connector so that outgoing traffic can be managed - the same function is used with HTTP explicit proxies which use http-tunnel. This is a very powerful feature - this means that we can create a virtual server which listens on that tunnel interface to be able to capture VPN user traffic before it leaves the BIG-IP. Bandwidth Controllers We have two types of bandwidth controller policy - Static and Dynamic. A Static policy sets the overall rate of traffic allowed, a dynamic policy allows us to set an overall rate but also a rate per user flow. For instance, we could allow ALL VPN traffic to be 1Gbps but each individual traffic flow within that could be limited to 1Mbps. iRules To apply the policy to the user traffic we are going to use an iRule with the BWC::policy command - this will set the policy on this flow for both uplink and downlink traffic in two different events - CLIENT_ACCEPTED and SERVER_CONNECTED when CLIENT_ACCEPTED { BWC::policy attach /Common/bwc-10M "[IP::remote_addr]:[TCP::remote_port]" } when SERVER_CONNECTED { BWC::policy attach /Common/bwc-10M "[IP::remote_addr]:[TCP::remote_port]" } Obviously the TCP::remote_port would be UDP::remote_port in a UDP virtual server. Putting it together Below you can see the virtual server configuration which I created using my iApp ltm virtual Common/vpn-1.app/vs_bwc_vpn-1_tcp_default { app-service /Common/vpn-1.app/vpn-1 creation-time 2020-04-29:10:26:39 destination Common/0.0.0.0:any ip-protocol tcp last-modified-time 2020-04-29:11:32:33 mask any profiles { Common/tcp { } } rules { Common/vpn-1.app/rule_bwc_vpn-1_tcp_default } serverssl-use-sni disabled source 0.0.0.0/0 source-address-translation { pool Common/snat-1 type snat } translate-address disabled translate-port disabled vlans { Common/connectivity-1 } vlans-enabled vs-index 7 } ltm virtual Common/vpn-1.app/vs_bwc_vpn-1_udp_default { app-service /Common/vpn-1.app/vpn-1 creation-time 2020-04-29:10:26:39 destination Common/0.0.0.0:any ip-protocol udp last-modified-time 2020-04-29:11:32:33 mask any profiles { Common/udp { } } rules { Common/vpn-1.app/rule_bwc_vpn-1_udp_default } serverssl-use-sni disabled source 0.0.0.0/0 source-address-translation { pool Common/snat-1 type snat } translate-address disabled translate-port disabled vlans { Common/connectivity-1 } vlans-enabled vs-index 6 } It works! iperf with no bandwidth controller $ iperf -c 10.20.20.3 ------------------------------------------------------------ Client connecting to 10.20.20.3, TCP port 5001 TCP window size: 64.0 KByte (default) ------------------------------------------------------------ [ 3] local 10.20.20.131 port 5957 connected with 10.20.20.3 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 184 MBytes 154 Mbits/sec iperf with a 10Mbps bandwidth controller applied $ iperf -c 10.20.20.3 ------------------------------------------------------------ Client connecting to 10.20.20.3, TCP port 5001 TCP window size: 64.0 KByte (default) ------------------------------------------------------------ [ 3] local 10.20.20.131 port 6066 connected with 10.20.20.3 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.2 sec 12.1 MBytes 9.98 Mbits/sec Make it easy I've shown you there how to put together the constituent parts yourself but brought to you by the magic of iApps, you can do it all automagically. You can use the iApp at APM VPN Bandwidth Controller iApp - just create the BWC policy you want to apply beforehand and run the iApp. You can even treat certain protocols differently - imagine if you want to limit HTTPS to 1Mbps but allow other traffic to have 10Mbps ( because a 1 second delay in a web page load is not noticeable but a 1 second delay in a Zoom session is very noticeable! )Securing your VMware Remote Solutions to Support COVID-19 Work From Home Scaling
Many of us are now working from home in unprecedented numbers. For infrastructure teams it's putting impressive strain on remote work solution. Building off our primary DevCentral COVID-19 article, our support teams and solution architects are hearing from many of you asking us for new and better ways to expand VMware capabilities with F5 BIG-IP Local Traffic Manager (LTM) and Access Policy Manager (APM). Get started securing your VMware remote working solutions with the field-recommended guides below. F5 with VMware Virtual Desktop Infrastructure (VDI) Solutions (Horizon View, Workspace ONE) How to deploy F5 with Horizon View using iApps This is a comprehensive guide for deploying F5 BIG-IP APM with VMware Horizon. Walk through the F5 iApp to assist in configuring APM with VMware Horizon View. How to use BIG-IP LTM in front of VMware Horizon Unified Access Gateway This guide will show step by step guidance on how to use F5 BIG-IP LTM to increase the scale and resiliency of either greenfield or brownfield VMware Horizon deployments. How to Deploy F5 APM with VMware ONE Providing a step-by-step instruction for setting up F5 BIG-IP APM as a proxy gateway for VMware Horizon with VMware Workspace ONE. How to deploy F5 BIG-IP LTM with VMware Workspace ONE Identity Manager (vIDM) This guide provides step-by-step instructions for setting up the first Identity Manager virtual appliance (Node 1), for production implementations. VMware recommends the deployment of two (2) additional nodes for three (3) total. Nodes 2 and 3 will be cloned from the first node after it's been configured and setup with the F5 BIG-IP to provide a fully load-balanced configuration. Reach Out To Us As our technical teams work with our users to provide continuous COVID-19 coverage, you may still need additional information we haven't surfaced yet. If you can't determine what best meets your requirements, let us know in the comments or reach out to our technical community. Don't forget to check out AskF5, our support knowledge center.
