How to get a F5 BIG-IP VE Developer Lab License
(applies to BIG-IP TMOS Edition) To assist DevOps teams improve their development for the BIG-IP platform, F5 offers a low cost developer lab license.This license can be purchased from your authorized F5 vendor. If you do not have an F5 vendor, you can purchase a lab license online: CDW BIG-IP Virtual Edition Lab License CDW Canada BIG-IP Virtual Edition Lab License Once completed, the order is sent to F5 for fulfillment and your license will be delivered shortly after via e-mail. F5 is investigating ways to improve this process. To download the BIG-IP Virtual Edition, please log into downloads.f5.com (separate login from DevCentral), and navigate to your appropriate virtual edition, example: For VMware Fusion or Workstation or ESX/i:BIGIP-16.1.2-0.0.18.ALL-vmware.ova For Microsoft HyperV:BIGIP-16.1.2-0.0.18.ALL.vhd.zip KVM RHEL/CentoOS: BIGIP-16.1.2-0.0.18.ALL.qcow2.zip Note: There are also 1 Slot versions of the above images where a 2nd boot partition is not needed for in-place upgrades. These images include_1SLOT- to the image name instead of ALL. The below guides will help get you started with F5 BIG-IP Virtual Edition to develop for VMWare Fusion, AWS, Azure, VMware, or Microsoft Hyper-V. These guides follow standard practices for installing in production environments and performance recommendations change based on lower use/non-critical needs fo Dev/Lab environments. Similar to driving a tank, use your best judgement. DeployingF5 BIG-IP Virtual Edition on VMware Fusion Deploying F5 BIG-IP in Microsoft Azure for Developers Deploying F5 BIG-IP in AWS for Developers Deploying F5 BIG-IP in Windows Server Hyper-V for Developers Deploying F5 BIG-IP in VMware vCloud Director and ESX for Developers Note: F5 Support maintains authoritativeAzure, AWS, Hyper-V, and ESX/vCloud installation documentation. VMware Fusion is not an official F5-supported hypervisor so DevCentral publishes the Fusion guide with the help of our Field Systems Engineering teams.Deploying F5 BIG-IP Virtual Edition on VMware Fusion
To deploy BIG-IP Virtual Edition on your workstation, VMware provides two great solutions: VMware Fusion Pro for OSX VMware Workstation Pro For this guide, we’ll use Fusion Pro 8 (v11 functions the same) due to it’s good network management abilities; for the non-Pro version refer to Jason Rahm’s article on setting up networking. Using the BIG-IP Virtual Edition, you can setup a development environment for most BIG-IP software solutions, including but not limited to LTM, APM Lite, ASM, AFM, and BIG-IP DNS. For more team oriented test or dev environments, you should probably install those to more robust infrastructure everyone has access too. Installation Instructions Installing and configuring VMware Fusion Pro Installing additional VMware networking Downloading the F5 BIG-IP Virtual Edition Importing BIG-IP VE to VMware Fusion F5 BIG-IP Configuration Configuring the Management Interface Obtaining an F5 BIG-IP Developer Edition License Configuring External and Internal Networks on BIG-IP VE Accessing BIG-IP VE GUI and Completing Setup and Licensing Configure BIG-IP System Settings Additional Information Installing and configuring VMware Fusion Pro Follow this link to purchase and download VMware Fusion Pro Install VMware and take advantage of their Getting Started Guide if unfamiliar with the product Installing additional VMware networking Start VMware Fusion Pro, and select the menu VMware Fusion > Preferences Click the Network icon Click the lock icon to authenticate and create additional networks Click the + icon 3 times to create vmnet2, vmnet3, and vmnet4 Select vmnet2 and configure the following network: Leave Allow virtual machines on this network to connect to external networks (using NAT) cleared Leave the Connect the host Mac to this network selected Leave Provide addresses on this network via DHCP selected In the Subnet IP field, enter 10.128.1.0 In the Subnet mask field, enter 255.255.255.0 Select vmnet3 and configure the following network: Select the Allow virtual machines on this network to connect to external networks (using NAT) to allow your BIG-IP VE to reach the internet Leave the Connect the host Mac to this network selected Leave Provide addresses on this network via DHCP selected In the Subnet IP field, enter 10.128.10.0 In the Subnet mask field, enter 255.255.255.0 Select vmnet4 and configure the following network: Leave Allow virtual machines on this network to connect to external networks (using NAT) Clear the Connect the host Mac to this network to prevent the system from having direct access to the internal network Leave Provide addresses on this network via DHCP selected In the Subnet IP field, enter 10.128.20.0 In the Subnet mask field, enter 255.255.255.0 Click Apply and close the window Downloading the F5 BIG-IP Virtual Edition Navigate and login at https://downloads.f5.com, if you do not have a support login, register here. Click Find a Download, select BIG-IP v12.x / Virtual Edition, and click Virtual-Edition again. Read the License Agreement and click I Accept (it’s a fantastic read) Select the BIGIP-currentversion.ALL-scsi.ovafile, with the description Image file set for VMware ESX/iServer Choose the nearest download location Importing BIG-IP Virtual Edition Image From VMware Fusion, navigate to File > Import Click Choose File Select the BIGIP-13.0.0.3.0.1679.ALL-scsi.ova image file from your download location and click Open Click Continue Name the new virtual machine whatever you want using common sense, for our example we’ll use BIGIP_v13_lab Click Accept After the import completes, click Finish, and Customize Settings Click Processors & Memory and adjust memory to provide the following: If System = 8GB, set VM memory to 4096 If System = 16GB, set VM memory to 8192 If System = 24GB+, set VM memory to 12416 Click Show All Click Network Adapter, and click vmnet2 Click Show All, then click Network Adapter 2, select vmnet3 Click Show All, then click Network Adapter 3, select vmnet4 Click Show All, then click Network Adapter 4, and uncheck the Connect Network Adapter to disable Close the Settings window F5 BIG-IP Configuration Configuring the Management Interface Click your BIG-IP VE Image from the Virtual Machine Library, then click Start Up After the BIG-IP VE powers up, you’ll be presented with the localhost login screen Log in to the BIG-IP system using the following default credentials localhost login: root Password: default At the CLI prompt, type: config Press Enter to activate the OK option Use the Tab key to activate the No option, then press Enter Edit the IP Address to 10.128.1.145, then press Tab to activate the OK option, and press Enter Ensure the Netmask is 255.255.255.0, then press Tab to activate the OK option, and press Enter Press Enter to activate the Yes option to create a default route for the management port Edit the Management Route to 10.128.1.1, then press the Tab to activate the OK option, and press Enter Press the Enter key to activate the Yes option to accept the settings Obtaining an F5 BIG-IP Developer Edition License Refer to How to get a F5 BIG-IP VE Developer Lab License to purchase your Developer License. Configuring External and Internal Networks on BIG-IP VE Open a terminal window, and type: ssh root@10.128.1.145 Use the following Password: default Copy or manually enter the following TMSH commands to your SSH session. You can copy and past all the lines simultaneously tmsh create net vlan external interfaces add { 1.1 { untagged } } tmsh create net vlan internal interfaces add { 1.2 { untagged } } tmsh create net self 10.128.10.240 address 10.128.10.240/24 vlan external tmsh create net self 10.128.20.240 address 10.128.20.240/24 vlan internal tmsh create net route Default_Gateway network 0.0.0.0/0 gw 10.128.10.1 tmsh save sys config exit Accessing BIG-IP VE GUI and Completing Setup and Licensing Open a web browser and access https://10.128.1.145 Log into the BIG-IP VE using the following credentials: Username: admin Password: admin On the Welcome Page click Next On the License page click Activate Open the email from F5 Networks with your Developer License Registration Key and copy the Registration Key text In the Setup Utility, in the Base Registration Key field, past the registration key text For Activation Method, select Manual, and click Next Select and copy all of the dossier text to your clipboard Select Click here to access F5 Licensing Server On the Activate F5 Product page, paste the dossier text in the field, then click Next Select to accept the legal agreement, then click Next Select and copy all of the license key text to your clipboard On the Setup Utility > License page, paste the license key text into the Step 3: License field, then click Next After the configuration changes complete, log into the BIG-IP VE system using the previous credentials On the Resource Provisioning page leave Local Traffic (LTM) as the only provisioned module and click Next On the Device Certificates page click Next On the Platform page, configure the Host Name, Root Account, and Admin Account to your desired settings, then click Next You’ll be prompted to log out and back into the BIG-IP VE. Do it. Under Standard Network Configuration, click Next Clear the Display configuration synchronization options checkbox, then click Next On the Internal Network Configuration page, review the settings, then click Next On the External network Configuration page, review the settings, then click Finished to complete the Setup Utility. Configure BIG-IP System Settings Open the System > Preferences page, and update the following settings, then click Update Records Per Screen: 30 Start Screen: Statistics Idle Time Before Automatic Logout: 100000 seconds Security Banner Text: Welcome to the F5 BIG-IP VE Lab Environment (or whatever you want this to say) Open the System > Configuration > Device > DNS page For DNS Lookup Server List, enter 8.8.8.8, and then click Add (you can use whatever DNS resolver you want here) Select 10.128.1.1, then click Delete, and click Update Open the Local Traffic > Nodes > Default Monitor page Click ICMP, and click << to move it to the Active list, then click Update Additional Information Using the 10.128.x.0/24 is intended only for ease of use and not a requirement. If you have alternate requirements, please replace our examples This guide builds a sufficient external and internal network the BIG-IP can use for proxy architecture testing and is intended for development purposes only If you opted not to purchase the Pro version of Fusion, you can still setup advanced networking. For more on this please see: VMware Fusion Custom Networking for BIG-IP VE Lab This guide is developed for VMware Fusion Pro on OSX. If you run VMware Workstation, setup is the same, only the UX and configuration locations change.
One Time Passwords via an SMS Gateway with BIG-IP Access Policy Manager
One time passwords, or OTP, are used (as the name indicates) for a single session or transaction. The plus side is a more secure deployment, the downside is two-fold—first, most solutions involve a token system, which is costly in management, dollars, and complexity, and second, people are lousy at remembering things, so a delivery system for that OTP is necessary. The exercise in this tech tip is to employ BIG-IP APM to generate the OTP and pass it to the user via an SMS Gateway, eliminating the need for a token creating server/security appliance while reducing cost and complexity. Getting Started This guide was developed by F5er Per Boe utilizing the newly released BIG-IP version 10.2.1. The “-secure” option for the mcget command is new in this version and is required in one of the steps for this solution. Also, this solution uses the Clickatell SMS Gateway to deliver the OTPs. Their API is documented at http://www.clickatell.com/downloads/http/Clickatell_HTTP.pdf. Other gateway providers with a web-based API could easily be substituted. Also, there are steps at the tail end of this guide to utilize the BIG-IP’s built-in mail capabilities to email the OTP during testing in lieu of SMS. The process in delivering the OTP is shown in Figure 1. First a request is made to the BIG-IP APM. The policy is configured to authenticate the user’s phone number in Active Directory, and if successful, generate a OTP and pass along to the SMS via the HTTP API. The user will then use the OTP to enter into the form updated by APM before allowing the user through to the server resources. BIG-IP APM Configuration Before configuring the policy, an access profile needs to be created, as do a couple authentication servers. First, let’s look at the authentication servers Authentication Servers To create servers used by BIG-IP APM, navigate to Access Policy->AAA Servers and then click create. This profile is simple, supply your domain server, domain name, and admin username and password as shown in Figure 2. The other authentication server is for the SMS Gateway, and since it is an HTTP API we’re using, we need the HTTP type server as shown in Figure 3. Note that the hidden form values highlighted in red will come from your Clickatell account information. Also note that the form method is GET, the form action references the Clickatell API interface, and that the match type is set to look for a specific string. The Clickatell SMS Gateway expects the following format: https://api.clickatell.com/http/sendmsg?api_id=xxxx&user=xxxx&password=xxxx&to=xxxx&text=xxxx Finally, successful logon detection value highlighted in red at the bottom of Figure 3 should be modified to response code returned from SMS Gateway. Now that the authentication servers are configured, let’s take a look at the access profile and create the policy. Access Profile & Policy Before we can create the policy, we need an access profile, shown below in Figure 4 with all default settings. Now that that is done, we click on Edit under the Access Policy column highlighted in red in Figure 5. The default policy is bare bones, or as some call it, empty. We’ll work our way through the objects, taking screen captures as we go and making notes as necessary. To add an object, just click the “+” sign after the Start flag. The first object we’ll add is a Logon Page as shown in Figure 6. No modifications are necessary here, so you can just click save. Next, we’ll configure the Active Directory authentication, so we’ll add an AD Auth object. Only setting here in Figure 7 is selecting the server we created earlier. Following the AD Auth object, we need to add an AD Query object on the AD Auth successful branch as shown in Figures 8 and 9. The server is selected in the properties tab, and then we create an expression in the branch rules tab. To create the expression, click change, and then select the Advanced tab. The expression used in this AD Query branch rule: expr { [mcget {session.ad.last.attr.mobile}] != "" } Next we add an iRule Event object to the AD Query OK branch that will generate the one time password and provide logging. Figure 10 Shows the iRule Event object configuration. The iRule referenced by this event is below. The logging is there for troubleshooting purposes, and should probably be disabled in production. 1: when ACCESS_POLICY_AGENT_EVENT { 2: expr srand([clock clicks]) 3: set otp [string range [format "%08d" [expr int(rand() * 1e9)]] 1 6 ] 4: set mail [ACCESS::session data get "session.ad.last.attr.mail"] 5: set mobile [ACCESS::session data get "session.ad.last.attr.mobile"] 6: set logstring mail,$mail,otp,$otp,mobile,$mobile 7: ACCESS::session data set session.user.otp.pw $otp 8: ACCESS::session data set session.user.otp.mobile $mobile 9: ACCESS::session data set session.user.otp.username [ACCESS::session data get "session.logon.last.username"] 10: log local0.alert "Event [ACCESS::policy agent_id] Log $logstring" 11: } 12: 13: when ACCESS_POLICY_COMPLETED { 14: log local0.alert "Result: [ACCESS::policy result]" 15: } On the fallback path of the iRule Event object, add a Variable Assign object as show in Figure 10b. Note that the first assignment should be set to secure as indicated in image with the [S]. The expressions in Figure 10b are: session.logon.last.password = expr { [mcget {session.user.otp.pw}]} session.logon.last.username = expr { [mcget {session.user.otp.mobile}]} On the fallback path of the AD Query object, add a Message Box object as shown in Figure 11 to alert the user if no mobile number is configured in Active Directory. On the fallback path of the Event OTP object, we need to add the HTTP Auth object. This is where the SMS Gateway we configured in the authentication server is referenced. It is shown in Figure 12. On the fallback path of the HTTP Auth object, we need to add a Message Box as shown in Figure 13 to communicate the error to the client. On the Successful branch of the HTTP Auth object, we need to add a Variable Assign object to store the username. A simple expression and a unique name for this variable object is all that is changed. This is shown in Figure 14. On the fallback branch of the Username Variable Assign object, we’ll configure the OTP Logon page, which requires a Logon Page object (shown in Figure 15). I haven’t mentioned it yet, but the name field of all these objects isn’t a required change, but adding information specific to the object helps with readability. On this form, only one entry field is required, the one time password, so the second password field (enabled by default) is set to none and the initial username field is changed to password. The Input field below is changed to reflect the type of logon to better queue the user. Finally, we’ll finish off with an Empty Action object where we’ll insert an expression to verify the OTP. The name is configured in properties and the expression in the branch rules, as shown in Figures 16 and 17. Again, you’ll want to click advanced on the branch rules to enter the expression. The expression used in the branch rules above is: expr { [mcget {session.user.otp.pw}] == [mcget -secure {session.logon.last.otp}] } Note again that the –secure option is only available in version 10.2.1 forward. Now that we’re done adding objects to the policy, one final step is to click on the Deny following the OK branch of the OTP Verify Empty Action object and change it from Deny to Allow. Figure 18 shows how it should look in the visual policy editor window. Now that the policy is completed, we can attach the access profile to the virtual server and test it out, as can be seen in Figures 19 and 20 below. Email Option If during testing you’d rather send emails than utilize the SMS Gateway, then configure your BIG-IP for mail support (Solution 3664), keep the Logging object, lose the HTTP Auth object, and configure the system with this script to listen for the messages sent to /var/log/ltm from the configured Logging object: #!/bin/bash while true do tail -n0 -f /var/log/ltm | while read line do var2=`echo $line | grep otp | awk -F'[,]' '{ print $2 }'` var3=`echo $line | grep otp | awk -F'[,]' '{ print $3 }'` var4=`echo $line | grep otp | awk -F'[,]' '{ print $4 }'` if [ "$var3" = "otp" -a -n "$var4" ]; then echo Sending pin $var4 to $var2 echo One Time Password is $var4 | mail -s $var4 $var2 fi done done The log messages look like this: Jan 26 13:37:24 local/bigip1 notice apd[4118]: 01490113:5: b94f603a: session.user.otp.log is mail,user1@home.local,otp,609819,mobile,12345678 The output from the script as configured looks like this: [root@bigip1:Active] config # ./otp_mail.sh Sending pin 239272 to user1@home.local Conclusion The BIG-IP APM is an incredibly powerful tool to add to the LTM toolbox. Whether using the mail system or an SMS gateway, you can take a bite out of your infrastructure complexity by using this solution to eliminate the need for a token management service. Many thanks again to F5er Per Boe for this excellent solution!
LTM: HTTP Monitoring with POST Request
I recently discovered that you can use LTM's built in HTTP monitor template to send a POST request to a server to verify its health. It's surprisingly straightforward in comparison with the external monitor approach I've used in the past. This article will cover the details you'll need to properly implement an HTTP POST monitor using the built in monitor template. The built in HTTP monitor template The HTTP monitor template provides a way to send a single request to the server, monitor the response for an expected string, and mark the pool member down if the expected response isn't seen within the configured timeout. The options of interest for this exercise are the Send string and the Receive string. The Send string is where the request is configured. It should include the method (in this case, POST), the URI to request from the server, the HTTP version, some standard HTTP headers (including Basic Auth, if applicable), and in this case, the POST body. The server should respond in a predictable way to this request each time it is issued. The Receive string should include a phrase or non-dictionary word that will appear only in that expected response. The Receive string can use basic regular expressions to accommodate per-server or other minor variations in the expected response. Capturing the the transaction The first thing you'll need to do, as with any monitor, is to identify and capture the transaction you'd like to replicate in order to validate the health of your application servers. You can run tcpdump to watch live production traffic and capture the target request and response, but it's safer to run an isolated test request at the command line to more exactly replicate the action of the LTM monitor. The approach I recommend uses both techniques. First capture a trace of production traffic so you have a known-good working example with all the expected headers. Then log into the LTM command line and use telnet or cURL to send the same request to the server along with the observed headers, and monitor the response, adjusting the request until the expected response is received. Here's the cURL syntax you'll need to use for command line request submission: curl -N -H <headers> -u <user:password> -d <post body> http://<pool_mmeber_IP>:<pool_mmeber_port><URI> You'll need to examine the actual data that's being passed, including whitespace characters. A packet trace can be used to look directly at the conversation on the wire. Capture on the server-facing interface. To examine only production traffic, filter for the IP/port of the pool member, excluding the non-floating self IP on the server-facing VLAN: tcpdump -nni <server_vlan> -Xs 0 host <pool_member_IP> and port <pool_member_port> and not host <internal_non-floating_selfIP> (Filtering out the non-floating self IP prevents monitor traffic from appearing in the trace.) To examine your command line request, filter instead for the non-floating self IP on the server-facing VLAN and the IP/port of the pool member: tcpdump -nni <server_vlan> -Xs 0 host <internal_non-floating_selfIP> and host <pool_member_IP> and port <pool_member_port> Here's a sample transaction trace for a SOAP/xml request: Once you have identified the transaction and validated the required headers, you are ready to build your custom monitor. Creating the custom HTTP monitor The information you've gathered so far should be everything you need to configure the custom HTTP monitor. Start by creating a monitor of type "http" in the LTM GUI (Local Traffic -> Monitors -> Create). Configure Interval and Timeout as appropriate for your application. Set Username and Password if Basic Auth credentials are required by the server. Then you'll need to dig into your transaction trace data to construct the Send string containing all the request details. For the Send string, the HTTP method must be specified along with the URI and HTTP version v1.1 on the same line, followed by an encoded LF/CR (\r\n), then the Host header (required for HTTP 1.1). For the trace above, we'll start with: POST /XAIApp/xaiserver HTTP/1.1\r\nHost: 191.168.1.1:4200 Add the other headers, all separated by a single encoded CR/LF. (Line breaks are only for display. No literal carriage returns are inserted into the Send string.) Include the Basic Auth header, if applicable, even though it is specified elsewhere in the monitor template. With headers, the Send string now looks like this: POST /XAIApp/xaiserver HTTP/1.1\r\nHost: 191.168.1.1:4200\r\n Authorization: Basic c3lzdXNlcjpzeXN1c2VyMDA= \r\n Pragma: no-cache\r\nAccept: */*\r\nContent-Length: 673\r\nContent-Type: application/x-www-form-urlencoded Finally add the POST body, preceded by a double CR/LF to indicate the end of the headers. If the POST body contains double quotes or braces, you will need to escape them with a preceding backslash so they are interpreted correctly by LTM. The completed Send string now looks like this: POST /XAIApp/xaiserver HTTP/1.1\r\nHost: 191.168.1.1:4200\r\n Authorization: Basic c3lzdXNlcjpzeXN1c2VyMDA= \r\n Pragma: no-cache\r\nAccept: */*\r\nContent-Length: 673\r\nContent-Type: application/x-www-form-urlencoded\r\n\r\n <SOAP-ENV:Envelope xmlns:SOAP-ENV=\"urn:schemas-xmlsoap-org:envelope\"> <SOAP-ENV:Body><SSvcInstallationOptionsMaintenance dateTimeTagFormat=\"CdxDateTime\" transactionType =\"READ\" ><SSvcInstallationOptionsMaintenanceService> <SSvcInstallationOptionsMaintenanceHeader InstallationOptionID=\"1\"/><SSvcInstallationOptionsMaintenanceDetails InstallationOptionID=\"1\"> <InstallMsg><InstallMsgHeader InstallationOptionID=\"1\"/></InstallMsg><InstallAlg><InstallAlgHeader InstallationOptionID=\"1\"/></InstallAlg> <Module></Module></SSvcInstallationOptionsMaintenanceDetails></SSvcInstallationOptionsMaintenanceService></SSvcInstallationOptionsMaintenance> </SOAP-ENV:Body></SOAP-ENV:Envelope> To complete the monitor definition, enter the phrase or string expected in a healthy server response into the Receive string field. For the trace above, we'll use: Standard automatic payment creation The resulting monitor configuration will look like this: monitor http-post-soapenv { defaults from http interval 10 timeout 31 password "password" recv "Standard automatic payment creation" send "POST /XAIApp/xaiserver HTTP/1.1\r\nHost: 191.168.1.1:4200\r\nAuthorization: Basic c3lzdXNlcjpzeXN1c2VyMDA= \r\n Pragma: no-cache\r\nAccept: */*\r\n Content-Length: 673\r\nContent-Type: application/x-www-form-urlencoded\r\n\r\n <SOAP-ENV:Envelope xmlns:SOAP-ENV=\"urn:schemas-xmlsoap-org:envelope\"> <SOAP-ENV:Body><SSvcInstallationOptionsMaintenance dateTimeTagFormat=\"CdxDateTime\" transactionType =\"READ\" ><SSvcInstallationOptionsMaintenanceService> <SSvcInstallationOptionsMaintenanceHeader InstallationOptionID=\"1\"/><SSvcInstallationOptionsMaintenanceDetails InstallationOptionID=\"1\"> <InstallMsg><InstallMsgHeader InstallationOptionID=\"1\"/></InstallMsg><InstallAlg><InstallAlgHeader InstallationOptionID=\"1\"/></InstallAlg> <Module></Module></SSvcInstallationOptionsMaintenanceDetails></SSvcInstallationOptionsMaintenanceService></SSvcInstallationOptionsMaintenance> </SOAP-ENV:Body></SOAP-ENV:Envelope>" username "user" } Related information These 2 previous articles are specific to external monitors, but contain some more useful details about command line testing and capturing traffic: LTM External Monitors: The Basics LTM External Monitors: Troubleshooting
The Full-Proxy Data Center Architecture
Why a full-proxy architecture is important to both infrastructure and data centers. In the early days of load balancing and application delivery there was a lot of confusion about proxy-based architectures and in particular the definition of a full-proxy architecture. Understanding what a full-proxy is will be increasingly important as we continue to re-architect the data center to support a more mobile, virtualized infrastructure in the quest to realize IT as a Service. THE FULL-PROXY PLATFORM The reason there is a distinction made between “proxy” and “full-proxy” stems from the handling of connections as they flow through the device. All proxies sit between two entities – in the Internet age almost always “client” and “server” – and mediate connections. While all full-proxies are proxies, the converse is not true. Not all proxies are full-proxies and it is this distinction that needs to be made when making decisions that will impact the data center architecture. A full-proxy maintains two separate session tables – one on the client-side, one on the server-side. There is effectively an “air gap” isolation layer between the two internal to the proxy, one that enables focused profiles to be applied specifically to address issues peculiar to each “side” of the proxy. Clients often experience higher latency because of lower bandwidth connections while the servers are generally low latency because they’re connected via a high-speed LAN. The optimizations and acceleration techniques used on the client side are far different than those on the LAN side because the issues that give rise to performance and availability challenges are vastly different. A full-proxy, with separate connection handling on either side of the “air gap”, can address these challenges. A proxy, which may be a full-proxy but more often than not simply uses a buffer-and-stitch methodology to perform connection management, cannot optimally do so. A typical proxy buffers a connection, often through the TCP handshake process and potentially into the first few packets of application data, but then “stitches” a connection to a given server on the back-end using either layer 4 or layer 7 data, perhaps both. The connection is a single flow from end-to-end and must choose which characteristics of the connection to focus on – client or server – because it cannot simultaneously optimize for both. The second advantage of a full-proxy is its ability to perform more tasks on the data being exchanged over the connection as it is flowing through the component. Because specific action must be taken to “match up” the connection as its flowing through the full-proxy, the component can inspect, manipulate, and otherwise modify the data before sending it on its way on the server-side. This is what enables termination of SSL, enforcement of security policies, and performance-related services to be applied on a per-client, per-application basis. This capability translates to broader usage in data center architecture by enabling the implementation of an application delivery tier in which operational risk can be addressed through the enforcement of various policies. In effect, we’re created a full-proxy data center architecture in which the application delivery tier as a whole serves as the “full proxy” that mediates between the clients and the applications. THE FULL-PROXY DATA CENTER ARCHITECTURE A full-proxy data center architecture installs a digital "air gap” between the client and applications by serving as the aggregation (and conversely disaggregation) point for services. Because all communication is funneled through virtualized applications and services at the application delivery tier, it serves as a strategic point of control at which delivery policies addressing operational risk (performance, availability, security) can be enforced. A full-proxy data center architecture further has the advantage of isolating end-users from the volatility inherent in highly virtualized and dynamic environments such as cloud computing . It enables solutions such as those used to overcome limitations with virtualization technology, such as those encountered with pod-architectural constraints in VMware View deployments. Traditional access management technologies, for example, are tightly coupled to host names and IP addresses. In a highly virtualized or cloud computing environment, this constraint may spell disaster for either performance or ability to function, or both. By implementing access management in the application delivery tier – on a full-proxy device – volatility is managed through virtualization of the resources, allowing the application delivery controller to worry about details such as IP address and VLAN segments, freeing the access management solution to concern itself with determining whether this user on this device from that location is allowed to access a given resource. Basically, we’re taking the concept of a full-proxy and expanded it outward to the architecture. Inserting an “application delivery tier” allows for an agile, flexible architecture more supportive of the rapid changes today’s IT organizations must deal with. Such a tier also provides an effective means to combat modern attacks. Because of its ability to isolate applications, services, and even infrastructure resources, an application delivery tier improves an organizations’ capability to withstand the onslaught of a concerted DDoS attack. The magnitude of difference between the connection capacity of an application delivery controller and most infrastructure (and all servers) gives the entire architecture a higher resiliency in the face of overwhelming connections. This ensures better availability and, when coupled with virtual infrastructure that can scale on-demand when necessary, can also maintain performance levels required by business concerns. A full-proxy data center architecture is an invaluable asset to IT organizations in meeting the challenges of volatility both inside and outside the data center. Related blogs & articles: The Concise Guide to Proxies At the Intersection of Cloud and Control… Cloud Computing and the Truth About SLAs IT Services: Creating Commodities out of Complexity What is a Strategic Point of Control Anyway? The Battle of Economy of Scale versus Control and Flexibility F5 Friday: When Firewalls Fail… F5 Friday: Platform versus ProductLTM HTTP Profile Option: Response Chunking
The HTTP profile contains an option to control HTTP response chunking, and it's not exactly obvious why you'd choose one over the other. This article gives a high level explanation of HTTP chunking, and (with lots of input from unRuleY) attempts to dispel the mystery and capture the subtle nuances of each response chunking mode. What is HTTP Response Chunking? Chunking is a message transfer encoding scheme for HTTP that enables dynamic content length management to help reduce latency. The introduction of chunked transfer encoding in HTTP 1.1 allows the server to begin sending data to the client before the content length is known, and in some cases before the entire response has been generated by the application. Chunked Transfer-Encoding Chunked transfer encoding modifies the body of a message in order to transfer it as a series of chunks. In chunked responses, the Content-Length header is replaced by a Transfer-Encoding header with a value of "chunked", and chunks of data are sent to the client as they become available. (You can identify a chunked response because it contains the "Transfer-Encoding: chunked" header.) The chunked response is a normal HTTP response beginning with HTTP headers, followed by 2 consecutive CR/LF's, then the message body. A complete chunked message body contains a series of data chunks ending with a 0 length chunk. Footers may follow the message body, and the chunked message is terminated by 2 consecutive CR/LF's. Once the end of the chunked transaction is seen, the connection is available for re-use. Each data chunk consists of 4 parts: a header indicating the size of the chunk data in hex (and possibly other non-standard parameters you can safely ignore) <CR><LF> a chunk of the payload data <CR><LF> Here's an unchunked server response: HTTP/1.1 200 OK Date: Fri, 31 Dec 1999 23:59:59 GMT Content-Type: text/plain Content-Length: 42<CR><LF> <CR><LF> abcdefghijklmnopqrstuvwxyz1234567890ABCDEF And the same response using chunked transfer encoding: HTTP/1.1 200 OK Date: Fri, 31 Dec 1999 23:59:59 GMT Content-Type: text/plain Transfer-Encoding: chunked<CR><LF> <CR><LF> 1a; non-std-params-here<CR><LF> abcdefghijklmnopqrstuvwxyz<CR><LF> 10<CR><LF> 1234567890ABCDEF<CR><LF> 0<CR><LF> <CR><LF> The response data may be split into any number of chunks of any size. The chunks can contain any binary data, and may be much larger than the examples here. Chunking Options in the LTM HTTP Profile The HTTP profile contains a "Response Chunking" option that can be set to 4 different modes depending on how server responses should be encoded. The effect of each mode depends on whether the server response is sent using chunked encoding. This section should help you choose the best response chunking setting for your application. Summary of the 4 response chunking modes: Response Chunking Mode Description Recommendation Unchunk All responses are buffered on LTM then sent unchunked with Content-Length header Not optimal for typical webserver / browser traffic Rechunk All responses are chunked after LTM processing. LTM determines optimal chunk sizes. Recommended for most applications, especially when other LTM features may change the size of the payload. Selective For unchunked responses, same as Unchunk. For chunked responses, same as Rechunk. Recommended for most applications, especially when other LTM features may change the size of the payload. Preserve Preserves transfer encoding as set by the origin server. Recommended for troubleshooting or when chunk size errors can't be resolved. Each mode has a different functional effect on chunked and unchunked server responses. Here are a few more details about how each mode affects each type of response. Unchunk Unchunk mode will send all content unchunked regardless of whether the server sent it chunked or unchunked. For content that is chunked, Unchunk buffers the response, removing the HTTP transfer encoding headers and chunk headers, processes the HTTP content, and then sends the unchunked response to the client. The client connection will be closed once the response is completed. For content that is already unchunked, Unchunk handles the response as a normal HTTP transfer: It buffers the response, processes the HTTP content and sends the response unchunked. Unchunk mode is only recommended when you need to force unchunked responses for all clients accessing a virtual server. Since the process of unchunking a response requires LTM to buffer the entire response before sending it, using Unchunk is not optimal for normal webserver/browser traffic. nchunk should not be used if LTM will be modifying the payload length. Rechunk Rechunk mode will re-chunk all content regardless of whether the server sent it chunked or unchunked. For content that is already chunked, Rechunk unchunks the HTTP content, then as it processes the data, creates and sends new chunks to the client. For optimization, this mode attempts to preserve the original chunks, but new chunks may be different sizes than the original chunks, especially if the original response was modified. For content that is unchunked, Rechunk buffers the response and processes the HTTP content, adds the transfer encoding and chunk headers to the response, and then sends a newly chunked response. Rechunk mode (or Selective) is recommended if you are using any other feature that modifies the length of the original server response (such as HTTP compression, or iRules / stream profile modifying HTTP payload). While your server may only chunk some responses, chunking all responses will help optimizes data transfer through multiple proxy hops. Selective Selective mode will re-chunk chunked content, and leave unchunked content alone. For content that is chunked, Selective is the same as Rechunk: It unchunks the HTTP content, then as it processes the data, creates and sends new chunks to the client, preserving the original chunks if optimal to do so. For content that is unchunked, Selective mode is the same as Unchunk: It processes the HTTP content and sends the response unchunked. Selective mode is recommended for most applications. Selective mode (or Rechunk) is recommended if you are using any other feature that modifies the length of the original server response (such as HTTP compression, or iRules / stream profile modifying HTTP payload). Preserve Preserve mode will preserve the original transfer encoding, changing nothing at all regardless of whether the server sends chunked or unchunked data. For content that is chunked, Preserve processes the HTTP content and as it is processed, sends the response to the client with the same chunk boundaries. For content that is unchunked, Preserve buffers the response, processes the HTTP content and sends the response unchunked. Preserve mode is recommended for troubleshooting or when chunk size errors can't be resolved, and should not be used if LTM will be modifying the payload length.Deploying F5 BIG-IP in VMware vCloud Director and ESX for Developers
F5’s BYOL (Bring Your Own License) model allows you to purchase a developer/lab license and install it in your VMware infrastructure. This model provides a stable development instance for the following software components of BIG-IP: LTM, GTM, DNS, AFM, ASM, APM Lite (10 users), AAM, CGN, SSL Forward Proxy, Advanced Protocols, and Crypto Offload all at a 10Mbps rate limit. Please refer to these resources to get started with VMware vCloud Director and ESX(i) How to get a F5 BIG-IP VE Developer Lab License BIG-IP Virtual Edition Setup Guide for VMware vCloud Director @ support.f5.com BIG-IP Virtual Edition Setup Guide for VMware ESXi v12.0 @ support.f5.com (PDF) NOTE: DevCentral recommends no less than 4GB RAM and prefer 8 to 24 depending on what modules requiring activation (the more the merrier right?). For Developer environments, you can deviate from staging and production requirements allowing you develop applications and solutions across multiple BIG-IP modules. DevCentral uses multiple vm’s each with standardized configurations found in BIG-IP’s product catalog to ensure that we don’t produce unsupportable materials for ingestion by the community. You can do whatever you want!
Interfaces uninitialized?
I am testing the VE in both Vmware ws 7 and Virtualbox. Unfortunatly with the same result. I can't get the interfaces to work. I have added them as host only but the interfaces show up as "UNINITIALIZED" and without a mac address. I can't see that I get any connectivity at all on the interfaces. The release notes mention UNINITIALIZED but that it should work anyway.F5 in AWS Part 1 - AWS Networking Basics
Updated for Current Versions and Documentation Part 1 : AWS Networking Basics Part 2: Running BIG-IP in an EC2 Virtual Private Cloud Part 3: Advanced Topologies and More on Highly-Available Services Part 4: Orchestrating BIG-IP Application Services with Open-Source Tools Part 5: Cloud-init, Single-NIC, and Auto Scale Out of BIG-IP in v12 If you work in IT, and you haven’t been living under a rock, then you have likely heard of Amazon Web Services (AWS). There has been a substantial increase in the maturity and stability of the AWS Elastic Compute Cloud (EC2), but you are wondering – can I continue to leverage F5 services in AWS? In this series of blog posts, we will discuss the how and why of running F5 BIG-IP in EC2. In this specific article, we’ll start with the basics of the AWS EC2 and Virtual Private Cloud (VPC). Later in the series, we will discuss some of the considerations associated with running BIG-IP as compute instance in this environment, we’ll outline the best deployment models for your application in EC2, and how these deployment models can be automated using open-source tools. Note: AWS uses the terms "public" and "private" to refer to what F5 Networks has typically referred to as "external" and "internal" respectively. We will use this terms interchangeably. First, what is AWS? If you have read the story, you will know that the EC2 project began with an internal interest at Amazon to move away from messy, multi-tenant networks using VLANs for segregation. Instead, network engineers at Amazon wanted to build an entirely IP-based architecture. This vision morphed into the universe of application services available today. Of course, building multi-tenant, purely L3 networks at massive scale had implications for both security and redundancy (we’ll get to this later). Today, EC2 enables users to run applications and services on top of virtualized network, storage, and compute infrastructure, where hosts are deployed in the form of Amazon Machine Images (AMIs). These AMIs can either be private to the user or launched from the public AWS marketplace. Hosts can be added to elastic load balancing (ELB) groups and associated with publicly accessible IPs to implement a simple horizontal model for availability. AWS became truly relevant for the enterprise with the introduction of the Virtual Private Cloud service. VPCs enabled users to build virtual private networks at the IP layer. These private networks can be connected to on-premise configurations by way of a VPN Gateway, or connected to the internet via an Internet Gateway. When deploying hosts within a VPC, the user has a significant amount of control over how each host is attached to the network. For example, a host can be attached to multiple networks and given several public or private IPs on one or multiple interfaces. Further, users can control many of the security aspects they are used to configuring in an on-premise environment (albeit in a slightly different way), including network ACLs, routing, simple firewalling, DHCP options, etc. Lets talk about these and other important EC2 aspects and try to understand how they affect our application deployment strategy. L2 Restrictions As we mentioned above, one of the design goals of AWS was to remove layer 2 networking. This is a worthy accomplishment but we lose access to certain useful protocols, including ARP (and gratuitious ARP), broadcast and multi-cast groups, 802.1Q tagging. We can no longer use VLANs for some availability models, for quality of service management, or for tenant isolation. Network Interfaces For larger topologies, one of the largest impacts given the removal of 802.1Q protocol support is the number of subnets we can attach to a node in the network. Because in AWS each interface is attached as a layer 3 endpoint, we must add an interface for each subnet. This contrasts with traditional networks, where you can add VLANs to your trunk for each subnet via tagging. Even though we're in a virtual world, the number of virtual network interfaces (or Elastic Network Interfaces (ENIs) in AWS terminology) is also limited according to the EC2 instance size. Together, the limits on number of interfaces and mapping between interface and subnet effectively limit the number of directly connected networks we can attach to a device (like BIG-IP, for example). IP Addressing AWS offers two kinds of globally routable IP address; these are “Public IP Addresses” and “Elastic IP Address”. In the table below, we outlined some of the differences between these two types of IP addresses. You can probably figure out for yourself why we will want to use Elastic IPs with BIG-IP. Like interfaces, AWS limits the number of IPs in several ways, including the number of IPs that can be attached to an interface and the number of elastic IPs per AWS account. Table 1: Differences between Public and Elastic IP Addresses Public IP Elastic IP Released on device termination/disassociation YES NO Assignable to secondary interfaces NO YES Can be associated after launch NO YES Amazon provides more information on public and elastic IP addresses here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-instance-addressing.html#concepts-public-addresses Each interface on an EC2 instance is given a private IP address. This IP address is routable locally through your subnet and assigned from the address range associated with the subnet to which your interface is attached. Multiple private secondary IP addresses can be attached to an interface, and is a useful technique for creating more complex topologies. The number of interfaces and private IPs per interface within an Amazon VPC are listed here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html#AvailableIpPerENI NAT Instances, Subnets and Routing When creating a VPC using the wizard available in the AWS VPC web portal, several default configurations are possible. One of these configurations is “VPC with Public and Private subnets”. In this configuration, what if the instances on our private subnet wish to access the outside world? Because we cannot attach public or elastic IP address to instances within the private subnet, we must use NAT provided by AWS. Like BIG-IP and other network devices in EC2, the NAT instance will live as a compute node within your VPC. This is good way to allow outbound traffic from your internal servers, but to prevent those servers from receiving inbound traffic. When you create subnets manually or through the VPC wizard, you’ll note that each subnet has an associated routing table. These route tables may be updated to control traffic flow between instances and subnets in your VPC. Regions and Availability Zones We know quite a few number of people who have been confused by the concept of availabilty zones in EC2. To put it clearly, an availabilty zone is a physically isolated datacenter in a region. Regions may contain mulitple availability zones. Availabilty zones run on different networking and storage infrastructure, and depend on seperate power supplies and internet connections. Striping your application deployments across availability zones is a great way to provide redundancy and, perhaps a hot standby, but please note that these are not the same thing. Amazon does not mirror any data between zones on behalf of the customer. While VPCs can span availability zones, subnets may not. To close this blog post, we are fortunate enough to get a video walk through from Vladimir Bojkovic, Solution Architect at F5 Networks. He shows how to create a VPC with internal and external subnets as a practical demonstration of the concepts we discussed above.
