iRules 101 - #12 - The Session Command
One of the things that makes iRules so incredibly powerful is the fact that it is a true scripting language, or at least based on one. The fact that they give you the tools that TCL brings to the table - regular expressions, string functions, even things as simple as storing, manipulating and recalling variable data - sets iRules apart from the rest of the crowd. It also makes it possible to do some pretty impressive things with connection data and massaging/directing it the way you want it. Other articles in the series: Getting Started with iRules: Intro to Programming with Tcl | DevCentral Getting Started with iRules: Control Structures & Operators | DevCentral Getting Started with iRules: Variables | DevCentral Getting Started with iRules: Directing Traffic | DevCentral Getting Started with iRules: Events & Priorities | DevCentral Intermediate iRules: catch | DevCentral Intermediate iRules: Data-Groups | DevCentral Getting Started with iRules: Logging & Comments | DevCentral Advanced iRules: Regular Expressions | DevCentral Getting Started with iRules: Events & Priorities | DevCentral iRules 101 - #12 - The Session Command | DevCentral Intermediate iRules: Nested Conditionals | DevCentral Intermediate iRules: Handling Strings | DevCentral Intermediate iRules: Handling Lists | DevCentral Advanced iRules: Scan | DevCentral Advanced iRules: Binary Scan | DevCentral Sometimes, though, a simple variable won't do. You've likely heard of global variables in one of the earlier 101 series and read the warning there, and are looking for another option. So here you are, you have some data you need to store, which needs to persist across multiple connections. You need it to be efficient and fast, and you don't want to have to do a whole lot of complex management of a data structure. One of the many ways that you can store and access information in your iRule fits all of these things perfectly, little known as it may be. For this scenario I'd recommend the usage of the session command. There are three main permutations of the session command that you'll be using when storing and referencing data within the session table. These are: session add: Stores user's data under the specified key for the specified persistence mode session lookup: Returns user data previously stored using session add session delete: Removes user data previously stored using session add A simple example of adding some information to the session table would look like: when CLIENTSSL_CLIENTCERT { set ssl_cert [SSL::cert 0] session add ssl $ssl_cert 90 } By using the session add command, you can manually place a specific piece of data into the LTM's session table. You can then look it up later, by unique key, with the session lookup command and use the data in a different section of your iRule, or in another connection all together. This can be helpful in different situations where data needs to be passed between iRules or events that it might not normally be when using a simple variable. Such as mining SSL data from the connection events, as below: when CLIENTSSL_CLIENTCERT { # Set results in the session so they are available to other events session add ssl [SSL::sessionid] [list [X509::issuer] [X509::subject] [X509::version]] 180 } when HTTP_REQUEST { # Retrieve certificate information from the session set sslList [session lookup ssl [SSL::sessionid]] set issuer [lindex sslList 0] set subject [lindex sslList 1] set version [lindex sslList 2] } Because the session table is optimized and designed to handle every connection that comes into the LTM, it's very efficient and can handle quite a large number of items. Also note that, as above, you can pass structured information such as TCL Lists into the session table and they will remain intact. Keep in mind, though, that there is currently no way to count the number of entries in the table with a certain key, so you'll have to build all of your own processing logic for now, where necessary. It's also important to note that there is more than one session table. If you look at the above example, you'll see that before we listed any key or data to be stored, we used the command session add ssl. Note the "ssl" portion of this command. This is a reference to which session table the data will be stored in. For our purposes here there are effectively two session tables: ssl, and uie. Be sure you're accessing the same one in your session lookup section as you are in your session add section, or you'll never find the data you're after. This is pretty easy to keep straight, once you see it. It looks like: session add uie ... session lookup uie Or: session add ssl ... session lookup ssl You can find complete documentation on the session command here, in the iRules, as well as some great examplesthat depict some more advanced iRules making use of the session command to great success. Check out Codeshare for more examples.Intermediate iRules: Nested Conditionals
Conditionals are a pretty standard tool in every programmer's toolbox. They are the functions that allow us to decided when we want certain actions to happen, based on, well, conditions that can be determined within our code. This concept is as old as compilers. Chances are, if you're writing code, you're going to be using a slew of these things, even in an Event based language like iRules. iRules is no different than any other programming/scripting language when it comes to conditionals; we have them. Sure how they're implemented and what they look like change from language to language, but most of the same basic tools are there: if, else, switch, elseif, etc. Just about any example that you might run across on DevCentral is going to contain some example of these being put to use. Learning which conditional to use in each situation is an integral part to learning how to code effectively. Once you have that under control, however, there's still plenty more to learn. Now that you're comfortable using a single conditional, what about starting to combine them? There are many times when it makes more sense to use a pair or more of conditionals in place of a single conditional along with logical operators. For example: if { [HTTP::host] eq "bob.com" and [HTTP::uri] starts_with "/uri1" } { pool pool1 } elseif { [HTTP::host] eq "bob.com" and [HTTP::uri] starts_with "/uri2" } { pool pool2 } elseif { [HTTP::host] eq "bob.com" and [HTTP::uri] starts_with "/uri3" } { pool pool3 } Can be re-written to use a pair of conditionals instead, making it far more efficient. To do this, you take the common case shared among the example strings and only perform that comparison once, and only perform the other comparisons if that result returns as desired. This is more easily described as nested conditionals, and it looks like this: if { [HTTP::host] eq "bob.com" } { if {[HTTP::uri] starts_with "/uri1" } { pool pool1 } elseif {[HTTP::uri] starts_with "/uri2" } { pool pool2 } elseif {[HTTP::uri] starts_with "/uri3" } { pool pool3 } } These two examples are logically equivalent, but the latter example is far more efficient. This is because in all the cases where the host is not equal to "bob.com", no other inspection needs to be done, whereas in the first example, you must perform the host check three times, as well as the uri check every single time, regardless of the fact that you could have stopped the process earlier. While basic, this concept is important in general when coding. It becomes exponentially more important, as do almost all optimizations, when talking about programming in iRules. A script being executed on a server firing perhaps once per minute benefits from small optimizations. An iRule being executed somewhere in the order of 100,000 times per second benefits that much more. A slightly more interesting example, perhaps, is performing the same logical nesting while using different operators. In this example we'll look at a series of if/elseif statements that are already using nesting, and take a look at how we might use the switch command to even further optimize things. I've seen multiple examples of people shying away from switch when nesting their logic because it looks odd to them or they're not quite sure how it should be structured. Hopefully this will help clear things up. First, the example using if statements: when HTTP_REQUEST { if { [HTTP::host] eq "secure.domain.com" } { HTTP::header insert "Client-IP:[IP::client_addr]" pool sslServers } elseif { [HTTP::host] eq "www.domain.com" } { HTTP::header insert "Client-IP:[IP::client_addr]" pool httpServers } elseif { [HTTP::host] ends_with "domain.com" and [HTTP::uri] starts_with "/secure"} { HTTP::header insert "Client-IP:[IP::client_addr]" pool sslServers } elseif {[HTTP::host] ends_with "domain.com" and [HTTP::uri] starts_with "/login"} { HTTP::header insert "Client-IP:[IP::client_addr]" pool httpServers } elseif { [HTTP::host] eq "intranet.myhost.com" } { HTTP::header insert "Client-IP:[IP::client_addr]" pool internal } } As you can see, this is completely functional and would do the job just fine. There are definitely some improvements that can be made, though. Let's try using a switch statement instead of several if comparisons for improved performance. To do that, we're going to have to use an if nested inside a switch comparison. While this might be new to some or look a bit odd if you're not used to it, it's completely valid and often times the most efficient you’re going to get. This is what the above code would look like cleaned up and put into a switch: when HTTP_REQUEST { HTTP::header insert "Client-IP:[IP::client_addr]" switch -glob [HTTP::host] { "secure.domain.com" { pool sslServers } "www.domain.com" { pool httpServers } "*.domain.com" { if { [HTTP::uri] starts_with "/secure" } { pool sslServers } else { pool httpServers } } "intranet.myhost.com" { pool internal } } } As you can see this is not only easier to read and maintain, but it will also prove to be more efficient. We've moved to the more efficient switch structure, we've gotten rid of the repeat host comparisons that were happening above with the /secure vs /login uris, and while I was at it I got rid of all those examples of inserting a header, since that was happening in every case anyway. Hopefully the benefit this technique can offer is clear, and these examples did the topic some justice. With any luck, you'll nest those conditionals with confidence now.How to get a F5 BIG-IP VE Developer Lab License
(applies to BIG-IP TMOS Edition) To assist DevOps teams improve their development for the BIG-IP platform, F5 offers a low cost developer lab license.This license can be purchased from your authorized F5 vendor. If you do not have an F5 vendor, you can purchase a lab license online: CDW BIG-IP Virtual Edition Lab License CDW Canada BIG-IP Virtual Edition Lab License Once completed, the order is sent to F5 for fulfillment and your license will be delivered shortly after via e-mail. F5 is investigating ways to improve this process. To download the BIG-IP Virtual Edition, please log into downloads.f5.com (separate login from DevCentral), and navigate to your appropriate virtual edition, example: For VMware Fusion or Workstation or ESX/i:BIGIP-16.1.2-0.0.18.ALL-vmware.ova For Microsoft HyperV:BIGIP-16.1.2-0.0.18.ALL.vhd.zip KVM RHEL/CentoOS: BIGIP-16.1.2-0.0.18.ALL.qcow2.zip Note: There are also 1 Slot versions of the above images where a 2nd boot partition is not needed for in-place upgrades. These images include_1SLOT- to the image name instead of ALL. The below guides will help get you started with F5 BIG-IP Virtual Edition to develop for VMWare Fusion, AWS, Azure, VMware, or Microsoft Hyper-V. These guides follow standard practices for installing in production environments and performance recommendations change based on lower use/non-critical needs fo Dev/Lab environments. Similar to driving a tank, use your best judgement. DeployingF5 BIG-IP Virtual Edition on VMware Fusion Deploying F5 BIG-IP in Microsoft Azure for Developers Deploying F5 BIG-IP in AWS for Developers Deploying F5 BIG-IP in Windows Server Hyper-V for Developers Deploying F5 BIG-IP in VMware vCloud Director and ESX for Developers Note: F5 Support maintains authoritativeAzure, AWS, Hyper-V, and ESX/vCloud installation documentation. VMware Fusion is not an official F5-supported hypervisor so DevCentral publishes the Fusion guide with the help of our Field Systems Engineering teams.iRule Security 101 - #1 - HTTP Version
When looking at securing up your web application, there are a set of fairly standard attack patterns that application firewalls make use of to protect against those bad guys out there who are trying exploit your website. A good reference for web application attacks is the Open Web Application Security Project (OWASP). In this series of blog posts, I'm going to highlight different attacks and how they can be defended against by using iRules. In the first installment of this series I will show how to only allow valid HTTP requests to your application server.The most common HTTP versions out there are 1.0 and 1.1 although version0.9 is still used in places. A common attempt to fool an application is by passing an invalid HTTP Version causing the server to not interpret the request correctly. The "HTTP version" iRules command contains the request version and you can ensure that only valid requests are processed and allowed to your app servers with this iRule: when RULE_INIT { set INFO 0 set DEBUG 0 #------------------------------------------------------------------------ # HTTP Version #------------------------------------------------------------------------ set sec_http_version_enabled 0 set sec_http_version_block 1 set sec_http_version_alert 1 set sec_http_versions [list \ "0.9" \ "1.0" \ "1.1" \ ] } when HTTP_REQUEST { #============================================================================ # HTTP Version #============================================================================ if { $::INFO } { log local0. "ASSERTION: http_version" } if { $::sec_http_version_enabled } { if { $::DEBUG } { log local0. " HTTP Version: [HTTP::version]" } if { ! [matchclass [HTTP::version] equals $::sec_http_versions ] } { if { $::sec_http_version_alert } { log local0. " SEC-ALERT: Invalid HTTP Version found: '[HTTP::version]'" } if { $::sec_http_version_block } { reject } } else { if { $::DEBUG } { log local0. " PASSED" } } } } In the RULE_INIT method I've created a few global variablesenabling one to turn on or off theverification. Without all the extraconditionals, the iRule can be stripped down to the following couple of lines: when RULE_INIT { set sec_http_versions [list "0.9" "1.0" "1.1" ] } when HTTP_REQUEST { if { ! [matchclass [HTTP::version] equals $::sec_http_versions ] } { reject } } Stay tuned for the next installment of iRules Security 101 where I'll show how to validate HTTP methods. -JoeDeploying F5 BIG-IP in VMware vCloud Director and ESX for Developers
F5’s BYOL (Bring Your Own License) model allows you to purchase a developer/lab license and install it in your VMware infrastructure. This model provides a stable development instance for the following software components of BIG-IP: LTM, GTM, DNS, AFM, ASM, APM Lite (10 users), AAM, CGN, SSL Forward Proxy, Advanced Protocols, and Crypto Offload all at a 10Mbps rate limit. Please refer to these resources to get started with VMware vCloud Director and ESX(i) How to get a F5 BIG-IP VE Developer Lab License BIG-IP Virtual Edition Setup Guide for VMware vCloud Director @ support.f5.com BIG-IP Virtual Edition Setup Guide for VMware ESXi v12.0 @ support.f5.com (PDF) NOTE: DevCentral recommends no less than 4GB RAM and prefer 8 to 24 depending on what modules requiring activation (the more the merrier right?). For Developer environments, you can deviate from staging and production requirements allowing you develop applications and solutions across multiple BIG-IP modules. DevCentral uses multiple vm’s each with standardized configurations found in BIG-IP’s product catalog to ensure that we don’t produce unsupportable materials for ingestion by the community. You can do whatever you want!Deploying F5 BIG-IP in Windows Server Hyper-V for Developers
F5’s BYOL (Bring Your Own License) model allows you to purchase a developer/lab license and deployinto your Microsoft Hyper-V environment. This provides a stable development instance for the following software components of BIG-IP: LTM, GTM, DNS, AFM, ASM, APM Lite (10 users), AAM, CGN, SSL Forward Proxy, Advanced Protocols, and Crypto Offload all at a 10Mbps rate limit. Please refer to these resources to get started with Microsoft Hyper-V How to get a F5 BIG-IP VE Developer Lab License F5 BIG-IP Virtual Edition Setup Guide for Microsoft Hyper-V @ support.f5.com Hyper-V Overview @ Microsoft Technet NOTE: For Development and Lab environments, DevCentral recommends 2 logical processors and no less than 4GB RAM, preferring 8GB. This will allow you to run more than one BIG-IP software module at a time, i.e. BIG-IP DNS & LTM. As you change development requirements you can enable and disable other modules within the Developer Lab license as needed. You can even build multiple environments and enable different modules accordingly using multiple Developer Lab licenses. For staging and production environments, follow F5 recommended guidelines; DevCentral recommendations are for development and sandbox environments.Deploying F5 BIG-IP Virtual Edition on VMware Fusion
To deploy BIG-IP Virtual Edition on your workstation, VMware provides two great solutions: VMware Fusion Pro for OSX VMware Workstation Pro For this guide, we’ll use Fusion Pro 8 (v11 functions the same) due to it’s good network management abilities; for the non-Pro version refer to Jason Rahm’s article on setting up networking. Using the BIG-IP Virtual Edition, you can setup a development environment for most BIG-IP software solutions, including but not limited to LTM, APM Lite, ASM, AFM, and BIG-IP DNS. For more team oriented test or dev environments, you should probably install those to more robust infrastructure everyone has access too. Installation Instructions Installing and configuring VMware Fusion Pro Installing additional VMware networking Downloading the F5 BIG-IP Virtual Edition Importing BIG-IP VE to VMware Fusion F5 BIG-IP Configuration Configuring the Management Interface Obtaining an F5 BIG-IP Developer Edition License Configuring External and Internal Networks on BIG-IP VE Accessing BIG-IP VE GUI and Completing Setup and Licensing Configure BIG-IP System Settings Additional Information Installing and configuring VMware Fusion Pro Follow this link to purchase and download VMware Fusion Pro Install VMware and take advantage of their Getting Started Guide if unfamiliar with the product Installing additional VMware networking Start VMware Fusion Pro, and select the menu VMware Fusion > Preferences Click the Network icon Click the lock icon to authenticate and create additional networks Click the + icon 3 times to create vmnet2, vmnet3, and vmnet4 Select vmnet2 and configure the following network: Leave Allow virtual machines on this network to connect to external networks (using NAT) cleared Leave the Connect the host Mac to this network selected Leave Provide addresses on this network via DHCP selected In the Subnet IP field, enter 10.128.1.0 In the Subnet mask field, enter 255.255.255.0 Select vmnet3 and configure the following network: Select the Allow virtual machines on this network to connect to external networks (using NAT) to allow your BIG-IP VE to reach the internet Leave the Connect the host Mac to this network selected Leave Provide addresses on this network via DHCP selected In the Subnet IP field, enter 10.128.10.0 In the Subnet mask field, enter 255.255.255.0 Select vmnet4 and configure the following network: Leave Allow virtual machines on this network to connect to external networks (using NAT) Clear the Connect the host Mac to this network to prevent the system from having direct access to the internal network Leave Provide addresses on this network via DHCP selected In the Subnet IP field, enter 10.128.20.0 In the Subnet mask field, enter 255.255.255.0 Click Apply and close the window Downloading the F5 BIG-IP Virtual Edition Navigate and login at https://downloads.f5.com, if you do not have a support login, register here. Click Find a Download, select BIG-IP v12.x / Virtual Edition, and click Virtual-Edition again. Read the License Agreement and click I Accept (it’s a fantastic read) Select the BIGIP-currentversion.ALL-scsi.ovafile, with the description Image file set for VMware ESX/iServer Choose the nearest download location Importing BIG-IP Virtual Edition Image From VMware Fusion, navigate to File > Import Click Choose File Select the BIGIP-13.0.0.3.0.1679.ALL-scsi.ova image file from your download location and click Open Click Continue Name the new virtual machine whatever you want using common sense, for our example we’ll use BIGIP_v13_lab Click Accept After the import completes, click Finish, and Customize Settings Click Processors & Memory and adjust memory to provide the following: If System = 8GB, set VM memory to 4096 If System = 16GB, set VM memory to 8192 If System = 24GB+, set VM memory to 12416 Click Show All Click Network Adapter, and click vmnet2 Click Show All, then click Network Adapter 2, select vmnet3 Click Show All, then click Network Adapter 3, select vmnet4 Click Show All, then click Network Adapter 4, and uncheck the Connect Network Adapter to disable Close the Settings window F5 BIG-IP Configuration Configuring the Management Interface Click your BIG-IP VE Image from the Virtual Machine Library, then click Start Up After the BIG-IP VE powers up, you’ll be presented with the localhost login screen Log in to the BIG-IP system using the following default credentials localhost login: root Password: default At the CLI prompt, type: config Press Enter to activate the OK option Use the Tab key to activate the No option, then press Enter Edit the IP Address to 10.128.1.145, then press Tab to activate the OK option, and press Enter Ensure the Netmask is 255.255.255.0, then press Tab to activate the OK option, and press Enter Press Enter to activate the Yes option to create a default route for the management port Edit the Management Route to 10.128.1.1, then press the Tab to activate the OK option, and press Enter Press the Enter key to activate the Yes option to accept the settings Obtaining an F5 BIG-IP Developer Edition License Refer to How to get a F5 BIG-IP VE Developer Lab License to purchase your Developer License. Configuring External and Internal Networks on BIG-IP VE Open a terminal window, and type: ssh root@10.128.1.145 Use the following Password: default Copy or manually enter the following TMSH commands to your SSH session. You can copy and past all the lines simultaneously tmsh create net vlan external interfaces add { 1.1 { untagged } } tmsh create net vlan internal interfaces add { 1.2 { untagged } } tmsh create net self 10.128.10.240 address 10.128.10.240/24 vlan external tmsh create net self 10.128.20.240 address 10.128.20.240/24 vlan internal tmsh create net route Default_Gateway network 0.0.0.0/0 gw 10.128.10.1 tmsh save sys config exit Accessing BIG-IP VE GUI and Completing Setup and Licensing Open a web browser and access https://10.128.1.145 Log into the BIG-IP VE using the following credentials: Username: admin Password: admin On the Welcome Page click Next On the License page click Activate Open the email from F5 Networks with your Developer License Registration Key and copy the Registration Key text In the Setup Utility, in the Base Registration Key field, past the registration key text For Activation Method, select Manual, and click Next Select and copy all of the dossier text to your clipboard Select Click here to access F5 Licensing Server On the Activate F5 Product page, paste the dossier text in the field, then click Next Select to accept the legal agreement, then click Next Select and copy all of the license key text to your clipboard On the Setup Utility > License page, paste the license key text into the Step 3: License field, then click Next After the configuration changes complete, log into the BIG-IP VE system using the previous credentials On the Resource Provisioning page leave Local Traffic (LTM) as the only provisioned module and click Next On the Device Certificates page click Next On the Platform page, configure the Host Name, Root Account, and Admin Account to your desired settings, then click Next You’ll be prompted to log out and back into the BIG-IP VE. Do it. Under Standard Network Configuration, click Next Clear the Display configuration synchronization options checkbox, then click Next On the Internal Network Configuration page, review the settings, then click Next On the External network Configuration page, review the settings, then click Finished to complete the Setup Utility. Configure BIG-IP System Settings Open the System > Preferences page, and update the following settings, then click Update Records Per Screen: 30 Start Screen: Statistics Idle Time Before Automatic Logout: 100000 seconds Security Banner Text: Welcome to the F5 BIG-IP VE Lab Environment (or whatever you want this to say) Open the System > Configuration > Device > DNS page For DNS Lookup Server List, enter 8.8.8.8, and then click Add (you can use whatever DNS resolver you want here) Select 10.128.1.1, then click Delete, and click Update Open the Local Traffic > Nodes > Default Monitor page Click ICMP, and click << to move it to the Active list, then click Update Additional Information Using the 10.128.x.0/24 is intended only for ease of use and not a requirement. If you have alternate requirements, please replace our examples This guide builds a sufficient external and internal network the BIG-IP can use for proxy architecture testing and is intended for development purposes only If you opted not to purchase the Pro version of Fusion, you can still setup advanced networking. For more on this please see: VMware Fusion Custom Networking for BIG-IP VE Lab This guide is developed for VMware Fusion Pro on OSX. If you run VMware Workstation, setup is the same, only the UX and configuration locations change.iControl 101 - #06 - File Transfer APIs
The main use we see for iControl applications is with the automation of control features such as adding, removing, enabling, and disabling objects. Another key step in multi-device management is the automation of applying hotfixes and other software updates as well as the downloading of configurations for archival and disaster recovery purposes. iControl has a set of methods that enable the uploading and downloading of files for these purposes. This article will discuss these "file transfer" APIs and how you can use them for various management purposes. The File Transfer APIs The API methods uses to transfer files to and from the device can be found in the System::ConfigSync interface. You may ask: Why are they ConfigSync interface? The answer is quite simple actually. In our first version of iControl, we had the need to transfer configurations across devices in a HA pair as part of the Configuration Sync process. So in doing so, we introduced the upload_configuration() and download_configuration() methods. When we later added more generic file transfer APIs, it seemed logical to place them next to the pre-existing configuration transfer methods. So, that's the reason... The following methods are used to download content: FileTransferContext System::ConfigSync::download_configuration( in String config_name, in long chunk_size, inout long file_offset ); FileTransferContext System::ConfigSync::download_file( in String file_name, in long chunk_size, inout long file_offset ); And the following two methods are used to upload content: void System::ConfigSync::upload_configuration( in String config_name, in System::ConfigSync::FileTransferContext file_context ); void System::ConfigSync::upload_file( in String file_name, in System::ConfigSync::FileTransferContext file_context ); The above methods use the following enum and structure as part of the control enum Common::FileChainType { FILE_UNDEFINED = 0 FILE_FIRST = 1, FILE_MIDDLE = 2, FILE_UNUSED = 3, FILE_LAST = 4, FILE_FIRST_AND_LAST }; struct System.::ConfigSync::FileTransferContext { char [] file_data, Common::FileChainType chain_type }; Chunks Due to the limitations with SOAP's with regards to payload and processing of large messages, we designed the file transfer APIs to work in "chunks". This means that for a multi-megabyte file, you a loop in your code to send up "chunks" in whatever chunk sizes you wish from 1 byte to 100's of Ks. We recommend not getting too extreme on either end of the chunk size ranges. Typically we recommend 64 to 256k per chunk. How to use the download methods. The process for downloading content is fairly straightforward. In this example, we'll use the download_configuration method. 1. Make a call to the download_configuration method with a given configuration name (a list of existing configurations can be returned from ConfigSync::get_configuration_list) with the requested chunk_size (ie 64k) and the starting file_offset of 0. 2. The first response will come back in the FileTransferContext with the data and the FileChainType describing whether this was the first chunk, a middle chunk, the last chunk, or the first and last chunk. 3. take the encoded data stored in the FileTransferContext.file_data array and save it locally. 4. If the FileChainType is FILE_LAST or FILE_FIRST_AND_LAST, you are done. 5. Otherwise, use the incremented file_offset go to step #1. The following snippet of code taken from the iControl SDK's ConfigSync C# sample application illustrates how to deal with downloading a file with an unknown size. void handle_download(string config_name, string local_file) { ConfigSync.SystemConfigSyncFileTransferContext ctx; long chunk_size = (64*1024); long file_offset = 0; bool bContinue = true; FileMode fm = FileMode.CreateNew; if ( File.Exists(local_file) ) { fm = FileMode.Truncate; } FileStream fs = new FileStream(local_file, fm); BinaryWriter w = new BinaryWriter(fs); while ( bContinue ) { ctx = ConfigSync.download_configuration(config_name, chunk_size, ref file_offset); // Append data to file w.Write(ctx.file_data, 0, ctx.file_data.Length); Console.WriteLine("Bytes Transferred: " + file_offset); if ( (CommonFileChainType.FILE_LAST == ctx.chain_type) || (CommonFileChainType.FILE_FIRST_AND_LAST == ctx.chain_type) ) { bContinue = false; } } w.Close(); } How to use the upload methods The upload methods work in a similar way but in the opposite direction. To use the upload_configuration configuration method, you would use the following logic. 1. Determine the chunk size you are going to use (64k - 256k recommended) and fill the file_data array with the first chunk of data. 2. Set the FileChainType to FILE_TYPE_FIRST_AND_LAST if all your data can fit in the first chunk size. 3. Set the FileChainType to FILE_FIRST if you fill up the data with your chunk size and there is more data to follow. 4. Make a call to upload_configuration with the configuration name and the FileTransferContext with the data and the FileChainType. 5. If the data has all been sent, stop processing. 6. Else if the remaining data will able to fit in the given chunk size, set the FileChainType to FILE_LAST, otherwise set it to FILE_MIDDLE. 7. Fill the file_data with the next chunk of data. 8. Goto Step #4. The following example taken from the iControl SDK ConfigSync perl sample and illustrates taking a local configuration file and uploading it to the device. sub uploadConfiguration() { my ($localFile, $configName) = (@_); $success = 0; $bContinue = 1; $chain_type = $FILE_FIRST; $preferred_chunk_size = 65536; $chunk_size = 65536; $total_bytes = 0; open(LOCAL_FILE, "<$localFile") or die("Can't open $localFile for input: $!"); binmode(LOCAL_FILE); while (1 == $bContinue ) { $file_data = ""; $bytes_read = read(LOCAL_FILE, $file_data, $chunk_size); if ( $preferred_chunk_size != $bytes_read ) { if ( $total_bytes == 0 ) { $chain_type = $FILE_FIRST_AND_LAST; } else { $chain_type = $FILE_LAST; } $bContinue = 0; } $total_bytes += $bytes_read; $FileTransferContext = { file_data => SOAP::Data->type(base64 => $file_data), chain_type => $chain_type }; $soap_response = $ConfigSync->upload_configuration ( SOAP::Data->name(config_name => $configName), SOAP::Data->name(file_context => $FileTransferContext) ); if ( $soap_response->fault ) { print $soap_response->faultcode, " ", $soap_response->faultstring, "\n"; $success = 0; $bContinue = 0; } else { print "Uploaded $total_bytes bytes\n"; $success = 1; } $chain_type = $FILE_MIDDLE; } print "\n"; close(LOCAL_FILE); return $success; } Other methods of data transfer The two methods illustrated above are specific to system configurations. The more generic upload_file() and download_file() commands may be used to do things like backing up other system files as well as, but not limited to, uploading hotfixes to be later installed with the System::SoftwareManagement::install_hotfix() method. The usage of those methods is identical to the configuration transfer methods except in the fact that the config_name parameter is now replaced with a fully qualified file system name. Note that there are some restrictions as to the file system locations that are readable and writable. You wouldn't want to accidentally overwrite the system kernel with your latest hotfix would you? Conclusion Hopefully this article gave you some insights on how to use the various file transfer APIs to manipulate file system content on your F5 devices. Get the Flash Player to see this player.iRule Security 101 - #04 - Masking Application Platform
In this session of iRules Security 101, I'll show you how to hide your backend server application platform from the outside public. When you are browsing a website, you are likely to look at the URI's to determine what platform the application is running on. For instance, if you saw "http://www.foo.com/foo.asp", you would likely assume that the backend application was an ASP application running under Microsoft IIS. Likewise, if you saw "http://www.foo.com/foo.jsp", you would assume it was a java server platform. Well, in the world of exploits, wouldn't be best to hide the platform from application is running on from the outside world? This article will illustrate one way to do so. Other articles in the series: iRule Security 101 – #1 – HTTP Version iRule Security 101 – #02 – HTTP Methods and Cross Site Tracing iRule Security 101 – #03 – HTML Comments iRule Security 101 – #04 – Masking Application Platform iRule Security 101 – #05 – Avoiding Path Traversal iRule Security 101 – #06 – HTTP Referer iRule Security 101 – #07 – FTP Proxy iRule Security 101 – #08 – Limiting POST Data iRule Security 101 – #09 – Command Execution In this article, we will look at the URI extensions that your application uses. For this example, let's assume that your backend webserver is running an ASP application and you don't want the world to know it. We'll create a new "public" extension that the outside world will interact. In this example, we'll use ".joe". Hey, I wrote the article, so I get to name the extension B-). Let's first look at the HTTP request: when HTTP_REQUEST { # Don't allow data to be chunked. This ensures we don't get # a comment that is spread across two chunked boundaries. if { [HTTP::version] eq "1.1" } { if { [HTTP::header is_keepalive] } { HTTP::header replace "Connection" "Keep-Alive" } HTTP::version "1.0" } set orig_uri [HTTP::uri] log local0. "Old URI: $orig_uri" switch -glob $orig_uri { "*.html*" { log local0. "Found request to internal resource. Redirect to external resource" set new_uri [string map {".html" ".joe"} [HTTP::uri]] HTTP::redirect "http://[HTTP::host]$new_uri" } "*.jsp*" { log local0. "Found request to internal resource. Redirect to external resource" set new_uri [string map {".jsp" ".joe"} [HTTP::uri]] HTTP::redirect "http://[HTTP::host]$new_uri" } "*.asp*" { log local0. "Found request to internal resource. Redirect to external resource" set new_uri [string map {".asp" ".joe"} [HTTP::uri]] HTTP::redirect "http://[HTTP::host]$new_uri" } "*.joe*" { log local0. "Found external resource request, mapping URI to internal name" HTTP::uri [string map {".joe" ".asp"} [HTTP::uri]] } } } The HTTP_REQUEST event looks at the incoming URI and for the some common application extensions (.html, .jsp, and .asp), we'll redirect to the public facing ".joe" uri. The reason I did this if we only masked the internal application type of .asp, then all other requests wouuld return 404 (file not founds) from the server and the .asp requests would redirect. This would be a good sign we are hiding the .asp extension. By redirectly multiple extensions, there is no indication what the true extension really is. Here are some example redirects that this would generate: http://www.foo.com/index.html -> http://www.foo.com/index.joe http://www.foo.com/default.asp -> http://www.foo.com/default.joe http://www.foo.com/login.jsp -> http://www.foo.com/login.joe The last case in the switch statement turns all .joe requests to the true .asp requests before sending passing the request to the application server. Here are some sample URL transformations that would be made http://www.foo.com/default.joe --> http://www.foo.com/default.asp http://www.foo.com/login.joe?username=foobar --> http://www.foo.com/login.asp?username=foobar Now that the request is taken care of, we most likely should modify the responses to change all embedded URLs in the response content so that they reflect the external extension. when HTTP_RESPONSE { if { $orig_uri ends_with ".joe" } { # Ensure all of the HTTP response is collected if { [HTTP::header exists "Content-Length"] } { set content_length [HTTP::header "Content-Length"] } else { set content_length 1000000 } if { $content_length > 0 } { HTTP::collect $content_length } } if { [HTTP::header exists "Server" ] } { HTTP::header replace "Server" "Joe's Awesome App Server" } } when HTTP_RESPONSE_DATA { set new_payload [string map {".asp" ".joe"} [HTTP::payload]] HTTP::payload replace 0 [HTTP::payload length] $new_payload } In the HTTP_RESPONSE event, we check to see if the original uri was to our external extension. If so, trigger a collection of the payload from the backend server and perform a simple string replacement in the HTTP_RESPONSE_DATA event and update the response payload with the "HTTP::payload replace" command (make sure you have rechunking enabled in your HTTP profile). Oh, and make sure you check out the last line in the HTTP_RESPONSE event. Another giveaway as to your server type, is the "Server" HTTP response header. I went ahead and modified this to remove any server based identification. There are several ways you could enhance this by adding support for default index pages as well as adding support for other response content coming from non ".joe" based uri requests. You may also have more than one internal extension that you want to hide (.dll clearly indicates a windows machine). One could create multiple mappings (.joe-a -> .asp, .joe-b -> .dll, .joe-c -> .cgi, ...) and make the approprate redirections in the request and modifications in the response. Keep in mind that there are many ways a server can identify itself to the outside world. This iRule doesn't protect against all types of server "signatures" but gives you a good start. Get the Flash Player to see this player.Getting the most out of DevCentral: User Accepted Answers
As you may have noticed, we recently added a few new features to our Q&A experience. In this article, we will explore the User Accepted Answer feature. When a user asks a question and that question receives several answers, another user who reads the entire Q&A thread might wonder which of the answers is really the best one. Maybe several different solutions were presented in all the various answers. Maybe one of them works better than the others. How do you know which answer is the best? User Accepted Answer The User Accepted Answer is exactly what it sounds like…the user who asked the question has marked a certain answer as “accepted” according to his own personal standards for acceptable answers. Of course, each user has different standards by which he will accept an answer, so the “User Accepted” answer simply means that the given answer has proven to be acceptable for that specific user. When an answer is marked “User Accepted”, it will move to the top of the list of answers and be highlighted as shown in the screenshot below. If you want to mark an answer as “User Accepted”, you simply navigate to the question you asked and look for the “checkbox” button shown to the left of each answer (just below the voting buttons). When you click on that checkbox, it will mark that specific answer as “User Accepted”. Of course, you will only see the “checkbox” button option on the questions that you ask. See the screenshot below with the checkbox button circled: Q&A Listing When we back out into the main Q&A page, you’ll notice three boxes on the right side of each question…the first is the number of votes it has received, the second is the total number of answers it has received, and the third is the number of views it has received. Our recent upgrade highlights the “answers” box with a darker shade of color when that specific question has an “accepted” answer. This allows you to know right away that a question has an accepted answer before you ever click to see the details of the question.
