F5 Friday: Load Balancing MySQL with F5 BIG-IP
Scaling MySQL just got a whole lot easier load balancing MySQL – any database, really – is not a trivial task. Generally speaking one does not simply round robin your way through a cluster of MySQL databases as a means to achieve scalability. It is databases, in fact, that have driven a wide variety of scalability patterns such as sharding and partitioning to achieve the ultimate goal of high-performance and scalability simultaneously. Unfortunately, most folks don’t architect their applications with scalability in mind. A single database is all that’s necessary at first, and because of the way in which the application interacts with the database, it doesn’t make sense to code in support for multiple database instances, such as is often implemented with a MySQL master-slave cluster. That’s because the application has to actually open a connection to the database in question. If you’re only starting with one database, you really can’t code in a connection to a separate instance. Eventually that application’s usage grows and the demands upon the database require a more scalable approach. Enter the MySQL master/slave relationship. A typical configuration is to maintain the master as the “write” database, i.e. all updates and/or inserts must use the master, while the slave instance is used as a “read only” instance. Obviously this means the application code must be changed to support this kind of functional sharding. Unless you leverage network server virtualization from a load balancing service capable of acting as a full-proxy at layer 7 (application) like BIG-IP. This solution leverages iRules to implement database load balancing. While this specific example is designed to perform the common functional sharding pattern of read-write separation for a master-slave MySQL cluster, the flexibility of iRules is such that other architectural solutions can easily be designed using the same basic functions. Location based sharding is another popular means of scaling databases, and using the GeoLocation capabilities of BIG-IP along with iRules to inspect and route database requests, it should be a fairly trivial architectural task to implement. The ability to further extend sharding or other distribution methodologies for scaling databases without modifying the application itself is a huge bonus for both developers and operations. By decoupling the application from the database, it provides a more flexibility set of scalability domains in which technology targeted scalability strategies can be leveraged independent of the other layers. This is an important facet of agile infrastructure architecture and should not be underestimated as a benefit of network server virtualization. MySQL Load Balancing Resources: MySQL Proxy iRule MySQL Proxy iApp (deployment package for BIG-IP v11) The Full-Proxy Data Center Architecture Infrastructure Scalability Pattern: Sharding Streams Infrastructure Scalability Pattern: Sharding Sessions Infrastructure Scalability Pattern: Partition by Function or Type IT as a Service: A Stateless Infrastructure Architecture Model F5 Friday: Platform versus Product At the Intersection of Cloud and Control… What is a Strategic Point of Control Anyway? All F5 Friday Posts on DevCentral Why Single-Stack Infrastructure Sucks
Horizon View "This Page is Not Secure"
I have a connection to my VDI desktops via F5 (build using the iApp) and it essentially works i.e. I can get a virtual desktop although with a slight issue. To start with I enter the URL e.g. https://myvdi.mydomain.com Then after authenticating on the connection server and making my choice of desktop, the URL in the address bar changes to an IP in the range of the private LAN for the virtual desktops e.g. https://10.180.0.80:22443/d/DE841123-FE72-4C6D-A9F3-2E6B7072D7E1/certAccept.html?numPages=3 This results in a typical "this site is not secure" page in IE which I have to manually press on "go on to the webpage." Once I manually continue everything is fine as the URL is then https://myvdi.mydomain.com/portal/webclient/index.html/desktop and I get my authenticated, secure desktop. Does anyone know how I can stop this behaviour?
HTTP Explicit Proxy - V11.5+
Problem this snippet solves: This iApp configures an Explicit Proxy using the new "Explicit" Proxy Mode that was introduced into the HTTP Profile in BIG-IP 11.5. You only need LTM or APM provisioned. It creates all configuration components required including: DNS Resolvers TCP Tunnel HTTP Profile (Explicit) Default Connect Handling set to Allow SNAT Pools (Optional) SNAT Default is Automap If you require the Explicit Proxy to listen on more than 1 port e.g 3128 and 8080, simply just create another Application Service. Contributed by: Brett Smith How to use this snippet:What is an iApp?
iApp is a seriously cool, game changing technology that was released in F5’s v11. There are so many benefits to our customers with this tool that I am going to break it down over a series of posts. Today we will focus on what it is. Hopefully you are already familiar with the power of F5’s iRules technology. If not, here is a quick background. F5 products support a scripting language based on TCL. This language allows an administrator to tell their BIG-IP to intercept, inspect, transform, direct and track inbound or outbound application traffic. An iRule is the bit of code that contains the set of instructions the system uses to process data flowing through it, either in the header or payload of a packet. This technology allows our customers to solve real-time application issues, security vulnerabilities, etc that are unique to their environment or are time sensitive. An iApp is like iRules, but for the management plane. Again, there is a scripting language that administrators can build instructions the system will use. But instead of describing how to process traffic, in the case of iApp, it is used to describe the user interface and how the system will act on information gathered from the user. The bit of code that contains these instructions is referred to as an iApp or iApp template. A system administrator can use F5-provided iApp templates installed on their BIG-IP to configure a service for a new application. They will be presented with the text and input fields defined by the iApp author. Once complete, their answers are submitted, and the template implements the configuration. First an application service object (ASO) is created that ties together all the configuration objects which are created, like virtual servers and profiles. Each object created by the iApp is then marked with the ASO to identify their membership in the application for future management and reporting. That about does it for what an iApp is…..next up, how they can work for you.VMware Horizon and F5 iAPP Deployments Backed by Ansible Automation
The Intro: A little over a year ago I knew barely anything about automation, zero about ansible, and didn't even think it would be something so tied to my life like it is now. I spend all my moments trying to think about how I can make Automation easier in my life, and being in Business Development I spend a lot of time testing F5 solutions and integrations between vendors (specifically between F5 and VMware as well as F5 and RedHat Ansible). I figured why not bring them a little closer together? It takes forever to build Labs and setup environments, and with automation I can do this in mere minutes compared to the hours it use to take (we are talking fresh builds, clean environments). I plan on sharing more about more of my VMware and Ansible automation integrations down the chain (like Horizon labs that can be built from scratch and ready to test in 30 minutes or less). But I wanted to start out with something that I get a lot of questions about:is it possible to automate iApp Deployments? Specifically the VMware Horizon iApp? The answer is YOU CAN NOW! grant you this like all automation is a work in progress. My suggestion is if you have a use case you want to build using what I have started with I encourage it!! TAKE, FORK and Expand!!!! The Code: All of the code I am using is completely accessible via the F5 DevCentral Git Repository and feel free to use it! What does it do? Well, if you are an F5 Guru then you might think it looks similar to how our AS3 code works, if you aren't a Guru its basically taking one set of variables and sending off a single command to the F5 to build the Application (I tell it the things that make it work, and how I want it deployed and it does all the work for me). Keep in mind this isn't using F5 AS3 code, it just mimics the same methods bytaking a JSON declaration of how I want things to be and the F5 does all of the imperative commands for me. --- - name: Build JSON payload ansible.builtin.template: src=f5.horizon.{{deployment_type |lower }}.j2 dest=/tmp/f5.horizon.json - name: Deploy F5 Horizon iApp f5networks.f5_modules.bigip_iapp_service: #Using Collections if not use - bigip_iapp_service: name: "VMware-Horizon" template: "{{iapp_template_name}}" parameters: "{{ lookup('template', '/tmp/f5.horizon.json') }}" provider: server: "{{f5_ip}}" user: "{{f5_user}}" password: "{{f5_pass}}" validate_certs: no delegate_to: localhost All of this code can be found at - https://github.com/f5devcentral/f5-bd-horizon-iapp-deploy/ Deployments: Using the F5 iApp for Horizon provided many options of deployment but they were all categorized into 3 buckets F5 APM with VMware Horizon - Where the F5 acts as the Gateway for all VMware Horizon Connections (Proxying PCoIP/Blast) F5 LTM with VMware Horizon - Internal Connections to an environment from a LAN and being able to secure and load balance Connection Servers F5 LTM with VMware Unified Access Gateway - Using the F5 to load balance the VMware Unified Access Gateways (UAGs) and letting the UAGs proxy the connections. The deployments offer the ability to utilize pre-imported certificates, set the Virtual IP, add additional Connection Servers, Create the iRule for internal connections (origin header check) and much more. All of this is dependent on your deployment and the way you need it setup. The current code doesn't import in the iApp Template nor the certificates, this could be done with other code but currently is not part of this code. All three of these deployment models are considered and part of the code and how its deployed is based on the variables file "{{code_directory}}/vars/horizon_iapp_vars.yml" as shown below. Keep in mind this is using clear text (i.e. username/password for AD) for some variables you can add other ways of securing your passwords like an Ansible VAULT. #F5 Authentication f5_ip: 192.168.1.10 f5_user: admin f5_pass: "my_password" f5_admin_port: 443 #All Deployment Types deployment_type: "apm" #option can be APM, LTM or UAG #iApp Variables iapp_vip_address: "172.16.192.100" iapp_template_name: "f5.vmware_view.v1.5.9" #SSL Info iapp_ssl_cert: "/Common/Wildcard-2022" # If want to use F5 Default Cert for Testing use "/Common/default.crt" iapp_ssl_key: "/Common/Wildcard-2022" # If want to use F5 Default Cert for Testing use "/Common/default.key" iapp_ssl_chain: "/#do_not_use#" #Horizon Info iapp_horizon_fqdn: "horizon.mycorp.com" iapp_horizon_netbios: "My-Corp" iapp_horizon_domainname: "My-Corp.com" iapp_horizon_nat_addresss: "" #enter NAT address or leave empty for none # LTM Deployment Type iapp_irule_origin: - "/Common/Horizon-Origin-Header" # APM and LTM Deployment Types iapp_horizon_connection_servers: - { ip: "192.168.1.50", port: "443" } # to add Connection Servers just add additional line - { ip: "192.168.1.51", port: "443" } #APM Deployment Type iapp_active_directory_username: "my_ad_user" iapp_active_directory_password: "my_ad_password" iapp_active_directory_password_encrypted: "no" # This is still being validated but requires the encrypted password from the BIG-IP iapp_active_directory_servers: - { name: "ad_server_1.mycorp.com", ip: "192.168.1.20" } # to add Active Directory Servers just add additional lines - { name: "ad_server_2.mycorp.com", ip: "192.168.1.21" } # UAG Deployment Type iapp_horizon_uag_servers: - { ip: "192.168.199.50", port: "443" } # to add UAG Servers Just add additional lines - { ip: "192.168.199.51", port: "443" } How do the Variables integrate with the Templates? The templates are JSON based code which Ansible will inject the variables into them depending on the deployment method called. This makes it easier to templates to specific deployments because we don't hard code specific values that aren't necessary or are part of the default deployments. Advanced Deployments would require modification of the JSON code to apply specialized settings that aren't apart of the default. If you want to see more about the templates for each operation (APM/LTM/UAG) check out the JSON Code at the link below: https://github.com/f5devcentral/f5-bd-horizon-iapp-deploy/tree/main/roles/ansible-deploy-iapp/templates The Results: Within seconds I can deploy, configure and make changes to my deployments or even change my deployment type. Could I do this in the GUI? Absolutely but the point is to Automate ALL THE THINGS, and being able to integrate this with solutions like Lab in a box (built from scratch including the F5) saves massive amounts of time. Example of a VMware Horizon iApp Deployment with F5 APM done in ~12 Seconds [root@Elysium f5-bd-horizon-iapp-deploy]# time ansible-playbook horizon_iapp_deploy.yaml PLAY [localhost] ******************************************************************************************************************************************************************** TASK [bypass-variables : ansible.builtin.stat] ************************************************************************************************************************************** ok: [localhost] TASK [bypass-variables : ansible.builtin.include_vars] ****************************************************************************************************************************** ok: [localhost] TASK [create-irule : Create F5 iRule] *********************************************************************************************************************************************** skipping: [localhost] TASK [ansible-deploy-iapp : Build JSON payload] ************************************************************************************************************************************* ok: [localhost] TASK [ansible-deploy-iapp : Deploy F5 Horizon iApp] ********************************************************************************************************************************* changed: [localhost] PLAY RECAP ************************************************************************************************************************************************************************** localhost : ok=4 changed=1 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0 real 0m11.954s user 0m6.114s sys 0m0.542s Links: All of this code can be found at - https://github.com/f5devcentral/f5-bd-horizon-iapp-deploy/
Health Score iCall - Dynamic Modification of a Pool Member's Ratio
Problem this snippet solves: Code Use Case: This iApp can be used to control an iCall script that will dynamically modify a pool member's ratio based on the result of a header received from each individual pool member. The iApp exposes a list of typical questions that will in turn modify the iCall scripts behavior without having to dive into the iCall code itself. The most common example and what the original iCall script was written and tested against is SharePoint web servers will return a header called X-SharePointHealthScore with a value ranging from 0-10(0 being best) based on internal monitored metrics. The iApp managed iCall will then look at this value on an interval and apply a ratio to the pool member ranging from 10-1 as the inverse to the score above. This allows for a more in-tune approach to load balancing. Note: This will work against a pool managed by another iApp (tested with SharePoint 2010/2013/2016 iApp) as well as a manually configured pool (or a pool managed by an iApp with strict updates turned off). Link to Original iCall: https://devcentral.f5.com/codeshare/prioritize-sharepoint-nodes-on-reported-health How to use this snippet: Unzip the file, then upload the iApp template to Big-IP. Then create an application service using the template. Code : 67795 Tested this on version: 12.0F5 automated backup script not removing ucs archives/executing post copy clean up commands.
I've installed the https://devcentral.f5.com/codeshare/complete-f5-automated-backup-solution components and when utilizing the scp option have noticed that, although the archives do get copied to remote storage, the /var/local/ucs archives are not being cleaned up. I've seen posts which recommend changing from exec rm -f BACKUPDIRECTORY/$fnameBACKUPFILESCRIPTEXTENSION to exec rm -f BACKUPDIRECTORY/${fname}BACKUPFILESCRIPTEXTENSION which did not help. To me it appears that once the call to execute the bash script, which handles the scp, is performed it never properly returns back to the calling script. I'm basing this on the fact that neither /var/local/usc archives or the /var bash script are ever cleaned up. The commands to remove them immediately follows the call to the bash script. I've modified the command that builds the scp.sh bash, appending the clean up commands to the end of it, so that the clean up happens w/in the bash script before it returns back to the calling script. Before: exec echo -e "put()\n{\n\tfields=\"username server directory\"\n\ti=1\n\tf5masterkey=\$(f5mku -K)\n\tfor current_field in \$fields ; do\n\t\tsedcommand=\"\${i}p\"\n\t\tcurrent_encrypted_value=\$(sed -n \"\$sedcommand\" $configfile)\n\t\tcurrent_decrypted_value=\$(echo \"\$current_encrypted_value\" | openssl aes-256-ecb -salt -a -d -k \$f5masterkey)\n\t\teval \"\$current_field=\$current_decrypted_value\"\n\t\tlet i=\$i+1\n\t\tunset current_encrypted_value current_decrypted_value sedcommand\n\tdone\n\tsed -n '4,\$p' $configfile | openssl aes-256-ecb -salt -a -d -k \$f5masterkey > /var/tmp/scp.key\n\tchmod 600 /var/tmp/scp.key\n\tscp -i /var/tmp/scp.key BACKUPDIRECTORY/${fname}BACKUPFILESCRIPTEXTENSION \${username}@\${server}:\${directory}\n\trm -f /var/tmp/scp.key\n\treturn \$?\n}\n\nput" > $scriptfile After: exec echo -e "put()\n{\n\tfields=\"username server directory\"\n\ti=1\n\tf5masterkey=\$(f5mku -K)\n\tfor current_field in \$fields ; do\n\t\tsedcommand=\"\${i}p\"\n\t\tcurrent_encrypted_value=\$(sed -n \"\$sedcommand\" $configfile)\n\t\tcurrent_decrypted_value=\$(echo \"\$current_encrypted_value\" | openssl aes-256-ecb -salt -a -d -k \$f5masterkey)\n\t\teval \"\$current_field=\$current_decrypted_value\"\n\t\tlet i=\$i+1\n\t\tunset current_encrypted_value current_decrypted_value sedcommand\n\tdone\n\tsed -n '4,\$p' $configfile | openssl aes-256-ecb -salt -a -d -k \$f5masterkey > /var/tmp/scp.key\n\tchmod 600 /var/tmp/scp.key\n\tscp -i /var/tmp/scp.key BACKUPDIRECTORY/${fname}BACKUPFILESCRIPTEXTENSION \${username}@\${server}:\${directory}\n\tcp /var/tmp/scp.key /var/tmp/scp.key.bak\n\trm -f /var/tmp/scp.key\n\trm -f BACKUPDIRECTORY/${fname}BACKUPFILESCRIPTEXTENSION\n\trm -f $scriptfile\n\treturn \$?\n}\n\nput" > $scriptfile This seems to work but I was wondering if anyone else has encountered this issue and has a better solution?
