Power of tmsh commands using Ansible
Why is data important Having accurate data has become an integral part of decision making. The data could be for making simple decisions like purchasing the newest electronic gadget in the market or for complex decisions on what hardware and/or software platform works best for your highly demanding application which would provide the best user experience for your customer. In either case research and data collection becomes essential. Using what kind of F5 hardware and/or software in your environment follows the same principals where your IT team would require data to make the right decision. Data could vary from CPU, Throughput and/or Memory utilization etc. of your F5 gear. It could also be data just for a period of a day, a month or a year depending the application usage patterns. Ansible to the rescue Your environment could have 10's or maybe 100 or even 1000's of F5 BIG-IP's in your environment, manually logging into each one to gather data would be a highly inefficient method. One way which is a great and simple way could be to use Ansible as an automation framework to perform this task, relieving you to perform your other job functions. Let's take a look at some of the components needed to use Ansible. An inventory file in Ansible defines the hosts against which your playbook is going to run. Below is an example of a file defining F5 hosts which can be expanded to represent your 10'/100's or 1000's of BIG-IP's. Inventory file: 'inventory.yml' [f5] ltm01 password=admin server=10.192.73.xxx user=admin validate_certs=no server_port=443 ltm02 password=admin server=10.192.73.xxx user=admin validate_certs=no server_port=443 ltm03 password=admin server=10.192.73.xxx user=admin validate_certs=no server_port=443 ltm04 password=admin server=10.192.73.xxx user=admin validate_certs=no server_port=443 ltm05 password=admin server=10.192.73.xxx user=admin validate_certs=no server_port=443 A playbook defines the tasks that are going to be executed. In this playbook we are using the bigip_command module which can take as input any BIG-IP tmsh command and provide the output. Here we are going to use the tmsh commands to gather performance data from the BIG-IP's. The output from each of the BIG-IP's is going to be stored in a file that can be referenced after the playbook finished execution. Playbook: 'performance-data/yml' --- - name: Create empty file hosts: localhost gather_facts: false tasks: - name: Creating an empty file file: path: "./{{filename}}" state: touch - name: Gather stats using tmsh command hosts: f5 connection: local gather_facts: false serial: 1 tasks: - name: Gather performance stats bigip_command: provider: server: "{{server}}" user: "{{user}}" password: "{{password}}" server_port: "{{server_port}}" validate_certs: "{{validate_certs}}" commands: - show sys performance throughput historical - show sys performance system historical register: result - lineinfile: line: "\n###BIG-IP hostname => {{ inventory_hostname }} ###\n" insertafter: EOF dest: "./{{filename}}" - lineinfile: line: "{{ result.stdout_lines }}" insertafter: EOF dest: "./{{filename}}" - name: Format the file shell: cmd: sed 's/,/\n/g' ./{{filename}} > ./{{filename}}_formatted - pause: seconds: 10 - name: Delete file hosts: localhost gather_facts: false tasks: - name: Delete extra file created (delete file) file: path: ./{{filename}} state: absent Execution: The execution command will take as input the playbook name, the inventory file as well as the filename where the output will be stored. (There are different ways of defining and passing parameters to a playbook, below is one such example) ansible-playbook performance_data.yml -i inventory.yml --extra-vars "filename=perf_output" Snippet of expected output: ###BIG-IP hostname => ltm01 ### [['Sys::Performance Throughput' '-----------------------------------------------------------------------' 'Throughput(bits)(bits/sec) Current 3 hrs 24 hrs 7 days 30 days' '-----------------------------------------------------------------------' 'Service 223.8K 258.8K 279.2K 297.4K 112.5K' 'In 212.1K 209.7K 210.5K 243.6K 89.5K' 'Out 21.4K 21.0K 21.1K 57.4K 30.1K' ' ' '-----------------------------------------------------------------------' 'SSL Transactions Current 3 hrs 24 hrs 7 days 30 days' '-----------------------------------------------------------------------' 'SSL TPS 0 0 0 0 0' ' ' '-----------------------------------------------------------------------' 'Throughput(packets)(pkts/sec) Current 3 hrs 24 hrs 7 days 30 days' '-----------------------------------------------------------------------' 'Service 79 82 83 63 62' 'In 41 40 40 34 32' 'Out 41 40 40 32 34'] ['Sys::Performance System' '------------------------------------------------------------' 'System CPU Usage(%) Current 3 hrs 24 hrs 7 days 30 days' '------------------------------------------------------------' 'Utilization 17 18 18 18 17' ' ' '------------------------------------------------------------' 'Memory Used(%) Current 3 hrs 24 hrs 7 days 30 days' '------------------------------------------------------------' 'TMM Memory Used 10 10 10 10 10' 'Other Memory Used 55 55 54 54 53' 'Swap Used 0 0 0 0 0']] ###BIG-IP hostname => ltm02 ### [['Sys::Performance Throughput' '-----------------------------------------------------------------------' 'Throughput(bits)(bits/sec) Current 3 hrs 24 hrs 7 days 30 days' '-----------------------------------------------------------------------' 'Service 202.3K 258.7K 279.2K 297.4K 112.5K' 'In 190.8K 209.7K 210.5K 243.6K 89.5K' 'Out 19.6K 21.0K 21.1K 57.4K 30.1K' ' ' '-----------------------------------------------------------------------' 'SSL Transactions Current 3 hrs 24 hrs 7 days 30 days' '-----------------------------------------------------------------------' 'SSL TPS 0 0 0 0 0' ' ' '-----------------------------------------------------------------------' 'Throughput(packets)(pkts/sec) Current 3 hrs 24 hrs 7 days 30 days' '-----------------------------------------------------------------------' 'Service 77 82 83 63 62' 'In 39 40 40 34 32' 'Out 37 40 40 32 34'] ['Sys::Performance System' '------------------------------------------------------------' 'System CPU Usage(%) Current 3 hrs 24 hrs 7 days 30 days' '------------------------------------------------------------' 'Utilization 21 18 18 18 17' ' ' '------------------------------------------------------------' 'Memory Used(%) Current 3 hrs 24 hrs 7 days 30 days' '------------------------------------------------------------' 'TMM Memory Used 10 10 10 10 10' 'Other Memory Used 55 55 54 54 53' 'Swap Used 0 0 0 0 0']] The data obtained is historical data over a period of time. Sometimes it is also important to gather the peak usage of throughout/memory/cpu over time and not the average. Stay tuned as we will discuss on how to obtain that information in a upcoming article. Conclusion Use the output of the data to learn the traffic patterns and propose the most appropriate BIG-IP hardware/software in your environment. This could be data collected directly in your production environment or a staging environment, which would help you make the decision on what purchasing strategy gives you the most value from your BIG-IP's. For reference: https://www.f5.com/pdf/products/big-ip-local-traffic-manager-ds.pdf The above is one example of how you can get started with using Ansible and tmsh commands. Using this method you can potentially achieve close to 100% automation on the BIG-IP.F5 Automation with Ansible Tips and Tricks
Getting Started with Ansible and F5 In this article we are going to provide you with a simple set of videos that demonstrate step by step how to implement automation with Ansible. In the last video, however we will demonstrate how telemetry and automation may be used in combination to address potential performance bottlenecks and ensure application availability. To start, we will provide you with details on how to get started with Ansible automation using the Ansible Automation Platform®: Backing up your F5 device Once a user has installed and configured Ansible Automation Platform, we will now transition to a basic maintenance function – an automated backup of a BIG-IP hardware device or Virtual Edition (VE). This is always recommended before major changes are made to our BIG-IP devices Configuring a Virtual Server Next, we will use Ansible to configure a Virtual Server, a task that is most frequently performed via manual functions via the BIG-IP. When changes to a BIG-IP are infrequent, manual intervention may not be so cumbersome. However large enterprise customers may need to perform these tasks hundreds of times: Replace an SSL Certificate The next video will demonstrate how to use Ansible to replace an SSL certificate on a BIG-IP. It is important to note that this video will show the certificate being applied on a BIG-IP and then validated by browsing to the application website: Configure and Deploy an iRule The next administrative function will demonstrate how to configure and push an iRule using the Ansible Automation Platform® onto a BIG-IP device. Again this is a standard administrative task that can be simply automated via Ansible: Delete the Existing Virtual Server Ok so now we have to delete the above configuration to roll back to a steady state. This is a common administrative task when an application is retired. We again demonstrate how Ansible automation may be used to perform these simple administrative tasks: Telemetry and Automation: Using Threshold Triggers to Automate Tasks and Fix Performance Bottlenecks Now you have a clear demonstration as to how to utilize Ansible automation to perform routine tasks on a BIG-IP platform. Once you have become proficient with more routine Ansible tasks, we can explore more high-level, sophisticated automation tasks. In the below demonstration we show how BIG-IP administrators using SSL Orchestrator® (SSLO) can combine telemetry with automation to address performance bottlenecks in an application environment: Resources: So that is a short series of tutorials on how to perform routine tasks using automation plus a preview of a more sophisticated use of automation based upon telemetry and automatic thresholds. For more detail on our partnership, please visit our F5/Ansible page or visit the Red Hat Automation Hub for information on the F5 Ansible certified collections. https://www.f5.com/ansible https://www.ansible.com/products/automation-hub https://galaxy.ansible.com/f5networks/f5_modules
How to use Ansible with Cisco routers
Quick Intro For those who don't know, there is an Ansible plugin called network_cli to retrieve network device configuration for backup, inspection and even execute commands. So, let's assume we have Ansible already installed and 2 routers: I used Debian Linux here and I had to install Python 3: I've also installed pip as we can see above because I wanted to install a specific version of Ansible (2.5.+) that would allow myself to use network_cli plugin: Note: we can list available Ansible versions by just typing pip install ansible== I've also created a user named ansible: Edited Linux sudoers file with visudo command: And added Ansible user permission to run root commands without prompting for password so my file looked liked this: Quick Set Up This is my directory structure: These are the files I used for this lab test: Note: we can replace cisco1.rodrigo.example for an IP address too. In Ansible, there is a default config file (ansible.cfg) where we store the global config, i.e. how we want Ansible to behave. We also keep the list of our hosts into an inventory file (inventory.yml here). There is a default folder (group_vars) where we can store variables that would apply to any router we ran Ansible against and in this case it makes sense as my router credentials are the same. Lastly, retrieve_backup.yml is my actual playbook, i.e. where I tell Ansible what to do. Note: I manually logged in to cisco1.rodrigo.example and cisco2.rodrigo.example to populate ssh known_hosts files, otherwise Ansible complains these hosts are untrusted. Populating our Playbook file retrieve_backup.yml Let's say we just want to retrieve the OSPF configuration from our Cisco routers. We can use ios_command to type in any command to Cisco router and use register to store the output to a variable: Note: be careful with the indentation. I used 2 spaces here. We can then copy the content of the variable to a file in a given directory. In this case, we copied whatever is in ospf_output variable to ospf_config directory. From the Playbook file above we can work out that variables are referenced between {{ }} and we might probably be wondering why do we need to append stdout[0] to ospf_output right? If you know Python, you might be interested in knowing a bit more about what's going on under the hood so I'll clarify things a bit more here. The variable ospf_output is actually a dictionary and stdout is one of its keys. In reality, ospf_output.stdout could be represented as ospf_output['stdout'] We add the [0] because the object retrieved by the key stdout is not a string. It's a list! And [0] just represents the first object in the list. Executing our Playbook I'll create ospf_config first: And we execute our playbook by issuing ansible-playbook command: Now let's check if our OSPF config was retrieved: We can pretty much type in any IOS command we'd type in a real router, either to configure it or to retrieve its configuration. We could also append the date to the file name but it's out of the scope of this article. That's it for now.Dig deeper into Ansible and F5 integration
Basics of Ansible and F5 integration were covered in a joint webinar held earlier in March 2017. To learn more about the integration and current F5 module support along with some use cases view the webinar . We had another joint webinar in June 2017, which went into details on the integration. We spoke about how to F5 ansible modules (that will be part of upcoming Ansible 2.4 release) can be used to perfrom administrative tasks on the BIG-IP, make the BIG-IP ready for application deployment in weeks rather than in days. We also touched upon usage of F5 ansible iApps module to deploy applications on the BIG-IP. The webinar's was very well received which ended with a great Q and A session. Some of the questions that came up were how do we create playbooks for different workflows, what are the best practices that F5 recommends, can we get a sample playbook etc. We will use this forum to answer some of the questions and dig deeper into the F5 and Ansible integration. Now what really is a playbook, it is nothing but a collection of tasks that are to be performed sequentially on a system. Let us consider a use case where a customer has just purchased 20 BIG-IP’s and needs to get all of them networked and to a state where the BIG-IPs are ready to deploy applications. We can define a playbook which consists of tasks required to perform Day0 and Day1 configuration on the BIG-IPs. Lets start with Day0, now networking the system consists of assigning it a NTP and DNS server address, assigning a hostname, making some ssh customizations etc., some of these settings are common to all the BIG-IPs. The common set of configurations can be defined using the concept of a ‘role’ in ansible. Let’s define a ‘onboarding’ role which will configure the common settings like NTP, DNS and SSHD settings. PLAYBOOK FOR ONBOARDING - name: Onboarding BIG-IP hosts: bigip gather_facts: false roles: - onboarding //playbook runs tasks defined in the ‘onboarding’ role This play book will run against all the BIG-IP’s defined in the inventory host file Example of inventory host file [bigip] 10.192.73.218 10.192.73.219 10.192.73.220 10.192.73.221 The above playbook will run tasks specified in the 'onboarding' role in file main.yaml (playbooks/roles/onboarding/tasks/main.yaml) - name: Configure NTP server on BIG-IP bigip_device_ntp: server: "{{ inventory_hostname }}" user: "{{ username }}" password: "{{ password }}" ntp_servers: "{{ ntp_servers }}" validate_certs: False delegate_to: localhost - name: Manage SSHD setting on BIG-IP bigip_device_sshd: server: "{{ inventory_hostname }}" user: "{{ username }}" password: "{{ password }}" banner: "enabled" banner_text: " {{ banner_text }}" validate_certs: False delegate_to: localhost - name: Manage BIG-IP DNS settings bigip_device_dns: server: "{{ inventory_hostname }}" user: "{{ username }}" password: "{{ password }}" name_servers: "{{ dns_servers }}" search: "{{ dns_search_domains }}" ip_version: "{{ ip_version }}" validate_certs: False delegate_to: localhost Variables will be referenced from the main.yaml file under default directory for the ‘onboarding’ role (playbooks/roles/onboarding/default/main.yaml) username: admin password: admin banner_text: "--------Welcome to Onboarding BIGIP----------" ntp_servers: - '172.27.1.1' - '172.27.1.2' dns_servers: - '8.8.8.8' - '4.4.4.4' dns_search_domains: - 'local' - 'localhost' ip_version: 4 The BIG-IP is now ready to deploy applications. One application is configuring the BIG-IP to securely load balance applications. This requires configuring the following on the BIG-IP Vlans Self-IPs Nodes/members (2) Pool (1) Assigning the nodes to the Pool Creating a HTTPS Virtual server Creating a redirect Virtual server, which will redirect all HTTP requests to the HTTPS virtual server (iRule is assigned to the virtual server to achieve this) This playbook will be run individually for each BIG-IP since each will use different values for VLANS/Self IP’s/Virtual server address etc. The variables values for this playbook is defined inline and not in a separate file. PLAYBOOK FOR APPLICATION DEPLOYMENT - name: creating HTTPS application hosts: bigip tasks: - name: Configure VLANs on the BIG-IP bigip_vlan: server: "{{ inventory_hostname }}" user: "{{ username }}" password: "{{ password }}" validate_certs: False name: "{{ item.name }}" tag: "{{ item.tag }}" tagged_interface: "{{ item.interface }}" with_items: - name: 'External' tag: '10' interface: '1.1' - name: 'Internal tag: '11’ interface: '1.2' delegate_to: localhost - name: Configure SELF-IPs on the BIG-IP bigip_selfip: server: "{{ inventory_hostname }}" user: "{{ username }}" password: "{{ password }}" validate_certs: False name: "{{ item.name }}" address: "{{ item.address }}" netmask: "{{ item.netmask }}" vlan: "{{ item.vlan }}" allow_service: "{{item.allow_service}}" with_items: - name: 'External-SelfIP' address: '10.10.10.10' netmask: '255.255.255.0' vlan: 'External' allow_service: 'default' - name: 'Internal-SelfIP' address: '192.10.10.10' netmask: '255.255.255.0' vlan: 'Internal' allow_service: 'default' delegate_to: localhost - name: Create a web01.internal node //Creating Node1 bigip_node: server: "{{ inventory_hostname }}" user: "admin" password: "admin" host: "192.168.68.140" name: "web01.internal" validate_certs: False delegate_to: localhost - name: Create a web02.internal node //Creating Node2 bigip_node: server: "{{ inventory_hostname }}" user: "admin" password: "admin" host: "192.168.68.141" name: "web02.internal" validate_certs: False delegate_to: localhost - name: Create a web-pool //Creating a pool bigip_pool: server: "{{ inventory_hostname }}" user: "admin" password: "admin" lb_method: "ratio_member" monitors: http name: "web-pool" validate_certs: False delegate_to: localhost - name: Add http node to web-pool //Assigning members to a pool bigip_pool_member: description: "HTTP Webserver-1" host: "{{ item.host }}" name: "{{ item.name }}" user: "admin" password: "admin" pool: "web-pool" port: "80" server: "{{ inventory_hostname }}" validate_certs: False with_items: - host: "192.168.168.140" name: "web01.internal" - host: "192.168.68.141" name: "web02.internal" delegate_to: localhost - name: Create a virtual server //Create a HTTPS Virtual Server bigip_virtual_server: description: "Secure web application" server: "{{ inventory_hostname }}" user: "admin" password: "admin" name: "https_vs" destination: "10.10.20.120" port: 443 snat: "Automap" all_profiles: - http - clientssl pool: "web-pool" validate_certs: False delegate_to: localhost - name: Create a redirect virtual server //Create a redirect virtual server bigip_virtual_server: description: "Redirect Virtual server" server: "{{ inventory_hostname }}" user: "admin" password: "admin" name: "http_redirect" destination: "10.10.20.120" validate_certs: False port: 80 all_profiles: - http all_rules: //Attach an iRule to the Virtual server - _sys_https_redirect delegate_to: localhost Bookmark this page if you are interested in learning more. We will be updating this blog with new F5 modules that are going to be supported with Ansible 2.4 release
Getting started with Ansible
Ansible is an orchestration and automation engine. It provides a means for you to automate the administration of different devices, from Linux to Windows and different special purpose appliances in-between. Ansible falls into the world of DevOps related tools. You may have heard of others that play in this area as well including. Chef Puppet Saltstack In this article I'm going to briefly skim the surface of what Ansible is and how you can get started using it. I've been toying around with it for some years now, and (most recently at F5) using it to streamline some development work I've been involved in. If you, like me, are a fan of dabbling with interesting tools and swear by the "Automate all the Things!" catch-phrase, then you might take an interest in Ansible. We're going to start small though and build upon what we learn. My goal here is to eventually bring you all to the point where we're doing some crazy awesome things with Ansible and F5 products. I'll also go into some brief detail on features of Ansible that make it relatively painless to interoperate with existing F5 products. Let's get started! So why Ansible? Any time that it comes to adopting some new technology for your everyday use, inevitably you need to ask yourself "what's in it for me?". Why not just use some custom shell scripts and pssh to do everything? Here are my reasons for using Ansible. It is agent-less The only dependencies (on the remote device) are SSH and python; and even python is not really a dependency The language that you "do" stuff in is YAML. No CS degree or programming language expertise is required (Perl, Ruby, Python, etc) Extending it is simple (in my opinion) Actions are idempotent Order of operations is well-defined and work is performed top-down Many of the original tools in the DevOps space were agent-based tools. This is a major problem for environments where it's literally (due to technology or politics) impossible to install an agent. Your SLA may prohibit you from installing software on the box. Or, you might legitimately not be able to install the software due to older libraries or other missing dependencies. Ansible has no agent requirement; a plus in my book. Most of the systems that you will come across can be, today, manipulated by Ansible. It is agent-less by design. Dependency wise you need to be able to connect to the machine you want to orchestrate, so it makes sense that SSH is a dependency. Also, you would like to be able to do higher-order "stuff" to a machine. That's where the python dependency comes into play. I say dependency loosely though, because Ansible provides a way to run raw commands on remote systems regardless of whether Python is installed. For professional Ansible development though, this method of orchestrating devices is largely not recommended except in very edge cases. Ansible's configuration language is YAML. If you have never seen YAML before, this is what it looks like - name: Deploy common hosts files settings hosts: all connection: ssh gather_facts: true tasks: - name: Install required packages apt: name: "{{ item }}" state: "present" with_items: - ntp - ubuntu-cloud-keyring - python-mysqldb YAML is generally composed of simple key/value pairs, lists, and dictionaries. Contrast this with the Puppet configuration language; a special DSL that resembles a real programming language. class sso { case $::lsbdistcodename { default: { $ssh_version = 'latest' } } class { '::sso': ldap_uri => $::ldap_uri, dev_env => true, ssh_version => $ssh_version, sshd_allow_groups => $::sshd_allow_groups, } } Or contrast this with Chef, in which you must know Ruby to be able to use. servers = search( :node, "is_server:true AND chef_environment:#{node.chef_environment}" ).sort! do |a, b| a.name <=> b.name end begin resources('service[mysql]') rescue Chef::Exceptions::ResourceNotFound service 'mysql' end template "#{mysql_dir}/etc/my.conf" do source 'my.conf.erb' mode 0644 variables :servers => servers, :mysql_conf => node['mysql']['mysql_conf'] notifies :restart, 'service[mysql]' end In Ansible, work that is performed is idempotent. That's a buzzword. What does it mean? It means that an operation can be performed multiple times without changing the result beyond its initial application. If I try to add the same line to a file a thousand times, it will be added once and then will not be added again 999 times. Another example is adding user accounts. They would be added once, not many times (which might raise errors on the system). Finally, Ansible's workflow is well defined. Work starts at the top of a playbook and makes its way to the bottom. Done. End of story. There are other tools that have a declarative model. These tools attempt to read your mind. "You declare to me how the node should look at the end of a run, and I will determine the order that steps should be run to meet that declaration." Contrast this with Ansible which only operates top-down. We start at the first task, then move to the second, then the third, etc. This removes much of the "magic" from the equation. Often times an error might occur in a declarative tool due specifically to how that tool arranges its dependency graph. When that happens, it's difficult to determine what exactly the tool was doing at the time of failure. That magic doesn't exist in Ansible; work is always top-down whether it be tasks, roles, dependencies, etc. You start at the top and you work your way down. Installation Let's now take a moment to install Ansible itself. Ansible is distributed in different ways depending on your operating system, but one tried and true method to install it is via pip ; the recommended tool for installing python packages. I'll be working on a vanilla installation of Ubuntu 15.04.2 (vivid) for the remaining commands. Ubuntu includes a pip package that should work for you without issue. You can install it via apt-get . sudo apt-get install python-pip python-dev Afterwards, you can install Ansible. sudo pip install markupsafe ansible==1.9.4 You might ask "why not ansible 2.0". Well, because 2.0 was just released and the community is busy ironing out some new-release bugs. I prefer to give these things some time to simmer before diving in. Lucky for us, when we are ready to dive in, upgrading is a simple task. So now you should have Ansible available to you. SEA-ML-RUPP1:~ trupp$ ansible --version ansible 1.9.4 configured module search path = None SEA-ML-RUPP1:~ trupp$ Your first playbook Depending on the tool, the body of work is called different things. Puppet calls them manifests Chef calls them recipes and cookbooks Ansible calls them plays and playbooks Saltstack calls them formulas and states They're all the same idea. You have a system configuration you need to apply, you put it in a file, the tool interprets the file and applies the configuration to the system. We will write a very simple playbook here to illustrate some concepts. It will create a file on the system. Booooooring. I know, terribly boring. We need to start somewhere though, and your eyes might roll back into your head if we were to start off with a more complicated example like bootstrapping a BIG-IP or dynamically creating cloud formation infrastructure in AWS and configuring HA pairs, pools, and injecting dynamically created members into those pools. So we are going to create a single file. We will call it site.yaml . Inside of that file paste in the following. - name: My first play hosts: localhost connection: local gather_facts: true tasks: - name: Create a file copy: dest: "/tmp/test.txt" content: "This is some content" This file is what Ansible refers to as a Playbook. Inside of this playbook file we have a single Play (My first play). There can be multiple Plays in a Playbook. Let's explore what's going on here, as well as touch upon the details of the play itself. First, that Play. Our play is composed of a preamble that contains the following name hosts connection gather_facts The name is an arbitrary name that we give to our Play so that we will know what is being executed if we need to debug something or otherwise generate a reasonable status message. ALWAYS provide a name for your Plays, Tasks, everything that supports the name syntax. Next, the hosts line specifies which hosts we want to target in our Play. For this Play we have a single host; localhost . We can get much more complicated than this though, to include patterns of hosts groups of hosts groups of groups of hosts dynamically created hosts hosts that are not even real You get the point. Next, the connection line tells Ansible how to connect to the hosts. Usually this is the default value ssh . In this case though, because I am operating on the localhost, I can skip SSH altogether and simply say local . After that, I used the gather_facts line to tell Ansible that it should interrogate the remote system (in this case the system localhost) to gather tidbits of information about it. These tidbits can include the installed operating system, the version of the OS, what sort of hardware is installed, etc. After the preamble is written, you can see that I began a new block of "stuff". In this case, the tasks associated with this Play. Tasks are Ansible's way of performing work on the system. The task that I am running here is using the copy module. As I did with my Play earlier, I provide a name for this task. Always name things! After that, the body of the module is written. There are two arguments that I have provided to this module (which are documented more in the References section below) dest content I won't go into great deal here because the module documentation is very clear, but suffice it to say that dest is where I want the file written and content is what I want written in the file. Running the playbook We can run this playbook using the ansible-playbook command. For example. SEA-ML-RUPP1:~ trupp$ ansible-playbook -i notahost, site.yaml The output of the command should resemble the following PLAY [My first play] ****************************************************** GATHERING FACTS *************************************************************** ok: [localhost] TASK: [Create a file] ********************************************************* changed: [localhost] PLAY RECAP ******************************************************************** localhost : ok=2 changed=1 unreachable=0 failed=0 We can also see that the file we created has the content that we expected. SEA-ML-RUPP1:~ trupp$ cat /tmp/test.txt This is some content A brief aside on the syntax to run the command. Ansible requires that you specify an inventory file to provide hosts that it can orchestrate. In this specific example, we are not specifying a file. Instead we are doing the following Specifying an arbitrary string (notahost) Followed by a comma In Ansible, this is a short-hand trick to skip the requirement that an inventory file be specified. The comma is the key part of the argument. Without it, Ansible will look for a file called notahost and (hopefully) not find it; raising an error otherwise. The output of the command is shown next. The output is actually fairly straight-forward to read. It lists the PLAY s and TASK s that are running (as well as their names...see, I told you you wanted to have names). The status of the Tasks is also shown. This can be values such as changed ok failed skipped unreachable Finally, all Ansible Playbook runs end with a PLAY RECAP where Ansible will tell you what the status of the various plays on your hosts were. It is at this point where a Playbook will be considered successful or not. In this case, the Playbook was completely successful because there were not unreachable hosts nor failed hosts. Summary This was a brief introduction to the orchestration and automation system Ansible. There are far more complex subjects related to Ansible that I will touch upon in future posts. If you found this information useful, rate it as such. If you would like to see more advanced topics covered, videos demo'd, code samples written, or anything else on the subject, let me know in the comments below. Many organizations, both large and small, use DevOps tools like the one presented in this post. Ansible has several features, per design, that make it attractive to these organizations (such as being agent-less, and having minimum requirements). If you'd like to see crazy sophisticated examples of Ansible in use...well...we'll get there. You need to rate and comment on my posts though to let me know that you want to see more. References copy - Copies files to remote locations. — Ansible Documentation raw - Executes a low-down and dirty SSH command — Ansible Documentation Variables — Ansible DocumentationBIG-IP ASM Automation with Ansible
My Background Back in September I started my Ansible journey, coming from no knowledge about Ansible and its automation capabilities I was asked to develop some code/playbooks to automate some of the BIG-IP's ASM functions for AnsibleFest 2019. I was pleasantly surprised on how easy it was to install Ansible, build playbooks and deliver the correct end-state for the BIG-IP. The playbooks and automation took me back down memory lane to when I was creating a universal network bootable Norton Ghost CD in DOS for all of the different models of PCs my work owned. The team I work for (Business Development) has been working hard at making sure our code is easily accessible to customers through GitHub. Our goal is to provide the necessary tools such as F5 Automation Sandbox and use-cases so that even if you are new to Ansible, or a die-hard coder with Ansible that there is a place for you to test, consume and bring life to the code. What is BIG-IP ASM? F5 BIG-IP® Application Security Manager™ (ASM) is a flexible web application firewall that secures web applications in traditional, virtual, and private cloud environments. BIG-IP ASM helps secure applications against unknown vulnerabilities, and enables compliance for key regulatory mandates. BIG-IP ASM is a key part of the F5 application delivery firewall solution, which consolidates traffic management, network firewall, application access, DDoS protection, SSL inspection, and DNS security. What is Ansible? Ansible is a radically simple IT automation engine that automates cloud provisioning, configuration management, application deployment, intra-service orchestration, and many other IT needs. Designed for multi-tier deployments since day one, Ansible models your IT infrastructure by describing how all of your systems inter-relate, rather than just managing one system at a time. It uses no agents and no additional custom security infrastructure, so it's easy to deploy - and most importantly, it uses a very simple language (YAML, in the form of Ansible Playbooks) that allow you to describe your automation jobs in a way that approaches plain English. What does the Code Do? IP Blocking - In ASM, there is a feature called IP address intelligence that can allow or block IP addresses from being able to access protected applications. This code creates a Virtual IP (VIP) and a blank ASM policy attached to that VIP. After the creation the code exports the ASM Policy into an XML format and is then modified by the code snip-it below to add blocked IP addresses and re-import that policy over the existing one. Prior to this snip-it we have code that checks to see if the IP address already exists for things like re-runs of the code and blocks duplicate IP addresses from being added to the XML. This is a snip-it of the Code where it modifies the ASM Policy XML File (this was exported in previous steps in the code) #Import Additional Disallowed IPs - name: Add Disallowed IPs xml: path: "{{ ASM_Policy_File }}" pretty_print: yes input_type: xml insertafter: yes xpath: /policy/geolocation add_children: "<whitelist><ip_address>{{ item.item }}</ip_address><subnet_mask>255.255.255.255</subnet_mask><policy_builder_trusted>false</policy_builder_trusted><ignore_anomalies>false</ignore_anomalies><never_log>false</never_log><block_ip>Always</block_ip><never_learn>false</never_learn><description>blocked</description><ignore_ip_reputation>false</ignore_ip_reputation></whitelist>" with_items: "{{ Blocked_IP_Valid.results }}" when: Blocked_IPs is defined and item.rc == 1 Here is a demonstration of an IP being blocked and unblocked by the BIG-IP ASM Policy. Disallowed URL Filtering - Another feature of ASM is the ability to disallowed URLs, this can be useful when working internally vs. externally and there are other reasons to why a specific URL would be blocked or protected by BIG-IP ASM. This code can be used independently, cooperatively, or not at all with this playbook. Since this playbook is merged with the IP Blocking code it follows the same flow (exporting/importing XML and error checking) as previously mentioned in the IP Blocking to ensure no duplicates are made in the XML. This is a snip-it of the Code where it modifies the ASM Policy XML File (this was exported in previous steps in the code) #Import Additional Disallowed URLs - name: Add Disallowed URLs xml: path: "{{ ASM_Policy_File }}" input_type: xml pretty_print: yes xpath: /policy/urls/disallowed_urls add_children: - "<url protocol=\"HTTP\" type=\"explicit\" name=\"{{ item.item }}\"/>" - "<url protocol=\"HTTPS\" type=\"explicit\" name=\"{{ item.item }}\"/>" with_items: "{{ Blocked_URLs_Valid.results }}" when: Blocked_URLs is defined and item.rc == 1 Here is a demonstration of specific URLs being blocked by the BIG-IP ASM Policy. (Note: the File Name in the repo has been changed but does the same use-case ) Where can you access the Playbook for this integration? https://github.com/f5devcentral/f5-bd-ansible-usecases/tree/master/03-F5-WAF-Policy-Management How to get all of the use-cases currently available. https://github.com/f5devcentral/f5-bd-ansible-usecases Want to try it out but need a Lab to work in? Try out our F5 Automation Sandbox built for AWS! https://clouddocs.f5.com/training/automation-sandbox/Parsing complex BIG-IP json structures made easy with Ansible filters like json_query
JMESPath and json_query JMESPath (JSON Matching Expression paths) is a query language for searching JSON documents. It allows you to declaratively extract elements from a JSON document. Have a look at this tutorial to learn more. The json_query filter lets you query a complex JSON structure and iterate over it using a loop structure.This filter is built upon jmespath, and you can use the same syntax as jmespath. Click here to learn more about the json_query filter and how it is used in Ansible. In this article we are going to use the bigip_device_info module to get various facts from the BIG-IP and then use the json_query filter to parse the output to extract relevant information. Ansible bigip_device_info module Playbook to query the BIG-IP and gather system based information. - name: "Get BIG-IP Facts" hosts: bigip gather_facts: false connection: local tasks: - name: Query BIG-IP facts bigip_device_info: provider: validate_certs: False server: "xxx.xxx.xxx.xxx" user: "*****" password: "*****" gather_subset: - system-info register: bigip_facts - set_fact: facts: '{{bigip_facts.system_info}}' - name: debug debug: msg="{{facts}}" To view the output on a different subset, below are a few examples to change the gather_subset and set_fact values in the above playbook from gather_subset: system-info, facts: bigip_facts.system_info to any of the below: gather_subset: vlans , facts: bigip_facts.vlans gather_subset: self-ips, facts: bigip_facts.self_ips gather_subset: nodes. facts: bigip_facts.nodes gather_subset: software-volumes, facts: bigip_facts.software_volumes gather_subset: virtual-servers, facts: bigip_facts.virtual_servers gather_subset: system-info, facts: bigip_facts.system_info gather_subset: ltm-pools, facts: bigip_facts.ltm_pools Click here to view all the information that can be obtained from the BIG-IP using this module. Parse the JSON output Once we have the output lets take a look at how to parse the output. As mentioned above the jmespath syntax can be used by the json_query filter. Step 1: We will get the jmespath syntax for the information we want to extract Step 2: We will see how the jmespath syntax and then be used with json_query in an Ansible playbook The website used in this article to try out the below syntax: https://jmespath.org/ Some BIG-IP sample outputs are attached to this article as well (Check the attachments section after the References). The attachment file is a combined output of a few configuration subsets.Copy paste the relevant information from the attachment to test the below examples if you do not have a BIG-IP. System information The output for this section is obtained with above playbook using parameters: gather_subset: system-info, facts: bigip_facts.system_info Once the above playbook is run against your BIG-IP or if you are using the sample configuration attached, copy the output and paste it in the relevant text box. Try different queries by placing them in the text box next to the magnifying glass as shown in image below # Get MAC address, serial number, version information msg.[base_mac_address,chassis_serial,platform,product_version] # Get MAC address, serial number, version information and hardware information msg.[base_mac_address,chassis_serial,platform,product_version,hardware_information[*].[name,type]] Software volumes The output for this section is obtained with above playbook using parameters: gather_subset: software-volumes facts: bigip_facts.software_volumes # Get the name and version of the software volumes installed and its status msg[*].[name,active,version] # Get the name and version only for the software volume that is active msg[?active=='yes'].[name,version] VLANs and Self-Ips The output for this section is obtained with above playbook using parameters gather_subset: vlans and self-ips facts: bigip_facts Look at the following example to define more than one subset in the playbook # Get all the self-ips addresses and vlans assigned to the self-ip # Also get all the vlans and the interfaces assigned to the vlan [msg.self_ips[*].[address,vlan], msg.vlans[*].[full_path,interfaces[*]]] Nodes The output for this section is obtained with above playbook using parameters gather_subset: nodes, facts: bigip_facts.nodes # Get the address and availability status of all the nodes msg[*].[address,availability_status] # Get availability status and reason for a particular node msg[?address=='192.0.1.101'].[full_path,availability_status,status_reason] Pools The output for this section is obtained with above playbook using parameters gather_subset: ltm-pools facts: bigip_facts.ltm_pools # Get the name of all pools msg[*].name # Get the name of all pools and their associated members msg[*].[name,members[*]] # Get the name of all pools and only address of their associated members msg[*].[name,members[*].address] # Get the name of all pools along with address and status of their associated members msg[*].[name,members[*].address,availability_status] # Get status of pool members of a particular pool msg[?name=='/Common/pool'].[members[*].address,availability_status] # Get status of pool # Get address, partition, state of pool members msg[*].[name,members[*][address,partition,state],availability_status] # Get status of a particular pool and particular member (multiple entries on a member) msg[?full_name=='/Common/pool'].[members[?address=='192.0.1.101'].[address,partition],availability_status] Virtual Servers The output for this section is obtained with above playbook using parameters gather_subset: virtual-servers facts: bigip_facts.virtual_servers # Get destination IP address of all virtual servers msg[*].destination # Get destination IP and default pool of all virtual servers msg[*].[destination,default_pool] # Get me all destination IP of all virtual servers that a particular pool as their default pool msg[?default_pool=='/Common/pool'].destination # Get me all profiles assigned to all virtual servers msg[*].[destination,profiles[*].name] Loop and display using Ansible We have seen how to use the jmespath syntax and extract information, now lets see how to use it within an Ansible playbook - name: Parse the output hosts: localhost connection: local gather_facts: false tasks: - name: Setup provider set_fact: provider: server: "xxx.xxx.xxx.xxx" user: "*****" password: "*****" server_port: "443" validate_certs: "no" - name: Query BIG-IP facts bigip_device_info: provider: "{{provider}}" gather_subset: - system_info register: bigip_facts - debug: msg="{{bigip_facts.system_info}}" # Use json query filter. The query_string will be the jmespath syntax # From the jmespath query remove the 'msg' expression and use it as it is - name: "Show relevant information" set_fact: result: "{{bigip_facts.system_info | json_query(query_string)}}" vars: query_string: "[base_mac_address,chassis_serial,platform,product_version,hardware_information[*].[name,type]]" - debug: "msg={{result}}" Another example of what would change if you use a different query (only highlighting the changes that need to made below from the entire playbook) - name: Query BIG-IP facts bigip_device_info: provider: "{{provider}}" gather_subset: - ltm-pools register: bigip_facts - debug: msg="{{bigip_facts.ltm_pools}}" - name: "Show relevant information" set_fact: result: "{{bigip_facts.ltm_pools | json_query(query_string)}}" vars: query_string: "[*].[name,members[*][address,partition,state],availability_status]" The key is to get the jmespath syntax for the information you are looking for and then its a simple step to incorporate it within your Ansible playbook References Try the queries - https://jmespath.org/ Learn more jmespath syntax and example - https://jmespath.org/tutorial.html Ansible lab that can be used as a sandbox - https://clouddocs.f5.com/training/automation-sandbox/Existing Ansible BIG-IP modules
Right around the time that I started at F5, I was at the pinnacle of my exposure to Ansible. So imagine my surprise when I saw BIG-IP modules in the Ansible core product! I immediately wanted to know which one of my colleagues I could go talk Ansible-fu with! I cracked open the source code, found the names of contributors, made haste to the corporate phone book to look up where they sat, and...wait a second... ...no entries in the phone book? ...that curious look in colleagues eyes that says "I haven't the foggiest idea what you are talking about". Then it hit me...the BIG-IP modules didn't originate at F5. Upon further investigation, it became clear to me that, indeed, we had no skin in this game. These modules originated from two enterprising individuals who I found on DevCentral, and I would be remiss to exclude their valuable contributions to the Ansible+F5 cause, so in this post I want to highlight those contributions and bring others up-to-speed on current Ansible+BIG-IP functionality. Credit where it is due As far as I can tell from perusing Ansible's git logs, the existing BIG-IP modules in Ansible are the work of Matt Hite and Serge van Ginderachter. Both are DevCentral contributors, and have had a presence in the Ansible community from (at least) 2013 when their modules landed in the tool. Among the modules that they had a hand in are bigip_facts bigip_node bigip_pool bigip_pool_member bigip_monitor_http bigip_monitor_tcp Since that time, I've seen even more folks step up to the plate with Ansible modules that will be making their appearance in upcoming releases. Well, I wanted to have a hand in this work too. I figured I had a perfect opportunity to help because, I had some background in Ansible I had access to unlimited BIG-IP technical resources It was only a question of how to get involved. And then, Matt posted this in a pull request (PR)... Unfortunately I don't have GTM to smoke test this with. Can you solicit some testers on the mailing list? And Serge followed with So same here, not tested as no GTM gear, but an offline code review. Bingo...I was in. If there's one area that I figured I could have the greatest impact, it's the area of technical resources. There is BIG-IP stuff all over the place at F5, and I sit next to, or at least near, many of the people who have knowledge of the products; far more advanced knowledge than I do. Many of those same people hang out and contribute on DevCentral. Paired with my knowledge of Ansible, I figured I could help test BIG-IP related PRs to validate their functionality beyond what a code review offered. So that's what I did (and what we're about to do). Before you begin If you were around for the earlier introductory article to Ansible that I posted a couple days ago, pull up a terminal to that machine. We need to install one more dependency that is common to all of the BIG-IP modules in Ansible; bigsuds . pip install bigsuds With that in place, you're ready to work with the Ansible modules. Additionally, I am going to be using a very minimal inventory file that just includes a single BIG-IP. You will want to adjust yours for your environment, but here is mine. [test] big-ip01.internal Note that the name above is available in my local DNS. If you only have an IP address to work with, you can just add that to your inventory file. For example [test] 192.168.10.2 Also note that I included a line called [test] . In Ansible this is referred to as a group. We will be revisiting this in the future as we begin orchestrating multiple BIG-IPs. Let's dive in to some of Matt and Serge's BIG-IP modules! bigip_facts In Ansible, there is a pattern you often see over and over. This is the pattern of modules that provide information, and modules that change settings. For this tutorial I'll refer to them as, reader modules writer modules Without exception, reader modules are suffixed with an _facts string. While writer modules, are not. So, with that in mind...pop quiz! If I asked you whether the bigip_pool module was a reader module or a writer module, you would say...writer! If I asked you whether the bigip_pool_facts module was a reader module or a writer module, you would say...reader! The bigip_facts module will return to you a number of facts about the BIG-IP in question. Let's take a look. First, my playbook. - name: Test bigip_facts hosts: test connection: local tasks: - name: Get all of the facts from my BIG-IP bigip_facts: server: "{{ inventory_hostname }}" user: "admin" password: "admin" include: "system_info" And now, let's just run it to see what sort of output it generates. ansible-playbook -i hosts site.yaml -vvvv You should be presented with information resembling the following. root@debian:~# ansible-playbook -i hosts site.yaml -vvvv PLAY [Test bigip_facts] ******************************************************* TASK: [Get all of the facts from my BIG-IP] *********************************** ... ok: [big-ip01.internal] => {"ansible_facts": {"system_info": {"base_mac_address": "8A:83:8B:B1:AE:F7", "blade_temperature": [], "chassis_slot_information": [], "globally_unique_identifier": "8A:83:8B:B1:AE:F7", "group_id": "DefaultGroup", "hardware_information": [{"model": "Common KVM processor", "name": "cpus", "slot": 0, "type": "HARDWARE_BASE_BOARD", "versions": [{"name": "cache size", "value": "4096 KB"}, {"name": "cores", "value": "2"}, {"name": "cpu MHz", "value": "2199.994"}]}], ... "os_version": "#1 SMP Mon Aug 11 19:54:07 PDT 2014", "platform": "Z100", "product_category": "Virtual Edition", "switch_board_part_revision": null, "switch_board_serial": null, "system_name": "Linux"}, "time": {"day": 28, "hour": 22, "minute": 1, "month": 1, "second": 7, "year": 2016}, "time_zone": {"gmt_offset": -8, "is_daylight_saving_time": false, "time_zone": "PST"}, "uptime": 854}}, "changed": false} root@debian:~# This module can output a lot of information that you can use in later tasks. Note that I just asked for the system_info facts, but there are a number of them that you can ask for, including address_class certificate client_ssl_profile device device_group interface key node pool rule self_ip software system_info traffic_group trunk virtual_address virtual_server vlan You can include multiple types of facts by separating them with a comma. For example - name: Test bigip_facts hosts: test connection: local tasks: - name: Get all of the facts from my BIG-IP bigip_facts: server: "{{ inventory_hostname }}" user: "admin" password: "admin" include: "system_info,software,self_ip" Returns facts representing the system_info , software , and self IPs. You can use the generated facts in later tasks by referencing their JSON keys. For example - name: Test bigip_facts hosts: test connection: local tasks: - name: Get all of the facts from my BIG-IP bigip_facts: server: "{{ inventory_hostname }}" user: "admin" password: "admin" include: "system_info,software,self_ip" - name: Mention the Self IP debug: msg: "I have self IP {{ self_ip['/Common/net1'].address }}" - name: Mention the software debug: msg: "I have software version {{ software[0].version }}" And the (truncated) output ... TASK: [Mention the Self IP] *************************************************** ok: [big-ip01.internal] => { "msg": "I have self IP 10.2.2.2" } TASK: [Mention the software] ************************************************** ok: [big-ip01.internal] => { "msg": "I have software version 11.6.0" } ... bigip_node This module allows you to manipulate nodes in a BIG-IP. Nodes are logical objects on your BIG-IP that identify the IP address of a physical node on your network. In terms of what we can do with them with the existing BIG-IP modules, you can use this module to create nodes that you can later assign to pool members. First, let's show you the playbook that I'm going to run. - name: Node shenanigans hosts: test connection: local tasks: - name: Add a new node bigip_node: server: "{{ inventory_hostname }}" user: "admin" password: "admin" host: "10.2.1.1" name: "member1" This is a simple example of creating a node in my BIG-IP. As is probably apparent if I am going to be working with pools, I would want to create many nodes. I'll hold off on that until a future example, but hopefully this clarifies the point of how to create the object that we can later assign to pool members. Let's run it! ansible-playbook -i hosts site.yaml And the output we should expect to see looks like this root@debian:~# ansible-playbook -i hosts site.yaml PLAY [Node shenanigans] ******************************************************* TASK: [Add a new node] ******************************************************** changed: [big-ip01.internal] PLAY RECAP ******************************************************************** big-ip01.internal : ok=1 changed=1 unreachable=0 failed=0 root@debian:~# Now, just to clarify what gets dropped where, and where these nodes can be used, let's first look at the Local Traffic > Nodes screen. Now, let's navigate over to the pool screen and try to create a new pool. On the new pool creation screen, we have the option of specifying members of that pool. If we click on Node List, well, look at that. Our new node. bigip_pool This module allows you to create pools on your BIG-IPs. To these pools we can later add pool members. With this and the bigip_pool_member module, you can control the basic load balancing functionality of the BIG-IP. First, just to prove that I have nothing up my BIG-IP sleeve, here's my current pool list. And now, for my playbook - name: Create a pool hosts: test connection: local tasks: - name: Create the pool1 pool bigip_pool: server: "{{ inventory_hostname }}" user: "admin" password: "admin" name: "pool1" With a wave of my magic wand... root@debian:~# ansible-playbook -i hosts site.yaml PLAY [Create a pool] ******************************************************* TASK: [Create the pool1 pool] *********************************** ... changed: [big-ip01.internal] => {"changed": true} PLAY RECAP ******************************************************************** big-ip01.internal : ok=1 changed=1 unreachable=0 failed=0 root@debian:~# ...and my pool list has been changed. The bigip_pool module has a number of other options that let you adjust settings of the pool. They are all documented on the module's page. bigip_pool_member With the bigip_pool_member module, you can manipulate the members of any of the pools on your BIG-IP. This module, in particular, is a crucial part of a rolling upgrade strategy that you may undertake when upgrading software on members of the pool. Consider the following scenario. Remove (or disable) the member from the pool Upgrade the member's software (you pick the software, but let's say Apache for example) Verify the software upgrade Add (or enable) the member back to the old pool, or, add the member to a new pool This module can be used for steps 1 and 4. Let's have a look at my playbook. - name: Pool member manipulation hosts: test connection: local tasks: - name: Drop members out of pool1 bigip_pool_member: server: "{{ inventory_hostname }}" user: "admin" password: "admin" state: "absent" host: "{{ item }}" pool: "pool1" port: "22" with_items: - member1 - member2 - name: Intermediate processing debug: msg: "Upgrade some software" - name: More intermediate processing debug: msg: "Install some configuration" - name: Add member1 node back bigip_node: server: "{{ inventory_hostname }}" user: "admin" password: "admin" name: "{{ item }}" host: "10.2.1.1" state: "present" with_items: - member1 - name: Add member2 node back bigip_node: server: "{{ inventory_hostname }}" user: "admin" password: "admin" name: "{{ item }}" host: "10.2.1.2" state: "present" with_items: - member2 - name: Add member1 back to pool1 bigip_pool_member: server: "{{ inventory_hostname }}" user: "admin" password: "admin" host: "{{ item }}" pool: "pool1" port: "22" state: "present" with_items: - member1 - name: Add member2 to pool2 bigip_pool_member: server: "{{ inventory_hostname }}" user: "admin" password: "admin" host: "{{ item }}" pool: "pool2" port: "22" state: "present" with_items: - member2 And before we run it, let's just take a peek at my current pools. As well as the members of pool1 And the members of pool2 Now, let's have a go at running the playbook. ansible-playbook -i hosts site2.yaml After it runs, you should have output that resembles something like this. root@debian:~# ansible-playbook -i hosts site.yaml PLAY [Pool member manipulation] *********************************************** TASK: [Drop members out of pool1] ********************************************* changed: [big-ip01.internal] => (item=member1) ok: [big-ip01.internal] => (item=member2) TASK: [Intermediate processing] *********************************************** ok: [big-ip01.internal] => { "msg": "Upgrade some software" } TASK: [More intermediate processing] ****************************************** ok: [big-ip01.internal] => { "msg": "Install some configuration" } TASK: [Add member1 node back] ************************************************* changed: [big-ip01.internal] => (item=member1) TASK: [Add member2 node back] ************************************************* ok: [big-ip01.internal] => (item=member2) TASK: [Add member1 back to pool1] ********************************************* changed: [big-ip01.internal] => (item=member1) TASK: [Add member2 to pool2] ************************************************** ok: [big-ip01.internal] => (item=member2) PLAY RECAP ******************************************************************** big-ip01.internal : ok=7 changed=3 unreachable=0 failed=0 root@debian:~# And your pool members should be different, check out the screenshots below. The pool list Members in pool1 Members in pool2 To Summarize BIG-IP would not have a presence in Ansible if it were not for Matt and Serge's initiative. Based on the work they started, I'm happy to assist with testing and contributing new modules moving forward. In my entirely too biased opinion, Ansible fits in well with the tools and products I work on. Perhaps it will also fit in well with your workflow. If we've piqued your interest, then the Ansible mailing list is a great place to ask more Ansible related questions and further solidify your understanding. If you liked this article and want to see more like it, rate it as such. Have comments? Questions? Something else? Leave a comment below! References http://docs.ansible.com/ansible/bigip_facts_module.html http://docs.ansible.com/ansible/bigip_node_module.html http://docs.ansible.com/ansible/bigip_pool_module.html http://docs.ansible.com/ansible/bigip_pool_member_module.html https://support.f5.com/kb/en-us/products/big-ip_ltm/manuals/product/ltm_configuration_guide_10_0_0/ltm_nodes.html#1188710 https://groups.google.com/forum/#!forum/ansible-projectAutomate BIG-IP in customer environments using Ansible
There are a lot of technical resources on how Ansible can be used to automte the F5 BIG-IP. A consolidated list of links to help you brush up on Ansible as well as help you understand the Ansible BIG-IP solution Getting started with Ansible Ansible and F5 solution overview F5 and Ansible Integration - Webinar Dig deeper into F5 and Ansible Integration - Webinar I am going to dive directly into how customers are using ansible to automate their infrastructure today. If you are considering using Ansible or have already gone through a POC maybe a few of these use cases will resonate with you and you might see use of them in your environment. Use Case: Rolling application deployments Problem Consider a web application whose traffic is being load balanced by the BIG-IP. When the web servers running this application need to be updated with a new software package, an organization would consider going in for a rolling deployment where rather than updating all servers or tiers simultaneously. This means, the organization installs the updated software package on one server or subset of servers at a time. The organization would need to bring down one server at a time to have minimal traffic disruption. Solution This is a repeatitive and time consuming task for an organization considering there are newer updates coming out every so often. Ansible’s F5 modules can be used to automate operations like disabling a node (pool member), performing an update on that node and then bring that node back into service. This workflow is applicable for every node member of the pool sequentially resulting in zero down time for the application Modules bigip_ltm_facts bigip_node Sample - name: Update software on Pool Member - "{{node_name}}" hosts: bigip gather_facts: false vars: pool_name: "{{pool_name}}" node_name: "{{node_name}}" tasks: - name: Query BIG-IP facts for Pool Status - "{{pool_name}}" bigip_facts: validate_certs: False server: "{{ bigip_ip_address }}" user: "{{ bigip_username}}" bigip_password: "{{ bigip_password}}" include: "pool" delegate_to: localhost register: bigip_facts - name: Assert that Pool - "{{pool_name}}" is available before disabling node assert: that: - "'AVAILABILITY_STATUS_GREEN' in bigip_facts['ansible_facts']['pool']['/Common/{{iapp_name}}/{{pool_name}}']['object_status']['availability_status']|string" msg: "Pool is NOT available DONOT want to disable any more nodes..Check your BIG-IP" - name: Disable Node - "{{node_name}}" bigip_node: server: "{{ bigip_ip_address }}" user: "{{ bigip_username}}" bigip_password: "{{ bigip_password}}" partition: "Common" name: "{{node_name}}" state: "present" session_state: "disabled" monitor_state: "disabled" delegate_to: localhost - name: Get Node facts - "{{node_name}}" bigip_facts: validate_certs: False server: "{{ bigip_ip_address }}" user: "{{ bigip_username}}" bigip_password: "{{ bigip_password}}" include: "node" delegate_to: localhost register: bigip_facts - name: Assert that node - "{{node_name}}" did get disabled assert: that: - "'AVAILABILITY_STATUS_RED' in bigip_facts['ansible_facts']['node']['/Common/{{node_name}}']['object_status']['availability_status']|string" msg: "Pool member did NOT get DISABLED" - name: Perform software update on node - "{{node_name}}" hosts: "{{node_name}}" gather_facts: false vars: pool_name: "{{pool_name}}" node_name: "{{node_name}}" tasks: - name: "Update 'curl' package" apt: name=curl state=present - name: Enable Node hosts: bigip gather_facts: false vars: pool_name: "{{pool_name}}" node_name: "{{node_name}}" tasks: - name: Enable Node - "{{node_name}}" bigip_node: server: "{{ bigip_ip_address }}" user: "{{ bigip_username}}" bigip_password: "{{ bigip_password}}" partition: "Common" name: "{{node_name}}" state: "present" session_state: "enabled" monitor_state: "enabled" delegate_to: localhost Use Case: Deploying consistent policies across different environment using F5 iApps Problem Consider an organization has many different environments (development/QA/Production). Each environment is a replica of each other and has around 100 BIG-IP’s in each environment. When a new application is added to one environment it needs to be added to other environments in a fast, safe and secure manner. There is also a need to make sure the virtual server bought online adheres to all the traffic rules and security policies. Solution To ensure that the virtual server is deployed correctly and efficiently, the entire application can be deployed on the BIG-IP using the iApps module. iApps also has the power to reference ASM policies hence helping to consistently deploy an application with the appropriate security policies Modules bigip_iapp_template bigip_iapp_service Sample - hosts: localhost tasks: - name: Get iApp from Github get_url: url: "{{ Git URL from where to download the iApp }}" dest: /var/tmp validate_certs: False - name: Upload iApp template to BIG-IP bigip_iapp_template: server: "{{ bigip_ip_address }}" user: "{{ bigip_username }}" password: "{{ bigip_password }}" content: "{{ lookup('file', '/var/tmp/appsvcs_integration_v2.0.003.tmpl') }}" #Name of iApp state: "present" validate_certs: False delegate_to: localhost - name: Deploy iApp bigip_iapp_service: name: "HTTP_VS_With_L7Firewall" template: "appsvcs_integration_v2.0.003" parameters: "{{ lookup('file', 'final_iapp_with_asm.json') }}" #JSON blob for body content to the iApp server: "{{ bigip_ip_address }}" user: "{{ bigip_username}}" password: "{{ bigip_password }}" state: "present" delegate_to: localhost Use Case: Disaster Recovery Problem All organizations should have a disaster recovery plan in place incase of a catastrophic failure resulting in loss of data including BIG-IP configuration data. Re-configuring the entire infrastructure from scratch can be an administrative nightmare. The procedure in place for disaster recovery can also be used for migrating data from one BIG-IP to another as well as for performing hardware refresh and RMA’s. Solution Use BIG-IP user configuration set (UCS) configuration file to restore the configuration on all the BIG-IP’s and have your environment back to its original configuration in minutes. Modules bigip_ucs Sample --- - name: Create UCS file hosts: bigip gather_facts: false tasks: - name: Create ucs file and store it bigip_ucs: server: "{{ bigip_ip_address }}" user: "{{ bigip_username }}" password: "{{ bigip_password }}" ucs: "/root/test.ucs" state: "installed" delegate_to: localhost Use Case: Troubleshooting Problem Any problem on the BIG-IP for example a backend server not receiving traffic or a virtual server dropping traffic unexpectedly or a monitor not responding correctly needs extensive troubleshooting. These problems can be hard to pin point and need an expert to debug and look through the logs. Solution Qkview is a great utility on the BIG-IP which that an administrator can use to automatically collect configuration and diagnostic information from BIG-IP and other F5 systems. Use this ansible module and run it against all BIG-IP’s in your network and collect the diagnostic information to pass it onto to F5 support. Modules bigip_qkview Sample - name: Create qkview hosts: bigip gather_facts: false tasks: - name: Generate and store qkview file bigip_qkview: server: "{{ bigip_ip_address }}" user: "{{ bigip_username }}" password: "{{ bigip_password }}" asm_request_log: "no" dest: "/tmp/localhost.localdomain.qkview" validate_certs: "no" delegate_to: localhost Conclusion These are just some of the use cases that can be tackled using our BIG-IP ansible modules. We have several more modules in Ansible 2.4 release that the ones mentioned above which can help in other scenarios. To view a complete list of BIG-IP modules available in ansible 2.4 release click here Module overview at a glance BIG-IP modules in Ansible release 2.3 New BIG-IP modules in Ansible release 2.4 To get started and save time download a BIG-IP onboarding ansible role from ansible galaxy and run the playbook against your BIG-IPF5 and Cisco ACI Essentials: Automate automate automate !!!
This article will focus on automation support by BIG-IP and Cisco ACI and how automation tools specifically Ansible can be used to automate different use cases. Before getting into the weeds let's discuss and understand BIG-IP's and Cisco ACI's automation strategies. BIG-IP automation strategy BIG-IP automation strategy is simple-abstract as much complexity as possible from the user, give an easy button to the user to deploy their BIG-IP configuration. This could honestly mean different methods to different people, some prefer sending a single API call to perform one action( A one-to-one mapping between your API<->Configuration). Others prefer a more declarative approach where one API call performs multiple actions, basically a one-to -many mapping between your API(1)<->Configuration(N). A great link to refresh and learn about the different options: https://www.f5.com/products/automation-and-orchestration Cisco ACI automation strategy Cisco Application Policy Infrastructure Controller (APIC) is the network controller for the ACI fabric. APIC is the unified point of automation and management for the Cisco ACI fabric, policy enforcement, and health monitoring. The Cisco ACI programmability model provides complete programmatic access using APIC. Click here to learn more https://developer.cisco.com/site/aci/ Automation tools There are a lot of automation tools that are being talked for network automation BUT the one that comes up in every customer conversation is Ansible. Its simplicity, maturity and community adoption has made it very popular. In this article we are going to focus on using Ansible to automate a service discovery use case. Use Case: Dynamic EP attach/detach Let’s take an example of a simple http web service being made highly available and secure using the BIG-IP Virtual IP address. This web service has a bunch of backend web servers hosting the application, the IP of this web servers is configured on the BIG-IP as pool members. These same web server IP’s are learned as endpoints in the ACI fabric and are part of an End Point Group (EPG) on the APIC. Hence there is a logical mapping between a EPG on APIC and a pool on the BIG-IP. Now if the application is adding or deleting web servers that is hosting the application maybe to save cost or maybe to deal with increase/decrease of traffic, what happens is that the web server IP will be automatically learned/unlearned on APIC. BUT an admin will still have to add/remove that web server IP from the pool on BIG-IP. This can be a burden on the network admin specially if this happens very often. Here is where automation can help and let’s look at how in the next section More details on the use case can be found at https://devcentral.f5.com/s/articles/F5-Cisco-ACI-Essentials-Dynamic-pool-sizing-using-the-F5-ACI-ServiceCenter Automation of Use Case: Dynamic EP attach/detach Automation can be achieved by using Ansible and Ansible tower where API calls are made directly to the BIG-IP. Another option it to use the F5 ACI ServiceCenter (a native F5 ACI integration) to automate this particular use case. Ansible and Ansible tower To learn more about Ansible and Ansible tower: https://www.ansible.com/products/tower Using this method of automation separate API calls are made directly to the ACI and the BIG-IP. Sample playbook to perform the addition and deletion of pool members to a BIG-IP pool based on members in a particular EPG. The mapping of pool to EPG is provided as input to the playbook. - name: Dynamic end point attach/dettach hosts: aci connection: local gather_facts: false vars: epg_members: [] pool_members: [] pool_members_ip: [] bigip_ip: 10.192.73.xx bigip_password: admin bigip_username: admin # Here we are mapping pool 'dynamic_pool' to EPG 'internalEPG' which belongs to APIC tenant 'TenantDemo' app_profile_name: AppProfile epg_name: internalEPG partition: Common pool_name: dynamic_pool pool_port: 80 tenant_name: TenantDemo tasks: - name: Setup provider set_fact: provider: server: "{{bigip_ip}}" user: "{{bigip_username}}" password: "{{bigip_password}}" server_port: "443" validate_certs: "false" - name: Get end points learned from End Point group aci_rest: action: "get" uri: "/api/node/mo/uni/tn-{{tenant_name}}/ap-{{app_profile_name}}/epg-{{epg_name}}.json?query-target=subtree&target-subtree-class=fvIp" host: "{{inventory_hostname}}" username: "{{ lookup('env', 'ANSIBLE_NET_USERNAME') }}" password: "{{ lookup('env', 'ANSIBLE_NET_PASSWORD') }}" validate_certs: "false" register: eps - name: Get the IP's of the servers part of the EPG set_fact: epg_members="{{epg_members + [item]}}" loop: "{{eps | json_query(query_string)}}" vars: query_string: "imdata[*].fvIp.attributes.addr" no_log: True - name: Get only the IPv4 IP's set_fact: epg_members="{{epg_members | ipv4}}" - name: Adding Pool members to the BIG-IP bigip_pool_member: provider: "{{provider}}" state: "present" name: "{{item}}" host: "{{item}}" port: "{{pool_port}}" pool: "{{pool_name}}" partition: "{{partition}}" loop: "{{epg_members}}" - name: Query BIG-IP facts bigip_device_facts: provider: "{{provider}}" gather_subset: - ltm-pools register: bigip_facts - name: "Show members belonging to pool {{pool_name}}" set_fact: pool_members="{{pool_members + [item]}}" loop: "{{bigip_facts.ltm_pools | json_query(query_string)}}" vars: query_string: "[?name=='{{pool_name}}'].members[*].name[]" - set_fact: pool_members_ip: "{{pool_members_ip + [item.split(':')[0]]}}" loop: "{{pool_members}}" - debug: "msg={{pool_members_ip}}" #If there are any membeers on the BIG-IP that are not present in the EPG,then delete them - name: Find the members to be deleted if any set_fact: members_to_be_deleted: "{{ pool_members_ip | difference(epg_members) }}" - debug: "msg={{members_to_be_deleted}}" - name: Delete Pool members from the BIG-IP bigip_pool_member: provider: "{{provider}}" state: "absent" name: "{{item}}" port: "{{pool_port}}" pool: "{{pool_name}}" preserve_node: yes partition: "{{partition}}" loop: "{{members_to_be_deleted}}" Ansible tower's scheduling feature can be used to schedule this playbook to be run every minute, every hour or once per day based on how often an application is expected to change and how important is it for the configuration on both the Cisco ACI and the BIG-IP to be in sync. F5 ACI ServiceCenter To learn more about the integration : https://www.f5.com/cisco The F5 ACI ServiceCenter is installed on the APIC controller. Here automation can be used to create the initial EPG to Pool mapping. Once the mapping is created the F5 ACI ServiceCenter handles the dynamic sizing of pools based on events generated by APIC. Events are generated when a server is learned/unlearned on an EPG which is what the F5 ACI ServiceCenter listens to and accordingly adds or removes pool members from the BIG-IP. Sample playbook to deploy the mapping configuration on the BIG-IP through the F5 ACI ServiceCenter --- - name: Deploy EPG to Pool mapping hosts: localhost gather_facts: false connection: local vars: apic_ip: "10.192.73.xx" big_ip: "10.192.73.xx" partition: "Dynamic" tasks: - name: Login to APIC uri: url: https://{{apic_ip}}/api/aaaLogin.json method: POST validate_certs: no body_format: json body: aaaUser: attributes: name: "admin" pwd: "******" headers: content_type: "application/json" return_content: yes register: cookie - debug: msg="{{cookie['cookies']['APIC-cookie']}}" - set_fact: token: "{{cookie['cookies']['APIC-cookie']}}" - name: Login to BIG-IP uri: url: https://{{apic_ip}}/appcenter/F5Networks/F5ACIServiceCenter/loginbigip.json method: POST validate_certs: no body: url: "{{big_ip}}" user: "admin" password: "admin" body_format: json headers: DevCookie: "{{token}}" #The body of this request defines the mapping of Pool to EPG #Here we are mapping pool 'web_pool' to EPG 'internalEPG' which belongs to APIC tenant 'TenantDemo' - name: Deploy AS3 dynamic EP mapping uri: url: https://{{apic_ip}}/appcenter/F5Networks/F5ACIServiceCenter/updateas3data.json method: POST validate_certs: no body: url: "{{big_ip}}" partition: "{{partition}}" application: "DemoApp1" json: class: Application template: http serviceMain: class: Service_HTTP virtualAddresses: - 10.168.56.100 pool: web_pool web_pool: class: Pool monitors: - http members: - servicePort: 80 serverAddresses: [] - addressDiscovery: event servicePort: 80 constants: class: Constants serviceCenterEPG: web_pool: tenant: TenantDemo application: AppProfile epg: internalEPG body_format: json status_code: - 202 - 200 headers: DevCookie: "{{token}}" return_content: yes register: complete_info - name: Get task ID of above request set_fact: task_id: "{{ complete_info.json.message.taskId}}" when: complete_info.json.code == 202 - name: Get deployment status uri: url: https://{{apic_ip}}/appcenter/F5Networks/F5ACIServiceCenter/getasynctaskresponse.json method: POST validate_certs: no body: taskId: "{{task_id}}" body_format: json headers: DevCookie: "{{token}}" return_content: yes register: result until: result.json.message.message != "in progress" retries: 5 delay: 2 when: task_id is defined - name: Display final result debug: var: result After deploying this configuration, adding/deleting pool members and making sure the configuration is in sync is the responsibility of the F5 ACI ServiceCenter. Takeaways Both methods are highly effective and usable. The choice of which one to use comes down to the operational model in your environment. Some pros and cons to help made the decision on which platform to use for automation. Ansible Tower Pros No dependency on any other tools Fits in better with bigger company automation strategy to use Ansible for ALL network automation Cons Have to manage playbook execution and scheduling using Ansible Tower If more logic is needed besides what’s described above playbooks will have to be written and maintained Execution of playbook is based on scheduling and is not event driven F5 ACI ServiceCenter Pros Only pool-epg mapping has to be deployed using automation, rest all is handled by the application User interface to view pool member to EPG mapping once deployed and view discrepancies if any Limited automation knowledge is needed, heavy lifting is being done by the application Dynamically adding/deleting pool members is event driven, as members are learned/unlearned by the F5 ACI ServiceCenter an action is taken Cons Another tool is required Customization of pool to EPG mapping is not present. Only one-to-one EPG to pool mapping is present. References Learn about the F5 ACI ServiceCenter and other Cisco integrations: https://f5.com/cisco Download the F5 ACI ServiceCenter: https://dcappcenter.cisco.com/f5-aci-servicecenter.html Lab to execute Ansible playbooks: https://dcloud.cisco.com (Lab name: F5 and Ansible )
