Security Automation with F5 BIG-IP and Event Driven Ansible
Updated (September 19th 2023) INTRODUCTION TO EVENT DRIVEN SECURITY: Event Driven Security is one of the projects I have been working on for the last year or so. The idea of creating automated security that can react similarly to how I would react in situations is fascinating to me, and then comes the BIG Question.... "Can I code it?" Originally our solution we had utilized ELK (Elastic Logstash Kibana) where Elasticsearch was my logging and monitoring tool, Kibana was the frontend GUI for helping me visualize and set up my watchers for my webhook triggers, Logstash would be an intermediary to receive my webhooks to help me execute Ansible related code. While using Logstash, if the Ansible code was simple it had no issues, however when things got more complex (i.e., taking payloads from Elastic and feeding them through Logstash to my playbooks), I would sometimes get intermittent results. Some of this could be my lack of knowledge of the software but for me it needed to be simple! As I want to become more complex with my Event Driven Security, I needed a product that would follow those needs. And luckily in October 2022 that product was announced "Event Driven Ansible" it made it so I didn’t need Logstash anymore i could call Ansible related code directly, it even took in webhooks (JSON based) to trigger the code, so I was already half way there! CODE FOR EVENT DRIVEN SECURITY: So now I have setup the preface let’s get down to the good stuff! I have setup a GitHub repository for the code i have been testing withhttps://github.com/f5devcentral/f5-bd-ansible-eda-demowhich is free for all to use and please feel free to take/fork/expand!!! There are some cool things worth noting in the code specifically the transformation of the watch code into something usable in playbooks. This code will take all the times the watcher finds a match in its filter and then then copies the Source IP from that code and puts it into a CSV list, then it sends the list as a variable within the webhook along with the message to execute the code. Here is the code I am mentioning above about transforming and sending the payloads in an elastic watcher. See the Full code in the GitHub repo. (Github Repo --> elastic -->watch_blocked_ips.json) "actions": { "logstash_exec": { "transform": { "script": { "source": """ def hits = ctx.payload.hits.hits; def transform = ''; for (hit in hits) { transform += hit._source.src_ip; transform += ', ' } return transform; """, "lang": "painless" } }, "webhook": { "scheme": "http", "host": "10.1.1.12", "port": 5000, "method": "post", "path": "/endpoint", "params": {}, "headers": {}, "body": """{ "message": "Ansible Please Block Some IPs", "payload": "{{ctx.payload._value}}" }""" } } } } In the Ansible Rulebook the big thing to note is that from the Pre-GA code (which was all CLI ansible-rulebook based) to the GA version (EDA GUI) rulebooks now are setup to call Ansible Automation Platform (AAP) templates. In the code below you can see that its looking for an existing template "Block IPs" in the organization "Default" to be able to run correctly. (Github Repo --> rulebooks -->webhook-block-ips.yaml) --- - name: Listen for events on a webhook hosts: all ## Define our source for events sources: - ansible.eda.webhook: host: 0.0.0.0 port: 5000 ## Define the conditions we are looking for rules: - name: Block IPs condition: event.payload.message == "Ansible Please Block Some IPs" action: run_job_template: name: "Block IPs" organization: "Default" This shows my template setup in Ansible Automation Platform 2.4.x, there is one CRITICAL piece of information i wanted to share about using EDA GA and AAP 2.4 code is that within the template you MUSTtick the checkbox on the "Prompt on launch" in the "variables section". This will allow the payload from EDA (given to it from Elastic) to pass on to the playbook. In the Playbook you can see how we extract the payload from the event using the ansible_eda variable, this allows us to pull in the event we were sent from Elastic to Event Driven Ansible and then sent to the Ansible Automation Platform template to narrow down the specific fields we needed (Message and Payload) from there we create an array from that payload so we can pass it along to our F5 code to start adding Blocked IPs to the WAF Policy.(Github Repo --> playbooks -->block-ips.yaml) --- - name: ASM Policy Update with Blocked IPs hosts: lb connection: local gather_facts: false vars: Blocked_IPs_Events: "{{ ansible_eda.event.payload }}" F5_VIP_Name: VS_WEB F5_VIP_Port: "80" F5_Admin_Port: "443" ASM_Policy_Name: "WAF-POLICY" ASM_Policy_Directory: "/tmp/f5/" ASM_Policy_File: "WAF-POLICY.xml" tasks: - name: Setup provider ansible.builtin.set_fact: provider: server: "{{ ansible_host }}" user: "{{ ansible_user }}" password: "{{ ansible_password }}" server_port: "{{ F5_Admin_Port }}" validate_certs: "no" - name: Blocked IP Events From EDA debug: msg: "{{ Blocked_IPs_Events.payload }}" - name: Create Array from BlockedIPs ansible.builtin.set_fact: Blocked_IPs: "{{ Blocked_IPs_Events.payload.split(', ') }}" when: Blocked_IPs_Events is defined - name: Remove Last Object from Array which is empty array object ansible.builtin.set_fact: Blocked_IPs: "{{ Blocked_IPs[:-1] }}" when: Blocked_IPs_Events is defined ... All of this combined, creates a well-oiled setup that looks like the following diagram below, with the code and the flows setup we can now create proactive event based security! Here is the flow of the code that is in the GitHub repo when executed. The F5 BIG-IP is pushing all the monitoring logs to Elastic. Elastic is taking all that data and storing it while utilizing a watcher with its filters and criteria, The Watcher finds something that matches its criteria and sends the webhook with payload to Event Driven Ansible. Event Driven Ansible's Rulebook triggers and calls a template within Ansible Automation Platform and sends along the payload given to it from Elastic. Ansible Automation Platforms Template executes a playbook to secure the F5 BIG-IP using the payload given to it from EDA (originally from Elastic). In the End we go Full Circle, starting from the F5 BIG-IP and ending at the F5 BIG-IP! Full Demonstration Video: Check out our full demonstration video we recently posted (Sept 13th 2023) is available on-demand viahttps://www.f5.com/company/events/webinars/f5-and-red-hat-3-part-demo-series This page does require a registration and you can check out our 3 part series. The one related to this lab is the "Event-Driven Automation and Security with F5 and Red Hat Ansible" Proactive Securiy with F5 & Event Driven Ansible Video Demo LINKS TO CODE: https://github.com/f5devcentral/f5-bd-ansible-eda-demoHappy 20th Birthday, BIG-IP TMOS!
I wasn’t in the waiting room with the F5 family, ears and eyes perked for the release announcement of BIG-IP version 9.0. I was a customer back in 2004, working on a government contract at Scott AFB, Illinois. I shared ownership of the F5 infrastructure, pairs of BIG-IPs running version 4.5 on Dell PowerEdge 2250 servers with one other guy. But maybe a month or two before the official first release of TMOS, my F5 account manager dropped off some shiny new hardware. And it was legit purpose-built and snazzy, not some garage-style hacked Frankenstein of COTS parts like the earlier stuff. And you wonder why we chose Dell servers! Anyway, I was a hard-core network engineer at this time, with very little exposure to anything above layer four, and even there, my understanding was limited to ports and ACLs and maybe a little high-level clarity around transport protocols. But application protocols? Nah. No idea. So with this new hardware and an entirely new full-proxy architecture (what’s a proxy, again?) I was overwhelmed. And honestly, I was frustrated with it for the first few days because I didn’t know what I didn’t know and so I struggled to figure out what to do with it, even to replicate my half-proxy configuration in the “new way”. But I’m a curious person. Given enough time and caffeine, I can usually get to the bottom of a problem, at least well enough to arrive at a workable solution. And so I did. My typical approach to anything is to make it work, make it work better, make it work reliably better, then finally make it work reliably and more performantly better. And the beauty here with this new TMOS system is that I was armed with a treasure trove of new toys. The short list I dug into during my beta trial, which lasted for a couple of weeks: The concept of a profile. When you support a few applications, this is no big deal. When you support hundreds, being able to macro configuration snippets within your application and across applications was revolutionary. Not just for the final solution, but also for setting up and executing your test plans. iRules. Yes, technically they existed in 4.x, but they were very limited in scope. With TMOS, F5 introduced the Tcl-based and F5 extended live-traffic scripting environment that unleashed tremendous power and flexibility for network and application teams. I dabbled with this, and thought I understood exactly how useful this was. More on this a little later. A host operating system. I was a router, switch, and firewall guy. Nothing I worked on had this capability. I mean, a linux system built in to my networking device? YES!!! Two things I never knew I always needed during my trial: 1) tcpdump ON BOX. Seriously--mind blown; and 2) perl scripting against config and snmp. Yeah, I know, I laugh about perl now. But 20 years ago, it was the cats pajamas. A fortunate job change Shortly after my trial was over, I interviewed for an accepted a job offer from a major rental car company that was looking to hire an engineer to redesign their application load balancing infrastructure and select the next gear purchase for the effort. We evaluated Cisco, Nortel/Alteon, Radware, and F5 on my recommendation. With our team’s resident architect we drafted the rubric with which we’d evaluate all the products, and whereas there were some layer two performance issues in some packet sizes that were arguably less than real-world, the BIG-IP blew away the competitors across the board. Particularly, though, in configurability and instrumentation. Tcpdump on box was such a game-changer for us. Did we have issues with TMOS version 9? For sure. My first year with TMOS was also TMOS's first year. Bugs are going to happen with any release, but a brand new thing is guaranteed. But F5 support was awesome, and we worked through all the issues in due time. Anyway, I want to share three wins in my first year with TMOS. Win #1 Our first production rollout was in the internet space, on BIG-IP version 9.0.5. That’s right, a .0 release. TMOS was a brand new baby, and we had great confidence throughout our testing. During our maintenance, once we flipped over the BIG-IPs, our rental transaction monitors all turned red and the scripted rental process had increased by 50%! Not good. “What is this F5 stuff? Send it back!!” But it was new, and we knew we had a gem here. We took packet captures on box, of course, then rolled back and took more packet captures, this time through taps because our old stuff didn’t have tcpdump on box. This is where Jason started to really learn about the implications of both a full proxy architecture and the TCP protocol. It turns our our application servers had a highly-tuned TCP stack on them specific to the characteristics of the rental application. We didn’t know this, of course. But since we implemented a proxy that terminates clients at the BIG-IP and starts a new session to the servers, all those customizations for WAN traffic were lost. Once we built a TCP profile specifically for the rental application servers and tested it under WAN emulation, we not only reached parity with the prior performance but beat it by 10%. Huzzah! Go BIG-IP custom protocol stack configuration! Win #2 For the next internal project, I had to rearchitect the terminal server farm. We had over 700 servers in two datacenters supporting over 60,000 thin clients around the world for rental terminals. Any failures meant paper tickets and unhappy staff and customers. One thing that was problematic with the existing solution is that sometimes clients would detach and upon reconnect would connect directly to the server, which skewed the load balancers view of the world and frequently overloaded some servers to the point all sessions on that server would hang until metrics (but usually angry staff) would notify. Remember my iRules comment earlier on differentiators? Well, iRules architect David Hansen happened to be a community hero and was very helpful to me in the DevCentral forums and really opened my eyes to the art of possible with iRules. He was able to take the RDP session token that was being returned by the client, read it, translate it from its Microsoft encoding format, and then forward the session on to the correct server in the backend so that all sessions continued to be accounted for in our load balancing tier. This was formative for me as a technologist and as a member of the DevCentral community. Win #3 2004-2005 was the era before security patching was as visible a responsibility as it is today, but even then we had a process and concerns when there were obstacles. We had an internal application that had a plugin for the web tier that managed all the sessions to the app tier, and this plugin was no longer supported. We were almost a year behind on system and application patches because we had no replacement for this. Enter, again, iRules.I was able to rebuild the logic of the plugin in an iRule that IIRCwasn’tmore than 30 lines. So the benefits ended up not only being a solution to that problem, but the ability to remove that web tier altogether, saving on equipment, power, and complexity costs. And that was just the beginning... TMOS was mature upon arrival, but it got better every year. iControl added REST-based API access; clustered multi-processing introduced tremendous performance gains; TMOS got virtualized, and all the home-lab technologists shouted with joy; a plugin architecture allowed for product modules like ASM and APM; solutions that began as iRules like AFM and SSLO became products. It’s crazy how much innovation has taken place on this platform! The introduction of TMOS didn’t just introduce me to applications and programmability. It did that and I’m grateful, but it did so much more. It unlocked in me that fanboy level that fans of sports teams, video game platforms, Taylor Swift, etc, experience. It helped me build an online community at DevCentral, long before I was an employee. Happy 20th Birthday, TMOS! We celebrate and salute you!A Makeover for DevCentral.
UPDATE: This maintenance has been moved to Jan 24th starting at approximately 8AM Pacific time. --- DevCentral Community, TL;DR - I am announcing some updates coming to the DevCentral community website in a few days - January 17th.24th. This is a mid-sized reboot, some upgrades that we expect will make our online environment a bit nicer to use now and enable more epic-ness to come. ***This update WILL include a maintenance downtime of up to 6 hours starting around 8AM PT (-8 UTC).*** A huge shoutout. I speak for everyone on our team when I say the DevCentral community rocks! We on the DevCentral team are inspired and motivated by your creativity and willingness to share. The depth of your capabilities and the mutual respect you have for individual learning never ceases to amaze. Asyou, collectively, work through increasingly complex problem-spaces we are always working to support your progress. Our recent observations and your suggestions have been buzzing in our ears for several months and so the changes you'll see soon represent next steps for what we think will make your DevCentral community truly exceptional. An updated UI is just the start. Interface updates will be the most obvious, but these are evolutionary rather than revolutionary - a paint job and some new tech. The new tech includes a simpler set of stylesheets, more configurable page templates, some advanced features in the editors, and an overall reductive approach to some historical bloat. One of my design tenets is best summarized as comfortably information dense. DevCentral is a functional website, mostly used during your workweek, and we are looking to strike a balance between engaging with the content quickly and easily (e.g., less scrolling and more filters) without getting eyestrain or needing to know where you are on the site. I'll get into a bit more detail next week and after we launch, I expect to produce some guided walkthroughs you can use to discover some gems. If you are coming to AppWorld in San Jose in February - be sure to stop by our booth and look me up - I can give you a personalized tour.Mention this article and I'll be forced to find some special swag for you too!😎 Wait. Did you say "reductive"? Yes. In a world where everyone backs up everything a subtractive mindset seems nutty. I'm no expert (case-in-point, the length of this announcement) but I embrace the subtractive sentiment and as such I have been ruthless about driving this upgrade for a host of reasons and that *may* have come at the expense of some existing features. Some things won't be present at launch (like Badges and Event calendars) but on-balance I'm sure we will be better off. That said, if something you love disappears - let me know in a comment, in a DM, or on the Suggestions page. I can't promise everything, butI do promise my level-best and that everything is done with intent: sharing valuable and useful technical information with you. Temporary Outage Most important, for now, is to plan for a short outage the morning of Jan 17th approximately 8AM Pacific Time (-8H UTC) for up to about 6 hours. Next: A reminder on Monday, Jan 15th 🚀2022 DevCentral MVP Announcement
Congratulations to the 2022 DevCentral MVPs! Without users who take time from their busy days to share their experience and knowledge for others, DevCentral would be more of a corporate news site and not an actual user community. To that end, the DevCentral MVP Award is given annually to the outstanding group of individuals – the experts in the technical F5 user community who go out of their way to engage with the user community. The award is our way of recognizing their significant contributions, because while all of our users collectively make DevCentral one of the top community sites around and a valuable resource for everyone, MVPs regularly go above and beyond in assisting fellow F5 users.We understand that 2021 was difficult for everyone, and we are extra-grateful to this year's MVPs for going out of their ways to help others. MVPs get badges in their DevCentral profiles so everyone can see that they are recognized experts. This year’s MVPs will receive a glass award, certificate, exclusive thank-you gifts, and invitations to exclusive webinars and behind-the-scenes looks at things like roadmaps, new product sneak-previews, and innovative concepts in development. The 2022 DevCentral MVPs are: Aditya K Vlogs AlexBCT Amine_Kadimi Austin_Geraci Boneyard Daniel_Wolf Dario_Garrido David.burgoyne Donamato 01 Enes_Afsin_Al FrancisD iaine jaikumar_f5 Jim_Schwartzme1 JoshBecigneul JTLampe Kai Wilke Kees van den Bos Kevin_Davies Lionel Deval (Lidev) LouisK Mayur_Sutare Neeeewbie Niels_van_Sluis Nikoolayy1 P K Patrik_Jonsson Philip Jönsson Rob_Carr Rodolfo_Nützmann Rodrigo_Albuquerque Samstep SanjayP ScottE Sebastian Maniak Stefan_Klotz StephanManthey Tyler.HattonAgility sessions announced
Good news, everyone! This year's virtual Agilitywill have over 100 sessions for you to choose from, aligned to 3 pillars. There will be Breakouts (pre-recorded 25 minutes, unlimited audience) Discussion Forums (live content up to 45 minutes, interactive for up to 75 attendees) Quick Hits (pre-recorded 10 minutes, unlimited audience) So, what kind of content are we talking about? If you'd like to learn more about how to Simplify Delivery of Legacy Apps, you might be interested in Making Sense of Zero Trust: what’s required today and what we’ll need for the future (Discussion Forum) Are you ready for a service mesh? (breakout) BIG-IP APM + Microsoft Azure Active Directory for stronger cybersecurity defense (Quick Hits) If you'd like to learn more about how to Secure Digital Experiences, you might be interested in The State of Application Strategy 2022: A Sneak Peak (Discussion Forum) Security Stack Change at the Speed of Business (Breakout) Deploy App Protect based WAF Solution to AWS in minutes (Quick Hits) If you'd like to learn more about how to Enable Modern App Delivery at Scale, you might be interested in Proactively Understanding Your Application's Vulnerabilities (Discussion Forum Is That Project Ready for you? Open Source Maturity Models (Breakout) How to balance privacy and security handling DNS over HTTPS (Quick Hits) The DevCentral team will be hosting livestreams, and the DevCentral lounge where we can hang out, connect, and you can interact directly with session presenters and other technical SMEs. Please go to https://agility2022.f5agility.com/sessions.html to see the comprehensive list, and check back with us for more information as we get closer to the conference.Office Hours - Let's do some home lab stuff!
The week between Christmas and New Year's here in the US is often a little slow. Ok, a lot slow. I always like to work this week because it allows me to clean my "work" house: trying to salvage my inbox, organizing files, shredding docs. It also affords me some time to plan and strategize for the new year, and to work on some projects that are hard to fit in during normal weeks. One of those projects will be to give my home lab some much-needed attention. If you are in the same boat and just need a dedicated block of time, why don't you join me and we can do a community dojo office hours and chat while we make progress on our labs, and share what we're struggling with, what we're excited about, and just enjoy a little community time? Come late and/or leave early; just come! Details Date: Dec 28th, 2023 Time: 9am - 12pm PST Link:https://f5networks.zoom.us/j/9056331793?pwd=S3pKRUc4NWwvSUpaQXBaZms0VENSQT09May the Fourth and Security Trivia on DevCentral with a Chance to Win Prizes
DevCentral is hosting two trivia games to celebrate National Password Day AND May The Fourth (Star Wars Day). Come show off your Star Wars security, password security, and F5 security knowledge in this live event hosted by the DevCentral team and some special guests. There will be three rounds of questions during this hour, with prizes for the winners of each round as well as one lucky random player per game. Get details on how to join by clicking on Option 1:8-9pm Pacific on Thursday, May 3 (7am in the UAE, 8:30am in Bangalore, 11am in Singapore, noon in Tokyo, 1pm in Melbourne and Sydney on May 4 th ) or Option 2:8-9am Pacific on Thursday, May 4(4pm BST, 5pm CET, 6pm EEST, and 7pm GST on May 4 th )DevCentral Community Lab - A&O Toolchain 212: Application Deployments with BIG-IP and AS3
DevCentral is hosting a lab for the Community! Lab Details We will be going through "A&O Toolchain 212: Application Deployments with BIG-IP and AS3" This topic was chosen by the members of the DevCentral Connects Group. How To Sign Up To sign up, simply mark yourself as attending this event and the email address from your profile will be used for the training platform. If you would like to use an alternative email address, just DM me with the email you'd like to use. Important Dates / Time December 13, 2022 9:30am - 10:30am Pacific - DevCentral Connects hosts will be on a live stream and walk through the start of the lab and get everyone logged in. For the following 3 days, the lab environment will remain open so you have plenty of time to start and complete the lab! How to get help throughout the lab For questions, you will be able to interact with the lab team online via a thread Community.f5.com (will be posted here). The lab team may also jump on live streams, as needed, to address questions as well.Hanging with DevCentral at RSA Conference 2023
AubreyKingF5andbuulamspent last week at RSA Conference 2023 in San Franciscoconnecting with security professionals and colleagues. And, of course, they filmed a lot of those talks to share with the community. Buu connected with Tanya Janca at B-Sides San Francisco, Jason Haddix (BuddoBot), and Joel Violette (IBM) who shares his thoughts on ChatGPT (AI was a hot topic this year across the board). Aubrey shared hisRSA Keynote takeaways, interviewed Alyssa Miller before her panel on CISO challenges, and connected with a ton of other organizations across the expo room floor. Also making an appearance:Scheff,Kyle_Roberts, and Cameron_Delanowho all shared insights from their conversations at the F5 booth. Check out the full playlist of RSA 2023 content here!
