Overview of MITRE ATT&CK Tactic - TA0011 Command and Control
Introduction In modern days, cyber violations, command and control are one of the main set of techniques with which attackers can gain control over the system within a victim’s network. Once control is gained over the system, the attackers can steal sensitive data, move laterally and blend into normal activity. Command and Control (MITRE ATT&CK Tactic TA0011) represents another critical stage of the adversary lifecycle, where the adversaries focus on communicating with the systems under their control. There are multiple ways to achieve this, either by mimicking the expected traffic flow to avoid detection or mimicking a normal behavior of the compromised system. To avoid the vulnerability, it is important for defenders to understand how communication is established to any system in the network and the various levels of stealth depending on the network structure. This article walks through the most common Command and Control techniques, and how F5 solutions provide strong defense against them. T1071 - Application Layer Protocol To communicate with the systems, the adversaries blend in with the existing traffic of the OSI layer protocols to avoid detection/network filtering. The results of these commands will be embedded within the protocol traffic between the client and the server. T1071.001 - Web Services Adversaries mimic normal, expected HTTP/HTTPS traffic that carries web data to communicate with the systems under their control within a victim network. T1071.002 - File Transfer Protocol Protocols used to implement this technique includes SMB, FTP, FTPS and TFTP. The malicious data is concealed within the fields and headers of the packets produced from these protocols. T1071.003 - Mail Protocols Protocols carrying electronic mail such as SMTP/S, POP3/S, and IMAP is utilized by concealing the data within the email messages themselves. T1071.004 - DNS An administrative function in computer networking is served by the DNS Protocol, and DNS traffic may also be allowed even before the authentication of the network. Data is concealed in the fields and headers of these packets. T1071.005 - Publish/Subscribe Protocols For message distribution managed by a centralized broker, where Publish/Subscribe design utilizes MQTT, XMPP, AMQP and STOMP protocols. T1092 - Communication Through Removable Media On disconnected networks, command and control between the compromised hosts can be performed using removable media to execute commands from system to system. For a successful execution, both systems need to be compromised and need to replicate the removable media through lateral movement. T1659 - Content Injection Adversaries may also gain control over the victim’s system by injecting malicious content into the systems, by initially accessing the compromised data-transfer channels where the traffic can be manipulated or content can be injected. T1132 – Data Encoding Another technique to gain control over the system is by encoding the information using a standard data encoding system. Encoding includes the use of ASCII, Unicode, Base64, MIME or other binary-to-text encoding systems. T1132.001 - Standard Encoding Data Encoding schemes utilized for Standard Encoding includes ASCII, Unicode, hexadecimal, Base64 and MIME. Data compression, such as gzip, are also an example of standard encoding. T1132.002 - Non-Standard Encoding Data Encoded in the message body of an HTTP request, such as modified Base64, is utilized as encoding schemes. T1001 – Data Obfuscation Obfuscation of command-and-control communication is hidden as part of this technique, making it even more difficult to discover or decipher. The focus is to make the communication less conspicuous and hidden, by incorporating several methods, which create below sub-techniques: T1001.001 - Junk Data Adversaries may abuse the protocols by adding random, meaningless junk data to the protocols, which can prevent trivial methods for decoding or deciphering the traffic. T1001.002 - Steganography Steganographic sub-techniques are used to transfer hidden digital data messages between systems, such as images or document files. T1001.003 - Protocol or Service Impersonation Adversaries can impersonate legitimate protocols or web services, to command-and-control traffic by blending in with legitimate network traffic. T1568 – Dynamic Resolution To establish connections dynamically to command-and-control the infrastructure and prevent any detections, adversaries use malware sharing a common algorithm with the infrastructure to dynamically adjust the parameters, such as a domain name, IP address, or port number. T1568.001 - Fast Flux DNS Fast Flux DNS is used to hide a command-and-control channel behind an array of rapidly changing IP addresses linked to a single domain resolution. T1568.002 - Domain Generation Algorithm Rather than relying on a list of static IP addresses or domains, adversaries may utilize Domain Generation Algorithms to dynamically identify a destination domain for command-and-control traffic. T1568.003 - DNS Calculations Instead of utilizing the predetermined port number or the actual IP address, to dynamically determine which port and IP address to use, adversaries calculate on addresses returned in DNS results. T1573 – Encrypted Channel Adversaries rely on an encrypted algorithm channel to conceal command-and-control traffic rather than depending on any inherent protections by the communication protocols. T1573.001 - Symmetric Cryptography Symmetric Encryption Algorithms, such as AES, DES, 3DES, Blowfish and RC4, use keys for plaintext encryption and ciphertext decryption. T1573.002 - Asymmetric Cryptography Asymmetric cryptography, or public key cryptography, uses a keypair per party: one public and one private. The sender encrypts the data with the receiver’s public key, and the receiver decrypts the data with their private key. T1008 – Fallback Channels If the primary channel is compromised or inaccessible, then in order to maintain reliable command and control, adversaries use fallback communication channels. T1665 – Hide Infrastructure To hide and evade detection of the command-and-control infrastructure, adversaries identify and filter traffic from defensive tools, masking malicious domains to abuse the true destination, and otherwise hiding malicious contents to delay discovery and prolong the effectiveness of adversary infrastructure. T1105 – Ingress Tool Transfer Tools or other files transfer from an external adversary-controlled source into the compromised environment through controlled channels or protocols such as FTP. Also, adversaries may spread tools across the compromised environment as part of Lateral Movement. T1104 –Multi-Stage Channels To make detection more difficult, adversaries create multiple stages for command-and-control for several functions and different conditions. T1095 – Non-Application Layer Protocol To communicate between the host and command-and-control server, adversaries use non-application layer protocols, such as ICMP (Internet Control Message Protocol), UDP (User Datagram Protocol), SOCKS (Secure Sockets), or SOL (Serial over LAN). T1571 – Non-Standard Port Adversaries communicate using port pairings that are not associated with the protocol, for, say, HTTPS over port 8088 or port 587 as opposed to the traditional port 443. T1572 – Protocol Tunneling Another approach to avoid detection/network filtering is to explicitly encapsulate a protocol within another protocol to enable routing of network packets which otherwise not reach their intended destination, such as SMB, RDP. T1090 – Proxy To direct network communications to a command-and-control server to avoid direct connections to the infrastructure and override the existing actual communication paths to avoid suspicion and manage command-and-control communications inside a compromised environment, proxy act as an intermediary between the systems, such as, HTRAN, ZXProxy and ZXPortMap. T1090.001 - Internal Proxy Internal proxies are primarily used to conceal the actual destination while reducing the need for multiple connections to external systems, such as peer-to-peer (p2p) networking protocols. T1090.002 - External Proxy External proxy is used to mask the true destination of the traffic with port redirectors. Purchased infrastructure such as Virtual Private Servers which are the compromised systems outside the victim's network, are generally used for these purposes. T1090.003 - Multi-Hop Proxy Multiple proxies can also be chained together to abuse the actual traffic directions, making it more difficult for defenders to trace malicious activity and identify its source. T1090.004 - Domain Fronting Adversaries can even misuse Content Delivery Networks (CDNs) routing schemes to infect the actual HTTPS traffic destination or traffic tunneled through HTTPS. T1219 – Remote Access Tools To access the target system remotely and establish an interactive command-and-control within the network, remote access tools are used to bridge a session between two trusted hosts through a graphical interface, a CLI, or a hardware-level access (KVM, Keyboard, Video, Mouse) over IP solutions. T1219.001 - IDE Tunneling IDE Tunneling combines SSH, port forwarding, file sharing and letting the developers gain access as if they are local, by encapsulating the entire session and tunneling protocols alongside SSH, allowing the attackers to blend in with the actual development workflow. T1219.002 - Remote Desktop Software Adversary may access the target systems interactively through desktop support software, which provides a graphical interface to the remote adversary, such as VNC, Team Viewer, AnyDesk, LogMein, are commonly used legitimate support software. T1219.003 - Remote Access Hardware To access the legitimate hardware through commonly used legitimate tools, including IP-based keyboard, video, or mouse (KVM) devices such as TinyPilot and PiKVM. T1205 – Traffic Signaling Traffic signaling is used to hide open ports or any other malicious functionality to prolong command-and-control over the compromised system. T1205.001 - Port Knocking To hide the open ports for persistence, port knocking is included, to enable the port, in which the adversary sends a series of attempted connections to a predefined sequence of closed ports. T1205.002 - Socket filters Socket Filters are filters to allow or disallow certain types of data through the socket. If packets received by the network interface match the filtering criteria, desired actions are triggered. T1102 – Web Service Adversaries use an existing, legitimate external Web Service to transfer data to/from the compromised system. Also, web service providers commonly use SSL/TLS encryption, which gives adversaries an additional level of protection. T1102.001 - Dead Drop Resolver Adversaries post content called dead drop resolver on Web Services with encoded domains. These resolvers will redirect the victims to the infected domain/IP addresses. T1102.002 - Bidirectional Communication Once the system is infected, they can send the output back to the Web Service Channel. T1102.003 - One-Way Communication Compromised Systems may not return any output at all in a few cases where adversaries tend to send only one way instructions and do not want any response. How F5 Can Help F5 security solutions provide multiple different functionalities to secure and protect applications and APIs across various platforms including Clouds, Edge, On-prem or Hybrid. F5 supports risk management solutions mentioned below to effectively mitigate and protect against command-and-control techniques: Web Application Firewall (WAF): WAF is supported by all the F5 deployment modes, which is an adaptable, multi-layered security solution that defends web applications against a broad spectrum of threats, regardless of where they are deployed. API Security: F5 offers to ease the security of APIs with F5 Web Application and API Protection (WAAP) solutions, which protects API endpoints and other API dependencies by restricting the API definitions using specified rules and schemas. Rate-Limiting & Bot Protection: Brute-force, credential stuffing, and session attacks can be mitigated with configurable thresholds and automated bot protection. For more information, please contact your local F5 sales team. Conclusion Command and Control (C2) encompasses the methods adversaries employ to communicate with compromised systems within a target network. Adversaries disguise their C2 traffic as legitimate network activity to evade detection. To defend against Command-and-Control techniques, defenders should gain a clear understanding of implementation of robust segmentation and egress filtering using Web Application Firewalls (WAF) to limit communication channels and regularly monitor traffic for anomalous patterns and leverage threat intelligence to identify any C2 indicator. Additionally, employing endpoint detection and response (EDR) using API Security solutions can help detect and block malicious C2 activity at the host level. Reference links MITRE | ATT&CK Tactic 09 – Command and Control MITRE ATT&CK: What It Is, how it Works, Who Uses It and Why | F5 Labs MITRE ATT&CK®Overview of MITRE ATT&CK Tactic - TA0002 Execution
Introduction: Execution refers to the methods adversaries use to run malicious code on a target system. This tactic includes a range of techniques designed to execute payloads after gaining access to the network. It is a key stage in the attack lifecycle, as it allows attackers to activate their malicious actions, such as deploying malware, running scripts, or exploiting system vulnerabilities. Successful execution can lead to deeper system control, enabling attackers to perform actions like data theft, system manipulation, or establishing persistence for future exploitation. Now, let’s dive into the various techniques under the Execution tactic and explore how attackers use them. 1. T1651: Cloud Administration Command: Cloud management services can be exploited to execute commands within virtual machines. If an attacker gains administrative access to a cloud environment, they may misuse these services to run commands on the virtual machines. Furthermore, if an adversary compromises a service provider or a delegated administrator account, they could also exploit trusted relationships to execute commands on connected virtual machines. 2. T1059: Command and Scripting Interpreter The misuse of command and script interpreters allows adversaries to execute commands, scripts, or binaries. These interfaces, such as Unix shells on macOS and Linux, Windows Command Shell, and PowerShell are common across platforms and provide direct interaction with systems. Cross-platform interpreters like Python, as well as those tied to client applications (e.g., JavaScript, Visual Basic), can also be misused. Attackers may embed commands and scripts in initial access payloads or download them later via an established C2 (Command and Control) channel. Commands may also be executed via interactive shells or through remote services to enable remote execution. (.001) PowerShell: As PowerShell is already part of Windows, attackers often exploit it to execute commands discreetly without triggering alarms. It’s often used for things like finding information, moving across networks, and running malware directly in memory. This helps avoid detection because nothing is written to disk. Attackers can also execute PowerShell scripts without launching the powershell.exe program by leveraging.NET interfaces. Tools like Empire, PowerSploit, and PoshC2 make it even easier for attackers to use PowerShell for malicious purposes. Example - Remote Command Execution (.002) AppleScript: AppleScript is an macOS scripting language designed to control applications and system components through inter-application messages called AppleEvents. These AppleEvent messages can be sent by themselves or with AppleScript. They can find open windows, send keystrokes, and interact with almost any open application, either locally or remotely. AppleScript can be executed in various ways, including through the command-line interface (CLI) and built-in applications. However, it can also be abused to trigger actions that exploit both the system and the network. (.003) Windows Command Shell: The Windows Command Prompt (CMD) is a lightweight, simple shell on Windows systems, allowing control over most system aspects with varying permission levels. However, it lacks the advanced capabilities of PowerShell. CMD can be used from a distance using Remote Services. Attackers may use it to execute commands or payloads, often sending input and output through a command-and-control channel. Example - Remote Command Execution (.004) Unix Shell: Unix shells serve as the primary command-line interface on Unix-based systems. They provide control over nearly all system functions, with certain commands requiring elevated privileges. Unix shells can be used to run different commands or payloads. They can also run shell scripts to combine multiple commands as part of an attack. Example - Remote Command Execution (.005) Visual Basic: Visual Basic (VB) is a programming language developed by Microsoft, now considered a legacy technology. Visual Basic for Applications (VBA) and VBScript are derivatives of VB. Malicious actors may exploit VB payloads to execute harmful commands, with common attacks, including automating actions via VBScript or embedding VBA content (like macros) in spear-phishing attachments. (.006) Python: Attackers often use popular scripting languages, like Python, due to their interoperability, cross-platform support, and ease of use. Python can be run interactively from the command line or through scripts that can be distributed across systems. It can also be compiled into binary executables. With many built-in libraries for system interaction, such as file operations and device I/O, attackers can leverage Python to download and execute commands, scripts, and perform various malicious actions. Example - Code Injection (.007) JavaScript: JavaScript (JS) is a platform-independent scripting language, commonly used in web pages and runtime environments. Microsoft's JScript and JavaScript for Automation (JXA) on macOS are based on JS. Adversaries exploit JS to execute malicious scripts, often through Drive-by Compromise or by downloading scripts as secondary payloads. Since JS is text-based, it is often obfuscated to evade detection. Example - XSS (.008) Network Device CLI: Network devices often provide a CLI or scripting interpreter accessible via direct console connection or remotely through telnet or SSH. These interfaces allow interaction with the device for various functions. Adversaries may exploit them to alter device behavior, manipulate traffic, load malicious software by modifying configurations, or disable security features and logging to avoid detection. (.009) Cloud API: Cloud APIs offer programmatic access to nearly all aspects of a tenant, available through methods like CLIs, in-browser Cloud Shells, PowerShell modules (e.g., Azure for PowerShell), or SDKs for languages like Python. These APIs provide administrative access to major services. Malicious actors with valid credentials, often stolen, can exploit these APIs to perform malicious actions. (.010) AutoHotKey & AutoIT: AutoIT and AutoHotkey (AHK) are scripting languages used to automate Windows tasks, such as clicking buttons, entering text, and managing programs. Attackers may exploit AHK (.ahk) and AutoIT (.au3) scripts to execute malicious code, like payloads or keyloggers. These scripts can also be embedded in phishing payloads or compiled into standalone executable files (.011) Lua: Lua is a cross-platform scripting and programming language, primarily designed for embedding in applications. It can be executed via the command-line using the standalone Lua interpreter, through scripts (.lua), or within Lua-embedded programs. Adversaries may exploit Lua scripts for malicious purposes, such as abusing or replacing existing Lua interpreters to execute harmful commands at runtime. Malware examples developed using Lua include EvilBunny, Line Runner, PoetRAT, and Remsec. (.012) Hypervisor CLI: Hypervisor CLIs offer extensive functionality for managing both the hypervisor and its hosted virtual machines. On ESXi systems, tools like “esxcli” and “vim-cmd” allow administrators to configure and perform various actions. Attackers may exploit these tools to enable actions like File and Directory Discovery or Data Encrypted for Impact. Malware such as Cheerscrypt and Royal ransomware have leveraged this technique. 3. T1609: Container Administration Command Adversaries may exploit container administration services, like the Docker daemon, Kubernetes API server, or kubelet, to execute commands within containers. In Docker, attackers can specify an entry point to run a script or use docker exec to execute commands in a running container. In Kubernetes, with sufficient permissions, adversaries can gain remote execution by interacting with the API server, kubelet, or using commands like kubectl exec within the cluster. 4. T1610: Deploy Container Containers can be exploited by attackers to run malicious code or bypass security measures, often through the use of harmful processes or weak settings, such as missing network rules or user restrictions. In Kubernetes environments, attackers may deploy containers with elevated privileges or vulnerabilities to access other containers or the host node. They may also use compromised or seemingly benign images that later download malicious payloads. 5. T1675: ESXi Administration Command ESXi administration services can be exploited to execute commands on guest machines within an ESXi virtual environment. ESXi-hosted VMs can be remotely managed via persistent background services, such as the VMware Tools Daemon Service. Adversaries can perform malicious activities on VMs by executing commands through SDKs and APIs, enabling follow-on behaviors like File and Directory Discovery, Data from Local System, or OS Credential Dumping. 6. T1203: Exploitation for Client Execution Adversaries may exploit software vulnerabilities in client applications to execute malicious code. These exploits can target browsers, office applications, or common third-party software. By exploiting specific vulnerabilities, attackers can achieve arbitrary code execution. The most valuable exploits in an offensive toolkit are often those that enable remote code execution, as they provide a pathway to gain access to the target system. Example: Remote Code Execution 7. T1674: Input Injection Input Injection involves adversaries simulating keystrokes on a victim’s computer to carry out actions on their behalf. This can be achieved through several methods, such as emulating keystrokes to execute commands or scripts, or using malicious USB devices to inject keystrokes that trigger scripts or commands. For example, attackers have employed malicious USB devices to simulate keystrokes that launch PowerShell, enabling the download and execution of malware from attacker-controlled servers. 8. T1559: Inter-Process Communication Inter-Process Communication (IPC) is commonly used by processes to share data, exchange messages, or synchronize execution. It also helps prevent issues like deadlocks. However, IPC mechanisms can be abused by adversaries to execute arbitrary code or commands. The implementation of IPC varies across operating systems. Additionally, command and scripting interpreters may leverage underlying IPC mechanisms, and adversaries might exploit remote services—such as the Distributed Component Object Model (DCOM)—to enable remote IPC-based execution. (.001) Component Object Model (Windows): Component Object Model (COM) is an inter-process communication (IPC) mechanism in the Windows API that allows interaction between software objects. A client object can invoke methods on server objects via COM interfaces. Languages like C, C++, Java, and Visual Basic can be used to exploit COM interfaces for arbitrary code execution. Certain COM objects also support functions such as creating scheduled tasks, enabling fileless execution, and facilitating privilege escalation or persistence. (.002) Dynamic Data Exchange (Windows): Dynamic Data Exchange (DDE) is a client-server protocol used for one-time or continuous inter-process communication (IPC) between applications. Adversaries can exploit DDE in Microsoft Office documents—either directly or via embedded files—to execute commands without using macros. Similarly, DDE formulas in CSV files can trigger unintended operations. This technique may also be leveraged by adversaries on compromised systems where direct access to command or scripting interpreters is restricted. (.003) XPC Services(macOS): macOS uses XPC services for inter-process communication, such as between the XPC Service daemon and privileged helper tools in third-party apps. Applications define the communication protocol used with these services. Adversaries can exploit XPC services to execute malicious code, especially if the app’s XPC handler lacks proper client validation or input sanitization, potentially leading to privilege escalation. 9. T1106: Native API Native APIs provide controlled access to low-level kernel services, including those related to hardware, memory management, and process control. These APIs are used by the operating system during system boot and for routine operations. However, adversaries may abuse native API functions to carry out malicious actions. By using assembly directly or indirectly to invoke system calls, attackers can bypass user-mode security measures such as API hooks. Also, attackers may try to change or stop defensive tools that track API use by removing functions or changing sensor behavior. Many well-known exploit tools and malware families—such as Cobalt Strike, Emotet, Lazarus Group, LockBit 3.0, and Stuxnet—have leveraged Native API techniques to bypass security mechanisms, evade detection, and execute low-level malicious operations. 10. T1053: Scheduled Task/Job This technique involves adversaries abusing task scheduling features to execute malicious code at specific times or intervals. Task schedulers are available across major operating systems—including Windows, Linux, macOS, and containerized environments—and can also be used to schedule tasks on remote systems. Adversaries commonly use scheduled tasks for persistence, privilege escalation, and to run malicious payloads under the guise of trusted system processes. (.002) At: The “At” utility is available on Windows, Linux, and macOS for scheduling tasks to run at specific times. Adversaries can exploit “At” to execute programs at system startup or on a set schedule, helping them maintain persistence. It can also be misused for remote execution during lateral movement or to run processes under the context of a specific user account. In Linux environments, attackers may use “At “to break out of restricted environments, aiding in privilege escalation. (.003) Cron: The “cron” utility is a time-based job scheduler used in Unix-like operating systems. The “crontab” file contains scheduled tasks and the times at which they should run. These files are stored in system-specific file paths. Adversaries can exploit “cron” in Linux or Unix environments to execute programs at startup or on a set schedule, maintaining persistence. In ESXi environments, “cron” jobs must be created directly through the “crontab” file. (.005) Scheduled Task: Adversaries can misuse Windows Task Scheduler to run programs at startup or on a schedule, ensuring persistence. It can also be exploited for remote execution during lateral movement or to run processes under specific accounts (e.g., SYSTEM). Similar to System Binary Proxy Execution, attackers may hide one-time executions under trusted system processes. They can also create "hidden" tasks that are not visible to defender tools or manual queries. Additionally, attackers may alter registry metadata to further conceal these tasks. (.006) Systemd Timers: Systemd timers are files with a .timer extension used to control services in Linux, serving as an alternative to Cron. They can be activated remotely via the systemctl command over SSH. Each .timer file requires a corresponding .service file. Adversaries can exploit systemd timers to run malicious code at startup or on a schedule for persistence. Timers placed in privileged paths can maintain root-level persistence, while user-level timers can provide user-level persistence. (.007) Container Orchestration Job: Container orchestration jobs automate tasks at specific times, similar to cron jobs on Linux. These jobs can be configured to maintain a set number of containers, helping persist within a cluster. In Kubernetes, a CronJob schedules a Job that runs containers to perform tasks. Adversaries can exploit CronJobs to deploy Jobs that execute malicious code across multiple nodes in a cluster. 11. T1648: Serverless Execution Cloud providers offer various serverless resources such as compute functions, integration services, and web-based triggers that adversaries can exploit to execute arbitrary commands, hijack resources, or deploy functions for further compromise. Cloud events can also trigger these serverless functions, potentially enabling persistent and stealthy execution over time. An example of this is Pacu, a well-known open-source AWS exploitation framework, which leverages serverless execution techniques. 12. T1229: Shared Modules Shared modules are executable components loaded into processes to provide access to reusable code, such as custom functions or Native API calls. Adversaries can abuse this mechanism to execute arbitrary payloads by modularizing their malware into shared objects that perform various malicious functions. On Linux and macOS, the module loader can load shared objects from any local path. On Windows, the loader can load DLLs from both local paths and Universal Naming Convention (UNC) network paths. 13. T1072: Software Deployment Tools Adversaries may exploit centralized management tools to execute commands and move laterally across enterprise networks. Access to endpoint or configuration management platforms can enable remote code execution, data collection, or destructive actions like wiping systems. SaaS-based configuration management tools can also extend this control to cloud-hosted instances and on-premises systems. Similarly, configuration tools used in network infrastructure devices may be abused in the same way. The level of access required for such activity depends on the system’s configuration and security posture. 14. T1569: System Services System services and daemons can be abused to execute malicious commands or programs, whether locally or remotely. Creating or modifying services allows execution of payloads for persistence—particularly if set to run at startup—or for temporary, one-time actions. (.001) Launchctl (MacOS): launchctl interacts with launchd, the service management framework for macOS. It supports running subcommands via the command line, interactively, or from standard input. Adversaries can use launchctl to execute commands and programs as Launch Agents or Launch Daemons, either through scripts or manual commands. (.002) Service Execution (Windows): The Windows Service Control Manager (services.exe) manages services and is accessible through both the GUI and system utilities. Tools like PsExec and sc.exe can be used for remote execution by specifying remote servers. Adversaries may exploit these tools to execute malicious content by starting new or modified services. This technique is often used for persistence or privilege escalation. (.003) Systemctl (Linux): systemctl is the main interface for systemd, the Linux init system and service manager. It is typically used from a shell but can also be integrated into scripts or applications. Adversaries may exploit systemctl to execute commands or programs as systemd services. 15. T1204: User Execution Users may be tricked into running malicious code by opening a harmful file or link, often through social engineering. While this usually happens right after initial access, it can occur at other stages of an attack. Adversaries might also deceive users to enable remote access tools, run malicious scripts, or coercing users to manually download and execute malware. Tech support scams often use phishing, vishing, and fake websites, with scammers spoofing numbers or setting up fake call centers to steal access or install malware. (.001) Malicious Link: Users may be tricked into clicking on a link that triggers code execution. This could also involve exploiting a browser or application vulnerability (Exploitation for Client Execution). Additionally, links might lead users to download files that, when executed, deliver malware file. (.002) Malicious File: Users may be tricked into opening a file that leads to code execution. Adversaries often use techniques like masquerading and obfuscating files to make them appear legitimate, increasing the chances that users will open and execute the malicious file. (.003) Malicious Image: Cloud images from platforms like AWS, GCP, and Azure, as well as popular container runtimes like Docker, can be backdoored. These compromised images may be uploaded to public repositories and users might unknowingly download and deploy an instance or container, bypassing Initial Access defenses. Adversaries may also use misleading names to increase the chances of users mistakenly deploying the malicious image. (.004) Malicious Copy and Paste: Users may be deceived into copying and pasting malicious code into a Command or Scripting Interpreter. Malicious websites might display fake error messages or CAPTCHA prompts, instructing users to open a terminal or the Windows Run Dialog and run arbitrary, often obfuscated commands. Once executed, the adversary can gain access to the victim's machine. Phishing emails may also be used to trick users into performing this action. 16. T1047: Windows Management Instrumentation WMI (Windows Management Instrumentation) is a tool designed for programmers, providing a standardized way to manage and access data on Windows systems. It serves as an administrative feature that allows interaction with system components. Adversaries can exploit WMI to interact with both local and remote systems, using it to perform actions such as gathering information for discovery or executing commands and payloads. How F5 can help? F5 security solutions like WAF (Web Application Firewall), API security, and DDoS mitigation protect the applications and APIs across platforms including Clouds, Edge, On-prem or Hybrid, thereby reducing security risks. F5 bot and risk management solutions can also stop bad bots and automation. This can make your modern applications safer. The example attacks mentioned under techniques can be effectively mitigated by F5 products like Distributed Cloud, BIG-IP and NGINX. Here are a few links which explain the mitigation steps. Mitigating Cross-Site Scripting (XSS) using F5 Advanced WAF Mitigating Struts2 RCE using F5 BIG-IP For more details on the other mitigation techniques of MITRE ATT&CK Execution Tactic TA0002, please reach out to your local F5 team. Reference Links: MITRE ATT&CK® Execution, Tactic TA0002 - Enterprise | MITRE ATT&CK® MITRE ATT&CK: What It Is, How it Works, Who Uses It and Why | F5 LabsOverview of MITRE ATT&CK Tactic : TA0009 - Collection
This article is a continuation of our MITRE ATT&CK series. In this article, we focus on the Collection tactic, and the techniques adversaries use to gather, stage, and organize data from compromised systems before exfiltration. As attackers progress through an intrusion, Collection becomes critical for assembling sensitive files, credentials, screenshots, and other high‑value information that will fuel data theft, espionage, or destructive operations.Overview of MITRE ATT&CK Tactic: TA0040 - Impact
This article focuses on the Impact Tactic, and the techniques adversaries use to manipulate, disrupt or damage the systems and data as they reach the final stage of an attack. This is one of the critical tactics, as it highlights the adverse effects attackers can cause, including exploitation, operational disruption, data destruction, or financial gainF5 Distributed Cloud (XC) Custom Routes: Capabilities, Limitations, and Key Design Considerations
This article explores how Custom Routes work in F5 Distributed Cloud (XC), why they differ architecturally from standard Load Balancer routes, and what to watch out for in real-world deployments, covering backend abstraction, Endpoint/Cluster dependencies, and critical TLS trust and Root CA requirements.Overview of MITRE ATT&CK Tactic - TA0010 Exfiltration
Introduction In current times of cyber vulnerabilities, data theft is the ultimate objective with which attackers monetize their presence within a victim network. Once valuable information is identified and collected, the attackers can package sensitive data, bypass perimeter defences, and finalize the breach. Exfiltration (MITRE ATT&CK Tactic TA0010) represents a critical stage of the adversary lifecycle, where the adversaries focus on extracting data from the systems under their control. There are multiple ways to achieve this, either by using encryption and compression to avoid detection or utilizing the command-and-control channel to blend in with normal network traffic. To avoid this data loss, it is important for defenders to understand how data is transferred from any system in the network and the various transmission limits imposed to maintain stealth. This article walks through the most common Exfiltration techniques and how F5 solutions provide strong defense against them. T1020 - Automated Exfiltration To exfiltrate the data, adversaries may use automated processing after gathering the sensitive data during collection. T1020.001 – Traffic Duplication Traffic mirroring is a native feature for some devices for traffic analysis, which can be used by adversaries to automate data exfiltration. T1030 – Data Transfer Size Limits Exfiltration of the data in limited-size packets instead of whole files to avoid network data transfer threshold alerts. T1048 – Exfiltration over Alternative Protocol Stealing of data over a different protocol or channel other than the command-and-control channel created by the adversary. T1048.001 – Exfiltration Over Symmetric Encrypted Non-C2 Protocol Symmetric Encryption uses shared or the same keys/secrets on all the channels, which requires an exchange of the value used to encrypt and decrypt the data. This symmetric encryption leads to the implementation of Symmetric Cryptographic Algorithms, like RC4, AES, baked into the protocols, resulting in multiple layers of encryption. T1048.002 – Exfiltration Over Asymmetric Encrypted Non-C2 Protocol Asymmetric encryption algorithms or public-key cryptography require a pair of cryptographic keys that can encrypt/decrypt data from the corresponding keys on each end of the channel. T1048.003 – Exfiltration Over Unencrypted Non-C2 Protocol Instead of encryption, adversaries may obfuscate the routine channel without encryption within network protocols either by custom or publicly available encoding/compression algorithms (base64, hex-code) and embedding the data. T1041 – Exfiltration Over C2 Channel Adversaries can also steal the data over command-and-control channels and encode the data into normal communications. T1011 – Exfiltration Over Other Network Medium Exfiltration can also occur through a wired Internet connection, for example, a WiFi connection, modem, cellular data connection or Bluetooth. T1011.001 – Exfiltration Over Bluetooth Bluetooth can also be used to exfiltrate the data instead of a command-and-control channel in case the command-and-control channel is a wired Internet connection. T1052 – Exfiltration Over Physical Medium Under circumstances, such as an air-gapped network compromise, exfiltration occurs through a physical medium. Adversaries can exfiltrate data using a physical medium, for example, say a removable drive. Some examples of such media include external hard drives, USB drives, cellular phones, or MP3 players. T1052.001 – Exfiltration Over USB One such circumstance is where the adversary may attempt to exfiltrate data over a USB connected physical device, which can be used as the final exfiltration point or to hop between other disconnected systems. T1567 – Exfiltration Over Web Services Adversaries may use legitimate external Web Service to exfiltrate the data instead of their command-and-control channel. T1567.001 – Exfiltration to Code Repository To exfiltrate the data to a code repository, rather than adversary’s command-and-control channel. These code repositories are accessible via an API over HTTPS. T1567.002 – Exfiltration to Cloud Storage To exfiltrate the data to a cloud storage, rather than their primary command-and-control channel. These cloud storage services allow storage, editing and retrieval of the exfiltrated data. T1567.003 – Exfiltration to Text Storage Sites To exfiltrate the data to a text storage site, rather than their primary command-and-control. These text storage sites, like pastebin[.]com, are used by developers to share code. T1567.004 – Exfiltration Over Webhook Adversaries also exfiltrate the data to a webhook endpoint, which are simple mechanisms for allowing a server to push data over HTTP/S to a client. The creation of webhooks is supported by many public services, such as Discord and Slack, that can be used by other services, like GitHub, Jira, or Trello. T1029 – Scheduled Transfer To exfiltrate the data, the adversaries may schedule data exfiltration only at certain times of the day or at certain intervals, blending the traffic patterns with general activity. T1537 – Transfer Data to Cloud Account Many a times, exfiltration of data can also be through transferring the data through sharing/syncing and creating backups of cloud environment to another cloud account under adversary control on the same service. How F5 Can Help F5 offers a comprehensive suite of security solutions designed to safeguard applications and APIs across diverse environments, including cloud, edge, on-premises, and hybrid platforms. These solutions enable robust risk management to effectively mitigate and protect against MITRE ATT&CK Exfiltration threats, delivering advanced functionalities such as: Web Application Firewall (WAF): Available across all F5 products, the WAF is a flexible, multi-layered security solution that protects web applications from a wide range of threats. It delivers consistent defense, whether applications are deployed on-premises, in the cloud, or in hybrid environments. HTTPS Encryption: F5 provides robust HTTPS encryption to secure sensitive data in transit, ensuring protected communication between users and applications by preventing unauthorized access or data interception. Protecting sensitive data with Data Guard: F5's WAF Data Guard feature prevents sensitive data leakage by detecting and blocking exposure of confidential information, such as credit card numbers and PII. It uses predefined patterns and customizable policies to identify transmissions of sensitive data in application responses or inputs. This proactive mechanism secures applications against data theft and ensures compliance with regulatory standards. For more information, please contact your local F5 sales team. Conclusion Adversaries Exfiltration of data often aims to steal sensitive information by packaging it to evade detection, using methods such as compression or encryption. They may transfer the data through command-and-control channels or alternate paths while applying stealth techniques like transmission size limitations. To defend against these threats, F5 provides a layered approach with its advanced offerings. The Web Application Firewall (WAF) identifies and neutralizes malicious traffic aimed at exploiting application vulnerabilities. HTTPS encryption ensures secure data transmission, preventing unauthorized interception during the attack. Meanwhile, a data guard policy set helps detect and block exposure of confidential information, such as credit card numbers and PII. Together, these F5 solutions effectively counteract data exfiltration attempts and safeguard critical assets. Reference links MITRE | ATT&CK Tactic 10 – Exfiltration MITRE ATT&CK: What It Is, how it Works, Who Uses It and Why | F5 Labs MITRE ATT&CK®Thinking Outside the Box: Rewriting Web Pages with F5 Distributed Cloud (XC)
This article demonstrates how to dynamically rewrite web page content, such as updating links or replacing text, by using native features in F5 Distributed Cloud (XC). It provides a creative workaround that leverages JavaScript injection to modify pages on the fly, avoiding the need for a separate proxy like NGINX or BIG-IP.F5 Distributed Cloud Kubernetes Integration: Securing Services with Direct Pod Connectivity
Introduction As organizations embrace Kubernetes for container orchestration, they face critical challenges in exposing services securely to external consumers while maintaining granular control over traffic management and security policies. Traditional approaches using NodePort services or basic ingress controllers often fall short in providing the advanced application delivery and security features required for production workloads. F5 Distributed Cloud (F5 XC) addresses these challenges by offering enterprise-grade application delivery and security services through its Customer Edge (CE) nodes. By establishing direct connectivity to Kubernetes pods, F5 XC can provide sophisticated load balancing, WAF protection, API security, and multi-cloud connectivity without the limitations of NodePort-based architectures. This article demonstrates how to architect and implement F5 XC CE integration with Kubernetes clusters to expose and secure services effectively, covering both managed Kubernetes platforms (AWS EKS, Azure AKS, Google GKE) and self-managed clusters using K3S with Cilium CNI. Understanding F5 XC Kubernetes Service Discovery F5 Distributed Cloud includes a native Kubernetes service discovery feature that communicates directly with Kubernetes API servers to retrieve information about services and their associated pods. This capability operates in two distinct modes: Isolated Mode In this mode, F5 XC CE nodes are isolated from the Kubernetes cluster pods and can only reach services exposed as NodePort services. While the discovery mechanism can retrieve all services, connectivity is limited to NodePort-exposed endpoints with the inherent NodePort limitations: Port Range Restrictions: Limited to ports 30000-32767 Security Concerns: Exposes services on all node IPs Performance Overhead: Additional network hops through kube-proxy Limited Load Balancing: Basic round-robin without advanced health checks Non-Isolated Mode, Direct Pod Connectivity (and why it matters) This is the focus of our implementation. In non-isolated mode, F5 XC CE nodes can reach Kubernetes pods directly using their pod IP addresses. This provides several advantages: Simplified Architecture: Eliminate NodePort complexity and port management limitation Enhanced Security: Apply WAF, DDoS protection, and API security directly at the pod level Advanced Load Balancing: Sophisticated algorithms, circuit breaking, and retry logic Architectural Patterns for Pod IP Accessibility To enable direct pod connectivity from external components like F5 XC CEs, the pod IP addresses must be routable outside the Kubernetes cluster. The implementation approach varies based on your infrastructure: Cloud Provider Managed Kubernetes Cloud providers typically handle pod IP routing through their native Container Network Interfaces (CNIs): Figure 1: Cloud providers' K8S CNI routes PODs IPs to the Cloud Provider Private Cloud Routing Table AWS EKS: Uses Amazon VPC CNI, which assigns VPC IP addresses directly to pods Azure AKS: Traditional CNI mode allocates Azure VNET IPs to pods Google GKE: VPC-native clusters provide direct pod IP routing In these environments, the cloud provider's CNI automatically updates routing tables to make pod IPs accessible within the VPC/VNET. Self-Managed Kubernetes Clusters For self-managed clusters, you need an advanced CNI that can expose the Kubernetes overlay network. The most common solutions are: Cilium: Provides eBPF-based networking with BGP support Calico: Offers flexible networking policies with BGP peering capabilities and eBPF support as well These CNIs typically use BGP to advertise pod subnets to external routers, making them accessible from outside the cluster. Figure 2: Self-managed K8S clusters use advanced CNI with BGP to expose the overlay subnet Cloud Provider Implementations AWS EKS Architecture Figure 3: AWS EKS with F5 XC CE integration using VPC CNI With AWS EKS, the VPC CNI plugin assigns real VPC IP addresses to pods, making them directly routable within the VPC without additional configuration. Azure AKS Traditional CNI Figure 4: Azure AKS with traditional CNI mode for direct pod connectivity Azure's traditional CNI mode allocates IP addresses from the VNET subnet directly to pods, enabling native Azure networking features. Google GKE VPC-Native Figure 5: Google GKE VPC-native clusters with alias IP ranges for pods GKE's VPC-native mode uses alias IP ranges to provide pods with routable IP addresses within the Google Cloud VPC. Deeper dive into the implementation Implementation Example 1: AWS EKS Integration Let's walk through a complete implementation using AWS EKS as our Kubernetes platform. Prerequisites and Architecture Network Configuration: VPC CIDR: 10.154.0.0/16 Three private subnets (one per availability zone) F5 XC CE deployed in Private Subnet 1 EKS worker nodes distributed across all three subnets Figure 6: Complete EKS implementation architecture with F5 XC CE integration Kubernetes Configuration: EKS cluster with AWS VPC CNI Sample application: microbot (simple HTTP service) Three replicas distributed across nodes What is running inside the K8S cluster? The PODs We have three PODs in the default namespace. Figure 7: The running PODs in the EKS cluster One running with POD IP 10.154.125.116, another one with POD IP 10.154.76.183 and one running with POD IP 10.154.69.183. microbot POD is a simple HTTP application that is returning the full name of the POD and an image. Figure 8: The microbot app The services Figure 9: The services running in the EKS cluster Configure F5 XC Kubernetes Service Discovery Create a K8S service discovery object. Figure 10: Kubernetes service discovery configuration In the “Access Credentials” activate the “Show Advanced Fields” slider. This is the key! Figure 11: The "advanced fields" slider Then provide the Kubeconfig file of the K8S cluster and select “Kubernetes POD reachable”. Figure 12: Kubernetes POD network reachability Then the K8S should be displayed in the “Service Discoveries”. Figure 13: The discovered PODs IPs One can see that the services are discovered by the F5 XC node and more interestingly, the PODs IPs. Are the pods reachable from the F5XC CE? Figure 14: Testing connectivity to pod 10.154.125.116 Figure 15: Testing connectivity to pod 10.154.76.183 Figure 16: Testing connectivity to pod 10.154.69.183 Yes, they are! Create Origin Pool with K8S Service Create an origin pool that references your Kubernetes service: Figure 17: Creating origin pool with Kubernetes service type Create an HTTPS Load-Balancer and test the service Just create a regular F5 XC HTTPS Load-Balancer and use the origin pool created above. Figure 18: Traffic load-balanced across the three PODs The result shows traffic being load-balanced across all EKS pods. Implementation Example 2: Self-Managed K3S with Cilium CNI One infrastructure subnet (10.154.1.0/24) in which the following components are going to be deployed: F5 XC CE single node (10.154.1.100) Two Linux Ubuntu nodes (10.154.1.10 & 10.154.1.11) On the Linux Ubuntu nodes, a Kubernetes cluster is going to be deployed using K3S (www.k3s.io) with the following specifications: PODs overlay subnet: 10.160.0.0/16 Services overlay subnet: 10.161.0.0/16 Default K3S CNI (flannel) will be disabled K3S CNI will be replaced by Cilium CNI to expose directly the PODs overlay subnet to the “external world” Figure 19: Self-managed K3S cluster with Cilium CNI and BGP peering to F5 XC CE What is running inside the K8S cluster? The PODs We have two PODs in the default namespace. Figure 20: The running PODs in the K8S cluster One running on node “k3s-1” with POD IP 10.160.0.203 and the other one running on node “k3s-2” with POD IP 10.160.1.208. microbot POD is a simple HTTP application that is returning the full name of the POD and an image. The services Figure 21: The services running in the K8S cluster Different Kubernetes services are created to expose the microbot PODs, one of type Cluster IP and the other one of type LoadBalancer. The type of service doesn’t really matter for F5XC because we are working in a full routed mode between the CE and the K8S cluster. F5XC only needs to “know” the PODs IPs, which will be discovered through the services. Configure F5 XC Kubernetes Service Discovery Steps are identical regarding what we did for EKS. And once done, services and PODs IPs are discovered by F5XC. Figure 22: The discovered PODs IPs Configure the BGP peering on F5XC CE In this example topology, BGP peerings are established directly between the K8S nodes and the F5 XC CE. Other implementations are possible, for instance, with an intermediate router. Figure 23: BGP peerings Check if the peerings are established. Figure 24: Verification of the BGP peerings Are the pods reachable from the F5XC CE? Figure 25: PODs reachability test They are! Create Origin Pool with K8S Service As we did for the EKS configuration, create an origin pool that references your Kubernetes service. Create an HTTPS Load-Balancer and test the service Just create a regular F5 XC HTTPS Load-Balancer and use the origin pool created above. Figure 26: Traffic load-balanced across the two PODs Scaling up? Let’s add another POD to the deployment to see how F5XC will handle the load-balancing after. Figure 27: Scaling up the Microbot PODs And it’s working! Load is spread automatically as soon as new PODs instances are available for the given service. Figure 28: Traffic load-balanced across the three PODs Appendix - K3S and Cilium deployment example Step 1: Install K3S without Default CNI On the master node: curl -sfL https://get.k3s.io | K3S_KUBECONFIG_MODE="644" \ INSTALL_K3S_EXEC="--flannel-backend=none \ --disable-network-policy \ --disable=traefik \ --disable servicelb \ --cluster-cidr=10.160.0.0/16 \ --service-cidr=10.161.0.0/16" sh - # Export kubeconfig export KUBECONFIG=/etc/rancher/k3s/k3s.yaml # Get token for worker nodes sudo cat /var/lib/rancher/k3s/server/node-token On worker nodes: IP_MASTER=10.154.1.10 K3S_TOKEN=<token-from-master> curl -sfL https://get.k3s.io | K3S_URL=https://${IP_MASTER}:6443 K3S_TOKEN=${K3S_TOKEN} sh - Step 2: Install and Configure Cilium On the K3S master node, please perform the following: Install Helm and Cilium CLI: # Install Helm sudo snap install helm --classic # Download Cilium CLI CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/main/stable.txt) CLI_ARCH=amd64 curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum} sha256sum --check cilium-linux-${CLI_ARCH}.tar.gz.sha256sum sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin Install Cilium with BGP support: helm repo add cilium https://helm.cilium.io/ helm install cilium cilium/cilium --version 1.16.5 \ --set=ipam.operator.clusterPoolIPv4PodCIDRList="10.160.0.0/16" \ --set kubeProxyReplacement=true \ --set k8sServiceHost=10.154.1.10 \ --set k8sServicePort=6443 \ --set bgpControlPlane.enabled=true \ --namespace kube-system \ --set bpf.hostLegacyRouting=false \ --set bpf.masquerade=true # Monitor installation cilium status --wait Step 3: Configure BGP Peering Label nodes for BGP: kubectl label nodes k3s-1 bgp=true kubectl label nodes k3s-2 bgp=true Create BGP configuration: # BGP Cluster Config apiVersion: cilium.io/v2alpha1 kind: CiliumBGPClusterConfig metadata: name: cilium-bgp spec: nodeSelector: matchLabels: bgp: "true" bgpInstances: - name: "k3s-instance" localASN: 65001 peers: - name: "f5xc-ce" peerASN: 65002 peerAddress: 10.154.1.100 peerConfigRef: name: "cilium-peer" --- # BGP Peer Config apiVersion: cilium.io/v2alpha1 kind: CiliumBGPPeerConfig metadata: name: cilium-peer spec: timers: holdTimeSeconds: 9 keepAliveTimeSeconds: 3 gracefulRestart: enabled: true restartTimeSeconds: 15 families: - afi: ipv4 safi: unicast advertisements: matchLabels: advertise: "bgp" --- # BGP Advertisement apiVersion: cilium.io/v2alpha1 kind: CiliumBGPAdvertisement metadata: name: bgp-advertisements labels: advertise: bgp spec: advertisements: - advertisementType: "PodCIDR"XC Distributed Cloud and how to keep the Source IP from changing with customer edges(CE)!
Code is community submitted, community supported, and recognized as ‘Use At Your Own Risk’. Old applications sometimes do not accept a different IP address to be used by the clients during the session/connection. How can make certain the IP stays the same for a client? The best will always be the application to stop tracking users based on something primitive as an ip address and sometimes the issue is in the Load Balancer or ADC after the XC RE and then if the persistence is based on source IP address on the ADC to be changed in case it is BIG-IP to Cookie or Universal or SSL session based if the Load Balancer is doing no decryption and it is just TCP/UDP layer . As an XC Regional Edge (RE) has many ip addresses it can connect to the origin servers adding a CE for the legacy apps is a good option to keep the source IP from changing for the same client HTTP requests during the session/transaction. Before going through this article I recommend reading the links below: F5 Distributed Cloud – CE High Availability Options: A Comparative Exploration | DevCentral F5 Distributed Cloud - Customer Edge | F5 Distributed Cloud Technical Knowledge Create Two Node HA Infrastructure for Load Balancing Using Virtual Sites with Customer Edges | F5 Distributed Cloud Technical Knowledge RE to CE cluster of 3 nodes The new SNAT prefix option under the origin pool allows no mater what CE connects to the origin pool the same IP address to be seen by the origin. Be careful as if you have more than a single IP with /32 then again the client may get each time different IP address. This may cause "inet port exhaustion " ( that is what it is called in F5BIG-IP) if there are too many connections to the origin server, so be careful as the SNAT option was added primary for that use case. There was an older option called "LB source IP persistence" but better not use it as it was not so optimized and clean as this one. RE to 2 CE nodes in a virtual site The same option with SNAT pool is not allowed for a virtual site made of 2 standalone CE. For this we can use the ring hash algorithm. Why this works? Well as Kayvan explained to me the hashing of the origin is taking into account the CE name, so the same origin under 2 different CE will get the same ring hash and the same source IP address will be send to the same CE to access the Origin Server. This will not work for a single 3-node CE cluster as it all 3 nodes have the same name. I have seen 503 errors when ring hash is enabled under the HTTP LB so enable it only under the XC route object and the attached origin pool to it! CE hosted HTTP LB with Advertise policy In XC with CE you can do do HA with 3-cluster CE that can be layer2 HA based on VRRP and arp or Layer 3 persistence based bgp that can work 3 node CE cluster or 2 CE in a virtual site and it's control options like weight, as prepend or local preference options at the router level. For the Layer 2 I will just mention that you need to allow 224.0.0.8 for the VRRP if you are migrating from BIG-IP HA and that XC selects 1 CE to hold active IP that is seen in the XC logs and at the moment the selection for some reason can't be controlled. if a CE can't reach the origin servers in the origin pool it should stop advertising the HTTP LB IP address through BGP. For those options Deploying F5 Distributed Cloud (XC) Services in Cisco ACI - Layer Three Attached Deployment is a great example as it shows ECMP BGP but with the BGP attributes you can easily select one CE to be active and processing connections, so that just one ip address is seen by the origin server. When a CE gets traffic by default it does prefer to send it to the origin as by default "Local Preferred" is enabled under the origin pool. In the clouds like AWS/Azure just a cloud native LB is added In front of the 3 CE cluster and the solution there is simple as to just modify the LB to have a persistence. Public Clouds do not support ARP, so forget about Layer 2 and play with the native LB that load balances between the CE 😉 CE on Public Cloud (AWS/Azure/GCP) When deploying on a public cloud the CE can be deployed in two ways one is through the XC GUI and adding the AWS credentials but this way you have not a big freedom to be honest as you can't deploy 2 CE and make a virtual site out of them and add cloud LB in-front of them as it always will be 3-CE cluster with preconfigured cloud LB that will use all 3 LB! Using the newer "clickops" method is much better https://docs.cloud.f5.com/docs-v2/multi-cloud-network-connect/how-to/site-management/deploy-site-aws-clickops or using terraform but with manual mode and aws as a provider (not XC/volterra as it is the same as the XC GUI deployment) https://docs.cloud.f5.com/docs-v2/multi-cloud-network-connect/how-to/site-management/deploy-aws-site-terraform This way you can make the Cloud LB to use just one CE or have some client Persistence or if traffic comes from RE to CE to implement the virtual site 2 CE node! There is no Layer 2 ARP support as I mentioned in public cloud with 3-node cluster but there is NAT policy https://docs.cloud.f5.com/docs-v2/multi-cloud-network-connect/how-tos/networking/nat-policies but I haven't tried it myself to comment on it. Hope you enjoyed this article!
