Copy that! Duplicate packets with iRules via an iApp

Articles F5 Technical Articles
, ,
1

No, that’s not a Jack Bauer reference (I’m lying…it totally is,) but rather the crux of the article: copying ingress packets to multiple destinations. This effort was accomplished more than a decade ago in an iApp created by Ken Bocchino. I don’t recall if it was built purposely for the Interop 2012 show it was showcased at in their NOC, or if we had it already and repurposed it, but regardless, it was built, and it was awesome. A lot of love throughout the years in the Codeshare comments, but it’s been a hot minute since it was updated, and modern versions don’t load it properly. This article covers the use cases I updated the iApp for, and links the resources should you have a need or desire to deploy and test in your environment.

Use Cases

The iApp did more than what I’ve updated, but I focused on these four use cases:

  1. UDP Spray - just replicate all the inbound packets to every defined destination. Useful for different types of ingress servers.
  2. UDP LB - replicate all the inbound packets to one of a group of defined destinations. Useful for several ingress servers of a specific type.
  3. UDP HA - replicate all the inbound packets in primary/secondary tiers. Useful for backup of primary destination failures.
  4. TCP Spray - same as UDP Spray, but with backend TCP individual sessions required, the IP/TCP layer of source details is compromised.

The UDP Spray application service, when deployed, will look like this:

udp_spray_discourse
udp_spray_discourse803×745 35.7 KB

You can review all the diagrams for these use cases in the repo at F5 BIG-IP packet duplication diagrams.

What’s Changed

Here are the major updates, plus a blurb on testing the most impactful changes to the iRules code that the iApp deploys.

Dual-Stack IPv4/IPv6 and Route Domain Support

  • Original: Hardcoded to IPv4 dot-decimal format, fixed 255.255.255.255 netmasks, and colon-delimited port strings (<ip>:<port>).
  • Latest: Fully dual-stack aware with dynamic netmasks, route domain (%rd) preservation, and adaptive endpoint formatting (dot-notation <addr>.<port> for IPv6, colon-notation <addr>:<port> for IPv4).

Header Serialization and Performance

  • Original: Used binary format ssssa256 and binary scan with 256-byte static padding, which caused UTF-8 string shimmering, memory allocation overhead, and payload truncation bugs.
  • Latest: Uses lightweight string-delimited framing (<ClientIP>#<Destination>#<Payload>) indexed via TMM C-level memchr routines, running over faster (test on your platforms, I tested in a lab environment on VEs) with zero data corruption.

TCP Traffic Duplication Optimization

  • Original: Called HSL::open -proto TCP on every incoming CLIENT_DATA event, adding per-packet handle allocation overhead in the fast path.
  • Latest: Pre-opens and pools HSL connection handles once during CLIENT_ACCEPTED, executing only lightweight list iteration in CLIENT_DATA. This one is hit or miss depending on the use case.

Monitor and Health-Check Modernization

  • Original: Used basic monitors with assumptions that failed across differing TMOS versions.
  • Latest: Integrated universal gateway_icmp monitoring and dynamic priority-group failover logic that works consistently across TMOS 17.1, 17.5, and 21.1.

Tooling and Test Automation

  • Original: Manual CLI deployment with static iApp presentation scripts.
  • Latest: Complete automation suite including targeted deployment scripts (deploy_171.sh, deploy_175.sh, deploy_211.sh), multi-node virtual wire test harness, and automated PCAP payload validation.

iRule Test Performance

With the big change from stuffing and extracting data from binary to using simple string commands, I wanted to make sure that I wasn’t designing in a performance drop. That testing can be done through traffic means, but you can also isolate the Tcl-native commands and run directly on the BIG-IP host or on a linux box to get a rough idea. I didn’t compile the iRules version of Tcl (8.4.6), but the later 8.4.x and 8.5.x versions I did test still showed marginal to significant improvement depending on the test run. This script is rule_bench_traffic_duplication.tcl in the repo. Just fire up the Tcl shell by typing tclsh at the bash prompt on BIG-IP or on any shell in a linux box, paste the contents of that file in and hit return and you’ll get a good idea on the performance diff in your environments. An example run from my MacOS Tcl shell:

::__irule_event_RULE_INIT
% % % % % Sep 30 12:50:14 local0.info     === STARTING SERIALIZATION BENCHMARK (Tcl 8.5.9, timer 1us, 5 rounds x 250000 iterations per test) ===

Sep 30 12:50:17 local0.info     --- BENCHMARK RESULTS (best of 5 rounds; 3.2ms loop overhead subtracted) ---
Sep 30 12:50:17 local0.info     Legacy Binary Approach    : Total=321.5ms | Avg=1.286us/pkt
Sep 30 12:50:17 local0.info     Modern Delimited Approach : Total=253.7ms | Avg=1.015us/pkt
Sep 30 12:50:17 local0.info     Performance : Modern approach is 1.27x FASTER (21.1% less time per packet)
Sep 30 12:50:17 local0.info     Integrity   : Both approaches round-tripped all fields correctly (1250000/1250000 each)
Sep 30 12:50:17 local0.info     ==========================================================================

If you want to see the original iRule version of this test, it’s saved in a previous commit under the same file name in the repo. But WARNING, don’t run that in production!

Client/Server Test Harness

Updating the iApp required a lot of validation, so I worked with Gemini Flash and Claude to create a test harness script that would be the client and server to pass traffic through the BIG-IP, so payloads could be unique and validated on the server side of things without complex deployment strategies. This test harness is run_all_tests.py in the repo. An example run from one of my test sessions:

root@webserver:/scripts# sudo python3 run_all_tests.py --ip-version all --interface ens7
[2026-09-30 17:45:10 UTC] Starting BIG-IP Multi-Version Dual-Stack Automated Test Runner...
[*] Sniffer Interface: ens7 (AF_PACKET raw kernel frame tap)
[*] Running full dual-stack battery: 24 test cases (12 IPv4, 12 IPv6)

================================================================================
                    EXECUTION MATRIX SUMMARY REPORT
================================================================================
#   | Fam  | Version  | Service Name            | VIP                  | Recv/Exp   | Status
--------------------------------------------------------------------------------
1   | IPv4 | TMOS 17.1 | app_171_udp_single      | 10.1.10.111          | 15/15      | PASS
2   | IPv4 | TMOS 17.1 | app_171_udp_lb          | 10.1.10.112          | 20/>=20    | PASS
3   | IPv4 | TMOS 17.1 | app_171_udp_failover    | 10.1.10.113          | 20/20      | PASS
4   | IPv4 | TMOS 17.1 | app_171_tcp_spray       | 10.1.10.114          | 15/15      | PASS
5   | IPv4 | TMOS 17.5 | app_175_udp_single      | 10.1.10.151          | 15/15      | PASS
6   | IPv4 | TMOS 17.5 | app_175_udp_lb          | 10.1.10.152          | 20/>=20    | PASS
7   | IPv4 | TMOS 17.5 | app_175_udp_failover    | 10.1.10.153          | 20/20      | PASS
8   | IPv4 | TMOS 17.5 | app_175_tcp_spray       | 10.1.10.154          | 15/15      | PASS
9   | IPv4 | TMOS 21.1 | app_211_udp_single      | 10.1.10.201          | 15/15      | PASS
10  | IPv4 | TMOS 21.1 | app_211_udp_lb          | 10.1.10.202          | 20/>=20    | PASS
11  | IPv4 | TMOS 21.1 | app_211_udp_failover    | 10.1.10.203          | 20/20      | PASS
12  | IPv4 | TMOS 21.1 | app_211_tcp_spray       | 10.1.10.204          | 15/15      | PASS
13  | IPv6 | TMOS 17.1 | app_171_v6_udp_single   | fd00:1:1:10::111     | 15/15      | PASS
14  | IPv6 | TMOS 17.1 | app_171_v6_udp_lb       | fd00:1:1:10::112     | 20/>=20    | PASS
15  | IPv6 | TMOS 17.1 | app_171_v6_udp_failover | fd00:1:1:10::113     | 20/20      | PASS
16  | IPv6 | TMOS 17.1 | app_171_v6_tcp_spray    | fd00:1:1:10::114     | 15/15      | PASS
17  | IPv6 | TMOS 17.5 | app_175_v6_udp_single   | fd00:1:1:10::151     | 15/15      | PASS
18  | IPv6 | TMOS 17.5 | app_175_v6_udp_lb       | fd00:1:1:10::152     | 20/>=20    | PASS
19  | IPv6 | TMOS 17.5 | app_175_v6_udp_failover | fd00:1:1:10::153     | 20/20      | PASS
20  | IPv6 | TMOS 17.5 | app_175_v6_tcp_spray    | fd00:1:1:10::154     | 15/15      | PASS
21  | IPv6 | TMOS 21.1 | app_211_v6_udp_single   | fd00:1:1:10::201     | 15/15      | PASS
22  | IPv6 | TMOS 21.1 | app_211_v6_udp_lb       | fd00:1:1:10::202     | 20/>=20    | PASS
23  | IPv6 | TMOS 21.1 | app_211_v6_udp_failover | fd00:1:1:10::203     | 20/20      | PASS
24  | IPv6 | TMOS 21.1 | app_211_v6_tcp_spray    | fd00:1:1:10::204     | 15/15      | PASS
================================================================================
Final Verdict: 24/24 Tests Passed (100.0% Success Rate across TMOS 17.1, 17.5, 21.1)

DevCentral Connects Walkthrough

Last week I hosted a live stream to walk through the iApp updates and run tests. You can watch that here:

Resources

The template, app service deployment scripts (tmsh commands in a shell script), and test harness are all in the repo. I’m linking the original codeshare entry for your viewing pleasure as well. Happy coding!

2 Likes
UDP TCP Packet Duplication