F5 CNF/BNK issue with DNS Express tmm scaling and zone notifications
I did see an interesting issue with DNS Express with Next for Kubernetes when playing in a test environment. When you have 2 TMM pods in the same namespace as the DNS zone mirroring is done by zxfrd pod and I you need to create a listener "F5BigDnsApp" as shown in https://clouddocs.f5.com/cnfs/robin/latest/cnf-dnsexpress.html#create-a-dns-zone-to-answer-dns-queries for the optional notify that will feed this to the TMM and then to the zxfrd pod. The issue happens when you have 2 or more TMM as then the "F5BigDnsApp" that is like virtual server/listener as then then on the internal vlans there is arp conflict as the two tmm on two different kubernetes/openshift nodes advertise the same ip address on layer 2. This is seen with "kubectl logs" ("oc logs" for Openshift) on the TMM pods that mention the duplicate arp detected. Interesting that the same does not happen when you do this for the normal listener on the external Vlan (the one that captures and responds to the client DNS queries) as I think by default the ARP is stopped for the external listener that can be on 2 or more TMM as ECMP BGP is used to redistribute the traffic to the TMM by design. I see 4 possible solutions as I see it. One is to be able to control the ARP for the "F5BigDnsApp" CRD for Internal or External Vlans (BGP ECMP to be used also on the server side then) and the second is to be able to select "F5BigDnsApp" to be deployed just one 1 TMM even if there are more. Also if an ip address could be configured for the listener that is not part of the internal ip address range but then as I see with "kubectl logs" on the ingress controller (f5ing-tmm-pod-manager) the config is not pushed to the TMM as also with "configview" from the debug sidecar container on the tmm pods there is no listener at all. The manager logs suggest that because the Listener IP address is not part of the Self-IP IP range under the intnernal Vlan as this maybe system limitation and no one thinking about this use case as in BIG-IP this is is supported to have VIP on non self ip address range that is not advertised with arp because of this. The last solution that can work at the moment is to have many tmm in different namespaces on different kubernetes nodes with affinity rules that can deploy each tmm on different node even if the tmm are on different namespaces by matching a configured label (see the example below) as maybe this is the current working design to have one zxfrd pod with one tmm pod in a namespace but then the auto-scaling may not work as euto scale should create a new tmm pod in the same namespace if needed. Example: affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchLabels: app: tmm # Match Pods in any namespaces that have this label namespaceSelector: {} # empty selector = all namespaces topologyKey: "kubernetes.io/hostname" Also it should be considered if the zxfrd pod can push the DNS zone to the RAM of more than one TMM pods as maybe it can't as maybe currently only one to one is supported. Maybe it was never tested what happens when you have Security Context IP address on the Internal Network and multiple TMM pods. Interest stuff that I just wanted to share as this was just testing things out😄F5 Kubernetes CNF/BNK GSLB functionality ?
Hello everyone is there F5 CNF/BNK GSLB functionality ? I see the containers gslb-engine (probably the main GTM/DNS module) and gslb-probe-agent (probably the big3d in a container/pod ) but no CR/CRD definitions about it and and can this data be shared between F5 TMM in different clusters (something like DNS sync groups) or probing normal F5 BIG-IP devices (not in kubernetes). https://clouddocs.f5.com/cnfs/robin/latest/cnf-software-install.html https://clouddocs.f5.com/cnfs/robin/latest/intro.html
Can I deploy the application study tool on Kubernetes?(AST)
Hi, everyone. I followed the AST guide to ensure that AST works on Docker. But, I want to deploy this on Kubernetes, is there a how-to or guide for that? If you have any suggestions on availability, or any alternative links or instructions I can refer to, I would really appreciate it. Thank you for your help.BIG-IP Next does not work on KubeVirt
Hi all, I use a KubeVirt based Hypervisor called Harvester (https://harvesterhci.io) and tried to start a BIG-IP Next Instance with the BIG-IP-Next-20.3.0-2.716.2+0.0.50.qcow2.tar.gz image. The VM does start but I cannot see any service on tcp port 5443 running after doing setup. What I have seen when I checked the logs of the f5-platform-manager deployment is the following: 2025-02-06T12:52:25.985842479Z stdout F "ts"="2025-02-06 12:52:25.985"|"l"="info"|"m"="Found an unknown Bios vendor"|"id"="19093-000230"|"lt"="A"|"vendor"="KubeVirt"|"pod"="f5-platform-manager-6f78695744-p48tr"|"ct"="f5-platform-manager"|"v"="1.0"|"src"="surveyor/z100_detector.go:60" 2025-02-06T12:52:25.98585795Z stdout F "ts"="2025-02-06 12:52:25.985"|"l"="error"|"m"="Failed to run surveyor probe"|"id"="19093-000259"|"lt"="A"|"error"="Unsupported virtual platform: '{ChassisAssetTag: MachineId: Mfr:KubeVirt Product:None Serial: Uuid:aeada22c-3bf3-5220-a678-91f04ac6db0d Version:pc-q35-7.1}'"|"pod"="f5-platform-manager-6f78695744-p48tr"|"ct"="f5-platform-manager"|"v"="1.0"|"src"="surveyor/surveyor.go:344" So what can I do to get this BIG-IP Next Instance running on the VM? Thanks, Peter
BIG-IP for SIP resources running in Kubernetes
Hello, We are trying to setup Virtual Server using BIG-IP that would server as a Load Balancer for SIP traffic for resources that are deployed in Kubernetes cluster and exposed through NodePort. Our F5 is not part of the Kubernetes cluster and it is a standalone Virtual Machine that sends its traffic to NodePort service of our SIP resources. We are facing few issues and hope someone can help us understand them. UDP not working When we try to use UDP the problem is that F5 (10.224.64.223) sends SIP OPTIONS to ip address/port that we defined as access point for SIP elements in Kubernetes (Node IP and NodePort port, 10.224.64.222, port:31131). But due to Kubernetes deployment, responses are sent from different IP address and port (10.224.64.222, port 30834). And this gets rejected by the F5. 10:17:23.695039 IP 10.224.64.223.51938 > 10.224.64.222.31131: UDP, length 575 out slot1/tmm1 lis=mon_mrf_sip_udp port=1.2 trunk= 10:17:23.700849 IP 10.224.64.220.30834 > 10.224.64.223.51938: UDP, length 520 in slot1/tmm0 lis= port=1.2 trunk= 10:17:23.700949 IP 10.224.64.223 > 10.224.64.220: ICMP 10.224.64.223 udp port 51938 unreachable, length 36 out slot1/tmm0 lis= port=1.2 trunk= Even the usage of macvlan on Kubernetes pods does not help. With macvlan we manage to achieve that IP address is preserved (10.226.64.225), but still the port changes (5060 -> 25404). And F5 rejects it. 10:42:07.370926 IP 10.224.64.223.54412 > 10.224.64.225.5060: SIP: OPTIONS sip:10.224.64.225:5060 SIP/2.0 out slot1/tmm0 lis= port=1.2 trunk= 10:42:07.378237 IP 10.224.64.225.25404 > 10.224.64.223.54412: UDP, length 425 in slot1/tmm0 lis= port=1.2 trunk= 10:42:07.378325 IP 10.224.64.223 > 10.224.64.225: ICMP 10.224.64.223 udp port 54412 unreachable, length 36 out slot1/tmm0 lis= port=1.2 trunk= So I guess there is no way to have it working for UDP at all with resources being deployed in Kubernets cluster? (host-network is not an option). TCP (in Message Routing mode) not working When we try to use TCP we found out that "Standard (SIP - legacy profile)" mode behaves differently then "Message Routing" one. In case when we use "Legacy" SIP monitor via TCP it establishes a TCP connection with destination server prior to sending the SIP Options message. This is OK for us. But when we try to use "Message Routing" (from what I understood this is generally advisable for SIP traffic) for TCP monitoring, TCP connection is not established before OPTIONS message is sent and this is not acceptable by our SIP servers. So I have few questions: Is it even possible to use F5 BIG-IP TLM VE as SIP LB for SIP resources operating in Kubernetes cluster (for both UDP and TCP) or the ONLY option is to use F5 BIG-IP Next Service Proxy Kubernetes (SPK) for SIP traffic? Is there a way to somehow force F5 that does Monitoring usin Message Routing mode to open TCP connection prior to sending SIP requests? Due to UDP problem above (that probably is solvable only if SPK version is used) is some way for F5 to do the UDP-2-TCP conversion of SIP traffic? Kind Regards, Zvonimir
F5 kubernetes f5 controller failing to compose 'poolMemberAddrs' and failing to generate F5 objects
Hi - I set this up: http://clouddocs.f5.com/products/connectors/k8s-bigip-ctlr/v1.0/ and getting errors after uploading my configmap and an applicable service: 2017/04/28 23:43:30 [INFO] File "/app/python/_f5.py", line 393, in _create_ltm_config_kubernetes 2017/04/28 23:43:30 [INFO] for node in backend['poolMemberAddrs']: 2017/04/28 23:43:30 [INFO] TypeError: 'NoneType' object is not iterable The config file generated by /app/bin/k8s-bigip-ctlr does not populate "poolMemberAddrs" so the python f5 handler /app/python/bigipconfigdriver.py is crashing since it cannot figure out the nodeport targets: /app cat /tmp/k8s-bigip-ctlr.config281602422/config.json {"bigip":{"username":"xxxxxxxxx","password":"yyyyyyyy","url":";:["k8s"]},"global":{"log-level":"INFO","verify-interval":30},"services":[{"virtualServer":{"backend":{"serviceName":"av-service","servicePort":30000,"poolMemberPort":0,"poolMemberAddrs":null},"frontend":{"virtualServerName":"default_av-service","partition":"k8s","balance":"round-robin","mode":"http","virtualAddress":{"bindAddr":"1.2.3.4","port":80},"iappPoolMemberTable":{"name":"","columns":null}}}}]}/app I ran out of anything helpful with debug statements or documentation about the closed source go binary... io:$ kubectl get services/av-service -o wide NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR av-service 10.25.104.158 80:30000/TCP 2h app=av io:$ kubectl get nodes -o jsonpath='{.items[*].status.addresses[?(@.type=="InternalIP")].address}' 10.25.82.193 10.25.82.65 10.25.83.54 Is this the right place to ask about what possible reasons the controller is crashing here?
An example of an AS3 Rest API call to create a GSLB configuration on BIG-IP.
Hi everyone, Below you can find an example of an AS3 Rest API call that creates a simple GSLB configuration on BIG-IP devices. The main purpose of this article is to share this configuration with others. Of course, on different sites (github, etc) you can find different bits of data, but I think this example will be useful, because it contains all the necessary information about how to create different GSLB objects at the same time, such as: Data Centers (DCs), Servers, Virtual Servers (VSs), Wide IPs, pools and more over. { "class": "AS3", "declaration": { "class": "ADC", "schemaVersion": "3.21.0", "id": "GSLB_test", "Common": { "class": "Tenant", "Shared": { "class": "Application", "template": "shared", "DC1": { "class": "GSLB_Data_Center" }, "DC2": { "class": "GSLB_Data_Center" }, "device01": { "class": "GSLB_Server", "dataCenter": { "use": "DC1" }, "virtualServers": [ { "name": "/ocp/Shared/ingress_vs_1_443", "address": "A.B.C.D", "port": 443, "monitors": [ { "bigip": "/Common/custom_icmp_2" } ] } ], "devices": [ { "address": "A.B.C.D" } ] }, "device02": { "class": "GSLB_Server", "dataCenter": { "use": "DC2" }, "virtualServers": [ { "name": "/ocp2/Shared/ingress_vs_2_443", "address": "A.B.C.D", "port": 443, "monitors": [ { "bigip": "/Common/custom_icmp_2" } ] } ], "devices": [ { "address": "A.B.C.D" } ] }, "dns_listener": { "class": "Service_UDP", "virtualPort": 53, "virtualAddresses": [ "A.B.C.D" ], "profileUDP": { "use": "custom_udp" }, "profileDNS": { "use": "custom_dns" } }, "custom_dns": { "class": "DNS_Profile", "remark": "DNS Profile test", "parentProfile": { "bigip": "/Common/dns" } }, "custom_udp": { "class": "UDP_Profile", "datagramLoadBalancing": true }, "testpage_local": { "class": "GSLB_Domain", "domainName": "testpage.local", "resourceRecordType": "A", "pools": [ { "use": "testpage_pool" } ] }, "testpage_pool": { "class": "GSLB_Pool", "resourceRecordType": "A", "members": [ { "server": { "use": "/Common/Shared/device01" }, "virtualServer": "/ocp/Shared/ingress_vs_1_443" }, { "server": { "use": "/Common/Shared/device02" }, "virtualServer": "/ocp2/Shared/ingress_vs_2_443" } ] } } } } } P.S. The AS3 scheme guide was very helpful: https://clouddocs.f5.com/products/extensions/f5-appsvcs-extension/latest/refguide/schema-reference.htmlF5 Kubernetes Container Integration
Two problems, finding docs to setup f5 kube-proxy. The doc is missing from this link - http://clouddocs.f5.com/products/asp/v1.0/tbd but I havn't gotten far enough to be able to test communication. The second is k8s-bigip-ctlr is not writing VIP or pool updates. I have k8s-bigip-ctlr and asp running. $ kubectl get pods --namespace kube-system -o wide NAME READY STATUS RESTARTS AGE IP NODE f5-asp-1d61j 1/1 Running 0 57m 10.20.30.168 ranchernode2.lax.verifi.com f5-asp-9wmbw 1/1 Running 0 57m 10.20.30.162 ranchernode1.lax.verifi.com heapster-818085469-4bnsg 1/1 Running 7 25d 10.42.228.59 ranchernode1.lax.verifi.com k8s-bigip-ctlr-deployment-1527378375-d1p8v 1/1 Running 0 41m 10.42.68.136 ranchernode2.lax.verifi.com kube-dns-1208858260-ppgc0 4/4 Running 8 25d 10.42.26.16 ranchernode1.lax.verifi.com kubernetes-dashboard-2492700511-r20rw 1/1 Running 6 25d 10.42.29.28 ranchernode1.lax.verifi.com monitoring-grafana-832403127-cq197 1/1 Running 7 25d 10.42.240.16 ranchernode1.lax.verifi.com monitoring-influxdb-2441835288-p0sg1 1/1 Running 5 25d 10.42.86.70 ranchernode1.lax.verifi.com tiller-deploy-3991468440-1x80g 1/1 Running 6 25d 10.42.6.76 ranchernode1.lax.verifi.com I have tried with k8s-bigip-ctlr 1.0.0 (Latest), which fails with different errors. Creating VIP with bigip-virtual-server_v0.1.0.json 2017/06/27 22:50:13 [WARNING] Could not get config for ConfigMap: k8s.vs - minLength must be of an integer Creating Pool with bigip-virtual-server_v0.1.0.json 2017/06/27 22:46:45 [WARNING] Could not get config for ConfigMap: k8s.pool - format must be a valid format . So I tired 1.1.0-beta.1 and it does produce something in the logs like its working but doesn't write any changes to the F5. (using f5schemadb bigip-virtual-server_v0.1.3.json) Here using f5schemadb://bigip-virtual-server_v0.1.3.json with 1.1.0-beta.1 seems get the farthest. 2017/06/27 22:58:19 [DEBUG] Delegating type *v1.ConfigMap to virtual server processors 2017/06/27 22:58:19 [DEBUG] Process ConfigMap watch - change type: Add name: hello-vs namespace: default 2017/06/27 22:58:19 [DEBUG] Add watch of namespace default and resource services, store exists:true 2017/06/27 22:58:19 [DEBUG] Looking for service "hello" in namespace "default" as specified by ConfigMap "hello-vs". 2017/06/27 22:58:19 [DEBUG] Requested service backend {ServiceName:hello ServicePort:80 Namespace:default} not of NodePort type 2017/06/27 22:58:19 [DEBUG] Updating ConfigMap {ServiceName:hello ServicePort:80 Namespace:default} annotation - status.virtual-server.f5.com/ip: 10.20.28.70 2017/06/27 22:58:19 [DEBUG] ConfigWriter (0xc42039b3b0) writing section name services 2017/06/27 22:58:19 [DEBUG] ConfigWriter (0xc42039b3b0) successfully wrote section (services) 2017/06/27 22:58:19 [INFO] Wrote 0 Virtual Server configs 2017/06/27 22:58:19 [DEBUG] Services: [] 2017/06/27 22:58:19 [DEBUG] Delegating type *v1.ConfigMap to virtual server processors 2017/06/27 22:58:19 [DEBUG] Process ConfigMap watch - change type: Update name: hello-vs namespace: default 2017/06/27 22:58:19 [DEBUG] Add watch of namespace default and resource services, store exists:true 2017/06/27 22:58:19 [DEBUG] Looking for service "hello" in namespace "default" as specified by ConfigMap "hello-vs". 2017/06/27 22:58:19 [DEBUG] Requested service backend {ServiceName:hello ServicePort:80 Namespace:default} not of NodePort type 2017/06/27 22:58:19 [DEBUG] ConfigWriter (0xc42039b3b0) writing section name services 2017/06/27 22:58:19 [DEBUG] ConfigWriter (0xc42039b3b0) successfully wrote section (services) 2017/06/27 22:58:19 [INFO] Wrote 0 Virtual Server configs 2017/06/27 22:58:19 [DEBUG] Services: [] Config Map kind: ConfigMap apiVersion: v1 metadata: name: hello-vs namespace: default labels: f5type: virtual-server data: schema: "f5schemadb://bigip-virtual-server_v0.1.3.json" data: |- { "virtualServer": { "frontend": { "balance": "round-robin", "mode": "http", "partition": "kubernetes", "virtualAddress": { "bindAddr": "10.20.28.70", "port": 443 } }, "backend": { "serviceName": "hello", "servicePort": 80 } } }How to define tcp_half_open monitor with k8s-bigip-ctlr
Hello, how I can define backend health monitor to be tcp_half_open through k8s-bigip-ctlr? currently backend configuration have a key "healthMonitors" but it take values only TCP, UDP or HTTP, I even tried to change health monitor manually - but it is reverted by controller pod. any ideas? https://clouddocs.f5.com/products/connectors/k8s-bigip-ctlr/v1.5/#backend I tried with: { "virtualServer": { "backend": { "servicePort": 8080, "serviceName": "my-service", "healthMonitors": [{"protocol": "tcp_half_open"}] }, "frontend": { "virtualAddress": { "port": 9090, "bindAddr": "123.44.11.11" }, "partition": "k8s", "balance": "least-connections-member", "mode": "tcp" } } } but getting obvious error: 2019/10/31 19:54:57 [WARNING] Could not get config for ConfigMap: f5.vs - configMap is not valid, errors: ["virtualServer.backend.healthMonitors.0.protocol: virtualServer.backend.healthMonitors.0.protocol must be one of the following: \"http\", \"tcp\", \"udp\""]
