Forum Discussion
NacreousF5 BIGIP VE Traffic exceeds %75
I am seeing some strange behaviour from my virtual edition running under VMware ESXi. This device is not even in production and yet I am getting warning messages like the following. We are just doing basic single user testing yet this is happening.
Bandwidth utilization is 1198 Mbps, exceeded 75% of Licensed 1000 Mbps
Bandwidth utilization is 1146 Mbps, exceeded 75% of Licensed 1000 Mbps
Bandwidth utilization is 1208 Mbps, exceeded 75% of Licensed 1000 Mbps
In this case the VLANS in ESXi are assigned to each interface so in the f5 configuration they are untagged traffic on each interface 1.1 through 1.4. As per SOL15831, a check of the shaper which controls maximum bandwith shows
root@pilot:Active:Standalone] config tmctl -d blade tmm/if_shaper
shaper_tid ingress_max ingress_avg ingress_red ingress_drops egress_drops
---------- ----------- ----------- ----------- ------------- ------------
1 1929688 99 62999 182560808 0
0 1878392 99 102985 158795922 0
Which clearly indicates we are heavily exceeding bandwidth. A view on the interfaces shows the following.
[root@pilot:Active:Standalone] config tmsh show net interface
------------------------------------------------------------------
Net::Interface
Name Status Bits Bits Pkts Pkts Drops Errs Media
In Out In Out
------------------------------------------------------------------
1.1 up 1.9G 9.4G 1.4M 1.2M 0 0 none
1.2 up 18.0T 261.2M 1.4G 552.7K 0 0 none
1.3 up 1.7T 377.7M 114.6M 383.7K 0 0 none
1.4 up 5.3G 23.9M 4.7M 49.4K 0 0 none
If you have any suggestions on VMware configuration or other diagnostic steps it would be appreciated.
9 Replies
Kevin_Davies_40Interface 1.2 is an internal network with servers on it so I dont understand how the F5 would be receiving so much traffic from it.- alex100
Cirrostratus
This is interesting. Looks like "traffic" is getting sucked into the black hole and not passed to any other interface. Anything interesting in tcpdump for int 1.2?
- Kevin_Davies_40
We have discovered that Promiscuous mode had been set to Accept on all vSwitches under ESXi. This effectively means they act as hubs and not switches. They copy any traffic they see to all members of the port group. The F5 was receiving traffic not only for it but every single server on any VLAN's to which it was connected. Every other server was seeing the traffic as well. It would have been placing quite a bit of network load on customer machines.
- Kevin_Davies_40The next change window Promiscuous mode was set to Reject and the F5 problem dissappeared. They re-enabled it some time later concerned it it had caused issues but this turned out not to be the case. It was turned off again. Why is unsolicited traffic arriving at the F5 being treated as licensed traffic? It's through traffic that should be considered by the licensing, not anything arriving at the interface which appeared to be the case. I raised this as an issue with F5. More diagnosis information was provided and it was escalated.
- Kevin_Davies_40
The issue was escalated to the product engineering team. A week later they came back with the following. It is posted here for your information.
1) The the issue appears to be due to the bug 434713 (VADC - non-load-balanced traffic (broadcast, internal, configsync) counts against licensed limit, causes license exceeded messages) This issue has no workaround at this time. And will be fix in the next software release. No specific timeline.
2) At the same time vmware does not encourage customers to turn on promiscuous mode as per: KB1004099
3) The current method of calculating bandwidth is: SOL15831
It means that we calculate all traffic incoming/outgoing via the data plane interfaces against the licensed performance limit. This will change in the future version where only traffic that is meant as production traffic will be counted towards the limit.
There is no workaround at this time and the only solution that we can suggest is to avoid using the promiscuous mode on the vswitch.
chuffaker_11557Nimbostratus
We are experiencing this issue in 6 different clusters... It doesn't look like 12.0 resolved the issue. Do you know if there are plans to have it resolved in the 12.1 release?
Additionally, if the only workaround available at this time is to avoid promiscuous mode on the vSwitch, then we could no longer use MAC Masquerading. Right? If that is the case, then we really don't have a use for failover in a VE environment.
- Kevin_Davies_40You would have to raise case with F5 to get an answer on that.
- Not true. There is no requirement to use mac masquerading to get failover working properly. There is a gratuitous arp sent when the failover occurs so the switch should update. Mac Masquerading was developed to decrease traffic on the wire ( ARPs) when the failover occurs. With Mac Masquerading, you will need to insure that the switch won't complain if the mac address floats from virtual port to virtual port. I am not aware of any of my customers using mac masquerading.
RuchitHi,
I am facing a similar situation on BIG-IP 12.1.2 Build 1.0.271 Hotfix HF1 with 1G throughput license.
When i check the throughput (tmsh show sys performance throughput), it's not exceeding 200Mbps but the logs are showing up quite frequently.
Also, when i tried to see the drops (tmctl -d blade tmm/if_shaper) the ingress_drops counter is incrementing.
Please can anyone shed some light on the situation that i am facing?
Thanks in advance.
Regards,
Ruchit
