Forum Discussion

sstafford's avatar
sstafford
Icon for Nimbostratus rankNimbostratus
Mar 21, 2018

Warning, ICMP error limit reached.

Upgraded a VCMP guest to version 13.1.0.3 Build 0.0.5 yesterday morning, and am now seeing new log warnings. While that's expected, I'd still like to know what they mean. For instance a message similar to "warning tmm3[19717]: 01200015:4: Warning, ICMP error limit reached." has appeared 40 or so times in the last couple of errors, and my searches as to their meaning have been unfruitful. Anyone have an idea what's going on?

 

  • I have also seen this messages after 13.1.0.3 Upgrade last weekend (4000s Plattform) In addition I have the Message "Limiting icmp unreach response from 501 to 500 packets/sec for traffic-group /Common/traffic-group-local-only" every 2 Minutes.

     

    In my LAB i haven't seen this warning.

     

    Did you try a tcpdump to search for the ICMP Errors?

     

  • I'm not seeing the traffic-group messages at this point, just "error limit reached." No tcpdump yet either--there's over 150 nodes on this LTM, so there's a ton of icmp checks just from the monitors.

     

  • You could try to filter out icmp echo and echo reply messages:

    tcpdump -enni internal-if 'icmp[0] != 8 and icmp[0] != 0'
    The Advanced Tcpdump Article shows some hints to do this

    I am currently unable to follow up on my boxes.

  • I have the same experience, but on a VE running on a VMware host. I noticed these warnings in /var/log/ltm on TMOS version 13.1.0.6. Then I upgraded to 13.1.0.7 but nothing has changed. I also tried to run this two tcpdumps - without success (nothing appeared in dump):

     

    tcpdump -enni 0.0 'icmp[0] != 8 and icmp[0] != 0' (as Kai suggested)

     

    tcpdump -ni 0.0:nnnp -s0 -v icmp

     

     

    When I go to Statistics ›› Module Statistics : Traffic Summary : ICMP I can see that horrible amount of IPv4 ICMP Packets has been transmitted. When clearing statistics and refreshing second by second the number can increase even by 300-400.

     

     

    Any idea, what can cause this? I already stopped all my virtual machines that are located in the same networks as my VE... no clue...

     

  • Jerry_Lees_4280's avatar
    Jerry_Lees_4280
    Historic F5 Account

    Try searching through tcpdumps for ICMP packets with a TTL of zero. Per RFC 792 page 6 and 7 this is against RFC, and the BIG-IP logs this state with this error message.

     

    Searching through the dumps will likely reveal the offending device. The following command may help: tcpdump -nnvi 0.0:nnn -s0 -w /var/tmp/icmp_testing.pcap -C 100

     

  • Running v13.1.0.2 VCMP guests. After running the capture listed above

        (tcpdump -nnvi 0.0:nnn -s0 -w /var/tmp/icmp_testing.pcap -C 100) 
    

    the offenders appeared to be the self IPs used for HA/failover. The error in the capture was "158 Destination unreachable (Port Unreachable)". Changing the port lock down settings on the self IP cleared the errors for me.

    • JG's avatar
      JG
      Icon for Cumulonimbus rankCumulonimbus

      Thanks for sharing your experience.

       

  • svs's avatar
    svs
    Icon for Cirrostratus rankCirrostratus

    I'm aware, that this question is already 3 years old, but I'd like to share my two cents:

    https://support.f5.com/csp/article/K13151

    https://support.f5.com/csp/article/K14813 (tm.maxicmprate)

    https://support.f5.com/csp/article/K14358

    https://support.f5.com/csp/article/K15003

     

    The BIG-IP uses some basic DDoS/DoS features, which explain your messages. I don't know if your HA interfaces are dedicated between the devices or shared. But it may indicate erroneous traffic or a real attack.