Upgrade Path for BIG-IP 2000 – Out of Warranty and EoRMA Status
Hi Team, We are currently running BIG-IP LTM and AFM version 14.1.4.6 on a BIG-IP 2000 appliance. While reviewing our setup in iHealth, we received the following message: "Your hardware reached its End of Return to Manufacturing (EoRMA) date on April 1, 2025. Support options may be limited and an upgrade is recommended." As the platform is no longer under warranty and we’re running an older software version, we are planning to upgrade to the latest supported version on this hardware. Based on the compatibility matrix, it appears that BIG-IP 15.1.10.x is the latest version supported on the 2000 series. We understand that 15.1.x will reach End of Technical Support in December 2025, and we plan to use this upgrade as a short-term solution while we evaluate options for hardware replacement. Our questions are: Since we are out of warranty and do not currently have an active support contract, can we still upgrade to 15.1.10.x? If the device doesn’t have internet access, will collecting the license dossier and uploading it to the F5 licensing portal allow us to reactivate the existing license? Are there any limitations in upgrading or re-licensing in this scenario that we should be aware of? Any guidance or confirmation would be appreciated.SSH Proxy Problem: Real Server Auth
Hi, while playing around with the SSH proxy feature, I'm encountering issues with the validation of the Real Server Auth key. I've configured the profile as described in https://support.f5.com/kb/en-us/products/big-ip-afm/manuals/product/network-firewall-policies-implementations-12-1-0/13.html. Unfortunately I got an error message in /var/log/sshplugin: err : SSHPLUGIN: sshplugin_2|SSHPlugin|ssh_setup_serverside|Core|the backend ssh server does not have a public key that matches the configuration! (0) Erroring out of this connection. I've checked and doublechecked the host key using ssh-keyscan and copied the key string into the field "Real Server Auth". The format of the key looks exactly like the one from the manual, except that my key is a one-liner instead of the block view in the manual. The manual shows the key in block view AAAAB3NzaC1yc2EAAAADAQABAAABAQCziS6yavPpFuRjLP9hzRiEBcVgLDynoW qNMuwCrOREkSiDqWqFRrydFCGy6Z1WwwJuDMIw5h3sIuqtOo78zd6pBabXpj0Q LUyLtGx80Oe3vInpwxvG2/YX9KaGjofkasZJ+tOqoOe5QscnUYr7Iw6CEuo2dB VIZyL/o1IyTvDfL8+yXO4vPzadmL0gvV1F56feRVsCF0HUrhWwdrQ6CpIpX6ac sY0HayrhOGPmVF4qRz7fLySHJ5XQz5IKXJRNHJEbXx2tiV1TuQlhz8gOMqMp2I iSqyKDcUTk2Oy0fPYkNAWPlifq7GplYkit85EL5UCgtHf595rqibOQJWFAAzHF while mine looks like AAAAB3NzaC1yc2EAAAADAQABAAABAQCziS6yavPpFuRjLP9hzRiEBcVgLDynoWqNMuwCrOREkSiDqWqFRrydFCGy6Z1WwwJuDMIw5h3sIuqtOo78zd6pBabXpj0QLUyLtGx80Oe3vInpwxvG2/YX9KaGjofkasZJ+tOqoOe5QscnUYr7Iw6CEuo2dBVIZyL/o1IyTvDfL8+yXO4vPzadmL0gvV1F56feRVsCF0HUrhWwdrQ6CpIpX6acsY0HayrhOGPmVF4qRz7fLySHJ5XQz5IKXJRNHJEbXx2tiV1TuQlhz8gOMqMp2IiSqyKDcUTk2Oy0fPYkNAWPlifq7GplYkit85EL5UCgtHf595rqibOQJWFAAzHF Hopefully this doesn't make a difference. I even don't know how to turn on debug logging for sshplugin. Maybe this would help. Any ideas? Greets, svs
7. SYN Cookie: Troubleshooting Stats
Introduction In this article I will explain what SYN Cookie stats you can consult and their meaning. There are more complex stats than the explained in this article but they are intended for helping F5 engineers when cases become complex. SYN Cookie stats First at all, in order to troubleshoot SYN Cookie you need to know how you can check SYN Cookie stats easily and understand what you are reading. The easiest way to see these stats in a device running LTM module based SYN Cookie is by running below command and focusing in SYN Cookies section of the output: # tmsh show ltm virtual <virtual> SYN Cookies Status full-hardware Hardware SYN Cookie Instances 6 Software SYN Cookie Instances 0 Current SYN Cache 2.0K SYN Cache Overflow 24 Total Software 4.3M Total Software Accepted 0 Total Software Rejected 0 Total Hardware 21.7M Total Hardware Accepted 3 Let’s go through each field to know what they means: Status: [full-software/full-hardware|not-activated]. This value describes what type of SYN Cookie mode has been activated, software or hardware. Once an attack has finished it is normal that LTM takes some time to deactivate SYN Cookie mode after attack traffic stops, see the second article in this series (SYN Cookie Operation). Note that we do not want SYN Cookie entering in an activating/deactivating loop in case TCP SYN packets per second reaching device are near to the configured SYN cache threshold. So delay of 30-60 seconds before SYN Cookie being deactivated is in normal range. Hardware SYN Cookie Instances: How many TMMs are under Hardware SYN Cookie mode. Software SYN Cookie Instances: How many TMMs are under Software SYN Cookie mode. It indicates if software is currently generating SYN cookies. Current SYN Cache: Indicates how many embryonic connections are handled by BIG-IP (refreshed every 2 seconds). SYN cache is always counting embryonic connections, regardless if SYN Cookie is activated or not. So even if SYN Cookies is completely turned off, we still have embryonic flows that have not been promoted to full flows yet (SYNs which has not completed 3WHS), this means that cache counter will still be used regularly, so is normal having a value different than 0. Disabling SYN Cookie will not avoid this counter increase but thresholds will not be taken into account. Since SYN cache is no longer a cache, as explained in prior articles, and the stat is merely a counter of embryonic flows, it does not consume memory resources. As the embryonic flows are promoted or time out, this value will decrease. SYN Cache Overflow (per TMM): It is incremented whenever the SYN cache threshold is exceeded and SYN cookies need to be generated. It increments in one per each TMM instance and this value is a counter that only increases, so we can consult the value to know how many times SYN Cookie has been activated since last stats reset, remember, per TMM. Total Software: Number of challenges generated by Software. This is increased regardless if client sends a response, does not send any response or the response is not correct. Total Software Accepted: Number of TCP handshakes that were correctly handled with clients. Total Software Rejected: Number of wrong responses to challenges. Remember that ALL rejected SYN cookies are in software, there is no hardware rejected. Total Hardware: Number of challenges generated by Hardware. Total Hardware Accepted: Number of TCP handshakes that were correctly handled with clients. In case you have configured AFM based SYN Cookie then you can use two easy sources of information, the already explained LTM command above, but also you can check stats for TCP half open DoS vector, at device or virtual server context, as you would do with any other DoS vector. Let’s check the most important fields at device context. # tmsh show security dos device-config | grep -A 40 half Statistics Type Count Status Ready Attack Detected 1 *Attacked Dst Detected 0 *Bad Actor Attack Detected 0 Aggregate Attack Detected 1 Attack Count 2 *Stats 1h Samples 0 Stats 408 *Stats Rate 408 Stats 1m 104 Stats 1h 0 Drops 1063 *Drops Rate 1063 Drops 1m 187 Drops 1h 4 *Int Drops 0 *Int Drops Rate 0 *Int Drops 1m 0 *Int Drops 1h 0 Status: This field confirm if this specific DoS vector is ready to detect TCP SYN flood attacks. You have to take into account that you could decide to configure this DoS vector with Auto-threshold, in which case AFM would be in charge of deciding the best threshold. Note that by enabling auto-threshold AFM would need some time to learn the traffic pattern of your environment, until it has enough information to create a correct threshold you will not see this field as Ready but as Learning. If vector is configured manually you always see it as Ready. Attack detected: It informs you if currently there is an ongoing attack and detected by AFM. Aggregate attack detected: Since DoS stats shown above are for device context the detected attack is aggregate. Attack Count: Gives information about how many attacks have been detected since stats were reset last time. It does not decrease. Stats: Number of embryonic connections at this current second. Remember that since this is a snapshot, the counter could go increase or decrease. This is different to other DoS vectors where it only increases. Stats 1m: Average of the Stats in the last minute. This is the average number of embryonic connections that AFM has seen when taking sampling every 1s. Stats 1h: Average of the Stats 1m in the last hour. Drops: It counts the number of wrong ACKs received. In other words this is the current snapshot of: Number of SYN Cookies – Number of validated ACKs received Drops 1m: Average of the Drops in the last minute Drops 1h: Average of the Drops 1m in the last hour I have added an asterisk to some fields to point to values that has not real meaning for SYN Cookie DoS vector, or information is not really useful from my perspective, but they are included to have coherence with other DoS attack stats information. Note: Detection logic for this vector is not based on the Stats 1m, as other DoS vectors, instead it is based on the current number of embryonic connections, that is, value seen in Stats counter. Interpreting stats Once you know what each important stat mean I will give some advises when you interpret SYN Cookie stats. You should know some of them already if you have read all articles in this series: Hardware can offload TMM for validation, so you can see a number of software generated SYN Cookies much bigger than software validated SYN Cookie since software generates cookies that are validated by hardware as well. Since it is possible that a software generated SYN Cookie be accepted by hardware and vice versa, hardware generated SYN Cookies be accepted by software, you could think that value for Total Software is the same than the result of adding the Total Software Accepted plus Total Software rejected. But that is NOT true since it can be possible that a SYN Cookie sent by BIG-IP does not have a response (ACK), quite typical during a SYN flood attack indeed. In this case nothing adds up because generated SYN Cookie was not accepted nor rejected. Remember that SYN cookie’s responses discarded by hardware will be rejected by software, so all rejects are in software. This is why there is not counter for Total Hardware rejected. This means that Total Software Rejected can be increase by Total Hardware. When there is an attack vectors definition that match this attack will be increased. So, for example, during a TCP SYN flood attack you will see that Stats increases for TCP half open and TCP SYN flood (maybe others like low TTL,... as well). Also Stats will increase in all contexts (device and virtual server). The first vector in an specific context whose limit is exceeded it will start to mitigate. This is important because in cases where TCP SYN flood DoS vector has a lower threshold than TCP half open DoS vector, you will notice that traffic is dropped but you did not expect this behavior. Check article 5 for more information. Example In this part you will learn what stats changes you should expect to see when a TCP SYN flood attack is detected and SYN Cookie is activated, so it starts to mitigate the attack. Let’s interpret below stats: During attack Status full-hardware Hardware SYN Cookie Instances 6 Software SYN Cookie Instances 0 Current SYN Cache 1 SYN Cache Overflow 24 Total Software 10.1K Total Software Accepted 0 Total Software Rejected 0 Total Hardware 165.1M Total Hardware Accepted 3 After attack Status not-activated Hardware SYN Cookie Instances 0 Software SYN Cookie Instances 0 Current SYN Cache 15 SYN Cache Overflow 24 Total Software 10.1K Total Software Accepted 0 Total Software Rejected 0 Total Hardware 171.0M Total Hardware Accepted 3 Before the attack, before SYN Cookie is activated, you will see Current SYN Cache stats starts to increase quickly since TCP SYN packets are causing an increment of embryonic connections. Once SYN Cache threshold has been reached in a TMM then SYN Cache Overflow will increase attending to the number of TMMs that detected the attack. In above example you see 24 because this is a 6 CPUs device and 4 TCP SYN Flood attacks were detected by SYN Cookie. Remember this counter will never decrease unless we reset stats. During attack SYN cookie is activated so Current SYN Cache will start to decrease until reaching 0 because SYN Cookie Agent starts to handle TCP 3WHS, in other words, TCP stack stops to receive TCP SYN packets. We can check how many TMMs have SYN cookie activated currently looking at Hardware/Software SYN Cookie Instances counter. Note this match with what I explained for SYN Cache Overflow value. Legitimate connections are counted under Total Hardware/Software Accepted. In this case, we can see that although several millions of TCP SYN packets reached the device only 3 TCP 3WHS were correctly carried out. As you can see this device has Hardware SYN Cookie configured and working, but you can read that software also generates SYN Cookie challenges (Total Software). As commented in article number six of these series, this is expected and can be due to collisions or due to validations of first challenges when SYN Cookie is activated and TMMs handles TCP SYN packets until it enables hardware to do it. This does not affect device performance. In order to change from ‘Status full-hardware’ to ‘Status not-activated’ both HSBs must exit from SYN Cookie mode. Conclusion Now you know how to interpret stats, so you know can deduce information about the past and the present status of your device related to SYN Cookie. In next two article I will end up this series and this troubleshooting part talking about traffic captures and meaning of logs.2. SYN Cookie: Operation
Introduction As concluded in the last article, in order to avoid allocating space for TCB, the attacked device needs to reject TCP SYN packets sent by clients. In this article I will explain how a system can do this without causing service disruption, and more specifically I will explain how this work in BIG-IP. Implementation Since attacked device need to alter default TCP 3WHS behaviour the best option is implementing SYN Cookie countermeasure in an intermediate device, so you offload your servers from this task. In this way if connection is not legitimate then it is just never forwarded to the backend server and therefore it will not waste any kind of extra resourcess. Since BIG-IP is already in charge of handling application traffic directed towards servers it is the best place to implement SYN Cookie. What BIG-IP does is adding an extra layer, which we can call SYN Cookie Agent, that basically implement a stateless buffer between client and BIG-IP TCP stack. This agent is in charge of handling client’s TCP SYN packets by modifying slightly the standard behaviour of TCP 3WHS, this modification comes in two flavours depending on what type of routing role BIG-IP is playing between client and server. Symmetric routing This is the typical situation. In this environment BIG-IP is sited in the middle of the TCP conversation and all the traffic from/to server pass through it. The SYN Cookie operation in this case is briefly described below: Client sends a TCP SYN packet to BIG-IP. BIG-IP uses a stateless buffer for answering each client SYN with a SYN/ACK. BIG-IP generates the 32 bits sequence number which will be included in SYN/ACK packet sent to the client. BIG-IP encodes essential and mandatory information of the connection in 24 bits. BIG-IP hashes the previous encoded information. BIG-IP also encodes other values like MSS in the remaining bits. BIG-IP generates the SYN/ACK response packet and includes the hash and other encoded information as sequence number for the packet. This is the so called SYN Cookie. BIG-IP sends SYN/ACK to client. BIG-IP discards the SYN from the stateless buffer, in other words, BIG-IP removes all information related to this TCP connection. At this point no memory has been allocated for TCP connection (TCB). Client sends correct ACK to BIG-IP acknowledging BIG-IP’s sequence number. BIG-IP validates ACK. BIG-IP subtracts one to this ACK. BIG-IP runs the hash function using connection information as input (see point 3-b above). Then it compares it with the hash provided in the ACK, if they match then it means client sent a correct SYN Cookie response and that client is legitimate. BIG-IP uses connection information in the ACK TCP/IP headers to create an entry in connflow. At this time is when BIG-IP builds TCB entry, so it's the first time BIG-IP uses memory to save connection information. BIG-IP increase related SYN Cookie stats. BIG-IP starts and complete a TCP 3WHS with backend server on behalf of the client since we are sure that client is legitimate. Now traffic for the connection is proxied by BIG-IP as usual attending to configured L4 profile in the virtual server. *If ACK packet received by BIG-IP is a spurious ACK then BIG-IP will discard it and no entry will be created in connection table. Since attackers will never send a correct response to a SYN/ACK then you can be sure that TCB entries are never created for them. Only legitimate clients will use BIG-IP resources as shown in diagram: Fig4. TCP SYN Flood attack with SYN Cookie countermeasure Asymmetric routing In an asymmetric environment, also called nPath or DSR, you face a different problem because Big-IP cannot establish a direct TCP 3WHS with the server (point 11 in last section). As you can see in below diagram SYN/ACK packet sent from server to client would not traverse Big-IP, so method used for symmetric routing cannot be used in this case. You need another way to trust in clients and discard them as attackers, so clients then can complete TCP 3WHS directly with the server. Fig5. TCP state diagram section for asymmetric routing + FastL4 In order to circumvent this problem BIG-IP takes the advantage of the fact that applications will try to start a second TCP connection if a first TCP 3WHS is RST by the server. What BIG-IP does in asymmetric routing environments is completing the TCP three way handshake with client, issuing SYN Cookie as I described for symmetric routing, and once BIG-IP confirms that client is trustworthy it will add its IP to SYN Cookie Whitelist, it will send a RST to the client and it will close the TCP connection. At this point client will try to start a new connection and this time BIG-IP will let the client talk directly to the server as it would do under normal circumstances (see diagram above). The process is briefly described below: Client sends a TCP SYN packet to BIG-IP. BIG-IP generates the sequence number as explained for symmetric routing. BIG-IP generates the SYN/ACK response packet and includes the calculated sequence number for the packet. BIG-IP sends SYN/ACK to client and then remove all information related to this TCP connection. At this point no memory has been allocated for TCP connection (TCB). Client sends correct ACK to BIG-IP acknowledging sequence number. BIG-IP validates ACK. BIG-IP substract one to this ACK so it can decode needed information and check hash. BIG-IP increase related SYN Cookie stats. BIG-IP sends a RST to client and discards TCP connection. Big-IP adds client’s IP to Whitelist. Clients starts a new TCP connection. BIG-IP lets client to start this new TCP connection directly to the server since it knows that server is legitimate. *If ACK packet received by BIG-IP is a spurious ACK then BIG-IP will discard it and no entry will be created in connection table. Fig6. TCP 3WHS flow in asymmetric routing DSR whitelist has some important characteristics you need to take into account: By default IP is added 30 seconds to the whitelist (DB Key tm.flowstate.timeout). Client’s IP is added to whitelist for 30 seconds but only if there is no traffic from this client, if BIG-IP have already seen ACK from this client then its IP is removed from whitelist since connection has been already established between client and server. Whitelist is common to all virtual servers, so if a client is whitelisted it will be for all applications. Whitelist it is not mirrored. Whitelist it is not shared among blades. SYN Cookie Challenge When a TCP connection is initiated the TCP SYN packet sent by the client specify certain values that define the connection, some of these values are mandatory, like source and destination IP and port, otherwise you would not be able to identify the correct connection when packets arrive. Some other values are optional and they are used to improve performance, like TCP options or WAN optimizations. Under normal circumstances, upon TCP SYN reception the system creates a TCB entry where all this information is saved. BIG-IP will use this information to set up the TCP connection with the backend server. The problem we face is that when SYN Cookie is in play device does not create the TCB entry, only a limited piece of connection information is collected, so the information that BIG-IP has about the TCP connection is limited. Remember that SYN Cookie is not handled by TCP stack but by the stateless SYN Cookie Agent, so we cannot save the connection details. What BIG-IP does instead is encoding key information in 24 bits, this left only some bits for the rest of data. While there is no room for values like Window Scale information, however we have space for other values, for example 3 bits reserved for MSS value. This limits MSS possible values to 8, the most commonly used values, and the one chosen will the nearest to the original MSS value. However, note that MSS limitation has a workaround through configuration in FastL4 profile that it can help in some cases. The option SYN Cookie MSS in this profile specifies a value that overrides the SYN cookie maximum segment size (MSS) value in the SYN-ACK packet that is returned to the client. Valid values are 0, and values from 256 through 9162. The default is 0, which means no override. You might use this option if backend servers use a different MSS value for SYN cookies than the BIG-IP system does. If this is not enough for some customers, BIG-IP overcomes this problem by using TCP timestamp space to save extra information about TCP connection, in other words we create a second extra SYN Cookie. This space is used to record the client and server side Window Scale values, or the SACK info which are then made available to the TCP stack via this cookie if the connection is accepted. Note that TCP connection will only be accepted if both SYN Cookies (standard and Timestamp) are correct. So all this means that if the client starts a connection with TCP TS set then Big-IP will have more space to encode information and hence the performance will be improved when under TCP SYN flood attack. NOTE: SYN Cookie Timestamp extension (software or hardware) only work for standard virtual servers currently. FastL4 virtual servers only supports MMS option in SYN Cookie mode. Deactivating In order to deactivate SYN Cookie three conditions need to be met, these conditions are the same for LTM or AFM SYN Cookie, although stat names can vary: Embryonic connections must be below the configured SYN Cookie threshold. The ratio of invalid packets has not been exceeded. We define this ratio as a percentage saved in the DB key dos.syncookiedeactivate which by default is the 50% of the configured threshold (do not change this value if it is not requested by a F5 engineer). The second condition above has to be met three consecutive times as a method for avoiding SYN Cookie activation/deactivation flapping. BIG-IP will receive information about the number of invalid packets every 7 seconds, so this means that at least 21 seconds are required to deactivate SYN Cookie assuming that the second condition is constantly met. Note the difference between embryonic connections and invalid connections in the above conditions: Embryonic connections are connections pending to be confirmed with a last ACK. Invalid connections include any connection that did not finish the TCP 3WHS with a correct last ACK, that is, connections for which BIG-IP did not receive the last ACK, or for which BIG-IP did receive a wrong ACK. Finally remember that for Hardware SYN Cookie all HSBs connected to TMM must exit from SYN Cookie mode in order to consider that the TMM exit SYN Cookie. And also all TMMs must exit from SYN Cookie mode in order to deactivate SYN Cookie completely for that context. Conclusion Now you know how SYN Cookie works under the hoods. In next article I will describe when SYN Cookie is activated and I will give specific details of BIG-IP SYN Cookie operation. Note that limitations related to SYN Cookie commented in this article are not caused specifically by BIG-IP SYN Cookie implementation but by SYN Cookie standard itself. In fact, F5 Networks implements some improvements on SYN Cookie to get a better performance and provide some extra features.Sizing for HW and SW based
I am looking for a data for dimensioning for r5800 / 6000 etc where I am deploying DNS+PEM+AFM+URL Filtering + some iRules on ONE rSeries The same witch I am looking for is for VE deployment Where I can find data about such figures I can only find for DNS QPS, but for the rest of the modules can't Are there any exact numbers? How can I combine and calculate this module and see if feet into rSeries and VE HP??
