F5 NGINX HTTP Request Header Rules: What’s Permitted and What’s Not
When managing web servers like F5 NGINX, it's crucial to understand the rules that govern HTTP headers—particularly which characters are allowed or disallowed in both header names and values. HTTP headers play a vital role in the communication between a client and server, carrying essential metadata such as content type, length, and caching policies. NGINX strictly enforces these rules to ensure compliance with HTTP/1.1, HTTP/2, and HTTP/3 protocols. Misconfigured headers or the use of improper characters can lead to a range of issues, including security vulnerabilities, degraded performance, or even rejected requests. One common security threat,HTTP Request Smuggling, exploits desynchronization between devices involved in handling the request, such as frontends, caching servers, load balancers, and web servers. These attacks rely on inconsistencies in how each device parses the request, enabling malicious actors to inject hidden or split requests. This article outlines the allowed and disallowed characters in HTTP request headers as enforced by NGINX, which I compiled throughcode review, test case review and manual testing for HTTP Smugglingon NGINX version "nginx/1.25.5 (nginx-plus-r32-p1)". If I missed something please let me know bycontacting the F5 SIRT. Allowed Characters Header Names Lowercase letters: (a-z) Uppercase letters: (A-Z) Allowed in HTTP/1.1 Disallowed in HTTP/2 and HTTP/3, which require lowercase. Digits: (0-9) Hyphen: (-) Underscore: (_)Allowed in all versions only ifunderscores_in_headers is enabled Colon: (:) Allowed only as a prefix inHTTP/2 and HTTP/3 pseudo headers. Header Values Printable ASCII characters: Characters from ! to ~, including digits, letters, punctuation, and symbols, are allowed. Space: (0x20) Allowed within the value. Horizontal Tab (HT): (0x09) Generally allowed within values but with exceptions in headers such as Content-Length and Host (details below). Disallowed Characters Header Names Control Characters: ASCII control characters (<= 0x20), including space and horizontal tab, are disallowed in header names. DEL (0x7f): The delete character is not allowed. Colon: (:) Disallowed in header names except forHTTP/2 and HTTP/3 pseudo headers. Uppercase Letters: (A-Z) Disallowed in HTTP/2 and HTTP/3 header names (must be lowercase). Special characters: Characters such as ()<>@,;:\"/[]?={} are implicitly disallowed as they don’t belong to any allowed category for header names. Header Values Null Character: (\0) Disallowed in header values. Line Feed (LF): (0x0A) Not allowed within the value and used only to terminate headers. Carriage Return (CR): (0x0D) Not allowed within the value, used for header termination. Control Characters (<= 0x20) with special conditions for Horizontal Tab (HT): Horizontal Tab (HT, 0x09): Disallowed in the Host header value in HTTP/1. Disallowed in the pseudo-header values in HTTP/2 and HTTP/3. Disallowed in the Content-Length header value in HTTP/1. Summary This list of allowed and disallowed characters provides an overview of how NGINX handles HTTP headers across different protocol versions. While HTTP/1.1 is somewhat lenient with uppercase letters and certain characters, HTTP/2 and HTTP/3 enforce stricter validation rules, particularly for header names. Understanding these restrictions ensures your server configuration remains compliant with protocol specifications and avoids potential issues with malformed headers.
SSL bridging without SSL proxy forward
Dear all, I would like to implement ssl brigding for SMTPS traffic in my organization. In my case, I would like the client devices to receive the certificate configured in clientssl profile. When adding the serverssl profile to the virtual server, I get an error: smtps configuration error: SSL forward-proxy must be enabled Configuring SSL forward-proxy is not a solution for me, because the clients do not accept SMTP server certificates. Is it possible to configure ssl bridging for SMTPS without configuring SSL forward-proxy or to configure SSL forward-proxy so that client device get the certificate defined in clientssl profile?
BIG-IP BGP Routing Protocol Configuration And Use Cases
Is the F5 BIG-IP a router? Yes! No! Wait what? Can the BIG-IP run a routing protocol? Yes. But should it be deployed as a core router? An edge router? Stay tuned. We'll explore these questions and more through a series of common use cases using BGP on the BIG-IP... And oddly I just realized how close in typing BGP and BIG-IP are, so hopefully my editors will keep me honest. (squirrel!) In part one we will explore therouting components on the BIG-IP and some basic configuration details to help you understand what the appliance is capable of. Please pay special attention to some of the gotchas along the way. Can I Haz BGP? Ok. So your BIG-IP comes with ZebOS in order to provide routing functionality, but what happens when you turn it on? What do you need to do to get routing updates in to the BGP process? And well does my licensing cover it? Starting with the last question… tmsh show /sys license | grep "Routing Bundle" The above command will help you determine if you’re going to be able to proceed, or be stymied at the bridge like the Black Knight in the Holy Grail. Fear not! There are many licensing options that already come with the routing bundle. Enabling Routing First and foremost, the routing protocol configuration is tied to the route-domain. What’s a route-domain? I’m so glad you asked! Route-domains are separate Layer 3 route tables within the BIG-IP. There is a concept of parent and child route domains, so while they’re similar to another routing concept you may be familiar with; VRF’s, they’re no t quite the same but in many ways they are. Just think of them this way for now. For this context we will just say they are. Therefore, you can enable routing protocols on the individual route-domains. Each route-domain can have it’s own set of routing protocols. Or run no routing protocols at all. By default the BIG-IP starts with just route-domain 0. And well because most router guys live on the cli, we’ll walk through the configuration examples that way on the BIG-IP. tmsh modify net route-domain 0 routing-protocol add { BGP } So great! Now we’re off and running BGP. So the world know’s we’re here right? Nope. Considering what you want to advertise. The most common advertisements sourced from the BIG-IP are the IP addresses for virtual servers. Now why would I want to do that? I can just put the BIG-IP on a large subnet and it will respond to ARP requests and send gratuitous ARPs (GARP). So that I can reach the virtual servers just fine. <rant> Author's opinion here: I consider this one of the worst BIG-IP implementation methods. Why? Well for starters, what if you want to expand the number of virtual servers on the BIG-IP? Well then you need to re-IP the network interfaces of all the devices (routers, firewalls, servers) in order to expand the subnet mask. Yuck! Don't even talk to me about secondary subnets. Second: ARP floods! Too many times I see issues where the BIG-IP has to send a flood of GARPs; and well the infrastructure, in an attempt to protect its control plane, filters/rate limits the number of incoming requests it will accept. So engineers are left to try and troubleshoot the case of the missing GARPs Third: Sometimes you need to migrate applications to maybe another BIG-IP appliance as it grew to big for the existing infrastructure. Having it tied to this interface just leads to confusion. I'm sure there's some corner cases where this is the best route. But I would say it's probably in the minority. </rant> I can hear you all now… “So what do you propose kind sir?” See? I can hear you... Treat the virtual servers as loopback interfaces. Then they’re not tied to a specific interface. To move them you just need to start advertising the /32 from another spot (Yes. You could statically route it too. I hear you out there wanting to show your routing chops.) But also, the only GARPs are those from the self-ip's This allows you to statically route of course the entire /24 to the BIG-IP’s self IP address, but also you can use one of them fancy routing protocols to announce the routes either individually or through a summarization. Announcing Routes Hear ye hear ye! I want the world to know about my virtual servers.*ahem* So quick little tangent on BIG-IP nomenclature. The virtual server does not get announced in the routing protocol. “Well then what does?” Eery mind reading isn't it? Remember from BIG-IP 101, a virtual server is an IP address and port combination and well, routing protocols don’t do well with carrying the port across our network. So what BIG-IP object is solely an IP address construct? The virtual-address! “Wait what?” Yeah… It’s a menu item I often forget is there too. But here’s where you let the BIG-IP know you want to advertise the virtual-address associated with the virtual server. But… but… but… you can have multiple virtual servers tied to a single IP address (http/https/etc.) and that’s where the choices for when to advertise come in. tmsh modify ltm virtual-address 10.99.99.100 route-advertisement all There are four states a virtual address can be in: Unknown, Enabled, Disabled and Offline. When the virtual address is in Unknown or Enabled state, its route will be added to the kernel routing table. When the virtual address is in Disabled or Offline state, its route will be removed if present and will not be added if not already present. But the best part is, you can use this to only advertise the route when the virtual server and it’s associated pool members are all up and functioning. In simple terms we call this route health injection. Based on the health of the application we will conditionally announce the route in to the routing protocol. At this point, if you’d followed me this far you’re probably asking what controls those conditions. I’ll let the K article expand on the options a bit. https://my.f5.com/manage/s/article/K15923612 “So what does BGP have to do with popcorn?” Popcorn? Ohhhhhhhhhhh….. kernel! I see what you did there! I’m talking about the operating system kernel silly. So when a virtual-address is in an unknown or enabled state and it is healthy, the route gets put in the kernel routing table. But that doesn’t get it in to the BGP process. Here is how the kernel (are we getting hungry?) routes are represented in the routing table with a 'K' This is where the fun begins! You guessed it! Route redistribution? Route redistribution! And well to take a step back I guess we need to get you to the ZebOS interface. To enter the router configuration cli from the bash command line, simply type imish. In a multi-route-domain configuration you would need to supply the route-domain number but in this case since we’re just using the 0 default we’re good. It’s a very similar interface to many vendor’s router and switch configuration so many of you CCIE’s should feel right at home. It even still lets you do a write memoryor wr memwithout having to create an alias. Clearly dating myself here.. I’m not going to get in to the full BGP configuration at this point but the simplest way to get the kernel routes in to the BGP process is simply going under the BGP process and redisitrubting the kernel routes. BUT WAIT! Thar be dragons in that configuration! First landmine and a note about kernel routes. If you manually configure a static route on the BIG-IP via tmsh or the tmui those will show up also as kernel routes Why is that concerning? Well an example is where engineers configure a static default route on the BIG-IP via tmsh. And well, when you redistribute kernel routes and that default route is now being advertised into BGP. Congrats! AND the BIG-IP is NOT your default gateway hilarity ensues. And by hilarity I mean the type of laugh that comes out as you're updating your resume. The lesson here is ALWAYS when doing route redistribution always use a route filter to ensure only your intended routes or IP range make it in to the routing protocol. This goes for your neighbor statements too. In both directions! You should control what routes come in and leave the device. Another way to have some disasterous consequences with BIG-IP routing is through summarization. If you are doing summarization, keep in mind that BGP advertises based on reachability to the networks it wants to advertise. In this case, BGP is receiving it in the form of kernel routes from tmm. But those are /32 addresses and lots of them! And you want to advertise a /23 summary route. But the lone virtual-address that is configured for route advertisement; and the only one your BGP process knows about within that range has a monitor that fails. The summary route will be withdrawn leaving all the /23 stranded. Be sure to configure all your virtual-addresses within that range for advertisement. Next: BGP Behavior In High Availability Configurations
VAPT or APT tools scan prevention
Hello When the security team starts Vulnerability Assessment and Penetration Testing (VAPT) or Application Security Testing (APT) on a web application, then it can go and test those web pages that only registered users can browse. Is there any way I can block this with Big-IP. Sorry if my question is silly.
High CPU utilization (100%).
I observed high CPU utilization (100%) on F5 device, resource provision ASM nominal. I checked the client-side throughput and server-side throughput both are normal but found management interface throughput is very high and what i noticed this is happening in same time period for last 30 days. What could be the reason for this spike. Many thanks in advanced for your time and consideration.
Unable to get Internet in server using SWG forward Proxy.
We are using SWG forward proxy. But we are unable to get internet in my Redhat Linux server. It showing unable to get local issuer certificate. The same certificate is working for Windows user PC. We have got the Sub CA certificate from our enterprise local CA. Any one could help to resolve the issue.Ridiculously Easy Bot Protection: How to Use BIG-IP APM to Streamline Bot Defense Implementation
Ever imagined how your Bot solution implementation would be with a standard entry page at your application side--a page that’s easily referred, with clear parameters, and structured customization options? In this article, we are exploring using F5 BIG-IP Access Policy Manager (BIG-IP APM) along side F5 Distributed Cloud Bot Defense (XC Bot Defense). Bot defense solutions' challenges Implementing bot defense solutions presents several challenges, each with unique considerations: Evolving Bot Tactics: Bot tactics constantly evolve, demanding adaptive detection methods to avoid both false positives (blocking legitimate users) and false negatives (allowing malicious bots through). Effective solutions must be highly flexible and responsive to these changes. Multi-Environment Integration: Bot defenses need to be deployed across diverse environments, including web, mobile, and APIs, adding layers of complexity to integration. Ensuring seamless protection across these platforms is critical. Balancing Security and Performance: Security measures must be balanced with performance to avoid degrading the user experience. A well-calibrated bot defense should secure the application without causing noticeable slowdowns or other disruptions for legitimate users. Data Privacy Compliance: Bot solutions often require extensive data collection, so adherence to data privacy laws is essential. Ensuring that bot defense practices align with regulatory standards helps avoid legal complications and maintains user trust. Resource Demands: Integrating bot defense with existing security stacks can be resource-intensive, both in terms of cost and skilled personnel. Proper configuration, monitoring, and maintenance require dedicated resources to ensure long-term effectiveness and efficiency. What F5 BIG-IP APM brings to the table? For teams working on bot defense solutions, several operational challenges can arise: Targeted Implementation Complexity: Identifying the correct application page for applying bot defense is often a complex process. Teams must ensure the solution targets the page containing the specific parameters they want to protect, which can be time-consuming and resource-intensive. Adaptation to Application Changes: Changes like upgrades or redesigns of the application page often require adjustments to bot defenses. These modifications can translate into significant resource commitments, as teams work to ensure the bot solution remains aligned with the new page structure. BIG-IP APM simplifies this process by making it easier to identify and target the correct page, reducing the time and resources needed for implementation. This allows technical and business resources to focus on more strategic priorities, such as fine-tuning bot defenses, optimizing protection, and enhancing application performance. Architecture and traffic flow In this section, let's explore how F5 XC Bot Defense and BIG-IP APM works together, let's list the prerequisites, F5 XC account with access to Bot Defense. APM licensed and provisioned. F5 BIG-IP min. v16.x for native connector integration. BIG-IP Self IP rechability to Internet to communicate with F5 XC, mainly to reach this domin (ibd-web.fastcache.net). Now, time to go quickly through our beloved TMM packet order. Due to the nature of BIG-IP APM Access events take precedence to the Bot enforcement, hence we will rely on simple iRule to apply Bot Defense on BIG-IP APM logon page. BIG-IP Bot Defense is responsible for inserting the JS and passing traffic from client to APM VS back and forth. BIG-IP APM responsible for logon page, MFA, API security or SSO integrations to manage client Access to the backend application. Solution Implementation Let's start now with our solution implementation, F5 Distributed Cloud Bot defense connector with BIG-IP was discussed in details in this Article F5 Distributed Cloud Bot Defense on BIG-IP 17.1 You will follow the steps mentioned in the article, with few changes mentioned below, API Hostname Web: ibd-web.fastcache.net For Per-session policies we use /my.policy as the target URL, while for Per-request and MFA implementation, you need to add /vdesk/*. Protection Pool - Web: Create pool with FQDN ibd-web.fastcache.net Virtual server, Create LTM virtual server to listen to incoming traffic, perform SSL offloading, HTTP profile and attach Bot Defense connector profile. Forwarding iRule, attach forwarding irule to the Bot virtual server. when CLIENT_ACCEPTED { ## Forwarding to the APM Virtual Server virtual Auth_VS } BIG-IP APM Policies,In this step we are creating two options of our deployment, Per-Session policy, where BIG-IP presents Logon page to the user. Per-Request policy, which services in case initial logon is handled at remote IdP and APM handles Per-request, MFA authentication or API security. Now, it's time to run the traffic and observe the results, From client browser, we can see the customer1.js inserted, From F5 XC Dashboard, Conclusion The primary goal of incorporating BIG-IP APM into the Bot Defense solution is to strike a balance between accelerating application development across web and mobile platforms while enforcing consistent organizational bot policies. By decoupling application login and authentication from the application itself, this approach enables a more streamlined, optimized, and secure bot defense implementation. It allows development teams to concentrate on application performance and feature enhancements, knowing that security measures are robustly managed and seamlessly integrated into the infrastructure. Related Content F5 Distributed Cloud Bot Defense on BIG-IP 17.1 Bot Detection and Security: Stop Automated Attacks 2024 Bad Bots Review
