Introducing F5 AI Gateway

F5 AI Gateway version 1.0 was released in February 2025, and it comes with support for a robust set of processors and use cases.

The 4 processors that come out-of-the-box with AI Gateway are: 

• prompt-injection: detecting and blocking prompt-injection attacks with support for English language
• system-prompt: allowing to set tighter guardrails that improve the security posture of the downstream LLM 
• language-id: predicting the language used in the prompt and supporting sophisticated context-based routing
• repetition-detect: detecting and blocking attacks that rely on repeating strings in the input prompt 

Of course, custom processors can be added, using the available SDK.

Supported main use cases 

Route Abstraction

AI Gateway can decouple the client connection from the downstream LLM one. It presents a single endpoint for client connections, regardless of the LLM that is ultimately servicing the request. This allows for excellent code portability, as the Generative AI application developers don't need to change the GenAI app code every time the downstream LLM changes. AI Gateway transparently performs this task.

Context Routing

Being able to make routing decisions based on the prompt context is a powerful feature that allows AI Gateway to support complex policies easily. AI Gateway will look at certain traffic characteristics, for example the HTTP headers, the JWT claims, or the tags generated by the AI Gateway itself, when making a decision to route the input prompt to a downstream LLM.  

Telemetry & Observability

Audit logging is critical in running GenAI applications in production, both from regulatory and operations perspectives. It facilitates troubleshooting in a non-deterministic environment and provides an audit trail that matches the input prompt and the LLM response. AI Gateway can record transactions for auditing. It records not only the input prompt and the LLM response, but also the AI Gateway’s internal stages and processors’ states.  

Security

Security is core to AI Gateway, and the security features that come out-of-the-box act at Layer 8, understanding the prompt context, as opposed to regular API Gateways that cannot understand the semantics of the submitted prompt.  

F5 AI Gateway in action

Conclusion

F5 AI Gateway natively understands the prompt context, enabling security controls tailored for GenAI traffic, route abstraction, and context routing, as well as supporting transaction logging for audit purposes.