# AI Quickstart: A Ready-to-Run Reference for Securing LLM Inference on OpenShift AI with F5 XC WAAP

**URL:** <https://community.f5.com/t/ai-quickstart-a-ready-to-run-reference-for-securing-llm-inference-on-openshift-ai-with-f5-xc-waap/76978>\
**Category:** F5 Technical Articles\
**Tags:** security\
**Created:** [March 27, 2026, 12:00pm UTC](https://community.f5.com/t/ai-quickstart-a-ready-to-run-reference-for-securing-llm-inference-on-openshift-ai-with-f5-xc-waap/76978 "2026-03-27T12:00:00Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Eric\_Ji](https://d1p9zq3aats0t8.cloudfront.net/user_avatar/community.f5.com/eric_ji/32/17001_2.png) [@Eric\_Ji](https://community.f5.com/u/Eric_Ji)\
**Post date:** [March 27, 2026, 12:00pm UTC](https://community.f5.com/t/ai-quickstart-a-ready-to-run-reference-for-securing-llm-inference-on-openshift-ai-with-f5-xc-waap/76978/1 "2026-03-27T12:00:00Z")

</div>

The F5 + Red Hat AI Security Quickstart is now live in the Red Hat AI catalog — a ready-to-run reference for securing LLM inference endpoints on OpenShift AI with F5 Distributed Cloud WAAP.

Picture a financial services team that just wrapped up a successful pilot of an AI assistant for their underwriting desk. The assistant is grounded in the firm’s own documents — policy manuals, risk frameworks, compliance filings — using Retrieval-Augmented Generation. Business stakeholders are asking when it goes into production.

Then the security review lands. The LLM inference endpoint has no WAF policy, no schema enforcement, no rate limiting. A pen tester sends a request with an embedded `<script>` tag in the chat payload. It goes straight through to the model. She sends it 500 times in a minute. Every request succeeds. She queries `/v1/version` — an internal operational endpoint never meant to be external — and it responds with model metadata.

The rollout is paused. The AI team is frustrated. The security team is not wrong.

This is the scenario we built the [F5 API Security AI Quickstart](https://docs.redhat.com/en/learn/ai-quickstarts/rh-f5-api-security) to solve — and why we partnered with Red Hat to publish it in the [Red Hat AI Quickstarts catalog](https://docs.redhat.com/en/learn/ai-quickstarts).

* * *

## What the Quickstart Deploys

The quickstart stands up a complete, secured AI stack on Red Hat OpenShift AI in two steps — a Helm install for the RAG stack, and an Ansible playbook for the F5 XC security layer.

**RAG Stack (Helm) — `make install` deploys all four components together:**

- **LlamaStack** (port 8321) — AI orchestration framework providing an OpenAI-compatible API surface for chat completion, embeddings, and model management
- **LLM Service (vLLM)** — GPU-accelerated model inference; supports Llama-3.2-3B-Instruct by default, with pre-configured options up to Llama-3.3-70B
- **Streamlit UI** (port 8501) — chat interface for interacting with the RAG assistant and uploading documents
- **PostgreSQL + pgvector** — vector database for semantic document retrieval; documents are embedded at ingestion and retrieved by similarity at query time

**Security Layer (Ansible) — deploys the F5 XC Customer Edge as pods inside the cluster:**

- The CE connects to the F5 Distributed Cloud backbone and discovers cluster services via the Kubernetes API
- All traffic to LlamaStack APIs passes through the CE before reaching the inference runtime or vector database
- Kernel configuration (HugePages) and storage validation are automated as part of the same playbook — not manual prerequisites

 ![Architecture diagram: F5 XC Customer Edge deployed inside OpenShift intercepts all traffic to LlamaStack APIs](https://d20hrnpixdzcsd.cloudfront.net/original/2X/d/d22b9dbef7ca7e110ad2e387ef208c11885e1c55.png)

_Figure 1: F5 XC Customer Edge co-located inside OpenShift, protecting LlamaStack APIs and model servers. All user traffic passes through the CE before reaching inference._

The key design decision: the F5 XC Customer Edge runs _inside_ the cluster, not in front of it as an external appliance. It enforces WAF, API specification, and rate-limiting policies within the cluster’s network fabric — no traffic tromboning, no added round-trip for large LLM payloads.

* * *

## Three Security Use Cases, Hands-On

The quickstart ships with an end-to-end security testing guide covering three use cases — a starting point that demonstrates the fundamentals, while only scratching the surface of what F5 Distributed Cloud WAAP can do across its full feature set (bot defense, client-side protection, DDoS mitigation, service policy, and more).

**Use Case 1 — WAF: Block injection attacks**  
Simulate an XSS payload inside a chat completion request body. Watch it pass through unprotected. Enable a F5 XC WAF policy, re-run the same request, and see it blocked — with the event logged in Security Analytics.

**Use Case 2 — API Specification Enforcement: Kill shadow APIs**  
Hit `/v1/version` — an internal endpoint never meant to be external. Upload the approved OpenAPI spec to F5 XC and enable API Inventory enforcement. Re-run the same request: **403 Forbidden**. Anything not in the spec is blocked by default.

**Use Case 3 — Rate Limiting: Protect GPU compute**  
Fire 20+ requests per minute from a single client — all succeed. Configure a per-client rate limit in F5 XC. Requests beyond the threshold return **429 Too Many Requests**. GPU compute is now a protected resource, not an open buffer.

* * *

## Why This Matters Beyond the Demo

LLM inference endpoints are HTTP APIs. They accept free-form text inside JSON and process it without validation — including injection payloads, shadow API calls, and unconstrained request floods. The “it’s only accessible inside the cluster” assumption breaks down faster than most teams expect: a misconfigured route, a developer exposing Swagger to test something, a service mesh misconfiguration. I’ve seen each of these in production-bound deployments over the past year.

The financial services framing is illustrative, not exclusive. Healthcare organizations protecting PHI, government agencies running citizen-facing AI, any enterprise with a private LLM and sensitive internal data — the same gaps exist, and the same architecture closes them.

What makes this a reference architecture rather than just a demo: the security controls are **native to the deployment**. The Ansible playbook that deploys the F5 XC Customer Edge is part of the infrastructure bring-up — not a post-go-live addition. Security teams get WAF coverage, API schema enforcement, and rate limiting in place before the first production request is ever sent.

* * *

**Ready to try it?**

Browse the quickstart in the [Red Hat AI Quickstarts catalog](https://docs.redhat.com/en/learn/ai-quickstarts/rh-f5-api-security) — deployment instructions, security use case guides, and architecture docs all in one place.

Clone the repo and run through the three use cases: [github.com/rh-ai-quickstart/f5-api-security](https://github.com/rh-ai-quickstart/f5-api-security)

_This quickstart was developed in collaboration with the Red Hat AI team and is part of the [Red Hat AI Quickstarts catalog](https://docs.redhat.com/en/learn/ai-quickstarts)._