Retain the URL on Browser
Hello Team, We have a requirement , where we have to retain the URL on the browser as is , However at the backend (pool member) user hits land on the other URL/URI. For example: Actual URL (hit by user) : 123.abc.com/test (this has to be retain on user's browser , that is .. user should always see this URL on his browser). At Backend (pool member) : This hit should forwarded or land on 123.XYZ.com/test (which is hosted on backend server). In short, For end user, URL should not be change anyway. Rgds ***F5 App Connect and NetApp S3 Storage – Secured Scalable AI RAG
F5 Distributed Cloud (XC) is a SaaS solution which securely exposes services to the correct service consumers, whether the endpoints of the communications are in public clouds, private clouds, or on-prem data centers. This is particularly top of mind now as AI RAG implementations are easy to set up but are really only effective when the correct, often guarded, enterprise data stores are consumed by the solution. It is a common scenario where the AI compute loads are executed in one location, on-prem or perhaps in a cloud tenant. However, the data to be ingested, embedded, and stored in a vector database to empower inferencing may be distributed through many different geographies. The data sources to be ingested into RAG are often stored in NetApp form factors, for instance StorageGrid, a native object-first clustered solution for housing S3 buckets. Also, the ONTAP family, where frequently files are accessed through NAS protocols like NFS or SMB, today can see the RAG source content exposed as objects through S3-compliant API calls and the corresponding protocol license. Technology Overview The XC module App Connect leverages L4-L7 distributed load balancers to securely provide a conduit to enterprise NetApp-housed data for the centralized AI workloads leveraging RAG. The following is the setup objective for this article, although many customer edge (CE) sites exist, we aim to bring together corporate documents (objects) in a San Jose, California datacenter to a self-hosted AI/RAG solution running in a Seattle area datacenter. The Advantage of Load Balancers to Securely Attach to Your Data Previous articles have leveraged the XC Network Connect module to bring together elements of NetApp storage through NAS protocols like NFS in order to run AI RAG workloads, both self-hosted and through secure consumption of Azure OpenAI. The Network Connect module provides secure L3 (IP) connectivity between disparate sites. An advantage is Network Connect will support all IP-based protocol transactions between sites, and firewall rules to preclude unwanted traffic. Network Connect is great when ease of deployment is paramount, however if you know the protocols to be supported are HTTP or TCP-based read on about App Connect, a solution that can address any IP overlap that may exist between your various sites to be interconnected. App Connect is a different take on providing connectivity. It sets up a distributed load balancer between the consumer (in our case an AI server running LLMs and a vector database) and the services required (NAS or S3 accessible remote data stores on NetApp appliances or hosted services). The load balancer may be an HTTPS offering, which allows the leveraging of F5’s years of experience in web security solutions, including an industry-leading web application firewall (WAF) component. For non-web protocols, think NFS or SMB, a TCP layer load balancer is available. Advantages are that only the subnets where the consumer exists will ever receive connectivity and advertisements around the service configured. The load balancer can also expose origin pools that are not just private IP addressed appliances, origin pools can also be Kubernetes services. A final App Connect feature that is noteworthy, the offering provided is an L4 through L7 service (such as HTTPS), and as such the local layer 3 environment of the consumer and, in our case, storage offering is irrelevant. A complete overlap of IP, perhaps both ends are using the same 10.0.0.0/16 allotments, is acceptable, something extremely valuable within divisions of large corporations that have separately embraced similar RFC-1918 address spaces. Also, perhaps through mergers and acquisitions the net result in a major institution are widespread instances of duplicate IP spaces in use, IP renumbering projects are legendary as they are lengthy and fraught with the risks of touching core routing tables. Lastly, applications that require users to configure IP addresses into GUIs are problematic as values are dynamic, App Connect provides services by name typically and this is less burdensome for IT staff who manage applications. A Working AI RAG Setup Using NetApp StorageGRID and XC App Connect A Ubuntu 22.04 Linux server was configured as a hosted LLM solution in a Seattle-area datacenter. The Ollama open-source project was installed in order to serve both generative AI LLMs quickly (llama3.1, mistral and phi3 were all used for comparative results) and the required embedding LLM. The latter LLM is needed to create vector embeddings of both source enterprise documents and subsequent real-time inference query payloads. Through semantic similarity analysis, RAG will provide augmented prompts with useful and relevant enterprise data to the Ollama-served models for better AI outcomes. Using the s3fs offering on Linux one can quickly mount s3 buckets as file systems using FUSE (file system in user space). The net result is that any S3 bucket, supported natively by NetApp StorageGRID and through a protocol license on ONTAP appliances, can now be mounted as a Linux folder for your RAG embedding pipeline to build a vector database. The key really is how to easily tie together S3-compliant data sources through your modern enterprise, no matter where they exist and the form-factor they are in. This is where XC App Connect enters the equation, dropping a modern distributed load balancer to project services across your network locations. The first step in configuring the HTTPS Load balancer to connect sites is to enter the Multi-Cloud App Connect module of the XC console. Once there, primarily three key items need to be configured: An origin pool that points at the StorageGrid nodes or the local load balancer sitting in front of the nodes, these are private addresses within our San Jose site. An HTTPS load balancer that ties a virtual service name (in our case the arbitrary name s3content.local), to our origin pool. Establish where the service name will be projected by DNS and connectivity will be allowed, the service s3content.local is not to be projected into the global DNS but rather will only be advertised by the Seattle CE inside interface, essentially making this load balancer a private offering. Here is the origin pool setup, in our case a BIG-IP is being used as a local load balancer for StorageGRID and thus its private San Jose datacenter address is used. To achieve the second item, an HTTPS load balancer, we key in the following fields, including the FQDN of the service (s3content.local), the fact that we will provide a TLS certificate/key pair to be used by the load balancer, and the one-check option to create an HTTP-to-HTTPS redirection service too. Lastly, the advertisement of our service will only be supported by the CE node at the Seattle site, requests for s3content.local from our AI server will resolve to the local CE node inside network interface IP address. The App Connect load balancer will ensure the underlying connectivity, through the underlay network, to the origin pool (StorageGRID) in San Jose. RAG Using Ollama-Based Models and Remote StorageGRID Content Various methods exist to leverage freely available, downloadable AI LLMs. One popular approach is huggingface.com whereas another can be to leverage the Ollama framework and download both embedding and generative models from ollama.com. The latter approach was followed in this article and in keeping with past explorations, Python 3 was used to manipulate AI programmatically, including the RAG indexing tasks and the subsequent inferencing jobs. Ollama supports a Docker-like syntax when used interactively from the command line, the one embedding model and three generative models are seen below (from Ubuntu 22.04 terminal). $ ollama ls NAME ID SIZE MODIFIED llama3.1:latest 42182419e950 4.7 GB 6 days ago mistral:latest 974a74358d6 4.1 GB 6 days ago nomic-embed-text:latest 0a109f422b47 274 MB 6 days ago phi3:latest 4f2222927938 2.2 GB 7 days ago The RAG tests includes ingestion of both .txt and .pdf documents provided by App Connect from NetApp StorageGRID. A private CA certificate was created using an OpenSSL-derived tool and loaded into the Seattle Linux and Windows hosts. That CA cert was then used to create a pkcs12-packaged certificate and key set for s3content.local and uploaded to the HTTPS load balancer setup on Distributed Cloud. A quick Windows-based S3browser test confirmed reachability from Seattle: To leverage RAG document ingestion on Linux, the s3fs project was made use of. A Linux folder can be mounted using NAS protocols like NFS, a simple example might be: #sudo mount -t nfs 10.50.0.202:/f5busdev /home/steve/f5bizdev/ Using s3fs, folders can similarly be mounted which tie back to buckets on the remote StorageGRID: #s3fs mybucket001 /home/steve/rag-files-001 -o url=https://s3content.local:443 -o use_path_request_style -o passwd_file=/home/steve/.passwd-s3fs At this point RAG ingestion can take place. The net effectiveness of RAG is often attributed to the quality of the embedding LLM used. In this case the Python scripts leveraged the Ollama API and the locally downloaded “nomic-embed-text” model, one that is generally considered just passable but is small and runs well on CPU-only hosts. The Gutenberg project novel The Great Gatsby by F. Scott Fitzgerald was consumed from the NetApp remote bucket and a detailed question was asked of llama3.1, first without RAG’s augmentations, meaning just the pre-trained data was leveraged. $ python search.py What university did Gatsby attend? Answered without RAG: “In the novel "The Great Gatsby" by F. Scott Fitzgerald, the character of Jay Gatsby does not reveal where he attended university, as it is never explicitly stated in the book. However, it has been speculated and suggested in various adaptations and interpretations that he may have studied at St. Olaf College in Minnesota or Yale University due to some similarities with their campus architecture in the novel. But again, this remains purely speculative and not confirmed by Fitzgerald himself.” Note, the user prompt shown (nor the system prompt) told llama3.1 that the question pertained to a specific novel, it figured that part out on its own based on the impressive base training that is baked into foundational models. However, the answer is imperfect. With RAG supporting the query the result was much better; it mimics a response of an individual who had actually just read the entire novel, cover-to-cover, and is able to provide more nuanced and accurate answers (trimmed for brevity). $ python search.py What university did Gatsby attend? Answered with RAG: "According to the text, Gatsby attended 'Oggsford College' in England, not Oxford University. It is also mentioned later in the text that Gatsby told Mr. Wolfshiem that he had been employed by Dan Cody on a boat, and that this occurred after World War I, but before he claimed to have attended Oggsford College (which suggests that his attendance at the college may be questionable)." Pointing Linux mounted folders towards buckets containing pdf documents seemed to work best when the source documents were smaller. For instance, user manuals worked well, even though the embeddings focused upon textual chunks and disregarded diagrams. This script was instructed to provide attributions within the augmented text provided to the LLM, specifically the manual page number and document chunk number from that page. The following is a result using the smallest Ollama generative LLM test, the Phi3 model and quizzing it about some lawn maintenance equipment. $ python query_data.py "Are there any concerns with the muffler on this lawn mower?" Response with RAG: Yes, according to the given context, there is a concern about the muffler on this lawn mower. It is stated that if the engine has been running, the muffler will be hot and can severely burn you. Therefore, it is recommended to keep away from the hot muffler. Sources: ['../rag-files-003/toro_lawn_mower_manual.pdf:12:2', '../rag-files-003/toro_lawn_mower_manual.pdf:12:1', '../rag-files-003/toro_lawn_mower_manual.pdf:21:1', The findings were less positive with very large PDF documents. For instance, Joseph Hickey’s 1975 book A Guide to Bird Watching is available in the public domain and totals almost 300 pages and is 32 megabytes in size. Regardless of the LLM, mistral or llama3.1 included, rarely were questions taken directly from specific pages answered with precision. Questions supported by statements buried within the text, “Where was the prairie chicken first observed?” or “Have greenfinches ever been spotted in North America and how did they arrive?” all went unanswered. Get Better Mileage from Your RAG To optimize RAG, it’s unlikely the generative LLMs are at fault; with Ollama allowing serial tests it was quickly observed that none of Mistral, Llama3.1 or Phi3 differed when RAG struggled. The most likely route to improve responses is to experiment with the embedding LLM. The ability to derive semantic meaning for paragraphs of text can vary, Hugging Face provides a leaders board for embedding LLMs with their own appraisal via a performance scoring system in the massive text embedding benchmark (MTEB). Other ideas are to use significantly larger chunk sizes for large documents, to reduce the overall number of vector embeddings being semantically compared, although a traditional 2,048 token context window in inferencing would limit how much augmented text can be provided per RAG-enabled prompt. Finally, multiple ways exist to actually choose similar vector embeddings from the database, approaches like cosine similarity or Euclidean distance. In these experiments, the native feature to find “k” similar vectors is provided by ChromaDB itself, as explained here. Other methods that play this critical search for related, helpful content would include Facebook AI Similarity Search (FAISS), which uncouples the search feature from the database, reducing risks of vector DB vendor lock in. Other libraries, such as the compute-cosine-similarity library are available on-line, including support for languages like JavaScript or TypeScript. Future experiments with better embedding models or possibly changing document chunking sizes to larger values might well produce even better results when scaling up your RAG deployment to enterprise scale and the expected larger documents. F5 XC App Connect for Security and Dashboard Monitoring The F5 Distributed Cloud load balancers provide a wealth of performance and security visibility. For any specific time range, just glancing at the Performance Dashboard for our HTTPS load balancer quickly gives us rich details including how much data has been moved to and from NetApp, the browser types used, and specifically which buckets are active on the StorageGRID. Enterprises frequently invest heavily in dedicated monitoring solutions, everything from raw packet loggers to application performance measurement (APM) tools, sometimes offering PCAP exports to be consumed in turn by other tools such as Wireshark. Although Distributed Cloud load balancers are a secure communications solution, a wealth of monitoring information is available. Both directions of transactions, consisting of requests coupled with their responses as a single entity, are monitored and available in both a decoded, information pane and a rawer .json format for consumption by other tools. Here is one S3 object write from Seattle, crossing the XC HTTPS load balancer, and storing content on the San Jose StorageGRID. The nature of the client, including browser agent (S3 Browser in the image) and TLS signature for this client are available, as well as the object name and bucket it was targeting on the NetApp appliance. Useful Approaches for Managing and Securing Storage API Traffic A powerful module in Distributed Cloud that locks in on NetApp traffic, in this case carried between cities by S3-compliant APIs, is the API Discovery module. As seen in the following image, an operator can add approved API endpoints to the “Inventory”, similar to adding the endpoint to a swagger file, something easily exported from the GUI and potentially integral in enterprise API documentation. As denoted in the next image, the remaining “Shadow” endpoints that were all automatically discovered, allow quick attention brought to them and is an enabler of a positive security approach whereby shadow traffic could be blocked immediately by the load balancer. In this case, a quick method of blocking unsanctioned access to specific StorageGRID buckets is arrived at. Also worth noting in the above screenshot, the most active APIs for a selected time period, 5 minutes to a full day perhaps, are brought to the operator’s attention. Finally, a last example of the potential value of the API discovery module are the sensitive data columns. Both custom provided data types observed in flight (such as phone numbers or national health ID values) as well as non-compliance to industry-wide guidelines (such as PCI_DSS) are flagged per S3 API endpoint. Automatic Malicious User Mitigation Distributed Cloud also offers a valuable real-time malicious user mitigation feature. Using behavioral observations, many harnessing AI itself, rogue consumers or generators of S3 objects can be automatically blocked. This may be of particular use when the distributed HTTPS load balancer provides NetApp S3 access to a wider range of participants, think of perhaps partner enterprises with software CE sites installed at their location. Or, ramping up, consider general Internet access where the App Connect load balancer projects access to FQDNs through global DNS and the load balancer is instantiated on an international network of regional edge (RE) sites in 30+ major metropolitan markets. This user mitigation feature can be enacted by first tying a user identity policy to the HTTPS load balancer. One valuable approach is to make use of state-of-the-art client-side TLS fingerprinting, JA4 signatures. Combining TLS fingerprinting with other elements, such as client source IP, can assist is categorizing the unique user driving any attempted transaction. With this selection in place, an operator need only flip “on” the automatic mitigations and security will be ratcheted up for the load balancer. As seen in the following screenshot, XC and its algorithms can gauge the threat level presented by users and respond accordingly. For detected low threat levels JavaScript challenges can be presented to a client browser, something not requiring human intervention but assists in isolating DDoS attackers from legitimate service clients. For behavior consistent with medium threat levels, something like a Captcha Challenge can be opted for, in cases where a human should be driving interactions with the service presented by the load balancer. Finally, upon the perception of high threat levels Distributed Cloud will see to it that the user is placed into a temporarily blocked state. Summary In this article we have demonstrated that the F5 Distributed Cloud App Connect module can be set up to provide a distributed L4-7 load balancer that can bring remote islands of storage, in this case NetApp appliances, to a centralized RAG AI compute platform. Although TCP-based NAS protocols like NFS could be utilized through TCP load balancing, this particular article focused upon the growing S3-compatible API approach to object retrieval, which uses HTTPS as transport. Through a pair of CE sites, an S3 service was projected from San Jose to Seattle, the origin pool was a cluster of NetApp StorageGRID nodes, and the consuming entity was Ubuntu running Ollama LLMs models in support of RAG. The service was exclusively projected to Seattle and the net result was AI outcomes that consumed representative novels, research guides and product user manuals for contextually meaningful responses. The App Connect module empowers features such as rich transactional metrics, API discovery coupled with enforcement of S3 bucket access, and finally the ability to auto-mitigate worrisome users with reactions in accordance with threat risk levels.
Priority group activation on GTM.
Hello All, I need to configure active standby configuration on GTM pool level, only one VS should be UP and second should be standby if one vs is down then traffic should pass to another VS and i can see there is one option Minimum-up Members but i do not know how to use it as a priority group activation on GTM level. If any one has any article or config suggestion please share. Many thanks in advanced for your time and consideration.VPN issues
Hi, I have Verizon 5G home internet with upload speeds of 200+ Mbps but that drops to 3.2-3.6 Mbps once I log into the Big-IP VPN that is required for work. I did not have this issue historically. My employer updated the VPN a few weeks ago, and the issue arose then but work IT is unable to assist thus far. Any help is appreciated.
f5 command document
hello everyone have a good day i want to ask, in some cases that experience repeated input data, or closing some server objects that only provide a little time in their implementation. often i use cli command to speed up the process. sometimes i don't find documentation for some situations that i need, so i search by doing 'tab' and try it in the lab, some i do just by trying. does anyone have information on cli command f5 with explanation of its usage. this is very helpful in work. thank you.
F5xC Migration
Hey Amigos, Need some advice.. I am implementing F5xC on our infra and migrating applications, however, ran into a small problem and need guidance.. There's an on-prem application sitting behind Citrix LB with the SSL offloaded directly on to the backend members i.e. SSL passthrough configured.. We have to migrate this app behind F5xC with SSL certificate on the F5xC as well.. Have below concerns ; Would this solution work if we get the SSL cert from the server itself and deploy it on the F5xC ? Has anyone implemented this sort of solution before, if yes, can anyone share their observations ? There's no test env so I can't really test this in non-prod.. This has to be implemented in prod directly and hence the precautions :)Default retry time inband monitor
Hi team I'm looking for a little information, about what causes the inband monitor on ltm to be set by default to have a retry time value of 300 seconds. Does this retry time rule affect performance in the data sending process? for example, when I set the retry time to 1s, the member health check will open every 1s. Does that pose a risk for this passive monitor? thank youNginx Plus LB as backend node member on F5 LTM
We have requirement for traffic load balancing as mentioned in below scenario. Main URL is configured on L5 LTM VIP , 80% traffic should go on Azure VM node , 20% should go on AKS via NGINX plus load balancer F5 LTM VIP - abc.com - 10.10.10.10 Pool member: Node 1 - Azure VM - 10.10.20.5 Node 2 - Nginx Plus virtual server - abcaks.com - 10.10.30.5
