Feed Your On-Premises Data into Amazon Bedrock RAG using F5 Distributed Cloud and NetApp

Retrieval-Augmented Generation (RAG) solutions have tended to require a more sophisticated approach to AI, frequently involving Python scripts to interact with LLMs and hands-on experience with libraries like Streamlit to expose user-friendly chatbot-style interfaces.   In short, the technical hurdles to operationalizing a RAG solution, where existing on-prem volumes, such as those hosted in NetApp ONTAP appliances, have been anything but trivial.
Today, a solution exists that is essentially a turnkey, end-to-end solution to offer RAG-infused AI outcomes to employees or other parties.  This solution is one that is largely SaaS-configured and is mostly a series of intuitive mouse-clicks that can achieve the goal of harnessing the latest AI LLMs to leverage your own existing data via RAG.   The elements of the solution include Amazon Bedrock, NetApp FSx for NetApp ONTAP, and F5 Distributed Cloud to onboard existing on-prem data volumes into the RAG solution.

 

RAG AI Knowledge Bases – Vastly Reduced Time to Deployment

The solution described in this article chiefly leverages two elements to quickly harness corporate data, historically on-premises, into a working RAG solution:

• NetApp ONTAP (on-premises), F5 Distributed Cloud, and Amazon FSx for Netapp ONTAP (FSxN) together provide an infrastructure solution to incorporate existing data as easily accessible content to infuse RAG with.
• NetApp BlueXP Workload Factory for AWS Automation (Workload Factory) to bind Amazon Bedrock LLMs to your data, including Active Directory awareness for RAG that produces responses for users in accordance with their user and group file permissions

The ease of this solution is demonstrated with Amazon-provided models, which are from industry leaders like Amazon, Anthropic, Cohere, DeepSeek and Meta.  The models are the necessities of a modern RAG implementation, including embedding, text and Vision LLMs.  The majority of models are now available to be applied to your own NetApp data volumes, securely brought into your own AWS VPC through F5 Distributed Cloud.
An interesting aspect of the solution is that on-premises volumes are simply SnapMirror-ed to FSxN, in your own VPC, and remain in the standard SnapMirror volume type of Data Protection (DP).   RAG, leveraging these leading LLMs, creates vector embeddings directly from the DP volume, automatically, and a chat interface is immediately made available to your users.   The following is a description of the elements, and a demonstration of a context-aware AI reply based upon on-prem NetApp volumes.

 

F5 Distributed Cloud for Secure On-Prem to Amazon Connectivity

The F5 Distributed Cloud (XC) is a SaaS console-configured secure application and network delivery solution.  The XC “Network Connect” module harnesses the built-out global F5 infrastructure to provide automatic reachability between customer edge (CE) sites on-prem, in private clouds or enterprise tenants within an array of public cloud providers.  Where WAN facilities may already exist, CE sites can also be directly inter-connected with those data planes, otherwise the aggregate 14Tbps+ XC infrastructure will be harnessed.
A key deliverable is that routing is automatic, and those simple troubleshooting and analytic tools required by NetOps are uniform and available as a single-pane-of-glass experience in the XC SaaS console.
The capabilities to segment an enterprise’s global network assets into dispersed managed network entities, such as network interfaces or VLANs, that are treated as walled-off communities of interest exist.   This previous article describes network segmentation, as provided by F5 XC Network Connect, in terms of securely connecting on-prem NetApp ONTAP volumes with Amazon FSxN volumes, for key issues like SnapMirror replication, which simplify disaster recovery.  Another high-value feature enabled by the secured F5 connectivity is CacheVolumes, where globally dispersed volumes are cached within the Amazon environment, as they are accessed, allowing for rapid, high-QoS access times in subsequent reads.
The following is a simple example of four secured and isolated segments simply labeled as color names, including a CE site in the lower right with attachments to multiple segments.

 

Amazon Bedrock for the Latest AI Models to Optimize RAG upon your Data

Amazon Bedrock is a fully managed service that simplifies building and scaling generative AI applications by providing access to a variety of leading foundational models.   To utilize Bedrock models, a simple AWS console access request check box exists, with chatbot-style support of granted models normally available within a minute or two.  The following demonstrates the Bedrock portion of an AWS Console session, and the spot where model access is configured.   Note the highlighted models that have already been granted access and the arrow indicating how easily one can add many more.

The value of using your own NetApp volumes with AI is seen in the following sequence, simply for illustrative purposes.   Background: The F5 Distributed Cloud CE (Client Edge) is the component that customers will frequently run on virtual machines or bare metal in on-premises environments, or perhaps within their cloud instances, to publish applications or facilitate network reachability.   Release notes from the second half of 2024 describe enhancements to the ease of CE deployments and registration with the XC global SaaS console. 
Without access to this data, an LLM, especially a foundational model with a training cut-off date too distant, or void of proprietary or context-specific information, may struggle or be unable to answer very specific questions around these related areas.  Note the very limited depth in response to our highlighted AI request below (double click to enlarge image).

The solution is, of course, to inject your own pertinent data sources to avoid an AI outcome such as the above.   The question: how to do this easily and securely?

 

NetApp BlueXP Workload Factory for AWS Automation

Amazon, direct from the AWS console, does offer some degree of RAG capabilities, but they may not always be aligned with what an existing NetApp on-premises storage user is seeking.  Knowledge Bases with vector stores, an Amazon term synonymous with the popular understanding of RAG, can ingest data from your AWS S3 endpoints, or by crawling publicly reachable HTTPS servers; other options are being added over time, such as tie-ins to SharePoint sites.
The enthusiasm for an established NetApp ONTAP corporate customer to transfer large and potentially sensitive volumes to AWS S3, over the insecure Internet, is likely muted.   Instead, a solution where NetApp volumes can be SnapMirror-ed directly to a customer’s own VPC and consumed only then as part of a knowledge base is much more enticing.   This is where Workload Factory shines.

 

Phase One of Two:  Setup Your Storage for RAG

The first step is to log into the Workload Factory UI and from the Storage menu (menu selections vertically arranged on the left of screen) simply “discover” all on-prem ONTAP appliances.   For brevity, only one site has been discovered in the following screenshot, located in the Seattle area.   Notice how it is being accessed by private RFC-1918 style addressing.   This is due to the F5 Distributed Cloud Network Connect module that securely ties together through private layer-3 reachability the customer’s AWS VPC and every on-premises site around the world with ONTAP appliances.

Workload factory in the image has discovered, on the customer premises, a version 9.16 ONTAP appliance and all volumes configured there on storage virtual machine 0 (SVM0).   By simply clicking the “Replicate” button next to each volume of interest, one can create a SnapMirror-ed volume on FSx for NetApp ONTAP (FSxN) in the local VPC.  This is what will inform the RAG knowledge base.  The Replicate button will be available for use as soon as an FSxN file system is created, something we will now do using the adjacent tab on Workload Factory.  Simply mouse-click on the “FSx for ONTAP” tab from the current “On-Premises ONTAP” tab.
Beyond configuring an existing FSxN file system, Workload Factory takes ease of use to the next level. It allows one to add an entire new FSxN instance completely from the current portal, as opposed to requiring a transition to the AWS Console FSxN module.   Here we see a sample screenshot of adding a new FSxN file system (double click to enlarge).

The entire creation process, using AWS credentials that are customizable, takes minutes and is performed in this single web form above.  Only some features are called out in the provided image. Other features exist, such as opting for a “Scale Out” approach where HA pairs of nodes are created in large quantities for huge capacities. 
Scale-out, as described here, offers significantly higher performance and capacity than a “scale-up” approach by distributing workloads across multiple file servers, while scale-up is suitable for general-purpose workloads with lower performance demands. A scale-up deployment, which is used in this project, spreads storage across multiple AWS availability zones and is as simple as toggling a single checkbox.
To prepare the relationship between the FSxN file system and the on-premises ONTAP cluster, only a single aspect of the workflow involved briefly using ONTAP CLI; all other aspects were mouse-click-driven from Workload Factory.  The one CLI use case involved peering the two clusters to each other; it is described within this guide.   The cluster peering involved issuing this command from the FSx cluster and the on-premises cluster via SSH access and providing the two Inter-Cluster LIF IP addresses used at each end, referred to as source_inter_[1,2] and dest_inter_[1,2]:

(From FSxN cluster SSH session)  #cluster peer create -address-family ipv4 -peer-addrs source_inter_1,source_inter_2
(From ONTAP on-premises SSH session) #cluster peer create -address-family ipv4 -peer-addrs dest_inter_1,dest_inter_2
(From ONTAP on-premises SSH session) #cluster peer show

Peer Cluster Name  Availability       Authentication
-----------------      --------------     --------------
FSx-Dest               Available          ok

The peer relationship came up within seconds (available and authentication “ok”), after which all further steps were conducted in Workload Factory.
The following shows our source on-premises sample volume, filled with our RAG corpus of documents, SnapMirror-ed to FSxN after clicking the “Replication” button in the GUI.  The volume contains various files that will help us prove out that RAG is working as it will contain information not presently in the foundational model’s knowledge base.   The volume is titled “companyAstorage”(double click to enlarge).

An examination of the volume now available for RAG within our VPC on FSxN, named “companyAstorage_copy”, indicates, as expected it is of type “DP” (Data Protection).  With our data in place, we now move on to setting up an embedding and generative text LLM from Amazon’s wide offering and conducting a RAG demonstration.

 

Phase Two of Two: Utilized Amazon Suite of LLMs to Offer Powerful Chatbot-based RAG AI Against Your Own Data Files

To finish the configuration, we tie together the power of Amazon’s LLM services with our own data sourced from on-premises ONTAP and SnapMirror-ed to FSxN.  Also, active read/write volumes already within FSxN are ripe for RAG, too.   By moving to the AI menu on the left of the screen, we can see the three key elements of the solution (double click to expand).

The first time a user enters the AI portion of Workload Factory they will be prompted to perform the one-time “Infrastructure” setup, which is largely automated through AWS CloudFormation in the backend.   As seen in the above image, a diagram indicates the solution consists of 1. Amazon Bedrock LLMs 2. FSxN for Data 3. AI Engine and Orchestration.   As depicted, an EC2 instance was created automatically to empower the NetApp AI Engine the first time the user entered the screen.  It will be created in the same AWS Region as your FSxN deployment and base features like Vector Database are called out, in our case, LanceDB.
With the one-time AI Infrastructure now built out by Workload Factory’s automation, we can choose the RAG solution details that suit us, picking the Bedrock models (embedding and generative text in our example) and the source volumes.  Simply move to the Knowledge Base’s tab and by choosing “Add Knowledge Base” you can have a RAG solution, with LLMs of your choice, fed by your own data within a minute or two.

The “Add knowledge base” workflow generally breaks down into two objectives. First up is the selection of RAG particulars, such as which embedding and text LLMs to use for our use case, while attributing a volume on an FSxN file system for a workspace.   
The following calls out a couple of aspects in terms of choices. We have notice in area 1 that this is just a small portion of the text LLMs available, based upon what models we have requested access to in Amazon Bedrock.   In area 2, we decide upon the location for our working volume.

The second of the two objectives of the “Add knowledge base” workflow is to point our knowledge base at our own data volumes to be employed when RAG provides AI services to our users.  These are the data volumes we own, which have often been actively used on-premises, possibly for years, and now copies of which will be chunked and embedded into vector embeddings automatically.
The ensuing “Add data source” screens can be added repeatedly to capture as many of our source volumes from as many offices as desired for RAG; we can even apply a simple filter to only pick up certain folders within a given volume.   In our demonstration, we will add the Seattle-area On-Prem ONTAP volume that was SnapMirror-ed to FSxN.
 

A key deliverable of this complete solution is “time to real, demonstrable AI value”, unlike self-hosted and often programmatic solutions requiring Python skills and knowledge around RAG, the default settings in this case will often be sufficient to start using the solution right away.   When necessary, things like file types in your volumes can be pruned from RAG, but by default note the wide selection of textual formats, including pdf, docx and parquet formats are available.  This file parsing service is a critical part of the pre-processing data pipe that all RAG implementations must deliver upon but which are easily included here as part of the offering.  Graphical formats such as .jpeg and .png are also enabled out of the box.

 

Outcome: A Test of RAG Against Our Data

As seen in our original test of Amazon with Anthropic Haiku 3, the foundational model struggled with detailed questions about the F5 Distributed Cloud solution.   To validate RAG against our own data, a volume on-premises (“companyAstorage”) was filled, using NFS as an access protocol, with recent F5 Distributed Cloud release notes.   These notes are available from the product’s subscribers’ portal.   The release note file names generally do not tip off what they refer to. In this case all were in .pdf format and included:

We can now ask the same question as earlier, “How has F5 Distributed Cloud Services streamlined CE registration?”.   Double-click to enlarge image of successful RAG outcome below.

We see the fruits of NetApp Workload Factory, F5 Distributed Cloud for secure access to on-premises volumes, and the AI-wide capabilities of Amazon Bedrock.   The same LLM, which previously was not empowered to deliver specifics on our question, now has successfully done so through interpreting our private data, gone ahead and provided tactical and usable feedback.  Interesting power-user tip:  clicking on the attributions does not simply open the source documents, but reveals the individual chunks from those documents that led to the AI chatbot response.

 

Summary and Further Exploration Candidates

In this demonstration, we have seen the ease and speed to incorporate existing NetApp ONTAP appliance-based volumes with the tremendous breadth of AI offered by Amazon Bedrock.   The secure access of enterprise data was accomplished by F5 Distributed Cloud secure connectivity between enterprise AWS VPCs and a physical office in the Seattle area.   NetApp’s FSx NetApp for ONTAP (FSxN) offering was configured through the new Workload Factory SaaS console, and volumes required for RAG were replicated securely from on-premises to FSxN.
With the data made securely accessible and models subscribed to, NetApp BlueXP Workload Factory corralled the storage and AI elements to produce a simple chatbot interface that produced meaningful responses that RAG can now produce, with our data being the key ingredient.
Other explorations beyond this initial setup would certainly include user-aware RAG.  Incorporate an on-premises Active Directory, which traditionally allows an access protocol like server message block (SMB) and its inherent elements of RBAC, to consider user and group memberships prior to granting file access.   This allows RAG to tie the data chunks that will augment customer AI inferences to Active Directory permissions.   A user, in our example documented, not granted access to the sample .pdf release note documents, would incur a RAG result that would instead simply rely upon the other documents or even the model’s base knowledge.
Another area to expand upon is multi-modal RAG.   This is the ability to “chat” with the non-text elements of your documents, documents, which include images, video, or audio.  Consider, purely as an example, a RAG response with this described solution that considers document images of bar graphs when showing, say, quarter-over-quarter sales results, and formulates richer AI replies with this content. Unlike a data science project in the past, where GitHub repositories, Jupiter Notebooks or Python freeform coding exercises might all be required, with the solution offered by NetApp, F5 and Amazon the RAG solution described simply needs your volumes containing your proprietary data and RAG will produce the results.