This deployment is based on the validated pattern framework, utilizing GitOps for seamless provisioning of all operators and applications. It deploys a Retail Recommendation System that leverages two-towe algorithm training technich to provide personalized item suggestions to customers, enhancing store sales by considering their preferences and demographics.

The pattern harnesses Red Hat OpenShift AI to deploy and serve recommendation at scale. It integrates the Feast Feature Store for feature management, EDB Postgres to store user and item embeddings, and a simple user interface (UI) to facilitate customer interactions with the system. Running on Red Hat OpenShift, this demo showcases a scalable, enterprise-ready solution for retail recommendations.

• Podman
• Red Hat Openshift cluster running in AWS. Supported regions are : us-east-1 us-east-2 us-west-1 us-west-2 ca-central-1 sa-east-1 eu-west-1 eu-west-2 eu-west-3 eu-central-1 eu-north-1 ap-northeast-1 ap-northeast-2 ap-northeast-3 ap-southeast-1 ap-southeast-2 ap-south-1.
• GPU Node to run Hugging Face Text Generation Inference server on Red Hat OpenShift cluster.
• Create a fork of the git repository.

• UI: Allows users to browse recommendations, add items to cart, purchase, or rate products.
• Feast Feature Store: Manages and serves features for training and real-time inference.
• EDB Postgres with PGVector: Stores user and item embeddings, enabling fast similarity searches.
• Kafka Integration: Records user interactions for continuous learning and dataset updates.
• Red Hat OpenShift AI
• Two-Tower Architecture: Utilizes separate neural networks to generate user and item embeddings for personalized recommendations.

The workflow consists of the following steps:

1. Data Ingestion

• Data originates from parquet files containing users, items, and interactions.
• Feast scans feature definitions, validates them, and syncs metadata to its registry.

2. Training - using the Two-Tower algorithm:

We have two encoders:

• User Tower: Encodes user features (age, gender, preference, ...) into embedding.
• Item Tower: Encodes item features (category, price, ...) into embedding.

For each interaction, positive or negative, train the encoders in such a way that positive interactions bring the item and the user closer in cosine similarity in the embedding space, and negative interactions move the user and the item embeddings farther apart.

3. Batch Scoring & Materialization

• Generates embeddings for all users and items using trained encoders.
• Computes the latest feature values and precomputes top-k recommendations for each user.
• Stores results in the online store for fast retrieval.

• Recommendation UI: A simple web application for users to interact with recommendations.
• Kafka & Kafka connect: ingest user interaction with items from the ui and sent them to Kafka, kafka connect move this intercation events into a EDB database.
• Feast Feature Store: Manages feature definitions and serves data for training and inference.
• EDB Postgres with PGVector: Acts as online (real-time embeddings) stores.
• Kubeflow job: A batch job that train the user and item encoders, then genrate the data generates embeddings into the vector database.

To run the demo, ensure the Podman is running on your machine.Fork the rag-llm-gitops repo into your organization

Replace the token and the api server url in the command below to login to the OpenShift cluster.

oc login --token=<token> --server=<api_server_url> # login to Openshift cluster

git clone https://github.com/<<your-username>>/rec-sys-gitops.git
cd rec-sys-gitops

Following commands will take about 15-20 minutes

./pattern.sh make install

• Login to the OpenShift web console.
• Navigate to the Workloads --> Pods.
• Select the rec-sys project from the drop down.
• Following pods should be up and running.

# TODO

• Click the Application box icon in the header, and select Retrieval-Augmented-Generation (RAG) LLM Demonstration UI

• It should launch the application

  

• It will use the default provider and model configured as part of the application deployment. The default provider is a Hugging Face model server running in the OpenShift. The model server is deployed with this valdiated pattern and requires a node with GPU.

• Enter any company name

• Enter the product as RedHat OpenShift

• Click the Generate button, a project proposal should be generated. The project proposal also contains the reference of the RAG content. The project proposal document can be Downloaded in the form of a PDF document.

  

You can optionally add additional providers. The application supports the following providers

• Hugging Face Text Generation Inference Server
• OpenAI
• NVIDIA

Click on the Add Provider tab to add a new provider. Fill in the details and click Add Provider button. The provider should be added in the Providers dropdown uder Chatbot tab.

Follow the instructions in step 3 to generate the proposal document using the OpenAI provider.

You can provide rating to the model by clicking on the Rate the model radio button. The rating will be captured as part of the metrics and can help the company which model to deploy in prodcution.

By default, Grafana application is deployed in llm-monitoring namespace.To launch the Grafana Dashboard, follow the instructions below:

• Grab the credentials of Grafana Application
  • Navigate to Workloads --> Secrets
  • Click on the grafana-admin-credentials and copy the GF_SECURITY_ADMIN_USER, GF_SECURITY_ADMIN_PASSWORD
• Launch Grafana Dashboard
  • Click the Application box icon in the header, and select Grafana UI for LLM ratings
  • Enter the Grafana admin credentials.
  • Ratings are displayed for each model.

GOTO: Test Plan

EDB Postgres for Kubernetes is distributed under the EDB Limited Usage License Agreement, available at enterprisedb.com/limited-use-license.