Skip to content

Commit 76b4576

Browse files
redhatHameedredhatHameed
authored
docs: Example to Deploy Feast Remote Server Components Using Podman Locally (feast-dev#4516)
* Added examples to deployed Feast remote server components podman container locally Signed-off-by: Abdul Hameed <[email protected]> * removed the script and used composed files Signed-off-by: Abdul Hameed <[email protected]> --------- Signed-off-by: Abdul Hameed <[email protected]>
1 parent 9a0398e commit 76b4576

File tree

11 files changed

+409
-0
lines changed

11 files changed

+409
-0
lines changed

‎examples/README.md‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# Feast Examples
2+
3+
1. **[Quickstart Example](https://github.com/feast-dev/feast/tree/master/examples/quickstart)**: This is a step-by-step guide for getting started with Feast.
4+
5+
2. **[Java Demo](https://github.com/feast-dev/feast/tree/master/examples/java-demo)**: Demonstrates how to use Feast with Java feature server and deployed with Kubernetes.
6+
7+
3. **[Python Helm Demo](https://github.com/feast-dev/feast/tree/master/examples/python-helm-demo)**: Demonstrates Feast with Kubernetes using Helm charts and Python feature server.
8+
9+
4. **[RBAC Local](https://github.com/feast-dev/feast/tree/master/examples/rbac-local)**: Demonstrates using notebooks how configure and test Role-Based Access Control (RBAC) for securing access in Feast using OIDC authorization type with in a local environment.
10+
11+
5. **[RBAC Remote](https://github.com/feast-dev/feast/tree/master/examples/rbac-local)**: Demonstrates how to configure and test Role-Based Access Control (RBAC) for securing access in Feast using Kubernetes or OIDC Authentication type with in Kubernetes environment.
12+
13+
6. **[Remote Offline Store](https://github.com/feast-dev/feast/tree/master/examples/remote-offline-store)**: Demonstrates how to set up and use remote offline server.
14+
15+
7. **[Podman/Podman Compose_local](https://github.com/feast-dev/feast/tree/master/examples/podman_local)**: Demonstrates how to deploy Feast remote server components using Podman Compose locally.
16+

‎examples/podman_local/README.md‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
2+
# Feast example using Podman and Podman Compose
3+
4+
This guide explains how to deploy Feast remote server components using Podman Compose locally and run an example using the client.
5+
6+
## Prerequisites
7+
8+
1. **Podman**: [Podman installation guide](https://podman.io/).
9+
2. **Podman Compose**: [Podman Compose Installation guide](https://github.com/containers/podman-compose/tree/main?tab=readme-ov-file#installation]).
10+
3. **Python 3.9+ environment**
11+
4. **Feast CLI**
12+
13+
## Setup
14+
15+
### 1. **Feast Project Setup**
16+
17+
- The project [feature_repo](feature_repo) already created using `feast init` command
18+
19+
### 2. **Run the Podman Compose File**
20+
21+
- Use the [docker-compose.yml](docker-compose.yml) file to install and run the Feast feature servers (online, offline, and registry) on podman. The docker-compose file uses the `feastdev/feature-server:latest` image. Each respective service has specific port mappings and maps the volume from the `./feature_repo` configuration.
22+
- To start the feature servers, run the following command:
23+
24+
```bash
25+
podman-compose up -d
26+
```
27+
28+
- This will launch the necessary containers for online, offline, and registry feature servers.
29+
30+
### 3. **Verify the Installation**
31+
32+
- Use the `podman ps` command to verify that the containers are running:
33+
34+
```bash
35+
podman ps
36+
```
37+
38+
Example output:
39+
40+
```
41+
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
42+
61442d6d6ef3 docker.io/feastdev/feature-server:latest feast -c /feature... 2 minutes ago Up 2 minutes 0.0.0.0:6566->6566/tcp online-feature-server
43+
1274c21716a6 docker.io/feastdev/feature-server:latest feast -c /feature... 2 minutes ago Up 2 minutes 0.0.0.0:8815->8815/tcp offline-feature-server
44+
4e38ca8c39db docker.io/feastdev/feature-server:latest feast -c /feature... 2 minutes ago Up 2 minutes 0.0.0.0:6570->6570/tcp registry-feature-server
45+
```
46+
47+
- Alternatively, you can verify the running containers through **Podman Desktop**:
48+
![podman.png](podman.png)
49+
50+
### 4. **Run Feast Apply**
51+
52+
- To apply the feature store definitions to the remote registry, run the following command:
53+
54+
```bash
55+
podman exec registry-feature-server feast -c /feature_repo apply
56+
```
57+
58+
### 5. **Run Client Examples**
59+
60+
- The [client](client) folder contains example client-side configurations and code:
61+
- [feature_store.yaml](client/feature_repo/feature_store.yaml): Configuration for the feature store.
62+
- [test.py](client/feature_repo/test.py): Example Python script to interact with the Feast server.
63+
64+
### 6. **Cleanup**
65+
66+
- To stop and remove the running containers, run the following command:
67+
68+
```bash
69+
podman-compose down
70+
```
71+
72+
- This will stop all the feature server containers and clean up the environment.

‎examples/podman_local/__init__.py‎

Whitespace-only changes.

‎examples/podman_local/client/feature_repo/feature_store.yaml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
project: my_project
2+
registry:
3+
registry_type: remote
4+
path: localhost:6570
5+
offline_store:
6+
type: remote
7+
host: localhost
8+
port: 8815
9+
online_store:
10+
type: remote
11+
path: http://localhost:6566
12+

‎examples/podman_local/client/feature_repo/test.py‎

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
import subprocess
2+
from datetime import datetime
3+
import pandas as pd
4+
from feast import FeatureStore
5+
from feast.data_source import PushMode
6+
7+
def run_demo():
8+
try:
9+
store = FeatureStore(repo_path=".")
10+
11+
print("\n--- Historical features for training ---")
12+
fetch_historical_features_entity_df(store, for_batch_scoring=False)
13+
14+
print("\n--- Historical features for batch scoring ---")
15+
fetch_historical_features_entity_df(store, for_batch_scoring=True)
16+
17+
print("\n--- Load features into online store ---")
18+
store.materialize_incremental(end_date=datetime.now())
19+
20+
print("\n--- Online features ---")
21+
fetch_online_features(store)
22+
23+
print("\n--- Online features retrieved (instead) through a feature service---")
24+
fetch_online_features(store, source="feature_service")
25+
26+
print(
27+
"\n--- Online features retrieved (using feature service v3, which uses a feature view with a push source---"
28+
)
29+
fetch_online_features(store, source="push")
30+
31+
print("\n--- Simulate a stream event ingestion of the hourly stats df ---")
32+
event_df = pd.DataFrame.from_dict(
33+
{
34+
"driver_id": [1001],
35+
"event_timestamp": [
36+
datetime.now(),
37+
],
38+
"created": [
39+
datetime.now(),
40+
],
41+
"conv_rate": [1.0],
42+
"acc_rate": [1.0],
43+
"avg_daily_trips": [1000],
44+
}
45+
)
46+
print(event_df)
47+
store.push("driver_stats_push_source", event_df, to=PushMode.ONLINE_AND_OFFLINE)
48+
49+
print("\n--- Online features again with updated values from a stream push---")
50+
fetch_online_features(store, source="push")
51+
except Exception as e:
52+
print(f"An error occurred in run_demo: {e}")
53+
54+
def fetch_historical_features_entity_df(store: FeatureStore, for_batch_scoring: bool):
55+
try:
56+
entity_df = pd.DataFrame.from_dict(
57+
{
58+
"driver_id": [1001, 1002, 1003],
59+
"event_timestamp": [
60+
datetime(2021, 4, 12, 10, 59, 42),
61+
datetime(2021, 4, 12, 8, 12, 10),
62+
datetime(2021, 4, 12, 16, 40, 26),
63+
],
64+
"label_driver_reported_satisfaction": [1, 5, 3],
65+
"val_to_add": [1, 2, 3],
66+
"val_to_add_2": [10, 20, 30],
67+
}
68+
)
69+
if for_batch_scoring:
70+
entity_df["event_timestamp"] = pd.to_datetime("now", utc=True)
71+
72+
training_df = store.get_historical_features(
73+
entity_df=entity_df,
74+
features=[
75+
"driver_hourly_stats:conv_rate",
76+
"driver_hourly_stats:acc_rate",
77+
"driver_hourly_stats:avg_daily_trips",
78+
"transformed_conv_rate:conv_rate_plus_val1",
79+
"transformed_conv_rate:conv_rate_plus_val2",
80+
],
81+
).to_df()
82+
print(training_df.head())
83+
except Exception as e:
84+
print(f"An error occurred in fetch_historical_features_entity_df: {e}")
85+
86+
def fetch_online_features(store, source: str = ""):
87+
try:
88+
entity_rows = [
89+
{
90+
"driver_id": 1001,
91+
"val_to_add": 1000,
92+
"val_to_add_2": 2000,
93+
},
94+
{
95+
"driver_id": 1002,
96+
"val_to_add": 1001,
97+
"val_to_add_2": 2002,
98+
},
99+
]
100+
if source == "feature_service":
101+
features_to_fetch = store.get_feature_service("driver_activity_v1")
102+
elif source == "push":
103+
features_to_fetch = store.get_feature_service("driver_activity_v3")
104+
else:
105+
features_to_fetch = [
106+
"driver_hourly_stats:acc_rate",
107+
"transformed_conv_rate:conv_rate_plus_val1",
108+
"transformed_conv_rate:conv_rate_plus_val2",
109+
]
110+
returned_features = store.get_online_features(
111+
features=features_to_fetch,
112+
entity_rows=entity_rows,
113+
).to_dict()
114+
for key, value in sorted(returned_features.items()):
115+
print(key, " : ", value)
116+
except Exception as e:
117+
print(f"An error occurred in fetch_online_features: {e}")
118+
119+
if __name__ == "__main__":
120+
try:
121+
run_demo()
122+
except Exception as e:
123+
print(f"An error occurred in the main block: {e}")

‎examples/podman_local/docker-compose.yml‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
version: '3.9'
2+
3+
x-defaults: &default-settings
4+
image: feastdev/feature-server:latest
5+
restart: unless-stopped
6+
7+
services:
8+
online-feature-server:
9+
<<: *default-settings
10+
container_name: online-feature-server
11+
command: feast -c /feature_repo serve -h 0.0.0.0
12+
ports:
13+
- "6566:6566"
14+
volumes:
15+
- ./feature_repo:/feature_repo
16+
17+
offline-feature-server:
18+
<<: *default-settings
19+
container_name: offline-feature-server
20+
command: feast -c /feature_repo serve_offline -h 0.0.0.0
21+
ports:
22+
- "8815:8815"
23+
volumes:
24+
- ./feature_repo:/feature_repo
25+
26+
registry-feature-server:
27+
<<: *default-settings
28+
container_name: registry-feature-server
29+
command: feast -c /feature_repo serve_registry
30+
ports:
31+
- "6570:6570"
32+
volumes:
33+
- ./feature_repo:/feature_repo

‎examples/podman_local/feature_repo/__init__.py‎

Whitespace-only changes.

‎examples/podman_local/feature_repo/data/driver_stats.parquet‎

34.3 KB
Binary file not shown.

0 commit comments

Comments
 (0)