The application brings together several technologies:
The journey starts at the React UI. When a user (yogi) clicks the login button, the React application redirects him to Keycloak, which acts as our Identity Provider (IdP). Keycloak maintains the user accounts, credentials, roles, and identity information. In our example, Yogi is already registered in the ai-lab realm. After successful authentication, Keycloak issues a JSON Web Token (JWT) and returns control to the React application. The JWT is a digitally signed token containing claims about the authenticated user, such as:
React keeps the token and includes it in subsequent API requests using the HTTP Authorization header. While large language models can process conversational context, they do not inherently provide persistent memory across independent sessions. At this point, React does not need to handle the user’s password or implement its own authentication mechanism. Keycloak takes care of identity management.
In this project, PostgreSQL takes on that responsibility by acting as the application’s episodic memory. Every conversation and message is stored persistently, linked to the authenticated user. When a user returns to an existing conversation, the application retrieves its message history and sends the relevant context to the language model.
FastAPI does not communicate with PostgreSQL directly in our application. Instead, we use an Object-Relational Mapper (ORM) as an intermediary between the Python application and the relational database.
Once Yogi submits a message, the request travels from the browser to Envoy Gateway.
Envoy acts as the entry point to our backend services. More importantly, it is responsible for validating the JWT before allowing the request to reach FastAPI.
When Envoy receives the request, it:
This means FastAPI does not need to independently implement JWT signature verification for every incoming request. Envoy handles that authentication boundary. In our implementation, Envoy also makes the authenticated identity available to FastAPI through a trusted request header. This is an important architectural decision: authentication is centralized at the Gateway rather than duplicated across individual backend services.
After successful validation, Envoy forwards the request to our FastAPI backend. FastAPI now handles the application logic. The backend identifies the authenticated user using the identity information forwarded by Envoy. It then checks the requested conversation and retrieves its message history from PostgreSQL. This is where PostgreSQL plays a particularly important role.We use it as the application’s episodic memory: it persistently stores conversations and their associated messages.
For example, the database contains three important entities:
When Yogi continues an existing conversation, FastAPI retrieves the previous messages belonging to that conversation.
An ORM allows us to work with database records as Python objects rather than writing SQL queries for every database operation. In our implementation, SQLAlchemy handles this responsibility.
Once FastAPI has retrieved the conversation history, it constructs the request for Google’s Gemini model. Instead of forwarding only the latest user message, the backend sends the relevant conversation history along with the new message. The request therefore contains both the current question and the context needed to understand it. Gemini processes the input and generates a response based on that context.
code is here
git clone https://github.com/yogenderPalChandra/keycloak-envoy-k8s.gitgit checkout postgres_chat_memory
kind create cluster - config kind-config.yaml
kind: ClusterapiVersion: kind.x-k8s.io/v1alpha4# Name your cluster directly in the config (optional)name: normal-clusternodes: # The control-plane node - role: control-plane kubeadmConfigPatches: - | kind: InitConfiguration nodeRegistration: kubeletExtraArgs: node-labels: "ingress-ready=true" # Forward local laptop ports (80/443) into the cluster (useful for Ingress) extraPortMappings: - containerPort: 80 hostPort: 80 protocol: TCP - containerPort: 443 hostPort: 443 protocol: TCP
cd keycloakkubectl create namespace identityk apply -f keycloak/postgres.yamlk apply -f keycloak/keycloak.yaml
Expose keycloak url in browser:
kubectl port-forward svc/keycloak 8080:8080 -n identity
visit via localhost:8080 username: admin, passowrd: admin
Please follow this article to create Realm, realm role, users, client in keycloak: https://pub.towardsai.net/seuring-ai-applications-with-keycloak-and-envoy-gateway-in-kubernetes-2ed6e3b324c7
Integrating a PostgreSQL database into this architecture is necessary to persistently store and retrieve all core data structures, including user profiles, active chats, and individual messages. By leveraging this persistent storage, our FastAPI backend can continuously retrieve the exact historical context of a conversation. This gathered history is then passed along with every new backend query, allowing the AI to deliver coherent, context-aware responses over time
k apply -f postgres_chat_memory/postgres.yaml
apiVersion: v1kind: Secretmetadata: name: ai-postgres-secret namespace: ai-platformtype: OpaquestringData: POSTGRES_USER: yogi POSTGRES_PASSWORD: yogi POSTGRES_DB: episodic_memory---apiVersion: v1kind: PersistentVolumeClaimmetadata: name: ai-postgres-pvc namespace: ai-platformspec: accessModes: - ReadWriteOnce storageClassName: standard resources: requests: storage: 2Gi---apiVersion: apps/v1kind: Deploymentmetadata: name: ai-postgres namespace: ai-platformspec: replicas: 1 selector: matchLabels: app: ai-postgres template: metadata: labels: app: ai-postgres spec: containers: - name: postgres image: postgres:17 imagePullPolicy: IfNotPresent ports: - containerPort: 5432 name: postgres envFrom: - secretRef: name: ai-postgres-secret volumeMounts: - name: postgres-data mountPath: /var/lib/postgresql/data readinessProbe: exec: command: - sh - -c - pg_isready -U "$POSTGRES_USER" -d "$POSTGRES_DB" initialDelaySeconds: 5 periodSeconds: 5 resources: requests: cpu: 100m memory: 128Mi limits: cpu: 500m memory: 512Mi volumes: - name: postgres-data persistentVolumeClaim: claimName: ai-postgres-pvc---apiVersion: v1kind: Servicemetadata: name: ai-postgres namespace: ai-platformspec: type: ClusterIP selector: app: ai-postgres ports: - name: postgres port: 5432 targetPort: 5432 protocol: TCP
network policy so that only FastAPI pod has access to our PostgreSQL DB:
k apply -f postgres_chat_memory/network_policy.yaml
apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata: name: allow-only-fastapi-to-postgres namespace: ai-platformspec: podSelector: matchLabels: app: ai-postgres policyTypes: - Ingress ingress: - from: - podSelector: matchLabels: app: ai-api ports: - protocol: TCP port: 5432
fastapi/│├── app/│ ├── __init__.py│ ├── app.py│ ││ ├── routers/│ │ ├── __init__.py│ │ └── conversations.py│ ││ └── db/│ ├── __init__.py│ ├── db.py│ └── models.py│├── Dockerfile├── .dockerignore└── fastapiDeployment.yaml
The FastAPI folder is organized into separate modules, each responsible for a specific part of the backend. app.py is the main entry point where the FastAPI application is initialized and API endpoints are registered. db.py manages the connection to PostgreSQL using SQLAlchemy, including the database engine, session factory, and session management. models.py defines Python classes representing database tables, such as users, conversations, and messages. These classes use SQLAlchemy ORM, allowing the application to interact with database records as Python objects instead of writing raw SQL queries for every operation. conversations.py contains the API endpoints and business logic for creating, retrieving, and deleting conversations. When a request arrives, FastAPI obtains a database session, uses the models to query or modify records, and SQLAlchemy translates those operations into SQL statements executed against PostgreSQL. The conversations module also ensures that users can access only their own chat history based on their authenticated identity. This separation of responsibilities makes the backend modular, maintainable, and easier to test. In simple terms, db.py connects to the database, models.py defines the data structure, conversations.py manages the conversation data, and app.py brings everything together.
cd fastapidocker build -t ai-fastapi:1.3.6 ./fastapikind load docker-image ai-fastapi:1.3.6 --name normal-cluster
Deploy FastAPI app:
kubectl create secret generic gemini-api \ -n ai-platform \ --from-literal=GEMINI_API_KEY='AQ.YOUR API KEY'
k apply -f fastapiDeployment.yamlkubectl port-forward -n ai-platform service/ai-api 8000:8000
Once the deployment rolls out, the FastAPI application initializes its data layer using an Object-Relational Mapping (ORM) architecture. Within models.py, database schema is defined programmatically as Python classes that map directly to relational database tables. During application startup, the engine triggers Base.metadata.create_all(bind=engine), automatically executing the underlying DDL to create the users, conversations, and messages tables inside the PostgreSQL database. Finally, to safely expose these newly initialized endpoints to the network, you must apply the matching HTTPRoute and SecurityPolicy resources to your (see Envoy step)
test the PostgreSQL DB:
kubectl exec -it deployment/ai-postgres -n ai-platform -- \ sh -c 'psql -U "yogi" -d "episodic_memory"'SELECT current_database(), current_user, version();SELECT role, content, created_atFROM messagesORDER BY created_at;
#Install Envou api gateway:helm install eg \ oci://docker.io/envoyproxy/gateway-helm \ --version v1.9.1 \ -n envoy-gateway-system \ --create-namespace #wait for 5 mins:kubectl wait \ --timeout=5m \ -n envoy-gateway-system \ deployment/envoy-gateway \ --for=condition=Available
cd gatewayk apply -f envoyProxy.yamlk apply -f envoyDeployment.yamlk apply -f security-policy.yamlkubectl -n envoy-gateway-system port-forward \ service/envoy-ai-platform-ai-gateway-2621731d 8888:80
This configuration file defines a Kubernetes HTTPRoute resource named ai-api inside the ai-platform namespace. It acts as a routing traffic director that attaches itself directly to a parent infrastructure controller named ai-gateway. Inside the configuration rules, it inspects incoming web requests to match specific URL paths. It uses PathPrefix logic to catch broad endpoints like /api/health and /api/conversations, meaning it catches any URL that starts with those strings. For specific endpoints like /api/chat, /api/whoami, and /api/admin/overview, it uses strict Exact matching logic. Finally, all matched web traffic is securely forwarded to a backend Kubernetes service named ai-api on port 8000.
---apiVersion: gateway.networking.k8s.io/v1kind: Gatewaymetadata: name: ai-gateway namespace: ai-platform annotations: gateway.envoyproxy.io/service-type: NodePortspec: gatewayClassName: eg listeners: - name: http protocol: HTTP port: 80 allowedRoutes: namespaces: from: Same---apiVersion: gateway.networking.k8s.io/v1kind: HTTPRoutemetadata: name: ai-api namespace: ai-platformspec: parentRefs: - name: ai-gateway rules: - matches: - path: type: PathPrefix value: /api/health backendRefs: - name: ai-api port: 8000 - matches: - path: type: Exact value: /api/chat backendRefs: - name: ai-api port: 8000 - matches: - path: type: Exact value: /api/admin/overview backendRefs: - name: ai-api port: 8000 - matches: - path: type: Exact value: /api/db-health backendRefs: - name: ai-api port: 8000 - matches: - path: type: Exact value: /api/whoami backendRefs: - name: ai-api port: 8000 - matches: - path: type: PathPrefix value: /api/conversations backendRefs: - name: ai-api port: 8000
This configuration file defines an Envoy Gateway SecurityPolicy named ai-api-jwt that secures the ai-api routing rules. It acts as an API gateway firewall, implementing Cross-Origin Resource Sharing (CORS) rules to safely accept web requests from your local Vite frontend variants. For authentication, it integrates directly with Keycloak to inspect incoming JSON Web Tokens (JWTs) and injects the verified user identifier into a custom x-authenticated-user header. By default, the policy locks down the system by blocking all unauthenticated or unmatched traffic completely. Finally, it uses flexible Common Expression Language (CEL) logic to authorize access to specific API endpoints exclusively for users carrying either the ai-user or ai-admin security roles.
This configuration file defines a Kubernetes NetworkPolicy named allow-only-envoy-to-fastapi that acts as a targeted network firewall inside the ai-platform namespace. It isolates your FastAPI pods labeled with app: ai-api by blocking all unapproved inbound (Ingress) traffic by default. The rule explicitly opens a secure channel only for network traffic originating from the ai-gateway pods running inside the envoy-gateway-system namespace. Finally, this trusted communication is restricted to pass exclusively over TCP port 8000 to protect your application layer from unauthorized internal access.
apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata: name: allow-only-envoy-to-fastapi namespace: ai-platform # Protects your application namespacespec: podSelector: matchLabels: app: ai-api # Protects your FastAPI pod policyTypes: - Ingress ingress: - from: - namespaceSelector: matchLabels: kubernetes.io/metadata.name: envoy-gateway-system podSelector: matchLabels: # Updated to match the exact label key visible in your output! gateway.envoyproxy.io/owning-gateway-name: ai-gateway ports: - protocol: TCP port: 8000 # Binds to your FastAPI application listening port
sanity check:
kubectl get all -n envoy-gateway-systemNAME READY STATUS RESTARTS AGEpod/envoy-ai-platform-ai-gateway-2621731d-57467b7796-jzth9 2/2 Running 2 (14d ago) 27dpod/envoy-gateway-674bd56d54-29ktt 1/1 Running 2 (14d ago) 27dNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEservice/envoy-ai-platform-ai-gateway-2621731d NodePort 10.96.33.241 <none> 80:32743/TCP 27dservice/envoy-gateway ClusterIP 10.96.34.123 <none> 18000/TCP,18001/TCP,18002/TCP,19001/TCP,9443/TCP 27dNAME READY UP-TO-DATE AVAILABLE AGEdeployment.apps/envoy-ai-platform-ai-gateway-2621731d 1/1 1 1 27ddeployment.apps/envoy-gateway 1/1 1 1 27dNAME DESIRED CURRENT READY AGEreplicaset.apps/envoy-ai-platform-ai-gateway-2621731d-57467b7796 1 1 1 27dreplicaset.apps/envoy-gateway-674bd56d54
Containerise ReAct-UI and inject the image into kind cluster and create a deployment:
cd ai-web-reactdocker build -t ai-web:1.0 .kind load docker-image ai-web:1.0 --name normal-clusterk apply -f ai-web-react/react-frontend-deployment.yaml
k get pods -n ai-platformNAME READY STATUS RESTARTS AGEai-api-7cfb6cc784-qgb9b 1/1 Running 0 2d18hai-postgres-74d4b84697-frdkt 1/1 Running 2 (3d5h ago) 7d4hai-web-6547f56fcb-z6s9m 1/1 Running 0 28h
Test via port forward:
kubectl port-forward -n ai-platform svc/ai-web 8082:80
Exposing it via Envoy gateway, not via port forward:
Exposing via portforward means creating an extra entry point. We need single entry point to everything at / via routes.
k apply -f gateway/ai-web-route.yaml
apiVersion: gateway.networking.k8s.io/v1kind: HTTPRoutemetadata: name: ai-web namespace: ai-platformspec: parentRefs: - name: ai-gateway rules: - matches: - path: type: PathPrefix value: / backendRefs: - name: ai-web port: 80
If you notice, my first question was ‘What is Kubernetes?’ followed by ‘Why should we use it?’ Without our PostgreSQL episodic memory integrated into the FastAPI backend, the LLM would have lost track and responded with a generic ‘What do you mean by it?’ However, because the historical context is continuously stored in and retrieved from Postgres, the full conversation history is preserved, merged, and sent to the LLM to deliver a perfectly context-aware response.
You can logout and log back in with another user like shalu and create history of chats. Both users do not have access to each others chats.
Building and deploying full blown ChatGPT with ReAct, PostgreSQL, FastAPI, Keycloak and Envoy… was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.