Go Completely Offline: Build a Privacy-First Personal Finance Assistant with LiteRT and Gemma 4

A developer has created a privacy-first personal finance assistant that runs entirely offline in the browser using LiteRT's LM Web API and the Gemma 4 E2B small language model. The application caches model weights in the browser's Cache Storage and processes transactions on-device with IndexedDB, ensuring complete privacy and zero token costs. The project, available on GitHub, demonstrates a three-phase approach including arithmetic grounding to prevent hallucination and conversation management to stay within browser token limits.

Imagine having a personal financial planner that analyzes your sensitive transaction history, flags spending anomalies, and suggests budget optimizations—all while running 100% offline, directly inside your web browser. In this guide, we will build an offline Retrieval-Augmented Generation RAG assistant powered by the LiteRT LM Web API and the highly efficient, on-device Gemma 4 E2B small language model. By caching the model weights directly in the browser's Cache Storage, your application will achieve complete privacy, zero token costs, and genuine offline capability. While Angular and TailwindCSS provide the user interface and styling, the LiteRT LM Web API and the on-device small model power this demo's AI capabilities. npm i --save-exact @litert-lm/core tailwindcss postcss @tailwindcss/postcss jsonrepair dexie npm i --save-exact --save-dev angular-eslint husky lint-staged serve @commitlint/cli We install additional dependencies for on-device AI, CSS Styling, JSON response streaming, and IndexedDB storage. The dev dependencies help automatically catch code smells, enforce code quality, and serve the production bundle. We designed this application to process transactions entirely on-device, ensuring financial data privacy inside the browser. The process operates in three key phases: Model and Data Storage: First, the application downloads and caches the Gemma 4 E2B model weights directly in the browser's Cache Storage so they are available offline. When users log expenses, they are saved locally in IndexedDB. Arithmetic Grounding: Small models hallucinate when processing arithmetic queries, so we aggregate expenses into a monthly total, a daily total, and an expense-by-category breakdown. We supply the precomputed values to the Gemma WebGPU engine as ground truth, along with the user query. Conversation Management: To prevent exceeding browser token limits, the application resets the conversation after three turns but retains a client-side memory of the last two queries. On reset, the application re-primes the model with immutable financial grounding data before replaying those queries. This maintains conversational continuity while keeping memory usage within on-device limits. Next, let's look at how we instantiate this LiteRT LM engine, define the insight service, and design the Angular user interfaces to generate these insights. While the full codebase is available in the NG Personal Finance Assistant https://github.com/railsstudent/ng-on-device-expense-tracker repository, our application relies on an IndexedDB database to track expenses offline. You can find the database implementation in the service file https://github.com/railsstudent/ng-on-device-expense-tracker/blob/main/src/app/core/services/database.service.ts , which is injected via an injection token https://github.com/railsstudent/ng-on-device-expense-tracker/blob/main/src/app/core/consts/app-database.const.ts and initialized at startup using provideAppInitializer https://github.com/railsstudent/ng-on-device-expense-tracker/blob/main/src/app/app.config.ts L18 . With this local database configured to log expenses, we can now focus on the AI integration. The following sections illustrate how to cache the Gemma 4 weights and leverage Angular 22's reactive features to generate secure, local insights directly from this transaction data. Running LLMs locally in the browser requires two steps: Caching the Weights: Storing the model weights ~2GB in the browser's Cache Storage so the application can run 100% offline. Engine Bootstrapping: Creating a single, reusable instance of the LiteRT LM engine so we can stream responses with WebGPU speed and avoid costly re-initialization latencies. When discussing on-device AI, people often assume offline operation, zero server dependency, and total privacy. However, this is only partially true for Web AI. Most tutorials make a fetch request to retrieve the Gemma model from HuggingFace, which works well when online. This fails if the user's connection is unstable or unavailable. This illustrates the benefit of caching a small Gemma model in Cache Storage. The device only needs to be online once to download the model, and the application runs 100% offline. The demo has a download button for users to download the Gemma 4 E2B model that is around 2GB. After a successful download, go to Application Storage Usage to verify that the model occupies 2GB in storage. Let's implement this logic in AiModelCacheService : js export const AI CACHE NAME = 'JMWebAIModels'; export const DEFAULT MODEL FILENAME = 'gemma-4-E2B-it-web.litertlm'; export const GEMMA MODEL URL = https://huggingface.co/litert-community/gemma-4-E2B-it-litert-lm/resolve/main/${DEFAULT MODEL FILENAME} ; @Service export class AiModelCacheService { readonly modelUrl = GEMMA MODEL URL; readonly state = signal { status: 'not-downloaded' } ; async downloadModel : Promise<string { if this. state .status === 'cached' { return await this.getModelUrl ; } this. state.set { status: 'downloading' } ; const localBlobUrl = await FileProxyCache.loadFromURL this. modelUrl ; this. state.set { status: 'cached' } ; return localBlobUrl; } async getModelUrl : Promise<string | null { return await FileProxyCache.loadFromURL this. modelUrl ; } } The AiModelCacheService class leverages FileProxyCache an open-source utility by Jason Mayes designed specifically for caching large Web AI weights . Instead of using a standard browser fetch which can crash or fail with large ~2GB assets , FileProxyCache handles chunked downloads and saves the Gemma model directly to the browser's Cache Storage. In subsequent runs, it instantly loads the cached local file and passes its blob URL to the LiteRT engine, completing the offline capability. The application has a single instance of the LiteRT LM engine. Initializing and destroying an engine for an individual query is an expensive operation. This introduces latency while waiting for the engine to become available, start a new conversation, and accept a message. We lazily initialize the engine using GemmaEngineService , which we then inject into our insight service in the next section. @Service export class GemmaEngineService { readonly cacheService = inject AiModelCacheService ; engine: Engine | null = null; private async initializeEngine : Promise<Engine { const localBlobUrl = await this. cacheService.getModelUrl ; const instance = await Engine.create { model: localBlobUrl, mainExecutorSettings: { maxNumTokens: 4096, }, } ; this. engine = instance; return instance; } getEngine : Promise<Engine { if this. engine { return Promise.resolve this. engine ; } return this.initializeEngine ; } ... other methods and lifecycle methods ... } If initialized, the method returns the engine immediately. Otherwise, the initializeEngine helper method loads the model from Cache Storage, constructs the LiteRT LM engine, and returns it. We define the system, priming, and user prompts to analyze and derive deep insights into personal finance and spending. You can view our exact prompts here https://github.com/railsstudent/ng-on-device-expense-tracker/blob/main/src/app/core/consts/insight-prompt.const.ts . To ensure high-quality, structured output from Gemma 4, the model must return a JSON response adhering to the InsightsResponse interface. You can view the response https://github.com/railsstudent/ng-on-device-expense-tracker/blob/main/src/app/shared/interfaces/insights-response.interface.ts and the Insight interface here https://github.com/railsstudent/ng-on-device-expense-tracker/blob/main/src/app/shared/interfaces/insight.interface.ts . Let's implement our InsightService to coordinate the streaming process: @Service export class InsightService { readonly engineService = inject GemmaEngineService ; conversation: Conversation | null = null; lastPrimedExpenses: Expense | null = null; turnsCount = 0; previousQueries: string = ; async streamInsights userQuery: string, expenses: Expense : AsyncGenerator<InsightsResponse { const isContextDifferent = this. lastPrimedExpenses == expenses; const isContextExhausted = this. turnsCount = 3; // Reset & prime context when threshold is met if this. conversation || isContextDifferent || isContextExhausted { await this.primeContext expenses, isContextExhausted ? this. previousQueries : undefined ; } const userPrompt = INSIGHTS USER PROMPT userQuery ; const stream = await this. conversation.sendMessageStreaming userPrompt ; this. previousQueries.push userQuery ; this. turnsCount = this. turnsCount + 1; // Yield repaired, streamed JSON chunks using jsonrepair yield this.processStream stream ; } / processStream, lifecycle, and other helper methods / } Our primeContext method formats transaction data, computes high-level spending statistics, and appends them to the query context—guaranteeing that Gemma 4 generates a reliable, grounded JSON response. The processStream method returns an asynchronous generator that iterates over the stream to extract each chunk's content. Because LLM response streaming delivers data fragment-by-fragment, the raw buffer is frequently incomplete e.g., an open brace or unclosed bracket . Attempting to run standard JSON.parse on these partial chunks would immediately throw a syntax error. To solve this, the jsonrepair library dynamically patches and closes the partial JSON string on the fly, allowing the application to successfully parse and render UI updates incrementally. HistoryInsightsService acts as a clean facade between our UI component and the core InsightService , letting us easily query expenses and stream insights based on user questions. Our component injects this service to trigger and render the JSON response asynchronously. Source code of HistoryInsightsService https://github.com/railsstudent/ng-on-device-expense-tracker/blob/main/src/app/features/history-insights/components/history-insights/services/history-insights.service.ts HistoryInsightsComponent comprises child components and a pipe. However, the primary method that communicates with Gemma 4 to generate a list of insights is onAskGemma . @Component { selector: 'app-history-insights', ... } export default class HistoryInsightsComponent { protected readonly vm = inject HistoryInsightsService ; readonly expenses = signal<Expense ; readonly streamingResponse = signal<InsightsResponse { insights: } ; async onAskGemma query: string : Promise<void { const trimmed = query.trim ; if trimmed { return; } this.streamingResponse.set { insights: } ; // The expenses are populated after a search query // Consume the stream generator and update the UI signal incrementally const generator = this.vm.streamInsights trimmed, this.expenses ; for await const response of generator { this.streamingResponse.set response ; } } } The onAskGemma event handler invokes HistoryInsightsService 's streamInsights method to retrieve the streamed response and incrementally updates the streamingResponse signal. The derived aiState signal computes the current status, captures any errors, and binds the values directly to the child component to render the streamed response in real-time. The full listing of HistoryInsightsComponent https://github.com/railsstudent/ng-on-device-expense-tracker/blob/main/src/app/features/history-insights/components/history-insights/history-insights.component.ts This concludes the journey of building an on-device personal finance assistant using the LiteRT LM Web SDK, the Gemma 4 E2B small-scale model, and Angular. This blog post shows how to build an offline RAG-based application by caching the model weights locally. Even when Wi-Fi is turned off, the personal finance assistant provides insights for user queries. We mitigated the lack of tool-calling capability in the API and the mathematical hallucinations of small models by providing precomputed aggregated data as the LLM context. This prevents incorrect "60 + 40.5 = 200.2" expressions in the final results. Engineers can build web applications with local models to perform financial analysis without leaking personal data to the training datasets of AI vendors.