{"slug": "go-completely-offline-build-a-privacy-first-personal-finance-assistant-with-and", "title": "Go Completely Offline: Build a Privacy-First Personal Finance Assistant with LiteRT and Gemma 4", "summary": "A developer has created a privacy-first personal finance assistant that runs entirely offline in the browser using LiteRT's LM Web API and the Gemma 4 E2B small language model. The application caches model weights in the browser's Cache Storage and processes transactions on-device with IndexedDB, ensuring complete privacy and zero token costs. The project, available on GitHub, demonstrates a three-phase approach including arithmetic grounding to prevent hallucination and conversation management to stay within browser token limits.", "body_md": "Imagine having a personal financial planner that analyzes your sensitive transaction history, flags spending anomalies, and suggests budget optimizations—all while running 100% offline, directly inside your web browser.\n\nIn this guide, we will build an offline Retrieval-Augmented Generation (RAG) assistant powered by the LiteRT LM Web API and the highly efficient, on-device Gemma 4 E2B small language model. By caching the model weights directly in the browser's Cache Storage, your application will achieve complete privacy, zero token costs, and genuine offline capability.\n\nWhile Angular and TailwindCSS provide the user interface and styling, the LiteRT LM Web API and the on-device small model power this demo's AI capabilities.\n\n```\nnpm i --save-exact @litert-lm/core tailwindcss postcss @tailwindcss/postcss jsonrepair dexie\nnpm i --save-exact --save-dev angular-eslint husky lint-staged serve @commitlint/cli\n```\n\nWe install additional dependencies for on-device AI, CSS Styling, JSON response streaming, and IndexedDB storage. The dev dependencies help automatically catch code smells, enforce code quality, and serve the production bundle.\n\nWe designed this application to process transactions entirely on-device, ensuring financial data privacy inside the browser. The process operates in three key phases:\n\n**Model and Data Storage:** First, the application downloads and caches the Gemma 4 E2B model weights directly in the browser's Cache Storage so they are available offline. When users log expenses, they are saved locally in IndexedDB.\n\n**Arithmetic Grounding:** Small models hallucinate when processing arithmetic queries, so we aggregate expenses into a monthly total, a daily total, and an expense-by-category breakdown. We supply the precomputed values to the Gemma WebGPU engine as ground truth, along with the user query.\n\n**Conversation Management:** To prevent exceeding browser token limits, the application resets the conversation after three turns but retains a client-side memory of the last two queries. On reset, the application re-primes the model with immutable financial grounding data before replaying those queries. This maintains conversational continuity while keeping memory usage within on-device limits.\n\nNext, let's look at how we instantiate this LiteRT LM engine, define the insight service, and design the Angular user interfaces to generate these insights.\n\nWhile the full codebase is available in the [NG Personal Finance Assistant](https://github.com/railsstudent/ng-on-device-expense-tracker) repository, our application relies on an IndexedDB database to track expenses offline. You can find the database implementation in the [service file](https://github.com/railsstudent/ng-on-device-expense-tracker/blob/main/src/app/core/services/database.service.ts), which is injected via an [injection token](https://github.com/railsstudent/ng-on-device-expense-tracker/blob/main/src/app/core/consts/app-database.const.ts) and initialized at startup using [provideAppInitializer](https://github.com/railsstudent/ng-on-device-expense-tracker/blob/main/src/app/app.config.ts#L18).\n\nWith this local database configured to log expenses, we can now focus on the AI integration. The following sections illustrate how to cache the Gemma 4 weights and leverage Angular 22's reactive features to generate secure, local insights directly from this transaction data.\n\nRunning LLMs locally in the browser requires two steps:\n\nCaching the Weights: Storing the model weights (~2GB) in the browser's Cache Storage so the application can run 100% offline.\n\nEngine Bootstrapping: Creating a single, reusable instance of the LiteRT LM engine so we can stream responses with WebGPU speed and avoid costly re-initialization latencies.\n\nWhen discussing on-device AI, people often assume offline operation, zero server dependency, and total privacy. However, this is only partially true for Web AI. Most tutorials make a `fetch`\n\nrequest to retrieve the Gemma model from HuggingFace, which works well when online. This fails if the user's connection is unstable or unavailable.\n\nThis illustrates the benefit of caching a small Gemma model in Cache Storage. The device only needs to be online once to download the model, and the application runs 100% offline.\n\nThe demo has a download button for users to download the Gemma 4 E2B model that is around 2GB.\n\nAfter a successful download, go to `Application`\n\n> `Storage`\n\n> `Usage`\n\nto verify that the model occupies 2GB in storage.\n\nLet's implement this logic in `AiModelCacheService`\n\n:\n\n``` js\nexport const AI_CACHE_NAME = 'JMWebAIModels';\nexport const DEFAULT_MODEL_FILENAME = 'gemma-4-E2B-it-web.litertlm';\nexport const GEMMA_MODEL_URL = `https://huggingface.co/litert-community/gemma-4-E2B-it-litert-lm/resolve/main/${DEFAULT_MODEL_FILENAME}`;\n\n@Service()\nexport class AiModelCacheService {\n  readonly #modelUrl = GEMMA_MODEL_URL;\n  readonly #state = signal({ status: 'not-downloaded' });\n\n  async downloadModel(): Promise<string> {\n    if (this.#state().status === 'cached') {\n      return await this.getModelUrl();\n    }\n\n    this.#state.set({ status: 'downloading' });\n    const localBlobUrl = await FileProxyCache.loadFromURL(this.#modelUrl);\n    this.#state.set({ status: 'cached' });\n\n    return localBlobUrl;\n  }\n\n  async getModelUrl(): Promise<string | null> {\n    return await FileProxyCache.loadFromURL(this.#modelUrl);      \n  }\n}\n```\n\nThe `AiModelCacheService`\n\nclass leverages `FileProxyCache`\n\n(an open-source utility by Jason Mayes designed specifically for caching large Web AI weights). Instead of using a standard browser fetch (which can crash or fail with large ~2GB assets), `FileProxyCache`\n\nhandles chunked downloads and saves the Gemma model directly to the browser's Cache Storage. In subsequent runs, it instantly loads the cached local file and passes its blob URL to the LiteRT engine, completing the offline capability.\n\nThe application has a single instance of the LiteRT LM engine. Initializing and destroying an engine for an individual query is an expensive operation. This introduces latency while waiting for the engine to become available, start a new conversation, and accept a message.\n\nWe lazily initialize the engine using `GemmaEngineService`\n\n, which we then inject into our insight service in the next section.\n\n```\n@Service()\nexport class GemmaEngineService {\n  readonly #cacheService = inject(AiModelCacheService);\n  #engine: Engine | null = null;\n\n  private async initializeEngine(): Promise<Engine> {\n    const localBlobUrl = await this.#cacheService.getModelUrl();\n    const instance = await Engine.create({\n      model: localBlobUrl,\n      mainExecutorSettings: {\n        maxNumTokens: 4096,\n      },\n    });\n\n    this.#engine = instance;\n    return instance;\n  }\n\n  getEngine(): Promise<Engine> {\n    if (this.#engine) {\n      return Promise.resolve(this.#engine);\n    }\n\n    return this.initializeEngine();\n  }\n\n  ... other methods and lifecycle methods ...\n}\n```\n\nIf initialized, the method returns the engine immediately. Otherwise, the `initializeEngine`\n\nhelper method loads the model from Cache Storage, constructs the LiteRT LM engine, and returns it.\n\nWe define the system, priming, and user prompts to analyze and derive deep insights into personal finance and spending. You can [view our exact prompts here](https://github.com/railsstudent/ng-on-device-expense-tracker/blob/main/src/app/core/consts/insight-prompt.const.ts).\n\nTo ensure high-quality, structured output from Gemma 4, the model must return a JSON response adhering to the `InsightsResponse`\n\ninterface. You can [view the response](https://github.com/railsstudent/ng-on-device-expense-tracker/blob/main/src/app/shared/interfaces/insights-response.interface.ts) and the [Insight interface here](https://github.com/railsstudent/ng-on-device-expense-tracker/blob/main/src/app/shared/interfaces/insight.interface.ts).\n\nLet's implement our `InsightService`\n\nto coordinate the streaming process:\n\n```\n@Service()\nexport class InsightService {\n  readonly #engineService = inject(GemmaEngineService);\n\n  #conversation: Conversation | null = null;\n  #lastPrimedExpenses: Expense[] | null = null;\n  #turnsCount = 0;\n  #previousQueries: string[] = [];\n\n  async *streamInsights(userQuery: string, expenses: Expense[]): AsyncGenerator<InsightsResponse> {\n\n    const isContextDifferent = this.#lastPrimedExpenses !== expenses;\n    const isContextExhausted = this.#turnsCount >= 3;\n\n    // Reset & prime context when threshold is met\n    if (!this.#conversation || isContextDifferent || isContextExhausted) {\n      await this.primeContext(expenses, isContextExhausted ? this.#previousQueries : undefined);\n    }\n\n    const userPrompt = INSIGHTS_USER_PROMPT(userQuery);\n    const stream = await this.#conversation.sendMessageStreaming(userPrompt);\n\n    this.#previousQueries.push(userQuery);\n    this.#turnsCount = this.#turnsCount + 1;\n\n    // Yield repaired, streamed JSON chunks (using jsonrepair)       \n    yield* this.processStream(stream);\n  }\n\n  /* processStream, lifecycle, and other helper methods */\n}\n```\n\nOur `primeContext`\n\nmethod formats transaction data, computes high-level spending statistics, and appends them to the query context—guaranteeing that Gemma 4 generates a reliable, grounded JSON response.\n\nThe `processStream`\n\nmethod returns an asynchronous generator that iterates over the stream to extract each chunk's content. Because LLM response streaming delivers data fragment-by-fragment, the raw buffer is frequently incomplete (e.g., an open brace or unclosed bracket). Attempting to run standard `JSON.parse()`\n\non these partial chunks would immediately throw a syntax error. To solve this, the `jsonrepair`\n\nlibrary dynamically patches and closes the partial JSON string on the fly, allowing the application to successfully parse and render UI updates incrementally.\n\n`HistoryInsightsService`\n\nacts as a clean facade between our UI component and the core `InsightService`\n\n, letting us easily query expenses and stream insights based on user questions.\n\nOur component injects this service to trigger and render the JSON response asynchronously.\n\n[Source code of HistoryInsightsService](https://github.com/railsstudent/ng-on-device-expense-tracker/blob/main/src/app/features/history-insights/components/history-insights/services/history-insights.service.ts)\n\n`HistoryInsightsComponent`\n\ncomprises child components and a pipe. However, the primary method that communicates with Gemma 4 to generate a list of insights is `onAskGemma`\n\n.\n\n```\n@Component({\n  selector: 'app-history-insights',\n  ...\n})\nexport default class HistoryInsightsComponent {\n  protected readonly vm = inject(HistoryInsightsService);\n\n  readonly expenses = signal<Expense[]>([]);\n  readonly streamingResponse = signal<InsightsResponse>({ insights: [] });\n\n  async onAskGemma(query: string): Promise<void> {\n    const trimmed = query.trim();\n    if (!trimmed) {\n      return;\n    }\n\n    this.streamingResponse.set({ insights: [] });\n\n    // The expenses are populated after a search query\n    // Consume the stream generator and update the UI signal incrementally\n    const generator = this.vm.streamInsights(trimmed, this.expenses());\n    for await (const response of generator) {\n      this.streamingResponse.set(response);\n    }\n  }\n}\n```\n\nThe `onAskGemma`\n\nevent handler invokes `HistoryInsightsService`\n\n's `streamInsights`\n\nmethod to retrieve the streamed response and incrementally updates the `streamingResponse`\n\nsignal. The derived `aiState`\n\nsignal computes the current status, captures any errors, and binds the values directly to the child component to render the streamed response in real-time.\n\n[The full listing of HistoryInsightsComponent](https://github.com/railsstudent/ng-on-device-expense-tracker/blob/main/src/app/features/history-insights/components/history-insights/history-insights.component.ts)\n\nThis concludes the journey of building an on-device personal finance assistant using the LiteRT LM Web SDK, the Gemma 4 E2B small-scale model, and Angular. This blog post shows how to build an offline RAG-based application by caching the model weights locally. Even when Wi-Fi is turned off, the personal finance assistant provides insights for user queries.\n\nWe mitigated the lack of tool-calling capability in the API and the mathematical hallucinations of small models by providing precomputed aggregated data as the LLM context. This prevents incorrect \"60 + 40.5 = 200.2\" expressions in the final results.\n\nEngineers can build web applications with local models to perform financial analysis without leaking personal data to the training datasets of AI vendors.", "url": "https://wpnews.pro/news/go-completely-offline-build-a-privacy-first-personal-finance-assistant-with-and", "canonical_source": "https://dev.to/railsstudent/go-completely-offline-build-a-privacy-first-personal-finance-assistant-with-litert-and-gemma-4-227l", "published_at": "2026-08-27 01:07:01+00:00", "updated_at": "2026-08-27 01:18:25.571418+00:00", "lang": "en", "topics": ["large-language-models", "ai-products", "developer-tools", "ai-infrastructure"], "entities": ["LiteRT", "Gemma 4", "Angular", "TailwindCSS", "IndexedDB", "GitHub", "Hugging Face"], "alternates": {"html": "https://wpnews.pro/news/go-completely-offline-build-a-privacy-first-personal-finance-assistant-with-and", "markdown": "https://wpnews.pro/news/go-completely-offline-build-a-privacy-first-personal-finance-assistant-with-and.md", "text": "https://wpnews.pro/news/go-completely-offline-build-a-privacy-first-personal-finance-assistant-with-and.txt", "jsonld": "https://wpnews.pro/news/go-completely-offline-build-a-privacy-first-personal-finance-assistant-with-and.jsonld"}}