Lean AI in LangBear

LangBear is fully bootstrapped. This means no investors' money to burn, no big budgets. All funding has been coming from my main job's income. This is why I made a decision early on to stay AI provider-agnostic as much as possible to have the flexibility to migrate to cheaper options.

At first, I tried LangChain.js. It was too heavy and was burdensome to maintain. Too many dependencies. The API wasn't stable and oftentimes broke when I had to bump the version for security fixes.

I decided to write a very lightweight interface that would satisfy all use cases of the app.

I present it to you with all its glory as it is today:

import type z from "zod/v3";

export type AIProvider = {

  generateText<T extends z.ZodTypeAny>(
    prompt: string,
    schema: T,
    options?: AIProviderGenerateTextOptions,
  ): Promise<z.infer<T>>;

  transcribeAudio(filePath: string): Promise<string>;
};

export type AIProviderThinkingConfig =
  | {
      // 0 disables thinking, -1 lets the provider choose automatically
      thinkingBudget: 0 | -1;
      thinkingLevel?: never;
    }
  | {
      thinkingBudget?: never;
      thinkingLevel: "MINIMAL" | "LOW" | "MEDIUM" | "HIGH";
    };

export type AIProviderGenerateTextOptions = {
  model?: string;
  thinkingConfig?: AIProviderThinkingConfig;
};

So far, I've already tried OpenAI, Gemini, Anthropic, OpenRouter and switching between providers only takes 1 commit.

In the code, you would use it like this:

function translateText(
  aiProvider: AIProvider,
  model: string,
  text: string,
  targetLang: string,
): Promise<string> {
  const { translatedText } = await aiProvider.generateText(
    getTranslateTextPrompt(text, targetLang),
    z.object({ translatedText: z.string() }),
    {
      model,
    },
  );

  return translatedText;
}

await translateText(geminiProvider, "gemini-3.5-flash", "hej", "en")
// hello

Implementation of this interface is trivial. Here's a partial implementation for the Gemini provider for generateText

function:

async function generateText<T extends z.ZodTypeAny>(
  prompt: string,
  schema: T,
  { geminiApiKey, model, thinkingConfig }: GenerateTextOptions,
): Promise<z.infer<T>> {
  const gemini = new GoogleGenAI({
    apiKey: geminiApiKey,
  });

  const config: GenerateContentConfig = {
    responseMimeType: "application/json",
    responseJsonSchema: zodToJsonSchema(schema),
  };
  const geminiThinkingConfig = toGeminiThinkingConfig(thinkingConfig);

  if (geminiThinkingConfig) {
    config.thinkingConfig = geminiThinkingConfig;
  }

  const response = await gemini.models.generateContent({
    model,
    contents: prompt,
    config,
  });

  if (!response.text) {
    throw new Error("no content");
  }

  return schema.parse(JSON.parse(response.text));
}