{"slug": "feature-flags-for-production-ai", "title": "Feature flags for production AI", "summary": "Pydantic Logfire managed variables give AI engineers a control plane for changing models, prompts, and tools in production without redeploying, enabling feature flags that are typed, versioned, and targetable. The tool supports A/B testing of AI experiences while balancing quality, cost, latency, and reliability, and is available via 'pip install logfire[variables]'.", "body_md": "AI makes software accessible to more people. One agent may serve an AI-native specialist and a first-time user who expects an ordinary request to work. It may support several languages, levels of domain expertise, and workflows that combine model responses with tools and application logic.\n\nA single default experience will rarely suit everyone. [Pydantic Logfire managed variables](https://pydantic.dev/docs/logfire/manage/managed-variables/) give AI engineers a control plane for changing the three primitives that shape an AI application: models, prompts, and tools. Here, feature flags are not limited to booleans. They are typed, versioned variables that teams can target, measure, and change without redeploying.\n\nThe foundation of A/B testing still applies: define alternatives, assign traffic, and compare outcomes. Generative AI expands what we can test and what we need to measure. Teams must find the right experience while balancing quality, cost, latency, and reliability.\n\nGrowth is now AI work\n\nGrowth engineering is now part of AI engineering. AI engineers run product experiments while building backends, frontends, and the analytics needed to understand how their work performs.\n\nThe result is more complex than a click or conversion. A production AI experiment should consider:\n\n- Answer quality and task completion\n- Cost per successful outcome\n- Latency and time to first useful response\n- Tool accuracy and retry behavior\n- Safety, reliability, and escalation\n- Performance across languages, cohorts, and levels of expertise\n\nThe questions are simple, even when the answers are not. Does it work? For whom? Under what conditions? How quickly? At what cost? Does it use the right tools, and can it fail safely?\n\nShipping faster can produce a better product as providers release models, frameworks add capabilities, and useful patterns emerge. A shorter feedback loop gives teams more chances to learn before the technology moves again.\n\nStart with three primitives\n\nAI teams can start with three things.\n\nInstall the managed variables extra before using these examples:\n\n```\npip install 'logfire[variables]'\n```\n\n-\n**Models** determine capabilities, speed, cost, context limits, and provider behavior. A string variable can route an opted-in group to a new model without changing application code:\n\n``` python\nimport logfire\n\nlogfire.configure()\nlogfire.instrument_pydantic_ai()\n\nmodel = logfire.var(\n    name='support_model',\n    type=str,\n    default='openai:gpt-5.2',\n)\n```\n\n-\n**Prompts** frame the task, provide domain guidance, and shape how the application communicates. A managed prompt can compare a guided experience with a concise one:\n\n```\ninstructions = logfire.var(\n    name='support_instructions',\n    type=str,\n    default='Explain the next step clearly and ask before taking action.',\n)\n```\n\n-\n**Tools** determine what the system can retrieve, calculate, or change. The variable should contain a typed policy, not executable code:\n\n``` python\nfrom typing import Literal\n\nfrom pydantic import BaseModel, Field\n\nclass ToolPolicy(BaseModel):\n    profile: Literal['answer_only', 'research', 'diagnostic']\n    max_tool_calls: int = Field(ge=0, le=8)\n\ntool_policy = logfire.var(\n    name='tool_policy',\n    type=ToolPolicy,\n    default=ToolPolicy(\n        profile='answer_only',\n        max_tool_calls=3,\n    ),\n)\n```\n\nThe application maps each profile to pre-approved tools and enforces the call limit. Tool code, credentials, authorization, and required approvals stay in code.\n\nThese primitives affect one another. A better prompt may make a smaller model viable. A stronger model may need fewer examples but call an expensive tool too often. Teams should test them together and connect each combination to product and operational results.\n\nPut variables to work\n\nDefining variables is only the start. The application must resolve and apply them during an agent run.\n\nThis example uses the user ID as a stable targeting key. It also supplies the tenant plan and workflow for conditional routing. The resolved tool policy becomes a limit that [Pydantic AI](https://pydantic.dev/docs/ai/overview/) enforces:\n\n``` python\nfrom pydantic_ai import Agent, UsageLimits\n\nAPPROVED_TOOLS = {\n    'answer_only': [],\n    'research': [support_search],\n    'diagnostic': [support_search, account_diagnostics],\n}\n\nasync def answer(user_id: str, tenant_plan: str, message: str) -> str:\n    attributes = {\n        'tenant_plan': tenant_plan,\n        'workflow': 'customer_support',\n    }\n\n    with (\n        model.get(targeting_key=user_id, attributes=attributes) as selected_model,\n        instructions.get(targeting_key=user_id, attributes=attributes) as selected_instructions,\n        tool_policy.get(targeting_key=user_id, attributes=attributes) as selected_policy,\n    ):\n        policy = selected_policy.value\n        agent = Agent(\n            selected_model.value,\n            instructions=selected_instructions.value,\n            tools=APPROVED_TOOLS[policy.profile],\n        )\n        result = await agent.run(\n            message,\n            usage_limits=UsageLimits(tool_calls_limit=policy.max_tool_calls),\n        )\n        return result.output\n```\n\nA high-volume support workflow might receive one research call. An approved diagnostic workflow might receive four. `UsageLimits`\n\nchecks the cap before it executes more tools, protecting cost and latency from a runaway loop.\n\nThe variable contexts also add the selected labels and versions to downstream spans. Engineers can see which model, prompt, and tool policy produced each result.\n\nEnter typed feature flags\n\nThe emphasis on **typed** matters. Pydantic began by helping developers turn untrusted data into validated objects. Managed variables bring the same idea to production configuration: [Logfire](/logfire) can change a value, but the application still declares its valid shape.\n\nVariables can hold:\n\n**Text** for prompts, instructions, and messages**Numbers** for token limits, thresholds, and budgets**Booleans** for on-or-off flags**Structured data** for dataclasses and Pydantic models**Templates** that combine runtime inputs with reusable fragments\n\nThe code default acts as a contract and a safety net. Logfire validates remote values against the declared type or JSON Schema. If a value is missing or invalid, the application uses its known-good default. Teams can change one primitive at a time or group a model, prompt, and tool policy in one Pydantic model so the whole experience changes together.\n\nTarget with live context\n\nVersions can receive labels such as `production`\n\n, `canary`\n\n, `control`\n\n, and `treatment`\n\n. Percentage routing controls how much traffic receives each label. A stable `targeting_key`\n\n, such as a user or tenant ID, keeps the assignment consistent across requests.\n\nSampling is only one option. [Conditional targeting rules](https://pydantic.dev/docs/logfire/manage/managed-variables/targeting/) can select an experience using explicit resolution attributes, OpenTelemetry resource attributes, or request baggage. Useful attributes include language, plan, region, workflow, service version, and beta enrollment. Logfire checks rules in order and uses the first match.\n\nThis gives engineers more control than sending 10% of all traffic to a treatment. A team might offer a guided French prompt only to opted-in users during onboarding. During a capacity incident, it might route one tenant plan to a cheaper model. Each resolution creates a span, and the chosen label and version flow into the work that follows.\n\nManage the lifecycle\n\nAfter defining the variables, push their metadata and generated schemas to Logfire:\n\n```\nif __name__ == '__main__':\n    logfire.variables_push()\n```\n\nThis command requires a `LOGFIRE_API_KEY`\n\nwith the `project:write_variables`\n\nscope. The runtime application uses the separate `project:read_variables`\n\nscope.\n\nThe rest of the lifecycle lives in Logfire. Teams create immutable versions, move labels, and change rollout percentages or targeting rules. To roll back, point a label at the last known-good version. No application deploy is required.\n\nConnect changes to evidence\n\nFeature flags determine which experience runs. Observability shows what happened next. Logfire connects variable versions to traces and [SQL-backed dashboards](https://pydantic.dev/docs/logfire/observe/dashboards/), so product and system results share the same OpenTelemetry context.\n\nTwo dashboards make a useful start:\n\n**Product results:** task completion, abandonment, repeat use, escalation, and feedback, grouped by variable label, version, and cohort.**System results:** latency, tokens, inference cost, tool calls, retries, and errors for the same groups.\n\nThe first shows whether the experience helped. The second shows what it cost. Alerts can identify a sustained problem and link an engineer to the relevant traces and configuration.\n\nTogether, these parts form a continuous evidence loop. Production traces reveal failures worth investigating. Teams preserve those cases, assess a proposed change, and use feature flags to return a new model, prompt, or tool configuration to production.\n\nPersonalization in practice\n\nSupport across languages\n\nA customer asking for help with an unfamiliar task may need translated instructions, guided steps, or a direct answer. Teams can compare prompts written for each language, route ambiguous requests to a stronger model, and vary tool access. Completion, clarification turns, escalation, latency, and cost show whether the added guidance helps.\n\nKnowledge by role\n\nOne user may know the specialist terms and want the source immediately. Another needs acronyms and assumptions explained. Teams can target configurations by organization, role, or selected expertise. They can vary domain instructions, retrieval sources, citation requirements, response depth, and analytical tools. Task completion, follow-up questions, and corrections from subject-matter experts point to the next experiment.\n\nSaaS agents by tenant\n\nAn agent embedded in a SaaS product may investigate problems across tenants. One tenant might generate large contexts, repeated tool calls, and high inference costs. Teams can test model routes, context limits, summarization prompts, and tool policies for that tenant before changing the experience for everyone.\n\nLogfire traces connect tenant context with tokens, cost, latency, retries, and results. A SQL alert can detect sustained abnormal usage. An event-triggered judge can then produce a trace-linked explanation for an engineer instead of returning only an `expensive`\n\nlabel.\n\nWhy embedded flags matter\n\nFeature flags belong inside the AI engineering and observability workflow. This keeps the change, the targeting decision, and the result connected. Engineers can release an experience to the right group, observe its product and system effects, notify the right people, and roll it back without losing attribution.\n\nThe boundary remains important. Code governs schemas, business logic, permissions, and safety rules. Managed variables govern the parts of models, prompts, and tool policies that should change at runtime. OpenTelemetry connects each decision to what follows.\n\nThis control plane can support more complex workflows. A model variable can select a [Logfire AI Gateway](https://pydantic.dev/docs/logfire/manage/ai-gateway/) routing group with provider failover and load balancing. Targeting can use OpenTelemetry attributes from real production traffic. An internal application can add roster or group membership as routing context.\n\nA feature flag never grants authorization. Code still decides what a user or agent may do. Managed variables make the behavior that should change visible, versioned, and reversible. They give AI engineers, and authorized agents working with them, the context needed to investigate results and improve production systems safely.", "url": "https://wpnews.pro/news/feature-flags-for-production-ai", "canonical_source": "https://pydantic.dev/articles/feature-flags-for-production-ai", "published_at": "2026-08-18 09:00:00+00:00", "updated_at": "2026-08-18 18:11:36.293423+00:00", "lang": "en", "topics": ["ai-tools", "ai-infrastructure", "mlops", "ai-products"], "entities": ["Pydantic Logfire", "Pydantic"], "alternates": {"html": "https://wpnews.pro/news/feature-flags-for-production-ai", "markdown": "https://wpnews.pro/news/feature-flags-for-production-ai.md", "text": "https://wpnews.pro/news/feature-flags-for-production-ai.txt", "jsonld": "https://wpnews.pro/news/feature-flags-for-production-ai.jsonld"}}