Outlines – Structured Outputs for LLMs

The .txt team released Outlines, an open-source library that guarantees structured LLM outputs during generation rather than fixing them afterward with parsing or regex, and it is trusted by NVIDIA, Cohere, HuggingFace and vLLM. Outlines works across OpenAI, Ollama, vLLM and other providers with the same code, letting developers specify an output type such as Literal["Yes", "No"], int, or a Pydantic model via model(prompt, output_type). The .txt API is in early access, and the team is working with select partners on new interfaces to structured generation for XML, FHIR, custom schemas and grammars.

🗒️ Structured outputs for LLMs 🗒️ Made with ❤👷️ by the team at .txt https://dottxt.co Trusted by NVIDIA, Cohere, HuggingFace, vLLM, etc. The .txt API is currently in early access. Request access here → https://h1xbpbfsf0w.typeform.com/to/fwQNWmS8?utm source=github&utm medium=organic&utm campaign=outlines We're working with select partners to develop new interfaces to structured generation. Need XML, FHIR, custom schemas or grammars? Let's talk. Audit your schema: share one schema, we show you what breaks under generation, the constraints that fix it, and compliance rates before and after. Sign up here https://h1xbpbfsf0w.typeform.com/to/rtFUraA2?typeform . LLMs are powerful but their outputs are unpredictable. Most solutions attempt to fix bad outputs after generation using parsing, regex, or fragile code that breaks easily. Outlines guarantees structured outputs during generation — directly from any LLM. - Works with any model - Same code runs across OpenAI, Ollama, vLLM, and more - Simple integration - Just pass your desired output type: model prompt, output type - Guaranteed valid structure - No more parsing headaches or broken JSON - Provider independence - Switch models without changing code Outlines follows a simple pattern that mirrors Python's own type system. Simply specify the desired output type, and Outlines will ensure your data matches that structure exactly: - For a yes/no response, use Literal "Yes", "No" - For numerical values, use int - For complex objects, define a structure with a Pydantic model https://docs.pydantic.dev/latest/ Getting started with outlines is simple: pip install outlines python import outlines from transformers import AutoTokenizer, AutoModelForCausalLM MODEL NAME = "microsoft/Phi-3-mini-4k-instruct" model = outlines.from transformers AutoModelForCausalLM.from pretrained MODEL NAME, device map="auto" , AutoTokenizer.from pretrained MODEL NAME python from typing import Literal from pydantic import BaseModel Simple classification sentiment = model "Analyze: 'This product completely changed my life '", Literal "Positive", "Negative", "Neutral" print sentiment "Positive" Extract specific types temperature = model "What's the boiling point of water in Celsius?", int print temperature 100 python from pydantic import BaseModel from enum import Enum class Rating Enum : poor = 1 fair = 2 good = 3 excellent = 4 class ProductReview BaseModel : rating: Rating pros: list str cons: list str summary: str review = model "Review: The XPS 13 has great battery life and a stunning display, but it runs hot and the webcam is poor quality.", ProductReview, max new tokens=200, review = ProductReview.model validate json review print f"Rating: {review.rating.name}" "Rating: good" print f"Pros: {review.pros}" "Pros: 'great battery life', 'stunning display' " print f"Summary: {review.summary}" "Summary: Good laptop with great display but thermal issues" Here are production-ready examples showing how Outlines solves common problems: 🙋♂️ Customer Support Triage This example shows how to convert a free-form customer email into a structured service ticket. By parsing attributes like priority, category, and escalation flags, the code enables automated routing and handling of support issues. python import outlines from enum import Enum from pydantic import BaseModel from transformers import AutoTokenizer, AutoModelForCausalLM from typing import List MODEL NAME = "microsoft/Phi-3-mini-4k-instruct" model = outlines.from transformers AutoModelForCausalLM.from pretrained MODEL NAME, device map="auto" , AutoTokenizer.from pretrained MODEL NAME def alert manager ticket : print "Alert ", ticket class TicketPriority str, Enum : low = "low" medium = "medium" high = "high" urgent = "urgent" class ServiceTicket BaseModel : priority: TicketPriority category: str requires manager: bool summary: str action items: List str customer email = """ Subject: URGENT - Cannot access my account after payment I paid for the premium plan 3 hours ago and still can't access any features. I've tried logging out and back in multiple times. This is unacceptable as I have a client presentation in an hour and need the analytics dashboard. Please fix this immediately or refund my payment. """ prompt = f""" <|im start| user Analyze this customer email: {customer email} <|im end| <|im start| assistant """ ticket = model prompt, ServiceTicket, max new tokens=500 Use structured data to route the ticket ticket = ServiceTicket.model validate json ticket if ticket.priority == "urgent" or ticket.requires manager: alert manager ticket 📦 E-commerce product categorization This use case demonstrates how outlines can transform product descriptions into structured categorization data e.g., main category, sub-category, and attributes to streamline tasks such as inventory management. Each product description is processed automatically, reducing manual categorization overhead. python import outlines from pydantic import BaseModel from transformers import AutoTokenizer, AutoModelForCausalLM from typing import List, Optional MODEL NAME = "microsoft/Phi-3-mini-4k-instruct" model = outlines.from transformers AutoModelForCausalLM.from pretrained MODEL NAME, device map="auto" , AutoTokenizer.from pretrained MODEL NAME def update inventory product, category, sub category : print f"Updated {product.split ',' 0 } in category {category}/{sub category}" class ProductCategory BaseModel : main category: str sub category: str attributes: List str brand match: Optional str Process product descriptions in batches product descriptions = "Apple iPhone 15 Pro Max 256GB Titanium, 6.7-inch Super Retina XDR display with ProMotion", "Organic Cotton T-Shirt, Men's Medium, Navy Blue, 100% Sustainable Materials", "KitchenAid Stand Mixer, 5 Quart, Red, 10-Speed Settings with Dough Hook Attachment" template = outlines.Template.from string """ <|im start| user Categorize this product: {{ description }} <|im end| <|im start| assistant """ Get structured categorization for all products categories = model template description=desc for desc in product descriptions , ProductCategory, max new tokens=200 Use categorization for inventory management categories = ProductCategory.model validate json category for category in categories for product, category in zip product descriptions, categories : update inventory product, category.main category, category.sub category 📊 Parse event details with incomplete data This example uses outlines to parse event descriptions into structured information like event name, date, location, type, and topics , even handling cases where the data is incomplete. It leverages union types to return either structured event data or a fallback “I don’t know” answer, ensuring robust extraction in varying scenarios. python import outlines from typing import Union, List, Literal from pydantic import BaseModel from enum import Enum from transformers import AutoTokenizer, AutoModelForCausalLM MODEL NAME = "microsoft/Phi-3-mini-4k-instruct" model = outlines.from transformers AutoModelForCausalLM.from pretrained MODEL NAME, device map="auto" , AutoTokenizer.from pretrained MODEL NAME class EventType str, Enum : conference = "conference" webinar = "webinar" workshop = "workshop" meetup = "meetup" other = "other" class EventInfo BaseModel : """Structured information about a tech event""" name: str date: str location: str event type: EventType topics: List str registration required: bool Create a union type that can either be a structured EventInfo or "I don't know" EventResponse = Union EventInfo, Literal "I don't know" Sample event descriptions event descriptions = Complete information """ Join us for DevCon 2023, the premier developer conference happening on November 15-17, 2023 at the San Francisco Convention Center. Topics include AI/ML, cloud infrastructure, and web3. Registration is required. """, Insufficient information """ Tech event next week. More details coming soon """ Process events results = for description in event descriptions: prompt = f""" <|im start system You are a helpful assistant <|im end| <|im start user Extract structured information about this tech event: {description} If there is enough information, return a JSON object with the following fields: - name: The name of the event - date: The date where the event is taking place - location: Where the event is taking place - event type: either 'conference', 'webinar', 'workshop', 'meetup' or 'other' - topics: a list of topics of the conference - registration required: a boolean that indicates whether registration is required If the information available does not allow you to fill this JSON, and only then, answer 'I don't know'. <|im end| <|im start| assistant """ Union type allows the model to return structured data or "I don't know" result = model prompt, EventResponse, max new tokens=200 results.append result Display results for i, result in enumerate results : print f"Event {i+1}:" if isinstance result, str : print f" {result}" else: It's an EventInfo object print f" Name: {result.name}" print f" Type: {result.event type}" print f" Date: {result.date}" print f" Topics: {', '.join result.topics }" print Use structured data in downstream processing structured count = sum 1 for r in results if isinstance r, EventInfo print f"Successfully extracted data for {structured count} of {len results } events" 🗂️ Categorize documents into predefined types In this case, outlines classifies documents into predefined categories e.g., “Financial Report,” “Legal Contract” using a literal type specification. The resulting classifications are displayed in both a table format and through a category distribution summary, illustrating how structured outputs can simplify content management. python import outlines from typing import Literal, List import pandas as pd from transformers import AutoTokenizer, AutoModelForCausalLM MODEL NAME = "microsoft/Phi-3-mini-4k-instruct" model = outlines.from transformers AutoModelForCausalLM.from pretrained MODEL NAME, device map="auto" , AutoTokenizer.from pretrained MODEL NAME Define classification categories using Literal DocumentCategory = Literal "Financial Report", "Legal Contract", "Technical Documentation", "Marketing Material", "Personal Correspondence" Sample documents to classify documents = "Q3 Financial Summary: Revenue increased by 15% year-over-year to $12.4M. EBITDA margin improved to 23% compared to 19% in Q3 last year. Operating expenses...", "This agreement is made between Party A and Party B, hereinafter referred to as 'the Parties', on this day of...", "The API accepts POST requests with JSON payloads. Required parameters include 'user id' and 'transaction type'. The endpoint returns a 200 status code on success." template = outlines.Template.from string """ <|im start| user Classify the following document into exactly one category among the following categories: - Financial Report - Legal Contract - Technical Documentation - Marketing Material - Personal Correspondence Document: {{ document }} <|im end| <|im start| assistant """ Classify documents def classify documents texts: List str - List DocumentCategory : results = for text in texts: prompt = template document=text The model must return one of the predefined categories category = model prompt, DocumentCategory, max new tokens=200 results.append category return results Perform classification classifications = classify documents documents Create a simple results table results df = pd.DataFrame { "Document": doc :50 + "..." for doc in documents , "Classification": classifications } print results df Count documents by category category counts = pd.Series classifications .value counts print "\nCategory Distribution:" print category counts 📅 Schedule a meeting from requests with Function Calling This example demonstrates how outlines can interpret a natural language meeting request and translate it into a structured format matching a predefined function’s parameters. Once the meeting details are extracted e.g., title, date, duration, attendees , they are used to automatically schedule the meeting. python import outlines import json from typing import List, Optional from datetime import date from transformers import AutoTokenizer, AutoModelForCausalLM MODEL NAME = "microsoft/phi-4" model = outlines.from transformers AutoModelForCausalLM.from pretrained MODEL NAME, device map="auto" , AutoTokenizer.from pretrained MODEL NAME Define a function with typed parameters def schedule meeting title: str, date: date, duration minutes: int, attendees: List str , location: Optional str = None, agenda items: Optional List str = None : """Schedule a meeting with the specified details""" In a real app, this would create the meeting meeting = { "title": title, "date": date, "duration minutes": duration minutes, "attendees": attendees, "location": location, "agenda items": agenda items } return f"Meeting '{title}' scheduled for {date} with {len attendees } attendees" Natural language request user request = """ I need to set up a product roadmap review with the engineering team for next Tuesday at 2pm. It should last 90 minutes. Please invite john@example.com, sarah@example.com, and the product team at product@example.com. """ Outlines automatically infers the required structure from the function signature prompt = f""" <|im start| user Extract the meeting details from this request: {user request} <|im end| <|im start| assistant """ meeting params = model prompt, schedule meeting, max new tokens=200 The result is a dictionary matching the function parameters meeting params = json.loads meeting params print meeting params Call the function with the extracted parameters result = schedule meeting meeting params print result "Meeting 'Product Roadmap Review' scheduled for 2023-10-17 with 3 attendees" 📝 Dynamically generate prompts with re-usable templates Using Jinja-based templates, this example shows how to generate dynamic prompts for tasks like sentiment analysis. It illustrates how to easily re-use and customize prompts—including few-shot learning strategies—for different content types while ensuring the outputs remain structured. python import outlines from typing import List, Literal from transformers import AutoTokenizer, AutoModelForCausalLM MODEL NAME = "microsoft/phi-4" model = outlines.from transformers AutoModelForCausalLM.from pretrained MODEL NAME, device map="auto" , AutoTokenizer.from pretrained MODEL NAME 1. Create a reusable template with Jinja syntax sentiment template = outlines.Template.from string """ <|im start user Analyze the sentiment of the following {{ content type }}: {{ text }} Provide your analysis as either "Positive", "Negative", or "Neutral". <|im end <|im start assistant """ 2. Generate prompts with different parameters review = "This restaurant exceeded all my expectations. Fantastic service " prompt = sentiment template content type="review", text=review 3. Use the templated prompt with structured generation result = model prompt, Literal "Positive", "Negative", "Neutral" print result "Positive" Templates can also be loaded from files example template = outlines.Template.from file "templates/few shot.txt" Use with examples for few-shot learning examples = "The food was cold", "Negative" , "The staff was friendly", "Positive" few shot prompt = example template examples=examples, query="Service was slow" print few shot prompt | Model type | Description | Documentation | |---|---|---| | Server Support | vLLM and Ollama | Server Integrations → https://dottxt-ai.github.io/outlines/latest/features/models/ | | Local Model Support | transformers and llama.cpp | Model Integrations → https://dottxt-ai.github.io/outlines/latest/features/models/ | | API Support | OpenAI, Gemini, and Dottxt https://h1xbpbfsf0w.typeform.com/to/fwQNWmS8?utm source=github&utm medium=organic&utm campaign=outlines | API Integrations → https://dottxt-ai.github.io/outlines/latest/features/models/ | | Feature | Description | Documentation | |---|---|---| | Multiple Choices | Constrain outputs to predefined options | Multiple Choices Guide → https://dottxt-ai.github.io/outlines/latest/features/core/output types/ multiple-choices | | Function Calls | Infer structure from function signatures | Function Guide → https://dottxt-ai.github.io/outlines/latest/features/core/output types/ json-schemas | | JSON/Pydantic | Generate outputs matching JSON schemas | JSON Guide → https://dottxt-ai.github.io/outlines/latest/features/core/output types/ json-schemas | | Regular Expressions | Generate text following a regex pattern | Regex Guide → https://dottxt-ai.github.io/outlines/latest/features/core/output types/ regex-patterns | | Grammars | Enforce complex output structures | Grammar Guide → https://dottxt-ai.github.io/outlines/latest/features/core/output types/ context-free-grammars | | Feature | Description | Documentation | |---|---|---| | Prompt templates | Separate complex prompts from code | Template Guide → https://dottxt-ai.github.io/outlines/latest/features/utility/template/ | | Custome types | Intuitive interface to build complex types | Python Types Guide → https://dottxt-ai.github.io/outlines/latest/features/core/output types/ basic-python-types | | Applications | Encapsulate templates and types into functions | Application Guide → https://dottxt-ai.github.io/outlines/latest/features/utility/application/ | Outlines is developed and maintained by .txt https://dottxt.co , a company dedicated to making LLMs more reliable for production applications. Our focus is on advancing structured generation technology through: - 🧪 Cutting-edge Research : We publish our findings on structured generation http://blog.dottxt.co/performance-gsm8k.html - 🚀 Enterprise-grade solutions : You can license our enterprise-grade libraries https://docs.dottxt.co . - 🧩 Open Source Collaboration : We believe in building in public and contributing to the community Follow us on Twitter https://twitter.com/dottxtai or check out our blog https://blog.dottxt.co/ to stay updated on our latest work in making LLMs more reliable. - 💡 Have an idea? Come chat with us on Discord https://discord.gg/R9DSu34mGd - 🐞 Found a bug? Open an issue https://github.com/dottxt-ai/outlines/issues - 🧩 Want to contribute? Consult our contribution guide https://dottxt-ai.github.io/outlines/latest/community/contribute/ . @article{willard2023efficient, title={Efficient Guided Generation for Large Language Models}, author={Willard, Brandon T and Louf, R{\'e}mi}, journal={arXiv preprint arXiv:2307.09702}, year={2023} }