Outlines is an open-source library that introduces deterministic certainty into LLMs' output generation process for better, more reliable generation of structured outputs.
# Introduction #
Usually, when asking an LLM — abbreviation for "Large Language Model" — for a neat, structured output like JSON objects, for instance, a mix of careful prompt crafting with a "pinch" of luck is required. Otherwise, it might be tricky to get the model to obtain the perfectly structured output you are expecting. Or so it was, until a novel [open-source library](http://A new, open-source library that introduces deterministic certainty into LLMs' output generation process for better, more reliable generation of structured outputs.) came onto the scene: outlines.
This library is designed to prevent typical issues experienced by LLMs in these specific output-oriented use cases, such as hallucinations. More precisely, it introduces a degree of deterministic certainty into the output generation process.
Let's uncover what outlines
allows us to do through this illustrative article, in which we will show some practical examples in Python!
# Use Case 1: Multiple-Choice Classification for Sentiment Analysis #
Before fully diving into the first use case, you might be wondering. How does outlines
work and how does it guarantee correctness in structured model outputs? At the inference level, it masks out "syntactically illegal" tokens during generation instead of attempting to fix poor text once generated. This makes it virtually impossible to break the rules underlying the specific output format sought.
Let's see a first example in which we are building an analysis pipeline for customer support tickets, and we want exactly one option from a limited, approved list of possible options. This is pretty much like a classification problem, and generate.choice()
is the function that helps us mimic it, by forcing the model at hand to choose one of the predefined literals or classes.
But first, let's install it alongside the transformers
to load pre-trained LLMs:
pip install outlines[transformers]
The following code uses outlines.from_transformers()
to load a pre-trained model aided by Hugging Face's auto classes for a model and its associated tokenizer. But the icing on the cake is: they are both wrapped in an outlines
object that will later help tell the model what exactly to obtain. At the inference stage, we pass not only the user prompt asking to classify a review, but also a Literal
object that contains the output constraints the model should limit to:
import outlines
from transformers import AutoTokenizer, AutoModelForCausalLM
from typing import Literal
model_name = "microsoft/Phi-3-mini-4k-instruct"
model = outlines.from_transformers(
AutoModelForCausalLM.from_pretrained(model_name),
AutoTokenizer.from_pretrained(model_name)
)
sentiment = model(
"Classify the sentiment of this customer review: 'I've been waiting two weeks for my delivery and it's still missing.'",
Literal["Positive", "Negative", "Neutral"]
)
print(sentiment)
Output:
Negative
A word of warning here: although the literal we defined is part of Python's built-in typing module, rather than outlines
, our out-of-the-box library still assumes model control here: both the model and tokenizer are wrapped into an object that enforces standard Python types, building a finite state machine under the hood that limits the output to the options provided only.
# Use Case 2: JSON Object Generation #
This example first defines a Pydantic object that defines the desired structure for a JSON object describing a fictional character with a name, description, and age. It then uses our previously wrapped outlines
model, passing the character object to ensure the output generated strictly follows this structure for the JSON object requested:
from pydantic import BaseModel
class Character(BaseModel):
name: str
description: str
age: int
json_output = model(
"Generate a JSON object describing a fictional character named 'Anya'.",
Character,
max_new_tokens=200
)
print(json_output)
Output:
{ "name": "Anya", "description": "Anya is a young, adventurous woman with a passion for exploring new places and meeting new people. She has long, curly hair and bright green eyes that sparkle with curiosity. Anya is always eager to learn and loves to share her knowledge with others. She is kind-hearted and always willing to lend a helping hand to those in need. Anya's favorite hobbies include hiking, reading, and playing the guitar. She is a free spirit who values freedom and independence above all else." ,"age": 25 }
# Use Case 3: Pure JSON Generation for REST APIs #
This third example, also JSON-related, is similar to the previous one but in a slightly different context. Imagine you are building an API backed requiring a well-defined JSON payload for updating a database. Asking a standard LLM to get this output will more often than not yield to annoying, trailing characters like commas that are likely to crash a JSON parser.
With outlines, we define our JSON payload schema once again with a Pydantic-based custom class object.
from pydantic import BaseModel
from typing import Literal
import json
class ServerHealth(BaseModel):
service_name: str
uptime_seconds: int
status: Literal["OK", "DEGRADED", "DOWN"]
raw_json_string = model(
"Report the current status of the main Auth database.",
ServerHealth,
max_new_tokens=50
)
print(type(raw_json_string)) # This will just print:
parsed_json = json.loads(raw_json_string)
print(json.dumps(parsed_json, indent=2))
Output:
{
"service_name": "auth_db_status",
"uptime_seconds": 1623456789,
"status": "OK"
}
# Closing Remarks #
Since LLMs are trained to be chat-lovers capable of breaking syntax or hallucinating to "sound like humans" in their conversations with us, getting them to produce reliable, structured outputs like clean JSON objects can feel like a bit of a pain. Outlines a new, open-source library that introduces deterministic certainty into LLMs' output generation process for better, more reliable generation of structured outputs. This article showed three simple yet useful use cases for beginners with this interesting tool.
is a leader, writer, speaker, and adviser in AI, machine learning, deep learning & LLMs. He trains and guides others in harnessing AI in the real world.