This tutorial explains a practical progression from traditional deterministic code to an agentic AI system. It follows one consistent example: a customer-support assistant that classifies issues, retrieves user/support knowledge, calls tools, and eventually makes multi-step decisions through an agent framework.
The goal is not to jump directly into an agent framework. Instead, each stage introduces one new capability so that the reason for RAG, tool calling, vector search, and orchestration becomes clear.
We will evolve the same support use case through these stages:
1. Algorithmic rules
↓
2. LLM classification
↓
3. LLM + tool definitions
↓
4. LLM + tool execution
↓
5. LLM + static application data
↓
6. Vector database
↓
7. Semantic retrieval
↓
8. RAG
↓
9. RAG + tools
↓
10. Agent orchestration with Pydantic AI
At the end, the system will be able to:
- understand a user's support request,
- retrieve relevant knowledge,
- decide which action is appropriate,
- execute a Python tool,
- use the tool result,
- and produce a final response.
llm_to_agentic_ai/
├── .env
├── a1algorithm.py
├── a2simple_llm.py
├── a3simple_llm_tool_1.py
├── a4simple_llm_tool_2.py
├── a5simple_llm_tool_3.py
├── a6simple_llm_db_1.py
├── a6simple_llm_db_2.py
├── a7simple_llm_db_3.py
├── a8simple_llm_db_4.py
├── a9simple_llm_db_5.py
├── a10simple_llm_db_6.py
└── a11simple_llm_agent.py
Using uv:
uv add groq python-dotenv pinecone pydantic-ai
If you also want to demonstrate Chroma locally:
uv add chromadb
Or with pip:
pip install groq python-dotenv pinecone pydantic-ai chromadb
Create a .env file:
GROQ_API_KEY=your_groq_key
PINECONE_API_KEY=your_pinecone_key
Never hard-code production API keys in source code.
Classify support requests using explicit rules.
def clarify_ticket(message: str):
message = message.lower()
if "password" in message or "login" in message:
return "Authentication Issue"
elif "payment" in message or "invoice" in message:
return "Payment Issue"
elif "slow" in message or "performance" in message:
return "Performance Issue"
else:
return "Other"
print(clarify_ticket("I cannot login"))
print(clarify_ticket("My payment failed"))
print(clarify_ticket("It is too slow"))
print(clarify_ticket("My username is not working"))
User input
↓
if / elif rules
↓
Category
- predictable,
- cheap,
- easy to test,
- deterministic,
- excellent when rules are known.
The code only understands patterns you explicitly define.
For example:
"I cannot sign in"
may mean the same thing as:
"I cannot login"
but your algorithm may not recognize it unless you add another rule.
This is the first motivation for using an LLM.
Let the model understand the meaning of the request instead of matching keywords.
from dotenv import load_dotenv
from groq import Groq
load_dotenv()
client = Groq()
def clarify_ticket(message: str):
completion = client.chat.completions.create(
model="openai/gpt-oss-120b",
messages=[
{
"role": "system",
"content": """
Classify the support request as one of:
- Authentication
- Payment
- Performance
- Other
Return only the category.
"""
},
{
"role": "user",
"content": message
}
],
temperature=0,
stream=False
)
return completion.choices[0].message.content
print(clarify_ticket("I cannot sign in to my account"))
User input
↓
LLM understands meaning
↓
Category
Developer specifies HOW to decide.
Developer specifies WHAT the desired result is.
The model interprets the language.
The LLM can reason and generate text, but it still cannot perform real application actions by itself.
For example, it can say:
You should reset your password.
But it has not actually reset anything.
That leads to tools.
Give the model a set of capabilities it may choose from.
First create normal Python functions:
def reset_password(username: str):
print(f"Email sent to {username}")
return f"Password reset link sent to user {username}"
def get_invoice(username: str):
print(f"Getting invoice for {username}")
return f"Latest invoice for user {username}: INV-1024"
These functions are ordinary application code.
Now describe them to Groq as tools:
from groq.types.chat import ChatCompletionToolParam
tools: list[ChatCompletionToolParam] = [
{
"type": "function",
"function": {
"name": "reset_password",
"description": "Send a password reset link to a user",
"parameters": {
"type": "object",
"properties": {
"username": {
"type": "string",
"description": "Username of the user"
}
},
"required": ["username"]
}
}
},
{
"type": "function",
"function": {
"name": "get_invoice",
"description": "Get the latest invoice for a user",
"parameters": {
"type": "object",
"properties": {
"username": {
"type": "string",
"description": "Username of the user"
}
},
"required": ["username"]
}
}
}
]
The tool schema does not execute the function.
It only tells the model:
These capabilities are available.
Here is what each one does.
Here are the arguments required.
The model may then choose a tool.
completion = client.chat.completions.create(
model="openai/gpt-oss-120b",
messages=[
{
"role": "system",
"content": """
Analyze the user's support request.
Categories:
- Authentication
- Payment
- Performance
- Other
Select the most appropriate available tool.
"""
},
{
"role": "user",
"content": "I cannot login, my username is dewmal"
}
],
tools=tools,
tool_choice="required",
temperature=0,
stream=False
)
message = completion.choices[0].message
if message.tool_calls:
selected_tool = message.tool_calls[0]
print(selected_tool.function.name)
print(selected_tool.function.arguments)
Possible output:
reset_password
{"username":"dewmal"}
User request
↓
LLM
↓
Understand intent
↓
Choose a tool
↓
Tool name + arguments
At this stage, the model has selected an action, but Python still needs to execute it.
The tool arguments returned by the model are JSON text.
Therefore this is wrong:
reset_password(**selected_tool.function.arguments)
because arguments is a string.
Convert it first:
import json
arguments = json.loads(
selected_tool.function.arguments
)
Then execute the selected function:
if selected_tool.function.name == "reset_password":
result = reset_password(**arguments)
elif selected_tool.function.name == "get_invoice":
result = get_invoice(**arguments)
python
import json
def clarify_ticket(message: str):
completion = client.chat.completions.create(
model="openai/gpt-oss-120b",
messages=[
{
"role": "system",
"content": "Select the most appropriate support tool."
},
{
"role": "user",
"content": message
}
],
tools=tools,
tool_choice="required",
temperature=0,
stream=False
)
assistant_message = completion.choices[0].message
if not assistant_message.tool_calls:
return assistant_message.content
selected_tool = assistant_message.tool_calls[0]
arguments = json.loads(selected_tool.function.arguments)
if selected_tool.function.name == "reset_password":
return reset_password(**arguments)
if selected_tool.function.name == "get_invoice":
return get_invoice(**arguments)
return "No supported tool selected"
LLM = decides WHAT to do
Python = controls HOW it is done
Tool = capability exposed to the LLM
This separation is important for security and reliability.
Before introducing RAG, first demonstrate a simple form of context injection.
users_db = [
{
"username": "dewmal",
"email": "dewmal@example.com",
"status": "active",
"plan": "Premium",
"full_name": "Dewmal Handapangoda"
},
{
"username": "john",
"email": "john@example.com",
"status": "locked",
"plan": "Basic",
"full_name": "John Smith"
},
{
"username": "sarah",
"email": "sarah@example.com",
"status": "active",
"plan": "Enterprise",
"full_name": "Sarah Kent"
}
]
Inject it into the prompt:
completion = client.chat.completions.create(
model="openai/gpt-oss-120b",
messages=[
{
"role": "system",
"content": f"""
You are a support assistant.
Here is the available user database:
{users_db}
Answer using only the information above.
If the information is unavailable, say you do not know.
"""
},
{
"role": "user",
"content": "What plan is John using?"
}
],
temperature=0,
stream=False
)
All application data
↓
Prompt
↓
LLM
↓
Answer
This works for small data.
10 records → easy
1,000 records → expensive
100,000 records → not practical
1,000,000 docs → impossible to send every time
Problems include:
- context-window limits,
- token cost,
- latency,
- irrelevant information,
- increased chance of poor grounding.
The solution is to retrieve only relevant data.
This is the reason for vector databases and RAG.
A vector database stores embeddings that represent semantic meaning.
Instead of asking:
Does this document contain exactly the word "login"?
semantic search can retrieve content related to:
sign in
access account
credentials not working
cannot authenticate
because the meanings are similar.
For a completely local demonstration:
import chromadb
chroma_client = chromadb.PersistentClient(
path="./db"
)
collection = chroma_client.get_or_create_collection(
name="user_support"
)
Add records only when empty:
if collection.count() == 0:
documents = [
"Dewmal has an active Premium account.",
"John's account is locked after five failed login attempts.",
"Sarah is an Enterprise customer.",
"Premium customers receive priority customer support.",
"Locked accounts require administrator approval.",
"Refunds normally take five to seven business days."
]
ids = [f"doc_{i}" for i in range(len(documents))]
collection.add(
ids=ids,
documents=documents
)
Search semantically:
results = collection.query(
query_texts=["Why can't John access his account?"],
n_results=2
)
print(results["documents"][0])
The query and retrieved document do not need to contain exactly the same wording.
Pinecone is useful for demonstrating a hosted vector database.
Assume you already created an index called:
user-support
and configured integrated embeddings with the source text field called:
text
python
import os
from dotenv import load_dotenv
from pinecone import Pinecone
load_dotenv()
pc = Pinecone(
api_key=os.getenv("PINECONE_API_KEY")
)
index_name = "user-support"
namespace = "support"
index = pc.Index(index_name)
stats = index.describe_index_stats()
namespace_stats = stats.namespaces.get(namespace)
count = namespace_stats.vector_count if namespace_stats else 0
if count == 0:
documents = [
"Dewmal has an active Premium account.",
"John's account is locked after five failed login attempts.",
"Sarah is an Enterprise customer.",
"Premium customers receive priority customer support.",
"Locked accounts require administrator approval.",
"Refunds normally take five to seven business days."
]
records = [
{
"_id": f"doc_{i}",
"text": document
}
for i, document in enumerate(documents)
]
index.upsert_records(
namespace=namespace,
records=records
)
The field name used in each record must match the field mapping configured for the Pinecone index.
If the index expects text, this is correct:
{
"_id": "doc_1",
"text": "John's account is locked..."
}
This would fail if the index does not use chunk_text:
{
"_id": "doc_1",
"chunk_text": "John's account is locked..."
}
php
def retrieve_context(question: str) -> str:
results = index.search(
namespace=namespace,
query={
"inputs": {
"text": question
},
"top_k": 3
},
fields=["text"]
)
hits = results["result"]["hits"]
documents = []
for hit in hits:
text = hit["fields"]["text"]
documents.append(text)
score = getattr(hit, "score", None)
if score is not None:
print(f"{score:.4f} -> {text}")
else:
print(text)
return "\n".join(documents)
Example:
context = retrieve_context(
"Why can't John access his account?"
)
print(context)
Possible relevant results:
John's account is locked after five failed login attempts.
Locked accounts require administrator approval.
The system did not send the entire database to the LLM.
It selected only the most relevant pieces first.
RAG has three conceptual parts.
R = Retrieval
A = Augmentation
G = Generation
context = retrieve_context(message)
Add the retrieved context to the model instructions:
messages = [
{
"role": "system",
"content": f"""
You are a customer support assistant.
Answer using only this retrieved context:
{context}
If the answer is unavailable, say you do not know.
"""
},
{
"role": "user",
"content": message
}
]
completion = groq_client.chat.completions.create(
model="openai/gpt-oss-120b",
messages=messages,
temperature=0,
stream=False
)
python
def clarify_ticket(message: str):
context = retrieve_context(message)
completion = groq_client.chat.completions.create(
model="openai/gpt-oss-120b",
messages=[
{
"role": "system",
"content": f"""
You are a customer support assistant.
Use only the following retrieved context:
{context}
If the answer cannot be determined from the context,
say "I don't know based on the available information."
"""
},
{
"role": "user",
"content": message
}
],
temperature=0,
stream=False
)
return completion.choices[0].message.content
User question
↓
Vector search
↓
Relevant documents
↓
Add to prompt
↓
LLM
↓
Grounded answer
Now we have two powerful capabilities:
Gives the model knowledge.
Give the model actions.
This creates the next architecture:
User
↓
Retrieve knowledge
↓
LLM understands request + context
↓
Choose tool
↓
Execute Python function
↓
Observe result
↓
Generate final response
python
def reset_password(username: str):
print(f"[TOOL] Sending password reset email to {username}")
return {
"status": "success",
"message": f"Password reset link sent to {username}"
}
def get_invoice(username: str):
print(f"[TOOL] Getting invoice for {username}")
return {
"status": "success",
"invoice": "INV-1024",
"username": username
}
def create_support_ticket(username: str, issue: str):
print(f"[TOOL] Creating ticket for {username}")
return {
"status": "success",
"username": username,
"issue": issue,
"ticket_id": "SUP-1001"
}
Describe them to Groq with tool schemas.
Then build the support flow.
import json
def support_agent(message: str):
context = retrieve_context(message)
messages = [
{
"role": "system",
"content": f"""
You are a customer support assistant.
Relevant retrieved context:
{context}
Classify the request as:
- Authentication
- Payment
- Performance
- Other
Rules:
- Authentication -> reset_password
- Payment -> get_invoice
- Performance -> create_support_ticket
- Other unresolved issue -> create_support_ticket
Select the most appropriate tool.
"""
},
{
"role": "user",
"content": message
}
]
completion = groq_client.chat.completions.create(
model="openai/gpt-oss-120b",
messages=messages,
tools=tools,
tool_choice="required",
temperature=0,
stream=False
)
assistant_message = completion.choices[0].message
if not assistant_message.tool_calls:
return assistant_message.content
selected_tool = assistant_message.tool_calls[0]
tool_name = selected_tool.function.name
arguments = json.loads(selected_tool.function.arguments)
if tool_name == "reset_password":
tool_result = reset_password(**arguments)
elif tool_name == "get_invoice":
tool_result = get_invoice(**arguments)
elif tool_name == "create_support_ticket":
tool_result = create_support_ticket(**arguments)
else:
tool_result = {"error": "Unknown tool"}
final_response = groq_client.chat.completions.create(
model="openai/gpt-oss-120b",
messages=[
{
"role": "system",
"content": f"""
You are a support assistant.
Relevant company context:
{context}
A support action has already been executed.
Explain the result briefly to the user.
"""
},
{
"role": "user",
"content": message
},
{
"role": "assistant",
"content": f"""
Selected action: {tool_name}
Tool result: {json.dumps(tool_result)}
"""
}
],
temperature=0,
stream=False
)
return final_response.choices[0].message.content
The system is no longer just answering questions.
It is performing a multi-step process:
Understand
↓
Retrieve
↓
Decide
↓
Act
↓
Observe
↓
Respond
This begins to resemble agentic behavior.
However, we still wrote the orchestration manually.
That is where an agent framework helps.
In the manual implementation, we had to manage:
- JSON tool schemas,
- tool-call inspection,
json.loads,if/eliffunction dispatch,- tool results,
- multiple model calls,
- orchestration logic.
An agent framework can manage much of that loop.
With Pydantic AI, ordinary Python functions can become tools through decorators.
We want the retrieval tool to access Pinecone without relying on global state.
from dataclasses import dataclass
@dataclass
class SupportDeps:
index: object
namespace: str
python
from pydantic_ai import Agent, RunContext
agent = Agent(
"groq:openai/gpt-oss-120b",
deps_type=SupportDeps,
instructions="""
You are a customer support agent.
You have tools for:
- retrieving company/user knowledge,
- resetting passwords,
- checking invoices,
- creating support tickets.
Always retrieve relevant support information before deciding how to respond.
Authentication problems:
- use reset_password when appropriate.
Payment or invoice problems:
- use get_invoice.
Performance or unresolved problems:
- use create_support_ticket.
Keep the final response short and clear.
"""
)
This is the major conceptual change.
Previously the application always executed retrieval first:
context = retrieve_context(message)
Now retrieval itself becomes a tool the agent can use.
@agent.tool
def retrieve_support_context(
ctx: RunContext[SupportDeps],
question: str
) -> str:
"""
Search the support knowledge base for information
relevant to the user's question.
"""
results = ctx.deps.index.search(
namespace=ctx.deps.namespace,
query={
"inputs": {
"text": question
},
"top_k": 3
},
fields=["text"]
)
hits = results["result"]["hits"]
documents = []
for hit in hits:
text = hit["fields"]["text"]
documents.append(text)
if not documents:
return "No relevant information found."
return "\n".join(documents)
The LLM can now decide when it needs more knowledge.
These tools do not need the dependency context, so tool_plain is enough.
@agent.tool_plain
def reset_password(username: str) -> str:
"""Send a password reset link to a user."""
print(f"[TOOL] Password reset requested for {username}")
return f"Password reset link sent to {username}."
php
@agent.tool_plain
def get_invoice(username: str) -> str:
"""Get the latest invoice for a user."""
print(f"[TOOL] Getting invoice for {username}")
return f"Latest invoice for {username}: INV-1024"
python
@agent.tool_plain
def create_support_ticket(
username: str,
issue: str
) -> str:
"""Create a support ticket for an unresolved issue."""
print(f"[TOOL] Creating ticket for {username}: {issue}")
return (
f"Support ticket SUP-1001 created for "
f"{username}: {issue}"
)
Pydantic AI can derive the tool input schema from the Python function signature.
deps = SupportDeps(
index=index,
namespace=namespace
)
def support_agent(message: str) -> str:
result = agent.run_sync(
message,
deps=deps
)
return result.output
Test:
print(
support_agent(
"I cannot login, my username is John"
)
)
Conceptually the agent can do this:
User
↓
Agent receives goal
↓
Agent calls retrieval tool
↓
Pinecone returns relevant knowledge
↓
Agent observes result
↓
Agent selects reset_password
↓
Python executes tool
↓
Agent observes tool result
↓
Agent generates final answer
This is the clearest transition from RAG to agentic AI.
Input
↓
Rules
↓
Output
The developer decides every branch.
Input
↓
LLM interpretation
↓
Output
The model handles ambiguity in language.
Input
↓
LLM
↓
Tool selection
↓
Python function
The model can choose actions.
All data
↓
Prompt
↓
LLM
The model can answer using application-specific data, but this does not scale.
Question
↓
Vector DB
↓
Relevant documents
The system retrieves based on meaning.
Question
↓
Retrieve
↓
Augment prompt
↓
Generate
The model answers using relevant private or domain-specific knowledge.
Question
↓
Retrieve knowledge
↓
LLM decision
↓
Tool execution
↓
Final answer
The model has both knowledge and capabilities.
Goal
↓
Agent
↓
Choose next step
↓
Retrieve / Tool / Reason
↓
Observe result
↓
Choose next step
↓
Finish
The model participates in deciding the sequence of steps.
The application decides the sequence.
Example:
context = retrieve_context(message)
result = call_llm(context, message)
execute_tool(result)
The flow is predefined.
The model can decide the next action from available tools.
Need knowledge?
→ retrieve
Need action?
→ call tool
Enough information?
→ answer
Agentic does not mean "use an LLM everywhere." It means the model has controlled decision-making responsibility within a bounded set of capabilities.
Use normal deterministic code when:
- the logic is fixed,
- compliance requires exact behavior,
- there are only a few known branches,
- the action sequence should never vary,
- latency and cost are critical.
For example:
if account_locked:
require_admin_approval()
may be much better than asking an LLM to decide.
A strong production system usually combines both:
AI handles ambiguity.
Code enforces business rules.
The examples in this tutorial are deliberately small. Real systems need stronger controls.
Never let the model directly execute arbitrary operations.
Instead expose a controlled function:
def reset_password(username: str):
...
Retrieval should respect permissions.
A user should not retrieve documents simply because they are semantically similar.
Production retrieval often needs metadata such as:
user_id
team_id
tenant_id
document_type
access_level
Validate:
- usernames,
- IDs,
- JSON,
- tool parameters,
- allowed actions,
- tool results.
Bad:
"Please remember users cannot refund more than $10,000."
Better:
if refund_amount > allowed_limit:
reject_refund()
Error:
argument after ** must be a mapping, not str
Fix:
arguments = json.loads(
selected_tool.function.arguments
)
Use the Groq tool type:
from groq.types.chat import ChatCompletionToolParam
tools: list[ChatCompletionToolParam] = [
...
]
For simple tool calling, use:
stream=False
Then:
completion.choices[0].message
For streaming, you must iterate through chunks.
404 NOT_FOUND: Resource user-support not found
pc.Index("user-support") assumes that index already exists.
Create it first in Pinecone or through the API before connecting.
Example error:
Missing field_mapping field 'text'
If your index expects text, records must contain:
{
"_id": "doc_0",
"text": "..."
}
not a differently named source field.
Do not assume every SDK response behaves like a plain dictionary.
A defensive pattern is:
score = getattr(hit, "score", None)
Then print it only when available.
A useful live-demo sequence is:
Show:
if "login" in message:
Then ask:
What happens if the user says "I can't sign in"?
Show the model understands varied wording.
Key point:
Rules → meaning
Show:
LLM can choose reset_password
but emphasize that the LLM still has not executed anything.
Execute the Python function.
Key point:
LLM provides intelligence.
Tools provide capabilities.
Python provides control.
This works and is intentionally simple.
Then ask:
What happens with one million records?
This creates the need for retrieval naturally.
Store support documents and run semantic search.
Show that:
"can't access account"
can retrieve:
"account is locked after failed login attempts"
without exact keyword matching.
Map the code explicitly:
R = index.search(...)
A = retrieved context inserted into prompt
G = Groq generates answer
Show knowledge and action working together.
RAG = knows
Tools = does
Show how manual orchestration becomes framework-managed tool usage.
Key message:
Agent = model + instructions + tools + controlled loop
| Stage | Who decides? | External knowledge | Can act? | Typical use |
|---|---|---|---|---|
| Algorithm | Python rules | No | Yes, deterministic | Fixed business logic |
| LLM | Model | Only prompt/model knowledge | No | Classification, generation |
| LLM + Tools | Model chooses tool | Limited | Yes | Intelligent action selection |
| Static Context | Model | Data copied into prompt | No/optional | Small private datasets |
| Vector Search | Retrieval system | Vector DB | No | Semantic search |
| RAG | Retriever + model | Relevant private data | No/optional | Grounded Q&A |
| RAG + Tools | Retriever + model | Relevant private data | Yes | Knowledge + action |
| Agentic AI | Model within constraints | Via retrieval/tools | Yes | Dynamic multi-step tasks |
A useful way to explain the entire progression is:
Algorithm
= Rules
LLM
= Language understanding + generation
Tools
= Capabilities
Vector DB
= Searchable semantic memory / knowledge store
RAG
= Relevant knowledge added at runtime
Agent
= LLM that can decide how and when to use tools and knowledge
Or even more simply:
LLM gives intelligence.
RAG gives knowledge.
Tools give actions.
Agent orchestration connects them.
- Start with deterministic code when the logic is fixed.
- Add an LLM when you need language understanding or flexible interpretation.
- Add tools when the model needs to perform real actions.
- Add external context when the model needs private or domain-specific information.
- Do not send all data to the model when the dataset grows.
- Use vector retrieval to find only relevant information.
- RAG combines retrieval with generation.
- Combine RAG with tools when the system must both know and act.
- Use an agent framework when the model needs controlled multi-step decision-making.
- Keep authorization, validation, security, and critical business rules in deterministic application code.
The most important engineering principle is:
Building an AI demo is easy. Building a reliable, secure, testable, and maintainable AI system is software engineering.