Building an AI-powered application is easy to describe.
Upload documents → ask a question → get an answer.
Actually building that flow is a different story.
While working on Guidely, an internal knowledge assistant, I wanted to understand what happens between those two endpoints. More importantly, I wanted to design the system so that each part had a clear responsibility and could be tested independently.
The result was an end-to-end Retrieval-Augmented Generation (RAG) system built around a simple pipeline:
Documents
↓
Parsing
↓
Chunking
↓
Embeddings
↓
Vector Store
↓
Semantic Search
↓
Response Generation
↓
Citations
↓
React Frontend
The interesting part wasn't simply getting an LLM to answer questions.
It was designing the architecture that makes the entire pipeline work reliably.
Guidely is an internal knowledge assistant that allows users to ask questions about a collection of organizational documents.
Instead of expecting an AI model to already know everything about an organization's internal knowledge, Guidely retrieves relevant information from the organization's documents and uses that information to construct an answer.
For example, a user might ask:
"What is TrustLayer?"
Guidely searches the organization's knowledge base, retrieves the most relevant sections, and uses those sections as context for generating the answer.
The response is then presented together with the sources that support it.
This is the basic idea behind RAG.
But I wanted the architecture to make the flow explicit rather than hiding everything inside one large function.
The first major decision was to separate the system into distinct stages.
┌─────────────────┐
│ Documents │
└────────┬────────┘
↓
┌─────────────────┐
│ Parser │
└────────┬────────┘
↓
┌─────────────────┐
│ Chunker │
└────────┬────────┘
↓
┌─────────────────┐
│ Embeddings │
└────────┬────────┘
↓
┌─────────────────┐
│ Vector Store │
└────────┬────────┘
↓
User Query
↓
┌─────────────────┐
│ Semantic Search │
└────────┬────────┘
↓
┌─────────────────┐
│ Response │
└────────┬────────┘
↓
┌─────────────────┐
│ React Frontend │
└─────────────────┘
Each component answers a different question.
That separation became one of the most important architectural decisions in the project.
The first stage is getting documents into the system.
Guidely supports documents such as:
The upload API receives a document and stores it in the document directory.
But up a file isn't the same thing as making it searchable.
The document needs to go through the ingestion pipeline.
Uploaded document
↓
Determine file type
↓
Parse document
↓
Extract text
↓
Chunk text
↓
Generate embeddings
↓
Store vectors + metadata
This separation means the upload layer doesn't need to understand embeddings or semantic search.
Its responsibility is simply:
Get the document into the system.
Once a document exists, Guidely needs to extract its text.
The parser provides a common interface:
parse_document(file_path)
Internally, the appropriate parser can be selected depending on the file type.
For example:
.txt → text parser
.pdf → PDF parser
.docx → DOCX parser
The important architectural idea here is that the rest of the pipeline doesn't need to care where the text came from.
Once parsing is complete, everything downstream works with:
text: str
This keeps the pipeline format-independent.
A document can be thousands of words long.
Sending an entire document into a retrieval system isn't ideal.
Instead, Guidely breaks the extracted text into smaller chunks.
The chunking strategy uses tokens rather than simply splitting every N characters.
For example:
Document
────────────────────────────
Paragraph 1
Paragraph 2
Paragraph 3
Paragraph 4
Paragraph 5
...
Chunk 1
────────────────
Paragraph 1
Paragraph 2
Chunk 2
────────────────
Paragraph 2
Paragraph 3
Chunk 3
────────────────
Paragraph 3
Paragraph 4
The overlap is intentional.
If a relevant sentence happens to sit near a chunk boundary, overlap reduces the chance that important context gets separated.
The chunking function therefore has two important parameters:
chunk_size = 800
overlap = 100
The exact values can be tuned later.
The important design decision was making chunking its own service rather than embedding the logic inside document ingestion.
This is where the system starts moving from traditional text processing into semantic search.
Each chunk is converted into a vector representation.
Conceptually:
"TrustLayer is a decentralized protocol..."
↓
Embedding Model
↓
[0.021, -0.143, 0.782, ...]
The same process happens when a user asks a question.
"What is TrustLayer?"
↓
Embedding Model
↓
Query Vector
Now the system can compare the query vector with document vectors.
This is the foundation of semantic retrieval.
One challenge I encountered here was model selection.
I initially explored hosted embedding APIs but ran into API limitations. I eventually moved toward a local Sentence Transformers model.
That decision had an architectural benefit beyond simply solving the immediate problem:
the embedding layer became independent from the rest of the application.
If I change the embedding model later, the search API doesn't need to change.
Once chunks have embeddings, the vectors need to be stored somewhere.
For Guidely, I used FAISS.
The basic relationship looks like:
Vector
│
├── FAISS index
│
└── Metadata
├── filename
├── chunk information
└── original text
The vector index handles similarity search.
The metadata provides the information needed to understand what a vector represents.
This separation is important.
FAISS answers:
Which vectors are closest to this query?
The metadata answers:
What do those vectors actually represent?
When a user submits a question, the query follows a shorter path:
User question
↓
Create embedding
↓
FAISS similarity search
↓
Top K results
↓
Relevant document chunks
The search service looks roughly like this:
query_embedding = create_embeddings([query])[0]
distances, indices = index.search(
query_vector,
top_k
)
The returned vector IDs are then mapped back to document metadata.
One useful property of this architecture is that the search service doesn't need to know anything about the frontend.
It simply returns structured results.
For example:
{
"filename": "faq.txt",
"text": "TrustLayer is a decentralized protocol..."
}
One of the more important problems I encountered was handling irrelevant questions.
A vector database will usually return something.
Even if the user asks a question completely unrelated to the knowledge base, FAISS can still return the nearest vectors.
That creates a dangerous situation:
Irrelevant question
↓
Similarity search
↓
Some vaguely similar chunks
↓
AI generates an answer
↓
Citation appears
The system can therefore look confident even when it shouldn't be answering.
This led to an important architectural requirement:
Retrieval needs a relevance boundary.
Instead of blindly accepting the top K results, the system needs to determine whether the retrieved results are actually relevant enough to support an answer.
This is also where citations need to be handled carefully.
A citation should not appear simply because a document happened to be returned by FAISS.
After retrieval, the relevant chunks become context for the response layer.
Conceptually:
User Question
+
Retrieved Context
↓
Response Generator
↓
Answer + Sources
The backend returns a structured response rather than exposing internal implementation details.
For example:
{
"answer": "TrustLayer is a decentralized protocol on Solana...",
"citations": [
{
"source": "faq.txt",
"snippet": "It allows clients and talent to collaborate directly..."
}
]
}
This distinction matters.
The backend can contain things such as:
Citations became an interesting part of the project.
Initially, returning the entire retrieved chunk produced poor results.
A citation could contain an entire section of a document even when only one sentence supported the answer.
For example:
faq.txt
TRUSTLAYER FREQUENTLY ASKED QUESTIONS
...
TABLE OF CONTENTS
...
Q1...
Q2...
Q3...
That's technically a citation, but it isn't particularly useful to a human.
The goal became:
faq.txt
"It allows clients and talent to collaborate directly..."
The citation should answer:
"Where did this information come from?"
not:
"Here is a large portion of the document."
This led to a query-aware citation strategy where the snippet is selected based on the information relevant to the user's question.
The frontend is intentionally separated from the retrieval system.
I built the interface with React.
The user sees:
Guidely
Ask your organization's knowledge
┌──────────────────────────────────────────┐
│ Ask a question... → │
└──────────────────────────────────────────┘
Answer
TrustLayer is a decentralized protocol...
Sources
┌──────────────────────────────────────────┐
│ 📄 faq.txt │
│ It allows clients and talent... │
└──────────────────────────────────────────┘
The frontend doesn't need to know how embeddings work.
It doesn't know what FAISS is.
It doesn't need to understand chunking.
It simply consumes the response contract from the API.
The second major frontend surface is the knowledge-base administration page.
The admin interface allows documents to be uploaded and viewed.
The architecture looks like:
Admin
↓
Upload document
↓
FastAPI
↓
Document storage
↓
Ingestion pipeline
↓
Embeddings
↓
FAISS
The interface intentionally hides implementation details.
An administrator doesn't need to know:
"Your document has been converted into a 768-dimensional vector and inserted at index 42."
They need to know:
"Your document has been uploaded and is available."
This distinction influenced a lot of the UI decisions.
FastAPI became the boundary between the frontend and the internal services.
The application is organized around responsibilities such as:
app/
├── routers/
│ ├── search.py
│ └── documents.py
│
├── services/
│ ├── parser.py
│ ├── chunker.py
│ ├── embeddings.py
│ ├── vector_store.py
│ └── response.py
│
└── main.py
This structure isn't about creating as many files as possible.
It's about making the data flow understandable.
A search request can be traced through:
search router
↓
search service
↓
embedding service
↓
vector store
↓
response service
Similarly, document ingestion has its own path.
That makes debugging considerably easier.
The biggest challenge wasn't writing the individual functions.
It was deciding where each responsibility belonged.
For example, it would have been possible to create one large function:
def ask_question(query):
It would probably work.
But it would also become difficult to test and modify.
Instead, Guidely separates those responsibilities.
That gives me the ability to change:
without necessarily rewriting the entire system.
After putting everything together, the final architecture looks like this:
DOCUMENT INGESTION
Document
↓
Parser
↓
Text
↓
Chunker
↓
Chunks
↓
Embedding Model
↓
Vectors
↓
FAISS + Metadata
│
│
│
▼
QUERY PIPELINE
User Question
↓
Embedding Model
↓
Query Vector
↓
FAISS Similarity Search
↓
Relevant Chunks
↓
Relevance Filtering
↓
Response Generation
↓
Answer + Query-Aware Citations
↓
React UI
The current architecture works, but there are several areas I would improve as the project evolves.
Similarity scores alone aren't enough.
I'd like to build a proper evaluation dataset containing:
This would make retrieval quality measurable rather than something I evaluate manually.
Different documents have different structures.
A fixed token-based chunk size isn't necessarily optimal for:
A future version could use structure-aware chunking.
Citation snippets could be selected more intelligently based on the query and the generated answer.
FAISS works well for a project like this, but a production deployment could benefit from a persistent vector database depending on scale and operational requirements.
The current admin functionality is primarily focused on document management.
A production knowledge assistant would also need authentication, authorization, document ownership, and potentially per-user or per-team knowledge bases.
The most valuable part of this project wasn't getting an AI model to answer a question.
It was learning to think about the system as a collection of independent stages.
A useful mental model is:
Don't start with:
"How do I make an AI answer questions?"
Start with:
"How does information move through the system?"
Once that question is answered, the architecture becomes much clearer.
Documents become text.
Text becomes chunks.
Chunks become vectors.
Vectors become searchable knowledge.
Search results become context.
Context becomes an answer.
And the answer becomes something a human can actually use.
That is the architecture behind Guidely: a small but complete end-to-end RAG system designed not just to work, but to make each stage understandable, replaceable, and testable.
The next challenge is measuring how well each stage works.