# learning Generative AI/LLM through practical projects > Source: > Published: 2026-09-01 13:23:23+00:00 If you want to become good at **Generative AI/LLM engineering**, watching tutorials is not enough. The fastest way to understand these technologies is to build projects where you are forced to solve real problems: prompting, context management, retrieval, model adaptation, and evaluation. Here is a practical project roadmap: **LLM → RAG → Fine-Tuning → Evals** Build an AI system that receives a new customer lead and decides: **Input:** "Hi, I'm looking for an enterprise plan for 200 employees. We need SSO and would like to schedule a demo next week." **Output:** ``` Lead Quality: High Intent: Enterprise Purchase Company Size: 200 employees Urgency: High Recommended Action: Schedule Demo ``` This project gives you a strong foundation in: Before jumping into RAG or fine-tuning, you should understand how an LLM behaves **without external knowledge or model customization**. This gives you the baseline against which you can later compare RAG and fine-tuning. Now take the same LLM and give it access to your own knowledge base. Upload documents such as: ``` Company Policies Product Documentation HR Policies FAQs Technical Documentation Pricing Documents ``` Users should be able to ask questions about these documents. **User:** "What is our work-from-home policy?" **RAG pipeline:** ``` User Question ↓ Query Embedding ↓ Vector Database ↓ Retrieve Relevant Documents ↓ Context + Question ↓ LLM ↓ Grounded Answer ``` RAG works by retrieving relevant information from an external data source and providing that information to the LLM as context. ([GitHub](https://github.com/langchain-ai/rag-from-scratch?utm_source=chatgpt.com)) Build the project in stages: **Level 1 — Basic RAG** **Level 2 — Better RAG** **Level 3 — Production RAG** A great reference is [LangChain — RAG From Scratch](https://github.com/langchain-ai/rag-from-scratch?utm_source=chatgpt.com). It builds RAG progressively from indexing, retrieval and generation, making it particularly useful for understanding **how RAG actually works rather than simply copying a framework implementation**. ([GitHub](https://github.com/langchain-ai/rag-from-scratch?utm_source=chatgpt.com)) You can also explore [LlamaIndex RAG example](https://github.com/danielbank/rag-llamaindex?utm_source=chatgpt.com) for a more application-oriented implementation. ([GitHub](https://github.com/danielbank/rag-llamaindex?utm_source=chatgpt.com)) Important:This should be treated as an educational AI project, not a real medical diagnostic system. The goal is to take an open-source LLM and adapt it to produce responses in a particular domain and format. For example, create a dataset containing: ``` Question ↓ Medical Context ↓ Expected Response ``` Then fine-tune an open model on your dataset. **Input:** "What are common symptoms associated with iron deficiency?" The model should learn to produce a response following your desired structure and style. This project teaches: Instead of trying to fine-tune a huge model from scratch, start with **LoRA/QLoRA**. These techniques make experimentation much more practical. For a simple introduction: [Fine-Tuning LLMs with LoRA and QLoRA](https://github.com/Apoorva-Udupa/Fine-Tuning_LLMs_with_LoRA_and_QLoRA?utm_source=chatgpt.com) This repository demonstrates LoRA and QLoRA fine-tuning using PyTorch and Hugging Face Transformers. ([GitHub](https://github.com/Apoorva-Udupa/Fine-Tuning_LLMs_with_LoRA_and_QLoRA?utm_source=chatgpt.com)) For a more complete implementation: [LLM Fine-Tuning — SFT, LoRA & QLoRA](https://github.com/gazelle93/llm-fine-tuning-sft-lora-qlora?utm_source=chatgpt.com) It includes dataset loading, tokenization, SFT, LoRA and QLoRA examples. ([GitHub](https://github.com/gazelle93/llm-fine-tuning-sft-lora-qlora?utm_source=chatgpt.com)) Don't think: Fine-tuning = giving the model more knowledge Instead, think: Fine-tuning = adapting model behavior, style, format or task performance. For frequently changing factual knowledge, RAG is often a better solution. This is the project most beginners skip. And it is one of the most important. Suppose your RAG system answers: "What is the company's leave policy?" How do you know whether the answer is actually good? You need an evaluation system. Create a test dataset: ``` Question Expected Answer Retrieved Context Generated Answer ``` Then evaluate the system automatically. **Retrieval** **Generation** ``` Question: What is our annual leave policy? Expected: Employees receive 24 days of annual leave. Model Answer: Employees receive 24 days of annual leave. Evaluation: Correctness: 1.0 Faithfulness: 1.0 Relevance: 1.0 ``` Now intentionally introduce a bad answer: ``` Model Answer: Employees receive 30 days of annual leave. Evaluation: Correctness: 0.0 Faithfulness: 0.0 ``` You have now started building an **LLM evaluation pipeline**. A good reference is [RAG Evaluation Framework](https://github.com/Aftabbs/RAG-Evaluation-Framework?utm_source=chatgpt.com). It separates evaluation into retrieval quality and generation quality and uses LLM-based evaluation with LangChain. ([GitHub](https://github.com/Aftabbs/RAG-Evaluation-Framework?utm_source=chatgpt.com)) Another useful project is [Ragas](https://github.com/vibrantlabsai/ragas?utm_source=chatgpt.com), which provides metrics and test-data generation for evaluating LLM applications and RAG systems. ([GitHub](https://github.com/vibrantlabsai/ragas?utm_source=chatgpt.com)) You can also study [LLM RAG Eval](https://github.com/sujitpal/llm-rag-eval?utm_source=chatgpt.com), which focuses specifically on evaluating RAG pipelines. ([GitHub](https://github.com/sujitpal/llm-rag-eval?utm_source=chatgpt.com)) Instead of building four unrelated projects, build them as a progression: ``` GENERATIVE AI │ ▼ ┌─────────────────────┐ │ 1. Lead Triaging Bot│ │ LLM │ └──────────┬──────────┘ │ ▼ ┌─────────────────────┐ │ 2. Knowledge │ │ Assistant │ │ RAG │ └──────────┬──────────┘ │ ▼ ┌─────────────────────┐ │ 3. Medical Advisor │ │ Fine-Tuning │ └──────────┬──────────┘ │ ▼ ┌─────────────────────┐ │ 4. Evaluation │ │ Framework │ │ Evals │ └─────────────────────┘ ``` | Project | Technology | You Learn | |---|---|---| Lead Triaging Bot | LLM | Prompting, structured output, tools | Knowledge Assistant | RAG | Embeddings, retrieval, vector DB, RAG | Medical Advisor | Fine-Tuning | SFT, LoRA, QLoRA, datasets | Evaluation Framework | Evals | Metrics, test datasets, hallucination detection | But the real value comes when you **connect them**. Your final architecture can look like: ``` User │ ▼ Lead / Query │ ▼ ┌──────────────┐ │ LLM │ └──────┬───────┘ │ ┌─────────┴─────────┐ ▼ ▼ RAG Fine-Tuned Knowledge Model │ │ └─────────┬─────────┘ ▼ Final Answer │ ▼ EVALUATION │ ┌─────────┼─────────┐ ▼ ▼ ▼ Correct? Relevant? Grounded? ``` Don't learn these technologies as isolated topics. **Build progressively.** Start with an LLM application → add your own knowledge with RAG → adapt the model with fine-tuning → finally build an evaluation layer to measure whether your system actually improved. That progression takes you from **"I know how to call an LLM API"** to **"I can design, improve and evaluate production-style LLM systems."**