{"slug": "building-a-local-rag-system-for-lab-notes-and-slides-with-amd-and-dgx-spark", "title": "Building a Local RAG System for Lab Notes and Slides with AMD and DGX Spark", "summary": "A developer built a local retrieval-augmented generation service that lets lab researchers query Markdown notes and PowerPoint slides and trace each answer back to a file and line or slide number. The system runs Gemma 4 31B and Qwen3-Embedding-4B on an AMD Ryzen AI Max+ 395 for slide description and embedding, stores 2,560-dimensional vectors in Qdrant, and uses DeepSeek-V4.1-Flash-1017 across two NVIDIA DGX Spark systems to generate cited answers. In a November 18, 2026 test over 143 files and 1,529 chunks, all four Japanese and English queries returned four excerpts, and the model correctly withheld a free-space figure the excerpts could not establish.", "body_md": "I built a local retrieval-augmented generation (RAG) service so researchers can ask questions about our lab’s Markdown notes and PowerPoint slides and trace each answer back to a file and line or slide number.\n\nTwo models run on the AMD Ryzen AI Max+ 395. **Gemma 4 31B** describes rendered slides, making details in figures available as searchable text. **Qwen3-Embedding-4B** embeds both the source text and the slide descriptions. When a researcher asks a question in Open WebUI, Qwen embeds the question in the same vector space so Qdrant can rank relevant passages. DeepSeek-V4.1-Flash-1017 runs across two NVIDIA DGX Spark systems and uses the retrieved excerpts to write the answer.\n\n**During an index refresh**, the AMD service reads the files from network-attached storage (NAS). It extracts native text, keeps its source locations, and adds descriptions of rendered slides from Gemma 4 31B. Ollama serves Qwen3-Embedding-4B locally to produce 2,560-dimensional embeddings. The service stores them with the passages and their source metadata in a local Qdrant index. A scheduled job checks for updates daily at 03:00 Japan time.\n\n**At question time**, a researcher selects the lab-document model in Open WebUI. The single-turn service uses Qwen3-Embedding-4B to embed the latest question and queries Qdrant using cosine similarity. It sends DeepSeek the question and up to four excerpts labeled `[S1]` through `[S4]`. Each excerpt gets up to 1,600 characters of content, and the whole evidence block, including its source headers, has a 5,500-character budget.\n\nThe service checks the citation IDs it recognizes in the answer against the IDs it supplied. It then lists **all supplied excerpts** by file path and starting line or slide number, including any excerpts the answer did not cite. A valid ID lets a researcher locate a passage. To check whether that passage supports a claim, someone still has to compare the two. \n\nOn November 18, 2026, I tested the direct route from AMD to DeepSeek. For this trial, the index contained native text from 143 files, split into 1,529 chunks. I asked two questions, each in Japanese and English. All four searches returned four excerpts. The table paraphrases the prompts and omits the private path.\n\n| Question | What the test checked | Recorded result | \n|---|---|---|\n| Where are the CSV files from test runs and MLflow stored? | Did the answer give the expected directory and cite an excerpt containing it? | Both languages returned the expected path with a supporting citation. | \n| How many GB are free on that volume right now? | Did the answer acknowledge that the excerpts could not establish the current value and avoid estimating it? | Both languages withheld a numeric capacity and passed the evidence check. | \n\nIn a separate retrieval check, the passage with the expected directory ranked **first for the Japanese query and second for the English query**. For the answer test, DeepSeek received the selected excerpts. After generation, the checker hashed path candidates from the answer and compared them with the known path hash. It also checked that at least one citation in the answer body pointed to an excerpt containing the expected path. This last check matters because the right source can appear in the appended list even when the answer cites the wrong passage.\n\nFor the free-space question, retrieval succeeded in both languages. A fallback that only handles empty search results would never have run. The checker looked for numeric capacity expressions, and a second call to the same DeepSeek model assessed whether the answer explicitly acknowledged the missing measurement. I also used that model to review grounding and citations across the trial. The four cases give me a regression test for these two behaviors. The model’s review provides no independent estimate of answer quality.\n\nA plot can occupy most of a slide while the native text extractor returns little more than its title. The title might help find the deck, but it gives DeepSeek little evidence about the figure. I had encountered a similar problem when testing text and figures from a PDF. [Earlier PDF evaluation](https://dev.to/nha6ki/evaluating-deepseek-v41-flash-1017-on-a-60-page-pdf-what-text-extraction-misses-2efo)\n\nTo make slides searchable, the pipeline indexes native text alongside descriptions that Gemma 4 31B generates from rendered slides. During refresh, LibreOffice renders the deck to PDF. The pipeline checks that the page count matches the slide count, then rasterizes visible slides using their original slide numbers. A Japanese prompt asks Gemma to describe the figure’s subject, layout and legible labels, preserve English abbreviations, and include small numbers or legend entries only when it can read them.\n\nWhen retrieval selects a slide caption, DeepSeek receives its text and slide number. The source entry labels it as an AI-generated image description. A researcher can use it to locate a figure, then open the slide to check an axis label, legend or numerical value.\n\nThe caption cache uses the rendered pixels and captioning configuration, so an unchanged image can reuse its description. The service also verifies that a retrieved caption matches the stored caption record and the indexed deck. This catches a mismatched record. A caption that misreads a legend can still pass the check. For one slide I reviewed, I tied a corrected caption to the exact rendered pixels so a replacement figure would not inherit the correction.\n\nThe **November 19 production refresh** recorded **143 files and 1,707 indexed chunks**. It retained descriptions for **160 slides** and omitted descriptions for **14 slides**. Those counts establish that captions entered the searchable index. The bilingual questions ran the previous day on native text. To measure the contribution of captions, I would build two indexes from the same document snapshot, enable captions in one, run the same figure questions with the same retrieval and generation settings, and grade the answers against the original slides.\n\nIn the bilingual trial, DeepSeek cited the handover note for the CSV directory and acknowledged that the same note could not establish the volume’s current free space. For researchers, the source location is as useful as the answer itself. They can check a documented procedure at the cited line, or use a caption retrieved by the system to find a figure and inspect its legend on the original slide.", "url": "https://wpnews.pro/news/building-a-local-rag-system-for-lab-notes-and-slides-with-amd-and-dgx-spark", "canonical_source": "https://dev.to/nha6ki/building-a-local-rag-system-for-lab-notes-and-slides-with-amd-and-dgx-spark-42a8", "published_at": "2026-11-20 00:37:22+00:00", "updated_at": "2026-11-20 00:43:14.484204+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "generative-ai", "ai-agents", "ai-infrastructure"], "entities": ["AMD", "NVIDIA", "DGX Spark", "Ryzen AI Max+ 395", "Gemma 4 31B", "Qwen3-Embedding-4B", "DeepSeek-V4.1-Flash-1017", "Qdrant"], "also_reported_by": [], "takeaway": "Appending all supplied excerpts with file and line references, not just the ones the model cited, lets reviewers catch answers that cite the wrong passage even when the right source was retrieved.", "recommendation": {"advice": "List every retrieved excerpt with its source location alongside the answer, and check that at least one cited ID points to the passage containing the expected fact.", "playbook": "output-verification", "playbook_url": "https://wpnews.pro/playbooks/output-verification"}, "alternates": {"html": "https://wpnews.pro/news/building-a-local-rag-system-for-lab-notes-and-slides-with-amd-and-dgx-spark", "markdown": "https://wpnews.pro/news/building-a-local-rag-system-for-lab-notes-and-slides-with-amd-and-dgx-spark.md", "text": "https://wpnews.pro/news/building-a-local-rag-system-for-lab-notes-and-slides-with-amd-and-dgx-spark.txt", "jsonld": "https://wpnews.pro/news/building-a-local-rag-system-for-lab-notes-and-slides-with-amd-and-dgx-spark.jsonld"}}