{"slug": "the-day-my-lecture-notes-bot-contradicted-itself", "title": "The Day My Lecture Notes Bot Contradicted Itself", "summary": "A developer built a lecture-notes chatbot using MonkeyCode's free tier, which initially gave overconfident, incorrect answers by ignoring contradictory slides. The developer added a crude contradiction check that flags when retrieved chunks are textually dissimilar, catching cases where the bot would quote only one side of a contradiction.", "body_md": "I was up at 2 AM, staring at seventeen PDFs that refused to tell me anything. My midterm was in six days, and my notes were a mess of arrows, acronyms, and half-typed definitions. I wanted a chatbot that could answer questions about my own lectures. Not a fancy one. Just something that would take a question, find the relevant slide, and answer in plain language.\n\nSo I built one. I used MonkeyCode for the free model access and free server space. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Their open-source platform's free tier includes 10 million tokens and a server slot, which is enough for a weekend prototype. The “why not” won.\n\nThe plan was simple: extract text from the PDFs, split it into chunks, retrieve the most relevant chunks with a dumb similarity search, then ask a model to answer from those chunks. No vector database. No fine-tuning. Just a few lines of Python and a POST request.\n\nThe extraction step was almost too easy.\n\n``` python\nfrom pypdf import PdfReader\n\ndef extract_pdf(path):\n    return \"\\n\".join(page.extract_text() for page in PdfReader(path).pages)\n```\n\nMost of my slides were text-heavy, so it worked. One deck came out as garbage because the pages were rotated. That was my first warning: garbage in, confident nonsense out.\n\nNext, chunking. I set a chunk size of 1,200 characters with an overlap of a hundred. Small enough to be relevant, big enough to contain a complete idea.\n\n``` python\ndef chunk_text(text, size=1200, overlap=100):\n    chunks = []\n    for i in range(0, len(text), size - overlap):\n        chunks.append(text[i:i+size])\n    return chunks\n```\n\nI didn't use a vector database. My whole corpus was about two hundred chunks, so TF-IDF plus cosine similarity was enough. More importantly, it made every retrieval transparent. I could see exactly which chunks the bot pulled, and why.\n\n``` python\nfrom sklearn.feature_extraction.text import TfidfVectorizer\nfrom sklearn.metrics.pairwise import cosine_similarity\n\ndef retrieve(query, chunks, k=3):\n    vec = TfidfVectorizer().fit(chunks + [query])\n    scores = cosine_similarity(vec.transform(chunks), vec.transform([query]))\n    indices = scores.flatten().argsort()[-k:][::-1]\n    return [chunks[i] for i in indices]\n```\n\nThe app itself was a thin FastAPI server.\n\n``` python\nfrom fastapi import FastAPI\napp = FastAPI()\n\n@app.post(\"/ask\")\ndef ask(question: str):\n    chunks = retrieve(question, all_chunks)\n    context = \"\\n---\\n\".join(chunks)\n    prompt = f\"Answer using only the context.\\n\\n{context}\\n\\nQ: {question}\"\n    answer = call_model(prompt)  # a POST to the free endpoint\n    return {\"answer\": answer, \"sources\": chunks}\n```\n\nI pushed it to MonkeyCode’s free server. It booted. I felt like a wizard.\n\nFor the first three days, it was great. It remembered details I had forgotten. It quoted slide numbers. I started to trust it. Then I asked: “Is gradient descent guaranteed to find the global minimum?”\n\nHere’s what it answered:\n\n```\nAnswer: Yes. According to slide 8, gradient descent always converges to the global minimum.\n```\n\nThat was a lie. Slide 8 actually said that for convex functions, gradient descent converges to the global minimum. Slide 29 said the same thing with extra care: “for non-convex problems, only a local minimum is guaranteed.”\n\nThe bot had retrieved slide 8, ignored slide 29 because it didn’t score high enough, and produced an answer that sounded exactly like the kind of overconfident nonsense I’d written in my own notes.\n\nThe scary part was the confidence. If I hadn’t known the subject, I would have believed it. So I added a contradiction check.\n\nThe idea was simple: retrieve the top three chunks. If the top two were textually dissimilar, there was a good chance they were saying different things. That’s a crude heuristic, but it catches the most dangerous case — when the bot is about to quote only one side of a contradiction.\n\n``` python\ndef conflict_score(chunks):\n    vec = TfidfVectorizer().fit(chunks)\n    sims = cosine_similarity(vec.transform(chunks))\n    return sims[0][1]\n\nif conflict_score(chunks) < 0.3:\n    return {\"warning\": \"Retrieved slides seem to disagree. I'm not confident.\"}\n```\n\nI picked 0.3 by testing a handful of questions. It’s not a principled threshold. It’s a cheap tripwire.\n\nWith that in place, the same question gave:\n\n```\nWarning: Retrieved slides seem to disagree. I'm not confident.\n```\n\nThat was the moment I understood something important about RAG. The retrieval step decides what the model can say. If the retrieval is greedy and only returns one side of a contradiction, the model will faithfully present it as truth. The fix isn’t a better prompt. It’s a better retrieval policy.\n\nThe free server handled the load fine throughout the test. The 10 million tokens lasted about a week of heavy experimenting. I did run into rate limits near the end, which forced me to think twice before every request. That’s not a bad habit.\n\nWho should not copy this? Anyone who needs reliable citations, or whose documents contain diagrams and equations that don’t extract as text. My PDFs lost all the math. The bot never saw a sigma or an integral. If I’d tried to ask about a formula, it would have made something up.\n\nThere’s also a deeper lesson for beginners like me. It’s tempting to treat a free-tier LLM as a magic oracle. In practice, the model is the most reliable part of the pipeline. The broken parts are the ones you write yourself: extraction, chunking, retrieval. Those are where the lies come from.\n\nIf you want to see what your own notes look like through a naive RAG pipeline, MonkeyCode’s free tier is a sensible place to start. Just remember: the model will always sound sure. The retrieval is what you should be suspicious of.", "url": "https://wpnews.pro/news/the-day-my-lecture-notes-bot-contradicted-itself", "canonical_source": "https://dev.to/magickong/the-day-my-lecture-notes-bot-contradicted-itself-1nja", "published_at": "2026-09-01 12:40:21+00:00", "updated_at": "2026-09-01 12:54:29.102367+00:00", "lang": "en", "topics": ["artificial-intelligence", "developer-tools", "ai-tools"], "entities": ["MonkeyCode", "FastAPI", "scikit-learn", "pypdf"], "alternates": {"html": "https://wpnews.pro/news/the-day-my-lecture-notes-bot-contradicted-itself", "markdown": "https://wpnews.pro/news/the-day-my-lecture-notes-bot-contradicted-itself.md", "text": "https://wpnews.pro/news/the-day-my-lecture-notes-bot-contradicted-itself.txt", "jsonld": "https://wpnews.pro/news/the-day-my-lecture-notes-bot-contradicted-itself.jsonld"}}