{"slug": "my-email-client-runs-my-first-ml-classifier-it-trains-itself-from-my-own", "title": "My email client runs my first ML classifier. It trains itself from my own keystrokes.", "summary": "A developer built an open-source email client, Ai-Email-summariser, that connects to Gmail via OAuth2 (or IMAP) and triages the inbox with a local scikit-learn pipeline: TF-IDF vectorization capped at 5,000 features with unigrams and bigrams feeding a 100-estimator Random Forest using class_weight='balanced'. Classification runs locally and free, while GPT-3.5-turbo is called only to summarize emails the user actually opens, and the model retrains in milliseconds from a deduplicated JSON feedback file, persisting to joblib. The developer argues the feedback loop, not the model, is the product: fresh installs return a neutral 0.5 confidence defaulting to important, and every correction refines a classifier trained on the user's own definition of junk.", "body_md": "Before I had any professional software title, I built the tool I actually needed: something that triages my inbox before I open it. [Ai-Email-summariser](https://github.com/Reaver1000/Ai-Email-summariser) connects to Gmail over OAuth2 (or any IMAP server as a fallback), classifies each email as important or junk, labels them accordingly, and uses GPT-3.5-turbo to write a short summary of the ones I open.\n\nThe design decision I keep getting asked about in interviews is why the classifier is TF-IDF plus Random Forest instead of an LLM. The honest answer has three parts, and the third one is the interesting one.\n\nSummarisation genuinely needs a language model. Classification does not. An API call to decide \"is this junk\" costs money, adds latency, and fails at a different level than local code fails. The classifier runs locally with scikit-learn, so triage is free, instant, and works offline. GPT-3.5-turbo only gets called when I actually open an email to read it. Splitting the two jobs meant one API call per email I read instead of one per email that exists.\n\nThe vectoriser is TF-IDF with a 5000-feature cap, English stop words removed, and unigrams plus bigrams. The classifier is a Random Forest with 100 estimators and `class_weight='balanced'`, which matters because my real inbox is not 50/50: the junk-heavy reality would otherwise let a model score well by predicting \"junk\" constantly.\n\nThe input features are deliberately crude: subject, sender, and body concatenated into one string. No fancy sender-reputation graph, no header analysis. I wanted the first version to be auditable, and with TF-IDF I can print which words pushed a classification either way. That turned out to matter for trust: when the model flags something important that I thought was junk, being able to see why makes me fix my assumptions instead of distrusting the tool.\n\nAn untrained classifier is useless, and a tool that demands training data before it works is a tool nobody sets up. So the first prediction on any fresh install returns a neutral 0.5/0.5 with the label defaulting to important, and the interface shows that confidence number next to every email. You are never asked to trust the model. You are asked to correct it.\n\nEvery correction (`i1` marks email one important, `j2` marks two junk) goes into a JSON feedback file that deduplicates by a composite id built from the subject, sender, and a hash of the body preview, so marking the same email twice updates the label instead of duplicating the training row. Each retrain happens on demand from that file, milliseconds, on my machine. The model persists to a joblib file and reloads on the next run.\n\nThis is the part I would defend hardest in a design review: the feedback loop is the product. The model itself is the boring part, a textbook scikit-learn pipeline, and that is fine. What makes the tool usable by week two is that it becomes *mine* by week two, trained on what I personally consider junk.\n\nEight tests in unittest with mocked Gmail responses: prediction structure, the train-and-persist path, feedback deduplication (adding the same email twice updates one row and flips the label), the Gmail integration with a mocked service, and three edge cases that taught me the most. An empty email, a 10,000-character body, and a body full of special characters including multi-script text all have to return a valid prediction structure rather than an exception, because the alternative is the classifier crashing on the one weird email that matters most.\n\nThe duplicate-feedback test is the one I would keep if I could keep only one. It encodes the assumption that feedback should correct, not accumulate, which is exactly the bug that would silently poison the training data if someone refactored the feedback manager later.\n\nTwo things, both known. First, the neutral fallback on prediction failure is silent; it should log so I can see how often the vectoriser is being fed garbage. Second, there is no evaluation set at all, because the feedback file doubles as training data. The moment the feedback pool gets large enough, I want a held-out split so accuracy is measured instead of felt.\n\nThe stack is Python with scikit-learn, the Gmail API with IMAP fallback, OpenAI for summaries only, and unittest with mocks. Nothing exotic, and that is the point: this is what a first production-shaped ML system looks like when the goal is a tool you run every day rather than a benchmark score.\n\nFull repo: [https://github.com/Reaver1000/Ai-Email-summariser](https://github.com/Reaver1000/Ai-Email-summariser)", "url": "https://wpnews.pro/news/my-email-client-runs-my-first-ml-classifier-it-trains-itself-from-my-own", "canonical_source": "https://dev.to/reaver1000/my-email-client-runs-my-first-ml-classifier-it-trains-itself-from-my-own-keystrokes-30ki", "published_at": "2026-11-16 09:44:31+00:00", "updated_at": "2026-11-16 09:51:14.613761+00:00", "lang": "en", "topics": ["machine-learning", "ai-tools", "ai-products", "developer-tools"], "entities": ["Gmail", "GPT-3.5-turbo", "scikit-learn", "Ai-Email-summariser", "GitHub"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/my-email-client-runs-my-first-ml-classifier-it-trains-itself-from-my-own", "markdown": "https://wpnews.pro/news/my-email-client-runs-my-first-ml-classifier-it-trains-itself-from-my-own.md", "text": "https://wpnews.pro/news/my-email-client-runs-my-first-ml-classifier-it-trains-itself-from-my-own.txt", "jsonld": "https://wpnews.pro/news/my-email-client-runs-my-first-ml-classifier-it-trains-itself-from-my-own.jsonld"}}