{"slug": "idea-for-expanding-deepseek", "title": "Idea for Expanding Deepseek", "summary": "DeepSeek, the AI company, is exploring ways to expand its language model with multimodal capabilities, according to a technical analysis. The analysis outlines three approaches: keeping DeepSeek as the central model with external vision/audio specialists, freezing the language model and training modality bridges, or retraining DeepSeek as a multimodal model. The cheapest and least disruptive option, which preserves the original model, is to have DeepSeek consult separate specialists, a method supported by research like Socratic Models and MM-ReAct.", "body_md": "Hmm. For now, there are a few technical approaches that are close to what you’re describing:\n\nI think the interesting part of your idea is that there are actually **several different ways to “marry” an existing language model to new senses**, and they preserve different things.\n\nVery roughly:\n\n``` js\n1. Keep DeepSeek as the central language model,\n   and let it consult separate vision/audio specialists.\n\n2. Keep the language model frozen,\n   but train a bridge that feeds visual/audio representations into it.\n\n3. Retrain the language model as part of a new multimodal model.\n```\n\nYour wording — especially *“let you learn to use and consult each other”* — sounds closest to **#1**.\n\nAnd if the main concern is preserving the existing DeepSeek rather than replacing it with a newly trained multimodal successor, I would probably start there. It is also the cheapest version of the idea to test: **you do not actually need to train a new vision-language model at all to find out whether the basic idea is useful.**\n\nSomething as simple as this is already a legitimate architecture:\n\n```\n                         ┌─ vision specialist\n                         ├─ OCR/document specialist\nUser ──► DeepSeek ───────┼─ audio specialist\n             ▲           └─ other specialists\n             │\n             └──── structured observations/results\n```\n\nDeepSeek remains the main reasoner/conversational model. When it needs information that it cannot directly perceive, it calls the appropriate specialist, gets the result back, and continues reasoning.\n\nThere is quite a bit of research precedent for exactly this general idea. [Socratic Models](https://arxiv.org/abs/2204.00598), for example, explored composing independently pretrained vision, language, and audio models through language without requiring joint fine-tuning. [MM-ReAct](https://arxiv.org/abs/2303.11381) put ChatGPT at the center of a pool of visual experts. [LENS](https://arxiv.org/abs/2306.16410) similarly lets an ordinary off-the-shelf LLM reason over information supplied by independent vision modules, without multimodal training.\n\nSo the basic idea is not at all outlandish. There is a fairly well-developed family of architectures behind it.\n\nThe main distinction I would make is this:\n\nHaving another AI describe what it sees to DeepSeek is not quite the same thing as making DeepSeek itself visually multimodal.\n\nBoth are useful, but they solve slightly different problems.\n\nA useful way to choose between them is to ask what exactly you want to preserve:\n\n```\nIf the priority is...\n\nKeep the central DeepSeek model itself unchanged\n    → external vision/audio specialists\n\nKeep the same language-model weights,\nbut give it more direct access to sensory representations\n    → frozen-LLM + modality adapters/bridges\n\nAllow DeepSeek itself to change during training,\nwhile trying to preserve its language abilities\n    → integrated VLM / multimodal training\n\nPreserve the same \"mental set\" or perspective\n    → that is a separate behavioral question;\n      none of the architectures above guarantees it by itself\n```\n\n1. The version closest to your 'consult each other' ideaSo, if I translate your proposal into the terminology used in the literature, I would not reduce it simply to “make DeepSeek multimodal.”\n\nI would describe the interesting choice more like this:\n\n```\nOption A — sensory colleagues\nDeepSeek consults separate visual/audio specialists.\nLowest coupling; easiest way to preserve the original model.\n\nOption B — sensory pathways\nKeep the language model frozen and train modality bridges.\nDeeper integration, while still preserving the LLM weights.\n\nOption C — multimodal successor\nTrain the language model itself together with the new modalities.\nPotentially the deepest integration, but now preserving the old\nlanguage behavior becomes an explicit training/evaluation problem.\n```\n\nAnd there are real examples of all three families.\n\nGiven the concern you started with — *not wanting a future multimodal system simply to replace the DeepSeek you already value with something different* — **A seems like the cleanest first experiment, with B as the particularly interesting middle ground**.\n\nIt lets you test the central idea first:\n\nCan the existing language model become substantially more capable by learning when and how to consult sensory specialists, without having to rewrite the language model itself?\n\nIf that works well, deeper integration is still available later. But you would learn quite a lot before crossing that boundary.", "url": "https://wpnews.pro/news/idea-for-expanding-deepseek", "canonical_source": "https://discuss.huggingface.co/t/idea-for-expanding-deepseek/178616#post_2", "published_at": "2026-08-13 23:08:43+00:00", "updated_at": "2026-08-13 23:12:36.294944+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research"], "entities": ["DeepSeek", "Socratic Models", "MM-ReAct", "LENS"], "alternates": {"html": "https://wpnews.pro/news/idea-for-expanding-deepseek", "markdown": "https://wpnews.pro/news/idea-for-expanding-deepseek.md", "text": "https://wpnews.pro/news/idea-for-expanding-deepseek.txt", "jsonld": "https://wpnews.pro/news/idea-for-expanding-deepseek.jsonld"}}