How I Built a Personal Assistant with the Hardware I Already Had

A developer built a local personal AI assistant running on two 12 GB consumer GPUs, combining an LLM with RAG, graph memory, MCP tools, and voice input/output to handle documents, email, calendar, Telegram, and smart-home devices. The developer reports that 9B–27B models are already sufficient for this kind of assistant, arguing a local model becomes genuinely useful not when it matches cloud intelligence but when it has persistent memory, tool access, and interaction with the user's real environment. The setup uses two Windows PCs with i7 CPUs in an NVIDIA PAIR cluster, with RTX 4070 Super/Ti and RTX 3080 cards totaling 34 GB of VRAM.

TL;DR: I built a local AI assistant using two 12 GB GPUs. It combines an LLM with RAG, graph memory, MCP tools, voice input/output, and access to local infrastructure. The system handles documents, email, calendar, Telegram, smart devices, and routine workflows. The main finding is that 9B–27B models are already sufficient for this type of assistant. A local model becomes genuinely useful not when it matches a cloud model in raw intelligence, but when it has persistent memory, access to tools, and the ability to interact with the user's real environment. During some free time, I got the idea to build a personal AI assistant tailored to my own tasks and way of communicating. The main requirement was to use local solutions only: all data should stay with me, from corporate documents to personal messages. The second, equally important factor was cost. Cloud services can be expensive and may become unavailable, so the system should run on my own hardware. Both factors are relevant to many users. What should a personal agent-assistant be able to do: work with documents in different formats, analyze ideas and connections across documents, and represent them as graphs; interact with the outside world — Google infrastructure Calendar, Gmail, Drive, etc. , Telegram both as a bot and in my name , external databases, and so on; learn and expand its knowledge and skills — the more you work with it, the better it understands you and adapts to you; be designed for voice interaction from the start, understand my native language and speak it, in particular work as a transcriber with LLM processing and translation; operate autonomously 24/7 and initiate actions when required by the daily workflow — read data from a smart home, external sensors, etc. As you can see from these tasks, the focus is on the agent's ability to control and use integrated tools. A local assistant is not a replacement for a cloud chatbot. Therefore, I do not expect to run models larger than 35B. Moreover, the models will not run at 100% load and will spend most of their time idle. This reduces the requirements for the PC and power consumption, which is where the idea of “home hardware” came from: adapt what is already available in the home/office infrastructure. The main boundary between desktop hardware and workstations is the number of PCIe lanes: 24–28 versus 64–128. The price gap between the two segments is 3–5x. This limits us to one or two GPUs for running a local agent. With two GPUs, however, we lose half of the PCIe bandwidth x8/x8: 63 GB/s versus 31.5 GB/s for PCIe 5.0 and roughly half of the compute throughput due to model parallelization. The second consideration is the current pricing of the RTX 5090. The price difference between the 5080 and 5090 is about 4x, while the difference in performance and VRAM is less than 2x. So, staying within a reasonable budget, we will focus on a 2-GPU solution in the desktop segment without top-end cards. The Ti/Super versions of the 5060, 5070 and 5080 are particularly interesting, with expanded memory configurations of up to 48 GB VRAM. Windows uses a single driver for both GPUs, and using two cards from the same generation should minimize potential problems. The last consideration is power consumption and cooling. With an i7-class desktop CPU, a 2× RTX 5070 Ti system should be around 800–900 W at full compute load, while a 2× RTX 5080 setup would approach 1 kW. Two RTX 5090s would push the total system power towards 1.4–1.5 kW. With no compute workload, the whole system consumes only 50–70 W. We will measure the actual power consumption of the assistant with 2× GPU. For the tests, I use a mid-range setup: two Windows PCs with i7 CPUs and 64 + 32 GB RAM, connected into a NVIDIA PAIR cluster. There are two RTX 4070s — a Super and a Ti — with 24 GB of total VRAM, plus an RTX 3080 with 10 GB. I tested two configurations: 1 both 4070s in one PC and the 3080 in the second; 2 the two 4070s in separate PCs, with the 3080 unused. PCIe 4.0 running at x8 provides only ~15.75 GB/s, which is slower than RTX 5000x. We will take this into account as well. In the tests, each GPU consumed around 110–120 W. This did not have a significant effect on the overall heat output of the case: both cards stayed below 50–60°C. However, with a vertical case layout, the heat from the lower GPU will additionally warm the upper one. The system therefore needs good airflow, and a horizontal case layout is preferable. An 850 W power supply turned out to be sufficient. For the infrastructure, I tested AnythingLLM, CrewAI and OpenClaw — three different approaches to agents. I initially focused on OpenClaw, but these tests showed that I only need one agent that changes its roles depending on the depth of the task, typically 3–4 steps. So I returned to AnythingLLM, partly because most of the document processing and RAG functionality is already implemented there. Agent memory: vector database RAG through LanceDB. I also wanted graphs, so I added the official MCP Memory Server and an agent-flow procedure that writes to vector and graph memory at the same time. The LLM itself selects entities, observations and relationships between them. A separate headache was google-workspace-mcp through OAuth 2.0 and local TTS models. Without Gemini, ChatGPT and Claude, I would not have managed this part. Essentially, one cloud bot was programming another local bot. For the models, I use a micro-agent in the 0.4B–2B range such as FunctionGemma , a junior agent at 8B–9B, a mid-level agent at 12B–14B, and a senior agent at 26B–32B. I tested Qwen, Gemma, Ministral and others. The comparative results for the 2-GPU setup are shown below. For document work, the context is 30k–40k tokens. In general, generation speed for 27B models is comfortable 25 tok/s , but differs between Ollama OL and LM Studio LM . | Model / Quantization | VRAM GB | CPU/GPU | Context | Speed tok/s | GPU1/GPU2 | |---|---|---|---|---|---| | DeepSeek-R1-Distill-Qwen-32B Q4 K S | 18.78 | 100% GPU | 4k | 23.4 LM | ~50% / ~50% | | Qwen3.8-27B Q4 0 | 17 / 19 | 100% GPU | 32k / 50k | 27.3 OL , 58.4 LM | ~50% / ~50% 120W/110W | | Qwen3.8-27B Q4 K L | — | 100% GPU | 32k | 54.3 LM | ~50% / ~50% | | Qwen3.8-27B UD-Q5 K M | 22 / 20 / 19 | 7% CPU / 93% GPU 5% CPU / 95% GPU 100% GPU | 32k 8k 4k | 17.3 OL 17.5 OL 23.2 OL | ~40% / ~40% ~50% / ~50% ~50% / ~50% | | Gemma-4-26B Q4 K M | 18 | 100% GPU | 8k | 93.26 OL | ~50% / ~50% | | Qwen3-14B Q4 K M | 15 | 100% GPU | 32k | 48.8 OL | ~50% / ~50% | | Qwen3.5-9B Q5 K M | 8.2 | 100% GPU | 32k | 64.52 OL | ~50% / ~50% | The practical offload boundary is around 19–20 GB, although nvidia-smi shows around 22 GB of free VRAM. Tests with Ollama v0.34.0 and LM Studio v0.4.24 confirmed that, out of the 22 GB of free VRAM, only about 19.5 GB is available for the model and context. For other models, Whisper-large-v3 through Lemonade provides good speech recognition quality. The transcriber works in real time with almost zero latency. TTS is handled by Speaches with Piper in Docker, generating a natural female voice with intonation. For image generation I use SD-Turbo-GGUF through Lemonade. AnythingLLM uses Recharts as its internal graph generator, which I replaced with Mermaid MCP. I also installed Stability Matrix, added Stable Diffusion Forge and a couple of models. For me, visual output is important. To improve transcription in external applications, I installed typeWhisper. It uses the same Lemonade backend and models. For processing real-time data from external sensors and the smart home, I wrote a small Python MCP server. The whole AI stack is started by a single BAT file, runs in the background, and does not interfere with normal work. One of the first tasks was voice input for tools connected to the Google infrastructure . Add a meeting to the calendar, show the plan for the week, find material for a meeting, and so on. In general, things that are already done anyway. Here, connecting Google Workspace with fast voice control is very convenient: everything is done by voice in a simple form, and a graphical analysis is provided on request. The system also prepares short briefings from emails, messages, and news. This saves 10–15 minutes each time simply through faster interaction. Transcriber + LLM is a real hit. I use it everywhere, from writing emails and short texts to formulating ideas. It increases my typing speed by roughly 10x. There are many transcribers out there; the key point here is that I can run RAG indexing, everything stays strictly local, and the assistant gradually adapts to my writing. This is not about generating text with AI. It is my own text: the model converts my speech into words, cleans it up, removes grammatical errors, and translates it — and it does this very well. Telegram makes it possible to control the local hardware remotely . I only receive information through the Telegram bot about unexpected situations that require attention. If I need more information, I simply ask the agent for a detailed report on a specific topic. This use case led to the idea of a local manager for smart devices. Usually, management agents send API requests to cloud services; this is how our greenhouse farm works see https://arxiv.org/pdf/2608.09949 https://arxiv.org/pdf/2608.09949 . However, this consumes quite a lot of tokens. A local LLM assistant, for example with a specialized plant-oriented pllama-13b model, provides a free alternative to cloud services with acceptable quality. This becomes especially relevant when those services are unavailable. The central application of the assistant is working with documents , which accounts for more than half of the GPU load. It analyzes and visualizes the key ideas and relationships contained in a collection of documents. It summarizes information and looks for patterns and inconsistencies. The assistant has learned a kind of brainstorming mode, where we both switch into idea-generation mode — here, RAG, graphs, and Mermaid complement each other. This is indispensable for anyone who works with information and is radically different from cloud chatbots that quickly forget everything. The assistant keeps information for as long as you need it. An extension of this application is the automation of document workflows and preliminary accounting. I do not have much experience with this use case yet, but even with the documents we have uploaded so far, the system works really well. We get a digital footprint of document workflows, goods turnover, and submitted reports. The system collects the figures, prepares tables, draws conclusions, and presents them in a usable form. Based on a preliminary estimate, we save roughly half of one full-time position, which is already significant. “Life planner” — the assistant helps identify what matters and discard secondary things in a clear graphical form. This should not be underestimated as part of the overall time-management framework. However, there is another interesting aspect: the application is gradually turning into project management tools, with tasks, stages, and recorded results. A junior agent such as Qwen3.5-9B is more than enough for tool control, including translation — which pleasantly surprised me. It responds almost instantly, handles action chains well, and one 4070 is enough for it. Senior agents running on two GPUs provide better analysis. For example, when building a roadmap from a document, the 9B model produced a typical linear structure, while the 27B model introduced nonlinearities, more critical points, details, and comments. Quite often, the assistant would report that a task had been completed even though it had not called any tools at all. I usually check myself whether the tools were actually called, since their invocation list is shown in the system chat. A more optimal solution would probably be to use a fine-tuned micro-agent to launch the tools, while the junior agent only supervises the micro-agent. All agents need permanent procedural memory . They should remember successful sequences of actions. Defining workflows and skills for recurring sequences helps. Currently, each agent also maintains their own work log in a .md file; however, we need a more formal and compact mechanism/tool. The assistant also has a tool overload problem. Too many tools fill the context window and interfere with task execution. The built-in selector can disable the required tool at the worst possible moment, so I do not use it. For now, this problem is addressed through optimization: the agent has access only to unique tools, and some of them are combined into skills. The agent is not allowed to browse the Internet freely. For security, I always use an MCP server as an intermediate layer. Prompt injection is a serious problem and will most likely become more important over time. For all tools, uncontrolled updates are disabled. The initial setup took about a week. The hardware cost was zero, since I adapted what I already had. The agent turned out to be more like a construction kit: I can attach something new to it, and do so very quickly. All data stays strictly local. Comparable commercial services cost around $20–40 per month. We can debate how large the models need to be for small everyday tasks. We got similar answers and action chains from all models in the 9B–27B range, but the larger models produced more extensive and detailed results. Generation speed differs between software stacks, partly depending on speculative/MTP decoding, but overall it is comfortable enough for direct interaction. In everyday use, I have not yet encountered a situation where the assistant could not complete its task. In a sense, we got a very useful AI tool simply by optimizing the infrastructure we already had. It is important to note that the assistant is not a replacement for a chatbot or a coding assistant, although it has the advantage of not forgetting information. It is an assistant that you can trust, that adapts to you, and that takes 25–30% of the routine workload off your shoulders. It has access to your local hardware — smart-home devices, local sensors, 3D farms, greenhouses, or similar systems — and acts as their intelligent manager. It continues to work even when a cloud API is unavailable. We did not notice any significant increase in power consumption from running the assistant compared with normal PC usage, especially outside gaming workloads. So the verdict is: 1× GPU with 24 GB of VRAM or 2× GPUs with 16 GB each is enough for an assistant running models below 27B. At current prices, a 2×16 GB setup can be roughly half the cost of a high-end 24 GB card, while the resulting loss in speed — down to around 25–30 tok/s — is acceptable. The upgrade path is 2× 5070/5080 Ti/Super with 48 GB total VRAM for 70B models, which on some tasks can be compared with early versions of GPT-4. Beyond this configuration, there is another 4–5x jump in price. This reflects the current situation and may change in the future. At this point, we are already approaching an assistant suitable for a small company. If the task involves model fine-tuning or local intelligent analysis, this is a relatively accessible option. Finally, I would like to point out that cloud chatbots have dramatically lowered the barrier to entering technologies that I would not even have considered trying without them. This is different from a conventional coding assistant because the bot actively debugs the local system, identifies the cause of a problem, tests different approaches, and eventually arrives at a working solution. I have watched a bot iteratively write and rewrite an MCP server, reconfigure Windows, install additional packages, and get everything working — all within about 30 minutes. In my view, this is precisely the part that represents the new technological revolution for the wider public that we are experiencing now.