This topic has background found here that was inadvertently marked closed New to This LLM
First off thanks @John6666 I have my testbench doc and files 90% complete. So I am almost ready to start testing in a more coherent way. However, I remembered an issue I ran into while setting my system up and went through my notes to check if it had been corrected.
Also to note, i am new to all this, not an IT worker, consider myself at the level slightly above a novice. I have to use Claude to explain things, help write up the technical stuff and help set up my stack. While working on my testing bench I remembered during setup Claude and I discovered that “native” mode does not work for RAG search.
Here are my specs:
MacBook M5 Max (18-core CPU, 40-core GPU, 128GB unified memory), macOS Tahoe
**•** Open WebUI v0.10.2, self-hosted via Docker Compose (“myLLM”)
**•** Docker Model Runner (llama.cpp-metal backend), not Ollama
**•** Custom model: Gemma 4 (base ai/gemma4, 7.52B, Q4_K_M)
**•** Embedding: SentenceTransformers (all-MiniLM-L6-v2)
**•** Knowledge Base: mix of PDFs and hand-written markdown reference files
The issue is that Native mode’s query_knowledge_files and kb_exec both fail to retrieve from attached knowledge bases with ‘str’ object has no attribute ‘items’; Legacy mode works fine with the same model and knowledge base. I did open a bug report via github (#26880). Yesterday I updated open webUI and docker to their latest versions and the issue still persists.
For my stack I considered LM Studio but went with Docker for the containerization. I actually started with Ollama, running inside Docker. It hit a hard wall on my Mac, Docker Desktop couldn’t pass my M5 Max’s GPU through to Ollama’s container, so it ran CPU-only, plus a 28GB memory ceiling regardless of my 128GB. Docker Model Runner replaced it because it runs natively on the Mac host, full Metal GPU access, full RAM, no container ceiling.
Now I am running my models in “legacy” mode which is now unsupported so until (or if) they fix this issue I seem to be stuck with this limit.
I have a few thoughts and questions.
In my mind I think I should rethink using open webUI and consider using AnythingLLM as my frontend interface before starting serious testing unless anyone has any ideas how to solve my current issue. My thought is that if I need to switch frontends, why not do it now.
Would love to get your perspective on 1) how to fix my native mode issue or 2) using AnythingLLM.
Tim