Local LLMs/VLMs for Physical AI engineering workflows — model suggestions wanted

Afrid Ali is developing EdgeAI Forge, an early-stage local-first architecture for using specialized AI agents in industrial vision, ROS 2, robotics, and edge-deployment workflows, and is seeking model recommendations and evaluation ideas for local coding agents, VLMs for industrial inspection, smaller reasoning models, embedding models, and runtimes across workstation GPUs and Jetson-class devices. The proof of concept routes Planner, Vision, and ROS prompts to Ollama, with optional cloud escalation and explicit human review before physical deployment.

I am experimenting with EdgeAI Forge , an early-stage local-first architecture for using specialized AI agents in industrial vision, ROS 2, robotics, and edge-deployment workflows. The intended split is: - local models for frequent coding, planning, documentation, private-data processing, and visual reasoning; - optional cloud escalation for difficult or large-context tasks; and - explicit human review before any physical deployment or machine-control action. The current proof of concept routes Planner, Vision, and ROS prompts to Ollama. I am looking for practical model recommendations and evaluation ideas for: - local coding agents that produce structured Python/C++/ROS 2 artifacts; - VLMs for industrial image inspection and pipeline selection; - smaller reasoning models for planning and validation; - embedding models for project memory and retrieval; and - runtimes that work well across workstation GPUs and Jetson-class devices. I am especially interested in measured tradeoffs—VRAM, latency, structured-output reliability, quantization, tool use, and license constraints—rather than leaderboard scores alone. Project: GitHub - afridali123/EdgeAI Forge · GitHub Longer background: EdgeAI Forge: My Journey Toward a Local Agentic AI Platform for Industrial Automation This is an early architecture discussion, not a finished product announcement. Suggestions for models, datasets, evaluation harnesses, or related projects are welcome.