# Using Agents to Maximize NVIDIA Jetson Memory Usage at the Edge > Source: > Published: 2026-08-13 12:00:00+00:00 An earlier article here on EE Times, “[Built-In Memory. Built-In Confidence](https://www.eetimes.com/built-in-memory-built-in-confidence/).”, covered how Jetson’s integrated memory architecture takes supply chain risk off the table. This article explores how far a developer can optimize the software stack within a fixed memory budget and why that budget can accommodate far more than developers may initially perceive. On any edge device, the memory has always felt like a hard ceiling. An application must operate within the board’s available memory capacity. [NVIDIA Jetson](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/) turns out to be more flexible thanks to hardware and software codesign and optimization. When properly configured and optimized, a Jetson module can run meaningfully bigger AI workloads than what would be initially perceived through its memory specifications. Plus, NVIDIA’s newest software releases, such as the Jetson agent skills, give engineering teams huge development velocity increases and make it easier to get to finished designs. ## The Memory Budget Is a Program Problem, Not Just a Technical One For program managers and lead engineers building edge AI systems, memory capacity shows up on a spec sheet as a fixed number, but its real effect is felt in the program plan. A model that does not fit in the memory budget of the module a team has already committed to does not just fail a benchmark. It forces a choice: descope the application, re-architect around a smaller model causing schedule delay, or move to a larger-memory, more expensive module late in the design cycle, often after the carrier board and thermal solution have already been locked. The last option is the one teams cannot afford. Memory pricing has risen sharply across the DRAM and LPDDR markets through 2026, and industry analysts expect supply constraints to persist. Critically, the impact is no longer limited to future design decisions, rising costs are now disrupting active production shipments, not just programs still in the design phase. A late-stage jump to a larger module does not just add additional CapEx and bill-of-material cost (COGS) to the design due to carrier board revisions, and new qualification cycle, etc. but also disturbs the existing market positioning of the product. The alternative is to treat the flexible software stack as part of the memory budget from the start. A structured optimization pass, applied before a developer concludes it needs more hardware, routinely recovers more effective memory than most teams assume is available. What’s more important, proper memory optimization can often help teams move one to two memory SKUs down to save cost. [View All](https://www.eetimes.com/category/sponsored-content/) ## Why This Problem Hits Hardest at the Edge Cloud infrastructure teams have a structural advantage when memory gets tight: elastic resource pooling. If a workload grows, a hyperscaler can shift it to different hardware, add capacity, or rebalance across a fleet. The software adapts, and the workload continues. Edge deployments do not have that flexibility. For edge developers, memory optimization is not a nice-to-have performance tweak, especially in the current memory market. Right-sizing your application and reducing memory usage are critical, not only during the design phase, but also for products already in production. ## The Solution: Software-Defined Memory Headroom NVIDIA breaks the optimizable memory on a Jetson module into five layers. Each layer is a configuration or software decision rather than a hardware change, and the savings compound as a developer works down the stack, from the board support package up through the model itself. Layer | What changes | Typical savings | | BSP & OS services | Disable unused services such as display graphics and network services; reclaim reserved memory from specific carveouts | ~1 GB | | Inference frameworks | Serving a serial workload on a lightweight runtime (llama.cpp) instead of a full vLLM instance | ~5 GB | | Model quantization | BF16 down to FP8, INT4, or NVFP4, matched to the accuracy floor the use case requires | 4 to 10 GB | | Combined | All savings combined | Up to ~10-12GB | **Jetson BSP and OS Services** The foundation layer covers the Jetson Board Support Package and [NVIDIA JetPack](https://developer.nvidia.com/embedded/jetpack/). Disabling unused display and GUI services, reclaiming memory reserved for unused hardware engines, and removing unnecessary drivers and background processes, such as display, audio, and camera components in a headless deployment—can recover roughly a gigabyte before application-level optimization begins. **Inference Frameworks** The serving runtime is one of the largest levers available. Handling a serial, lower-priority workload with a lightweight runtime such as llama.cpp, rather than standing up a second full vLLM instance to run it, can save roughly 5 GB on its own, memory that would otherwise be reserved for a heavier framework the workload does not require. **Model Quantization** Quantization is the largest memory saving opportunity in the stack. Moving a model from BF16 down to FP8, INT4, or NVIDIA’s NVFP4 format powered by NVIDIA Blackwell architecture can reclaim 4 to 10 GB, depending on model size. To make that concrete: quantizing a vision-language model from FP16 to 4-bit on the NVIDIA Jetson Orin Nano 8GB reduced its memory footprint from 6.6 GB to 2.2 GB, a three-to-one compression on the model alone, as [documented in this Reachy Mini Assistant Project](https://www.jetson-ai-lab.com/tutorials/reachy-mini-jetson-assistant/#how-jetson-device-skills-helped) tutorial. The discipline that matters here is choosing the lowest precision that still clears the accuracy bar for the specific use case, not defaulting to the most aggressive setting available. ## Jetson Agent Skills: Automating the Optimization Loop NVIDIA’s newest tool for working through this stack is [Jetson Agent Skills,](https://forums.developer.nvidia.com/t/jetson-agent-skills-ai-assisted-workflows-for-device-bsp-customization/374150) introduced with JetPack 7.2. We have two open-source Jetson Agent Skills. First, [Jetson BSP Skills](https://github.com/NVIDIA-AI-IOT/jetson-bsp-skills) which guide the BSP customization lifecycle, from workspace setup through flashing and validation. Second, [Jetson Device Skills](https://github.com/NVIDIA-AI-IOT/jetson-device-skills) which help with post-boot device management, diagnostics, performance tuning, and AI workload setup The skills distill more than a decade of NVIDIA Jetson development expertise into automated workflows that handle key tasks: optimizing memory usage across the full software stack, customizing the operating system for the target hardware, and benchmarking models to find the best fit for the device. Each step runs directly on the physical device and is verified against real telemetry, replacing manual trial and error with evidence-backed configuration decisions. With the new Jetson Agent Skills, teams can optimize the entire software stack and reduce memory usage in days instead of weeks. Available across the entire Jetson family including Jetson Thor and Jetson Orin, these AI-driven workflows enable more capable applications to run on lower-memory footprints, reducing system cost and accelerating deployment. NVIDIA JetPack 7.2 also adds official support for the [Yocto Project](https://www.yoctoproject.org/), letting teams build lean OS images containing only the services, drivers, and libraries a given deployment actually needs, and introduces a Super Mode for the NVIDIA Jetson AGX Orin 32GB module that raises its rated AI performance from 200 to 241 TOPS with no hardware change. Teams can often move down one memory SKU without compromising performance, reducing system cost. ## Proof It Works: Real Savings, Real Products The payoff for all of this is straightforward. A team that assumed it needed a larger-memory module for a given workload may find that a smaller-memory one now clears the bar, at the same performance target and a lower module cost, with no hardware redesign. NVIDIA frames the outcome as “the flexibility to move down one memory SKU within the same product tier” without giving up performance. It also reframes the sourcing conversation: a module chosen for its integrated, validated memory can turn out to be the cheaper module too, once software is accounted for. **Case Study: Connect Tech / CTai LABS** NVIDIA Elite Partner [Connect Tech](https://connecttech.com/), through its [CTai LABS](https://ctailabs.ai/) engineering group, [published a detailed account of this in practice.](https://ctailabs.ai/wp-content/uploads/2026/07/We_Budgeted_64GB_Super_Mode_Said_Otherwise-CTaiLABS.pdf) The team had scoped a real-time video AI system — their Scene Analyzer Agent (SAA), a dual-model agent running simultaneous scene analysis and natural-language alerting built on NVIDIA Video Search and Summarization (VSS) and Cosmos — for a 64GB Jetson AGX Orin module. After applying agent-driven memory optimization, they shipped the finished system on a single 32GB module instead. Headless operation, quantizing the vision-language model from BF16 to FP8, right-sizing the serving framework’s memory reservation, and running the second model on llama.cpp rather than a duplicate serving instance together freed enough room to add the entire second model rather than cut anything. By Connect Tech’s account, idle free memory grew from roughly 5.3GB to about 10GB, VLM throughput improved by 65 percent, time to first token dropped roughly sixfold, and the finished system sustained three live 720p video streams on a single module. “This is work that would typically require a multi-engineer team several weeks — compressed into days by a single engineer supervising the agent-driven bring-up loop,” said Rob Callaghan, Chief Product Officer, Connect Tech **Beyond a Single Deployment** [NVIDIA’s blog for the new Jetson T3000 and T2000](https://blogs.nvidia.com/blog/jetson-thor-robotics-edge-ai-agent/) modules cite similar results elsewhere in the ecosystem: - Humanoid robotics leader UBTech ,working with Connect Tech, reduced memory usage by up to 15 GB, enough to move from a 64 GB AGX Orin module to the 32 GB version. - Smart retail company SandStar cut memory use by up to 4 GB, enabling a move from a 16 GB Orin NX to the 8 GB version with no performance loss. - Companion robotics maker GROOVE X, creator of the LOVOT robot, used NVIDIA Jetson’s heterogeneous accelerators to redistribute workload and open up lower-memory deployment options. - Intelligent transportation company NoTraffic cut memory use by 30 percent on NVIDIA Jetson TX2 NX, freeing room to add AI capability without upgrading hardware. ## The Scalable Jetson Platform NVIDIA recently extended the Jetson Thor family with two new modules, the T3000 and T2000, aimed at mainstream robotics and edge AI rather than only high-end deployments. The T3000 delivers inference performance similar to the flagship T5000 at roughly half the size and power; the T2000 serves as a smaller entry point into the Thor architecture. Both are expected to ship in the first quarter of 2027. Developers can begin working with T3000 emulation mode on the Jetson AGX Thor developer kit through JetPack 7.2.1. With their arrival, NVIDIA’s edge AI lineup spans a wide performance range, all sharing the same JetPack and CUDA-X software foundation. Teams do not rebuild their stack when they move up or down that range: the optimization discipline covered above applies whether the target module is an Orin Nano or a Thor-class board. Module | Performance | Description | Availability | | Jetson Orin Nano 8GB | 67 TOPS | Entry-level edge AI. With software optimization, quantized multi-billion-parameter models today. | Available now | | Jetson Orin NX 8GB | 117 TOPS | Multi-sensor AI with headroom for concurrent pipelines. | Available now | | Jetson Orin NX 16GB | 157 TOPS | Complex models and high-resolution simultaneous streams. | Available now | | Jetson AGX Orin 32GB (Super Mode) | 241 TOPS | New JetPack 7.2 Super Mode approaches 64GB-class performance at a lower module cost, no hardware change required. | Available now | | Jetson AGX Orin 64GB | 275 TOPS | Full-scale autonomy and complex 3D perception. | Available now | | Jetson T2000 | 400 FP4 TFLOPS, 16GB | Entry point to the Thor architecture for visual AI agents and autonomous mobile robots. | Q1 2027 | | Jetson T3000 | 865 FP4 TFLOPS, 32GB | Similar inference performance to the T5000 at roughly half the size and power. | Q1 2027 | | Jetson T4000 | 1,200 FP4 TFLOPS | Blackwell-powered edge AI for LLMs and real-time human-robot interaction. | Available now | | Jetson T5000 | 2,070 FP4 TFLOPS | Frontier generative AI and real-time reasoning at the edge. | Available now | NVIDIA is also extending that discipline to the model layer with [Cosmos 3 Edge](https://huggingface.co/blog/nvidia/cosmos3edge), a compact world foundation model that developers can post-train for a specific robot or sensor setup in about a day and deploy on NVIDIA Jetson Thor modules for real-time vision analysis and on-device policy. ## The Decision-Maker Summary **Lower Module Cost** Software-optimized memory headroom means a workload that appeared to require a larger, more expensive module can often run on a smaller one instead. That is a direct bill-of-materials saving that compounds across a production run, on top of whatever advantage a team already gets from Jetson’s integrated memory architecture. **Faster Development** Jetson Agent Skills replace manual, trial-and-error memory tuning with an automated, telemetry-verified process. Bring-up and optimization work that used to take a multi-engineer team several weeks, as in the Connect Tech example above, can be completed by a single engineer in days. **Platform-Wide Payoff** The same JetPack and CUDA-X software investment carries forward across the entire Jetson family, from Orin Nano through the newly announced T2000 and T3000. Optimization work done at one tier is not thrown away when a program scales up or down; it is the same discipline applied to a different module. ## Next Steps - Explore the [complete Jetson family](https://www.arrow.com/en/resources/articles/2026/03/plan-for-tomorrow-with-nvidia-jetson.html)at Arrow Electronics and developer kits to get started. Modules are also in stock and ready to ship. - Developers targeting the new T3000 can start now using emulation mode on the [Jetson AGX Thor](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-thor/)developer kit with JetPack 7.2.1; T2000 emulation support follows in a later release, with both modules shipping in Q1 2027. - Leverage [agent skills](https://forums.developer.nvidia.com/t/jetson-agent-skills-ai-assisted-workflows-for-device-bsp-customization/374150)and latest[JetPack SDK page](https://developer.nvidia.com/embedded/jetpack/downloads)to accelerate development and right size your application memory to move one SKU down. - The same module that eliminates memory sourcing risk can now also be the cheaper module. Software optimization and supply chain resilience point to the same hardware choice.