# Build On-Device AI Companions with the NVIDIA ACE Game Agent SDK and Unreal Engine 5 Plugins > Source: > Published: 2026-06-16 17:00:00+00:00 NVIDIA RTX technologies are deeply integrated into Unreal Engine 5 through the [NVIDIA RTX Branch of Unreal Engine](https://developer.nvidia.com/game-engines/unreal-engine/rtx-branch) and the [NVIDIA DLSS Unreal Engine plugin](https://developer.nvidia.com/rtx/dlss). This provides developers with direct access to advanced rendering, frame generation, and ray-traced lighting. NVIDIA is expanding this integration with new tools for building on-device AI characters and gameplay, as announced at [Unreal Fest 2026](https://www.unrealengine.com/unreal-fest). This post covers: - The new [NVIDIA ACE Game Agent SDK](https://developer.nvidia.com/ace-for-games)for building on-device AI companions - New [NVIDIA ACE Unreal Engine 5 plugins](https://archive.docs.nvidia.com/ace/ace-unreal-plugin/2.5/)for ASR, LLM, and TTS with blueprint and C++ support - A live [Level Up with NVIDIA webinar](https://gateway.on24.com/wcc/experience/elitenvidiabrill/1407606/3849291/level-up-with-nvidia?linkId=100000374605923&linkId=100000415844321&linkId=100000419915956)on June 30 covering the new ACE plugins - How to get started with DLSS 4.5 in Unreal Engine **Build on-device AI companions more easily with NVIDIA ACE ** AI is unlocking next-generation NPCs that deliver experiences that were previously impossible. For example, KRAFTON recently showcased Ally, an AI teammate in *PUBG: BATTLEGROUNDS *that uses natural voice interaction to understand player intent, interpret gameplay, and respond dynamically in real time. Instead of following rigid scripts, Ally cooperates, adapts to combat or looting, and communicates naturally. And it runs AI fully on-device powered by GeForce RTX. *PUBG* Ally will soon be available in open beta. Building on-device AI NPCs requires not only choosing the right AI models, but also integrating them into a seamless, high-performance pipeline. Developers must overcome hurdles to deliver a viable experience, such as minimizing conversational latency for instantaneous responses, syncing global and dynamic game states, and preventing NPCs from falling into infinite loops. With the NVIDIA ACE Game Agent SDK Beta and Unreal Engine 5 plugins, game developers and ISVs can more easily build and deploy highly responsive, on-device AI companions. **What is the NVIDIA ACE Game Agent SDK?** The NVIDIA ACE Game Agent SDK is a lightweight, C/C++ agentic framework designed for seamless, native in-game integration. It is an open source, fully customizable solution optimized for small models and hardware-accelerated on NVIDIA RTX. Developers interact with the SDK through three core API categories: **Agent API:** A stateful API that owns chat history and autonomously drives multistep, tool-assisted reasoning with minimal application code**Chat API:** A stateless API that gives developers direct control over inference**RAG API:** An API that enables semantic, lexical, and hybrid knowledge retrieval from developer-built databases, grounding model responses in up-to-date knowledge Together, these tools empower NPCs to dynamically perceive, reason, and act within the game world based on real-time player input. The tools have been used and battle-tested in real games, such as with the experimental in-game AI advisor in *Total War: PHARAOH*. The advisor—rendered as an ancient Egyptian pharaoh—provides contextual advice like recommending court actions and building construction to suppress rebellions, with the playtest program launching in 2026. It uses a [retrieval-augmented generation (RAG)](https://www.nvidia.com/en-us/glossary/retrieval-augmented-generation/) architecture that queries 1,200+ interlinked game data tables to answer player strategy questions in real time. To start building the future of gaming, download the [NVIDIA ACE Game Agent SDK](https://developer.nvidia.com/ace-for-games) in beta. **What’s in the new suite of NVIDIA ACE Unreal Engine 5 plugins?** For [Unreal Engine 5](https://developer.nvidia.com/game-engines/unreal-engine) developers and technical artists, a new suite of NVIDIA ACE plugins is now available. These local, runtime AI models support both Blueprint and C++ integration out of the box, covering three pillars: **Automatic speech recognition (ASR):** Includes a ready-to-use English language model, nemo-conformer-ctc-120m, with download options for seven additional languages, a sample level, and Blueprint examples for rapid setup.**Small language models (SLM):** Provides local GGUF support, for low-latency text generation and function calling capabilities, optimized for immediate character dialogue and decision-making, also including a ready-to-use Qwen 3.5 4B model.**Text-to-speech (TTS):** Includes high-quality Chatterbox Turbo 350M TTS model, example voices, a sample level, and dedicated plugin content to bring characters to life instantly. These plugins enable creators to implement live transcription, dynamic character dialogue, voice commands, and contextual action responses directly within their game logic. Unlike cloud-based services that suffer from high latency and unpredictable operational costs, these plugins offer local, RTX-optimized workflows bundled with ready-to-use models. They deliver an immediate, end-to-end AI NPC pipeline directly to your Unreal Engine 5 project. [Download the NVIDIA ACE Unreal Engine 5 plugins](https://developer.nvidia.com/ace-for-games) to get started and join the [NVIDIA Developer Discord community](https://discord.com/invite/nvidiadeveloper) for support and discussion. To learn more, [register for Building AI-Powered NPCs in Unreal Engine 5.7](https://gateway.on24.com/wcc/experience/elitenvidiabrill/1407606/3849291/level-up-with-nvidia?linkId=100000374605923&linkId=100000415844321&linkId=100000419915956), a live Level Up with NVIDIA webinar on June 30. The webinar will provide more details on these plugins and you’ll have the opportunity to ask us questions. **What is the new NVIDIA DLSS 4.5 Unreal Engine plugin?** The new [DLSS 4.5 Unreal Engine plugin](https://developer.nvidia.com/rtx/dlss) adds support for [DLSS 4.5 Dynamic Multi Frame Generation, Multi Frame Generation 6X Mode, and the second-resolution transformer for Super Resolution](https://www.nvidia.com/en-us/geforce/news/dlss-4-5-dynamic-multi-frame-generation-6x-mode-released/). The plugin provides direct access to the latest DLSS Super Resolution and Frame Generation updates. Built on Streamline, the plugin offers a consistent integration path across DLSS features, allowing developers to selectively adopt capabilities like Ray Reconstruction or Dynamic Multi Frame Generation. Updated APIs, documentation, and sample code help reduce integration time and make it easier to bring DLSS into both new and existing projects. **Learn more at Unreal Fest 2026 ** Join NVIDIA at [Unreal Fest 2026](https://www.unrealengine.com/unreal-fest) to learn how NVIDIA ACE and Unreal Engine are redefining the future of interactive character performance. The June 17 session, [Ready, Set, Action: Why Your Next NPC Should Be Their Own Method Actor](https://unrealfest.unrealengine.com/event/46281576-79df-4821-8d6c-01c67e89bbb8/agenda?session=11020ae6-e0ab-4152-916b-5b869f106a44&shareLink=true), walks through a new workflow for building autonomous game characters that separates the *role* of an NPC from its *performer. *Learn how to create deeply reactive, always-in-character AI personalities. Using synthetic training data generated by large teacher models, you can distill character behaviors into lightweight, deployable LoRA-powered performances that run locally on NVIDIA GeForce RTX hardware. The session features how game developers can easily craft interactive MetaHuman NPCs in Unreal Engine 5.7, powered entirely on-device using NVIDIA ACE: - ACE UE Speech Plugins: Riva Parakeet ASR for real-time speech recognition, Chatterbox TTS for expressive voice synthesis - Audio2Face for facial animation and performance delivery The latest workflow allows NPCs to perform dynamically in-character during gameplay, then break character to collaborate directly with developers by refining motivations, generating backstory material, and helping build future training data. #### Full-scale first-person *Chalk Warfare *game with AI-powered features The studio behind the [ Chalk Warfare series](https://www.youtube.com/playlist?list=PLyVkV6PxrgitYdj0X2p6M8ZMbIm9MsOoC) boasting 350 million views, SOKRISPYMEDIA, is building a full-scale first-person action game on the franchise, and its standout feature leans on local AI. Players hand-draw their weapon of choice and the game integrates it in seconds, with models trained to handle how each weapon is held, fired, recoils, and reloads. To train that bespoke matching model without paying for expensive cloud compute, the team ran on a Puget Systems workstation built around dual [NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition](https://www.nvidia.com/en-us/products/workstations/professional-desktop-gpus/rtx-pro-6000-max-q/) GPUs. Real-time rendering veterans making the jump from 2D content to a playable medium, on hardware tuned for AI training. To see the game in action at Unreal Fest, visit the Puget Systems booth. #### Promptable, controllable human motion now available In Unreal Engine NVIDIA has released [NVIDIA Kimodo](https://research.nvidia.com/labs/sil/projects/kimodo/), an open source project for promptable, controllable human motion. Describe a motion in plain language (“a person walks forward happily, then jumps in the air”), pin specific bones or full-body keyframes for precise art direction, and generate complex human motion in seconds. Building on this project, [Animotive Kimodo](https://kimodo.animotive.com/) is a plugin that brings Kimodo directly into Unreal Engine, that enables animators to refine, layer, and hand-key AI-generated motion on their own rigs without leaving the editor. AI motion is a starting point rather than locked output that replaces craft. The Unreal Engine plugin ships today, with Blender and Maya integrations to follow. Available self-hosted or through Animotive Cloud. Future updates will allow AAA studios to securely upload their own animation datasets, training Animotive Kimodo to follow their unique, in-house style. Learn more about the [Animotive Kimodo launch](https://www.youtube.com/watch?v=AFtBFvKe2xI). **Get started with NVIDIA ACE and DLSS 4.5 in Unreal Engine 5 ** At [Unreal Fest 2026](https://www.unrealengine.com/unreal-fest), NVIDIA is announcing more features to make on-device AI characters and gameplay achievable through NVIDIA ACE—including the ACE Game Agent SDK, new Unreal Engine 5 plugins, and DLSS 4.5. Mark your calendars to join NVIDIA AI research leaders Sanja Fidler, Ming-Yu Liu, and David Luebke at [SIGGRAPH 2026](https://www.nvidia.com/en-us/events/siggraph/) on July 20 to explore the latest breakthroughs shaping the future of computer graphics and simulation. **Resources for game developers** Check out the full list of [game developer resources](https://developer.nvidia.com/game-development) and stay up to date with the latest NVIDIA game development news: - Subscribe to our [newsletter](https://developer.nvidia.com/email-signup)(select gaming as your industry) - Follow us on social: [X](https://x.com/NVIDIAGameDev),[LinkedIn](https://www.linkedin.com/showcase/79124990/),[Facebook](https://www.facebook.com/NVIDIAGameDev/), and[YouTube](https://www.youtube.com/@NVIDIAGameDeveloper/featured) - Join our [Discord community](https://discord.com/invite/nvidiadeveloper)