{"slug": "nvidia-eye-contact-nim", "title": "Nvidia Eye Contact Nim", "summary": "NVIDIA released the Eye Contact NIM, a microservice that uses AI models to redirect a user's gaze toward the camera in video frames to simulate natural eye contact. The NIM runs on CUDA, TensorRT and Triton for GPU acceleration, extracts an eye patch via NVIDIA's face tracking pipeline using 2D facial landmarks and 6DOF head pose, and uses a disentangled encoder-decoder network to estimate gaze angle and redirect it before blending the patch back into the frame. The service supports streaming mode, which runs inference incrementally on video chunks, and transactional mode for whole files, and is available to try at build.nvidia.com/nvidia/eyecontact.", "body_md": "# Overview[#](#overview)\n\nNVIDIA Eye Contact NIM leverages state-of-the-art AI models to dynamically redirect a user’s eye position toward the camera to simulate natural eye contact and enhance remote digital engagement. NVIDIA Eye Contact NIM models are built on the NVIDIA software platform, incorporating CUDA, TensorRT, and Triton to offer out-of-the-box GPU acceleration.\n\n## Architecture[#](#architecture)\n\nNVIDIA Eye Contact operates on a region of interest around the eyes, also known as the eye patch. The eye patch is extracted from a video frame using the NVIDIA face tracking pipeline, which computes the 2D facial landmarks and the 6DOF head pose from the video frame. This head pose is then fed into the eye contact network.\n\nThe eye contact network has a disentangled encoder-decoder architecture. The encoder estimates the gaze angle from the input eye patch along with a set of features, also known as embeddings. Based on these embeddings, the decoder performs redirection of the gaze in the input patch to make the face look forward.\n\nThe final stage of the pipeline involves blending the eye patch back into the original video frame using an inverse transformation.\nFor more details about the model, read [Improve Human Connection in Video Conferences with NVIDIA Maxine Eye Contact](https://developer.nvidia.com/blog/improve-human-connection-in-video-conferences-with-nvidia-maxine-eye-contact/) in the NVIDIA Developer Technical Blog.\n\n## Input Modes[#](#input-modes)\n\nNVIDIA Eye Contact NIM supports two modes of operation:\n\n- Streaming mode (recommended) In this mode, the input files are streamed to the NIM in chunks. As the chunks arrive, NIM runs inference incrementally and streams the output back to the client in chunks, even before the whole input file is received by the NIM. The server automatically detects streamable videos and uses the streaming inference path internally, so no extra flag is required to enable this behavior on the server. For best performance, use streamable videos as input.\n- Transactional mode In this mode, whole input files are uploaded to the NIM before the inference begins. The inference is run on the input files to obtain the output file, which is then sent back to the client. The server automatically falls back to this mode when the input video is not streamable.\n\nFor more details, refer to the [Input Modes](basic-inference.html#input-modes) section of Basic Inference.\n\n## Try It Out[#](#try-it-out)\n\nTry the NVIDIA Eye Contact NIM at [build.nvidia.com/nvidia/eyecontact](https://build.nvidia.com/nvidia/eyecontact).\n\nTo experience the NVIDIA Eye Contact NIM API without having to host your own servers, use the [Try API](https://build.nvidia.com/nvidia/eyecontact/api) feature, which uses the NVIDIA Cloud Function backend.", "url": "https://wpnews.pro/news/nvidia-eye-contact-nim", "canonical_source": "https://docs.nvidia.com/nim/maxine/eye-contact/latest/overview.html", "published_at": "2026-11-04 18:46:43+00:00", "updated_at": "2026-11-04 18:50:24.043372+00:00", "lang": "en", "topics": ["artificial-intelligence", "computer-vision", "ai-products", "ai-tools", "ai-infrastructure"], "entities": ["NVIDIA", "NVIDIA Eye Contact NIM", "CUDA", "TensorRT", "Triton", "NVIDIA Maxine Eye Contact"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/nvidia-eye-contact-nim", "markdown": "https://wpnews.pro/news/nvidia-eye-contact-nim.md", "text": "https://wpnews.pro/news/nvidia-eye-contact-nim.txt", "jsonld": "https://wpnews.pro/news/nvidia-eye-contact-nim.jsonld"}}