Make Long-Running NVIDIA TensorRT Engine Builds Observable and Cancelable in Python or C++ NVIDIA TensorRT's IProgressMonitor API lets developers make long-running engine builds observable and cancelable in Python and C++, preventing wasted GPU-hours in agent workflows. The interface provides phase_start, step_complete, and phase_finish callbacks, with step_complete returning a boolean to abort the build. A tutorial demonstrates a drop-in implementation with Ctrl-C cancellation and progress streaming for IDEs, services, or agent runtimes. A TensorRT engine build can take seconds to many minutes. Large strongly typed models, deep tactic search, and a cold timing cache on a brand-new GPU SKU can leave developers, end users, or AI agents staring at a frozen terminal with no idea whether to wait, retry, or kill the process. Most NVIDIA TensorRT integrations report nothing during a build or provide no way to abort early. In a long-running agent workflow, this turns into wasted GPU-hours and stuck sessions. TensorRT provides IProgressMonitor , an API for fixing this issue, and it has been in NvInfer.h for several releases. This tutorial walks through a minimal drop-in implementation for Python and C++, adds a cancel path that responds to Ctrl-C or a programmatic stop signal from an outer event loop, and shows where to surface the resulting progress stream so an IDE, a service, or an agent runtime can use it. Every code block in this post is lifted from or modeled on two NVIDIA-maintained OSS samples: Python: samples/python/simple progress monitor/ ResNet-50, strongly typed network C++: samples/sampleProgressMonitor/ MNIST What IProgressMonitor gives you IProgressMonitor is an abstract base class that TensorRT calls during the engine build. You subclass it and override three methods. The shape is identical in Python and C++; only the spelling differs. | Concept | Python method | C++ method | What you do | |---|---|---|---| | Phase entered | phase start phase name, parent phase, num steps | phaseStart phaseName, parentPhase, nbSteps | Reserve a progress row and record num steps . | | Step within phase complete | step complete phase name, step - bool | stepComplete phaseName, step - bool | Advance the bar. Return False / false to cancel the build. | | Phase exited | phase finish phase name | phaseFinish phaseName | Tear down the row. | Table 1. The IProgressMonitor interface mirrored across Python and C++. The three methods have identical semantics, and step complete is the only callback whose return value changes the builder’s behaviorA phase whose parent phase is non-null is nested inside another phase, so the monitor sees a tree of progress rather than a flat list. The implementation must be thread-safe because TensorRT can call the same monitor instance from multiple internal threads. Wire the monitor to the builder by setting it on the IBuilderConfig . It is a single call in either language: config.progress monitor = MyMonitor Python config->setProgressMonitor &myMonitor ; // C++ Read the diagram from top to bottom. The builder opens the Building Engine phase with phase start , then opens Tactic Selection nested inside it with its parent phase pointing back at Building Engine . As the build proceeds, the builder calls step complete the solid arrows and your monitor returns a Boolean the dashed arrows : true lets the build continue and false requests cancellation. In the run shown here, the monitor returns false at step 47, which is the red cancel path, and the builder stops issuing new steps and unwinds. It calls phase finish early on Tactic Selection and then on Building Engine , closing every active phase in reverse order. What this tutorial builds This tutorial shows how to implement IProgressMonitor in Python and C++, add cancellation through step complete , and route progress updates to a terminal, IDE, service, or agent runtime. Prerequisites - One NVIDIA GPU. - TensorRT current OSS release and its Python bindings, or a build of the C++ samples. - Python 3.10 or newer Python path . - The TensorRT sample data: ResNet-50 ONNX for Python and MNIST ONNX for C++. Both ship with the sample-data archive or are mounted under /usr/src/tensorrt/data in the official NGC containers. - A terminal that supports ANSI virtual-terminal escapes. Any modern Linux shell qualifies; Windows Terminal works if VT is enabled. 1. Subclass IProgressMonitor in Python The subclass is small. It only tracks which phases are active and how many steps each phase contains. python import tensorrt as trt from dataclasses import dataclass, field from threading import Lock @dataclass class PhaseState: num steps: int current step: int = 0 parent: str | None = None class RichProgressMonitor trt.IProgressMonitor : def init self : super . init self. lock = Lock self. phases: dict str, PhaseState = {} self. cancelled = False self. rendered lines = 0 def phase start self, phase name, parent phase, num steps : with self. lock: self. phases phase name = PhaseState num steps=num steps, parent=parent phase self. render def step complete self, phase name, step - bool: with self. lock: if phase name in self. phases: self. phases phase name .current step = step self. render return not self. cancelled def phase finish self, phase name : with self. lock: self. phases.pop phase name, None self. render Two things to notice. First, the Lock is not optional. TensorRT will call into the monitor from multiple internal threads, and rendering from a thread that doesn’t own the state will tear the display. Second, step complete is the only callback that can stop the build. phase start returns None , so you cannot reject a phase before it begins. The earliest cancellation point is the first step complete of that phase. 2. Render nested progress bars with virtual-terminal escapes The renderer is the part that varies most by environment, so this section gives the shape and points to the upstream sample for the production-grade implementation. The pattern is: python def render self : Order phases by nesting depth so children draw under parents. rows = sorted self. phases.items , key=lambda kv: kv 1 .parent or "", kv 0 , Move the cursor up by the number of lines the PREVIOUS render printed, not the current row count — phases are added on nesting and removed on phase finish, so the two differ exactly when the tree changes shape. if self. rendered lines: print f"\x1b {self. rendered lines}A", end="" for name, st in rows: step is a 0-based index in 0, num steps ; +1 turns it into a completed count so the bar can actually reach 100%. done = min st.current step + 1, st.num steps pct = done / max st.num steps, 1 bar = "█" int 40 pct + "·" 40 - int 40 pct indent = " " if st.parent else "" print f"\x1b 2K{indent}{name:<28} {bar} {done}/{st.num steps}" Clear rows left behind when a phase finishes and the count shrinks. for in range self. rendered lines - len rows : print "\x1b 2K" self. rendered lines = len rows The upstream simple progress monitor.py renders the same shape with improved color and width handling. The escape sequence \x1b NA moves the cursor up N lines, and \x1b 2K clears a line. The first render call writes blank rows; subsequent calls overwrite them in place. When this monitor is attached, do not redirect stdout to a file or pipe. The escape codes will be written verbatim into the log and make it unreadable. For non-terminal sinks, replace render with a structured emitter. 3. Add a cancel path Cancellation is a three-line addition once the monitor exists. Install a SIGINT handler that flips the flag, then let step complete honor it. python import signal def install cancel monitor: RichProgressMonitor : def handler signum, frame : monitor. cancelled = True print "\nCancelling TensorRT build at next step boundary..." signal.signal signal.SIGINT, handler Wire the monitor and run the builder: builder = trt.Builder TRT LOGGER network = builder.create network 1 << int trt.NetworkDefinitionCreationFlag.STRONGLY TYPED parser = trt.OnnxParser network, TRT LOGGER with open onnx path, "rb" as f: parser.parse f.read config = builder.create builder config monitor = RichProgressMonitor config.progress monitor = monitor install cancel monitor serialized = builder.build serialized network network, config if serialized is None: if monitor. cancelled: print "Build cancelled cleanly." else: print "Build failed." build serialized network returns None on cancellation. The builder unwinds at the next step boundary, usually quickly, but not instantaneously, especially inside a long tactic-search step. Applications should surface cancellation latency to users. A simple “Cancelling…” message during the unwind window goes a long way. The same flag can be set from any non-signal path, such as an IDE Stop button, an agent timeout, or a CI cancel webhook. Set monitor. cancelled = True , and the build aborts at the next step boundary. 4 . The same pattern in C++ include