{"slug": "enabling-real-time-ai-for-scientific-instruments-with-gpu-native-data", "title": "Enabling Real-Time AI for Scientific Instruments with GPU-Native Data Acquisition", "summary": "NVIDIA released DAQIRI, a Holoscan platform library that streams high-bandwidth instrument and sensor data directly into GPU memory so measurements can be processed during acquisition instead of being written to disk first. The library adds hardware packet reordering, throughput improvements, and a low-latency NIC polling mode for time-sensitive packet processing. In a case study, a high-dynamic-range electron microscopy camera built at Lawrence Berkeley National Laboratory's Molecular Foundry generates more than 500 Gbps of data — over 500x faster than conventional EELS streams — which DAQIRI and GPU processing handle to recover energy-loss spectra with submicron spatial resolution from 5-micrometer pixels.", "body_md": "In science and industry, analyzing sensor measurements in real time can enable AI to adjust processes, guide experiments, and search for better operating conditions. Yet limits on how quickly the measured data can be moved and processed often shape how an instrument is designed, from readout electronics to acquisition timing. What could scientific instrument makers simplify—or make possible—if processing could keep pace with their sensors?\n\n[NVIDIA DAQIRI](https://developer.nvidia.com/daqiri) is a software-defined data acquisition library that moves high-bandwidth instrument and sensor data into GPU-ready workflows. It enables developers to process measurements during acquisition without first writing them to disk. An [NVIDIA Holoscan](https://holoscan.org) platform library, DAQIRI bridges sensors optimized for measurement with programmable GPU platforms for scientific analysis, AI inference, and feedback applications.\n\nFor existing Ethernet-capable instruments, this opens opportunities to add processing and feedback capabilities, as described in the previous post, [Enable Real-Time AI for High-Speed Data Acquisition with NVIDIA DAQIRI](https://developer.nvidia.com/blog/enable-real-time-ai-for-high-speed-data-acquisition-with-daqiri/). For new instruments, it allows opportunities for reconsidering acquisition constraints from the start.\n\nThis post explores how a new electron microscopy camera and GPU processing enable analysis during acquisition. It then introduces new DAQIRI features, including hardware packet reordering, performance and throughput improvements in the library, and a low-latency NIC polling mode for time-sensitive packet processing.\n\n## Case study: Connecting high-dynamic-range imaging to GPU processing\n\nA high-dynamic-range electron microscopy camera developed at [Lawrence Berkeley National Laboratory’s](https://www.lbl.gov/) [Molecular Foundry](https://foundry.lbl.gov/) illustrates what becomes possible when advanced detectors and GPU processing are designed to work together.\n\nElectron energy-loss spectroscopy (EELS) measures the energy electrons lose as they interact with a sample, helping scientists identify which elements are present and how they are bonded together. These measurements contain both strong and faint signals. To capture those differences in intensity, the conventional approach uses two separately optimized acquisition processes in rapid succession.\n\nThis requires precise coordination of the electron optics, exposure timing, and detector readout. The resulting measurements then need to be aligned and combined, adding complexity to both acquisition and processing. The new high-dynamic-range camera developed at the Molecular Foundry captures strong and faint signals together.\n\nThis advancement simplifies the experimental setup and microscope optics, but generates more than 500 gigabits of data per second (Gbps)—a stream that must be received and processed to make those measurements useful during acquisition. This is over 500x faster than conventional EELS data streams, which greatly enhances the raw detector information available for further processing and analysis.\n\nHandling that stream requires an acquisition and processing pipeline designed to keep pace with the camera. This is where the collaboration between Berkeley Lab and NVIDIA brings together the camera, DAQIRI, and GPU-accelerated processing. FPGA readout electronics transmit detector data over Ethernet, DAQIRI receives the packets directly into GPU memory, and GPU processing assembles and corrects detector frames before reducing them to energy-loss spectra.\n\nComputation also improves the spatial information recovered from the detector, enabling submicron spatial resolution from sensor pixels measuring 5 micrometers across. Through this pipeline, the team enables inspection of the data during the measurement and adjusting the acquisition parameters and regions of interest in real time, all while processing high bandwidth data incoming at more than 500 Gbps.\n\nThe benefits of this approach extend beyond electron microscopy. Connecting sensors to GPU processing can extract more information from measurements and give instrument makers greater flexibility in how they acquire, analyze, and act on data. Realizing those benefits requires efficient data movement, ordered payloads ready for processing, and low-latency communication for feedback.\n\n## Designing acquisition paths for intelligent instruments\n\nThe [latest DAQIRI release](https://github.com/NVIDIA/daqiri/releases/tag/v2026.11.00) adds hardware packet reordering, an optimized library backend for improved throughput, and direct polling for low-latency packet transmission and reception.\n\n### Reorder data as it arrives with NIC hardware\n\nMost networks rely on protocol processing or receive-side buffering to handle out-of-order packets, which can add latency and memory overhead. DAQIRI can now offload packet reordering tasks to supported [NVIDIA ConnectX-7](https://www.nvidia.com/en-us/networking/ethernet-adapters/) or newer NICs, preserving CPU and GPU resources for analysis. For compatible sequence formats, the NIC parses sequence fields and places each payload directly into its ordered position in memory. The application receives the assembled, ordered data without requiring a separate software reorder copy step.\n\nHardware-based reordering is defined here for a specific receive queue. The configuration dictates how to parse the sequence number from the packet header, which reordering method to use, and how missing packets should be handled.\n\n```\n- name: \"rx_reorder_seq_1024\"\n  reorder_engine: \"hw\"          # NIC places payloads in order\n  cyclic_sequence: true         # Confirm compatible cyclic numbering\n  missing_action: “drop”        # Drop partial batches on RX timeout\n  reorder_type: \"gpu\"           # Deliver packets to GPU memory\n  memory_region: \"Reorder_RX_GPU\"\n  payload_byte_offset: 42       # Skip 42-byte Ethernet/IPv4/UDP header\n  packet_size: 8192             # Payload bytes per output slot\n  flow_ids: [201]               # Apply reorder to RX flow 201\n  method:\n    seq_packets_per_batch:      # Reorder by one sequence field\n      sequence_number:\n        bit_offset: 336         # Sequence starts at bit 336\n        bit_width: 32           # 32-bit sequence field\n      packets_per_batch: 1024   # 1024 packets per output buffer\n```\n\n### Use the ibverbs engine for efficient acquisition\n\nDAQIRI introduces an ibverbs engine for high-rate receive and transmit of raw Ethernet packets. Built on libibverbs and NVIDIA DevX, the engine is selected by default for raw Ethernet streams. Developers can also select it explicitly by adding engine: “ibverbs” to their DAQIRI configuration, without changing how they use the application API. By reducing per-packet CPU overhead, the ibverbs engine delivers approximately twice the per-core throughput of the previous default DPDK backend at small packet sizes (up to 1024 bytes) and extreme data rates (greater than 200 Gbps), based on benchmark tests performed on the [NVIDIA DGX Spark](https://nvidia.github.io/daqiri/benchmarks/performance-dgx-spark/), [NVIDIA IGX Thor](https://nvidia.github.io/daqiri/benchmarks/performance-igx-thor/), and [NVIDIA RTX PRO](https://www.nvidia.com/en-us/products/workstations/) server platforms.\n\n### Reduce feedback latency with direct polling\n\nIntelligent instruments may require latency-sensitive feedback from GPU analysis to adjust acquisition or control settings. The DAQIRI direct polling mode reduces communication latency by letting the application thread receive and submit packets directly, eliminating the handoff to a separate I/O worker. In cabled loopback benchmark tests, application send-to-receive latency on the order of 2-4 µs has been demonstrated in direct polling mode. When using the ibverbs engine, developers can select direct polling for control queues while retaining batched processing for high-throughput data streams.\n\nSet `poll_mode: \"direct\"` on the latency-sensitive TX and RX control queues so the application thread submits and receives packets directly.\n\n```\n# TX/RX queue settings in example configuration file\ntx:\n  queues:\n  - name: \"control_tx\"\n    id: 0\n    poll_mode: \"direct\"         # App thread submits packets\n    memory_regions: [\"Control_TX\"]\n\nrx:\n  queues:\n  - name: \"control_rx\"\n    id: 0\n    poll_mode: \"direct\"         # App thread polls received packets\n    memory_regions: [\"Control_RX\"]\n```\n\nRun the direct-polling latency benchmark using a cabled loopback between TX and RX ports on the same host.\n\nFirst, replace the configuration’s hardware address placeholders and other system-specific settings with values for your system:\n\n```\nsudo ./build/examples/daqiri_bench_raw_latency \nexamples/daqiri_bench_raw_latency_ibverbs.yaml --samples \n10000 --warmup 1000 --csv latency.csv\n```\n\n## Get started with GPU-native data acquisition\n\nDuring the process of making scientific instruments, GPU-native data acquisition enables you to design around what will be measured and your desired means of response. Advanced detectors and GPU processing can work together for reconsidering acquisition constraints.\n\nThe latest DAQIRI capabilities extend the options for moving and organizing measurements efficiently and connecting analysis to experimental control. That means more engineering effort can focus on measurement quality and scientific insight.\n\nTo explore these capabilities, start with the [DAQIRI documentation](https://nvidia.github.io/daqiri/getting-started/) and [DAQIRI configuration examples](https://nvidia.github.io/daqiri/config-generation/). Then use the [DAQIRI benchmarks](https://nvidia.github.io/daqiri/benchmarks/performance-igx-thor/) to evaluate throughput and communication latency on your hardware.\n\nTo learn more, check out these related resources:\n\n- Explore the [NVIDIA/daqiri](https://github.com/NVIDIA/daqiri) GitHub repo\n- Browse the [DAQIRI plus TensorRT Streaming Inference Reference Application](https://nvidia.github.io/daqiri/tutorials/daqiri-resnet-inference/)\n- Generate [DAQIRI Configurations](https://nvidia.github.io/daqiri/config-generation/) for your hardware", "url": "https://wpnews.pro/news/enabling-real-time-ai-for-scientific-instruments-with-gpu-native-data", "canonical_source": "https://developer.nvidia.com/blog/enabling-real-time-ai-for-scientific-instruments-with-gpu-native-data-acquisition/", "published_at": "2026-11-19 17:00:00+00:00", "updated_at": "2026-11-19 17:14:10.558803+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-tools", "machine-learning", "developer-tools"], "entities": ["NVIDIA", "DAQIRI", "NVIDIA Holoscan", "Lawrence Berkeley National Laboratory", "Molecular Foundry"], "also_reported_by": [], "takeaway": "DAQIRI receives Ethernet detector packets directly into GPU memory, letting a 500+ Gbps electron microscopy camera stream be assembled, corrected and reduced to spectra during acquisition rather than written to disk first.", "recommendation": null, "alternates": {"html": "https://wpnews.pro/news/enabling-real-time-ai-for-scientific-instruments-with-gpu-native-data", "markdown": "https://wpnews.pro/news/enabling-real-time-ai-for-scientific-instruments-with-gpu-native-data.md", "text": "https://wpnews.pro/news/enabling-real-time-ai-for-scientific-instruments-with-gpu-native-data.txt", "jsonld": "https://wpnews.pro/news/enabling-real-time-ai-for-scientific-instruments-with-gpu-native-data.jsonld"}}