# How to Read Data into GPU Memory in Python with NVIDIA GPUDirect Storage > Source: > Published: 2026-10-22 16:39:09+00:00 GPU applications are processing ever-growing datasets with increasingly powerful accelerators. According to the [2025 Stanford AI Index](https://hai.stanford.edu/ai-index/2025-ai-index-report/research-and-development), compute for notable AI models has been doubling approximately every five months, while the size of datasets used to train LLMs has been doubling approximately every eight months. As computation and data scale together, moving data efficiently from storage becomes a necessity for keeping accelerators supplied. [NVIDIA Magnum IO GPUDirect Storage (GDS)](https://developer.nvidia.com/gpudirect-storage) addresses this challenge by enabling a direct data path between supported storage and GPU memory. This avoids unnecessary staging through CPU memory and reduces overall CPU utilization in the system. GDS is supported by a growing ecosystem that includes many [major storage vendors](https://docs.nvidia.com/gpudirect-storage/release-notes/index.html#support-matrix). At the application layer, Python plays a central role in modern AI development. As of August 2025, [nearly half of all new AI projects on GitHub were built primarily in Python](https://github.blog/news-insights/octoverse/what-the-fastest-growing-tools-reveal-about-how-software-is-being-built/) while the main application interface to GDS is the [cuFile](https://docs.nvidia.com/gpudirect-storage/api-reference-guide/index.html) C API. [NVIDIA KvikIO](https://docs.nvidia.com/kvikio/latest/), a Python and C++ library for high-performance file I/O, bridges this gap. It provides high-level, idiomatic Python and C++ interfaces, along with additional controls over how I/O requests are performed. The tutorial in this post demonstrates how to use the NVIDIA KvikIO Python API to configure storage I/O, read data into GPU memory, and confirm that the accelerated path within GDS was activated. It also shows how to inspect the KvikIO and cuFile settings that influence the path each operation takes. ## How does NVIDIA KvikIO leverage GDS? GDS is the broader technology that enables accelerated data movement between supported storage and GPU memory. For file system-based storage, cuFile provides the C library and API through which applications and frameworks access GDS. KvikIO sits above cuFile, providing higher-level Python and C++ interfaces along with its own controls over how I/O requests are executed. Figure 1 shows these layers and their two separate path-selection points. Initialization failures, unsupported configurations, and error-handling paths are omitted for clarity. The application supplies a file system path and GPU buffer, and KvikIO determines whether to send the request through cuFile or use its POSIX compatibility path, which stages the data through host memory. When cuFile is selected, KvikIO registers the file and issues the I/O using the target GPU buffer. cuFile then determines the actual transfer path based on system support, the file system, and its configuration. It may use the accelerated GDS path or its CPU-staged compatibility path. Therefore, reaching the cuFile API confirms that KvikIO selected cuFile, but does not by itself confirm that the accelerated GDS path was activated. The following sections walk through how to configure the system, perform a KvikIO read, and verify the resulting path. ## Prerequisites Before beginning this tutorial, you will need: - A Linux system with an NVIDIA GPU and GPUDirect Storage installed. - A GDS-supported storage and file system configuration. - Permission to create, read, and write files at the storage endpoint, along with access to GDS tools such as `gdscheck` and`gds_stats` . For complete installation and platform requirements, refer to the [NVIDIA GPUDirect Storage documentation](https://docs.nvidia.com/gpudirect-storage/). ### Tutorial system This tutorial was tested with the following configuration: - An `x86_64` system with an NVIDIA H100 GPU, CUDA 13.3, and cuFile 1.18. - A local NVMe device formatted with `ext4` . - A PIX connection between the selected GPU and NVMe device, meaning the path traverses at most one PCIe bridge. - IOMMU and ACS disabled to preserve PCIe peer-to-peer transfers. - The `nvidia-fs` kernel module is loaded to provide the accelerated path. You can inspect the GPU-to-storage topology with the following command: ``` bash $ nvidia-smi topo -nvme ``` ## Step 1: Set up NVIDIA KvikIO First, create a Python environment and install the CUDA 13 build of KvikIO. CuPy will be installed as a dependency. ``` bash $ python3 -m venv kvikio-env $ source kvikio-env/bin/activate $ python -m pip install kvikio-cu13 ``` You can verify the installation with the following: ``` bash $ python -m pip show kvikio-cu13 cupy-cuda13x ``` ## Step 2: Inspect the configuration Before performing any I/O, inspect the active KvikIO settings and confirm that the system recognizes the GDS components. In the KvikIO defaults API, the compatibility-mode setting is named `compat_mode`, and its name attribute reports the active mode as `AUTO`, `ON`, or `OFF`. The following commands report the loaded KvikIO and cuFile versions, the active compatibility mode, and the minimum request size KvikIO sends to cuFile: ``` python import kvikio import kvikio.cufile_driver as cufile_driver import kvikio.defaults as defaults kvikio.__version__ cufile_driver.libcufile_version() defaults.get("compat_mode").name defaults.get("gds_threshold") ``` On the tutorial system, KvikIO reports compatibility mode as `AUTO` and a GDS threshold of 16 KiB. `AUTO` instructs KvikIO to try cuFile and fall back to its POSIX backend if cuFile is unavailable. Reads smaller than the threshold use the KvikIO POSIX path directly. These settings can also be controlled through `KVIKIO_COMPAT_MODE` and `KVIKIO_GDS_THRESHOLD`. For the complete list, refer to the [KvikIO runtime settings](https://docs.rapids.ai/api/kvikio/stable/runtime_settings/). Next, inspect the GDS platform and cuFile configuration: ``` $ /usr/local/cuda-13.3/gds/tools/gdscheck.py -p 2>&1 | less -R ``` The output lists the installed GDS components, supported file systems, detected GPUs, and active cuFile configuration. Figure 2 shows how to inspect the KvikIO and GDS configuration used in this tutorial. KvikIO settings can also be changed in Python. For example, `defaults.set` accepts a dictionary to change several settings at once, or a name and value to change one setting: ``` defaults.set({"task_size": 1024**3, "num_threads": 8}) defaults.set("compat_mode", kvikio.CompatMode.OFF) ``` ## Step 3: Read a file into GPU memory Next, create the input file once before running the Python example. The following command writes a 1 GiB file with random data using direct I/O and flushes it to storage: ``` bash $ dd if=/dev/urandom \ of=/mnt/nvme1/kvikio-tutorial/kvikio-example.bin \ bs=1M count=1024 \ oflag=direct conv=fsync \ status=progress ``` The following program allocates a 1 GiB CuPy array in GPU memory and supplies it to KvikIO as the destination buffer: ``` python from pathlib import Path import cupy as cp import kvikio file_path = Path( "/mnt/nvme1/kvikio-tutorial/kvikio-example.bin" ) # Allocate a GPU buffer large enough to hold the file. gpu_buffer = cp.empty( file_path.stat().st_size, dtype=cp.uint8, ) # Read the file into the GPU buffer. with kvikio.CuFile(file_path, "r") as file: bytes_read = file.read(gpu_buffer) # Validate that 1 GiB was read. assert bytes_read == gpu_buffer.nbytes ``` The context manager opens the file and completes the read into GPU memory. Although the return value confirms that the complete file was read, it does not identify the underlying data path. ## Step 4: Determine the data path One way to validate the data path used by any cuFile-based application is to observe the cuFile runtime per-process statistics. These statistics can be enabled by setting `CUFILE_STATS_LEVEL=3` and then inspected with `gds_stats` while the process remains active. For this KvikIO example, the environment variable keeps the setting scoped to a single run, although it can also be configured in `cufile.json`. ``` bash $ CUFILE_STATS_LEVEL=3 python3 -i read_with_kvikio.py ``` Running Python interactively keeps the process alive after the read, allowing `gds_stats` to inspect its accumulated counters. Figure 3 demonstrates this workflow. In the per-GPU output, `nvfs` counts operations representing the path taken through the `nvidia-fs` kernel module, while `posix` counts operations handled by the cuFile compatibility path. The KvikIO default 4 MiB task size divides the 1 GiB request into 256 read operations. The per-GPU statistics report `n=256`, `nvfs=256`, and `posix=0`, showing that all 256 operations used the `nvidia-fs` path. ## Step 5: Compare the available paths The same read can be repeated with different settings to demonstrate the two control layers. First, require KvikIO to use cuFile: ``` bash $ CUFILE_STATS_LEVEL=3 \ KVIKIO_COMPAT_MODE=off \ python3 -i read_with_kvikio.py ``` On the tutorial system, `gds_stats` reports `nvfs` operations, confirming that cuFile selected the accelerated path. Next, force cuFile to use its compatibility path: ``` bash $ CUFILE_STATS_LEVEL=3 \ KVIKIO_COMPAT_MODE=off \ CUFILE_FORCE_COMPAT_MODE=true \ python3 -i read_with_kvikio.py ``` KvikIO still calls cuFile, but the operations now appear under `posix` rather than `nvfs`. Finally, the KvikIO compatibility mode bypasses cuFile entirely: ``` bash $ KVIKIO_COMPAT_MODE=on \ python3 read_with_kvikio.py ``` In this case, KvikIO performs the read through its CPU-staged POSIX backend, so cuFile does not record the operation. The next section examines the cuFile settings that control request execution. ## Step 6: cuFile configuration for a KvikIO read After KvikIO selects its cuFile path, cuFile applies another set of controls that determine how the request is executed. These controls are defined in `cufile.json` and operate independently of KvikIO runtime settings. ### Request decomposition and parallel execution KvikIO and cuFile can both divide a large operation into smaller pieces. KvikIO applies `KVIKIO_TASK_SIZE` first, with `KVIKIO_NTHREADS` controlling how those tasks are scheduled. Each task on the cuFile path becomes a separate cuFile request. Once cuFile receives one of those requests, its execution settings determine whether it can be divided again and processed using the cuFile internal thread pool. The following cuFile settings control this behavior: | **Setting** | **Purpose** | |---|---| | `execution.parallel_io` | Enables cuFile internal parallel processing. | | `execution.max_io_threads` | Limits the number of cuFile worker threads available per GPU queue. | | `execution.min_io_threshold_size_kb` | Sets a target minimum size for internally scheduled pieces; a final remainder may be smaller. | | `execution.max_request_parallelism` | Limits how much parallelism cuFile prepares for one request. | | `execution.max_io_queue_depth` | Limits the number of pending work items in the internal queue. | *Table 1. cuFile execution settings that shape how KvikIO requests are processed* ### A 1 GiB read, step by step The configuration used here has four cuFile worker threads, `max_request_parallelism` set to four, and `min_io_threshold_size_kb` set to 8192 (8 MiB). With the KvikIO default 4 MiB task size, a 1 GiB read becomes 256 cuFile requests. Each 4 MiB request is handled as one piece by cuFile. This is the run that produced `nvfs=256` in the earlier statistics. If `KVIKIO_TASK_SIZE` is set to 1 GiB, KvikIO instead sends one large read request to cuFile. The CuPy destination has not been explicitly registered with cuFile, so cuFile uses its unregistered buffer scheduling path. It limits the request to min (4 worker threads, 4 maximum request parallelism) = 4 pieces. Dividing 1 GiB four ways gives 256 MiB per piece, already larger than `min_io_threshold_size_kb` which is 8 MiB. With room in its queue, cuFile assigns three pieces to workers, processes the fourth on the calling thread, and waits for them to finish. Those 256 MiB pieces are units of scheduled work, not individual storage transfers. Because the destination buffer is not registered for storage DMA, cuFile reads into a registered GPU bounce buffer and copies the data into the destination. Since GDS 1.17, these bounce buffers can be configured as slabs. The `nvidia-fs` path in this configuration uses slabs no larger than 16 MiB, so each 256 MiB piece is processed through smaller transfers. The 8 MiB scheduling target does not set their transfer size. ### When the 8 MiB split target matters In this configuration, `execution.min_io_threshold_size_kb` is set to 8192 (8 MiB). That target does not change the 1 GiB split, but it would affect a 16 MiB cuFile request. Four equal pieces would be 4 MiB each, so cuFile schedules two 8 MiB pieces instead. KvikIO must use a task size of at least 16 MiB for cuFile to receive that request intact. ## KvikIO in NVIDIA CUDA-X for data science KvikIO is part of the [NVIDIA CUDA-X](https://www.nvidia.com/en-us/technologies/cuda-x/) libraries for data science (formerly RAPIDS). Its development is closely connected to the GDS integration originally built for [NVIDIA cuDF](https://github.com/NVIDIA/cudf) GPU DataFrame library, where parallel cuFile reads accelerated Parquet and ORC ingestion. This design was later incorporated into KvikIO, building on the [cuDF original GDS data-ingest work](https://developer.nvidia.com/blog/boosting-data-ingest-throughput-with-gpudirect-storage-and-rapids-cudf/). KvikIO has since been used with GDS in [cuCIM digital pathology workflows](https://developer.nvidia.com/blog/accelerating-digital-pathology-workflows-using-cucim-and-nvidia-gpudirect-storage/) and in the [cuPhoton xDataReader for scientific-image analysis](https://developer.nvidia.com/blog/faster-scientific-image-analysis-with-nvidia-cuphoton/), while related [GPU-accelerated Presto work](https://developer.nvidia.com/blog/running-low-latency-analytical-workloads-with-gpu-accelerated-presto-on-nvidia-gb200-nvl72/) shows benefits of GDS in data analytics workloads. Together, these examples show the broader role of KvikIO as an application-facing I/O layer over the storage acceleration GDS provides. ## Get started with NVIDIA KvikIO This tutorial has demonstrated how to read data into GPU memory and use level-three GDS cuFile statistics to distinguish the accelerated `nvidia-fs` path from CPU-staged compatibility paths. These path decisions matter significantly for popular libraries such as [cuDF that use KvikIO for many I/O operations](https://docs.rapids.ai/api/cudf/latest/user_guide/io/io/#kvikio-integration). The key takeaway is that KvikIO and cuFile make separate decisions. KvikIO selects whether to call cuFile, while cuFile determines the final transfer path and execution strategy. Understanding these layers enables applications to understand the path the I/O is taking and tune the settings that matter for their workload. Ready to get started? Run the KvikIO Python example featured in this post on your GDS-enabled system. For more examples, explore the C++ and Python KvikIO code in the [MagnumIO GDS samples directory](https://github.com/NVIDIA/MagnumIO/tree/main/gds/samples). Use `gds_stats` to check whether reads appear under `nvfs` or `posix`, then apply the same verification to your own KvikIO workloads.