Polars GPU Engine

NVIDIA's cuDF library now provides GPU-accelerated execution engines for the Polars Lazy API, enabling users to run dataframe operations on GPUs with automatic fallback to CPU when operations are unsupported. The engine, available in Open Beta through the cudf-polars package, delivers significant speedups for datasets exceeding hundreds of gigabytes and supports scaling from single GPU to multi-GPU and multi-node configurations.

Polars GPU engine polars-gpu-engine cuDF provides GPU-accelerated execution engines for Python users of the Polars Lazy API. The engines support most of the core expressions and data types as well as a growing set of more advanced dataframe manipulations and data file formats. When a GPU engine is selected, Polars converts expressions into an optimized query plan and determines whether the plan is supported on the GPU. If it is not, the execution transparently falls back to the standard Polars engine and runs on the CPU. Install install Follow the RAPIDS installation guide https://docs.rapids.ai/install and pick the cudf-polars package for your CUDA and Python versions. For example, with conda: conda install -c rapidsai -c conda-forge -c nvidia cudf-polars Or with pip CUDA 13 wheels; use cudf-polars-cu12 for CUDA 12 : pip install cudf-polars-cu13 Quick start quick-start RayEngine api/ cudf polars.engine.ray.RayEngine with no arguments uses every GPU visible to the process, so the same code runs on one GPU and scales to multi-GPU / multi-node setups automatically: python import polars as pl from cudf polars.engine.ray import RayEngine query = pl.scan parquet "/data/dataset/ .parquet" .filter pl.col "amount" 100 .group by "customer id" .agg pl.col "amount" .sum with RayEngine as engine: result = query.collect engine=engine See Usage usage/ for the full tutorial, Engines engines/ for a conceptual overview of the available engines, Configuration Options options/ for the StreamingOptions api/ cudf polars.engine.options.StreamingOptions configuration, and Understanding Memory Use in the GPU Streaming Engine memory errors/ for guidance on out-of-memory errors and memory tuning. Benchmark benchmark Polars delivers high performance across a wide range of data scales through multiple execution engines. The default CPU engine is highly optimized for interactive and medium-scale analytics on a single node. The Polars GPU engine lets you move seamlessly to GPU nodes, providing meaningful acceleration when your dataset grows to hundreds of gigabytes or more. We ran the Polars Decision Support PDS benchmarks to compare the Polars GPU engine with the CPU engine at larger scale factors to show how the GPU engine delivers meaningful speedups as dataset size grows: On a single GPU, you can run TB-scale workloads with significant speedups compared to running on CPU. You can also scale up to run on multiple GPUs for processing even larger workloads: For more information on the benchmarks being run, see the PDS-DS queries in the cuDF GitHub repository https://github.com/rapidsai/cudf/tree/release/26.06/python/cudf polars/cudf polars/streaming/benchmarks . Learn More learn-more The GPU engine for Polars is now available in Open Beta and the engine is undergoing rapid development. To learn more, visit the GPU Support page https://docs.pola.rs/user-guide/gpu-support/ on the Polars website.