Polars GPU Engine

cuDF provides GPU-accelerated execution engines for Python users of the Polars Lazy API. The engines support most of the core expressions and data types as well as a growing set of more advanced dataframe manipulations and data file formats. When a GPU engine is selected, Polars converts expressions into an optimized query plan and determines whether the plan is supported on the GPU. If it is not, the execution transparently falls back to the standard Polars engine and runs on the CPU.

Install# #

Follow the RAPIDS installation guide and pick the cudf-polars

package for your CUDA and Python versions. For example, with conda:

conda install -c rapidsai -c conda-forge -c nvidia cudf-polars

Or with pip (CUDA 13 wheels; use cudf-polars-cu12

for CUDA 12):

pip install cudf-polars-cu13

Quick start# #

RayEngine with no arguments uses every GPU visible to the process, so the same code runs on one GPU and scales to multi-GPU / multi-node setups automatically:

import polars as pl
from cudf_polars.engine.ray import RayEngine

query = (
    pl.scan_parquet("/data/dataset/*.parquet")
    .filter(pl.col("amount") > 100)
    .group_by("customer_id")
    .agg(pl.col("amount").sum())
)

with RayEngine() as engine:
    result = query.collect(engine=engine)

See Usage for the full tutorial, Engines for a conceptual overview of the available engines, Configuration Options for the StreamingOptions configuration, and

Understanding Memory Use in the GPU Streaming Enginefor guidance on out-of-memory errors and memory tuning.

Benchmark# #

Polars delivers high performance across a wide range of data scales through multiple execution engines. The default CPU engine is highly optimized for interactive and medium-scale analytics on a single node. The Polars GPU engine lets you move seamlessly to GPU nodes, providing meaningful acceleration when your dataset grows to hundreds of gigabytes or more.

We ran the Polars Decision Support (PDS) benchmarks to compare the Polars GPU engine with the CPU engine at larger scale factors to show how the GPU engine delivers meaningful speedups as dataset size grows:

On a single GPU, you can run TB-scale workloads with significant speedups compared to running on CPU. You can also scale up to run on multiple GPUs for processing even larger workloads:

For more information on the benchmarks being run, see the PDS-DS queries in the cuDF GitHub repository.

Learn More# #

The GPU engine for Polars is now available in Open Beta and the engine is undergoing rapid development. To learn more, visit the GPU Support page on the Polars website.