Book Review: GPU-Accelerated Computing with Python 3 and CUDA #
I was offered the opportunity to read GPU-Accelerated Computing with Python 3 and CUDA by Niels Cautaerts and Hossein Ghorbanfekr in exchange for an honest review. As a Google Developer Expert in AI/PyTorch, I work with PyTorch a lot, so a deep book on GPU computing in Python was in my area of interest. The book teaches Python developers and scientists to write GPU-accelerated code without dropping into C or C++, and it does this by moving up an abstraction ladder: you start by writing raw CUDA kernels with Numba-CUDA, then switch to high-level libraries like CuPy, RAPIDS, and JAX, and finally build four full applications end-to-end. I often use GPUs, but I use PyTorch/JAX without thinking about the kernels, the memory hierarchy, and the profiler. This book is about the layer underneath that, and I liked how consistently it pushes you to measure what is actually happening on the device rather than guess.
The overall structure
The book is organized into five parts:
- Part 1 (Chapters 1–4) covers the fundamentals: why GPUs are fast, how to set up an environment locally or in the cloud, how to write your first Numba-CUDA kernels, and how to profile and debug them with the Nsight tools.
- Part 2 (Chapters 5-7) is about performance: the GPU execution, occupancy, coalesced memory access, shared memory, overlapping work with CUDA streams, and scaling across multiple GPUs with Dask and JAX.
- Part 3 (Chapters 8–10) moves on to high-level libraries: CuPy as an alternative to NumPy and SciPy; RAPIDS cuDF and cuML for pandas and scikit-learn; and JAX for optimization problems.
- Part 4 (Chapters 11–14) brings it together in four applications: solving the heat equation, image processing and computer vision, a molecular dynamics simulation, and a transformer language model built from scratch.
- Part 5 (Chapter 15) is an overview of where to go next, from Tensor Cores and other CUDA Python libraries to other platforms like ROCm and OpenCL.
Every chapter follows the same structure: derive the math or the concept, write a CPU version, port it to the GPU, then profile and optimize, and see a set of questions and worked answers. The environment is set up using the Pixi package manager, and you need an NVIDIA GPU (Chapter 2 shows how to rent if you do not have one).
What I liked
There were many things I liked in this book, and I want to highlight several in particular:
- Nearly every optimization is measured, and there are cases when something does work: manual loop unrolling that the compiler had already done or a shared-memory version of the heat-equation kernel that barely contributes.
- The sections about high-level libraries include “when to use” guidance. You can accelerate an existing pandas codebase with
cudf.pandas
and a single%load_ext
, but the book shows when the host-to-device transfer outweighs the speedup and the GPU is not worth using. - Every chapter ends with a set of questions and worked answers, and the running examples are real scientific problems.
The CUDA streams chapter was interesting. Overlapping data transfer with computation reduces a 600 ms image pipeline to 285 ms, and we can see the actual gain in this illustration.
The computer vision case compares three ways of classifying noisy objects in an image: seven Hu moment shape descriptors, template matching in log-polar space, and a small CNN. The Hu moments reach 66.7% accuracy in milliseconds. Template matching gets 80% but takes over a minute per run. The CNN reaches 88% on clean held-out Fashion-MNIST, then drops to 60% on the noisy extracted objects, which the authors call “somewhat disappointing”.
The JAX chapter implements a physics-informed neural network, a great demonstration of why automatic differentiation matters. A plain MLP fits the training window for a damped RLC-circuit oscillation but fails to extrapolate past it; incorporating the circuit’s differential equation into the loss lets the network predict the new region.
The final project is about building a small transformer language model with JAX, Flax, and Optax: scaled dot-product attention, multi-head attention, the causal mask, the decoder stack, and a top-k sampling head. In the past, I have built transformers from scratch and by them from Hugging Face, and I’m convinced it is very useful to know the implementation details.
What could have been better
There are a couple of small things that could have been handled differently:
- The transformer chapter uses IMDb (a sentiment-classification dataset), which is a bit of a weird choice for demonstrating LLM capabilities. The model output is barely coherent, but that is to be expected given such a short training.
- The book is closely tied to specific library versions, and some implementation details may become outdated. But it isn’t a problem for the core concepts, and the authors have a GitHub repository with the code and a “known issues” page.
But these are small nitpicks that are completely overshadowed by the good sides of the book.
Conclusion
This book is a good fit for Python developers, scientists, and data scientists who want to understand GPU performance instead of treating the GPU as a black box behind a framework. It is especially useful if you do scientific or numerical computing and occasionally need more control than PyTorch or a high-level library gives you.
GPU tooling moves fast, and the specifics here will change. What lasts is the way of working the book teaches: profile before you optimize, know where your data lives in the memory hierarchy, and understand when the GPU actually earns its place.