{"slug": "book-review-gpu-accelerated-computing-with-python-3-and-cuda", "title": "Book Review: GPU-Accelerated Computing with Python 3 and CUDA", "summary": "Niels Cautaerts and Hossein Ghorbanfekr's book 'GPU-Accelerated Computing with Python 3 and CUDA' teaches Python developers to write GPU-accelerated code using Numba-CUDA, CuPy, RAPIDS, and JAX. The book emphasizes profiling and optimization, covering fundamentals, performance tuning, high-level libraries, and end-to-end applications. It is aimed at scientists and engineers seeking to leverage GPU computing without dropping into C or C++.", "body_md": "## Book Review: GPU-Accelerated Computing with Python 3 and CUDA\n\nI was offered the opportunity to read **GPU-Accelerated Computing with Python 3 and CUDA** by Niels Cautaerts and Hossein Ghorbanfekr in exchange for an honest review. As a Google Developer Expert in AI/PyTorch, I work with PyTorch a lot, so a deep book on GPU computing in Python was in my area of interest. The book teaches Python developers and scientists to write GPU-accelerated code without dropping into C or C++, and it does this by moving up an abstraction ladder: you start by writing raw CUDA kernels with **Numba-CUDA**, then switch to high-level libraries like **CuPy**, **RAPIDS**, and **JAX**, and finally build four full applications end-to-end. I often use GPUs, but I use PyTorch/JAX without thinking about the kernels, the memory hierarchy, and the profiler. This book is about the layer underneath that, and I liked how consistently it pushes you to measure what is actually happening on the device rather than guess.\n\n### The overall structure\n\nThe book is organized into five parts:\n\n- Part 1 (Chapters 1–4) covers the fundamentals: why GPUs are fast, how to set up an environment locally or in the cloud, how to write your first Numba-CUDA kernels, and how to profile and debug them with the Nsight tools.\n- Part 2 (Chapters 5-7) is about performance: the GPU execution, occupancy, coalesced memory access, shared memory, overlapping work with CUDA streams, and scaling across multiple GPUs with Dask and JAX.\n- Part 3 (Chapters 8–10) moves on to high-level libraries: CuPy as an alternative to NumPy and SciPy; RAPIDS cuDF and cuML for pandas and scikit-learn; and JAX for optimization problems.\n- Part 4 (Chapters 11–14) brings it together in four applications: solving the heat equation, image processing and computer vision, a molecular dynamics simulation, and a transformer language model built from scratch.\n- Part 5 (Chapter 15) is an overview of where to go next, from Tensor Cores and other CUDA Python libraries to other platforms like ROCm and OpenCL.\n\nEvery chapter follows the same structure: derive the math or the concept, write a CPU version, port it to the GPU, then profile and optimize, and see a set of questions and worked answers. The environment is set up using the **Pixi** package manager, and you need an NVIDIA GPU (Chapter 2 shows how to rent if you do not have one).\n\n### What I liked\n\nThere were many things I liked in this book, and I want to highlight several in particular:\n\n- Nearly every optimization is measured, and there are cases when something does work: manual loop unrolling that the compiler had already done or a shared-memory version of the heat-equation kernel that barely contributes.\n- The sections about high-level libraries include “when to use” guidance. You can accelerate an existing pandas codebase with\n`cudf.pandas`\n\nand a single`%load_ext`\n\n, but the book shows when the host-to-device transfer outweighs the speedup and the GPU is not worth using. - Every chapter ends with a set of questions and worked answers, and the running examples are real scientific problems.\n\nThe CUDA streams chapter was interesting. Overlapping data transfer with computation reduces a 600 ms image pipeline to 285 ms, and we can see the actual gain in this illustration.\n\nThe computer vision case compares three ways of classifying noisy objects in an image: seven Hu moment shape descriptors, template matching in log-polar space, and a small CNN. The Hu moments reach 66.7% accuracy in milliseconds. Template matching gets 80% but takes over a minute per run. The CNN reaches 88% on clean held-out Fashion-MNIST, then drops to 60% on the noisy extracted objects, which the authors call “somewhat disappointing”.\n\nThe JAX chapter implements a physics-informed neural network, a great demonstration of why automatic differentiation matters. A plain MLP fits the training window for a damped RLC-circuit oscillation but fails to extrapolate past it; incorporating the circuit’s differential equation into the loss lets the network predict the new region.\n\nThe final project is about building a small transformer language model with JAX, Flax, and Optax: scaled dot-product attention, multi-head attention, the causal mask, the decoder stack, and a top-k sampling head. In the past, I have built transformers from scratch and by loading them from Hugging Face, and I’m convinced it is very useful to know the implementation details.\n\n### What could have been better\n\nThere are a couple of small things that could have been handled differently:\n\n- The transformer chapter uses IMDb (a sentiment-classification dataset), which is a bit of a weird choice for demonstrating LLM capabilities. The model output is barely coherent, but that is to be expected given such a short training.\n- The book is closely tied to specific library versions, and some implementation details may become outdated. But it isn’t a problem for the core concepts, and the authors have a GitHub repository with the code and a “known issues” page.\n\nBut these are small nitpicks that are completely overshadowed by the good sides of the book.\n\n### Conclusion\n\nThis book is a good fit for Python developers, scientists, and data scientists who want to understand GPU performance instead of treating the GPU as a black box behind a framework. It is especially useful if you do scientific or numerical computing and occasionally need more control than PyTorch or a high-level library gives you.\n\nGPU tooling moves fast, and the specifics here will change. What lasts is the way of working the book teaches: profile before you optimize, know where your data lives in the memory hierarchy, and understand when the GPU actually earns its place.\n\n[blogpost](/tag/blogpost)\n\n[books](/tag/books)\n\n[gpu](/tag/gpu)\n\n[cuda](/tag/cuda)\n\n[python](/tag/python)\n\n[jax](/tag/jax)", "url": "https://wpnews.pro/news/book-review-gpu-accelerated-computing-with-python-3-and-cuda", "canonical_source": "https://andlukyane.com/blog/book-review-gpu-cuda-python", "published_at": "2026-07-06 00:00:00+00:00", "updated_at": "2026-07-07 01:09:09.688248+00:00", "lang": "en", "topics": ["machine-learning"], "entities": ["Niels Cautaerts", "Hossein Ghorbanfekr", "Numba-CUDA", "CuPy", "RAPIDS", "JAX", "NVIDIA", "PyTorch"], "alternates": {"html": "https://wpnews.pro/news/book-review-gpu-accelerated-computing-with-python-3-and-cuda", "markdown": "https://wpnews.pro/news/book-review-gpu-accelerated-computing-with-python-3-and-cuda.md", "text": "https://wpnews.pro/news/book-review-gpu-accelerated-computing-with-python-3-and-cuda.txt", "jsonld": "https://wpnews.pro/news/book-review-gpu-accelerated-computing-with-python-3-and-cuda.jsonld"}}