by Jim Ledin
Published by Packt Publishing.
I am pleased to present the updated and expanded third edition of Modern Computer Architecture and Organization. Reader comments and questions from the first and second editions have provided valuable insights that helped shape this new edition. Thank you to everyone who took the time to offer feedback!
This edition updates technologies that have advanced since the publication of the second edition and adds substantial new content on critical areas of computer architecture. New chapters provide detailed analyses of Graphics Processing Units (GPUs) and Large Language Models (LLMs).
Over the last few years, the explosive growth of artificial intelligence (AI) has led to drastic shifts in the direction of large-scale computing architecture. The expansion of AI-focused datacenters across the United States and around the world has led to the allocation of enormous quantities of capital, electricity, and water resources to the construction and operation of these facilities.
The two new chapters in this edition provide detailed explanations of how these AI systems operate, beginning at the lowest levels of GPU parallel processing. We examine the detailed processing requirements of an example LLM (we analyze GPT-2, which has a structure that substantially represents current frontier LLMs) and datacenter-scale requirements for physical space, electrical power, and cooling systems.
Chapter 10, Graphics Processing Units, examines modern GPUs both as engines for generating synthetic video scenes and as accelerators for AI training and inference. This new chapter explores the hardware architecture of a modern GPU, from its high-level subsystems that perform data distribution and processor scheduling to its deeply parallel low-level instruction-processing.
Chapter 15, Large Language Model Computing Architectures, explores computing architectures for advanced artificial intelligence systems, specifically large language models. We begin by analyzing early LLM design, using GPT-2 as a case study to illustrate model structure and processing requirements. The chapter examines the computational demands for LLM training and inference, concluding with an analysis of the datacenter infrastructure underpinning today’s largest and most sophisticated AI models.
Continuing the tradition of the first two editions, each new chapter contains end-of-chapter exercises designed to deepen your understanding of the material and reinforce key concepts.
I hope you find this updated edition as rewarding to read as it was to develop.
Jim Ledin
This repository contains source code and related files referenced in the book. The content is organized by chapter and includes code examples and solutions to the end-of-chapter exercises.