{"slug": "google-and-meta-open-source-torchtpu-to-bring-native-pytorch-support-to-google", "title": "Google and Meta Open-source TorchTPU to Bring Native PyTorch Support to Google TPUs", "summary": "Google, working with Meta, released TorchTPU, an open-source native PyTorch backend for Google TPUs that lets existing PyTorch workloads run on TPU hardware with minimal code changes, such as swapping the `cuda` device for `tpu` and the `nccl` distributed backend for `tpu_dist`. Google says its DeferAndFuse hybrid eager/compiled mode \"can be 50% or more faster than Strict Eager while reducing peak HBM memory,\" and it validated the backend on about 5,000 Hugging Face models across 500+ architectures on TPU generations from v5e, measuring 25.7k token/s inference at 5120 sequence length on a single TPUv7 chip for a 30-billion parameter Muse Glimmer model. The release replaces the 2020 PyTorch/XLA approach, which Google says required intrusive source-code changes, suffered severe compile-time latency and lacked native eager execution, and TorchTPU remains at an early stage with a roadmap still in progress.", "body_md": "[TorchTPU is an open-source project developed by Google in collaboration with Meta that provides a native PyTorch backend for Google TPUs](https://pytorch.org/blog/introducing-torchtpu/). It enables PyTorch workloads to execute on [TPU hardware](https://cloud.google.com/tpu) with minimal code modifications, supporting eager and compiled execution, distributed computing, and hardware-specific optimizations.\n\nGoogle began enabling PyTorch workloads to run on TPUs in 2020 through PyTorch/XLA, which leveraged JAX, [XLA](https://github.com/openxla/xla), and lazy tensors. According to Google, this approach had significant shortcomings:\n\nCrucially, PyTorch/XLA required intrusive source-code modifications to explicitly define compilable regions, suffered from severe compile-time latency, lacked support for PyTorch's native imperative (“eager”) execution mode, and lagged in supporting emerging distributed computing primitives within the PyTorch ecosystem.\n\n[TorchTPU](https://github.com/google-pytorch/torch_tpu) takes a fundamentally different approach, making eager execution a top priority, and enabling existing PyTorch workloads to run out with minor code modifications.\n\nSpecifically, migrating from GPUs to TPUs involves replacing the `cuda` device with `tpu` and the `nccl` distributed backend with `tpu_dist`. Alternatively, developers can use the new `torch.accelerator.current_accelerator` API to identify the available accelerator, as shown in the following snippet.\n\n``` python\nimport torch\nimport torch.distributed as dist\n\ndef main():\n    device = torch.device(\"cuda\") # -> device = torch.accelerator.current_accelerator\n    dist.init_process_group(backend=\"nccl\") # -> dist.init_process_group()\n    rank = dist.get_rank()\n    word_size = dist.get_world_size()\n    ...\n    model = model.to(device)\n    ...\n```\n\nTorchTPU support three eager execution modes: strict, debug, and DeferAndFuse. The latter is a new hybrid eager/compiled execution mode that Google claims \"can be 50% or more faster than Strict Eager while reducing peak HBM memory\".\n\nTo test TorchTPU's ability to run existing PyTorch models with no modifications, Google built an integration pipeline to evaluate about 5,000 Hugging Face models across 500+ architectures in both eager and compiled modes on TPU hardware generations starting from v5e.\n\nWe use a robust testing methodology grounded in corpus stratification and balanced architectural selection. Our approach relies on a two-tiered selection process designed to balance real-world adoption with broad architectural diversity, avoiding redundant testing of minor variations while ensuring real production weights are thoroughly evaluated.\n\nTo validate TorchTPU's performance, Google ran a 30-billion parameter Muse Glimmer model implemented in standard PyTorch. On a single TPUv7 chip, inference throughput reached 25.7k token/s at 5120 sequence length and 22.4k token/s at 10,240 sequence length. For training, using four TPUv7 chips, throughput reached 5.6k token/s and 4.6k token/s at 5120 and 10,240 sequence length, respectively.\n\nGoogle also conducted experiments using Qwen3 MoE 235B for training workloads, scaling from 1,024 to 8,192 TPUv7 chips. For inference, Google reported competitive performance-per-dollar for large-scale MoE workloads using vLLM.\n\nCommenting on the announcement, [Hacker News user tingkuanpei observed](https://news.ycombinator.com/item?id=50180165):\n\nVery interesting work. Google should have done this a long time ago. PyTorch's ecosystem is just so rich.\n\nOther users pointed out that beyond offering TPUs through GCP, Google sells them to [\"enterprise client for on premises deployments\"](https://news.ycombinator.com/item?id=50175919), which include [Anthropic](https://news.ycombinator.com/item?id=50171644) and [in Meta following potentially in the future](https://news.ycombinator.com/item?id=50174910).\n\nTorchTPU is still at an early stage and therefore in continuous evolution. The project's roadmap include work on dynamic shapes, compilation performance, operator coverage, reinforcement learning, and quantization. For further details, make sure to read the original announcement, which provides a more comprehensive view of TorchTPU's architecture and scope.", "url": "https://wpnews.pro/news/google-and-meta-open-source-torchtpu-to-bring-native-pytorch-support-to-google", "canonical_source": "https://www.infoq.com/news/2026/11/torchtpu-native-pythorch-tpu/?utm_campaign=infoq_content&utm_source=infoq&utm_medium=feed&utm_term=global", "published_at": "2026-11-02 17:00:00+00:00", "updated_at": "2026-11-02 17:22:24.863896+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "ai-infrastructure", "ai-chips", "developer-tools"], "entities": ["Google", "Meta", "TorchTPU", "PyTorch", "PyTorch/XLA", "TPU", "Hugging Face", "Anthropic"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/google-and-meta-open-source-torchtpu-to-bring-native-pytorch-support-to-google", "markdown": "https://wpnews.pro/news/google-and-meta-open-source-torchtpu-to-bring-native-pytorch-support-to-google.md", "text": "https://wpnews.pro/news/google-and-meta-open-source-torchtpu-to-bring-native-pytorch-support-to-google.txt", "jsonld": "https://wpnews.pro/news/google-and-meta-open-source-torchtpu-to-bring-native-pytorch-support-to-google.jsonld"}}