TorchTPU Goes Open Source: 4 Repos, Oct. 20 [2026]

Google and Meta open-sourced TorchTPU on October 20, 2026, releasing four repositories — TorchTPU, vLLM-TorchTPU, SGLang-TorchTPU and TorchTitan — that give PyTorch developers a native backend for Google TPUs, according to Google Cloud's announcement at PyTorch Conference North America 2026 in San Jose. Google began the effort in mid-2025 with Meta's PyTorch team, and the release lets the vLLM and SGLang inference engines run natively on TPUs while TorchTitan supports dense, mixture-of-experts and diffusion training workloads. The announcement included no price, benchmark or adoption figure.

Google and Meta open-sourced TorchTPU on October 20, 2026, giving PyTorch developers a native backend for Google TPUs. According to Google Cloud’s announcement https://cloud.google.com/blog/topics/ai-infrastructure/torchtpu-is-now-open-source-run-native-pytorch-on-tpus , the project lets TPUs join GPUs and CPUs as a native option for PyTorch. The reveal came at PyTorch Conference North America 2026 in San Jose, which ran October 20 to 21. The pitch is simple. Take a PyTorch workload you already run, point it at a TPU, and change as little code as you can. For years that promise sat behind a translation layer or a rewrite into another framework. TorchTPU tries to remove that tax, and it does so with Meta’s PyTorch team as a co-developer rather than a bystander. If you already serve models with vLLM, as in our walkthrough for running an LLM on DGX Spark https://shattered.io/run-llm-dgx-spark-ollama-vllm-12-steps-2026/ , the new repositories suggest the same engine can now target a second kind of silicon. This analysis covers what shipped, what the sources confirm, what stays unproven, and what the release means for CUDA, JAX and the cloud accelerator market. Don't miss new tech stories on Google Add Shattered once in the Google app and our stories show up in your news suggestions. Add now https://www.google.com/preferences/source?hl=en&q=https%3A%2F%2Fshattered.io%2F What Google and Meta Announced at PyTorch Conference North America 2026 TorchTPU is an open-source collaboration between Google and Meta. It provides a native PyTorch backend for Google’s Tensor Processing Units. Google Cloud and InfoQ both report the announcement date as October 20, 2026, and the official event pages place the conference in San Jose, California. One naming detail deserves a correction. Many headlines say “PyTorch Conference 2026,” but the official event pages call it PyTorch Conference North America 2026. The distinction matters because the PyTorch community runs regional events, and a reader searching for session videos or slides needs the exact name. Two people introduced the project on stage. Bill Jia of Google Cloud and Zongwei Zhou, a Meta software engineer, presented TorchTPU alongside a segment on agentic developer workflows. The PyTorch project account announced the session on X, and the SGLang project posted its own note the same day saying it was officially open-sourcing TorchTPU for native, hardware-accelerated PyTorch execution on Cloud TPUs. The timeline also tells a story. Google started the effort in mid-2025 in collaboration with the PyTorch team at Meta. That means the project spent roughly a year or more in development before the public release, which is a normal runway for a compiler-heavy backend but a long one by the standards of AI tooling announcements. The release is not a product launch with a price tag. Nothing in the primary announcements gives a price, a benchmark or an adoption figure. What it gives is code, in the form of public repositories, and a clear statement of direction. That is the lens this article uses throughout. The Four Open-Source Repositories Behind TorchTPU Google and Meta did not ship a single library. They open-sourced four foundational repositories: TorchTPU, vLLM-TorchTPU, SGLang-TorchTPU and TorchTitan. Each covers a different layer of the stack, and together they map the path from a training run to a served model. TorchTPU is the core backend. It is the piece that lets PyTorch tensors and operations execute on a TPU as a first-class device. The other three repositories sit on top of it or beside it. vLLM-TorchTPU and SGLang-TorchTPU target inference, the phase where models answer user requests. TorchTitan targets large-scale training. InfoQ reports that the vLLM and SGLang inference engines run natively on the new backend, and that TorchTitan supports dense, mixture-of-experts and diffusion workloads. The PyTorch project’s conference preview described the same idea from a different angle: a unified backend that lets inference engines deploy current models on Cloud TPUs with minimal code changes. Google’s post also lists a wider set of familiar tools that now have a TPU route, including llm-d, Ray, TorchRec and LiteRT next to the four repositories above. The exact depth of support for each of those extras is not spelled out in the sources reviewed here, so treat the list as a statement of intent until the repository documentation says otherwise. | Component | Role in the stack | Reported by | |---|---|---| | TorchTPU | Native PyTorch backend for Google TPUs | Google Cloud, InfoQ | | vLLM-TorchTPU | Runs the vLLM inference engine on TPUs | Google Cloud, InfoQ | | SGLang-TorchTPU | Runs the SGLang inference engine on TPUs | Google Cloud, SGLang project | | TorchTitan | Large-scale training, covering dense, mixture-of-experts and diffusion workloads | Google Cloud, InfoQ | | torch.compile integration | Optimization path for production execution | Google Cloud | | Hugging Face Transformers | Model layer with TPU integration | PyTorch conference materials | | TPU generations documented | v5e, v5p, v6e and v7 Ironwood | Google TPU documentation | How TorchTPU Works: torch.compile, Eager Mode and Native Acceleration The technical pitch rests on two execution modes that PyTorch developers already know. Eager execution runs operations one at a time, which makes debugging easy. Compilation through torch.compile captures the program and optimizes it for the target hardware, which makes production runs fast. Google says TorchTPU integrates with torch.compile , supports eager-execution optimizations and provides native hardware acceleration. Why eager support matters on a TPU TPU software has long leaned on ahead-of-time graph compilation. That approach squeezes out performance, but it punishes anyone trying to step through a model with a debugger. Eager support closes part of that gap. A researcher can iterate the way they do on a GPU, then switch to the compiled path once the model works. Why torch.compile is the production path PyTorch itself pushed torch.compile as its main optimization route, so building TorchTPU around it keeps the backend aligned with where the core project is headed. A backend that fought the compiler would drift away from upstream with every release. One that rides the compiler stays closer to the mainline, and Meta’s involvement on the PyTorch side supports that alignment. The official line is that existing PyTorch workloads should run on TPUs with minimal code changes. That phrase is a goal, not a guarantee. Real models use custom operators, odd data loaders and third-party extensions, and each one can break a clean migration. Expect the first months of public use to turn up a long tail of unsupported operations. Which TPU Generations Fit Into the Picture Google’s documentation lists several TPU versions in current use: v5e, v5p, v6e and v7, the last one also called Ironwood. The sources reviewed for this article do not publish a support matrix that says which TorchTPU features work on which generation. That gap is worth flagging, because TPU generations differ in memory, interconnect and intended workload. The “e” parts have historically targeted cost-efficient inference and smaller training jobs, while the “p” parts target large-scale training. Ironwood sits at the new end of the line. Developers planning a migration should check the repository documentation for the exact combination they intend to rent, since a feature that works on one version may lag on another. Capacity is the other half of the question. Google Cloud and its rivals have all signaled tight supply for AI accelerators. Our coverage of Sundar Pichai’s comments that compute stays tight into 2027 https://shattered.io/pichai-compute-shortage-capacity-constrained-2027-2026/ shows why a software path to more kinds of silicon has real strategic value right now. A native PyTorch route does not create new chips. It does widen the pool of developers who can use the chips that exist. If a team could not justify a rewrite into JAX, TorchTPU lowers the bar to testing a TPU rental. That is a distribution play as much as an engineering one. Hugging Face, vLLM and SGLang: The Model and Serving Layer Most teams do not write models from scratch. They pull weights through Hugging Face Transformers and serve them with vLLM or SGLang. TorchTPU supports integration with all three of those, plus TorchTitan on the training side. That coverage is what turns a backend from a research curiosity into something a platform team might adopt. The vLLM and SGLang pieces carry special weight. Both engines have become default choices for open-weight model serving, and both bring their own schedulers and batching logic. Running them natively on TPUs means a team can keep its serving setup and change the hardware underneath. That separates TorchTPU from earlier TPU serving paths that required a separate stack. Open-weight models are the obvious beneficiary. Anyone shopping the current field, as we did in our ranking of the best open-source LLMs for October 2026 https://shattered.io/best-open-source-llm-october-2026-kimi-glm-deepseek/ , can now ask whether a given model also runs well on TPUs. Before this release, the honest answer for many PyTorch-first teams was “possible, but painful.” Training gets less attention than serving, but TorchTitan is arguably the more ambitious piece. InfoQ reports it supports dense, mixture-of-experts and diffusion workloads. Mixture-of-experts models stress networking and memory in ways that dense models do not, so credible support there says something about how seriously the teams treated the training path. What the People Behind TorchTPU Said Three statements from the launch materials frame how the companies want the project read. Bill Jia, who wrote the Google Cloud announcement, put the headline claim in one line: “With TorchTPU, TPUs join GPUs and CPUs as a native option for PyTorch.” The sentence positions the TPU as a peer device, not a special target that needs its own tooling. Zongwei Zhou, the Meta engineer who co-presented, kept the tone modest. Speaking in the conference keynote, as InfoQ reported https://www.infoq.com/news/2026/10/google-meta-torchtpu/ , he said: “All of this is open source and available today and it is just a beginning.” The phrase “just a beginning” is a fair warning that the repositories will change quickly. Jana van Greunen, Meta’s Director of PyTorch Engineering, gave the clearest reason for Meta’s involvement in a statement included in Google’s post: “At Meta, advancing the frontier of AI requires immense compute scale and deep hardware diversity. We cannot be constrained by a single hardware architecture.” Her second line, from the same post, said the project “enables us to deploy our PyTorch workloads to Google TPUs, provide direct feedback to the engineering teams, and build the ecosystem flexibility everyone needs.” Read carefully, those statements describe capability and direction. They do not announce a migration. None of the three speakers said Meta will move its largest language model workloads onto TPUs, and that claim, which has circulated in some summaries, is not confirmed by the primary announcements. Why Meta Backs a Google Chip Path Meta created PyTorch and still steers its development with the broader community. A company in that position gains from a framework that runs well on every accelerator, because it shrinks the switching cost between suppliers. Van Greunen’s remark about not being constrained by a single hardware architecture reads as a procurement argument as much as an engineering one. Her second statement adds a feedback loop. By deploying PyTorch workloads on TPUs and sending feedback to the engineering teams, Meta shapes how the backend evolves. That is a classic open-source bargain. The hardware vendor supplies the chips and compiler expertise, and the framework owner supplies the user-side pressure that keeps the API sane. For Google, the motive is easier to read. TPUs compete for customers against NVIDIA’s GPU line, and the main barrier for many customers has been software familiarity, not raw silicon. A native PyTorch route removes the strongest objection a PyTorch-first buyer could raise. It also gives Google Cloud a stronger story at a moment when spending on data centers keeps climbing, a trend our piece on Bank of America’s $2.1 trillion data center forecast https://shattered.io/bofa-data-center-spending-2-1-trillion-2028-2026/ lays out. Whether either company benefits in dollars is not something the sources can answer. No revenue figure, customer count or capacity commitment accompanied the release. Anyone claiming a measurable market effect this week is guessing. Market Impact: Pressure on the CUDA Default CUDA’s strength never came from one product. It came from the habit of writing PyTorch code that assumes an NVIDIA GPU is underneath. Every tutorial, every model repository and every cloud notebook reinforced that habit. TorchTPU attacks the habit directly by making a second device behave like a normal PyTorch target. The effect, if it works, shows up first in procurement conversations. A platform lead who can credibly say “our stack runs on either” gains leverage when negotiating GPU prices or waiting on delivery slots. Even teams that never deploy on TPUs may benefit from the threat. That is how portability tends to pay off in infrastructure markets. Cloud supply is the counterweight. Large providers report capacity commitments stretching well into 2027, as shown in our report that AWS capacity is largely sold out for 2027 https://shattered.io/aws-capacity-largely-sold-out-2027-jassy-220b-2026/ . If TPU capacity is also tight, a better software path helps only as far as Google can supply hardware. The release changes the developer experience far faster than it changes physical supply. No adoption data exists yet. The repositories went public days ago, and the primary announcements publish no usage, download or customer figures. Watch for three early signals instead: issue-tracker activity on the repositories, third-party benchmark posts, and cloud providers adding TorchTPU images to their managed AI services. Competitive Comparison: TorchTPU, PyTorch/XLA, JAX and CUDA TorchTPU does not replace everything that came before. It joins a field of established routes, and each has a different audience. The table below compares them on the dimensions the sources support. It deliberately omits speed and cost columns, because no verified benchmark exists for the new backend. | Path | Developer writes | Target hardware | Position after Oct. 20, 2026 | |---|---|---|---| | TorchTPU | Native PyTorch | Google TPUs | New open-source backend, built with Meta’s PyTorch team | | PyTorch/XLA | PyTorch through an XLA bridge | Google TPUs | Existing route that TorchTPU aims to make less necessary | | JAX with XLA | JAX programs | Google TPUs and GPUs | Google’s own TPU-first software path, still a major option | | PyTorch on CUDA | Native PyTorch | NVIDIA GPUs | The default most tutorials and model repositories assume | | vLLM-TorchTPU and SGLang-TorchTPU | Existing serving configs | Google TPUs | New TPU routes for two popular inference engines | | Hugging Face Transformers integration | Standard model code | Google TPUs through TorchTPU | Integration described in conference materials | The sharpest contrast is with PyTorch/XLA. That bridge let PyTorch programs reach TPUs, but it treated the TPU as a guest rather than a native device. TorchTPU’s claim is that PyTorch can now treat the TPU as a native device, backed by torch.compile and eager support. Whether developers feel that difference will depend on operator coverage and error messages, two things no press release can prove. JAX is a different case. Google built it with TPUs in mind, and it remains Google’s own software route. TorchTPU does not threaten that base. It targets the large population of teams that chose PyTorch first and never moved. For a closer look at the GPU side of the same rivalry, see our Nvidia versus AMD comparison for local AI https://shattered.io/nvidia-vs-amd-gpu-local-ai-32gb-vs-24gb-2026/ . Historical Context: From Side Bridge to First-Class Backend Google introduced TPUs as in-house accelerators and later opened them to cloud customers. Their software story grew up around TensorFlow and then JAX, both Google projects. PyTorch, meanwhile, won much of the research community and a large share of production work. The two worlds met through XLA bridges that worked but never felt native. PyTorch’s own evolution set the stage. The arrival of torch.compile gave the framework a compiler-driven path that vendors could target without forking the project. TorchTPU builds on that shift. Instead of asking PyTorch users to adapt to TPU tooling, it asks the TPU backend to meet PyTorch where it already is. The governance angle matters too. PyTorch sits in a vendor-neutral community setting, and the conference where TorchTPU launched is a community event, not a Google keynote. Launching there signals that the backend aims to live in the PyTorch ecosystem rather than beside it. Our earlier explainer on fine-tuning an LLM with QLoRA https://shattered.io/fine-tune-llm-qlora-12-steps-2026/ shows how much of everyday AI work already runs on PyTorch-native tooling. This history explains why the announcement drew attention despite lacking benchmarks. The idea of a native PyTorch TPU path has been on developer wish lists for a long time. Seeing two companies commit code in public turns a wish into something testable. What Remains Unconfirmed Several details circulating in summaries do not appear in the primary announcements. The list below separates what the sources confirm from what they leave open. Keeping the two apart protects readers from planning around numbers nobody has published. | Claim | Status | Basis | |---|---|---| | TorchTPU is open source and built by Google with Meta | Confirmed | Google Cloud, InfoQ | | Announced Oct. 20, 2026 at PyTorch Conference North America | Confirmed | Google Cloud, event pages | | Project began at Google in mid-2025 | Confirmed | Google Cloud | | Four repositories open-sourced | Confirmed | Google Cloud | | Existing PyTorch workloads run with minimal code changes | Stated goal, not independently tested | Google Cloud | | Price, performance benchmark or adoption figure | Not published | No primary source | | Meta moving larger LLM workloads onto TPUs | Not confirmed | No primary source | One more caution applies to secondary write-ups. Some aggregator pages quote efficiency figures from large Ironwood runs. Those numbers may be accurate, but this article does not repeat them because the fact check for this piece could not tie them to a primary source it could verify. If Google publishes a technical report, the picture will sharpen quickly. What Developers Should Do This Week Start small. Pick one model you already run on a GPU, clone the TorchTPU repository, and follow its setup notes on a rented TPU. Measure whether the model loads, whether outputs match your GPU baseline, and how long compilation takes. Record every unsupported operator you hit, because that list is the real migration cost. The snippet below shows the general shape of device-agnostic PyTorch code. It is an illustration of the pattern, not an excerpt from the TorchTPU docs. The device string is a placeholder, so take the real value from the repository. python import torch Placeholder: replace with the device name the TorchTPU docs specify device = get device from torchtpu docs model = MyModel .to device compiled = torch.compile model for batch in loader: out = compiled batch.to device loss = criterion out, batch.labels.to device loss.backward optimizer.step optimizer.zero grad For serving, test vLLM-TorchTPU or SGLang-TorchTPU with a model you already host. Compare time to first token and throughput against your current setup, but wait for independent benchmarks before you commit budget. Treat any cost comparison as provisional until you have run your own traffic through both stacks. Finally, plan for churn. Zhou called this “just a beginning,” and young backends change fast. Pin your versions, file issues when something breaks, and keep a GPU fallback in your deployment pipeline until the TPU path earns trust on your own workloads. Five Predictions for TorchTPU These are analysis, not reporting. They reflect the direction the launch materials point toward, and any of them could miss. 1. Independent benchmarks will define the story. Within a few months, third-party testers will publish cost-per-token and training-throughput comparisons against GPUs. Until then, claims on either side remain unproven. 2. Operator-coverage gaps will surface first. Early adopters will report unsupported operations and rough edges in custom kernels. Fast fixes in the public repositories will matter more than any marketing. 3. More open-source projects will add TPU paths. With vLLM and SGLang already covered, other serving and training tools have a template to follow. 4. JAX will keep its core audience. Frontier teams already invested in JAX have little reason to switch, so TorchTPU will grow mainly among PyTorch-first teams. 5. Hardware portability becomes a standard procurement question. Buyers will ask vendors whether a stack runs on more than one accelerator family, and cloud sales teams will prepare answers. The takeaway: TorchTPU is a credible bid to make TPUs a normal PyTorch target, backed by two companies with strong reasons to make it work. Proof will come from public benchmarks and months of real use, not from the launch week. Frequently Asked Questions About TorchTPU What is TorchTPU? TorchTPU is an open-source native PyTorch backend for Google TPUs, built by Google in collaboration with the PyTorch team at Meta. It lets PyTorch treat a TPU as a native device alongside GPUs and CPUs. When was TorchTPU announced? Google and Meta announced it on October 20, 2026, at PyTorch Conference North America 2026 in San Jose. The conference ran October 20 to 21. Google says development began in mid-2025. Which repositories were open-sourced? Four foundational repositories went public: TorchTPU, vLLM-TorchTPU, SGLang-TorchTPU and TorchTitan. They cover the core backend, two inference engines and large-scale training. Do I need to rewrite my PyTorch code to use it? Google’s stated goal is that existing PyTorch workloads run on TPUs with minimal code changes. That is a design target, and results will vary by model, custom operators and extensions. Test your own workload before you rely on it. Which TPU versions are documented? Google’s documentation lists v5e, v5p, v6e and v7, also known as Ironwood. The sources reviewed here do not give a feature-by-feature support matrix per generation, so check the repository notes. Is TorchTPU faster or cheaper than GPUs? Nobody can say yet. The primary announcements publish no price, benchmark or adoption figure. Wait for independent tests and run your own measurements. Is Meta moving its large language model workloads to TPUs? That is not confirmed. Meta’s statements describe hardware diversity and the ability to deploy PyTorch workloads on Google TPUs, but no source says it will shift larger-scale LLM work.