{"slug": "aws-launches-torchneuron-to-run-pytorch-models-on-aws-trainium-without", "title": "AWS Launches TorchNeuron to Run PyTorch Models on AWS Trainium Without Modifications", "summary": "Amazon introduced TorchNeuron, a native PyTorch backend that runs existing PyTorch code on AWS Trainium AI chips with virtually no code modifications, at the PyTorch Conference North America 2026. According to AWS documentation, the only required change when using PyTorch's automatic device detection through torch.accelerator is replacing .to('cuda') with .to('neuron'), and the backend natively supports distributed training and parallelism APIs including DDP, FSDP2 and DTensor. Amazon Annapurna Labs vice president Colin Brace said his team validated the 100,000 most popular models and more than 800 ML architectures, and Amazon says a single configuration change is sufficient to switch hardware, letting enterprises reallocate Trainium capacity between training and inference.", "body_md": "At the recent [PyTorch Conference North America 2026](https://pytorch.org/event/pytorch-conference-2026/), Amazon introduced [TorchNeuron](https://github.com/aws-neuron/torch-neuronx), a native PyTorch backend that enables developers to run existing PyTorch code on [AWS Trainium AI chips](https://aws.amazon.com/ai/machine-learning/trainium/) with virtually no code modifications.\n\nBeyond reducing engineering effort, TorchNeuron allows enterprises to share hardware capacity between training and inference workloads, reallocating resources from inference to training during off-peak hours.\n\nAmazon says that [a single configuration change is sufficient to switch hardware](https://www.aboutamazon.com/news/innovation-at-amazon/torchneuron-pytorch-aws-trainium-chips-ai), eliminating the need to rewrite and revalidate models when moving to Trainium.\n\nToday, 85% of AI researchers and developers write their code in PyTorch. Until now, the hardware that code could run on was far more limited than the code itself, and expanding to new chips meant rewriting and revalidating work that took months to perfect.\n\nAccording to the [TorchNeuron documentations](https://awsdocs-neuron.readthedocs-hosted.com/en/latest/torch/index.html), when leveraging PyTorch's automatic device detection through `torch.accelerator`, the only required change is replacing `.to('cuda')` to `.to('neuron')`. The backend also natively supports distributed training and PyTorch parallelism APIs, including DDP, FSDP2, DTensor, and others. In addition, TorchNeuron provides native support for Torch streams and events through `torch_neuronx.Stream` and `torch_neuronx.Event`.\n\nAmazon Annapurna Labs vice president Colin Brace, who leads the team behind TorchNeuron, emphasized that his team [validated the 100,000 most popular models and more than 800 ML architectures](https://lnkd.in/p/ev2BSv2J) to ensure a smooth developer experience when adopting the new backend.\n\nAdditionally, TorchNeuron enables developers and researchers to use the [Neuron Kernel Interface](https://awsdocs-neuron.readthedocs-hosted.com/en/latest/nki/index.html) to develop, optimize, and execute custom operators:\n\nDevelopers can obtain finer performance control beyond Adaptive Eager and torch.compile by calling performance critical NKI kernels directly in model code to replace sequences of standard PyTorch operations (ATen ops).\n\nCommenting on Amazon's announcement on LinkedId, Liner Notes Consulting CEO [Yasuo Maeda noted](https://lnkd.in/p/embbfUmx) that:\n\nThe real test will be whether developers can achieve competitive performance and total cost of ownership in production. The AI infrastructure race is not just about who builds the best chips. It is also about who makes them easiest to use at scale.\n\nSimilarly, user Pratham pratham hightlighted that, while reducing migration friction is important to avoiding accelerator lock-in, [the real test lies in workload-specific performance, cost per token, and reproducibility](https://lnkd.in/p/e3RbJ-bZ).\n\nAmazon also emphasizes that TorchNeuron has already been adopted by several organizations, including Cartesia, which is using it for voice AI inference, and the Lieber Institute for Brain Development, which explores applications in drug target identification and drug discovery.\n\nAWS TorchNeuron announcement comes on the heels of [Google's launch of TorchTPU](https://www.infoq.com/news/2026/11/torchtpu-native-pythorch-tpu/), its native PyTorch backend for Google's TPU hardware.", "url": "https://wpnews.pro/news/aws-launches-torchneuron-to-run-pytorch-models-on-aws-trainium-without", "canonical_source": "https://www.infoq.com/news/2026/11/aws-torchneuron-trainium-pytorch/?utm_campaign=infoq_content&utm_source=infoq&utm_medium=feed&utm_term=global", "published_at": "2026-11-09 11:00:00+00:00", "updated_at": "2026-11-09 11:22:10.728377+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "ai-chips", "ai-infrastructure", "developer-tools"], "entities": ["Amazon", "AWS Trainium", "TorchNeuron", "PyTorch", "Colin Brace", "Amazon Annapurna Labs", "Neuron Kernel Interface", "Cartesia"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/aws-launches-torchneuron-to-run-pytorch-models-on-aws-trainium-without", "markdown": "https://wpnews.pro/news/aws-launches-torchneuron-to-run-pytorch-models-on-aws-trainium-without.md", "text": "https://wpnews.pro/news/aws-launches-torchneuron-to-run-pytorch-models-on-aws-trainium-without.txt", "jsonld": "https://wpnews.pro/news/aws-launches-torchneuron-to-run-pytorch-models-on-aws-trainium-without.jsonld"}}