{"slug": "meta-bakes-networking-on-chip-in-custom-silicon-designed-to-cut-cpu-hops", "title": "Meta bakes networking on-chip in custom silicon designed to cut CPU hops", "summary": "Meta Platforms has unveiled its MTIA 300 custom AI accelerator, which integrates two network chiplets with six 800 Gb/s RDMA NICs each, delivering 1.2 TB/s of I/O bandwidth and eliminating CPU-mediated network hops. The chip, designed for training recommendation models, achieves 3.9-times faster total communication time than equivalent GPU clusters on a 150 billion parameter model across 40 accelerators, and includes 216 GB of HBM3E memory. Meta says the integrated networking and offloaded collective execution could also benefit broader AI workloads such as reasoning and agentic inference.", "body_md": "Meta has put networking front and center in its next-generation custom silicon, placing network interface cards (NICs) inside the chip package itself to reduce interconnectivity bottlenecks.\n\nIts Meta Training and Inference Accelerator (MTIA) 300 packs two network chiplets – each packing six custom 800 Gb/s remote direct memory access (RDMA) NICs – to provide 1.2 TB/s of total I/O bandwidth without the need to touch a peripheral component interconnect express (PCIe) bus.\n\nNormally, NICs would sit off-chip, with the central processing unit (CPU) used to broker traffic between accelerators and the network. Meta engineers have essentially removed that extra hop to eliminate that bottleneck.\n\nMeta claims the on-board NICs make the MTIA 300 specially suited for training recommendation and ranking models, which are the engine behind what shows up in feeds of Meta apps like Facebook and Instagram\n\nUnlike large language models (LLMs), which require vast floating-point throughput, recommendation models require constant communication between accelerators. Their embedding tables, which contain over 99% of the model’s parameters, require memory capacity and high memory bandwidth (HBM).\n\nMeta's [latest earnings](https://investor.atmeta.com/investor-news/press-release-details/2026/Meta-Reports-Second-Quarter-2026-Results/default.aspx) showed $59.4 billion in advertising revenue, highlighting that having custom hardware is a competitive necessity.\n\nThe hyperscaler has been designing its own custom MTIA silicon to power its social media empire since 2023. Back in March, it unveiled the [next four generations of MTIA](https://www.datacenterdynamics.com/en/news/meta-unveils-next-four-generations-of-its-mtia-chip/), with the 400 version a compute powerhouse offering six petaflops of eight-bit floating point (FP8) compute performance.\n\nThe network-enhanced MTIA 300s boast impressive performance details. On a 150 billion parameter recommendation model running across 40 accelerators, the chip’s total communication time is 3.9-times faster than equivalent graphic processing unit (GPU) clusters. Meta uses the same 12 Ethernet-based NICs for both scale-up and scale-out, which allows for flexible partitioning of the NICs to adjust the changing workload needs.\n\nThe chip houses huge on-chip memory, 216 gigabytes of high-bandwidth memory 3E (HBM3E), which supports larger local batch sizes and reduces communication overheads. It's paired with Meta's HCCL custom communication library that can execute network operations without host CPU involvement by compiling data patterns into autonomous work-queue subgraphs, while high-network bandwidth lets Meta use higher-precision datatypes to maintain precision.\n\n“Rather than driving communication from the host during execution, HCCL compiles each collective into a complete set of subgraphs – arrays of work-queue entries with explicit dependencies – dispatched to message engines for fully autonomous execution. Once work reaches the device, the host is uninvolved,\" [Meta noted in a blog post](https://engineering.fb.com/2026/08/24/networking-traffic/mtia-300-meta-training-chip-built-in-nics/).\n\nMeta engineers contend that while the package is designed for training recommendation models, the integrated networking and offloaded collective execution concepts could apply to broader workloads, including AI.\n\n“As AI inference evolves toward reasoning, agentic, and long-context use cases, the communication demands a shift: messages become smaller, more frequent, and latency-sensitive, with tighter per-collective budgets,” the blog post reads.", "url": "https://wpnews.pro/news/meta-bakes-networking-on-chip-in-custom-silicon-designed-to-cut-cpu-hops", "canonical_source": "https://www.sdxcentral.com/news/meta-bakes-networking-on-chip-in-custom-silicon-designed-to-cut-cpu-hops/", "published_at": "2026-08-28 13:30:14+00:00", "updated_at": "2026-08-28 13:50:57.436882+00:00", "lang": "en", "topics": ["ai-chips", "ai-infrastructure", "ai-research"], "entities": ["Meta Platforms", "MTIA 300", "Facebook", "Instagram", "HCCL"], "alternates": {"html": "https://wpnews.pro/news/meta-bakes-networking-on-chip-in-custom-silicon-designed-to-cut-cpu-hops", "markdown": "https://wpnews.pro/news/meta-bakes-networking-on-chip-in-custom-silicon-designed-to-cut-cpu-hops.md", "text": "https://wpnews.pro/news/meta-bakes-networking-on-chip-in-custom-silicon-designed-to-cut-cpu-hops.txt", "jsonld": "https://wpnews.pro/news/meta-bakes-networking-on-chip-in-custom-silicon-designed-to-cut-cpu-hops.jsonld"}}