The semiconductor industry has advanced for several decades by widening the same highway. Shrink the transistor, add more of them to a chip, and systems become faster, smaller, and more efficient. But AI is now turning that highway into a sprawling transportation network.
AI differs from previous computing workloads because it simultaneously increases demand for compute, memory bandwidth, interconnect capacity, and power. Solving one bottleneck often exposes another, forcing engineers to optimize entire systems rather than individual components.
Data needs to move continuously between processors, memory stacks, photonic interconnects, power-delivery circuits, and cooling systems—often across multiple layers of silicon. The challenge is no longer simply building faster transistors. It’s orchestrating the movement of data, power, and heat across increasingly complex systems.
According to imec, that shift is pushing the semiconductor industry beyond the traditional SoC paradigm and into a new era it calls heterogeneous large-scale integration, or HLSI.
View All “We don’t need to break Moore’s Law,” Julien Ryckaert, VP of R&D at imec, told EE Times. “Moore’s Law only says I want more compute per dollar. There are other ways to accomplish that.”
The answer, Ryckaert argued, isn’t a single breakthrough technology but a growing ability to combine specialized technologies into a coherent system. “The rules of the game have changed,” he said. “Now that I can blend four or five technologies, how do I rethink the architecture of my compute system accordingly?”
Much of this shift is enabled by advances in 3D integration and hybrid bonding. As interconnect densities increase and communication costs between stacked dies approach those found within a monolithic chip, designers gain new freedom to partition systems across multiple layers.
“With the progress of 3D and backside technology, you can connect two dies at the granularity of how objects are connected on the same chip,” Ryckaert said. “So, there’s no energy cost anymore. Right away, that completely breaks the SoC paradigm.”
Instead of constructing a single larger floor, chip designers are increasingly building upward, stacking specialized functions on top of one another and connecting them through dense vertical links.
“The package becomes the new chip,” Ryckaert said.
Manufacturing becomes system optimization
The move toward HLSI has significant implications for manufacturing. For decades, semiconductor innovation focused largely on transistor technology, lithography, and process scaling. Those areas remain critical, but manufacturers increasingly see future progress emerging from how multiple technologies are integrated together.
TSMC describes this approach as system-technology co-optimization.
“Transistor scaling is indeed becoming more and more challenging, and costs are growing higher,” a company spokesperson said in response to questions from EE Times. “However, the die itself will always be an important center of innovation for TSMC.”
The company pointed to ongoing research in CFET transistors, new channel materials, and advanced interconnect architectures. At the same time, TSMC argued that future innovation increasingly depends on bringing multiple technologies together within a single platform.
“Because it is system-level, it brings together many different types of players into a single ecosystem,” the company spokesperson said. “We are seeing many more new ideas and innovations now that the door has been opened to system-technology co-optimization.”
That vision closely mirrors the HLSI framework proposed by imec. Asked whether HLSI aligns with TSMC’s view of future AI infrastructure, the company pointed to systems that combine memory, photonics, chip stacking, and advanced packaging technologies within a single package.
“It is an excellent example of system-technology co-optimization,” the TSMC spokesperson said.
The trend is already visible in advanced AI hardware. Package sizes continue to grow as designers combine more compute, memory, and interconnect resources into integrated systems. Optical links are also moving closer to the processor as copper-based interconnects face growing power and bandwidth limitations.
TSMC said its co-packaged optics technology can deliver significant improvements in both latency and power efficiency compared to conventional copper connections, and that the technology’s already entering production.
The implication is that future scaling increasingly happens at the system level rather than inside a single die.
Design tools must evolve
If HLSI resembles the construction of a vertical city, the software used to design it must change as well. Historically, EDA tools benefited from abstraction. Designers could optimize transistors, chips, packages, and systems in largely separate domains. Heterogeneous integration begins to collapse those boundaries. “We’ve been using abstraction for decades to enable complexity,” Rob Knoth, group director of solutions marketing at Cadence, told EE Times. “Now we need to optimize across those layers.”
In HLSI systems, thermal behavior influences placement decisions. Memory architecture affects package design. Photonics influences system layout. Power delivery must be optimized alongside compute architecture. The result is a dramatically more interconnected design process.
“We’re moving from optimizing chips to optimizing systems,” Knoth said.
One example is physical design itself. Traditional placement tools were largely built around X and Y coordinates on a two-dimensional plane. Emerging 3D systems require optimization across a third dimension as well, forcing EDA vendors to rethink some of the industry’s most fundamental algorithms.
At the same time, designers increasingly need to model interactions between silicon, packaging, photonics, thermals, and mechanical behavior simultaneously.
That challenge goes beyond geometry. In increasingly heterogeneous systems, thermal behavior, photonics, packaging, and silicon all interact. “The physics don’t care where your abstraction boundaries are,” Knoth said.
That complexity also increases the importance of simulation and automation. Engineering teams are struggling to scale at the same pace as AI system complexity. As a result, AI is increasingly incorporated into design tools to automate tasks that would otherwise require larger engineering organizations.
The irony is that the technology creating much of the complexity may also help solve it. As semiconductor systems become more complex, AI is increasingly being used to automate parts of the design process itself, creating a feedback loop in which AI is both driving the industry’s scaling challenge and helping engineers manage its solution.
Memory becomes the new battleground
If there’s one area where the HLSI transition is likely to be felt most strongly, it may be data storage and movement. Traditional memory hierarchies were optimized for CPU-centric computing. AI workloads place different demands on systems, creating pressure for new architectures and more specialized forms of integration. “Those memories were developed for CPU organization, not for these AI systems,” Ryckaert said. “Now people need to design memories in such a way that the movement of the data between where it’s processed and where it’s stored is the most efficient possible.”
The next question is no longer how many transistors fit onto a chip, but how effectively compute, memory, power, and connectivity can be orchestrated across an increasingly heterogeneous system.
The industry spent the past 50 years perfecting very large-scale integration. Its next challenge may be learning how to engineer heterogeneity at scale. And fittingly, AI may prove to be both the reason that transition became necessary and one of the tools that make it possible.
Read also:
[AI and Semiconductors Dance a Quick Two-Step](https://www.eetimes.com/ai-and-semiconductors-dance-a-quick-two-step/)
[Why Chip Sovereignty Is No Longer About Chips—But Systems](https://www.eetimes.com/why-chip-sovereignty-is-no-longer-about-chips-but-systems/)