How Semiconductor Companies Use Multi-Die Systems

Shekhar Kapoor

May 18, 2023 / 5 min read

As system designers seek to pack ever-more transistors into smaller spaces, monolithic system-on-chips (SoCs) are quickly nearing the reticle ceiling for manufacturing. Simply put, conventional SoCs are becoming too big and costly to produce, especially for compute-intensive applications such as machine learning (ML), high-performance computing (HPC), and advanced driver assistance systems (ADAS).

Indeed, these workloads are incredibly demanding due to massive parallelism requirements for multiply-accumulate (MAC) operations such as dot product functions. Thanks to multi-die systems, purpose-built chips can now support compute-intensive tasks with a diverse lineup of high-end processors, sophisticated memory arrays, and reliable real-time data connectivity between dies.

Consisting of multiple heterogeneous dies integrated into a single package, multi-die systems offer semiconductor companies a more revolutionary way of designing and fabricating a new generation of silicon. “To be able to tackle these very large workloads, we need to put more silicon in the package than would fit in a single monolithic die,” Gerry Talbot, AMD corporate fellow, explains in a recent MIT Technology Review Insights report. “You physically could not print it in a single reticle.”

Titled “Multi-Die Systems Define the Future of Semiconductors,” the MIT Technology Review report explores why multi-die systems will be instrumental in meeting burgeoning industry demand for compute power. In this blog post, we summarize key sections of the MIT Technology Review report and highlight additional quotes from senior Synopsys executives and industry experts—with a focus on how semiconductor companies can leverage the advances of multi-die systems to create new business opportunities.

Multi-Die System (abstract)

Why Multi-Die Systems?

According to Synopsys Chair and CEO Aart de Geus, multi-die systems enable semiconductor companies to implement heterogeneous dies or chiplets on different process nodes.

“The processor needs to be as fast as possible,” de Geus said during his keynote speech opening the SNUG Silicon Valley 2023 conference. “Other [components] can be manufactured in older, better yielding, easier technologies.”

As de Geus explains, multi-die systems are having a massive impact on the industry and empowering designers to deliver advanced silicon with more computation and transistors. Synopsys President and COO Sassine Ghazi expressed similar sentiments during a SNUG Silicon Valley 2023 Q&A session, noting that multi-die systems continue to spark a tremendous amount of excitement across the industry.

Semiconductor companies, says Ghazi, can now efficiently disaggregate chips and cost-effectively stack components—with 2.5D and 3D techniques—in different locations on the same package. “Once you get into manufacturing, packaging, and testing, there is a new wave of complexity with lots of runway for industry innovation,” he adds during the Q&A session.

According to Patrick Moorhead, founder, CEO, and chief analyst at global technology consulting firm Moore Insights & Strategy, multi-die systems allow semiconductor companies to optimize the design of customized and differentiated processors. “People are looking at more custom silicon as a way to differentiate what they bring to the table, [and] that’s what businesspeople should be looking at,” he explains in the report. “Chiplets enable smaller companies with smaller pocketbooks to use semiconductors for unique competitive advantage.”

As Moorhead points out in the report, innovation isn’t nearly as capital intensive or as resource dependent with multi-die systems. “It doesn’t take five years to get to market,” he states. “You’re shaving off at least a year, maybe two years, if you have a chiplet implementation because you’re not spending time hardening the monolithic design.”

Accelerating Autonomous Driving and Enhancing Generative AI

François Piednoël, distinguished chief mSoC (multiple system on chip) architect at Mercedes-Benz, sees custom multi-die systems benefiting the automotive industry by facilitating complex power management for autonomous driving. “If a user wants to drive with Level Two autonomy, where they still need to be in control of the car and pay attention but with some lane-keeping assistance, chiplets can provide this capability without consuming the power of an entire chip,” he states in the report.

During his SNUG Silicon Valley 2023 keynote, Piednoël said multi-die systems will play a major role in accelerating adoption of Level Four autonomy while ushering in a new era self-driving vehicles that will safely navigate roads and highways. With ADAS processing and inferring dozens of images every 0.7 milliseconds, compute power is clearly a top priority for the automotive industry. “Chiplets allow us to scale, with [sophisticated] functions that will [tier] down from level four to level three and two,” he adds.

Uri Frank, vice president of engineering, Google, says in the report that multi-die systems can improve the viability of large language models (LLMs) and other generative AI engines like ChatGPT. These technologies are currently seeing exponential growth, enabling them to be deployed at scale. While beta testing generative AI engines is an important step, Frank notes that these tools “require a lot of hardware, and they are actually hardware-limited today.”

According to Frank, multi-die systems also enable better and more nuanced customization. With multi-die systems, circuits supporting conventional tasks can be placed on less advanced nodes, while newer nodes are dedicated to more advanced workloads such as AI and ML. “You can mix and match and optimize for a better performance-at-cost solution,” he says in the report.

Multi-Die Systems Facilitate Cross-Industry Collaboration

For many years, semiconductor companies reliably secured their respective market positions by pushing engineers to the next process node. However, the slowdown of Moore’s law prompted companies to re-evaluate chip design with multi-die systems and re-envision their participation in the wider semiconductor ecosystem.

To be sure, multi-die systems are creating exciting new partnerships, opportunities, and standards including Universal Chiplet Interconnect Express (UCle). This open specification for die-to-die interconnects between dies or chiplets is rapidly gaining popularity and sparking innovative multi-die designs across the industry. Chip foundries like TSMC help foster this innovation—although the company doesn’t design products. To successfully manufacture multi-die systems for its customers, TSMC works closely with ecosystem partners to facilitate collaboration between players in chip design, materials, testing, and packaging.

“We create a technology platform and allow our customers to innovate on it,” Kevin Zhang, senior vice president of business development, TSMC, explains in the report. “We bring in different product players—including fabless design companies, system companies, for example—to develop a product that can be seamlessly integrated with our underlying technology and create innovative products to deliver significant benefits to the end users.”

As Zhang notes in the report, TSMC offers a family of chip stacking and packaging technologies, called TSMC 3DFabric™, to address some common chiplet design challenges. “We’ve developed a broad portfolio of different ways to integrate multiple chips together,” he elaborates. “[With 3D stacking], you can create a system with high-bandwidth interconnects between the chips while maintaining very small form factors.”

In the future, Zhang says in the report, multi-die systems will become a “prevailing way to bring more function and more capabilities together to achieve better system-level performance and power efficiency.”

While multi-die systems aren’t a new concept, it is fast gaining traction across the semiconductor ecosystem. According to Ghazi, if late 2022 was an inflection point for multi-die systems, 2023 looks to be the year when these architectures really take off.

“We’re not talking about the industry just trying to test the water—it’s well beyond that,” he emphasizes in the report, noting that Synopsys is currently tracking hundreds of unique systems designed around multi-die architecture.

“The need is there, the push from industry is absolutely there,” Ghazi concludes in the report. “What makes it exciting is the innovation we’re seeing across the chain—from architecture all the way to manufacturing—and how we’re collaborating to optimize the whole technology stack so multi-die systems can reach scale across global markets.”

Continue Reading