From Silicon To Software

 

The Evolution to Co-Packaged Optics in Data Centers

By Scott Durrant, Strategic Marketing Manager, and Priyank Shukla, Product Marketing Manager; Synopsys Solutions Group and Mitch Heins, Sr. Business Development Manager, and Jigesh Patel, Staff Technical Marketing Manager; Synopsys Custom Design and Manufacturing Group

Did you know that Group Zoom meetings use up between 810 MB and 2.4 GB of data per hour? Your “Work from Home Productivity” playlist uses 2 MB+ of data per three-minute song, amounting to around 40 MB per hour. And that HD movie you just streamed from Netflix for family movie night used about 6 GB data. All that data adds up. There’s no arguing that over the past several years, there has been a rapid increase in the amount of data generated, transmitted, and analyzed (intensified by the ongoing COVID-19 pandemic and increased amounts of remote learning and working).

This has caused a dramatic increase in demand for faster data network and device interface speeds. With these higher data transfer speeds come increased energy requirements for data interfaces and increased sensitivity to latency in data transfer. Driven by a need to reduce power and increase bandwidth density in data center network switches and other devices, the data networking industry is moving toward the adoption of co-packaged optics (CPO).

Today, we’ll share a brief overview of the history of copper and optical interconnects used in data centers, the limitations of existing interconnect solutions, and the future of co-packaged optics.

Copper to Optic Cables: a Humble History

Copper has a major presence in networks because it is highly conductive, cheap, malleable, and resistant to heat. For these reasons and more, copper cables were (and continue to be in some cases) used for data networking even for long runs between data centers and around metropolitan areas. However, as network speeds increased, so did the power and bandwidth required to reliably drive data signals over long runs of copper cable. This had engineers searching for a more efficient material. The 1990s brought a transition from copper to optical cabling for long runs because optical fiber offered less lossy transmission, higher bandwidth, and lower energy requirements.

Not only did optical fiber introduce all the above benefits, but it also allowed for more convenient upgrades to network infrastructure as new technologies were introduced. That’s because optical fiber cables utilize pluggable optical modules that include an optical engine (OE) to convert optical signals to electrical signals and vice versa. These pluggable modules provide a simple and flexible method to connect fiber optic cables to network equipment. The pluggable modules are inserted into a connector that is mounted to the edge of a PC board (PCB) and to the front panel of network equipment. The modules utilize an electrical interface between the module and the switch/router ASIC in the network device.

However, as data network speeds continue to increase beyond 400 Gbps, optical fibers on their own aren’t going to cut it. The necessary power required to drive electrical signals even the relatively short distance from the switch ASIC near the center of a PCB to the pluggable modules at the front panel is becoming problematic. That’s where co-packaged optics come into play.

What Are Co-Packaged Optics?

It’s helpful context to know that electrical to optical conversion requires an electrical PHY to retime the incoming electrical signal and photonic components such as laser, modulator, and photodiodes to drive the optical signal. These functions are usually implemented in different IC packages and integrated on a PCB as shown below.

what are co packaged optics

But today’s advanced technology offers unprecedented miniaturization, making in-package integration of electrical and optical dies possible and practical. This single package integration of electrical and photonic dies is called CPO (see below).

co packaged optics

Future generations of switch ASICs running at 51.2 TBps and faster to support 800 GbE and 1.6 TbE will incorporate CPO so that data can be transmitted via light all the way to the switch ASIC package. This reduces the length of the electrical interface between the optical engine and the switch ASIC to only a few millimeters. Additionally, this addresses the need for energy reduction and cuts the latency associated with extracting the clock and data from the electrical signal.

Ultimately, hyperscale data center applications are driving this new wave of co-packaged optics designs to address “power wall” and faceplate density limitations by offering a way to keep rack unit power constant while increasing bandwidth capacity.

What’s Needed for Broad Adoption of Co-Packaged Optics?

While all the above sound promising, there are still several hurdles that need to be overcome before the industry sees a broad adoption of CPO in data centers. For example, because CPO requires silicon to be near the photonics, the traditional design rules of faceplate pluggable optics no longer apply. Each of the three broader groups of CPO applications today (Ethernet switches, machine learning, and disaggregation) requires different sets of design trade-offs and serviceability concerns. Professionals in the industry are currently debating exactly what those design rules and interface specifications should be.

Cost is another challenging factor. Prices must come down for CPO to compete with the projected cost of USD$0.60/Gb in 2024 for 400G-DR4 optics. Take heart though that significant savings are expected due to CPO’s lack of retimers, clock-and-data-recovery (CDR) chips, expensive ultra-low-loss PCB material, and enclosure hardware.

Despite these obstacles, it’s clear that CPO is the next major step on the path toward integration of optical and electrical data interfaces. The Co-Packaged Optics Collaboration and the Optical Interconnect Forum (OIF) have taken leadership roles in coordinating input from industry leaders like Synopsys to define CPO specifications and more, including:

  • A defined standard for the channel between electrical and optical components
  • A standardized interface to enable the electrical and optical components to communicate effectively and reliably
  • Design tools to develop and verify integrated electrical and photonic ICs

Synopsys provides OptoCompiler™, an integrated platform for the design, layout, simulation, and verification of electrical and photonic integrated circuits. OptoCompiler enables designers to capture their designs as schematics and choose domain-specific circuit simulators and DesignWare® IP to analyze the performance of their electrical/optical interface channel and CDR. As the above standards evolve and the demand for CPO rises, we continue to update our tools and IP to help customers efficiently migrate toward co-packaged optics and all the innovation that can occur through single package integration of electrical and photonic dies.

In Case You Missed It

Catch up on these past data center blog posts for additional insights: