How to Maximize PCIe 6.0’s Advantages with End-to-End PCIe Design Solutions

If you dream of networking at the speed of the latest 800G Ethernet, you’ll need a serial bus interface to match it. Data center solid state drives (SSDs) and the accelerators used in AI applications are also hungry for speed. That’s why PCI Express® (PCIe®) 6.0 is gaining traction in AI, HPC, and data centers. PCIe 6.0 operates at 64GT/s and that’s lightning fast, twice as fast as the previous generation. But while network servers, SSDs, switches, and AI accelerators are all early adopters of PCIe 6.0, and network interface cards (NIC) and CPU host chips are on the horizon, how do you know if PCIe 6.0 is right for your next chip design?

Every time you advance, everything gets a little bit tougher, even as the time-to-market (TTM) pressure increases. If you want to take a step forward into a first mover position, but are hesitating, here’s a tip: Leveraging the leader in complete end-to-end PCIe hardware design tools can ease your journey to achieving industry-leading quality and security while lowering your risk. If you are considering making the move from PCIe 5.0 to 6.0, this overview will cover what you need to consider before getting into the PCIe 6.0 fast lane.

The Challenges of a New Generation of PCI Express

If your application has the need for the speed and bandwidth that PCIe 6.0 has to offer, there are a couple important factors to consider in your adoption:

1. The expense incurred in adopting a new PCIe specification: Added costs can come from moving to a smaller foundry process, new IP, and what the implication of adoption means to the whole system, including losses associated with cables and backplanes and adding new cards.
2. The timing of PCIe 6.0 maturity: The PCIe 6.0 ecosystem hasn’t solidified and compliance, gated on PCI-SIG consortium workshops, is roughly a couple years out.

Any time you advance to a new generation of a standard there are cost considerations. Whether these costs are born now or later depends on your urgency in realizing the advantages PCIe 6.0 has to offer weighed against your goals and unique market pressures.

In terms of PCIe 6.0 maturity, the current adoption trajectory is similar to previous generations of the spec. Because PCIe is ubiquitous, you can be assured that the ecosystem to support the 6.0 generation is growing. You could wait until completion of consortium workshops, for the proverbial dust to settle, and for the spec to have a track record of product in the field before adopting it. But, adopting PCIe 6.0 now gives you all the amazing advantages it has to offer while helping to future proof your design, enabling you to be on the leading edge of innovation as the 6.0 ecosystem comes online.

Backwards Compatibility Hedges Your Risk

While engineers love designing advanced systems and technologies, they are also notoriously risk averse, continuously walking the fine edge of decisions that can determine first movers, fast followers, the muddle in the middle, late to market, or dead on arrival. Despite the significant technology advancements, PCIe has carefully written the specification for backward compatibility with previous versions of the spec. In other words, if you adopt the spec early, you don’t have to rely on a robust PCIe 6.0 ecosystem for your system to work. Whether it’s an endpoint or a complex system, PCIe 6.0 designs can plug into technology that uses any version of the PCIe standard, including PCIe 1.0 @ 2.5GT/s. If a device cannot support the new PCIe 6.0 64GT/s data rate, the link will be negotiated at the highest data rate supported by both link partners.

One note on backward compatibility, once a link negotiates 64GT/s FLIT mode, it must remain in FLIT mode: if an anomalous event impacts signal integrity—for instance, if a giant motor turns on causing a glitch in the power supply, or there is extra noise, or somebody moves a cable—the system may need to fall back to the previous PCIe generation data rate (such as 32GT/s or even 16GT/s). Even when a glitch such as this causes a fall back, the system will remain in FLIT mode. Supporting a previous generation while remaining in FLIT mode is a new state that never existed before. While this is not necessarily likely, it is a possible complication. To hedge for this possibility, you must support FLIT mode for all PCIe data rates.

Backward compatibility is one of the key beauties of the spec. PCIe 6.0 is already part of a whole PCIe ecosystem that dominates the industry, providing ubiquitous connectivity. Afterall, you cannot change the entire data center and all the devices in it at once when you move from Gen. 5 to 6. PCIe enables a fall back option, so the whole universe doesn’t need to upgrade simultaneously.

Network Switch Design with PCIe 6.0

Navigating network switch design development illustrates the complexity of an implementation of PCIe 6.0 in practical application. To get the advantages of 64GT/s and PAM-4 signal integrity in a design with 256 lanes, for instance, you will not only need the full switching solution, but you will also need to integrate with technology beyond your system. You need to think through the co-design of the package, the board, and all the integrations, as well as how you put it together, into a holistic solution. This entails activities such as optimizing the bump map, reduction of escaping signals, and optimizing the beachfront of your die, on all die edges. On top of activities like these, and more, you must make sure that you get the lowest cost package for the device.

Designing a switch with PCIe 6.0 requires an understanding of a vast array of different perspectives so the system will work holistically. Implementing PCIe 6.0 in a switch requires different configurations with multiple links and multiple controllers. Because of this, you will need an IP provider who has deep experience in PCIe and has already done the analysis for PCIe 6.0, including working with customers on PCIe 6.0 switch designs.

To achieve PCIe 6.0 success in switches or any other application, you need a partner who can provide you with expertise and a complete solution that maps to the end-to-end design process, from IP and design through production. A solid design partner can help you lower risk, overcome the challenges of complexity, and ease your journey to success.

Is PCIe Verification a Big Deal Again?

Yes. The changes introduced with PCIe 6.0 affect all layers, creating increased verification complexity.

At the physical layer, 64GT/s speed support is achieved using PAM4 encoding. PCIe 6.0 introduces 256B FLIT, which demands certain packing rules for protocol packets into FLITs, increasing design complexity. It also brings in FEC complexity along with the existing CRC mechanism.

For backward compatibility reasons, 256B FLITs are supported at 2.5/5/8/16/32 GT/s speeds. This demands verification of FLIT mode at all supported speeds.  

At the data link layer, addition of new DLLP types–namely optimized updatefc and link management, for exchanging link information, and change in sequence number/replay rules–demands in-depth verification of sequence numbering, FLIT replay command handshakes, and selective/full replay mechanisms to provide guaranteed FLIT transfer across to the link partners. 

Along with the introduction of FLIT, new TLP framing rules are also defined, which requires extensive verification.

PCIe 6.0 also introduces a new power state L0p, which enables power reduction without impacting the traffic flow. Link management DLLPs are used to establish L0p handshake between link partners. This adds to the design complexity necessitating in-depth verification.

The disruptive nature of PCIe Gen6 specification will create new verification challenges not only for backward compatibility, bandwidth, and performance of the interface but also for dependent NVMe, SSD, and other PCIe-based storage technologies. Synopsys Verification IP (VIP) and test suite are designed to handle this verification complexity. Synopsys VIP is used to verify silicon-proven Synopsys IP.

In addition, running system-level payload on SoCs requires a faster hardware-based pre-silicon solution. Synopsys transactors based on Synopsys IP enable fast verification hardware solutions including Synopsys ZeBu® emulation systems and Synopsys HAPS® prototyping systems for validation use cases.