Full Utilization of 16 GT/s PCIe Gen 4 Bandwidth – 2

PCI express Gen 4 implementation is marching towards the Gen 4 0.7 release. It’s important that not only physical layer delivers the 16 GT/s rate, but also the entire protocol stack should be capable of saturating the full allocated bandwidth. To saturate the full bandwidth, following two key features are gaining traction:

• 10-bit extended tag support
• Scaled flow control credits

In our earlier blog,  we discussed about 10-bit extended tag. In this blog, we will discuss about the second feature scaled flow control credits. We will give a brief introduction of the feature to give a jump start to any one ramping up on the latest specifications, and also discuss corresponding verification challenges and solutions.

Scaled flow control credits

Current flow control mechanism allows maximum of up to 127 outstanding header credits and 2047 outstanding data credits. The Gen3 x16 link can saturate with these limits in certain scenarios. The flow control mechanism is enhanced with scaled flow control. In scaled flow control mechanism, the maximum outstanding header and data credits can be scaled by a factor of 1, 4 or 16 based on the programmable setting. Note that the credits remain unchanged, one header credit will still be 1 TLP header and one data credit remains to be 16 bytes. Following updates have been done to support scaling flow control: –

• New capability structure called “Data Link Feature Extended Capability” has been added. It contains programmable control/status information about the local and peer support of the “Data Link Feature Support”
• New DLLP called “Data Link Feature Exchange” has been added
• New state called “DL Feature” has been added in the Data Link Control and Management State Machine as a part of the initialization
• During the new state DL Feature a new extension is added to initialization. The new DLLP “Data Link Feature Exchange” is exchanged at every 34 us to figure out if the Flow control scaling is enabled
• Init FC1, Init FC2 exchanged subsequently decide the scaling factor through the re-definition of the reserved bits as shown below
• Subsequent UpdateFC protocol remains unchanged. The scaling factor in the UpdateFC must continue to match to the factor indicated during initial credit exchange protocol

Verification Challenges and Solution

Verification of the feature can be divided in 3 categories as described below: –

• Normal operation
  • Directed test to verify if credit scaling with different programmed settings is reaching to its maximum values
  • Random credits usage with different scaling enabled
  • With multiple VCs, when one VCx is blocked the other VCy can continue to make forward progress if there is pending traffic
• DL Feature state
  • Possible state transitions from and to DL Feature state
• Error injection cases
  • New HdrScale and DataScale field corruptions
  • Issuing more credits than advertised
  • All the error injections leading to reporting of “Flow Control Protocol Error”, as defined in specification.

Synopsys VC VIP for PCIe supports latest specifications and makes it very easy to verify all the new features.

Authors:  Anand Shirahatti, Mohd Adil Khan, Jamshed Alum