Posted by Navraj Nandra on January 28th, 2014
PCI Express (PCIe) has long been a dominant standard for communications inside of computers, servers and blades but sometimes you need to think (and send data) outside the box. Since the first release of PCIe cabling standard in early 2007, creative engineers have been looking to utilize this new capability as a box-to-box interconnect, in order to expand PCIe into high end networking applications.
Through the three versions of the PCIe specification two reference clock (Refclk) receiver (Rx) architectures were defined, the common Refclk Rx architecture (Figure. 1a) and the data clocked Rx architecture (Figure. 1b). The difference in the two architectures is the following: In the common Refclk Rx architecture the receiver uses the same reference clock in its Clock and Data Recovery (CDR) as the transmitter and thus the CDR is synchronous with the incoming data. In a data clocked Rx architecture the CDR uses only the edges of the incoming data in its CDR and the Rx CDR needs to track all the jitter on the incoming signal. However, these architectures pertain only to the receiver and in either Rx architecture it is expected that the associated transmitter is synchronous with a system reference clock.
What this means is that the same reference clock (usually with Spread Spectrum Clocking, a.k.a. SSC) needs to be delivered to both ends of the link. For in-box applications this is not overly burdensome since the PCIe edge connector already has pins to carry this reference clock. However, if it is desired to send PCIe data outside of a box using a cable, the requirement that the reference clock needs to be sent in addition to the data signals is troublesome. While sending the Refclk does not sound like much of a burden it creates two issues: First, the cables and connectors need to be thicker, heavier and more expensive to carry the extra clock signals. Second, even though the Refclk does have SSC on it there can still be a large amount of Electromagnetic Interference (EMI) resulting from sending the Refclk.
How does this EMI arise? If the Refclk were perfectly differential and if the pair of conductors carrying the differential clock were perfectly matched then there would be no return current and little EMI. However, any non-differential element in the clock (either from the clock going into the cable or from mismatch in the conductors carrying the clock signals) causes some Common Mode (CM) signal to develop and the return path for that CM current is through ground. The shield of the cable is ground and thus shield would be a ”good” antenna for the CM signal. This is more problematic for clock signals than data signals because even a clock with SSC on it is relatively narrowband compared to the width of the data spectrum and thus is a stronger aggressor.
What is the solution? Well like the saying goes, I told my doctor it hurts when I do this and he said don’t do that. Thus SRIS (Separate Refclks Independent Spread) was developed to define a usage model with separate reference clocks at each end of a link (Figure 2). Thus, the root complex Refclk does not need to be sent across the cable in a PCIe SRIS application. Also, we note that the Refclk at each end of the link is independently spread to help reduce EMI. This is similar to other standards supporting data exchange of cables such as SATA and USB.
Is there any penalty for not sending the Refclk? The only disadvantage is that now the CDR in the Rx must track the full range of the incoming SSC. However, SATA and USB data clocked architecture receivers already have this situation. Thus, the challenges placed on a receiver in this environment are well understood and manageable by anyone offering any of the aforementioned products.
Thus, SRIS for PCIe is a new usage models to allow PCIe links to exist outside the box, especially for high data rate PCIe 3.0 (8Gbps) and the upcoming PCIe 4.0 (16Gbps) that are planned for quite a few new chassis to chassis interconnect for the high-end networking systems.